About our checker bot

Agent Directory records how companies treat AI agents. To do that, an automated checker reads each tracked site's public robots.txt file and notes which AI agents it mentions.

What it does

• Requests only https://your-site/robots.txt, at most about once a day per site, following up to 5 redirects.
• Identifies itself with the user agent AgentDirectoryBot/0.1 (+https://agentaccepted.com/bot).
• Does not log in, crawl pages, submit forms, run scripts or try to get around blocks. If a site refuses it, that is recorded as "denied" and nothing else is concluded.
• Stores the robots.txt text, its hash and the date, and shows the relevant rules on the site's listing.

Opt out or ask questions

Send a message with the contact form. For an opt-out, give the site's domain and we will stop checking it. You can also block the bot yourself with User-agent: AgentDirectoryBot in robots.txt; the listing will then say the check was blocked.

If a listing about your company is wrong, use the form too and include a link to the page that shows the correct policy.