Agent Directory

← Back to all companies

InstacartUnited States

Food · United States · Q22909236

Verdict
SelectiveCovers all 5 agentsTechnical signals differ between agents (7 agents with a signal); no company statement found.
Company statement
None found
robots.txt signals
Selective
Last checked
Fee
Unknown
x402
Unknown
Notes
Auto-discovered (Tranco + Wikidata); consumer classification not yet reviewed.
Human verified
Not yet

By agent

AgentResultBasis
ChatGPTConditionalrobots.txt rule (advisory: the vendor says this agent may ignore robots.txt)
ClaudeBlockedrobots.txt rule
PerplexityBlockedrobots.txt rule (advisory: the vendor says this agent may ignore robots.txt)
GeminiBlockedrobots.txt rule (advisory: the vendor says this agent may ignore robots.txt)
Meta AIBlockedrobots.txt rule (advisory: the vendor says this agent may ignore robots.txt)
Meta MuseNo dataReported to send a generic Chromium-on-Linux user agent, so it cannot be targeted or detected via robots.txt; only company statements can speak to it.

Consumer surfaces

Evidence

Source typeAgentsResultEvidence levelObservedSource
robots.txt Check
Consumer agents
  • ChatGPT-User (the vendor says it may ignore robots.txt)
Italic: the vendor says it may ignore robots.txt.		 Conditional Our Check2026-11-08 View source
18 rules
Allow: /
Allow: /store/brands/l/pads-tampons-liners-wipes
Disallow: /accounts/unlock
Disallow: /api/
Disallow: /ex/
Disallow: /orders/
Disallow: /store/orders/
Disallow: /redeem_coupon_token
Disallow: /store/order_changes/
Disallow: /store/*/browse_departments*
Disallow: /store/*/ng/l/*
Disallow: /store/referrals*
Disallow: /store/item*
Disallow: /store/meals/*
Disallow: /store/pickup/status/*
Disallow: /store/*/buy_it_again*
Disallow: /store/*/l/*
Disallow: /store/*/mea
robots.txt Check
Consumer agents
  • Claude-User
  • Google-Agent (the vendor says it may ignore robots.txt)
  • Meta-ExternalFetcher (the vendor says it may ignore robots.txt)
  • Amzn-User (the vendor says it may ignore robots.txt)
  • MistralAI-User (the vendor says it may ignore robots.txt)
Search crawlers
  • Claude-SearchBot
  • Amzn-SearchBot
  • Meta-WebIndexer
  • DuckAssistBot
Training crawlers
  • GPTBot
  • ClaudeBot
  • Google-Extended
  • Applebot-Extended
  • CCBot
  • Amazonbot
  • Bytespider
Italic: the vendor says it may ignore robots.txt.Applies through the wildcard (*) rule.		 Blocked Our Check2026-11-08 View source
Disallow: /
robots.txt Check
Consumer agents
  • Perplexity-User (the vendor says it may ignore robots.txt)
Search crawlers
  • PerplexityBot
Italic: the vendor says it may ignore robots.txt.		 Blocked Our Check2026-11-08 View source
Disallow: /
robots.txt Check
Search crawlers
  • OAI-SearchBot
 Conditional Our Check2026-11-08 View source
19 rules
Allow: /
Disallow: /accounts/unlock
Disallow: /api/
Disallow: /ex/
Disallow: /orders/
Disallow: /store/orders/
Disallow: /redeem_coupon_token
Disallow: /store/order_changes/
Disallow: /store/*/browse_departments*
Disallow: /store/*/ng/l/*
Disallow: /store/referrals*
Disallow: /store/item*
Disallow: /store/meals/*
Disallow: /store/pickup/status/*
Disallow: /store/*/buy_it_again*
Disallow: /store/*/l/*
Disallow: /store/*/meals/*
Disallow: /widgets/*
Disallow: /client-side
robots.txt Check
Training crawlers
  • Meta-ExternalAgent
 Blocked Our Check2026-11-08 View source
Disallow: /
Disallow: /

robots.txt checks

HostResultLast checkedUnchanged sinceContent-Signal
instacart.comRead (200)2026-11-08 00:00 UTC2026-11-02None

A robots.txt with no agent-specific rules produces no result, and silence is not the same as allowed. A refused or unreachable check says nothing about the site's rules.

Evidence levels: Our Check is a direct read of the site's robots.txt, Company Document is confirmed by a quote, News Report is reported by a reputable outlet, and Unverified Lead never sets a verdict. Methodology v2. View as JSON