Who visited
User-Agent strings can be spoofed. Identity needs stronger evidence.
Read the rules.
Don’t guess the visits.
Inspect the robots.txt rules declared for major search and AI tokens.
Check a website's declared access
SeeingOwl follows the page to its final host, then checks that exact path against robots.txt.
Declared access
Allowed means permitted by the matching rule. It does not record a visit or verify the requester.
Use the token for the job.
Search, model development, and user-requested fetching use different controls.
User-agent: OAI-SearchBot
Allow: /
User-agent: GPTBot
Disallow: /
User-agent: Claude-SearchBot
Allow: /
User-agent: ClaudeBot
Disallow: /Example only. Test your exact paths.
What robots.txt leaves unanswered.
What was indexed
Permission does not confirm inclusion in search or answer systems.
What drove traffic
Rules cannot prove retrieval, citations, recommendations, or conversions.
See actual visits.
SearchSeal reads your access logs.