Free tools / AI Crawler URL Inspector
See this page like GPTBot sees it
Paste one URL. We fetch it as a browser and as GPTBot, and show the exact title, meta description, word count, content excerpt and structured data each one gets back.
By Raphaël Aubry, founder of Howseen AI · Updated
Free, no signup. Your result shows right here.
How it works
Most AI crawlers, including OpenAI's GPTBot, fetch the raw HTML of a page and do not run JavaScript. We request your URL twice, once as a normal browser and once with the GPTBot user agent, and show you both results side by side.
“GPTBot is used to crawl content that may be used to train our generative AI foundation models.”
OpenAI, Overview of OpenAI crawlers
Paste a URL
Any public page: home, pricing, a blog post.
We fetch it twice
Once as a browser, once with the GPTBot user agent, and compare the response.
See the extracted content
Title, meta description, word count, a content excerpt and the structured data found.
FAQ
Short answers to the questions people ask most before they run the crawler URL inspector, based on what the tool actually checks.
Why would GPTBot see something different from my browser?+
If your content is built with JavaScript after the page loads, a crawler that does not run JavaScript sees an empty or near-empty page even though your browser shows the full content.
What does “blocked” mean here?+
Your browser request succeeded but the GPTBot request failed or was refused, usually a firewall or bot-protection rule blocking the GPTBot user agent specifically.
Does a short excerpt mean a problem?+
Not always. A short page can be normal. Compare the word count to what you expect your page to contain; a near-zero count on a content-heavy page usually means the text is not in the raw HTML.
What is the structured data list?+
The schema.org @type values found in any JSON-LD script tag on the page, such as Organization, Product or FAQPage. An empty list means no structured data was found.
Go further: Check your whole site's robots.txt rules
More free tools
Each free tool answers one question about how AI assistants see your brand. Together they cover visibility, crawling, llms.txt, searches and schema.
Sources
The official documentation this tool follows, from the companies that run the AI crawlers it checks against.
