robots.txt, sitemap and llms.txt
robots.txt, sitemap and llms.txt validator, free, no sign-up
Checks the three files that guide search engines and AI assistants through your site. It tells you whether Google and AI bots can get in and whether your page is listed where it should be.
How it works
- 01
Paste a URL
Use any page on your site. We look for the files at the domain root.
- 02
Review all three files
See whether robots.txt, sitemap.xml and llms.txt exist, whether they are valid and what they allow.
- 03
Adjust what is needed
Get priorities: which rule to change, what to declare and which pages are missing from your sitemap or llms.txt.
What it checks
Frequently asked questions
What is robots.txt for?
It is the file that tells crawlers which parts of your site they may visit. A badly written rule can lock out Google or AI bots without you noticing. The tool checks that it exists, that it lets Google in and which AI bots it allows.
What does it check in sitemap.xml?
It checks that the sitemap is declared, that it is valid XML and whether the page you analyzed is listed in it. A sitemap helps search engines discover your important pages. If a key page is missing, it is worth adding.
What is llms.txt?
It is a Markdown file at the root of your site that summarizes your content and links your main pages for language models. It is a recent, optional proposal. The tool checks whether you have one and how much of your sitemap it covers.
Why is my page not indexed if robots.txt allows it?
It may carry a noindex directive in the HTML or headers, or be missing from the sitemap. These are different things: robots.txt controls crawling and noindex controls indexing. The tool checks both so you can see which one affects you.
What data do you keep from the validation?
We keep the URL analyzed and the results, to show history and improve the tool. If you give an email for the PDF, we keep it too. Your IP is stored only as a hash. Marketing consent is optional and unchecked by default.
Is there a limit on validations?
Yes. There is a per-person limit per hour and per day, so the tool can stay free for everyone. If the same URL was validated in the last 24 hours, you see the saved result instead of a new run.
Market OS
Make sure no change locks you out of search or AI
Market OS checks robots.txt, your sitemap and llms.txt continuously, together with Search Console, GA4 and your competitors. Ask for a demo and we will review it with your site.
More free tools
Page analyzer
SEO, GEO, speed, schema and security score for one URL, with what to fix first.
Try it →FreeFree GEO audit
Whether the crawlers behind ChatGPT, Gemini, Perplexity and Claude can reach, read and cite your content.
Try it →FreeSpeed and Core Web Vitals
LCP, INP and CLS for your page with field and lab data.
Try it →FreeStructured data
Checks your page's JSON-LD and generates what is missing.
Try it →FreeHTTPS, headers and redirects
Redirect chain, HTTPS, HSTS and headers that affect search engines.
Try it →FreeTitle and meta description
Title, description, canonical, Open Graph and the directives that keep you out of snippets and AI Overviews.
Try it →FreeEmail security: SPF, DKIM and DMARC
Whether your domain meets what Gmail and Yahoo require to deliver your email.
Try it →FreeAI brand visibility
Whether ChatGPT and Gemini mention your brand, and who they name instead.
Try it →Keywords, AI Overviews and competitors
What your market searches, who shows up and whether Google answers with AI. It needs search volume and difficulty data we haven't connected yet.

Review with WITS
Want us to review it with you?
The free tool tells you what is missing. If you want WITS to prioritize and fix it with you, tell us what you want to achieve.
WhatsApp +52 56 6107 7931 · ventas@wits.mx