robots.txt, sitemap and llms.txt

robots.txt, sitemap and llms.txt validator, free, no sign-up

Checks the three files that guide search engines and AI assistants through your site. It tells you whether Google and AI bots can get in and whether your page is listed where it should be.

  • Free
  • No sign-up
  • Spanish and English
  • GEO included

We store the URL analyzed and the result; the IP only hashed. Privacy policy.

How it works

  • 01

    Paste a URL

    Use any page on your site. We look for the files at the domain root.

  • 02

    Review all three files

    See whether robots.txt, sitemap.xml and llms.txt exist, whether they are valid and what they allow.

  • 03

    Adjust what is needed

    Get priorities: which rule to change, what to declare and which pages are missing from your sitemap or llms.txt.

What it checks

  • robots.txt exists and lets Google in
  • AI bots allowed in robots.txt
  • sitemap.xml declared and valid
  • Whether the page is listed in the sitemap
  • noindex on the page
  • llms.txt present and how much of the sitemap it covers
  • Markdown alternate version
  • IndexNow (ping to Bing): checked when the key is known

Frequently asked questions

What is robots.txt for?

It is the file that tells crawlers which parts of your site they may visit. A badly written rule can lock out Google or AI bots without you noticing. The tool checks that it exists, that it lets Google in and which AI bots it allows.

What does it check in sitemap.xml?

It checks that the sitemap is declared, that it is valid XML and whether the page you analyzed is listed in it. A sitemap helps search engines discover your important pages. If a key page is missing, it is worth adding.

What is llms.txt?

It is a Markdown file at the root of your site that summarizes your content and links your main pages for language models. It is a recent, optional proposal. The tool checks whether you have one and how much of your sitemap it covers.

Why is my page not indexed if robots.txt allows it?

It may carry a noindex directive in the HTML or headers, or be missing from the sitemap. These are different things: robots.txt controls crawling and noindex controls indexing. The tool checks both so you can see which one affects you.

What data do you keep from the validation?

We keep the URL analyzed and the results, to show history and improve the tool. If you give an email for the PDF, we keep it too. Your IP is stored only as a hash. Marketing consent is optional and unchecked by default.

Is there a limit on validations?

Yes. There is a per-person limit per hour and per day, so the tool can stay free for everyone. If the same URL was validated in the last 24 hours, you see the saved result instead of a new run.

Market OS

Make sure no change locks you out of search or AI

Market OS checks robots.txt, your sitemap and llms.txt continuously, together with Search Console, GA4 and your competitors. Ask for a demo and we will review it with your site.

Talk to WITS →
The Market OS character, in a lab coat, waves in front of the Minerva monument in Guadalajara on a sunny day.

Review with WITS

Want us to review it with you?

The free tool tells you what is missing. If you want WITS to prioritize and fix it with you, tell us what you want to achieve.

WhatsApp +52 56 6107 7931 · ventas@wits.mx

Message us on WhatsApp