Why isn't my site's llms.txt file showing up for AI crawlers?

Your site returns a 404 error when AI systems look for llms.txt. This file helps AI crawlers understand how to properly index your content. You need to create and upload an llms.txt file to your root domain to fix this issue.

What does it mean when robots.txt returns a 404 error?

Your robots.txt file is missing or inaccessible, returning a 404 error. Search engines expect this file at your root domain to understand crawling permissions. Create a robots.txt file even if it's just basic instructions.

How do missing date signals affect your site's search performance?

Your pages lack publication or update dates, making it hard for search engines to assess content freshness. This hurts rankings for time-sensitive topics. Add visible dates to your articles and structured data markup for better SEO.

Why is content freshness critical for your site's authority?

Search engines can't determine when your content was created or updated without date signals. This impacts your site's perceived expertise and trustworthiness. Adding clear publication dates helps establish content credibility and improves search visibility.

What happens when your site has no internal linking structure?

Your pages aren't connected through internal links, making it harder for search engines to discover and understand your content hierarchy. This weakens page authority distribution and user navigation. Add relevant internal links between related pages.

‹ return Home

Caddy SEO & AI Visibility Audit

Caddy is an open-source web server and reverse proxy written in Go that automates HTTPS certificate management. It provides automatic TLS, load balancing, and API-driven configuration that enable developers to deploy secure web services without manual certificate handling.

trycaddy.ai

SEO + AEO + E-E-A-T combined

44%

Audited March 06, 2026 · 1 pages analyzed

Overall Scores

Area	Score	Grade
SEO	77%	B
AEO (AI Visibility)	11%	F
E-E-A-T	44%	F
Combined	44%	—

Fastest way to fix your site is to use our Custom Connector for Claude Desktop, or other IDEs.

E-E-A-T Breakdown

Signal	Score	Issue
Trustworthiness	57%	No about/team page, social profiles, or LinkedIn links detected, 6 security headers missing, 1/4 contact signals found
Expertise	67%	All checks passing
Authoritativeness	20%	No Organization JSON-LD schema found, No JSON-LD schema types detected, No third-party review or trust signals detected
Experience	19%	Found, Freshness issues, Found 3 headings, No first-hand experience signals detected

Stobo finds what's broken and generates the fixes

Claude Desktop MCP integration settings showing how to add a custom MCP server

Try these in Claude Desktop

stobo my site: trycaddy.ai

stobo this article: trycaddy.ai/blog/my-article

Generate a robots.txt for trycaddy.ai

stobo my blog: trycaddy.ai/blog

Recommendations

robots.txt not accessible (HTTP 404) AEO
Update robots.txt to allow AI crawlers like GPTBot and ClaudeBot. Stobo generates an AI-friendly version.
```
Generate a robots.txt for trycaddy.ai
```
Freshness issues: no date signals found AEO
Add date markup (datePublished, dateModified) so AI engines know content is current. Stobo generates the JSON-LD.
```
Generate freshness code for trycaddy.ai
```
llms.txt not found (HTTP 404) AEO
Create an llms.txt file so AI models can understand what trycaddy.ai does. Stobo generates one automatically.
```
Generate an llms.txt for trycaddy.ai
```

Track your score over time

We'll re-audit your site in 7 days and email you what changed. Free, automatic.

Embed your audit badge

Show your score on your README or website.

Markdown

HTML

Frequently asked questions

Why isn't my site's llms.txt file showing up for AI crawlers?: Your site returns a 404 error when AI systems look for llms.txt. This file helps AI crawlers understand how to properly index your content. You need to create and upload an llms.txt file to your root domain to fix this issue.
What does it mean when robots.txt returns a 404 error?: Your robots.txt file is missing or inaccessible, returning a 404 error. Search engines expect this file at your root domain to understand crawling permissions. Create a robots.txt file even if it's just basic instructions.
How do missing date signals affect your site's search performance?: Your pages lack publication or update dates, making it hard for search engines to assess content freshness. This hurts rankings for time-sensitive topics. Add visible dates to your articles and structured data markup for better SEO.
Why is content freshness critical for your site's authority?: Search engines can't determine when your content was created or updated without date signals. This impacts your site's perceived expertise and trustworthiness. Adding clear publication dates helps establish content credibility and improves search visibility.
What happens when your site has no internal linking structure?: Your pages aren't connected through internal links, making it harder for search engines to discover and understand your content hierarchy. This weakens page authority distribution and user navigation. Add relevant internal links between related pages.

Audit a competitor