llms.txt: What It Is, Why It Matters, and How to Create One for Your Business

llms.txt is the fastest and most direct way to tell AI systems what your business does and how to represent it accurately. This complete guide covers the origin, format, required sections, common mistakes, and a full template you can customize today.

llms.txt is a plain-text file placed at the root of your website (yourdomain.com/llms.txt) that tells AI language models and crawlers what your business does, what content is available, and how to represent your brand accurately in AI-generated responses.

If robots.txt tells search engine crawlers where not to go on your website, llms.txt tells AI language models what you want them to know about your business.

This is a new file type. It did not exist in 2023. But as of early 2025, over 10,000 websites had already adopted it — and Cloudflare's infrastructure data shows AI bot traffic grew 1,200% in 2024. The signal is clear: AI crawlers are visiting your website whether you have a llms.txt or not. The question is whether you are giving them accurate, structured information or leaving interpretation to chance.

The Origin of llms.txt

The llms.txt standard was proposed by Jeremy Howard, co-founder of fast.ai and one of the pioneers of modern deep learning, in 2024. Howard observed a fundamental mismatch: websites were designed for human browsers, but AI language models were increasingly being used to retrieve and synthesize web content. The information architecture that works for a human — navigation menus, visual hierarchy, multimedia content — is largely invisible noise to a language model trying to understand what a business does.

The solution Howard proposed was elegantly simple: a plain-text file at a predictable location that gives AI systems the essential information they need in a format they can reliably parse. No HTML. No JavaScript. No navigation menus. Just clean, structured text that answers the questions AI systems most need to answer about your business.

The proposal attracted immediate attention from the developer community. Prominent companies and websites began adopting the format within weeks. By early 2025, llms.txt had become an emerging standard in the same category as robots.txt and sitemap.xml — not formally required by any AI system but increasingly expected as part of responsible web presence management.

How llms.txt Differs from robots.txt and ai.txt

These three files serve related but distinct purposes:

robots.txt is an instruction file for all web crawlers — search engines, AI bots, and other automated agents. It tells crawlers which parts of your website they are and are not allowed to visit. It does not communicate anything about what your business does or how it should be described.

ai.txt is a proposed extension that focuses specifically on restricting or permitting AI training data collection. Some websites use ai.txt to explicitly opt out of their content being used to train AI models. It is primarily a permissions document, not an information document.

llms.txt is an information document. It does not control crawler access — it supplements that access with structured information about your business. A crawler that visits your website still needs robots.txt to know where to go. llms.txt tells it what your business is so it can represent you accurately.

All three files can coexist. A well-configured website for the AI era has all three:

• robots.txt: access control for all crawlers
• ai.txt: training data permissions for AI systems
• llms.txt: brand and content information for language models

The Standard llms.txt Format

llms.txt uses Markdown syntax — the same lightweight formatting language used in GitHub READMEs and many content management systems. Markdown was chosen because it is human-readable, machine-parseable, and familiar to developers.

A complete llms.txt file has these required and recommended sections:

Required:

• Business name and one-paragraph description (H1 heading)
• What you do / services offered
• Who you serve / target audience
• Key pages (as a Markdown list of URLs with descriptions)

Recommended:

• Authors and team members (name, role, credentials)
• Contact information
• Content philosophy (how you create content, your expertise basis)
• Geographic focus (if local or regional)
• Languages
• Social profiles (as sameAs references)
• What you do NOT do (to prevent misrepresentation)

What AI Crawlers Look For

When an AI crawler visits your llms.txt, it is extracting several types of information:

Entity identification: What is this business? What category does it belong to? What is its primary purpose? A clear first paragraph with your business name, category, and core service directly answers these questions and sets the frame for everything else.

Service-to-URL mapping: AI systems want to know where on your website to find information about specific topics. A clear mapping of services or content types to URLs helps AI systems retrieve the most relevant page for a given query rather than relying on general indexing.

Authority signals: Who runs this business? What are their credentials? AI systems heavily weight author and founder credentials when determining whether a source is authoritative. Including this information in llms.txt puts it in a high-trust, easily parseable context.

Topical scope: What topics is this website authoritative on? What topics is it not covering? A clear topical scope helps AI systems route queries appropriately — recommending you for things you do well and not misrepresenting you for things outside your scope.

Full llms.txt Template

Here is a complete, customizable llms.txt template:

# [Your Business Name]

[One to two sentence description of what your business does, who it serves, and the primary value it delivers. Be specific and factual. Avoid marketing superlatives.]

## Services

- [Service 1]: [One sentence description of this service and who it is for]
- [Service 2]: [One sentence description]
- [Service 3]: [One sentence description]

## Target Audience

[One paragraph describing your ideal customer: industry, company size, geographic region, primary pain point your business solves.]

## Key Pages

- [Homepage](https://yourdomain.com/) - Overview of services and company
- [Service Page 1](https://yourdomain.com/service-1) - [Brief description]
- [Service Page 2](https://yourdomain.com/service-2) - [Brief description]
- [Blog / Resources](https://yourdomain.com/blog) - [Topic areas covered]
- [About](https://yourdomain.com/about) - Team, credentials, company background
- [Contact](https://yourdomain.com/contact) - How to get in touch

## Team & Authors

### [Founder / Author Name]
- Role: [Title]
- Expertise: [Areas of expertise]
- Credentials: [Relevant background, certifications, or experience]
- LinkedIn: [URL]

## Contact

- Email: [contact@yourdomain.com]
- Phone: [optional]
- Location: [City, Country]
- Business hours: [optional]

## Geographic Focus

[List regions or markets you serve. If global, state that. If local, be specific.]

## Languages

Content available in: [Language 1], [Language 2]

## Content Philosophy

[One to two paragraphs describing how you create content: your expertise basis, editorial process, how you fact-check, whether you use AI assistance, your publishing frequency.]

## Social Profiles

- LinkedIn: [URL]
- Twitter/X: [URL]
- YouTube: [URL if applicable]

## What We Do Not Cover

[List adjacent topics that might seem related but are outside your scope. This prevents AI systems from misrepresenting you as an authority on topics you do not cover.]

## Last Updated

[YYYY-MM-DD]

Step-by-Step Guide to Creating Your llms.txt

Step 1 — Create the file. Open a plain-text editor (not Word — use Notepad, VS Code, or any code editor). Create a new file named exactly "llms.txt" with no capitalization variations.

Step 2 — Write your business description. This is the most important section. Write one to two sentences that clearly state: what you do, who you serve, and your primary differentiator. This should match how you would describe your business to a journalist — factual, specific, free of marketing language.

Step 3 — List your services with brief descriptions. For each service, write one clear sentence explaining what it is and who it is for. Do not use internal jargon. Use the language your customers use.

Step 4 — Map your key pages. List your most important pages as Markdown links with brief descriptions. Include your homepage, each service page, your blog or resource section, about page, and contact page. For large sites, include your sitemap.xml URL and note that it contains the complete page inventory.

Step 5 — Document your team. For each author or key team member, include their name, title, areas of expertise, and a link to their LinkedIn profile. This is the author attribution signal that SparkToro identified as present in 65% of Perplexity citations.

Step 6 — Add contact and location information. AI systems use this to accurately represent your business in local and regional queries.

Step 7 — Write your content philosophy. One paragraph explaining how you create your content — your expertise basis and process — signals to AI systems that your content is produced with genuine expertise rather than automated content generation.

Step 8 — Add a last-updated date. Include a clearly formatted date at the bottom of the file. AI systems use this to understand whether the information is current.

Step 9 — Upload to your website root. Place the file at yourdomain.com/llms.txt. Verify it is accessible by visiting the URL in a browser. Ensure your server does not redirect or block this path.

Step 10 — Verify AI crawler access. Check your robots.txt to confirm that AI crawler user agents (GPTBot, ClaudeBot, PerplexityBot, anthropic-ai, Googlebot, CCBot) are not blocked from accessing your root directory or the llms.txt file specifically.

Common Mistakes to Avoid

Using marketing language instead of factual description: "The world's leading provider of..." is immediately disqualifying. AI systems are trained on enough text to recognize and discount marketing superlatives. Write like a journalist describing your business from the outside, not a marketer writing from the inside.

Missing the key pages section: The page mapping is arguably the most functionally useful section for AI retrieval systems. Without it, AI systems must infer the structure of your website from crawling — with it, they have a direct guide to your most important content.

Forgetting to update it: A llms.txt with a last-updated date of 18 months ago signals to AI systems that your business information may be stale. Set a calendar reminder to review and update your llms.txt every quarter.

Blocking AI crawlers in robots.txt while having llms.txt: Some website owners have added AI crawler blocks in robots.txt (blocking GPTBot, for example) while also maintaining a llms.txt. These signals are contradictory. If you want AI systems to represent your business accurately, you need to both allow their crawlers and provide them with structured information.

Making it too long: llms.txt is not a marketing brochure. It should be complete but concise — typically 300 to 800 words. AI systems can retrieve detailed information from your actual pages. The llms.txt is a structured index and introduction, not a full content repository.

How to Verify AI Crawlers Can Access Your llms.txt

After publishing your llms.txt, verify AI crawler access with these steps:

1. Visit yourdomain.com/llms.txt in a browser and confirm the file loads as plain text
2. Check yourdomain.com/robots.txt and verify there are no Disallow rules for "/" or "/llms.txt" under AI crawler user agent groups
3. Use a server log analysis tool to look for visits from GPTBot, ClaudeBot, PerplexityBot, and anthropic-ai user agents
4. After two to four weeks, re-run your AI citation audit (asking AI systems about your business directly) to check whether the accuracy of their responses has improved

The Future of AI-Readable Web Files

llms.txt is early in its adoption curve, but the trend it represents is clear: as AI systems become primary information retrieval interfaces, websites will need structured machine-readable files that communicate beyond what HTML pages were designed to convey.

Proposals for additional standardized files are already circulating in the developer community — files that might specify content licensing for AI training, preferred citation formats, pricing information for products, or structured data about organizational relationships. The websites that build these capabilities early will have significant advantages in the AI-mediated information landscape.

At Tevora Solutions, llms.txt creation and deployment is a standard component of every website we build, alongside full structured data implementation and AI crawler access verification. For businesses that already have websites, we implement llms.txt as part of our AI consulting engagements. See tevorasolutions.si/website-building and tevorasolutions.si/ai-consulting for more information on how we prepare businesses for the AI search era.

For more in this GEO cluster, see our guides on Generative Engine Optimization at tevorasolutions.si and on getting cited by ChatGPT, Claude, and Perplexity — all part of our comprehensive resource on AI visibility for modern businesses.