Create an AI-optimized llms.txt file from your website. Paste a sitemap URL, enter your domain, or list specific pages — no AI, no credits, no login.
Choose one input method below. We crawl your public pages and assemble a valid llms.txt from titles and meta descriptions.
An llms.txt file is a plain-text Markdown index at your site root (yoursite.com/llms.txt) that lists your most important pages with short descriptions for AI systems and developers.
The format was proposed by Jeremy Howard in 2024 and is documented at llmstxt.org. It is a community convention — not an IETF or W3C standard — but documentation sites, SaaS products, and blogs increasingly publish one.
Think of it as a curated reading list for language models: an H1 site title, a blockquote summary, and H2 sections of markdown links with one-line notes. It complements sitemap.xml and robots.txt rather than replacing them.
A valid llms.txt follows a simple, deterministic Markdown layout defined at llmstxt.org:
Absolute URLs are strongly recommended. Serve the file as text/plain or text/markdown at a well-known path. Most sites only need the index file — not a separate llms-full.txt unless you want full page text inlined.
These three root-level files serve different audiences. Publishing all three is common; they do not conflict.
llms.txtrobots.txtsitemap.xmlSet realistic expectations before you ship. llms.txt is low-cost to publish but adoption by major AI products is still evolving.
Enter your sitemap URL for blogs and large sites, your homepage URL for smaller sites, or paste specific URLs. Review the preview, download llms.txt, and upload it to your domain root.
Straight answers for common llms.txt questions — useful for SEO long-tail queries and AI-citable Q&A.
An llms.txt file is a Markdown text file at your domain root (/llms.txt) that lists your site's most important pages with short descriptions for AI systems. It gives language models a curated map of your content without requiring them to crawl your entire site. The convention is documented at llmstxt.org.
Yes — this generator is 100% free with no signup, no credits, and no login required. We crawl your public pages and build the file from HTML titles and meta descriptions without using AI. You can copy or download the result instantly.
Upload it to the root of your domain so it is reachable at https://yourdomain.com/llms.txt. Some documentation sites use a subpath like /docs/llms.txt, but the root is the most common location crawlers check first.
llms.txt is a curated link index with short descriptions — what most sites need. llms-full.txt inlines the full text of those pages for deeper context and can be very large. Start with llms.txt; add llms-full.txt only if you have a specific use case that needs full page content.
No — llms.txt is not a Google ranking factor, and Google has said it does not use llms.txt for search indexing. Its real value is in AI tooling, developer discovery, and giving coding assistants a structured overview of your site. Keep investing in sitemaps, meta tags, and content quality for SEO.
No major AI provider has officially committed to using llms.txt in production inference. Adoption is uncertain, though some crawlers occasionally fetch /llms.txt. The strongest confirmed use today is AI coding assistants and developer tooling that consume the file when pointed at your repo or docs.
No — llms.txt is an inclusion and curation file, not a blocking mechanism. It does not tell crawlers what they may or may not access. To restrict AI crawlers like GPTBot or ClaudeBot, use robots.txt rules and each provider's opt-out documentation.
Filter your server access logs for requests to /llms.txt and inspect the User-Agent header. Look for identifiers such as GPTBot, ClaudeBot, PerplexityBot, and Google-Extended. A single fetch does not mean ongoing use — track patterns over weeks.
Curate your best pages rather than listing every URL on your site. Aim well under typical model context limits — roughly 150K words or ~700KB as a practical ceiling — and split into llms-full.txt or section-specific files if you need more. Quality and focus beat exhaustive dumps.
Generally no — publishing indexable .md mirrors of every HTML page can create duplicate-content issues if search engines crawl them. llms.txt links to your canonical URLs with descriptions; that is usually enough. Use llms-full.txt only when you deliberately want inlined full text for LLM context.
Update when you add, remove, or significantly change key pages — new product areas, docs sections, or landing pages. For active sites, regenerating from your sitemap monthly or on each deploy (via CI) keeps the file current without manual edits.
No. The generator crawls your public HTML and extracts page titles, meta descriptions, and URL structure programmatically. Nothing is sent to Gemini or any other AI model to write your file.
Use sitemap URL for blogs and large sites — it discovers every listed page quickly. Use website URL for smaller sites where we start at the homepage and follow internal links. Paste specific URLs when you only want a hand-picked subset.
Some sites block automated requests through bot protection, Cloudflare, or WAF rules. Try pasting specific URLs manually, or use your sitemap URL if it is publicly accessible without authentication.