What is the difference between robots.txt, llms.txt, and agents.md?

robots.txt is an access-control file that tells crawlers and bots which paths they may fetch. llms.txt is an index file that points AI assistants at your most important content in one place. agents.md is the canonical agent-discovery file: it describes what your store is, what it sells, and how an agent should transact with it. robots.txt controls permission, the other two control understanding. On Shopify, agents.md is now the canonical file and /llms.txt and /llms-full.txt mirror it.

Did agents.md replace llms.txt?

On Shopify, effectively yes. As of Spring 2026, /agents.md is the canonical AI-discovery file and /llms.txt and /llms-full.txt mirror or redirect to it, so customizing one updates all three. The llms.txt convention still exists across the wider web, but on a Shopify store the three paths resolve to the same content.

Do I still need robots.txt if I have agents.md?

Yes. They do different jobs. robots.txt decides whether an AI crawler is allowed to fetch your pages at all; agents.md describes your store to an agent once it is there. Block a crawler in robots.txt and your agents.md never gets read by that crawler. You want robots.txt to allow the AI agents you care about, and agents.md to speak to them when they arrive.

Where do these files live on a Shopify store?

All three are served at the root of your domain: yourstore.com/robots.txt, yourstore.com/llms.txt, and yourstore.com/agents.md. Shopify generates robots.txt and agents.md automatically. You customize robots.txt with a robots.txt.liquid template and agents.md with an agents.md.liquid template, both in your theme.

Does agents.md help my Google ranking?

Not directly. agents.md is read by AI shopping agents and assistants, not by Google's classic search index, so it is not a traditional SEO ranking signal. Its job is agentic commerce: helping an AI agent understand and recommend your store. Classic SEO still runs on your pages, structured data, and links.

robots.txt vs llms.txt vs agents.md: What Each File Does (and Which You Need)

Three files sit at the root of your store with overlapping-sounding names, and they get mixed up constantly. robots.txt, llms.txt, and agents.md are not three versions of the same idea. They do three different jobs, and you generally want all three doing theirs.

Here is the one-line version: robots.txt controls access, llms.txt was the index, and agents.md is the brief. One decides who gets in, the others decide what they understand once they do.

The short comparison

	robots.txt	llms.txt	agents.md
Job	Access control	Content index for AI	Agent brief + transaction handshake
Reader	Crawlers and bots	AI assistants	AI shopping agents
Answers	"May I fetch this?"	"Where's the substance?"	"What is this store and how do I buy?"
Format	Directives (allow/disallow)	Markdown links	Markdown prose + endpoints
On Shopify	Auto-generated, editable	Mirrors agents.md	Canonical, auto-generated, editable
Affects classic SEO?	Yes	No	No
You still need it?	Yes	Optional (covered by agents.md)	Yes

robots.txt: who is allowed in

robots.txt is the oldest of the three and the only one about permission. It lives at yourstore.com/robots.txt and tells automated visitors which paths they may and may not fetch. It has governed search-engine crawlers for decades, and it now governs AI crawlers too.

This is the file that decides whether GPTBot, ClaudeBot, PerplexityBot, Google-Extended and the rest are even allowed to read your store. If you disallow them here, nothing downstream matters: a blocked crawler never reaches your product pages, your structured data, or your agents.md. The permission gate comes first.

The catch on Shopify is that the default robots.txt was written for search engines, and some stores inherited blanket rules that quietly block the AI crawlers they would actually want. We cover exactly which agents to allow, and how to check what yours is doing today, in which AI crawlers to allow in your Shopify robots.txt.

What robots.txt does not do: describe your store. It is a bouncer with a guest list, not a host. It says "you may enter," never "here is what we sell."

llms.txt: the index that pointed the way

llms.txt was the web's first real answer to "how does an AI assistant find the important parts of my site without crawling everything?" It is a Markdown file at yourstore.com/llms.txt that lists your key pages and content in one place, the way sitemap.xml does for search engines, but written for a language model to read in a single fetch.

The convention is still alive across the wider web, and the full history and format are in llms.txt, explained. But on Shopify specifically, its role changed in Spring 2026. The platform made agents.md the canonical AI-discovery file, and /llms.txt and /llms-full.txt now mirror it. You no longer maintain a separate index. Customize agents.md and all three paths update together. The Shopify-specific details are in llms.txt for Shopify.

So on a Shopify store, llms.txt is not a third thing to maintain. It is one of three URLs that resolve to the same content.

agents.md: the brief and the handshake

agents.md is the newest and, for a store, the most important. It is a Markdown file at yourstore.com/agents.md that an AI shopping agent fetches to learn three things fast: what this store is, what it sells, and how to transact with it programmatically.

That last part is what separates it from llms.txt. agents.md is not just an index of links. It carries the transaction handshake: the Universal Commerce Protocol (UCP) discovery endpoint and the Model Context Protocol (MCP) endpoint an agent uses to search your catalog and build a checkout without scraping your storefront. It is the difference between "here are my pages" and "here is how you do business with me."

On Shopify, agents.md is platform-generated and native. Every store already serves one at the domain root, no app or app proxy required. That is the good news. The bad news is that the default is generic: it reads near-identically to every other store and points agents at Shopify's own rails rather than describing your brand. Most stores are sitting on the default and do not know it, because a checker that only asks "does the file exist?" shows green. We dig into why that quiet sameness costs you recommendations in your agents.md is identical to every other store, and the full reference for the file is the pillar, agents.md on Shopify.

How the three work together

Think of an AI agent arriving at your store in order:

robots.txt decides whether the agent's crawler is allowed to fetch anything. Permission.
agents.md (and its llms.txt mirror) tells the agent what your store is and how to transact. Understanding.
Structured data on your pages gives the agent the specific facts, per-variant price, availability, ratings, it needs to actually recommend and check out. Proof.

Skip the first and the rest never runs. Skip the second and the agent has to guess what you are from raw HTML. Skip the third and the agent understands your store but cannot trust the details. The three files are a permission layer, an understanding layer, and a proof layer, and a recommendation needs all three. The complete agentic commerce guide maps how the proof layer fits on top.

So which do you actually need?

On a Shopify store:

robots.txt: keep it, and make sure it allows the AI crawlers you care about. This is the most common silent mistake. A store can do everything else right and still be invisible because a stale robots.txt blocks GPTBot.
llms.txt: you do not maintain it separately. It mirrors agents.md. One file, three URLs.
agents.md: this is the one to invest in. The default exists and is valid, but it describes the platform, not you. Customizing it is the highest-leverage move of the three, because it is the file that decides how an agent represents your brand.

Customizing agents.md: paste one file, keep it current

The only supported way to override the generated agents.md on an Online Store 2.0 theme is a template at templates/agents.md.liquid. You can write it by hand, but then every catalog change becomes a theme edit, and the file drifts out of date the moment your store does.

AgentReady generates a brand-specific agents.md from your live catalog, collections, policies, FAQs and reviews, then gives you the finished file to paste into that template once. When your store changes, it flags you and hands you the updated version to paste over the old one. It is free, the content is plain Markdown you own outright, and removing it is as simple as deleting the template to fall back to Shopify's default. The step-by-step is in how to customize your Shopify agents.md.

Check all three at once

The free AI-readiness checker fetches your store the way an agent does and reports on all three files together: whether your robots.txt allows the major AI crawlers, and whether your agents.md is Distinctive, Generic-default, or Missing, alongside the structured data on your pages.

Run the checker to see what an AI agent sees when it arrives at your store today.