Skip to content

When ChatGPT Reads Your Page Live: A Fetch-Readiness Guide for Agencies

A live fetch means a real buyer is looking at your client through ChatGPT right now. The job is not to block it. It is to be ready for it.

John Cravey with AIFounder9 min readUpdated Jul 6, 2026

There is a moment most businesses never think about: a person is sitting in ChatGPT, asking about a company, a service, or a specific question, and ChatGPT reaches out and reads a live web page to answer them. That page might be your client's. The agent doing the reading is called ChatGPT-User, and it is different in kind from the crawlers everyone argues about. It is not indexing the web in the background. It is fetching one page, right now, because a real person asked. For an agency, that changes the job from being found to being ready.

Free estimate · 2 minutes

AI search is already sending buyers. Be citable.

Build the estimate to see the AI-visibility work we would ship for your site in the first thirty days. Sixty seconds, no opt-in.

The plain-English version

OpenAI runs an agent called ChatGPT-User that fetches a live page when a ChatGPT user's request needs one, to help answer the question and to provide a source link back (OpenAI's crawler docs). It is explicitly not an automated bulk crawler, unlike OAI-SearchBot, which surfaces you in search, or GPTBot, which trains on content. ChatGPT-User acts on behalf of a specific person in a specific moment. When it hits a client's page, it means someone is actively researching that client, or the topic the client serves, inside ChatGPT, and the model went to read the page to give that person a real answer with a link.

That framing is the whole reason this matters commercially. A live fetch is not background noise, it is a buyer with their hand halfway up. The agency's job is not to decide whether to allow it, because these are your client's potential customers, and blocking them would be self-defeating. The job is to make sure that when the fetch happens, the page is fast enough to be read, readable enough to be understood, and complete enough to answer on its own. Being fetch-ready is a real deliverable, and most client sites are not.

Source: OpenAI, Overview of OpenAI crawlers. ChatGPT-User is the only one tied to a live, present user.

Why you cannot, and should not, block it

There is an important technical wrinkle an agency needs to understand before advising a client. Per OpenAI's December 2025 documentation update, ChatGPT-User no longer complies with robots.txt rules for user-initiated actions. Because the fetch happens on behalf of a real person who asked, it is treated more like a browser loading a page than like a crawler indexing one, so a robots.txt disallow is not a reliable way to stop it. That surprises people who assume robots.txt is a universal off switch. It is not, for this agent.

But the more important point is that you would not want to block it even if you could. A ChatGPT-User fetch is a potential customer being handed your client's page as an answer, with a link back. Blocking that is like refusing to let a walk-in customer read your sign. The entire posture here is the opposite of the training conversation, where blocking is sometimes right. With live fetches, the goal is to welcome them and serve them well, because each one is a person your client wants to reach, arriving at the exact moment of interest.

The technical version: making a client fetch-ready

Fetch-readiness is a specific, auditable property, and it overlaps with performance and accessibility work you may already do, but the framing is sharper: can a machine, in the moment, fetch this page, render its content, and read a clear answer. Run this checklist on a client's key pages.

  1. Serve content as real, server-rendered text. A live fetch may not execute heavy client-side JavaScript, so content that only appears after scripts run can be invisible to it. The core answer of every key page should be in the initial HTML, not painted in later.
  2. Be fast. A live fetch is happening while a person waits for an answer. Slow responses risk being abandoned or truncated. Core Web Vitals and server response time matter here for a reason beyond Google.
  3. Keep key pages free of hard gates. Login walls, aggressive interstitials, and cookie modals that block content can stop a fetch from reaching the substance. Gate what you must, but keep the pages you want cited open and readable.
  4. Make each page answer on its own. A live fetch usually pulls one deep page, not your whole site, so the specific service or answer page has to stand alone. It cannot rely on context the user never loaded.
  5. Keep information current. Because the fetch is live, it reads what is on the page today. Stale hours, prices, or availability get served as current. Freshness is now a correctness issue, not just a nicety.

None of this requires blocking anything or editing robots.txt. It is content and performance hygiene aimed at a new consumer: a real-time agent reading on behalf of a live buyer. It also compounds with the search work in the OAI-SearchBot agency playbook, because the same fast, readable, standalone pages that get fetched well also get surfaced well.

Every step downstream of the fetch depends on the page being fast, readable, and complete when ChatGPT-User arrives.

Turning live fetches into a client signal

The reporting opportunity here is genuinely new and most agencies are not using it. Because ChatGPT-User fetches are tied to live user interest, they are a leading indicator of demand forming inside ChatGPT, before any click, form, or call. Verify the fetches against OpenAI's published ranges at openai.com/chatgpt-user.json so you are counting the real agent, then log them per client and per page. A client whose service pages are getting a rising number of live fetches is being actively researched inside ChatGPT, and that is a story worth telling in a monthly report, alongside the search-visibility and access numbers.

It also sharpens where to invest. If the fetches cluster on a handful of pages, those are the pages buyers are being handed as answers, so those are the pages to make fastest, clearest, and most complete. The signal tells you which content is actually doing the work in front of live buyers, which is more useful than guessing. Over time, the mix of which pages get fetched is a map of what ChatGPT thinks your client is good for, which is a strategic input, not just a metric.

Fix from the top. A live fetch cannot wait, cannot run heavy scripts, and cannot see past a hard gate.

A worked example: reading the fetch signal

Make the signal concrete. Suppose you run this for a mid-sized law firm client with practice-area pages for estate planning, business formation, and personal injury. You verify and log ChatGPT-User fetches per page for a quarter. The estate-planning page shows a steady, rising line of live fetches; the business-formation page shows a trickle; the personal-injury page shows almost none. That pattern is a map of where ChatGPT-mediated demand is forming for this client, and it is available nowhere else in their analytics, because it sits upstream of every click and call.

What do you do with it? Three things. You tell the client, because a rising line of live fetches on estate planning is a real story about demand they cannot see any other way. You prioritize fetch-readiness work on that page first, making it the fastest, clearest, most standalone page on the site, because it is the one being handed to live buyers most often. And you ask why personal injury gets no fetches: is the firm not positioned for it in a way ChatGPT recognizes, or is the page not fetch-ready enough to be chosen. The signal turns vague optimization into a prioritized, evidence-backed plan, which is exactly what a client wants to see in a report.

Over several quarters, the shifting mix of which pages get fetched becomes a longitudinal read on what ChatGPT associates the client with, and how that is changing. That is strategic intelligence, not a vanity metric, and being the agency that surfaces it positions you well above the one reporting rankings and bounce rate. It is a new column in the report that only exists because you understood that a live fetch is a buyer, and you instrumented for it.

Packaging fetch-readiness as a deliverable

Fetch-readiness fits cleanly into the productized model you already use for AI access and search. The audit is concrete: for a client's top pages, confirm server-rendered content, measure performance, check for hard gates, and verify freshness, then hand back a scored list of what to fix. The ongoing work is the fetch-signal reporting plus rechecks after any site change, since a replatform or a new performance regression can silently make a previously fetch-ready page slow or script-dependent. Priced as an add-on to an existing retainer or folded into a care plan, it carries its own margin because the audit is repeatable and the reporting is largely automated once the logging is in place.

The pitch to the client writes itself once you frame it right. When a buyer asks ChatGPT about them, the client is either handed a fast, clear, current page or a slow, broken, stale one, and right now nobody is checking which. You check it, you fix it, and you report the live demand it reveals. That is a real service tied to a channel the client's competitors are almost certainly ignoring, which is exactly the kind of offer that both retains a client and differentiates your agency.

The thread tying the audit and the reporting together is a single reframe worth stating plainly to every client: a live fetch is a buyer, not a bot. Once a client internalizes that, the fast page, the readable text, the current information, and the open access all stop feeling like technical chores and start feeling like customer service, which is exactly what they are. Your job as the agency is to make that reframe stick and then deliver on it.

The mistakes that cost a client here

  • Assuming robots.txt controls it. For user-initiated fetches it does not, per OpenAI's current docs. Do not promise a client you blocked ChatGPT-User via robots.txt.
  • Rendering the answer only in JavaScript. If the substance appears after heavy scripts, a live fetch may read an empty shell. Put the core content in the server-rendered HTML.
  • Ignoring the signal. Live fetches are a leading demand indicator. Not verifying and logging them means leaving a real client story on the table.
  • Letting key pages go slow or stale. A live fetch serves whatever is on the page, at whatever speed. Slow or outdated pages get read as the client's current, best answer.
  • Hard-gating pages you want cited. A cookie wall or login in front of a service page can stop the fetch from reaching the substance a buyer needed.

One more reason to move on this now rather than later: the field is wide open. Most businesses have not thought about fetch-readiness at all, because most agencies have not either. Their deep pages are slow, their answers are painted by scripts, and their edge posture was never checked against a live agent. That means the bar to being the fastest, clearest, most reliably-served option in a category is currently low, and an agency that gets a client there early is banking an advantage while competitors are still unaware the channel exists. As with the search work, the window is open and uncontested today, and it will not stay that way once fetch-readiness becomes table stakes.

What changes by client size inside your book

Search gets a client into consideration. Training shapes what the model generally knows. The live fetch is the moment a specific buyer is handed the client as the answer, so it is where readiness pays off most directly. Being fetch-ready and being the firm the model chooses to hand over are two halves of the same job, and the second half is the authority work in the answer engine optimization cornerstone. See also how the training decision fits in the GPTBot agency guide.

Want us to audit your client book for fetch-readiness and stand up the live-fetch reporting with you? Run discovery or see what we ship.

Answers

Frequently asked questions

ChatGPT-User is the agent that fetches a live web page when a ChatGPT user's request needs it, to help answer and provide a source link. It is not an automated bulk crawler like OAI-SearchBot or GPTBot. It fires in real time, in response to a specific person's prompt, so a fetch signals active, present interest.

Does ChatGPT-User obey robots.txt?

Per OpenAI's December 2025 documentation update, ChatGPT-User no longer complies with robots.txt rules for user-initiated actions. Because it acts on a real user's behalf, a robots.txt disallow is not a reliable way to block it. Since these fetches represent interested users, you generally want to serve them well rather than block them.

How should an agency use ChatGPT-User traffic in reporting?

Treat it as a real-time, high-intent signal. Verify hits against OpenAI's published ChatGPT-User ranges, then log them per client. A rising count of live fetches on a client's key pages indicates people are actively researching that client inside ChatGPT, which is a leading indicator worth surfacing to the client.

What makes a page fetch-ready?

Speed, readable server-rendered content, and standalone pages. A live fetch has to render and read your page in the moment, so content buried behind heavy JavaScript, a login, or slow responses may not be usable. Every key page should answer its question on its own, in real text, quickly.

Can a client tell the difference between a live fetch and a crawl in their logs?

Yes, by the user agent and the pattern. The live fetcher identifies itself distinctly and arrives as single requests for specific deep pages, rather than the broad sweep of an indexing crawl. That difference is what makes the log readable as demand rather than as traffic.

What is a live fetch and how does it differ from crawling?

It happens when a user's question requires reading a specific page right now, so the fetch is triggered by a person rather than by an index refresh. That means it is real-time, it targets one page, and what it finds is what the user is about to be told.

Why should a client not block it?

Because the request originates from someone actively asking about them. Blocking it means the assistant reports it could not read the page, in front of a prospective customer, which is a worse outcome than anything being read. It is the one crawler with a person waiting behind it.

How do live fetches become a client signal?

The server log shows which pages are being fetched and how often, which is a direct read on what people are asking assistants about that client. It is demand data nobody else is collecting, and it points at which pages deserve better answers.

Why does speed matter more here than for classic crawling?

Because a person is waiting. A crawler will come back; a live fetch has a budget measured in seconds and a user watching. A slow page is functionally an unreadable page in this context, even though it works fine for every other visitor.

How should this be packaged as a deliverable?

As a fetch-readiness pass: confirm the crawler is allowed, check the highest-value pages render server-side and fast, put direct answers near the top, then report which pages are being fetched monthly. It fits inside existing technical work rather than beside it.

What mistakes cost clients here?

Blocking the fetcher as part of a blanket AI rule. Client-rendering the content that answers the question. Slow pages that time out mid-fetch. And never looking at the logs, which means the demand signal sits there unread for a year.

How does this vary by client size inside a book?

Small clients need the check once and rarely again. Growing clients need it applied across the deep pages, which is where fetches actually land. Larger clients need it as a standing check, because a template change on one section can quietly break readability for all of it.

Question we did not answer? Ask us directly and we will answer it here.

John Cravey, Founder
Written by
John Cravey
Founder

Founder of Frontend Horizon. Writes most of the long-form work on the FH blog.

Newer post
Should You Let AI Train on Your Content? A Mid-Market Guide
Older post
When ChatGPT Reads Your Page Live: A Guide for Growing Businesses
Keep reading

More from the blog

AI·10 min

Should You Let AI Train on Your Content? An Agency Guide

Training and search are two different doors. The expensive mistake is slamming the search door shut while trying to close the training one. Here is how to advise clients on both.

SEO·10 min

On-Page SEO Automation: Titles, Meta, Schema, and Internal Links

The boring on-page work is where most SEO is won and lost. It is also exactly the work AI does faster than you and never forgets.

AI·10 min

How to Control the OpenAI Crawlers With robots.txt: An Agency Playbook

One small file decides whether a client is found, trained on, or invisible. Standardize it, and a fragile per-client guess becomes a repeatable, defensible deliverable.