Same Assistant, Opposite Weeks

01A step down on a Wednesday evening

After the tracking bug I described in the week ChatGPT lost its referrer, my products recovered together. Then on the evening of 21 October they split. ChatGPT tagged visits to a CV product fell from about 63 a day to about 7, at roughly 20:00 UTC, and stayed there for the next ten days. Every page fell. Every country fell. A voice product on the same server, measured the same way, kept its 120 to 126 ChatGPT sign ups a day without a dent. When one assistant drops one product and keeps another, the cause is in the answers, not the pipes.

Ruling out the usual suspects

It was not the tracking, because direct visits to the same pages fell at the same moment, so the visitors were missing, not mislabelled. It was not a deploy: the first change to that site that day went out at 21:22, after the step. It was not a block: ChatGPT's live agent and its search crawler both got normal responses. In fact the search crawler became far more interested, jumping from about 2 fetches a day on that site to about 125 from 22 October. And it was not Google, where the same product gained, from 1,136 to 1,593 clicks a week.

02Asking the model directly

Thirty nine questions, zero mentions

On 3 November I asked the question the logs could not answer: when a model searches the web for ordinary CV questions, does it pick this product? Through OpenAI's API with web search switched on, I sent 39 generic questions, the kind a person writing a CV asks, to three current models. The product was cited zero times and named zero times. When the questions included its name, the models found the site and read it without trouble. As a control, the voice product was named in 3 of 12 generic questions in its own category. Being findable by name and being chosen for a question are different tests, and only the second sends traffic.

What the test cannot say

I have no baseline from before 21 October, so I cannot prove the product used to be named and stopped. The API is not the ChatGPT app either, and the app has its own memory, ranking and personalisation. What the test does show is the current state: for generic questions in its category, this product is not among the sources these models choose. That matches the shape of the drop, a step across every page at once, which looks like a change in how answers are assembled rather than in any single page. It is the pattern from how ChatGPT chooses who to recommend: reputation decides the shortlist before any page is read.

03Build a baseline before you need it

A weekly question set

The single most useful thing I could have had was last month's answer to the same 39 questions. So the next step is to run the set weekly, the same questions to the same models, storing every answer's cited sources, so that the next time a channel steps down there is a before. A citation test you start after the drop can describe the hole but never measure its depth. The method is the one I set out in one question, five engines, run on a schedule instead of once.

Do not panic edit the pages that still rank

The temptation after a drop like this is to rewrite pages until the number comes back. Google traffic for this product was rising in the same weeks. Editing its core pages to chase an assistant risks the channel that is working. The changes worth making are additive: new pages that answer the questions the models were asked, consistent facts across the site, and outside mentions that give a model a reason to include the name. When one channel falls and another rises, protect the one that is rising first.

Tomorrow: an error report endpoint that a search crawler kept posting to.