Webinar: State Of Agentic AEO For 2027: What’s Scaling & What’s Stalling

Register Now
  1. SEJ
  2.  ⋅ 
  3. AI Search

AI Citations: What To Check Before Reacting To A Lost One

Most AI citations that disappear later come back. Here are four things to check about a lost citation before you decide to change a page.

AI Citations: What To Check Before Reacting To A Lost One

Around three in four AI citation drops were temporary, according to Ahrefs’ study of the changes in five different AI products over 20 days.

When sharing AI citation analysis results with clients or bosses, you should be aware that a loss may not represent what it seems like at first glance. Sites in the study that had been cited in answers but then dropped out of those answers were more likely to have another citation occur for them as opposed to staying missing. If someone rewrote a page in the meantime, that rewrite could receive credit for a citation it didn’t earn.

Ahrefs studied about 37,500 first-day citations made by Google’s AI Mode and AI Overviews, along with ChatGPT, Gemini and Copilot. The study included only one organization’s view based on its own group of questions answered daily by each tool for 20 days.

The study’s findings point to four items to look into before making an edit to a webpage based upon the disappearance of a citation.

What The Study Tracked

Tracking the same citations for consecutive days, the study was able to see what can happen after a site has disappeared from an answer.

These results were documented by Mateusz Makosiewicz, an SEO and marketing educator at Ahrefs. His colleague Xibeijia Guan collected all websites that had been cited on day one, then traced each reference through to day 20.

By day two, about half of those references were gone from AI Overviews. Even more were gone on each of the other tools. Most of these returned prior to the conclusion of the study.

Across the five tools, about half of the citations that dropped out came back within three days, Ahrefs found.

Stacked bars show when dropped citations returned across AI Overviews, ChatGPT, AI Mode, Copilot and Gemini.
Each bar shows when citations returned after their first disappearance. “Never came back” refers to the 20-day study window. Source: Ahrefs.

The data collected in this study reflects the number of websites, thus when a website comes back, this study cannot determine whether the exact same webpage came back as well.

Another measure was taken (in Brand Radar’s own tracking of citations), revealing a somewhat comparable pattern with lower rates of return. About two-thirds of all the citations that disappeared from view there ultimately did come back. Ahrefs sells Brand Radar, and the write-up has a section on using it.

How This Fits Earlier Findings

Earlier research I have discussed demonstrated changes in AI responses over multiple runs of a prompt, and this current study adds what happens across different days.

I reported on SparkToro’s findings in January regarding their discovery that both ChatGPT and Google’s AI in Search provided the same list of brands less than 1% of the time across repeated runs of the same prompt. They also found that the frequency with which a brand appeared throughout multiple runs was generally more stable than the specific ranking of that brand within those lists.

The new study’s authors recommend counting how many of the last seven days a site was cited. This is a frequency count by day for the sites that were cited and the research in January favored a frequency count by run for brand names.

Check One: Did The Tool Cite Anyone?

When a loss occurred on Gemini or Copilot, it often meant there was no one cited at all in the response.

Gemini had responses without citations about 64% of the time in the data from this study, while Copilot showed answers without citations around 53% of the time. Without citations in an answer, a site will appear as a loss when you track your results.

Many of the comebacks on those two tools were the tool citing sources again after an answer that cited none. With the empty answers left out, Gemini and Copilot looked much like AI Mode and ChatGPT.

AI Overviews, AI Mode and ChatGPT almost always cited sources. Fewer than 1% of their answers had no citations, and Makosiewicz wrote that a lost citation on those tools almost always means another site took the spot.

The first way to verify a lost citation is by looking at the answer. If the answer has no link then you are getting information about the AI tool that day, while if the answer links to a competitor then you have found the site that was cited in your place.

On Gemini and Copilot, per the write-up, you should keep track of two metrics, the number of times your site gets cited, as well as the frequency with which references are made to another source instead.

Check Two: How Long Had The Citation Lasted?

Citations that had stayed active longer before going away were more likely to come back.

The study ranked all citations on three of the tools (AI Overviews, AI Mode and ChatGPT) in order of how long they had been there until they went away the first time.

For those with a short life span (one day), 34% to 41% of the citations did not return during the study period.

And for those that remained there for eight to 14 days, this percentage was at or below 4%.

This suggests that a citation’s history may be more important than where it stands on a given day. A site referenced once that then disappears is an entirely different circumstance than a site that has maintained its position for 10 days and disappeared yesterday.

An analysis of occasional check-in reports shows if a site was cited on a specific date (so every site with only one appearance looks like every other site with continuous appearances). Using a seven-day count allows a team to make a distinction. It counts, for each question being asked, the number of days out of the last seven the site was a source referenced. This shows how often a site turns up. How long one citation held is a separate question.

This process also provides a team with an ongoing assessment of whether an AI tool is consistently relying on a given site to answer those questions.

Similarly, it can be applied to new citations. In terms of evaluating the usefulness of a new citation as a lasting win, Makosiewicz suggests waiting a few days before declaring victory. His general rule is to view one missing citation as no reason to change a page, but to review the page if the citation is still missing after two or three weeks.

Check Three: Is The Brand Still Named?

A brand can still be named in an answer even after the link to its site disappears.

A citation is a link readers can follow. A mention is a brand name in the answer text, whether or not it includes a link.

Each tool used saw approximately 50% to 70% of all cited sites appear on only one day, as opposed to around one in five of all brands named.

Then your report can display a lost citation while the answer still names the company. The link is still absent, and it still counts as a lost citation, and looking at the answer will tell you which measure has changed.

Also, AI tools use different versions (longer or shorter) of the same brand name from answer to answer. And a mention that seems to be gone could be the same company referred to by a different name, so the write-up advises checking for this before subtracting it from your mention totals.

Since the data showed brand names were not moving nearly as much as citations, Makosiewicz suggests that mentions may be the better metric to follow.

Check Four: Does The Page Rank For The Question?

On AI Overviews and AI Mode, pages that also ranked in Google Search for the same question kept their citations longer.

In this portion of the study, the authors examined pages. They compared cited pages that appeared somewhere within Google’s results for a question versus those that did not appear anywhere in Google’s results for the same question. On AI Overviews and AI Mode, cited pages that ranked had greater longevity of citation, more often lasting a week or longer; however, the difference between them was smaller for ChatGPT.

Although appearing in Google’s search results at all correlated with longer-lasting citations more than a page’s exact position in the results list did, according to the write-up, this finding was based on an examination of 230 questions. That comparison showed a correlation between whether pages rank in Google searches for each question and how long the citations to these pages last. Ranking wasn’t a requirement, though. Most of the pages AI Mode and ChatGPT cited didn’t rank in Google for the question they were cited on.

When a citation stays missing, the authors suggest examining two items, how the page performs in Google Search for the question and how well the page answers the question.

In September, I proposed a related item to be examined when citations drop while rankings hold steady. That item is to examine the pages being cited in lieu of those originally cited.

The methodology of this examination also relates to the work performed by SEO teams. When a page loses its Google ranking for a particular question and subsequently drops off of the AI Overview citations, looking into why the page lost its ranking would be a logical starting point to investigate.

A Comeback Isn’t Proof The Edit Worked

Most citations that disappeared came back during the study, so one comeback alone may not be strong evidence that a page change caused it.

A team that edits a page after losing a citation and then sees the citation come back has two explanations for it. An edit can’t be credited as working (it may have been due to some other reason) when there is only one comeback to support this.

Makosiewicz wrote, “Don’t credit a page edit for a single comeback.”

In essence, he wants us to look at an increase in the number of times a site reference appears over a longer time span.

In addition, it would be weak evidence that an edit damaged a citation when that citation disappears the very next day after you edit a page, because half or more of citations disappeared by the next day in the study.

Why This Matters

A number of things can be checked by someone before they change the page behind a lost citation. First, they can look again over the next several days to see whether the citation is still missing. Second, they can read the response itself.

Management will learn a lot more from a count like five of the last seven, when an in-house team reports how many days a site was cited, than from simply receiving a yes or no from one check.

An agency has an order to follow on a client call, checking the text of the response, the history of the citation and the ranking of the page prior to making a proposal for new content.

A content team will get a warning about correlation vs. causation. One comeback after an edit does not provide you with sufficient evidence to make similar edits to other pages.

Open Questions

The researchers measured the disappearance of citations; they did not test if changing a page brings back a citation that has gone missing.

They only tracked the very first time a citation went missing. The data collected does not reveal how frequently those same citations disappeared again after returning.

The researchers wrote that their figures may represent an underestimate of citations returning as citations that disappeared at the end of the study had less time to return than citations that were lost early in the study. Changes to models could also affect the stability of citation patterns over time.

Looking Ahead

The next lost citation will be much easier to read when you have something to reference already. Having a daily log of which sites are cited, related to the questions that matter most, can help provide your team’s own internal tally of days cited prior to a client or manager asking why citations are down.

When a citation is still missing after two or three weeks, the study’s advice is to look at the page, beginning with how it ranks for the question.

More Resources:


Featured Image: Tetiana Yurchenko/Shutterstock

Category SEO AI Search
SEJ STAFF Matt G. Southern Senior News Writer at Search Engine Journal

Follow on YouTube. Matt G. Southern is the Senior News Writer at Search Engine Journal, where he’s covered Google, SEO, ...