AI Search Tracking Checklist: How to Attribute Calls From ChatGPT, Claude, and Perplexity

Download the Checklist
  1. SEJ
  2.  ⋅ 
  3. SEO Strategy

Rank Trackers Find Google Results About One Word Of A Query

  • Unrelated Google results collected by rank trackers were mostly about one word of the query.
  • Two rank-tracking companies found them mainly below the top results and on page 2.
  • Check whether replacing results follow one word of the query before reporting a drop.

Two rank trackers' data shows unrelated Google results that follow one word of the query, mostly below the top spots. Here's how to check a ranking drop.

Rank Trackers Find Google Results About One Word Of A Query

In mid-September, most unrelated results discovered by SE Ranking using Google rank checking were only about one word of the original search term. Almost all of them started at position 9 or worse.

SE Ranking released an analysis today based on 10,000 Google US desktop keywords it tracks for research purposes. Rank tracking firm Nozzle documented a related pattern in an Oct. 22 report on what it describes as Google ignoring part of a query. According to Nozzle’s findings, page 2 started dropping words on Sept. 16 while page 1 dropped words four days later (Sept. 20). Both firms sell rank tracking software and track ranks through their respective products.

On Oct. 23 I covered how rank tracking providers say they handle the altered Google results; prior to that, on Oct. 14 Roger Montti reported Nozzle’s initial analysis. Both data sets show that most changes occurred in positions below the top positions. Nozzle’s data shows that most of the changes it saw occur on page 2. As such, when reviewing your late September/October ranking reports, check drops below the top positions and on page 2 first.

Results About One Word Of The Query

The primary example of a single-word result set offered by SE Ranking is for the search term “business bank line of credit.” The top 9 results on this search were all financial institutions (banks/lenders) such as Chase Bank, PNC Bank and LendingTree. Results found on page 2 were unrelated to banking or lending as they included videos of music, the LINE messaging application and The Line Hotel; therefore these results only used the word “line” in relation to the original query, rather than answering the query.

In 84% of the cases SE Ranking flagged, the unrelated listings were about one word of the query. The rest followed two or more words. Single-word keywords in its data never came back with unrelated results, which the company says could be because there was no other word for the results to follow.

In one of Nozzle’s examples, “easy strawberry jam” on Sept. 30, the AI Overview still gave a jam recipe while the organic results were about the word “easy,” including the TV series Easy. Nozzle found the page title still showed the full query on 99.8% of the pages that dropped words, with no “Showing results for” notice.

Below The Top Spots And On Page 2

SE Ranking saved up to 50 organic results per keyword, and none of the groups of unrelated results it found started in the top five. They started at positions 6 to 8 in 3.5% of affected result sets and at position 9 or lower in the other 96.5%. SE Ranking notes that a quick look at the top results could make such a page seem fine.

Since Sept. 16, page 2 ignored part of the query on 12.6% of the desktop searches Nozzle could judge on both pages, compared with 2.2% for page 1. On 93.6% of the page 2s that dropped words, page 1 still answered the query.

How Often, And What The Data Can’t Show

The way SE Ranking and Nozzle assess a search issue differs, therefore their rates cannot be compared directly to one another.

On Sept. 16, the estimated percentage of keywords in SE Ranking’s sample whose saved result list included any unrelated listing was 0.09%. On Sept. 17 the estimate was 57.3%, and by Sept. 24, which was the end of the test period, the percentage had grown to 67.2%.

Nozzle will flag a search results page if a term that has historically appeared in search results for the keyword appears in 20% or less of those results. Using this method, Nozzle measured that page 1 or page 2 lost terms on approximately 13.9% of the desktop searches it judged on both pages from Sept. 16 through Oct. 21. The highest rate of losing terms was approximately 39.9% on Sept. 28. Nozzle also observed short-term spikes in the occurrence of the pattern since March.

Nozzle viewed this as a different failure type from the “poisoned” search results it reported in October where the query is still answered; however the keyword’s typical websites do not appear. According to Nozzle’s data, when a page lost terms, the keyword’s subsequent search page(s) within 3 days provided an answer to the query about 85.8% of the time.

The numbers shown in SE Ranking’s daily report spanned from Sept. 17 through Sept. 24. SE Ranking’s baseline tests run in October utilized differing collection settings. A sampling of cases was assessed by two independent AI reviewers, and 40 disputed cases were manually reviewed. SE Ranking adds that its data only covers organic listings and can’t tell Google’s intent or what searchers saw, and that Google hasn’t publicly explained the results.

Why This Matters

SE Ranking notes that if an unrelated result has taken over the lower part of a search engine results page (SERP), a website which normally appears there may appear to be missing from a tracker report without its rank necessarily changing for people using the Google search engine.

Before reporting a late September/October drop at position 9 or lower or on a second page, go through the collected results from your last update to see if the pages that took your place were related to just one word in the query. Then go back to your next update and look at Search Console data for impressions and clicks on the same query, as SE Ranking suggests.

Looking Ahead

AccuRanker, whose page 1 history check I described on Oct. 23, has updated its post to say pages 2 and beyond have a related problem, with results for only one word of the query. It says it will cover that in a separate article. Check with your tracker to see whether it reviews page 2 or stops at page 1.

Featured Image: Accogliente Design/Shutterstock

Category News SEO Strategy
SEJ STAFF Matt G. Southern Senior News Writer at Search Engine Journal

Follow on YouTube. Matt G. Southern is the Senior News Writer at Search Engine Journal, where he’s covered Google, SEO, ...