Flawed Google Mobile Usability Test Results

SMS Text
Flawed Google Mobile Usability Test Results

UPDATE: Google Webmasters and Maile Ohye responded to our request for comment on this article:

As many of you know, Google recently announced a major change related to how sites show up in mobile search results, beginning on April 21st.

The official Google Webmaster announcement stated:

Starting April 21, we will be expanding our use of mobile-friendliness as a ranking signal. This change will affect mobile searches in all languages worldwide and will have a significant impact in our search results. Consequently, users will find it easier to get relevant, high quality search results that are optimized for their devices.

The announcement then went on to suggest two different ways site owners could find out whether their site might be impacted:

If you want to test a few pages, you can use the Mobile-Friendly Test. If you have a site, you can use your Webmaster Tools account to get a full list of mobile usability issues across your site using the Mobile Usability Report.

Note how that second option was for site owners who have GWT set up. Many do these days.

Flawed Testing Service

Since I do extensive testing for sites, I routinely rely upon GWT, GA, and several other tools, one of which is Google Page Speed Insights tool (GPSI). Part of the process Google themselves recommends for testing mobile usability is GPSI, which presents conflicting information that is already confusing some site owners and developers.

My Discovery Process

One of the sites I’ve previously audited received a GWT generated alert about how the client’s site currently fails mobile usability.

Google Webmaster Mobile Usability Alert

According to this automatically generated notice, this site has eight hundred and twenty pages with errors that “severely affect how mobile users are able to experience your website.

The notice then goes on to state “These pages will not be seen as mobile-friendly by Google Search and will therefore be displayed and ranked appropriately for smartphone users.” (emphasis mine)

That’s a big deal. Nearly HALF of that site is supposedly suffering from mobile usability problems and this notice specifically states those pages will lose rankings because of it.

That alert then goes on to offer help:

GWT Alert issue fix link

When you click on that “inspect mobile issues” link, it takes you to the recently added “Mobile Usability” report right in GWT.

That’s a great resource (well it will be, when they fix the conflicting messages I discovered) because it lists what appear to be the various flaws in mobile usability.

GWT Mobile Usability Issue ListClicking on one of those listed issues takes you to a list of the pages on the site that failed that particular test.

Clicking on one of the URLs listed gives you a pop-up with more information:

Mobile Usability Error Explanations

The Vortex of Crazy Begins

Note how on the right there’s a button that says “Check live version”? That button launches Google Page Speed Insights tool.

If you’re unfortunate enough to have at least one “challenge” with your site’s framework the current Google mobile usability system is NOT capable of dealing with yet, you’ll end up seeing this:

Google Page Speed Insights Flawed Data

Wait. How did THAT happen?

I clicked on the link to do a live test directly from an entry in GWT that had been included in the errors list.

Except GPSI says “Nope, you’re golden!” This obviously isn’t going to help webmasters.

Here’s where everything went off the rails:

  1. GWT says “Your site sucks, and we’re going to slap you into the mobile search grave-yard for those pages we think suck the most!”
  2. GPSI says “Your site is awesome! Kick back, relax, and laugh all the way to the bank because for mobile usability, your site is going to continue to rank well while sites some other suckers own are going to the mobile search grave-yard!”

This means some developers will think this means there’s NOTHING to fix!

Developers I have worked with on many sites offered the opinion that this is nothing to worry about – that Google is just “working things out” as they approach April 21st, and that since “GPSI says it’s good to go,” there’s NO NEED TO TAKE ACTION.

Now, in all my history of observing how Google’s systems are continually tweaked, changed, modified, improved, and otherwise re-calibrated, I know they do a LOT of testing. And I know that sometimes they make mistakes in their assessments, only to later recant, after sending out notices by mistake.

So maybe they’re right, I thought…

But what if they aren’t?

Ask Someone Who Knows

After I blasted a half-dozen rant-tweets on Twitter, with no response from the GWT team (not sure if they were too busy that day or just didn’t like my tone – a common challenge I face when I rant), I posted this up on my Facebook page with a cc to a few people at Google:

A call for help

No, I don’t have an “inside connection”. I’ve just worked hard over years of time to build trust with people I respect. So this was one of those rare situations where I pinged them, hoping one of them would be able to find out what’s up and provide some answers.

Maile Ohye Responds

I <3 Maile. She’s Developer Teams Program Lead at Google. She’s not usually someone SEOs go to for info – it used to be Matt Cutts, and more recently, it’s mostly been John Mueller.

However Matt is out of the loop these days (and I am ecstatic that he’s living a more complete life), and John does the best he can, however, he’s been bombarded non-stop since Matt stepped down.

Maile though – she’s amazing. Always goes right to the advanced stuff when it comes to her own posts and guidelines to help site owners. So I really appreciate that she checked into this for me.

The Core Problem

Google Page Speed Insights is not currently set up to process pages the same way GWT’s other internal systems are.

The specific problem? GPSI doesn’t emulate Googlebot.

That’s a really big deal. Because for Mobile Usability, if any of your pages are blocked in the robots.txt file, GPSI won’t know it.

And since GWT is RELYING on respecting the robots.txt file, if you have any critical layout or display files blocked there, GWT won’t find them, and that can result in your site FAILING the Mobile Usability test.

Here’s Maile’s exact response confirming that:

Maile's Response

I highlighted the two critical points:

“Some of the examples’ files are blocked to Googlebot” and

“And you’re totally right — PSI doesn’t fetch with Googlebot so not possible for you to tell.”

Wow.

BOMBSHELL!

So GWT is currently communicating to site owners when there are errors on the site.

Except if files vital to site functionality exist so as to ensure THE SITE WORKS FOR MOBILE USERS are blocked by robots.txt file, Google is now (at least for now) taking the position that your site is NOT MOBILE FRIENDLY.

All because Googlebot is involved, REGARDLESS of whether a site renders properly for mobile visitors or not.

Google Shoots First, Sometimes Never Asks Questions

So what I interpret that to mean is: Google is, at this point, of the opinion that “either you do what we demand you do, regardless of whether its important for any other reason OTHER than our use, or we’ll penalize you or filter your pages out, which might as well be a penalty.”

We can argue six ways to next summer whether that’s right, wrong, fair, unfair, or whatever. I can only hope Google will make a retraction and give innocent site owners the benefit of the doubt, if mobile layout scripts are behind the robots.txt file’s protection system, and not be harmed on April 21st.

I DOUBT they’ll retract, though. Just like so many other issues, at this point it’s probably full steam ahead.

So what do we do?

Google Mobile Friendly Testing Tool

One of the testing methods they suggest (and that Maile says is the best way to test now) is to run individual URLs through Google’s Mobile Friendly Testing Tool.

When I ran that same page previously tested and “100% good to go” in GPSI, here’s what I got:

Mobile Friendly Testing Tool Results

Note that the first thing you see here is along the lines of, “NO, THIS PAGE SUCKS FOR MOBILE.”

Except we know that it may or may not be true in this situation.

Fortunately, that screen does communicate the possibility that their test was flawed.

Mobile Friendly Test Caveat

Sadly, even though the report states this may be due to a robots.txt file issue, (and that means “not necessarily an actual flaw in your site design”), even this screen statement goes on to link to GPSI!

Which would just generate more confusion.

Just Confirming Facts

Just to confirm what I thought I found to this point, and what Maile stated was an issue, I went back to this client’s site and looked into the source on an example page. And sure enough I found this:

Some of the scripts blocked from Googlebot

That’s just two of the many files I found at the code level used for site functionality.

Note how they’re both in the /wp-includes/ folder.

Robots.txt File Layout Blocking Entry

And sure enough – there you have it – the /wp-includes/ folder is blocked in the robots.txt file.

So Is It Or Isn’t It Mobile Friendly?

Until we open up the robots.txt file or change where .js and css files are located to give Googlebot accessibility, we can only assume. And that’s dangerous.

The “Google’s Playground” Question

So at this point, I’ve communicated with the dev. Of course we’re in agreement that this should NEVER have had to happen.

Not because .js files shouldn’t have been blocked from search engines. Because until Google decided they want to do everything they can algorithmically to emulate users, this wouldn’t have been a problem.

And because of that I can’t blame them. I’ve been saying for a couple of years “SEO is the search engines attempt to emulate user experience through formulaic methods”.

So from THAT perspective, yeah, sure – give them access to JavaScript and CSS files.

Except now it’s the bully on the block issue again – where site owners will likely be forced to “comply or suffer” and how many site owners won’t know this until it’s too late?

How many site owners will suffer in mobile search and never have a clue why?

How many sites will lose real revenue, have to lay off employees, cut back on quality of life activities at home, or go out of business altogether because of this?

Cover Your Assets

Bottom line, though, is this – at this moment, we cannot rely on GPSI anymore regarding its Mobile Usability data, at least in situations where files may be blocked from access in the robots.txt file.

Except that leaves all sorts of “what if” other questions. If GPSI doesn’t rely on Googlebot, what other flaws exist in GPSI specific to page speed data? And what OTHER tools Google provides are like that? How many other flawed data results points exist across the entire sphere of their testing system?

Hopefully, we’ll get more information at some point and with any luck, April 21st may NOT be the day they bang the “unfriendly sites” up against the wall.

Alan Bleiweiss
Alan Bleiweiss is a Forensic SEO audit consultant with audit client sites consisting of upwards of 50 million pages and tens of millions of visitors... Read Full Bio
Alan Bleiweiss
Subscribe to SEJ!
Get our weekly newsletter from SEJ's Founder Loren Baker about the latest news in the industry!