Understanding Google’s Keyword ‘Not Provided’ Data

Google quietly released major changes to its search product this month to, among other things, move toward encrypting all search activity. The move is touted as an effort to protect searchers’ privacy. But search marketers faced with the demise of a key dataset wonder if Google has other motives.

When Google’s secure search came online for logged-in searchers in 2011, marketers began seeing that some of their keyword data was “not provided.” In essence, Google’s secure search stripped the referring keyword data out of the information passed to sites with each organic search visit. Originally Google stated that the missing keyword data should hover in the single digits as a percent of overall organic search visit.

The percentage of organic search keyword not provided data has steadily grown, however, to the point where many sites are facing over 50 percent not provided. One site dedicated to tracking the not provided counts across 60 websites shows an average of nearly 79 percent of keyword referrals as not provided, following a sharp increase at the beginning of September.

Sample of not provided organic search keyword data via (Not Provided) Count.

In essence, the keyword data we’re provided with is more of a sample set than a truly accurate data set. In fact, we lack the ability to even determine how representative the sample is because the data simply is not provided. With the recent changes to increase encrypted searches, it’s possible that soon 100 percent of Google’s organic search referred visits will not be tied to any specific keyword.

Because the base goal of SEO is to optimize a specific page for a specific keyword phrase, the lack of accurate reporting on the keywords that drive traffic and conversions to each individual page of the site seriously impairs our ability to plan and measure the success of our optimizations.

Why Google Encrypts Searches

The most confusing aspect of Google’s move is that it only affects organic search referred data. AdWords customers still have full access to the keywords that drive visits and conversions to their sites. As a result, many search marketers suspect that encrypted search is more about pushing sites to increase their AdWords’ activity to receive visibility into keyword data rather than protecting searchers’ privacy.

An unnamed spokesperson from Google provided this statement to Search Engine Land, “We want to provide SSL protection to as many users as we can, in as many regions as we can — we added non-signed-in Chrome omnibox searches earlier this year, and more recently other users who aren’t signed in. We’re going to continue expanding our use of SSL in our services because we believe it’s a good thing for users … The motivation here is not to drive the ads side — it’s for our search users.”

Using Google’s Other Tools to Solve ‘Not Provided’

Regardless of why Google is encrypting searches, the impact is clear. The percentage of organic search referred visits that include keyword data are decreasing. As a result, we need to identify new data sources to be able to effectively optimize organic search performance.

Interestingly, the tools best suited to analyzing the extent of the Google keyword not provided issue are provided by Google, further raising suspicion of Google’s intent.

• Google Webmaster Tools. The “Search Queries” report shows the top 2,000 keyword phrases that drove traffic to the site. The “clicks” measured are approximations, however, rather than true representations of the number of visits Google drove to the site.
• Google AdWords Paid and Organic Report. Part of the Dimensions tab in the AdWords Campaign interface, this report shows keyword referral data for paid and organic searches side by side. The organic search data is reportedly pulled from Google Webmaster Tools. For more information on this report read my previous article, “Google Offers Paid and Organic Search Report.”

How the Data Stacks Up

Unfortunately, as typically happens when you compare data from different sources, comparing the organic search referred keyword data from your analytics package to the Google AdWords Paid and Organic report and the Google Webmaster Tools Search Queries report raises some questions.

The charts below show a comparison between one site’s Site Catalyst Google organic search-referred data, Google AdWords Paid and Organic report, and Google Webmaster Tools Search Queries report. The numbers in red highlight the largest number of visits recorded for that keyword across the three data sets.

Keyword comparison from three different Google tools.

If the ratio of numbers was consistent — if one data set always had the largest number of visits for each keyword — I would feel more comfortable about this data. As you can see, however, ratios are inconsistent in a way that looks almost random.

Further, because the AdWords report pulls data from the Webmaster Tools report, I would expect the two data sets to be far more consistent on a keyword-by-keyword basis. Instead, AdWords shows 295 keyword phrases driving 4,973 total organic search clicks and Google Webmaster Tools shows 1,359 keyword phrases driving 7,828 total organic search clicks. That’s a large difference in both the number of keywords reported as driving organic search clicks and the number of clicks received.

Your experience may differ with these reports. Check your reports and compare the data keyword by keyword to see how reliable the data feels for your site.

Diligently adding keyword phrases that drive organic search traffic to your paid search campaigns may close the gap a bit between the data reported in AdWords versus Webmaster Tools. The time lag between identifying those keywords and adding them to AdWords campaigns means that businesses will always be a step behind in analyzing the data. It also means that Google stands to increase its AdWords business significantly.

Segmenting the Data

In addition, if your business needs to analyze data in a way that is inconsistent with your AdWords campaign structure, the Google AdWords Paid and Organic report has its own portion of inscrutable data.

For example, the company whose site is represented below serves the U.S. and Canada. Its AdWords account contains campaigns for both countries, with the campaign names designating their country target, such as “US – Campaign 1” and “CA – Campaign 1.” The report offers ways to filter based on campaign name. So it’s possible to run a report that shows only the paid and organic data for just the U.S. or Canada based on a filter of the campaign name.

However, for keywords that drove organic search clicks when the site’s paid ads were not displayed, no campaign is associated. Because Google doesn’t offer a location filter on this report, those organic search clicks that aren’t associated with a campaign cannot be identified as searches from a particular country. Google certainly has this data, but they’ve chosen not to provide access to it.

Imagine the lack of visibility into organic search this causes a business that serves tens of countries or manages multiple distinct businesses through a single AdWords account.

In the chart below, this business cannot depend on AdWords paid and organic data to understand where its organic search visitors live. For “Keyword 1,” 10 percent of the clicks are not attributable to a country based on the lack of available filters in the report. If all 256 of those visitors were from Canada it would increase the number of people searching for that phrase in Canada by more than one third. That’s a big swing.

Sample AdWords organic data across countries.

For “Keyword 6,” 100 percent of the organic search clicks occurred without exposure to one of the company’s paid ads. As a result, the business has no way of knowing if U.S. or Canadian customers are performing those searches. Providing a landing page URL would help to the extent that the searcher is landing on the appropriate content for her country. Providing additional filters or columns in the report would actually solve the issue.

In short, it remains to be seen whether marketers can feel comfortable making data-based SEO decisions with the dramatic differences between organic search-referred keyword data from your analytics package to the Google AdWords Paid and Organic report and the Google Webmaster Tools Search Queries report.