Jonathan has helped optimize...

« Measuring and Optimizing Engagement: The Audible.com Case Study | Main | Marketing Speak »

What Google’s Quality Score Is & What It Is Not

Picture_2_2Over the past few months at OTTO Digital we have done a series of search ad optimizations for Google advertisers using multivariate content delivery to optimize ad titles, descriptions and visible URLs based on landing page engagement and conversion rate. In addition to improving client ROI we’ve been provided a fascinating look at the effectiveness, or lack thereof, for Google’s Quality Score. Across multiple tests we have found that ads with the highest engagement (as measured by click action on the page) and conversion rates are being displayed less than ads with the highest click-through rates. Substantially less.

Since this is really hurting advertisers ability to generate the highest ROI from AdWords (and my job is to ensure that they do) I’ve attempted to find out all I could about Quality Score.

What do we know about Google’s Quality Score?

Let’s take a look at what Google has revealed about their three different Quality Scores.

Search Ads:

QS1- Minimum Bid: There is a quality score that determines your minimum bid. This is based on these factors of relevance to the keyword:

• Ad Text (as measured by CTR)

• Landing Page (AdsBot reads page text and informs QS based on keyword relevance and ad relevance to page text)

QS2- Position: There is another quality score that determines your ads position. This is based on these factors:

• Ad Text (as measured by CTR)

• Keyword Relevance to Query

Content:

QS3- Content Position: This is the new kid in town. The quality score for content determines you ads position based on these factors:
• Max CPC (as set by the advertiser)

• CTR (on the site you may appear on or on similar sites)

• Landing Page Relevance (to the Ad copy)

Also of interest is how Google’s Ad Optimizer works in relation to Quality Score. In fact, according to Google, prior to the recent Quality Score update, Quality Score was not taken into account at all the past few months if Ad Optimizer was turned off. Now if Optimizer is off Quality Score is indeed a factor. Hmmm.

How is Quality Score affecting advertisers?

Quality600It seems many advertisers are up in arms. Especially with this latest update that now includes the landing page factor of contextually-targeted ads. What is clear is that Google is great at measuring the effectiveness of the ad based on click-through but as I suspected at the launch of AdsBot they have no idea or ability to do what they claim, measure relevance of the keyword or ad with the landing page and effectively factor that into ad serving to deliver the best user experience.

Look at these impression levels from a recent multivariate optimization for an AdGroup. There were 9 ads live in the AdGroup and Optimizer was turned off.

Headlines & Impressions

Recipe A-C
Headline: “{keyword}”
Impressions
Recipe A 265909
Recipe B 265045
Recipe C 264784

Recipe D-F
Headline: “Find {keyword}”
Impressions
Recipe D 47093
Recipe E 47289
Recipe F 47066

Recipe G-I
Headline: “The Wait is Over”
Impressions
Recipe G 46518
Recipe H 46795
Recipe I 46579

Remember the aforementioned note about having optimizer off? This same test done before the recent Quality Score update had impression levels almost identical across all tested recipes with the optimizer was turned off.

But here’s the most interesting thing of all. Let’s take a look at the conversion rate for each tested recipe above.

Impressions & Conversion

Recipe A-C
Headline: “{keyword}”
Impressions & Conversion
Recipe A 265909 5.72%
Recipe B 265045 5.09%
Recipe C 264784 5.49%

Recipe D-F
Headline: “Find {keyword}”
Impressions & Conversion
Recipe D 47093 8.59%
Recipe E 47289 5.24%
Recipe F 47066 7.76%

Recipe G-I
Headline: “The Wait is Over”
Impressions & Conversion
Recipe G 46518 9.84%
Recipe H 46795 8.11%
Recipe I 46579 11.76%

Clearly the ads in Recipes G-I are more relevant. Based on the data about 100% more relevant than the ads in A-C that had 6x the impression levels. Even the ads in D-F were significantly better. Why were these six ads served the least? The answer is obvious. Their CTR was much less than the ads in A-C.

The ads in recipes G-I did two things. They drove highly qualified traffic and they converted incredibly well. The advertiser was spending less and getting much better ROI on these ads than the others. Yet their “quality score” is much lower than the other ads. This is because Google does not factor landing page relevance into the quality score for ad position.

Another important point from this study (I will presenting this in its entirety at SES Chicago at the session Ads in a Quality Score World) is that if keyword query relevance of the ads is a factor than we can assume that keyword insertion would benefit quality score (It seems to have been a factor above). But what happens to users when everyone is doing kw insertion? What happens is the description lines become the overriding factor for determining ad relevance, based on conversion rate. How is Google weighing descriptions into quality score? My feeling is minimally, if at all.

At SES Chicago, I will present how description line relevance factors into to ad impressions, CTR and conversion rate. We discovered this only because with multivariate optimization we can take the individual element of the description lines and measure their factor of influence on these metrics.

How should advertisers respond to quality score?

The notion that Google is optimizing for anyone’s performance other than Google’s is misleading and shortsighted. Validating this idea is the lack of transparency for advertisers. For example, we know there are historic factors that affect Quality Score. What are they? Google needs to provide advertisers transparency. Andrew Goodman who’s joining me at SES on the QS panel has just written eloquently about this subject. The second thing that needs to be done is that landing page quality must be a quality score factor for position.

Advertisers deserve better. Google, the experts in user experience should know better. Yahoo and Microsoft, are you listening?

TrackBack

TrackBack URL for this entry:
http://www.typepad.com/services/trackback/6a00d8341ccaa353ef00d83454296769e2

Listed below are links to weblogs that reference What Google’s Quality Score Is & What It Is Not:

» Google Quality Score Helps Determine AdWords Ad Delivery from Search Engine Roundtable
AussieWebmaster has posted information in a Search Engine Watch Forums thread where a Google representative has told him that Google's quality score helps determine the ad delivery. Here is the quote: Regarding the serving, this actually is a common qu... [Read More]

» In The Trenches, March 28, 2008 from Search Engine Land: News About Search Engines & Search Marketing
In The Trenches is a weekly spotlight of tips, tricks, and news about the tools search engine marketing professionals use to give them a leg up on the competition. Today: News from the search engines, today's in-depth look, "What You Don't Know About ... [Read More]

Comments

Lots of questions and some mind maps about quality score:
http://www.adwordsmadscientist.com/slap_q_6.php

Thanks for this article. I agree with you that Google might be holding a gun to their own head, with this new quality score nonsense. I wish they would actually listen to their advertisers. Personally I've given up on Adwords, it just seems to difficult to make it work. In the past 2 days I've set up 2 new campaigns and had them taken down, in just a few hours by this supposed quality score. I think Google had some user focus group session, and what the users asked for is basically "Organic Paid Results". From the Quality Score guidelines, this seems to be the case.

What Google is missing is that if their advertisers cannot make a profit, it is just a matter of time, until they go bust.

Regards

David Lucas

Read Great Articles on Stress Relief
http://www.GreatnesCircle.com/articles-on-stress.html

This is because Google does not factor landing page relevance into the quality score for ad position.

I have to disagree. A landing page optimized to a specific keyword has been the best method of raising quality scores. Now the algorithm for how a quality score is translated into an ad position is a mystery to me, but I think your statement is simply too broad.

The comments to this entry are closed.