Jonathan Mendez's Blog

How Semantic Ad Apps Deliver Relevance to Implicit, Explicit & Latent Intent

Every online endeavor starts with a goal. In search that goals is often explicit. In content that goal becomes much more implicit. What’s important to note is that just because a person is in content it doesn’t mean their goal is lost or not as important as it was in search. People spend much more time in content then they do on search engines. In fact, in content there’s a higher propensity for secondary and latent goals to emerge. Content is a goal farm but nobody has been plowing these fields.

Google owns the farm in search but where the online interest paths begin, recovery & discovery, transcend marketing channels. Building marketing applications for interest and intent in another channel needs begin in a different place. It just so happens where interest and intent end in Search they begin anew in Publisher content.

Actually, in 97% of searches the intent never ended, we just have marketed like it has and failed to create solutions meeting the needs of “searchers” other than the back button. Semantic ad applications are one solution. They create interest paths that can maintain intent, create intent, trigger latent intentions or marketing to implicit intentions. This is powerful stuff.

I briefly touched on the concept of how to capture these intentions by flipping the SEM experience in a previous post. I think the concept deserves a deeper exploration because it gets to the root of how semantic ads deliver relevance to the goals and intentions of an audience. Let’s compare the two experiences.

Search Experience

Step 1 - User Control: A query is the strongest element in all of marketing. In the user-controlled medium of the web it has added power. The act of input - the ultimate in user control over the medium. The expression of language – the definition of relevance.

Step 2 - Targeted Content: While the query is the goal for the user, it is the rule for the delivery of content. There are a number of rule inputs associated with Google’s algorithm -- some say as many as 500. However, the words that are entered and the order of them are rules 1 & 2 from which everything else follows.

Note: This proves the value of content targeting greatly exceeds that of behavioral targeting but I’ll save that for another post.

Anyway, what happened here is the Search Engine Result Page (SERP) content is delivered dynamically on the results page and as subsequent queries are entered, new rules are used.

Step 3 - Contextual Relevance: The content that comprises the search engine results page is the expression of the best efforts of the algorithm to deliver content relevant to the query. After this content is delivered to user regains control over the deciding relevance based on the context of their original goal and the secondary & latent goals that are involved in the consideration of the results set but not expressed in the query.

Step 4 - Publisher Content: Once a decision is made as to what result content seems most relevant to our searcher she then crosses experiences landing on a new page with where she can (and will) proceed down any number of new paths.

If you’re familiar with SEM those concepts aren’t new to you but if you come from the Display side or other channels that process is important to understand. Search is an application (content delivery) built on top of a platform (the web). Successful marketing strategies of the future will have to be created around this same formula.

Semantic ads have the good fortune of being built around this formula and are able to replicate the entire Search experience, effectively continuing the interest path by dynamically building new paths, or as I’m now calling them, RAMPs.

RAMP Experience

Step 1 – Publisher Content: We start where the people are. Content. Semantic is defined as "the meaning of words." It makes sense that the semantic ad application experience begins with content and the ability to extract meaning from it.

Step 2 – Contextual Relevance: The interest path we are providing is rooted in the content experience. To trigger interest and intent we must be contextually relevant to the content. While you may discover some interesting counterintuitive relationships of ads and content (in one semantic campaign laptop ads got better response on content about desktops than on laptop content) there is no real marketer genius at work here. This is the semantic technology at work.

Step 3 – Targeted Content: The “wow” factor of Semantic Ad Applications is taking the meaning of the page and delivering content into the ad dynamically through the API. Just like in search, semantic ad applications use rules to deliver content and functionality.

Step 4 – User Control: One of my optimization rules has always been to give users control of the experience. We are in a user controlled medium and no matter how smart you are as a marketer you will never understand the interest and intent of your audience better than they themselves will. Semantic Ad Applications give the audience the ability to query and navigate the information and content. This is at the core of the performance we’re seeing -- making these ads not “marketing speak” but useful, helpful and ultimately powerful utilities.

I look forward to exploring the idea of creating interest paths further in the coming weeks. There is a lot here to chew on and things are coming together quickly and we're building more RAMPs even as I try to define what this can ultimately look like. Like the rest of Semantic Web it’s looking bigger, better and more relevant every day.

The Psychology of Optimization

I saw this presentation from Tara Hunt on Dave Kellogg’s blog. Dave is CEO of Mark Logic where they are doing some amazing things with XML and content management. In any case, this preso is exceptional. It touches on a number of optimization factors that I feel very strongly about and incorporate whenever possible, namely user control, creating flows, alleviating confusion and consideration. My favorite slide of all -- “you just can’t build prediction models for this stuff.” It’s worth your time.

| View | Upload your own

.

The Dawn of Semantic Ads

Peer39 announced Friday their release of SemanticMatch™ ad technology. A few months ago I had an opportunity to preview SemanticMatch with Peer39’s CMO James Oppenheim. It further validated for me the value that is created using semantic technology to deliver relevant content in a context that will create consumer intent. This announcement follows on the heels of Dapper releasing their MashupAds technology a few months ago. Clearly, semantic advertising and the use of semantic web for marketing is starting to take root.

Personally I feel privileged to be working with advertisers and technology providers in pioneering the use and success of semantic ad technology. After just a few projects I am certain semantic web will redefine online advertising -- especially display (and if you’ve been reading this blog you know why I feel that way). There’s one piece I’ve yet to touch on however and I think it’s important.

Early adopters both on the publisher and advertiser side will have an incredible competitive advantage. Remember the early days of Overture and AdWords? Where was your business when CPC was so low and users had no idea those were ads? Ironically, those same paradigms apply now with semantic ads.

Semantic ads are really dynamic applications thus they don’t look, feel or seem like banners advertisements. When created well they are relevant and rather helpful tools to further information/content discovery. Here lies the key to advertising success. Don’t believe me? Here’s what David Ogilvy had to say on the matter:

The other huge opportunity is that semantic ads now are leveraging CPMs based on banner performance. This effectively exploits undervaluation of publisher content both in raw ad performance and the added value that the content achieves when it’s matched with semantic ads. Simply put, Semantic Ads, Applications & Widgets provide higher value to advertisers and thus higher value to the publisher -- but only the advertisers are reaping the benefits at the moment.

Ideally, advertisers and publishers will work in concert to increase value for everyone in the online ad ecosystem with a focus on creating great experiences by mashing up content and advertising. However, for the next two years forward thinking marketers will have an ROAS field day.

There are so many other things going on in the online ad space that semantic and intelligent web seems to get lost in the noise over Social Media, Behavioral Targeting, Widgets & Video. Even when it’s mentioned the perspective seems to be focused on semantic search and a Google killer. This has been a silver lining.

Not being under the spotlight has benefited these semantic advertising start-ups as they’ve quietly gotten their early rounds of VC and developed under the radar. Those days however are coming to an end. Semantic web and the ads that feed off it will crossover every digital media format and channel. With the performance and consumer benefits I’ve witnessed I wouldn’t be surprised if in a few years they single handedly destroy the BT. The value being created for advertisers, publishers and consumers is just too great to be ignored much longer.

Landing Page Speed Hits Google Quality Score

As I predicted last year, today Google officially incorporated another landing page factor into their quality score algo. The new quality score factor is page load time.

It seems what Google is trying to do is prevent multiple redirects. For example, if you click the Sony Style ad on this SERP you will see it has no less than 5 redirects. That’s the major issue being addressed though it will be interesting to see as noted above if there is any other design, tracking or testing practices that could come into play in a negative way for minimum CPCs.

Besides multiple domain redirects for things like analytics and tracking, potentially this could have an impact on landing pages that use javascript to make calls for content, AJAX for navigation and forms, as well as flash and video use. No other guidelines were issued around speed so we’ll have to see how sensitive the variances are.

The Sleeping Giant: Ad Factors in Conversion Rates & ROI

If you've done multivariate tests across AdWords to a single landing page in order to optimize the ad you know a little known truth about conversion rate optimization from media that not many realize. The ad creative (alone) can have an enormous factor of influence on conversion rates.

I’ve also seen results from multivariate tests on display ads that included factors of the banner and factors of the landing page across a single test. What I’ve seen is the main factors of influence that effect conversion rate are from the ad. Yet, when we think about optimizing for conversion rate we almost always think about optimizing landing pages. Why?

In the past few years I’ve given quite a few presentations on ad testing & optimization - here's one of them. This became really important in search with the advent of quality score in 2006. As ads become more advanced both through third-party ad serving in display and with Google’s ability to serve rich media ads and widgets on Search results pages, the factor or influence ads can and will have over ROI and conversion rates will only increase. The other factor on the horizon that may wake the sleeping giant is the rising use of PPA or CPA based pricing from search and display.

Ads set expectations that your product/service must deliver on. We all know the Friday newspaper auto ads that bait and switch. Ads that “just get them to the showroom” are not good enough online. The ad is part of the user experience as much as the SERP and the landing page and deserves the same level of attention to testing and optimization. Soon it will demand it.

Intelligent Web: Where Search & Display Advertising Meet

This past Tuesday at SMX Advanced in Seattle, I presented my ideas and work relating to how the Intelligent Web will fundamentally change and realize the promise of digital marketing. In this post I expand the points and provide my presentation.

Since the inception of online display advertising it has been mismatched with the web. It is fundamentally old media in pricing, form, strategies and executions -- passive persuasion and technology in an active medium. This is why display performance and ROI sucks. Yet, half of all online ad dollars are spent on display advertising.

As Google and SEM have shown, advertising in a user-controlled medium requires something much different. There is no debate that the web is a user-controlled medium and little argument that search is the application that drives the medium.

Search is user controlled. That is why it works so well for people. The experience of providing control over the query, results and refinement of content (and the subsequent back-end ability to deliver it) are much more important to digital marketing success than what you know about people. The simple fact is that consumers know more about what they want than you ever will.

Q: How powerful is this?
A: User control and the user experience it delivers make it possible to build a brand like Google with no advertising.

It’s also important for marketers of all stripes to realize about search that it is not a marketing channel. It is an essential human behavior. Search is both a means to discover and recover information, products, services, etc. and an essential part of social interactions. Search takes place offline all the time. The web just makes it much easier and does it insanely better (for more of this thinking I would refer you to Gord Hotchkiss excellent blog series on MediaPost).

Q: So where does Intelligent Web (APIs, RIAs, JavaScript, & Semantic Markup) fit with Display and Search?
A:It blurs the lines between the two.

At the core of Intelligent Web is a database with an amazing ability to query, mashup, return, present and optimize relevant information in ways not possible prior. Advertising applications built on top of this can be distributed with third-party serving across millions of impressions. This effectively takes your site data & functionality and distributes it in a scale never possible before. Best of all, Marketers can retain control over the delivery, targeting & placement of the ads but consumers retain the ability to define their experience. This changes all the rules and the results.

These advertising applications (or adplications) also do another really cool thing. They flip the consumer search funnel. Here’s the general search hierarchy:

1. Tools – Start with a query
2. Intelligence – Deliver content based on the keyword
3. Context – Select a (perceived) relevant link
4. Content – Go to a web page

Here’s how this it is flipped with adplications.

1. Content - Reach people at the page they’re on
2. Context – Serve ad when/where contextually relevant
3. Intelligence – Deliver content based on a rule
4. Tools – Allow people to query the DB

The most important piece of success with this 180 is the first piece, creating intent. I’ve written extensively about creating intent in digital as the key strategy to marketing success and how these digital strategies differ to what has been done in the past. This is the creative part of this process. Technology handles the rest!

Intelligent Web, APIs and the ability to distribute applications or “widgetize” the web are for the first time changing the idea of search as a destination and distributing that power in an effective and meaningful way that becomes part of the holistic experience of the web for people. This is a sea change that requires new ways to market.

Instead of waiting for people to come to your site we can now take your site to people and present the ability to query and consume it in a helpful and useful way. As Google has shown this allows for great experiences to be created that build brand value. In many respects it also provides a consumer detour around Google while retaining advertiser controls over ad serving not present in SEM.

So...here is my presentation from Tuesday. It's very simple and most of it was talking to the points I covered in this post.

| View | Upload your own

Sorry, but I’m not linking to the demos at the present time. Check back soon. :)

Macy’s Magical Online Disappearing Act

In today’s Wall Street Journal Macy’s CMO and Chairman of the Online Division Peter Sachse talks about Search Marketing. It’s rare to hear a CMO of a major retailer discuss search marketing so it was with great interest that I read his comments.

WSJ: How do you measure the effectiveness of search marketing?

Mr. Sachse: It's not as precise as we would like it to be. ...We have certain techniques -- surveys, tests, all sorts of things -- to find out how many people "activated." If they searched for "perfume" did they buy it in a Macy's store? We do know if they bought it on the Web site. We have that. What we don't know is what action they took in the store.

With rising gas prices, increased consumer time pressure, higher offline operations costs and lower offline margins his cross-channel answer to how well the intent driven medium of Search was working for Macy's was a bit puzzling. So, I decided to see how well Macy’s digital marketing experience was doing using the very scenario he described, a person searching for “perfume” on Google.

Unfortunately for Mr. Sachse (or his Search Agency & PR team) Macy’s does not appear to be buying the keyword “perfume” nor are they listed in the first five pages of the natural results (I stopped looking after page 5). Not being sure why Macy's is not buying what would seem to be a good keyword where they could leverage their brand – maybe low conversion rates due to poor landing pages, maybe a missed opportunity - I decided to take a look at the Macy's website and see what their perfume category and product pages looked like.

Ouch. I haven't seen a page like that since 2002.

Once I was able to get on the homepage a quick view of the source code (below) shows that they have a number or errors with their website analytics tracking and implementation. I wonder if the numbers Mr. Sachse's team has and that he refers to are truly accurate?

One more coal on this fire. Macy's does not buy their brand keyword. From what I've seen I doubt they've tested the ROI and brand impact of this decision. Especially since every one of the five competitors I searched on (including one I worked with in the past and know their brand kw ROI) buys their own branded terms. Also, with their interest in cross-channel metrics you would think at the very least Macy's would want to know the impact TV and other media campaigns have on search volume for their brand term. How are they not?

This is Q2, 2008. It is absolutely ridiculous that C-level execs have no clue about their digital marketing strategies and execution - especially in retail. It's completely unacceptable for a major retailer's website to be down during lunch - likely the highest temporal period for conversion rate and volume during the weekday.

Search is the single most important channel for branding and advertising (and yes, I said branding). Your digital experiences define your brand and that starts with the keyword query. According the the interview it seems Mr. Sachse is pouring millions of dollars into online marketing initiatives yet Macy's doesn't even have the basics down. You can build a beautiful house but if it's on a poor foundation eventually it will crumble.

Some previous posts that Mr. Sachse and his Agencies might find interesting.

Buy Branded Keywords? A Case Study on Traffic, Conversion and RPV

Brand Marketing in the Digital Age

Platforms, Applications and the Future of Digital Marketing

Expert Guide to Multivariate Testing Success

I’ve been doing MVT for three years across landing pages and websites with over 40 clients like Citibank, Disney, Esurance, H&R Block, Inuit, Microsoft and T-Mobile. As the Founder & former Chief Strategy Officer of OTTO Digital, the services division of the market-leading MVT tool, Offermatica (now Omniture Test&Target) I’ve authored many case studies on multivariate testing – six of which are featured in MarketingSherpa’s Landing Page Handbook. Many more have been posted here on Optimize and Prophesize.

However, even after two years of steady blogging I’ve never captured or shared a complete multivariate testing overview. So I’ve tossed aside two book offers (the client work is more fun and pays way better anyway) and shared everything I could think that would be helpful to digital marketers in a series of posts here on my blog.

If you’re just getting started with MVT hopefully this helps make sense of it all to you. If you’re already experienced hopefully this will give you some new ideas and methods. Whoever you are, you should know that when used properly, advanced marketing technology and analytic data can provide business intelligence that can quickly improve performance with your campaigns and content by making it more relevant to your customers.

Part 1: Introduction to Factorial Designs

Part 2: Choosing a Test Array

Part 3: Strategy & Tactics for Element Selection

Part 4: Creating Element Variations

Part 5: Test Set-Up & Implementation

Part 6: Understanding Result Data

It is unlikely that I will be posting about multivariate testing again in the near future but feel free to reach out with MVT questions. While my new company RAMP Digital will continue to help clients with testing services I expect most of my blog fodder to reflect what I'm most keenly interested in at the moment - pioneering the creation and success of dynamic advertising and landing page solutions (some embedded with MVT) using APIs and the intelligent web. Happy testing!

Multivariate Test Results: Confidence, Stability & Determining a Winner

Getting your MVT tests designed, developing your creative and implementing code are all just precursors to getting your test live. However, once your tests are live success will be determined by how well you did all those things. At this point you are just monitoring results - so relax and kick your feet up...if only it were that easy.

Over three years of doing MVT most of the questions I’ve received have been about determining when a test is over. This is because when a test is live, results are always going to be changing. But testing forever defeats the purpose. Every test needs a time to cut the cord.

So, how to figure out when you’re finished? You need to weight the following factors:

Confidence: Confidence is simply calculating the discrepancy between the results. The better the performance of one test recipe vs. another the more statistical confidence will be achieved. For example if our success metric is conversion rate and one test recipe is A at 5.0% and recipe B is 5.5% there will not be a lot of confidence (that B is 10% better). However, if A was at 5.0% and B was at 10.0% there would be a great deal of confidence (that B was 100% better than A).

Important Point: The confidence metric is based on the data that has been collected. This is not a predictive calculation.

Margin of Error: MOE simply looks at the confidence stats and factors in the amount of data that has been collected. The more data the smaller the increments for MOE. I generally don’t pay much attention to MOE as these swings can be very wide. I know some stat heads might get their panties in a bunch about this but as a marketer who relies on speed this can be a paralyzing metric since so much data is needed in most cases, even with fractional factorial testing.

Stability: Stability coupled with confidence are the two most important things to look at in determining if your test is over. There are two graphs you want to be looking to judge test stability. One is cumulative stability and the other is daily results. Let’s see what these reports look like in the Omniture Test&Target tool.

Cumulative

The main things we're looking for in the cumulative reports are trending and consistency. Once things seem to level off for a period of a week or so, we're looking good.

Daily

The main things we're looking for in the daily results reports are outliers and fluctuation. Once we have a recipe that wins most of the days we're looking good.

Account for Temporal Changes!

Generally a best practice is to let your multivariate tests run a minimum of two weeks. This way you can get week over week results and see if there are any strange temporal behaviors that could be skewing the results. Here it is helpful to look at the daily results. I’m hoping Omniture’s Test&Target will soon be able to graph results week over week (or in other comparative timeframes) like Google can.

Don’t look back!

Successful multivariate testing is about speed (how quickly), velocity (how many) and iteration (how intelligent) based on analytic data. I’ve never regretted stopping a test with a big winner because even after is test is done you are going to be monitoring results. More often than not early results hold up as winners even if the overall improvement levels subside a little bit. For best results I’d much rather run 10 small, quick tests over a month period than 2 large ones.

This post effectively wraps up my multivariate testing overview in six parts. My final thoughts:

Multivariate testing can be a tremendous amount of fun and get you great results but it requires highly dedicated marketers and great creative methodology. Matt Roche the founder of Offermatica once shared three learnings from his time building the most successful multivariate testing tool. I'll end with his great advice for digital marketers.

1. Great marketing comes from great marketers, machines help them aim better

2. Engaged marketers lead to engaged customers

3. Speed is everything

Happy testing!

Speaking on Landing Page Optimization & Widget Ads

I have a couple of upcoming speaking engagements on landing page optimization. If you’re going to be attending shoot me an email and let’s meet-up.

MediaPost Search Insider Summit
May 18th-21st Captiva Island, Fl

SMX Advanced
June 3rd,4th Seattle, WA

Also, I’m very excited that I was just asked to speak at the WidgetWebExpo. I’ll be sharing some of RAMP Digital’s recent work on Adplications, discussing how the Intelligent Web and the use of APIs and Semantics is going to redefine display advertising and showing the first ever Google Gadget Ad that RAMP has been working with Dapper and Google on.

Setting Up a Great Multivariate Test

Part 5 in my MVT series will focus on the key considerations and decisions for setting up your multivariate test so that you get the most learning and best results possible. I’m not going to get too technical here since I am not a developer however I want to share what I feel will ensure happy testing.

JavaScript Implementation
The first and often most difficult part of setting up MVT is getting the javascript implemented on the pages to serve and change the test content and getting the proper tracking in place. I have seen it done in days I have seen it done in weeks and I have seen it done in months. You probably know your site architecture and IT department well enough to know how long this will take. Plan accordingly. The good news is that once that JavaScript is in place it can be used for future tests and content targeting. Every vendor has implementation guides and most of the implementation with JavaScript based testing tools is very similar.

No Ad-Side Variables
Ads can be quite influential on landing page performance, so they are a great (and rather easy) thing to test. In fact MVT done across ads and landing pages most often you will see that ad elements haves the higher factors of influence on conversion than page elements. Because of this, just like if we were working in a science lab we want to control as many variables as possible to prevent noise from getting into our test results. For a multivariate test on a landing page this means having only a single ad that sends people to the page.

Clearly Defined Channel Sources
It’s incredibly important to look at your results by segments. Channel segments like Natural Search, Paid Search (down to the keyword level), Email and others can have be very high impact and may deliver different results. Make sure that your tests are set-up to recognize the proper channel attributions (usually URL parameters) so your results can be sliced and diced and your learning can be many.

Segmentation (Profiles)
As soon as people are coming into your environment you should have your campaigns set-up so they can create non-PII (Personally Identifiable Information) rules based profiles. These profiles can create constant real-time segment creation based on behavior, source (mentioned above), temporal and environment. This profile create no only lets you look at your data in more helpful ways but it allows you to create test and target rules for these visitors either in-session or on a return visit based on what you know about their intentions and affinity.

Tagging the Funnel
Understanding how your test campaign users navigate through your experience can be very useful information for optimization and future test ideas for content and routing. You want to make sure that each step of your conversion path is being tracked. I have seen horrendous conversion funnels that have nullified some great landing page test results. Keep this in mind. Every page has a goal -- and every user has a goal on that page. The goal of the landing page is to meet or exceed the interest created in the ad and get people into the funnel. Keep this in mind as you develop testing success metrics discussed in Part 3.

Value Attribution (AOV, RPV)
Higher conversion rates with lower orders values is usually not a good thing. All this testing work is about two things. How much money did I spend and how much money did I make. Therefore it is imperative that you feed back price attribution into the test results so you can look at your results by what matters. With Omniture Test&Target this is a pretty simple process. This also allows you to do what is many times the most important testing you can be doing, price testing. If you are a publisher site you can create your own value attributions or scoring systems to help quantify testing results. This is a great idea that is unfortunately hardly ever used.

Proper test set-up is an essential part of testing. Make sure you allocate the time and manpower necessary for code implementation and QA. If your site uses redirects you want to take extra precaution to make sure that the first party cookie is passed consistently across domains throughout the user experience.

Once you are set up you can go live and start the best part of testing, monitoring the results. The next part of this series will look at results monitoring and will attempt to answer the question I get asked the most, how do I know when my test is over?

MVT Element Selection - Strategy & Tactics

In part 2 of this series we discussed the first part of multivariate test design -- determining your test array -- how many elements and how many variations of each you you should test. In parts 3 & 4 we will cover the rest of MVT test design, element selection and element variation.

Successful testing starts with clear goals. It is these goals (metrics) that will inform your strategies for element selection and variation. Being that we are interactive marketers our metrics tend to be an action of some sort or another. This action (sometimes referred to as conversion) can be a click, it can mean a sale, it can refer to any outcome. For example, if I am trying to increase Average Order Value (AOV) or Revenue per Visitor (RPV) I may select different elements (and most certainly create different variations) than if I’m optimizing Order Conversion Rate. Making a decision what success means before you start test design is important but it is essential before you start this part of multivariate test design.

Once we begin testing a large part of the value in MVT is the element contribution reporting. The best tests will answer questions for you around your goal but the element contribution reporting will raise many additional questions that you’ll want to find the answers to. Therefore the next consideration we want to make with our test design strategy is to select elements where we desire to achieve greater understanding of their influence on the outcome.

The nature of MVT and the learning created lend itself to ongoing and iterative testing cycles. In Part 1 I discussed that the nature of fractional factorial testing requires us as marketers to select the elements that we believe are the most important factors to the metric we are trying to optimize. What influences an action? It can vary quite a bit. There are no defined best practices for element selection for each type of test array. Generally however we look to the following elements as those that have the most influence on people’s experience.

Navigation - There are some strong opinions in optimization around having or not having navigation on landing pages. The best thing of course is to test. As you can see with this case study.

Headlines - Headlines are often the most influential element on the page. Many elements may not be noticed on your page but more often than not the headline is noticed as people have been been trained to look for and scan headlines for clues to relevance.

Offers- Offer testing can lead to huge learning. How does your audience best respond to offers and how does this effect your bottom line results? Does 10% or 25% Off make a difference for AOV or Conversion? Do more act with a dollar offer or a percentage. Do gated offers ($10 off orders of $50 or more) get better metrics than straight percentage off offers?

Messaging - How do you present your benefits and unique selling propositions. Are bullet points enough or do you need more copy? In what voice are you speaking to visitors (more on this in element variation). What is your mix of content?

Price - Sometimes nothing makes more of an impact than price. If you have the ability to synch price testing with your back end reporting and analytics this is essential to understanding the market value of your products and services.

Calls to Action - Those little buttons and links really matter a lot. I love doing MVT on the buttons themselves however as an element in an array you can often get a quick win testing a button or call to action link. Highly recommended.

Next up is the most creative and my favorite part of MVT, element variation.

5 Ways to Optimize Google Search Result Video Ads

Video ads are now part of search. Most advertisers will be throwing up made for TV commercials and fail to understand or take advantage of this new medium. That will be their loss. Google’s video ads present a huge opportunity for those that take the time to create compelling video specifically for the SERP (Search Engine Results Page).

Like any other creative you need to tailor your messaging to the unique benefits that the medium has to offer. Here are some basic guidelines in understanding how to use this new SERP media to generate qualified traffic and optimize your search spend.

1. Keep it Short: Search is a fast twitch medium. You have only a few seconds to keep my attention. Get to the point, and get me to your landing page.

2. Keep it Big: The size of the video is 160x120. That’s tiny! Words, offers, messages, coupons, whatever copy or offers you have need to be big in order to make an impact.

3. Create a Flow: The end of the video can’t be the end of your ad. If someone clicks your video you have already pushed them two steps down the funnel -- step 1 getting their attention, step 2, taking an action. Your video should tee up steps 3 and 4 so don’t spend this valuable time rehashing the same messaging in the ad title and description in video form.

4. Use Images to Your Advantage: The brain processes images much faster than text to generate understanding. Make sure your ads have video images that are helpful to describing your product or service instead of a talking head.

5. Study DRTV: A few years ago I was fortunate enough to partner with Sy Sperling in a DRTV (Direct Response Television) venture after he sold Hair Club, a brand he built through video marketing. He taught me quite a bit about video advertising and much of it is transferable to video ads on the SERP. You can learn quite a bit studying this advertising format. DRTV ads are tested, targeted and backed with tons of analytic data. In fact I'll leave you with my old buddy Sy talking about creating a compelling video ad.

Multivariate Testing - Test Design

Part 2: Choosing a MVT Test Array
Also see Part 1: Introduction to Factorial Designs

The results of your tests are often pre-determined by your test designs. Good designs will provide measurable increases in performance and learning about what tested elements are most important in driving that lift. Good test designs will also provide learning even if you don’t get large increases in lift or statistical confidence in the results. As some of you may know designing a good multivariate test is a little harder than it seems. It takes a good amount of experience to understand the inputs and drivers needed. Hopefully sharing some of my experience will help you get better results.

There are two parts of multivariate test design that need to be considered and determined.

1) The test array
2) The test elements

I always pick the array first and then select the elements based on the array selected. Sometimes people do this in reverse and I’ve seen it lead to problems more often than not. Usually because they don’t have the data (traffic) to support the test. Also once you know your array sometimes there is flexibility in the amount of elements and alternatives you can use.

MVT Test Arrays
NOTE: As mentioned in Part 1 of this series MVT arrays are creating using fractional factorial or full factorial design of experiments -- I will only be covering fractional factorial array determination.

An array (sometimes referred to as an orthogonal array) is the sequence of elements that make up the test designs. Arrays are based on the decision of how many elements you want to test and how many variations of these elements. The array you choose will also inform how many designs you need to create and exactly how those pages need to be designed.

Example of an L4 MVT Array
3 Elements x 2 Variations = L4
Note: The control or existing always counts as a variation

In the example above three new designs would need to be created in addition to the control for a total of four. These designs are usually created by placing JavaScript around the areas on the page you decide are the most important elements to what you are trying to improve. As mentioned if you look across the table you can clearly see what mix of elements you will need to create the designs correctly.

In some instances the arrays needed to test more elements are the same as those testing less. For instance 5x2, 6x2 and 7x2 are all L8 arrays. Here is a great tool for determining orthogonal arrays. Some of the most successful arrays in my experience have been 7x2 L8 and 4x3 L9 and with less traffic or faster results a 3x2 L4. The nature of these arrays also reduce margin for error because the use less aliased interactions as described in Part 1.

That brings us to how to find the right array for your test. The main factors to determine the proper array are:

• Time available to run test
• Estimated traffic into test
• Estimated or baseline conversion rate

Let’s explore each of these.

Time Needed for MVT

The main considerations for testing time are the amount of data that needs to be collected in order to achieve statistical confidence and ensuring temporal changes in behavior are accounted to ensure result stability.

Data means visits and visits mean time. Depending on what you are trying to accomplish you can choose to run a shorter test with less traffic and fewer elements and alternatives or a longer test with more traffic and more elements and alternatives. This is one of the many trade-offs we encounter while running tests, especially MVT. It’s where the art of testing meets the science of testing and experience with these decisions can be very helpful.

It is important to understand statistical confidence as it is an important part of all testing. Statistical confidence is a percentage score out of 100 that the resulting data set is accurate. It has nothing to do with how confident the performance will be in the future (this is sometimes where people get confused). Statistical confidence is a result of two factors, the amount of data collected and the discrepancy between the performances of the designs. For example if I have 10,000 tested visits and the best performing design has 42% lift over the next best one I will have 99.9 confidence in those results are accurate. If amount of traffic and/or the difference in performance is reduced the confidence will also drop. With less data or closer results the margin of error is increased.

As far as time considerations, you want give yourself a minimum of one week to run a MVT so you can capture any temporal changes in day/week behavior. Two weeks is ideal. Be cautious before launching tests that there are not any unusual events that would cause different behavior during the estimated test period such as special offers, seasonality and holidays.

NOTE: Estimating the time needed to run a test and determining if a test is over are two different things. I will go into greater detail in a future post on determining when the test is complete.

Traffic and Conversions Needed for MVT

Along with time the amount of traffic and their resulting volume of conversions will determine what array you should use. The best way to determine traffic needed is to back into it based on conversions data. A general rule of thumb is 100 conversions for each design (also called recipes, branches or experiments). Using our L4 example from earlier here’s the formula for a sample landing page test:

L4 = 400 conversions needed for test
Baseline conversion rate 5%
Traffic needed for 400 conversions is 8,000 visitors

So if I average 4,000 clicks per week to this landing page then the estimated length of testing to get statistical confidence and stability is 2 weeks.

Having this information it’s pretty easy to determine you could run a 7x2 L8 for this AdGroup if you can wait another two weeks for results. Or maybe you can raise spend caps to drive more traffic and get results in one week. Once you understand all the inputs there are a lot of levers you can pull. Still, at the end of the day even if you have the right array your test results are only as good as the elements you selected and the messaging/creative used for your alternatives. I’ll cover that in Part 3.

Multivariate Testing Overview

Part 1: Introduction to Factorial Designs
I'm not sure how many parts this overview will be but since it seems MVT has finally "arrived" I'm starting at the beginning and hope to pound out a complete overview in the next month or two of every important factor to understanding and being successful with multivariate testing.

Multivariate testing (also called MVT, multivariable testing or experimental design) is one of the three common techniques for landing page and onsite optimization (A/B testing and onsite targeting being the other two) though it can be used effectively for ad testing as well. While multivariate testing does not usually provide the performance lifts of A/B testing or the ability of onsite targeting to deliver relevance, when used properly it is an essential part of ongoing and iterative online marketing testing and optimization.

MVT provides incredible intelligence and learning. The greatest benefit to MVT is what is referred to as element contribution. Element contribution informs the marketer of the percentage contribution each tested element has on conversion rate. This learning is highly valuable as it provides understanding of the triggers that influence behavior and can lead to numerous iterative test ideas that can provide further learning and improvements in conversion.

There are two main design of experiments that marketers use and multivariate testing vendors base their technology around. These are fractional factorial design and full factorial design. Let’s examine the pros and cons of each.

Fractional Factorial Testing
Fractional factorial testing (also commonly referred to as Taguchi Methodology) requires the marketer to choose two or more elements to test using one or more variations of the selected elements. This method is called fractional because not every page element is tested. The marketer selects what they believe are the most important elements based on what they are testing.

The combination of elements and alternatives is called an array and is used to create the pages used for testing. Certain arrays mitigate the effects of aliased interactions better than others. In my experience the L8 and L9 arrays are very good at this. L8 arrays are used for 7x2 (7 elements and two variations) and L9 for 4x3 (4 elements and 3 variations) MVT.

Example of a 7x2 L8 Array

Pros
The benefit of fractional factorial testing is that less data is required in order to reach statistical confidence in the results. This means that with a solid test design results can usually be achieved in shorter time periods. Usually weeks. This is highly beneficial for marketers that like to “fail faster” and take more iterative approaches to testing.

Cons
Since not every element is being tested certain interactions are aliased. This means that while the results may achieve statistical confidence full interactions between all elements are not measured. If the marketer selected the most important elements these interaction effects can be mitigated. Of course it's not always easy to determine the most important test elements.

Full Factorial Testing
Full factorial testing uses every element of the page to create a test array measuring all interactions. Because so many interactions are measured this method requires a great amount of data (thus time) to achieve results.

Pros
As mentioned each interaction is accounted for so when confidence in the results are achieved there are no questions about possible interactions skewing the results.

Cons
Since every element must be considered and tested a large amount of data is necessary to achieve results. Incorporating variations of each element can also be a taxing creative effort. Because of these reasons full factorial tests need a much longer period of time to gain results, usually months. SInce online behavior can change (for many reasons) over such long periods results may tend to benefit certain temporal swings.

Summary:
As a marketer I have always used fractional methodology and with tremendous success. I like the idea of velocity in marketing -- test, learn, test, learn, test. Instead of one large test I prefer focusing attention on certain areas or elements to achieve deeper understanding. I also believe this iterative approach lends itself the dynamic nature of online marketing.

Part 2 will focus on designing MVT test arrays based on traffic analysis, page design and conversions.

The $130 Billion Dollar Landing Page

Google is a fully digital company. Everything Google has done and is today revolves around the web. Google is also by every account one of the top 3 brands in the world. Until billboards this past year Google has never spent a dollar on advertising. No TV, no print, no radio, No banners, no search ads (that it is pays for ;) - Nothing.

So how did Google build their brand?

It stared with a landing page. The Google homepage is the greatest landing page ever created.

Of course there’s more to it. Google, and Marissa Mayer in particular have been insanely focused on user experience forever. It’s no accident all the links are blue underlined text links or that testing takes place regularly and is a core Google belief.

The other part of the user experience of course is their ability use technology to deliver of relevance.

Still, it started with a landing page. A very simple landing page.

So if you can build a brand -- maybe the world’s leading brand, through digital experience alone what does this mean for brand marketing once everything is digital? What does it mean when a killer landing page may hold the key to your digital success?

What it means is that the experiences people have with your digital footprint will define your brand. They will build it or the can take it down.

Crappy website? Crappy brand!
Irrelevant advertising? Irrelevant brand!
Bad landing page? End of brand conversation!

Of course the offline experience will flow back online as well. Rats in the kitchen? Fell asleep on the couch? Battery issues? These all become part of your brand’s digital footprint - forever!

Success for almost every brand and business is now defined by bringing people to your digital presence and optimizing their experience once there. Time and money spent now on anything else but optimizing your digital footprint is short sighted and likely will be wasted. At this stage of the digital game when the tools and technologies are accessible to everyone there are simply no excuses for a failure to optimize.

The $130 Billion Dollars? That’s Google’s market cap today. Not a bad ROI on testing and optimization.

SMX West Afterthoughts

I wasn’t sure what to expect with SMX West as this was the first large SMX event. I heard great things about at last summer’s SMX Advanced from clients and friends but there was short notice to fit this event into annual budgets of attendees and exhibitors. Now that it’s over I can say that the event was a big success all the way around and portends well for the SMX brand.

The first thing that made an impression on me was the free wifi throughout the conference area and table seating in the sessions. This allowed easy use of laptops for note taking and increased productivity during the sessions. With one exception the sessions that I sat in and spoke on were excellent. Jonah Stein who’s been to many a search conference commented that the landing page session was one of the best sessions he’s ever attended and modestly I agree.

The only negative I took away from the presentations were from the search engines themselves. The presentations (at least the half dozen I saw) from Google, Yahoo and Live were consistently weak. I eagerly anticipated The Blended Search Revolution session only to be bored to tears as the engines added little more insight than a basic product demonstration and no more substance than a cry of “use us!”

The exhibit hall had many of the usual suspects and as always a few new faces. Yahoo was MIA again as they were in SES Chicago. One thing that continues to amaze me is the ever increasing number of SEM tools for campaign and bid management. There has to be some shakeout in the next 12 months here as I don’t know how the SMB or Mid Agency market is going to afford anything like 5% of spend when GOOG and MSFT are rocking free tools.

The “most crowded booth award” went to SEOmoz. I swear there were times where the overflow in front of their booth completely blocked anyone from walking down the aisle. Now that’s a great tactic to build booth traffic -- block the aisle! Congrats to Gillian and Rand on a home run with their first exhibit.

My speaking slots were both on Day 2 and my first session was Landing Pages & Multivariate Testing. I was very impressed by the panel especially Sandra Niehaus, Vice President User Experience & Creative Director, Closed Loop Marketing whom I had not met before. She accurately summarized the UX and creative benefits of MVT and also emphasized the need for clients to quantify goals and objectives of testing prior to launching into it. All great points.

I lead off the session focusing on the basics of landing page optimization and testing. I’ve been trying to make my presentations more conversational and less all about bullet points so I’m not sure how well they’ll play on Slideshare but I’m posting them here anyway for those interested. Also, I'm aware the font looks terrible - it's really Bank Gothic, I swear.

| View | Upload your own

My second session was Search Ads & Behavioral Targeting. Once again I lead off so I provided an overview of segmentation and targeting methodology. I then showed some of the great site targeting abilities and success I’ve had with the Omniture Testing and Optimization tool. This dovetailed into some of the ad targeting work we’re doing at RAMP within Social Media using the great Lotame Crowd Control technology and with adplications developed using the incredible Dapper technology. More on that work in future posts.

| View | Upload your own

Somehow I double booked dinners on Wednesday so I brought together both parties and headed out to steaks with Ian White of Urban Mapping and his investor/local search guru Jake Baille as well as some of my good friends from the Omniture SF office. If you've never checked out Urban Mapping do yourself a favor - they have some of the most killer geo applications out there.

Thursday was spent catching up with work, visiting with some potential clients and partners around the valley and then 4 hours of work (including this post) at SJC. This was my third time in Silicon Valley in the past few months and I'm still not sure what to make of it. I guess it's like Hollywood for geeks (no, really).

Overall a fantastic start to the big SMX shows. I met a lot of great folks and as always enjoyed reconnecting and talking shop with the search pros I admire. Congrats to Danny, Chris and everyone at Third Door Media. It was an honor to be part of of the event.

RAMP is Born to Run

Two and a half weeks. That’s all it’s been since I left to start my own company for the third time in the past decade. In that brief period I’ve signed four clients including leaders in Social Media, B2B Services, Politics and Auto. Every day is filled with talking to prospects, potential investors, potential employees, current and potential partners. Buried in all of this are the more practical and often more fun parts of a start-up that include finding a name, creating an identity, building a website and ultimately establishing a culture. Oh, and there’s real work to do on top of all this! The momentum has my head spinning. So, without further ado I’m thrilled to present my new company to the world.

RAMP Digital

RAMP stands for Relevance Amplified but it means much more. The name combines my love for improving results, launching new ideas and the one thing I know is true about digital marketing -- the more you amplify relevance to people in their digital experiences, the happier they will be and the more successful you will be.

What is RAMP?

RAMP’s foundation is made from the tried and true digital marketing methodologies I’ve developed over the past decade and three basic principles that I believe will define success for digital marketing in the coming decade.

Intent & Affinity are the Root of RAMP’s Segmentation Strategies

I believe that understanding intention is the key to direct marketing success and discovering affinity is the key to brand marketing success. I hate to stovepipe these practices because there is cross-over in each but this basic understanding helps deliver relevance both contextually and directly. Most importantly, understanding consumer intent and affinity allows for the development of segmentation strategies to target creative and messaging. This segmentation provides a core foundation of intelligence to optimize the presentation and delivery of marketing and advertising.

Achieving Success with Advanced Marketing Technologies is RAMP’s Core Value to Clients

In my last post I briefly mentioned the new wave of emergent technologies that are coming upon marketers fueled by the investment boom of the past two years. They will create new ways of advertising, new tools for optimization and a higher degrees of intelligence and analytics than ever before. The problem is most agencies don’t have a clue how to be successful with them. RAMP will be the go-to agency for what we believe will be the best-in-breed of these technologies. We are partnering with emerging players in Social Media, Display & Semantic Web to serve as a conduit between their mind-blowing technology and real world client results. In the process we’ll build a few really cool applications of our own that I hope people and businesses will want to be a part of.

RAMP will also work with marketers using Omniture Offermatica. Having already built a services company around the technology and having used the technology across more clients and with greater success than anyone over the past three years, I anticipate this will be the biggest part of RAMP’s business for the foreseeable future. Certainly the services market around optimization technology is going to explode.

Testing, Targeting & Optimization is RAMP’s DNA

For most agencies and marketers these practices are add-ons. For RAMP, A/B and Multivariate testing, targeted content and profile creations are embedded as part of everything we do. Every RAMP landing page, microsite, widget or application comes with a test plan, optimization roadmap and targeting plan. The technology to measure and optimize results are an embedded as part of the deliverable. We know digital marketing success is based on iterative strategy and the velocity of intelligence collection -- both of which lead to consistent performance improvement. We will always be testing, targeting and optimizing and our clients will have the results to show for it.

So we’re blasting off. These are incredibly exciting times for our industry and I’m privileged to have the opportunity of a lifetime to help shape what’s possible. As always I look forward to sharing success stories (and failures) with you here to help further the dialog around digital marketing optimization. I will keep speaking about optimization, delivering case studies and developing the next generation of marketing solutions to keep pace with the needs of the most aggressive marketers.

Rave on, turn those amplifiers up to 11 and make your performance spreadsheet look like a ramp.

Google: Landing Page Quality & CTR

Some interesting data last week from Efficient Frontier on Search Advertising click-thru rates (CTR). EF quantifies the disparity between CTR on contextual and search advertising (search is 31x greater). Most interesting to me however was the chart below on Search CTR.

Take a look at Google. You can see there was a CTR decline on Google ads until September when things turned around and have continued to improve -- making up 8 months of losses in half that time. CTR now shows a year-over-year increase.

What happened in September?

Google made changes to its quality score (QS) incorporating more landing page factors. Regular readers will recall I predicted even more of this in 2008.

Here is all Google had to say formally on the September 2007 changes. In brief they did this to squash the following:

• Data collection sites that offer free gifts, subscription services etc., in order to collect private information
• Arbitrage sites that are designed for the sole purpose of showing ads
• Malware sites that knowingly or unknowingly install software on a visitor's computer

Going back and checking initial reaction on the blogosphere (and my own recollection) opinion on the change from an advertiser perspective was very negative. Here’s a long post by Marc Grote who leads internal SEM for many Microsoft properties. Marc rails against the changes and sites examples from Hotmail and Games. Microsoft buys lots of ads on Google.

In contrast to advertiser outrage the chart shows the QS changes in September had the intended effect of improving ad relevance. It also shows that while QS was a big winner in driving revenue in 2006 there were some bumps in the road in 2007 as Google attempted to add more data into the QS algo (though it seems they made up the bottom line difference in higher CPCs).

I expect the quality score troika of keyword, ad and landing page to continue to fuel Google’s bottom line. With so much at stake it’s more important than ever to understand how each element impacts your performance.

The Brand Optimization Revolution - The Metrics are Coming! The Metrics are Coming!

A common trait I find among many “red-coat” marketers is a lack of respect for the collective intelligence of consumers. This expresses itself in many ways. One is the “we know what our users want” mindset. Another is the Boston/NY-SF/LA view that folks in Middle America are unsophisticated and stupid. These attitudes are constantly reflected in marketing strategy. In fact I’d argue they express themselves every time a digital marketer does not perform real-time testing and optimization.

Common Sense
The reality is communication technology has always made consumers savvier; more educated and amplified their diversity. We will all agree that power has shifted in the marketplace to consumers as they take on an ever-active voice in the marketing and the ultimate success of products. Reviews, recommendations and social networks necessitate factual and helpful messages and marketing that quantifies benefits to consumers. This is the only way consumers will accept your voice. Some might say straight talk lends itself to more DR focused marketing – and it does —but it can also become best friend of a brand builder.

The Brand Tax
What the brand marketers on Madison Avenue don’t know or care about is that brand has always ruled the highly transactional practices of email, affiliate and search marketing both on volume and ROI. There’s no reason brand can’t have the same impact on other forms of digital – or in other words eventually rule digital the way it does traditional. No medium wrings emotions out of people and allows them to share them as easily and instantly as online. The medium itself was created as a means to share. So if brand is about a relationship the digital domain is the place where relationships can be easily established and maintained.

New Metrics
As we establish “digital” brand metrics optimization and success needs to be based on user behavior and action. Google is all over this. A brand revolution has begun with its seeds being sown in Mountain View, CA. This extends far past Google holding hands with Publicis. It permeates every part of the digital marketing landscape.

Crossing the Delaware
Marketers and Agencies can either be with this revolution or against it. So pick up a musket and don’t leave money (or your client) on the table. Here are the battlefields and the ammunition you’ll need to fire.

Search & Contextual Ads - Bids: I expect Googl