Will Scully-Power: Managing Director, Datarati

Optimisation is a fuzzy word. It’s been used to describe everything from decreasing page load time to improving user experience.

1) Testing - the process of testing alternatives on your site to see what converts best. Testing is most often implemented as simultaneously delivering alternative content to your visitors in a randomised fashion.

2) Targeting – the process of displaying relevant content to a particular segment or individual based on implicit and explicit variables such as gender, search terms, and intent. Targeting is most often implemented as delivering content to your visitors in a rules-based fashion. For some companies with enough site traffic, targeting can be delivered to the individual in an automated fashion.

Testing and Targeting: 1+1=3

Both efforts rely on the ability to deliver content dynamically on your site. Both are designed to increase key success metrics on your site. However, they are often separated as two completely different initiatives within a marketing organisation. While people seem to be interested in the idea of combining them, they are usually already siloed as different projects with different teams, budgets and timelines by the time we have the discussion.

Let’s first talk about testing by itself. It’s a great way to figure out what version your visitors prefer. If you’re just throwing new designs up on your site today on a wish and a prayer, testing is definitely a must. But then think about who your visitors are. Are they all people who are coming for the first time? Are they all visiting during work hours? Do they all enter on the same page looking for the same thing?

If your answer is no to any of these questions, then you have to ask yourself if it makes sense to assume that they all prefer the same winner in any given test. The easiest way to figure this out is to segment your reporting. That doesn’t change anything about how the test is designed or implemented, but it can give you additional learnings in the reporting and analysis.

What if you found out that the reason a particular version won was because it resonated deeply with your new visitors coming organically to your site? And balancing out that great lift was actually a negative result with other smaller segments such as direct mail and branded queries coming from paid search? This type of data is incredibly valuable for any organisation trying to get deeper insights into what their visitors are looking for and responding to. What makes it invaluable is to then take action on those learnings by delivering that targeted, winning experience to your organic, first-time visitors and then simultaneously beginning a new test on the rest of the traffic.

eHarmony ran a test with Omniture to understand whether adding tabbed navigation to a landing page would be more effective in getting users to convert. The hypothesis was that providing more information about the service directly on the landing page would be a positive change. Here are the different versions.

What they found was that the test actually performed worse for overall traffic, and that people preferred the simplified page without the AJAX navigation. Once they dug into their segments though, they found a more complex and interesting story. They had set up geo-segments to track how people from different countries behaved, and it turned out that visitors from Canada significantly preferred the navigation. This learning would never have been found without segmentation, and worse, the tabbed navigation would have probably been thrown out based on the overall negative lift.

What is Targeting Without Testing?

What is targeting without testing though? Isn’t it the same as just putting your hypothesis out onto the site and hoping your hunch aligns with your visitors? For example, let’s say you own a retail clothing site, and you want to target visitors who look at accessories and then return to the home page. You may wonder which accessories to now reinforce upon their return, but what if you should instead be asking yourself whether accessories is the right category to show? What if you should be featuring women’s clothing instead because there is a high correlation between the two or the potential for higher margin profit? Or what if showing 2 featured products instead of 6 would make all the difference? Do you know which price range you should be staying between? All of these questions are great opportunities to combine testing and targeting into a single initiative focused on creating conversion and revenue on your site.

BabyCenter ran a test with Omniture’s Test&Target module to understand the impact of increasing relevancy on one of their most highly-trafficked keywords: “baby names.” Below is the control version, along with the alternatives tested against it. What’s interesting to note here is that all of the alternatives have some element of targeting in them. Recipe B offers some relevant text, Recipe C adds some category links, Recipe D provides top 5 names of 2005 for both girls and boys, and Recipe E builds on Recipe B with a few more bullets of information. Can you guess which one won?

If you guessed Recipe D, then you are in sync with most of the marketers who see this case study. However, the answer is Recipe B, the alternative that provides the simplest reinforcement of just a headline and a couple lines of copy. Here’s the test data:

You can see that Recipe B provided 65.32% lift against control with very high statistical confidence. In fact, nearly every alternative performed better than the control except for Recipe D, the version that people most often pick as the winner. It makes sense that we naturally gravitate to Recipe D because it provides us with the most relevant information given that we are searching on the term, “baby names.” However, since we are measuring the success of the landing page based on those who sign up for the magazine, we have to balance driving both relevancy and engagement. Without testing this targeting effort though, we would have no idea whether we picked the right mix.

If you are in the process of testing today, I’d strongly advocate setting up segments so you can begin to see which ones behave differently. Targeting is a natural extension of testing, but don’t forget to go back and test your hunches as well, whether they apply to one segment or all of your traffic. Taking action on your data is the best way to do more with less, a strategy we will all have to follow more closely through the immediate future.

Content Referened from the Omniture Blog. http://blogs.omniture.com/