Try for Free
Advertisement
  1. SEJ
  2.  » 
  3. Paid Search

Microsoft Ads Introduces Cookie-Based Experiment Feature

Microsoft Ads Introduces Cookie-Based Experiment Feature

Microsoft Ads is adding an additional feature to the Experiment functionality launched last July.: cookie-based audience specifications.

What is the Cookie-Based Option?

Unlike experiments that are search-based only, cookie-based audience splits means that the system will “remember” which version they were shown.

Moving forward, they will only see ads from that campaign, vs. being treated as a new user every time they search. The audience “buckets” will occur at the cookie level vs. at the session level.

Microsoft notes this is preferable over Search-based Experiments for testing items such as creative testing, since the user is only responding to one source or another and it will remain that way for their cookie ID.

How Did It Work Before?

The first iteration of Experiments was search-based.

It relies on randomization for showing the ads between the original campaign and the Experiment, and it happens every time a user searches.

There was no “memory” of the user having performed the action previously, unlike cookie-based audience splitting, which buckets audiences by cookie identity vs. what they searched in that single session.

This method means users could search multiple times and be shown ads from both the control and experiment versions.

Microsoft recommends this for testing things such as bidding methods, since the comparison would be at the bid level and is ultimately a study in cost.

How to Enable Cookie-Based Splitting

Users can access Experiments via the Experiments tab.

Microsoft Ads Introduces Cookie-Based Experiment Feature

The “Create an experiment” module will appear. Here, users select the Campaign they want to run the test on, name it, specify the dates, and what percentage of the audience should see the Experiment.

Advertisement
Continue Reading Below

The new feature for cookie-based audience splits is now under the “Advanced Options” section:

Microsoft Ads Introduces Cookie-Based Experiment Feature

The Difference in Rate of Experimentation

There is a trade-off for having the potentially higher accuracy of cookie-level data:

The amount of time it will take for statistically significant results.

Advertisement
Continue Reading Below

Here is a simplistic example:

If you have 10,000 users eligible for your Experiment:

  • They search an average of 3 times
  • That’s 30,000 times your Experiment would run against your control.
  • It would give each instance roughly 15,000 searches of data if you run your Experiment split at 50%.

However, if you have 10,000 users using cookie-based:

  • Their multiple searches don’t count.
  • You would have a 50% split to each instance of only 5,000 users per each group.
  • The smaller pools mean it will probably take longer for the results to be statistically significant.

The cookie-based feature is available in all accounts now. You can find Microsoft’s full announcement here, along with an overview of using Experiments here.

Advertisement
Continue Reading Below

More Resources


Image credits: Susan Wenograd

ADVERTISEMENT

Subscribe to SEJ

Get our daily newsletter from SEJ's Founder Loren Baker about the latest news in the industry!

Ebook

Susan Wenograd

Consultant

Susan is the Paid Media Reporter for Search Engine Journal. A marketing veteran of almost 20 years, she has managed ... [Read full bio]

ADVERTISEMENT
Advertisement
Read the Next Article
Read the Next