The Difficult Task of Offline Media Measurement

By Giles Martin, EVP, Strategy & Insights

Why is measuring offline media difficult? There are many reasons. Some of the more obvious ones that distinguish offline from digital media are the lack of measurable clicks, a more limited data set, and no immediately obvious or trackable response path. These can certainly make measurement a challenge. However, many of the companies that come to us for help with offline media will often share such concerns without acknowledging the related concerns that also exist for digital. While digital is flooded with data, people often don’t take the time to consider what they should measure and why, making digital attribution arguably as challenging as offline.

Last week, we wrote about the methodology behind the ”Hi Dee How” survey for offline attribution. Many new clients, particularly those that consider themselves to be more sophisticated, have substantive concerns about using a survey to attribute offline contribution. To them, it seems inaccurate, crass, and evocative of 1999, and they insist that there must be a better way.

For many organizations, Market Mix Models have emerged as a best practice to measure offline, as they seem more robust than a simple survey. People seldom heed George Box’s wise words, however: “All models are wrong, but some models are useful.” MMM’s are often not as valuable as they could be for young, D2C-type companies because they don’t know how to correctly use and interrogate them. They’re not asking if their model is useful, and if so, how and where. Not to mention, these models are expensive, slow, and time-consuming to build and run.

Multitouch (MTA) solutions often fail to incorporate offline attribution in any way that’s meaningfully congruent with their capture of every digital touch on the customer journey. Some vendors shoehorn in offline with a different methodology and claim that they’ve made a coherent ‘holistic’ picture of every touchpoint — but it’s simply not true. The methodologies are totally different.

Even if we look at online, the results are disappointing. Solutions that are restricted to the digital world also struggle to add value. At least 80% of the clients we meet use last-click attribution as their main methodology, even though click-based optimization has been poo-pooed for arguably a decade or more (note: that study is authored by one of the world’s foremost data scientists, Claudia Perlich.)

In essence, people don’t have good solutions, even online. It’s rare to find a client who’s had a really positive experience with any of these expensive attribution vendors, or who has had a real success story with media optimization using these findings. Tellingly, a 2017 survey by The Data & Marketing Association found that only 7% of marketers are satisfied with their attribution efforts.

So to those clients who poo-poo the not-so-sexy survey, we ask: why are you skeptical? The survey does not claim or strive to be the magic answer. It merely claims to give you an approximation of what a channel is doing for your business, in a reliable, easily implemented, and understandable way. Simplicity is sometimes regarded negatively, but we’re with E.F. Schumacher on this: “Any intelligent fool can make things bigger, more complex, and more violent. It takes a touch of genius — and a lot of courage to move in the opposite direction.”

The survey does not falsely claim precision, as so many of these other approaches do. It will, however, show you the exposure in that channel, and allow you to measure a growing contribution to your business. Here is an example that demonstrates weekly spend for a new client and growth in survey responses.



The data reflects the channel’s growing contribution to customer acquisition, and disputes that the channel’s overall value to the business cannot be estimated from a seemingly simple survey.

The survey is not a perfect form of attribution, nor does it claim to be very precise. That said, there is no perfect form of attribution, and few, if any, are truly precise. Millions have been invested in attribution solutions, but there are no obvious winners and few ‘right answers’ flying around. The bottom line is that clients who use a survey have a vastly larger chance of scaling than those who don’t, and it’s very hard to succeed without one.

While new digital-style attribution in Podcast offers a welcome alternative, the industry is still learning about their strengths and weaknesses. We have been testing them with success, but caution clients to not dismiss the value of the old-fashioned survey.

2 thoughts on “The Difficult Task of Offline Media Measurement

  1. Fascinating graph. Do you find clients are able to fully justify their podcast spend based on HDYHAU surveys? And, as a result, to increase it?

  2. Many of them do, yes. It’s not perfect, but it works well enough to grow a channel – and a business. Many of our clients are very successful with this and that’s why it’s commonly practiced. The ones that don’t use a survey frequently fail to increase their spend.

    That help?

Leave a Reply

Your email address will not be published. Required fields are marked *