"Client, Measure Thyself," or "BYO Metrics"

In the Hutong

Upgrading the Windows machine

1126 hrs.

The pitch seems to be going well. There is good chemistry between the prospective client and the agency. The ideas are great, the concept clear, the tactics doable, the team qualified.

And then comes that moment where everyone sits back, and someone on the client side folds their arms, looks the agency leader in the eye, and asks "so how are you going to measure the effectiveness of your campaign?"

Usually the agency will have some sort of answer, and occasionally it will be enough to satisfy even the  procurement person sitting in on the pitch. But as someone who has sat through countless pitches on both sides of the table and have made more than my share, I have yet to hear an agency give the correct answer:

"I am sorry, but isn't that your problem?"

You Want Us To Do What?

The idea of holding an agency accountable is a great one. Far too much money has been hurled at marketing (not just television advertising) based on little more than the assumption that the only bad marketing is none at all, with the primary beneficiaries being media and agencies. But accountability goes bad when the agency is handed the task of measurement.

The first problem with asking an agency to be responsible for monitoring the efficacy of their own campaign should be self-evident: it is like asking the fox to guard the hen-house. The agency has an implicit conflict of interest.

When asked to provide measurement, most agencies will endeavor to use the metrics that best validate their own approach. What is more, by giving the agency (under pressure from both client and upper management to perform and retain the client) the responsibility to handle measurement, you give them  both the motive and opportunity to "game" the results. This is not a condemnation of agencies: it is merely an acknowledgment of human nature. 

One Yardstick to Rule Them All?

The second problem is that measurement, particularly in this day and age, is not (or should not) be the same for every company and in every situation. The use of standardized metrics like gross impressions, page views, ratings, ad-value equivalencies and click-throughs is helpful when comparing one marketing program to another. Yet they all suffer the same shortcoming: they are weak measures of how close a company is getting to reaching its specific business goals.

In any market, but especially in a place like China, goals are constantly changing as a company evolves and targets shift, and I would argue that different markets (geographic or product based) need different metrics. The way you would measure the marketing campaign for a "hero" product in a developed country is different than the way you would measure a campaign for a very-low-margin product in a developing one.

For these reasons, agencies should not be in the business of designing marketing metrics for companies. That responsibility belongs with each company, deciding what its business goals are, understanding how a marketing campaign can get the company to that goal, and then devising the way the agency (and the marketing team as a whole) will be measured in its performance. 

Chief Measurement Officer

This means that the first and most important challenge of a new Chief Marketing Officer (CMO) - is not to call an agency pitch. Rather, it is to create a measurement system that all of a company's leaders agree is tightly aligned with the firm's business goals. The second challenge, in that case, is update and evolve that system as goals change, as tools change, and as more accurate means of measurement become available.

When, and only when, that system is in place is it time to seek agency help. Yet even when that time comes, the only measurement responsibility an agency should be given is to perform against the metrics set by the client on the basis of the client's own business goals. Those metrics should be made clear in the pitch, and should be the focus of regular update meetings. Ideally, the measurement system should be real-time, so both agency and client can respond to critical data quickly. 

Revolution is No Cocktail Party

If you think all of this is designed to go easy on the agency, you miss the point. This approach would require an agency to perform against different metrics for each client, and for that reason I suspect many agencies would be dead-set against it. (Many in-house marketers will be as well, because it places a greater burden on them, too.)

But before we dismiss this approach out-of-hand, consider the future. The marketing craft is going through a time of profound crisis. A deep vein of skepticism, disbelief, and distrust of traditional marketing tools is coming to the surface, exposed by the erosion caused by the financial crisis, a general movement for greater fiscal accountability, and what can best be characterized as Advertising Budget Growth Fatigue.

If we want to have the credibility that entitles us to influence at the highest levels of our companies, it is incumbent on us to do everything possible to demonstrate the value of what we practice, and to guide our practices toward strategies and tactics that deliver greater value. That means a creating and building a toolkit of metrics unique to each company that ties every marketing dollar to sales, to repeat business, and to individual customer relationships.

Comments

[Will]

Perhaps it reflects my professional pedigree, but I'm a big fan of knowing what target audiences actually think of my clients, something that surprisingly few people seem to know or seem particularly interested in testing. You'd *think* companies would want to know. As a PR man, my job is to shift perceptions (ideally in a direction that supports the business objective). If I don't know current perceptions, and I can't test perceptions at the end of a campaign, well then I'm really just playing a very expensive version of blind-man's-bluff. I have a general idea of where the target is, and sometimes I get the pin more or less where I want it, but the rest of the time it's going into the couch, the wall, the dog, your eye, etc.

A common response to proposing this soft of testing is something along the lines of, "oh, well, just shift the tone in the media", that being easy and cheap to measure. And it would be great if the media was the main thing shifting perceptions or I could draw a clear line between what's in the media and the business results. But these days the media is just one element among many and often the line is not so clear cut.

So when I ask myself, at what point to all the channels of influence I work with converge in such a way that it forms the perfect target for measurement, I always come up with the same answer: It's the audience, stupid.

To your point, I don't really care whether we do it or the client does it. I just hope *someone* does it.

[David]

Will, I agree - we still see too many clients and agencies who behave as though measuring is too hard. The agency avoids the question in most pitch situations unless the client brings it up, and when the client DOES bring it up, the solution is either easy-but-dumb, or they lay the burden on the agency to come up with the metrics, then to handle the measurement.

Which goes back to my point. Metrics and measurement tools should be determined by and managed from the C-suite.

Oh, and one other thing: education. Marketing and communications programs need to start building quantitative analytical skills alongside the "softer" bits of the marcoms toolkit. We don't all need to be quants, but we all could stand to be a lot more comfortable with the tools.