Engagement and the ARF Model

On March 21, at its Annual Conference, the Advertising Research Foundation (ARF) will announce its new official definition of Engagement and how it is to be validated and measured. In the ARF program for that conference, the first line about the opening session says: “Engagement is being defined as the new currency for advertising ROI.”

The ARF et al have to step very carefully here – these are tricky issues. As Erwin Ephron points out, advertising Engagement does not just depend on the media, it depends even more on the creative itself – so basing transaction currency on a medium’s average advertising Engagement numbers is to some extent holding the media responsible for something they do not control. This whole vortex has roots going back at least to the 1950s and probably earlier.

In the 1950s, spurred on by the arrival of television, the ARF struggled to create a Model which would permit advertisers and agencies to make fair intermedia comparisons, knowing how to relate print circulation and audience to broadcast audience on an apples to apples basis. After years of remarkably intellectual debate by some of the brightest minds in the industry (then or now), in 1961 the ARF published its first Model defining the levels on which media could be compared. That Model had 6 levels and ignored the existence of multistage direct marketing where a person can interact with an ad yet not commit to purchase. This omission became increasingly relevant as direct response ballooned and as New Electronic Media with interactivity appeared on the scene. In the late 90s the ARF asked me to form a committee to update the Model. The choice of me as the first chair of this new committee reflected the fact that I had been using an unofficial updated ARF Model for our clients at Next Century Media, and that (13 level) Model had been published in a paper in the ARF Journal of Advertising Research.

Erwin Ephron agreed to be co-chair of the committee which also consisted of Bill Moran, Jim Spaeth, Denman Maroney, and graphic artist genius Phil Brandt whose responsibility was to create a new visual representation of the Model. In 2002 the new (8 level) Model was published.

The subtext of the Model from the 1950s through the present moment is the reasonable desire of advertisers and agencies to move the basis of buying media up from the low level called “Vehicle Exposure” where it has been for as long as anyone can remember. This is the level measured by Nielsen, Arbitron, MRI, Simmons, Scarborough, et al. It represents potential exposure to the ad in that it purports to measure the audience to the vehicle in which the ad is embedded – e.g. the program or publication.

Ideally, the buyer side would like the transaction with the media seller to be based on the highest level of the Model: Sales produced by the advertising. A few direct marketers and Internet sellers have already accepted this deal but 99% have resisted, saying that since they don’t control the creative, they could do their job perfectly and not get paid because of the agency’s creative failure. “Not fair!” they say, reasonably enough.

The ARF through its Engagement Proclamation seeks to cut this baby somewhere like in half…by finding a middle ground acceptable to all parties…and yet finding a metric that is predictive of Return On Investment (ROI), which is part of the Sales level of the Model. This is a ‘cake and eat it too’ strategy. Can it work?

Here are the hurdles which ARF must overcome:

1. The media must find the metric fair to them.
2. The metric must be proven to validly predict ROI.
3. Measuring the metric must be affordable and practical.
4. No one research company must be unfairly favored.

So, when we say ‘if successful’, it’s because of those hurdles.

Some practitioners have suggested that Engagement ought to be based on some characteristics of the medium which do not involve the creative. For example, the Durationists believe that in television one could infer Engagement based on the length of time the average audience member stays tuned to a program. Loyalists believe that the number of telecasts out of four seen by an average audience member would be a useful indicator of Engagement. And there are some Duratio-Loyalists who argue that Engagement ought to be based on both of those measurements.

This would make sense if we knew for sure that loyalty or tuning duration to a program translates to more attention to an ad placed in that program. It’s also possible that the longer one stays tuned to a program the more one needs to run to the washroom when the commercial comes on. As Erwin points out, we really don’t know the degree to which media Engagement leads to advertising Engagement. Some direct marketers insist that, leaving cost aside, the response rate to direct response TV is greatest in the least involving programming, and 2-6AM is often cited as an example of such programming. In my own experience I’ve never had an opportunity to verify the latter assertion.

Some of the other places in the Model where one might plant the flag called “Engagement” include:

• Advertising Exposure (commercial audience, ad page exposure) – this is level 3 of the updated Model, right above Vehicle Exposure
• Commercial Retention (the percent of those starting a commercial who stay tuned to the end of it without muting or turning the sound down)
• Advertising Attentiveness (based on self-report or psychophysiological measurement) – level 4 of the updated Model
• Advertising Communication (e.g. commercial recall) – level 5
• Advertising Persuasion – level 6
• Advertising Interaction – level 7
• Sales produced by the advertising – level 8
  
  

This picture of the model shows the eight levels at which media performance can be measured as eight segments of a helix seen from above. The segments are numbered and colored in increasing order of advertising relevance. In this example the relative sizes of the slices suggest the relative numbers of people involved at each level. The number of people who buy the advertised product (Sales Response, segment 8, colored red) is smaller than the number persuaded by the advertising (segment 6), which is smaller than the number attentive to the advertising (segment 4) and so on. The number of people exposed to the vehicle (segment 2) is usually larger than the number of vehicles in circulation (segment 1), since more than one person is likely to read a copy of an issue of a magazine or watch a TV set tuned to a TV program. The picture is not meant to suggest that the levels are necessarily consecutive; beyond advertising exposure, they need not be. Rather, it simply is a way of laying them out in an easily observable format for purposes of comparison.

In all of these cases, one would have to find a practical, affordable way to obtain valid averages across all of the advertising recently placed in each medium. A new company in which I’m involved, Touchpoints ROI Audit (TRA), and Apollo, the VNU/Arbitron coventure midwifed by P&G, could calculate average sales effect medium by medium. This would go all the way to the top of the Model. The media who were shown to have low averages however would argue that it wasn’t their fault that the ads placed in their medium had been so lousy, and that if someone gave them some good ads, they would move to the top of the ranking in sales effect.

This same criticism will be leveled by the media at everything in the bulletpoint list above except Advertising Exposure. From Commercial Retention through Sales, the creative is more important than the medium in determining these numbers.

So what is ARF likely to announce on March 21?

Guess #1: ARF will stop short of limiting the definition of Engagement to one specific metric derived from one specific method, and will instead lay down the groundrule that it must be a metric validated to be predictive of ROI, and which can be applied to audience data as a weight in the CPM calculation so as to be integrated into the buying process. The prestigious Richard D. Lysaker Award might be offered to whichever company comes up with the best solution, in the hopes that many research companies rise to the challenge.

Guess #2: ARF will actually elect one metric based on one method. I find this hard to believe given the long history of staying neutral on issues which could tilt favoritistically toward one or another research company. But I’ve been surprised before.

Guess #3: ARF might select ROI measurement itself as the goal, and this would point toward TRA and Apollo, while encouraging others to create their own contenders in this category. This is highly unlikely because from the get-go, as Bob Barocci, ARF CEO, explained it to me, the purpose of MI4 was to come up with a surrogate for ROI.

Guess #4: There has lately been close collaboration between ARF and Harvard professor Gerald Zaltman, whose book How Customers Think threw marketing research into a cocked hat by saying that 95% of the time consumer buying behavior is unconscious whereas nearly 100% of the time, marketing researchers measure the conscious mind only. What if ARF selects a psychophysiological metric for Engagement? Interesting to imagine, but hard to believe. By doing this, ARF would be taking about as radical a stand as Galileo did (and might be just as right as Galileo, but would likely take as much flak as he did too).

Guess #5: ARF calls for two measures, one emotional and the other behavioral. The emotional measure might be psychophysiological, or not, or perhaps the door is left open either way on that. The behavioral measure would involve purchase measurement and real ROI.

Guess #6: This is the only guess I would put money on. This guess is that all of the above guesses are wrong and ARF does something I have not anticipated. It will be very interesting to find out. I hope someone emails me as I will be on a cruise ship in the Pacific with my wife and Mom (already booked before ARF set the date). And I hope the ship has connectivity!

All the best,

Bill