1.13.08
By the numbers
by Matt Worley

About a year ago I was talking to my old boss at a market research firm. I told him I was thinking of starting a company called "All Opinion, No Data." Basically, I'd charge 10K a commercial, watch it and write a report about how I thought it might do in the marketplace. The service would be for large companies (not, say, local car dealerships), of course, because if you can afford 10K for "research," you gotta be spending many multiples of that to produce and run the ads.

This price, by the way, is significantly less than what my old company charged per ad.

The critical problem with my genius idea is twofold. One: I don't have any clout in the advertising industry, so who would care what I thought about an ad? And, two: the whole reason for testing advertising is to cover marketing executives' asses (collectively). Even if an ad that does gangbusters in research bombs in actual practice, they can always go back and say the research was reasonably sure it would work. And their ass is kind of covered. They have all these charts and graphs and numbers to show their work.

By the way, most ads are good enough to pass and get run. And horrible ads are rather easy to spot.

But really, all I have going for my 10K idea is price and speed. I could do this work in a day. A typical study takes a couple of weeks (from start of project to final report), if not more.

The reason I know this idea of not actually testing ads (just giving my opinion) would work almost as well (but much cheaper!) as the regular way is this: there were and are many studies where the result is anticipated, and the data is just layered in to "prove" what was already the opinion of the market researcher.

Testing commercials is not universal. Many companies do not test their ads. And, most likely, they do okay without it. Because of this basic principle of market research: it's about the opinion of people. You can ask a hundred people from various places around the country and then mush all that data together to get an answer, or you could just show the ad to a small room of people and take a tally from there. Smaller set of people, larger margin of error. And, yes, they do both of these things in the advertising market research industry.

So when the day before polls in New Hampshire came out and Obama surged to a huge lead, the Democratic Party's race for Prez was declared over. Then, the next day, Obama came in a few percentage points down from Hillary in the actual vote. And the polling shit hit the fan.

More than two hundred thousand people voted in the NH Democratic Primary. Most polls asked less than a thousand people their views. Of course, no one was questioning the huge surge Obama showed in those last minute polls. They only questioned it after the real poll came out.

No matter how good your sample is, or how well your questions are asked, or how well the results are analyzed, there is a chance you are completely wrong.

And, actually, the polls were mostly just wrong about the huge surge Obama had. When you look at most of the numbers, it was as if Obama was picking up all of his extra votes from everyone other than Hillary. There is, of course, a margin for error built into every poll (except for the actual vote, because it's the actual vote). Wiggle room. They put those numbers in really small type underneath the poll results. And when more people actually vote than anticipated, it makes the pre-election polls even less accurate. The number of voters in NH was a record for a primary.

Picking a Presidential race is harder than picking a good commercial. All you need for most ads is a nice joke or twist that makes the product the hero. And a lot of times you don't even need that.

And all of those expensive Superbowl ads? Most of them are not researched because there is no time. Which brings up a side question I've been having about this contest the NFL has been running about which ad they should run during the Superbowl: isn't the Superbowl itself a huge day long ad for the NFL?

So your guess about how all this Presidential hooha comes out is as good as mine. No one really knows (although the number of people running has gotten smaller, so you've got a better chance of getting it right now). I'm inclined to say I like Obama better than Hillary. That's just my opinion. Which is all a poll is: a few hundred opinions tallied up and extrapolated to the rest of the population.

And if you've got an ad, ten grand and a couple of days, I'll give you an opinion that's at least as likely to be correct as the polls in New Hampshire.


Matt Worley just wants to say that people who write and produce advertising are geniuses. And the likelihood of that statement being correct might be around fifty-fifty.


e-mail Matt Worley
return to the Shut up, I'm talking page
return to the LIES home page
return to the A&A home page