Why Quantcast Doesn't Use a Panel

At MPG’s Collaborative Alliance meeting on December 2^nd, Current TV’s head of research, Theresa Falcon, bravely stood up and pleaded with industry executives to help find new ways to improve audience measurement for small or emerging media outlets. In her opinion, existing panel-based methodologies such as Nielsen’s national TV panel of 14K households are ill equipped to handle Current TV’s evolving measurement needs.

While the various panel-based syndicated measurement sources for TV, radio, and online are considered gospel by many in the industry, there is a growing number of media executives, like Ms. Falcon, who view their respective audience results with increasing skepticism. They perceive an increasing dissonance between media forms offered and the ability to consistently understand their audience size and demographic characteristics. As a consequence, even subtle changes in panel-based data may dramatically impact performance and, ultimately, the bottom line.

It’s not just emerging media outlets like Current TV looking for new solutions beyond existing panel-based approaches. For example, earlier this year NBC faced the daunting challenge of showcasing their multi-platform content for the Beijing Olympics to marketers without consistent third-party validation of performance.

Alan Wurtzel, president of research at NBC Universal, was quoted in the Wall Street Journal on their measurement efforts, “Management said to me we have to figure out a way to go beyond Nielsen to measure this stuff."

Subsequently, NBC utilized a number of alternate measurement services, including direct measurement firm Quantcast, to better understand the audience characteristics of their Olympics content.

Challenge: In an increasingly diverse and addressable digital media marketplace, can panel-based methodologies offer enough actionable insight to enable publishers and marketers to properly evaluate target audiences for advertising investments?

Quantcast feels the answer lies well outside the realm of existing panel-based methodologies. Digital content and related marketing are delivered to consumers dynamically and in real time, while panel-based methodologies deliver supportive audience data weeks or even months later. Such latency in reporting may have been acceptable in the past for radio, television, or an early-simplified view of the digital space.

Not today. Panel-based measurement solutions lack the scale, flexibility and immediacy to deliver actionable insights for an increasingly complex and fragmented digital media marketplace.  

The case for ‘direct measurement’

In the book Mass Media Research, writers Roger D. Wimmer and Joseph R. Dominick state quite succinctly (in their apparent defense of panels), “it would be impossible for any ratings company to conduct a census of media use.”

The 8^th edition of this text was released in 2005. Perhaps if they witnessed first hand the challenges of measuring an evolving digital marketplace via panels they would reconsider their position (or perhaps even add a chapter in their next edition focusing on media fragmentation)?

As the media landscape continues its transition from analog to digital, enabling ever more discrete forms of content creation and distribution, Quantcast feels, despite Professor Wimmer & Dominick’s assertions, that a census or direct measurement approach is the most viable and consistent method available to understand the increasingly fragmented audience activity taking place on these emerging digital media platforms and services.

In fact, today Quantcast processes upwards of 4 billion new media consumption events every day, and for the media firms who participate this is a census. Rapid changes in technological capabilities mean that a full census of media consumption is possible today – sure its technically challenging, but it is possible. And, when you can comprehensively measure media consumption via a census, it’s a game changer.

Put simply, Quantcast doesn’t have a panel.  

In a major departure from existing panel-based measurement methodologies, Quantcast’s inference-based solution couples machine learning with massive quantities of directly measured data from over 85K partner publishers and marketers to deliver detailed audience data in real-time for all forms of digital media including websites, video, audio, widgets, downloads, and advertising campaigns. 

The direct measurement approach allows Quantcast to deliver a census view of all respondent level activity for reach and inventory measures, as well as fine-grained profiles on demographic, geographic, business and lifestyle attributes.

Note: For a detailed overview of Quantcast’s inference-based, direct measurement methodology, please visit www.quantcast.com/white-papers/quantcast-methodology.pdf 

Advantages to direct measurement

Quantcast’s direct measurement approach addresses, quite elegantly, some of the most nagging measurement challenges facing the digital media marketplace including latency of reporting, granularity of data, geography and cost:

    Latency

• Panel: Limited overnight data available ‘by special request’ only. Syndicated reports typically take anywhere from one week (for the basics) to a month or more, while custom reports may take even longer

• Direct: Audience reports for quantified publishers are delivered in near real time, typically within hours of the observation window

     Granularity

• Panel: Even with a sample base of nearly 2 million, panel-based methodologies consistently report monthly data on only 30K-40K domains
• Direct: Daily audience profile reports on over 10 million media assets with as few as 500 unique exposures

    Geography

• Panel: Inconsistent methodologies comparing US to globe, while granularity limits depth of local reporting
• Direct: Both US and global data (by country) for all quantified publishers in the U.S. Local demographic reports available for all 210 US DMAs

    Home/Work/School

• Panel: Unclear Fortune 1000 representation in workplace. Student representation also unclear
• Direct: Reports on industry type as well as small, medium and large business owners. Similar reports available for both schools and universities

In addition to the direct measurement advantages listed above, Quantcast’s methodology is also technology and content agnostic, meaning any new addressable media platform or content type including video, audio, widgets, ad campaigns, (all with as few as 500 unique exposures) can be configured to support direct measurement. 

Panel-based approaches, however, typically have a difficult time adjusting to technology changes from publishers, marketers or even consumers. For example, a re-emergent Apple now garners a sizable portion of the new computer market. According to Apple, nearly 1 in 5 new computers purchased (either desktop or laptop) is Mac OS-based. This figure rises to nearly one-third among students.

Despite Mac’s newly found prominence in the consumer market, existing panel-based measurement services do not include them in their sample, citing software compatibility issues.

Panel-based services must also resort to what are called site/panel intersects to report audience data for video, widgets, ad creative and other addressable media types. In such scenarios, panel-based data is matched against pixels or tagged media from either the publisher or marketer.

Site-panel intersect models suffer from all of the same sample size limitations and biases of traditional panel-based approaches (as listed above). Each piece of tagged content must reach a significant threshold of panel exposures among the panel before projectable demographic data can be generated.

A last point worth making is that while panel-based services have steep fees associated with them, Quantcast provides its direct measurement services absolutely free to all publishers and marketers. This is not a feature of direct-measurement, but rather an aspect of Quantcast’s business model which is based on enabling addressable advertising. However, it does mean that anyone can take advantage of this pioneering direct measurement approach at no cost.


Direct measurement promotes a common language

Quantcast is gratified that esteemed media companies such as MTV Networks, Disney, Hulu, NBC, Time Inc, Bloomberg, Gawker, and over 85K other publishers and marketers have embraced inference-based direct measurement as a viable alternative to existing panel-based audience measurement solutions.

As digital media publishers bring new advertising opportunities to marketers, Quantcast seeks to enable a ‘common language’ between all parties using this inference-based direct measurement approach, finally putting to rest some of the most nagging audience measurement issues currently facing the digital media marketplace,

Publisher and marketer exchanges once clouded by a collective disagreement over which numbers matter most, can get back to the more appropriate topics of brand goals and performance.

No panel required.

Share on Facebook

Comments

[An Interested Party]

Interesting change in postioning given this:

http://www.clickz.com/3630165
"The result is a unique solution that marries the benefits of direct measurement (cookies) with those of panel-based reference points (people/audience). Our model is based on inference. We don't just intersect our pixel data with our panel, do a simple extrapolation, and call it a day (e.g., for a property, if we see 100 directly measured cookies and see 10 of them across our panel, we don't just ascribe the characteristics of the people we saw in the panel to the full, directly measured audience). We are literally collecting billions of new directly measured data points every day and building a model that continually improves."

--and this from the cookies-to-people white paper from June--

http://www.quantcast.com/pr/pr-quantcast-cookie-to-people-translation.jsp
* "first audience measurement solution to provide comprehensive visibility into cookie-based traffic counts and people-based audience"
* "Quantcast's revolutionary hybrid methodology leverages directly-measured cookie counts combined with panel and third party data sets within a sophisticated statistical framework."
* "Bridging these measurement methodologies marks an important step ..."

Could it be that scrutiny of the so-called Quantcast panel forced your hand?

[Quantcast]

As we’ve grown and spoken to more media companies, marketers and agencies we’ve come to recognize that when people think of ‘panels’ they often think of a single pool of individuals who are recruited one by one. We don’t do that. The vast majority of our data comes from direct measurement through our Quantified Publisher program, which now generates over 4 billion new records every day. Our pioneering mathematical inference approach http://www.quantcast.com/white-papers/quantcast-methodology.pdf combines this directly measured data with a variety of sample based reference points. A panel is a sample, and in some ways a number of our sample sources are similar to panels, but in order to build lots of these relationships and avoid the sample size and bias limitations of any single sampling approach (including panels) we’ve chose to work with multiple partners rather than recruit sample participants one by one.

For our approach, which is focused on helping the industry better understand audiences and data to enable the delivery of real-time addressable content and advertising solutions http://www.quantcast.com/corpcom/videos , this is really the only path.

Our modeling approaches are constantly under scrutiny, as they should be, and no one focuses on them more than us. The attention that we most welcome is from the myriad of companies who participate in our open program. Ultimately, they have to understand our approaches and have trust in our models and their output so that they can sell on the basis of the information. And many companies we work with have undertaken extensive reviews of our approach and this process continues as our market presence expands. You may have recently seen that leading companies such as MTV Networks http://www.adweek.com/aw/content_display/news/digital/e3if0ded1d484f712c4ba11415504632ab0 , Disney-ABC and Hulu http://www.tvweek.com/news/2008/12/quantcast_tracks_online_audien.php have adopted our Quantified Publisher program, joining a growing roster of top tier firms that includes CBS, FOX, NBC, Business Week, Bloomberg, Discovery, WordPress, TypePad, Time Inc., Newsweek, The Huffington Post and over 85,000 others.

We’ve come a long way in two years and we’re delighted to work with so many leading firms but we’ll still make mistakes from time to time. That’s part of challenging a status quo in audience analysis that does a disservice to our industry. The most important thing is that we continue to learn and add value to our community and we’ll continue to work hard at that.

[Phil Ellis]

So, I am still a little confused. Is the number of visitors shown on my Quantified Site the number of REAL, UNIQUE visitors, or is the number adjusted by statistical projections?

www.divesports.com

[Quantcast]

Phil, as divesports.com is quantified the numbers shown are real, based on direct measurement of your audience. Thanks for being part of the program.