SM Global Report: KD Paine Analyzes Measurement

[KD Paine and unidentified Old Friend at SNCR Conference.]

I had only met  KD Paine once previously when we were slated to debate at a software publishing conference in the middle 1980s. Our topic was whether or not public relations could be measured. At the time, I was a PR operative and she had started a PR measurement firm. I expected it would be a slam-dunk to win the debate, but at best, I fought her to a draw.

I learned then, and I understand now–that KK Paine is a force to be reckoned with when it comes to measurement issues in marketing programs. She is tough and wise, candid and complex. And while the issues of how and what to measure have always been important in business, they have become the key points of debate and contemplation for decision makers considering the strategic and tactical aspects of social media.

I have been writing often about a new term called “social analytics,” and explaining why the term is more complex that most people realize and the need to understand the implications are of great importance.

This is a challenge for most of us. For KD Paine, this is her time.  Let’s pick up my conversation with her.

Just how do you define the term “social analytics?”

It’s whole lot of BS. Let me explain.

It’s not that the term isn’t real, or that applying solid business analytics isn’t important, but in general, most of the people saying, ‘social analytics’ these days  don’t seem to have a clue about measurement or analysis, and probably don’t know much about social media either.

Social analytics is too often used synonymously with social media monitoring or measurement. But, in fact, it is a totally different concept.

So here’s my take on definitions:

Social media monitoring is what companies like SM2 and Radian6 do. They look at vast amounts of data and send you a stream of it that indicates how often your brand or your products are mentioned in that stream.  They essentially count stuff. Most enterprises today are still at the monitoring phase in social media.

Social media measurement is what I do. I develop measurement systems that reflect the specific goals and objectives of a social media program and determine whether or not the efforts are having the desired effect –be it frequency of mention, development of brand awareness, increase in engagement or generating traffic.  Sodexo, SAS, Dell, Southwest Airlines, Cisco, Adobe, Vico Software are all measuring their social media effort and this is a phase beyond just monitoring.

Social analytics is where measurement and business intelligence converge.

Essentially, social analytics is a combination of two important concepts – ‘social’ – i.e. things having to do with our society, our interactions and our relationships – the warm fuzzy stuff; and ‘analytics’ – which refers to the analysis of data and statistics. This stuff is neither warm, nor fuzzy but is very important to your business.

Social analytics looks at the myriad data points generated by social media measurement and runs a variety of statistical regressions [Measuring how changing one variable results in changes to other variables]. They determine what impact, if any, all that activity, awareness and engagement is having on the bottom line.  Comcast, Dell, Southwest and IBM are among the few companies that are actually this far along.

In the interest of transparency,  I should note that my business partner SAS Software calls itself  “the leader in business analytics software” and they have a product called Social Media Analytics.

To them, analytics means taking huge volumes of data and running all kinds of statistical analysis on it to draw conclusions about what makes a comp profitable or unprofitable, what works or doesn’t work. Among the data they look at is what is being said about a person or company in social media.

You’ve been measuring PR and marketing since about the time the wheel first launched. What’s changed so much? What’s so different about today’s social analytics, from yesterday’s PR campaign?

Lots.

PR campaign success was measured in terms of “hits.” We used to say that stands for “How Idiots Track Success.”  The more hits, the better, some thought. About two decades ago, a few more sophisticated companies started measuring what I call “Opportunities to See,”  or OTS, impressions – based on the audited circulation figures for the publications.

Somehow, PR people assumed that the more impressions, the more awareness, even though there was no evidence to support that.

Social Media has changed all that.

• First, impressions no longer count. What’s the “circulation” of Global Neighbourhoods, for example? If I look your blog up on Compete it tells me you get 207 unique visitors per month.  Hardly reflective of the influence you wield.  And do 40 million people really see every story on Yahoo News?

What matters now is what people do after they read your post. Do they click thru, register, ask for more information? It’s no longer how many people you reach, it’s what they do AFTER you’ve reach them.

• Measuring influence is a whole new ballgame. In old time PR measurement you had a list of influencers that you had to pay attention to and that list got updated maybe once a year. Now that list needs to be updated every month.

• PR could only promise engagement, in social media you can actually measure it.  In the olden days, we could only hope that people would be engaged enough with what we were feeding reporters that they might act, or get more engaged with the brand. Now we can measure a whole spectrum of engagement.

• PR’s job was to control the message—essentially a one-way process. Now, control is no longer part of the vocabulary. Nowadays, it’s PR’s job to build relationships and conduct two-way conversations. Jim Grunig [l.] defined “excellent PR” as a two-way synchronous conversation – TWO DECADES AGO. PR is just now being forced to come around to accept that philosophy. So now you need to measure the message in a way that was not so important a decade ago.

You and I have disagreed on more than a few occasions—sometimes quite publicly.  But we do agree on one issue: computers lack common sense. Got a good story for our studio audience that illustrates how obviously computers sometimes miss the obvious?

My favorite is recent.  My team was measuring press coverage for our client,  the Federal Reserve Bank of Cleveland, generally referred to as  the Cleveland Fed.

Suddenly, last July our computer-driven news feed showed a remarkably high level of coverage tagged as negative. This puzzled us, since people usually didn’t have strong emotions attached to the Cleveland Fed.

But when actual humans dug into the data, we quickly discovered that the press was filled with reports saying that Cleveland basketball fans were fed up with a certain basketball star who has since moved on to Miami.

Computers saw the words “Cleveland” and “Fed” in close proximity. They observed a negative feeling and based on solid data gave absolutely false results.

Another of my favorites was looking up the phrase cloud for a  manufacturer interested in “storage virtualization.” Most of the conversation the computers found turned out to be reports on people spending their weekend clearing out personalstorage bins.

All you have to do is to run your own name through any typical “sentiment analysis” engine and you’ll find dozens of examples. Here are the latest negatives for you, Shel:

And for me:

Just how important is social analytics to mainstream adoption of social media?

Huge.

Unless we can tie social media to the actual business results, the fearful, the skeptical and dinosaurs will continue to claim that because it can’t be measured it should not be adopted.

Longterm, the dinosaurs will die, and those companies that do use social analytics to statistically prove the connection to the bottom line will be the ones that will soar in this environment.

I recently had an interesting interchange with the folks at Klout. On one hand, a lot of people argued that Klout results make no sense. On the other, these guys have 17 technologists working fulltime, checking 35 data points to get the results they get. My question: What do you think of Klout’s efforts to measure influence? How could it be done better?

First of all, influence exists only in the minds of the customers or the target audience. Klout can’t possibly know what is influential to me or to you, they can only hope to program computers to guess correctly.  You’ve done enough PR to know that in almost every industry there are those “gurus of your marketplace’ who influence the influencers who influence the media who influence the buying public.

How did you create that list? You read their stuff, you saw who responded, and you saw where they were quoted. Can computers be taught to do that? Yes, are they there yet? NO! Which is why I like Traackr’s system so much – -they actually use humans to check the data, read the writings of those influencers, measure the comments etc.

Do you see a time when there will be universal definition and measurement standards for soft social media terms like “engagement,” “Influence” “sentiment,” etc.

Ketchum, Maggie Fox, Weber, You, Me, Zoetica etc. having a Kumbaya moment and THEN having enough money to actually persuade everyone that we’re right?  I think mid-East peace is more likely.

Companies like Dell are catching about 15,000 mentions a day. If the KDPaine secret sauce is to insert a human element into social analytics. how can it possibly scale?

First, how many of those 15,000 are relevant – - how many are talking about the Farmer in the Dell, or my father Del Paine. How many talk about a Dell employee dying or getting married? How many are appearing on fake blogs that no one will ever see?  My guess is that they probably get closer to 5000 a day in valid mentions. Then it is just a question of HOW you insert that human element.

You can either randomly sample the computer output to make sure it is correct  — which is how my group does it.  Or, you get trained humans to read 5000 or so to a level of 90% accuracy. Then you use that coded data to teach the computer how to do it right. Then you test the computer against the human coding until the computer is 90% accurate. THAT’s how you make it both accurate and scalable (and that’s what we’re doing with SAS).

All this being said, I am repeatedly told that most c-level decision-makers still want an ROI measurement for social media. Can social analytics provide it? Should it?

Yes, absolutely, but the problem is that most c-level decision makers can’t define the “R” in the ROI that they expect, nor are they willing to do what is necessary to measure it.  To measure ROI you need to have seamless data sharing and integration between the business intelligence people, the social media people, customer satisfaction folks, and the marketing team. How often does that happen?

Not nearly often enough.  Also, they need to understand that the “R” doesn’t always mean sales, or market share, it means bottom line benefit. So the ROI is in fact more likely to be measured in terms of money saved, or greater efficiency or a shortened sales cycle.

Please give me your vision for the state of social analytics, 3-5 years down the pike? What will that do for/to social media in the enterprise?

Great question, short answer: who the fuck knows?

Within the enterprise today, there is way more technology than people know what to do with. They aren’t using half the stuff they have.

They are doing really simplistic things with counting likes and mentions, but they aren’t drawing any meaningful conclusions.

My vision is that with better use of analytics AND statistical analysis and more of a marketing mix modeling approach they will be able to do a better job teasing out cause and effect between social media and consumer behavior. Of course, in order to do this we will need to move beyond this stupid level of “sentiment” analysis and train computers to measure trust and commitment and other elements of relationships. But given the pace already and the competitive environment, I’m guessing they’ll crack that code by 2015.

Additional comments?

I think what is really interesting to day is the impact that Groupon and Foursquare will have on measurement – because they are so highly measurable, marketers who are screaming for ROI may well pull money out of Twitter and Facebook and put it into them just because they are so much more measureable.