So we’re currently trying to tease out, exactly, what people are using analytics for, and what ultimate goals those actions feed into. When we’re on the phone with members, overwhelmingly we’re hearing that data brings enterprises closer to the consumer, leading to all sorts of better outcomes: more resonant marcomms, higher margins through more effective price discrimination, and, for some companies, better products that arise through access to protected, proprietary data assets (like Nike+).

I could imagine two ways that data might feed into customer centricity (whether it’s helping or hurting). Story number one more or less goes: we as a company had very little idea who our customers were, what they liked, how they socialized and what kind of products they bought from others that they could be buying from us. When we integrated advanced marketing analytics and unstructured data, the numbers told us more about our customers than we already knew, and we became more customer-centric.

The other story goes: we as a company had very little idea who our customers were, and therefore we integrated big data and advanced analytics. But we couldn’t choose which data to use, and our analysts and marketers got caught up in a never-ending cycle of analysis paralysis. Moreover, thinking about the consumer as an abstract concept in data led to people forgetting the importance of experience and observation. In the process, we lost sight of the softer, qualitative ways that we learned about customers, and ended up becoming less customer-centric.

Which of these is more plausible? I’m not sure, but my gut says it’s the second story. I can count the number of companies with great, consumer-apparent uses of data on my fingers and toes, and analytics vendors have bigger appetites than that; there are surely hundreds of companies out there with data on their hands of varying effectiveness.

So, we thought we’d bring the question to you. Answer the poll below to let us know how you feel about data and analytics’ role in customer-centricity. Want to add some details? Let us know in the comments section.

This list was eerily reminiscent of the results from MLC’s Marketer Quick Poll from a couple of months ago.  Only in our case, we had asked marketers about their top challenges on the data management frontier.  If these challenges are so similar between such different subjects, then perhaps it’s time to reposition and take a step back to look at the broader marketing environment.

The easiest big-picture framework that came to me was the traditional supply-and-demand curves.  For simplicity’s sake, we can consider the consumer market for baby food.

Assume we hold the supply curve constant.  To increase the amount of consumer surplus under the demand curve, we can do one of two things:

1. Try to make our captured demand hug the full consumer demand closer.  (Gerber battles Baby’s Best!)
2. Attempt to shift both demand curves further out along the supply curve.  (Expand the economic pie – for instance: Gerber using analytics to discover that older adults without teeth were an underserved market)

Most marketers would agree that achieving both would be ideal, and if they had to pick, they’d aim for the latter.  But if we look at actual practices, most marketing departments are focusing their social media and analytics efforts in the first one.

Their thought process might go something like this:

Sure I’d like to just burst through the innovation bubble and find a whole new untouched consumer population…

But we don’t have the innovative power, and it certainly won’t be easy justifying riskier, creative ventures to the rest of the organization.

Besides, the consumer landscape is changing so fast, I’m having a hard-enough time just keeping up with my competitors!

So let’s just work on speeding up current activities and getting as much consumer information as possible.  Who knows, maybe we’ll get lucky and come across something that will push innovation forward!

However, while aiming for “real-time” relevance has its merits, it may not be the smartest way to secure customer value and loyalty.  Consider the following: are marginal increases in market share sustainable?  Are consumer preferences really changing so quickly, or does it just seem that way with recent technological/analytical advances?

We’ve recently been thinking that focusing on speed may lead to smaller marketing improvements with fleeting market advantage.   Keep an eye out for our survey in February, when we’ll be gauging Marketing Agility (speed, flexibility, and all the factors that represent entrepreneurial readiness).  Participating companies will get a benchmarking report.  Email me if you’re interested in taking the survey or learning more: yzhang@executiveboard.com

But bridging the gap from data to action almost always requires an intermediate step – testing. And with big, real-time data, the B2C space just might be entering a golden age of testing and learning, as the test-to-results cycle speeds up and decisions can be made faster.

It’s important, though, to keep the possibilities here realistic – tweaking products, experiences, or messages will almost certainly produce marginal results, ones that might be eaten up by the macro factors at play in any business success – technological trends, the economy, that sort of thing. Case in point is Sears/KMart – a company that’s come on some hard times in the last few years, and one that recently announced that it would close between 100 and 120 stores in the coming year.

Of course, a lot of this has to do with economic and technological shifts facing all big-box retailers: the improving quality of mobile and e-commerce, easy price comparisons, an overall dip in consumer spending. But Sears hasn’t sat idly back, waiting for the waves of change to render it irrelevant – they’ve been extraordinarily active in testing different sales-floor formats, trying to figure out which resonates with consumers the best.

As Jesse Eisinger wrote in Portfolio a few years back:

Retailers experiment all the time. But Sears Holdings celebrates its test-and-learn culture as a matter of corporate pride. [Sears CEO Eddie] Lampert’s idea is that he can, using data and good business sense, eventually figure out what’s wrong and fix it. “One of the great advantages of having approximately 2,300 large-format stores…is that we can test concepts in a few stores before undertaking the risk and capital associated with rolling out the concept to a larger number of stores or to the entire chain,” he wrote in a letter to shareholders.

Lampert’s tests are peppered throughout the country. In Zephyrhills, Florida, the company put a Sears within a Kmart. In Houston, it’s trying out a huge home-appliance showroom. In Rockford, Illinois, Sunderland and his team are testing a new Kmart design that has an outdoor-marketplace feel. Duluth, Georgia, has a retro-themed store. Maureen McGuire, Sears Holdings’ chief of marketing, says that testing is now so embedded in the culture that the company put two different covers on its famous Christmas staple, the Sears Wish Book catalog, which it reintroduced this year after a 14-year hiatus. The blue one with stars tested better than the red one with pictures of Christmas cookies in the shapes of power drills and high-heeled shoes, she says.

The lesson? Sears designed smart tests and has learned a lot about what its customers like and do not like, but these tweaks to the experience did not – and probably could not – overcome the immense challenges all big-box retailers face in the next few years. That runs counter to a lot of our conversations with marketers, some of whom seem to believe that marginally-better customer understanding will lead to big step changes in revenue.

We’re still in the early stages of our work, but I think we’re going to find that one of the biggest challenges with data and analytics won’t be solving problems with data, but rather figuring out what problems to solve.

(from guest blogger Jonathan Rudick, VP of Customer Experience at HSBC)

Good use of analytics is the only way to know what transactions are happening on a large scale.  Especially with financial services, data measures both transactions and behaviors, and leveraging this data can aggregate these trends and analyze them, providing a better understanding of the customer.  This deep customer understanding can help improve service across time, allowing the customer to develop a deeper and more loyal relationship with the brand.

Many marketers understand that analytics are creating more possibilities to understand the customer than before, but a recent IBM study found that 71% of CMOs aren’t prepared to use this data to help their business.  There are four main keys to getting the most out of data:

1. Hiring imaginative analysts. Analysts don’t just need to be good with numbers – they need to have enough imagination to look through and past the numbers to find the connections that are important, and have the industry and operational knowledge to know how insights relate back to the business.  One way to ensure that analysts have this breadth of skills is by hiring people who have worked in both marketing and operations; this diversity of experience encourages creative analysis without sacrificing quantitative skills.
2. Developing a taxonomy for unstructured data. Many analysts are now using the phrase “unstructured data” as a catch-all for everything that they don’t know how to structure.  HSBC develops taxonomies to give a bit of structure to unstructured data as we dig for insights – such taxonomies make the data much more useful.  We use these taxonomies in analyzing informal text and speech data where it has helped to uncover new insights that never would have been found without unstructured data.
3. Contextualizing all numbers. A common mistake is using numbers without explaining them, yet this can be very dangerous.  One manifestation of this is that numbers are considered outside of their original context; for example, if a number is one in a series, analyzing it on its own will remove all of the original context.  Another example: metrics created to measure a specific area of business activity are applied to other dissimilar activities; this is dangerous because it creates false certainty by numerically comparing the metrics – the “apples-and-oranges” trap.
4. Automating some analysis. There are some key metrics that are analyzed each and every month.  For many of these metrics, analytics teams have analyzed them so many times that they know exactly what they’re looking for, yet developing these monthly reports can take a lot of the team’s time.  Automating this analysis allows the analytics team to focus our time doing far more useful (and interesting) work, such as searching the data for new and sometimes counterintuitive insights that can help grow the business.  In an ideal world, a team would spend about 80% of time hunting for fresh insights – something that can only be accomplished through the application of human wisdom and perception.

Through a combination of quantitative and qualitative research we discovered a few key lessons that everyone considering, implementing, or optimizing marketing automation tools should know.  Our findings, ideas, tips, and best practices (including data from a benchmarking survey of 161 B2B marketers) are all collected on a dedicated Marketing Automation Key Findings page on our website.  Top takeaways include:

• Marketing automation success is a long, hard road.  Just 31% of those we surveyed said they had seen positive, measurable ROI from marketing automation that met or exceeded their expectations.  One of the things that distinguished those folks from the rest of the crowd was the length of time they had been using marketing automation – the overwhelming majority had been at it for a year or more.

• Have reasonable expectations for returns.  The most commonly realized returns included things like better alignment between Marketing and Sales, better qualified leads (and a higher volume of leads), and more engagement with marketing collateral (in the form of higher email click-through-rates, more thought piece downloads, and improved website metrics).  Very few respondents had seen things like bigger or faster deals or higher close rates.

• Put people first.  While the majority of marketing-automation-related angst we heard was about which software vendor to select, it is the people and processes that plug into and overlay the software that are going to lead to success or failure.  One of the best practices featured on the Key Findings page is a great profile of Sutherland about Marketing and Sales collaboratively hammering out a lead hand-off process.

Marketing automation is not a magic solution to all that ails B2B marketers, but it can enable those who use it smartly and strategically to accomplish some great things.  Check out our profile of Telus for what can happen when marketing automation is firing on all cylinders.

MLC Members – find all this and more on our Marketing Automation Key Findings page and share your marketing automation thoughts, opinions, and experiences in the comments section below.

Mr. Laursen’s latest book, Business Analytics for Sales and Marketing Managers, tackles marketing analytics head on.  He explains a pragmatic “top-down” approach for determining what data to use, which algorithm to select, and how to implement the results.

Intrigued by his insights, we took some time last week (a windy Friday afternoon in Denmark) to chat with Mr. Laursen about the role of data and analytics in marketing.  Here are a few interesting nuggets from the conversation:

• “Include the data guys.” We’re all aware of the cultural schism where the “boring analysts” are never invited to creative meetings.  But let’s face it, if nobody in strategy knows how to use information, then how can anyone make a strategy that can be informed by data and measured?  Someone in the strategy creation phase needs data and analytics knowledge, or else you might end up setting unreasonable and/or unmeasurable goals for yourselves.
• “Target the young ones!” There’s virtually no career for an analyst in a marketing department because s/he could just make a better career elsewhere.  That means that the key to analytics talent lies not in hiring, but in training.  The best way: nurture relatively young/junior marketers by having them stick with data guys for a half their time over a year or two.  If the majority of junior marketers learn data and analytics skills early on, eventually the culture and approach of the whole department will become more scientific.
• “Market your data capabilities and customer knowledge.” Maersk Line is a market leader with some of the best data in the shipping industry.  However fragmented that data may be, it would be smart to use that knowledge power as a strategic axis.  After all, who wouldn’t follow the company that is updated about its customers and can improve their products and services accordingly?

To enable stronger business intelligence, you have to strengthen your weak data link.  Otherwise, as Mr. Laursen puts it, “you will become an infosaurus that is eventually outsmarted.”

By Chris Frank, a former marketer at Microsoft and the author of a new book on decision making.

We have become a world of data addicts. But with more data comes the feeling of, “How do I make sense of all this? Can’t we break this down into a handful of simple points?” Critical questions about market demand, customer buying behavior, subscriber acquisition, brand positioning and your product’s roadmap are great. However, the interesting discussion comes to a screeching halt when the data arrives. Instead of being a well-arranged piece of music, it is a mash-up of sounds. The volume drowns the substance.

Data Rehab

Information is essential to making intelligent decisions, but more often than not, it simply overwhelms us. The 24/7 data explosion around us is both troubling and addictive. Consider that this year, The Economist estimates we will create 1,200 exabytes of data or more than 22 million times the amount of information contained in all the books ever written. That’s eight times the amount in 2005 and the annual volume is increasing geometrically.  The question isn’t how to stop the deluge, but how to get real value from it. How do you find the truly essential nuggets of information and use them with confidence to effectively grow your business and distinguish yourself in your company?

The answer, ironically enough, is found in asking questions. The smartest person in the room is the one that knows the questions to ask to separate the wheat from the chaff. This leads to discovering relevant facts, developing insights and delivering them with impact. Adapted from a new book, “Drinking From the Fire Hose”, below are three questions to ask yourself whenever you are suffering from information overload:

Ask “What Surprised You?” to Foster Dialgoue

The key is to foster real dialogue, so try asking your colleagues, “What surprised you?” At first you may not get a response but press the point and wait for new information to surface. Why? Because surprises are bias killers. Surprises make people think differently. This question will spur new discussion, uncover fresh learning and lead to new insights that separate meaningless facts from relevant information. The question exposes outliers in the data, draws connections between seemingly unrelated conclusions and opens different avenues of discussion with your colleagues. We must get in the habit of looking for things we don’t expect.

Ask “Should You Believe the Squiggly Line?” to Counter Short-term Thinking

Short-term thinking can be fatal to businesses, especially those focused on quarterly reporting. Relying on short-term data is not just misleading – it also robs you and your business of the continuity and equilibrium on which long- term success depends. The most effective way to discuss results is to consider just three factors: the absolute score, or today’s numbers; the competitive score, which is how your company is doing relevant to its competitors; and the score over time, or how you are faring against your competitors over the long-term.  By triangulating these three easily measured data points, you can gain almost all the perspective you’ll need and ensure that you don’t get hung up on short-term thinking.

Ask “Who are Our Swing Voters?” to Revive the Customer Conversation

Successful businesses generate additional revenue without incurring new marketing cost. This simple but powerful maxim is the key to driving profitable growth. It is important to know this and figure out how to apply it to your business. Popular wisdom states that neutral customers do not matter. Businesses typically focus on the extremely satisfied customers, yielding no new revenue or critics who’ll never switch to your brand. Yet, winning the neutral customer – or swing voter –  can be the most cost-effective method for driving growth.

Whenever you feel yourself getting lost in the data, consider asking smarter questions to reveal better answers, and be the catalyst that changes the dialogue in your company and with your customers.

For the full details on each question and how to apply them to your business, check out www.firehosethebook.com.

Christopher Frank is the co-author of Drinking from the Fire Hose: Making Smarter Decisions Without Drowning in Information (Portfolio/Penguin, September 2011). He previously spent 10 years at Microsoft as senior director of corporate research, worked at Accenture as a consultant in the consumer and technology practices, and founded an online start-up called Drei Tauben Ltd.  @chris_j_frank

Readers familiar with the outer reaches of the social media space may have heard of a concept called “the quantified self“, an ethos and accompanying suite of technologies designed to give average people data and insights into one of the most mysterious things in the world – the workings of their own bodies. Nike+, which we wrote about a few months ago, is a “quantified self” technology, as are more comprehensive solutions like the Jawbone Up, which – in addition to physical activity – can track daily calories burned and sleep quality.

Part of the value of quantified self technologies is ease of comparison – you can easily figure out if something that’s going on with your body is normal or cause for concern. But in order to get enough data for comparison, you have to have a critical mass of users – and quantified self stuff is probably not mainstream enough for statistically-significant sample sizes for any given question.

Well, what if there were organizations with hundreds of thousands of health consumers – enough to provide adequate sample sizes for just about any “is this normal” question, with a vested interest in those consumers taking preventative health measures? Oh, wait, there is!

Snark aside, I know that there are serious regulations around patient data, although I won’t pretend to know the finer points about what can be shared, and what can’t, as well as how patient consent changes that calculus. If HIPAA won’t allow reporting of aggregate data with patient opt-in, then HIPAA should be changed to allow it given the clear health benefits it would make available to customers.

But it kills me that this general idea – take data from the giant mass of health consumers under insurance plans, and give patients better insight into what good looks like for people of their rough demographic – is being pioneered by startups, established tech companies, and athletic shoe manufacturers, not health companies. We know, for instance, that providing insight into what others have done reduces consumer stress appreciably. It’s not a giant leap to think that consumers would appreciate the same from their health insurers.

As a health insurance consumer myself, the feeling I most want to have upon thinking about my insurance company is “they’re on my side”. Sometimes that means a trusted adviser, working one-on-one to solve a problem, but other times it means the strength of a corporate behemoth, knocking down walls to improve my health. Health marketers have little problem doing the first; I’d like to see a lot more of the second.

If your’e interested in Big Data’s potential for Marketing, please join us for a webinar in a few weeks, featuring experts from Intel, Hearst, and Maersk Line.

In the meantime, here are a few reasons we’ve spotted to question the potential, with the caveat that we’re purposefully not covering the positive here:

The map is not the territory. I’ve been reading an interesting book lately called Seeing Like a State: How Certain Schemes to Improve The Human Condition Have Failed. (Bear with the incredibly nerdy title for a moment.) The main point is that, as governments grew in power and in ambition, they tried in various ways to make their people and territory “legible” – in other words, easily understood from a distance. For instance, surnames and fixed addresses are relatively modern inventions, mostly imposed upon people in order to better identify them for the purposes of taxation and conscription. Some of these schemes have been good, others have been bad, but the main point is that they are massive simplifications, and miss important things about the way people really live.

Marketers aren’t as powerful as kings, presidents and prime ministers, but in embracing big data, we’re essentially doing a high-tech version of what governments do all the time, which is try and get a schematic picture of what society looks like. The schematic will necessarily be simplified, and occasionally it will be simplified in important ways that mean whatever you do with the data you have won’t work. For instance: if you’re younger, you probably have seen that some of your friends are in fake Facebook “relationships” with their best friends. If you’re putting out Facebook ads that suggest gifts for a significant other’s upcoming birthday, that money is wasted on those folks. Similarly, I understand that listing one’s friend group as family members is also prevalent, particularly among high-schoolers. And every single one of us on Facebook is friends with people we don’t really know, have fallen out of touch with, or don’t even like – and algorithms that treat these friendships as equally-weighted are necessarily suboptimal.

Your organization might not be ready. There’s a reason that the seminal book on marketing analytics, Tom Davenport’s Analytics at Work, focuses less on high-tech data platforms and cutting-edge analysis techniques than it does on organizational structure and team skill-sets. You can have the greatest data in the world, but if your organization is not capable of acting on it – for instance, if your supply chain is not flexible enough to quickly route inventory to where its most in demand, or if your marcomm process involves too much in the way of approvals to capitalize on short-term spikes in conversation, then all that data is getting you mostly nowhere.

We think some of the movement on talent in the next five to ten years will fix a lot of these kinds of problems. But it’s important to remember that data isn’t just plugging in better numbers to existing processes – processes themselves have to be revamped if you’re going to take full advantage of data.

There could be diminishing returns to what can be done with data. Okay, I’ll admit that this is pure conjecture. But it seems, to me, that there are diminishing returns to what Big Data can provide an organization. We’re testing this in our research this year, but one of our hypotheses is that, after a certain point, additional personalization in marcomms gets tuned out by consumers or is regarded as creepy; other data-oriented interventions, like inventory shifts and product personalization, have self-limiting returns.

If you already know a good deal about your consumers, Big Data platforms could be a Big Investment with little to show.

The technology might not be ready, and it could be getting pushed for the wrong reasons. As Pat discussed in the post I linked above, a lot of Big Data discussion is hype-driven, by folks with a vested interest in hype. The natural tendency is to idealize features and benefits, and minimize costs; market understanding of how these platforms work is limited enough that there’s not a lot of appropriate skepticism about what data can do for organizations.

Cautious organizations, I think, should tread carefully until the market for data solutions unfroths itself a little. There’s a decent chance that the provider/vendor you choose will get surpassed from a technology or capabilities perspective; why not wait until there’s a clear leader?

I know, it’s tempting to put in just a little bit more deliciousness. If we have it, why not use it?  Be it the extra stuffing or the bunch of variables that haven’t made it into your very comprehensive analysis. Maybe it’s the regret at seeing the extras “go to waste”; maybe it’s the urge to supersize in response to complexity and diminished control.  Cooking up an insightful model from a mountain of data can be just as stressful as cooking a banquet.

As analytics makes its way into marketing, much of what we do is “exploratory data analysis” or “fishing” or “hunting” in the vernacular. We dive in with a hunch of how things will turn out but aren’t really sure. What follows is a lot of poking around to see what variables we have, whether they make sense together and what to keep versus throw out. Add to that a lot of proving and disproving. The process is unstructured, impromptu, slightly disorienting and involves a lot of back and forth. Often, as more and more people get involved, your once lean model ends up as unwieldy as an overstuffed turkey.

And when that happens, your model basically spews out everything that “is” but says little in the way of causation. Variables will interfere with each other and conclusions will include “noise” like “Red and white décor drives 20% increase in foot traffic”. Duh, it’s Christmas. This is called “overfitting” your model, which means erroneously taking into account every random kink and confounding detail in the data.

Don’t feel guilty – this happens even to the best of us.  If anything, you may end up there more often precisely because you care more than others about the outcome of the project. In failing to take a step back we get buried in analysis, here are two ways to help you get some distance and perspective.

• Don’t be a control freak, pick your battles. Much as we like to account for everything from individual demographics, industry characteristics, macroeconomic trends all the way to business models and geographies, 80% of the controls will turn out to be non-differentiators. While these controls are important, don’t feel bad if you need to throw most of them out in favor of a better level of abstraction. For example, a data cut by geography on key parameters can help you figure out whether you need to control for all 50 states or will larger geographical blocks like “East Coast”, “West Coast” do just fine.
• Don’t try to feed the model your own conclusions. Sometimes we start out with dead-set conclusions (instead of a proper hypothesis), using data for support rather than for illumination. By refusing to let data correct existing beliefs, you could be clinging to conventional wisdom beyond its expiration date. As a customer service rep you may no longer need to “delight” your customers as exceeding expectations during service interactions has negligible impact on customer loyalty. As a salesperson you may need to start challenging them. As a marketer you may also be better off helping customers simplify their decisions instead of constantly trying to “actively engage” them.

Hard as it may be to learn to get cozy with numbers, our agenda poll this year shows marketers turning the heat up on analytics – a majority of B2Bs and B2Cs sees it coming their way in 3 years or less. I don’t think you want to put off roasting that particular bird.

Moving too slowly, or too quickly. Companies either move too slowly, or too quickly, when it comes to analytics.  Most companies continue to rely on Analytics 1.0 solutions that simply don’t work in today’s complex world.  Conversely, some companies move too quickly, without getting proper buy-in from the organization and the various stakeholders – sales, finance, marketing, etc.  By focusing so heavily on the speed with which they are introduced, companies can often stumble truly integrating the data and analytics into existing structures and knowledge.  Without a thoughtful plan to integrate and adopt, the quality of the insights from the analytics and their adoption within the company can be limited. Wes suggested that companies should instead take baby steps toward becoming more analytics-driven to maximize the effect of analytics on decision-making.

Using data in internal silos. Many companies have siloed organizational structures, where one group focuses on direct marketing, one on media, one on digital, and so on.  Because of this common structure, it isn’t surprising that many of these companies execute, then analyze, within these same silos.  But analyzing data and attempting to get a true read on a campaign’s actual impact from within silos can be problematic because of one simple fact: Consumers’ purchase journeys are integrated, not siloed. Today, online and offline channels are intertwined, having a profound effect on each other and a consumer’s path back and forth through these different types of media to a purchase is complex. But by ignoring these consumer behaviors and just analyzing campaigns to maximize within each channel, companies miss the interactions between channels.  Focusing so heavily on a few individual activities can result in huge errors when allocating budgets and marginalize the most important metrics – business results.

Overrelying on data. While analytics can provide a more in-depth view of everything from consumer insights to media measurement, it isn’t a silver bullet.  As I wrote last week, 43% of employees ignore their own gut instincts in favor of analytics.  This overreliance on data can cause Marketing to make poor decisions. So, make sure you find a balance between the art and science of marketing.

MLC members, stay tuned for updates on our analytics research.

In our most recent survey on the future of marketing, 54% of B2B and 71% of B2C marketers tell us they expect analytics to inform their day-to-day business decision making within 3 years (more on this in later posts). While you don’t need to become a quant person overnight, it’s helpful to understand a few key statistical concepts. Here are two things that every marketer should know about data:

Sample size. The sample size gives you a sense whether or not you have that “critical mass” of people (customers, for most purposes) on which to base your conclusions. When dealing with sample sizes, watch out for a few things:

• Too small. A small sample size will result in “really?!” glances from your audience unless you have a very good reason (i.e. there are 10 pandas in the world and 9 are in my dataset).
• Too big. If you already have 5000 people, you don’t need another 2000. It’s the Law of Large Numbers – i.e. if you’re large, being larger doesn’t make you much different.
• Not representative. Marketers prefer larger samples if only to feel buttressed against hostile rebuttals but statisticians would direct you to check for “representativeness” — how well the composition of your sample fits the needs of the project at hand. If your online customers have different buying preferences compared to your brick-and-motar customers, I’d make sure to get the right names out of the CRM system when testing the efficacy of an in-store campaign before worrying about response rates. The same goes with any substantial differentiating factors like race, gender and age.

Significance level. Significance is an econometrics term; basically, it’s a measure of to what degree a particular variable is responsible for reaching an outcome. For instance, price might be a more significant variable when it comes to market share than, say, store placement. When dealing with significance, here’s what to watch for:

• Not really causal. When a variable is suspected of causing an outcome we don’t really know whether it’s guilty as charged, acting as cover for another factor, or whether the causation runs the opposite way. Corporate culture, for example, could show up as a “driver” of business outcomes when all that’s going on is people attributing openness and innovation to their company when the business is doing well. Phil Rosenzweig covered this aptly in “The Halo Effect”.
• Insignificance might matter. If significance is statistically prominent, then the insignificance of something you thought would matter should capture your attention like an awkward silence. This is your chance to debunk some conventional wisdom and find potential areas to innovate/change. Surprised that simple cheerleading on Facebook didn’t drive loyalty? Time to ask fans what would worth their while.

Keeping these two concepts in mind will help a lot of marketers – particularly folks at the mid and junior levels – conversant in data, and will help them apply statistical concepts to their work. MLC members, we’re curious – how are you training non-quant staff in statistical concepts? Let us know in comments.

But not every company is maximizing their use of data.  One common problem is that individual employees aren’t using the data correctly.  As this quarter’s Executive Guidance says, only 38% of employees are Informed Skeptics, or employees who make good decisions based on data by combining the knowledge gleaned from the data with their own personal judgment.  Instead of balancing the qualitative and quantitative, almost half of employees (43%) trust empirics over their own judgment, while 19% ignore the data.  MLC members, to learn how your employees can become part of the 38% of Informed Skeptics, read the Executive Guidance here.

There are also two common organizational challenges to achieving results through data.  First, companies often provide too much data to too many employees.  This quarter’s Executive Guidance shows that employees currently spend an average of 36% of their time gathering and analyzing information and that poor information accessibility creates a 52% drag on employee productivity.  Because efficient information-gathering can have a huge boost in employee productivity, many companies are investing in making this process more efficient.  To do this, many companies are developing dashboards and other tools to share the data with as many employees as possible.  Unfortunately, this can be the wrong approach.  In our initial interviews with members, one common problem is that sharing all of the information handicaps – rather than empowers – marketers.  By sharing the information with everyone, executives create a tragedy of the commons situation in which no one takes ownership because everyone is involved.

Another common problem is that too much time is spent digging for random insights.  Since 85% of data is unstructured, many companies believe that they need to garner valuable insights from this vast data frontier.  To do this, many companies are focusing much of their time and resources on ad-hoc data requests rather than doing standard analyses on critical topics.  Unfortunately, this process can be incredibly resource-intensive: Sending employees on an insight-fishing expedition can yield useful results, but these insights often come at the expense of necessary analyses on critical business measures.

MLC members, please look out for more posts on analytics as we delve further into our 2012 research.  If you would like to be interviewed on your use of analytics, please email me at colong@executiveboard.com.

This evolution is to a great extent accelerated by the proliferation of consumer data and the many good things you can get out of it. While you can certainly observe customers in store (people still do), it is infinitely more scalable to use web mining to “observe” thousands perusing the net, estimate what stage they are at in their purchase process and tailor your messages accordingly. Few would have heard of Signet Banking Corp but many are familiar with its spin-off Capital One, the 8^th largest bank in the country – and one that was built to a great extent on the strength of analytics.

But just like anything with massive impact, analytics can also go massively wrong and the failure very often has less to do with the tool itself than with the one wielding it. The only thing worse than no data is bad data, and the only thing worse than bad data is bad interpretations. Here’s how you go down that slippery slope and how you can climb back up:

• Your purpose should drive the questions you ask. For example, on MLC’s Discussions forums we often get questions about Net Promoter Scores and how to interpret them. While it’s perfectly reasonable to benchmark business unit performance year-on-year based on NPS, without data transformation it tells you little, about things like share of wallet. Your competitor maybe only 0.1 point ahead of you for all 5000 customers, but when it comes down to picking between you two, they win every time. Your NPS difference is only 0.1 yet the difference in share of wallet is a gaping 100. Office Depot learned a similar lesson when they found what’s asked in their mystery shopper questionnaire makes for a nice store but not necessarily for a nice shopping experience, leading to the odd phenomenon in which shopper scores are up but sales are down (customers don’t care about recently cleaned bathrooms). In this case their president actually took the time to go and ask customers incognito. Hey, who said you can’t combine traditional market research methods with up-and-coming analytics?

• Your data should inform, not dictate your interpretation. The claim that churches cause crime is a classic example of not taking into account a key factor – that larger cities have more room for both. The observation that Harvard dropouts earn more on average than Harvard graduates is a typical case of ignoring outliers. Developing metrics from different angles and building models using multiple suitable techniques will increase the robustness of your conclusions. People from different departments will also see the same data point differently, this is valuable troubleshooting. If your stores’ restocking speed lags behind that of competitors, is it simply because you are short on people or is it because store managers assign associates to different sections each time? Is it because your deliveries arrive when the store is busy or because inventory control is slack and wrong items get ordered? Think, and then think again from a different angle.

MLC members, for more of our thoughts on data and analytics, stay tuned to Wide Angle and our Twitter account for the next few months, where we’ll be sharing the best insights from our ongoing research into Big Data.

The problem is a “bottom-up” approach.  Marketers spend a lot of time collecting data to show progress against marketing-specific targets.  And  they typically start with data that is easy to track.   But the volume of likes and re-tweets isn’t likely to convince a CFO that money is being well-budgeted.

Instead, marketers should work their way backwards.

When I was little, I loved solving those maze puzzles in my local newspaper.  Only I always did them backwards, starting at the center and tracing my way out.  I had trouble defending my method, but to this day, I’m still a firm believer in starting from the end goal when developing a problem-solving strategy.

The marketing function at Vista – a high-tech firm that has asked us to use a pseudonym on this case – seems to agree.  Facing scrutiny from senior management, they set out to link themselves to overarching corporate goals, rather than focus on internal (marketing-specific) measurements.  The key to their dashboard: a bold, 4-step reverse-engineering process that ensures alignment between marketing and other senior management.  In other words, work your way out from the center of the maze!

1. Begin with stated corporate goals.
2. Determine the supporting marketing goals and the metrics that best capture their progress.
3. Select the activities best-suited to impact the chosen metrics.
4. Capture this information with a broadly accessible web-based dashboard.  (MLC Members, see an example of the dashboard here.)

See how Vista worked out which activities did/didn’t align with corporate goals here.

The explicit alignment of company goals and marketing progress boosts non-marketers’ understanding of the function’s impact on firm performance.

MLC members, read more about how to effectively re-engineer your marketing plan to align with your company’s goals.

Loyal readers of Wide Angle may have noticed that I am not the first MLCer to wax poetic about the power of data recently.  Not surprisingly, this is not a coincidence.  We are starting to map out our 2012 research agenda and data – more specifically, how to collect it, how to analyze it, and how to make it a bigger part of decisions – is at the top of many members’ lists.  In the B2B space, that question of how to best use data is frequently presented in the context of working towards becoming a more customer-centric organization through better understanding of who customers are and what they need.  The increasingly digitized (and therefore, trackable) nature of marketing and Sales activities through tools like website tracking, CRM, marketing automation means there is a huge amount of data to be potentially collected and analyzed.

But, as the marketer I referenced at the top of this post realized, raw data and actual actionable information are not one and the same.  Statistics are one thing, insight is another.  People who blindly attach accuracy and importance to claims simply because they have numbers attached to them are in danger of following those numbers right off a cliff (this idea is not lost on my comic hero, Dilbert).  At the same time, those who eschew all data and rely solely on their instinct aren’t likely to fare much better.   When making an important decision, there must be a balance between hard data and gut instinct (honed through years of marketing experience).  Just what they balance is and how marketers can work to achieve it will be the subject of a lot of MLC study over the next few months.

Stay tuned as we explore this topic in greater depth.  In the meantime, I’ll offer my early reading list:

• Analytics at Work: Smarter Decisions, Better Results by Thomas Davenport, Jeanne Harris, and Robert Morrison (if 240 pages is too much of a commitment for you right now, check out these two summary articles by Accenture and HBR)
• Friends With (Digital) Benefits: CMOs Link With CIOs by Natalie Zmuda
  
• How We See It: Three Senior Executives on the Future of Marketing in McKinsey Quarterly (pay special attention to the thoughts of Yahoo! Research’s Duncan Watts)

MLC members, what are the biggest challenges you are facing in the coming year?  What would you like us to put our research muscle into?  B2Bs can tell us here.  B2Cs can offer their take here.

We all secretly wish for a magic weapon to vanquish competition. Where better to place our faith than in a model churning out intimidating, neat lines of output? A perfect deus ex machina to get out of a sticky situation. However, as with anything complex, we forget that it’s one thing to own analytical infrastructure and another entirely to be able to use it well.  If you’ve ever gotten your parents a smartphone, you know what I mean.

Regardless of whether you’re advanced enough to implement agent based modeling or just taking baby steps beyond bar charts, a few ground rules remain the same:

• Set a target because your analysis is only as good as your data. Whether you’re pulling data from the CRM system or fielding a new survey to gauge consumer loyalty, you should know up front what you’re looking for, starting with a list of hypothesis to prove or disprove. Which metrics, markets and consumer segments are relevant? Gather the quant and non-quant people together at this stage will help you get at what is both interesting and feasible to test. If you’re pulling from existing data, be generous in allotting time to data cleaning, especially if they come from disparate business lines and geographies, something as simple as an erroneous currency or unit conversion could derail your conclusion.
• More/ fancier analytics does not equal higher “analytical maturity”. For many companies, the arrival of analytics elicits two common responses – “yay-for-hard-definitive-proof” and “who-cares-I-use-my-gut”. In their survey of 5,000 employees at 22 global companies,  our sister program for IT executives, the CIO Executive Council, found that 43% of all employees are “unquestioning empiricists”, while another 19% are “visceral decision makers” – overly reliant on or lack appreciation for data.  Only 38% are the “informed skeptics”, people who apply judgment to data, are aware of its limitations, their own biases and are ready to teach. Which group do you belong to? Take their quiz to find out.
• Build your team of “T”s. “T-shaped” talent describes people with deep knowledge in one area (the “I” part) and comfortable operating less deeply in a host of others (the “―” part). For a quant person, that means statistical expertise complemented by a healthy dose of business acumen and communication skills.  Depending on one’s level, the shape of the “T” should vary. An executive “analytical champion” (a la Thomas Davenport) would be something of a “fat T” (broader rather than deeper) whereas a “data scientist” would fit the profile of a “skinny T” (deeper rather than broader). The remaining 80% of the analytical taskforce falls somewhere in the middle.

It is easy to get lost in the numbers and forget all about intuition and judgment. In my daily encounters with data, I’m often reminded of this sobering fact: “The combination of some data and an aching desire for an answer does not ensure that a reasonable answer can be extracted from a given body of data.” Tip of my hat to John Tukey, the statistician who championed exploratory data analysis.

Here’s the cynical interpretation of what is evolving in the marketing space right now.  All this talk of Big Data, smarter commerce and real-time marketing is science fiction for most marketing functions. With some exceptions (high tech, some retailers and some areas of financial services), the marketing ecosystem in which we operate isn’t structured to support real-time, hyper-targeted, Big Data-driven marketing.  It’s still 5-10 years off.

And the hype around all of this is being driven by an unholy trinity of bankers/VCs, the entrepreneurs in social, mobile, and location tech they represent, and vendors selling data and analytics solutions.  It’s not a conspiracy at all – it’s just that each of these parties have huge financial incentives to drive the hype.  And so they do.

But consider the barriers facing a typical large enterprise marketing organization that wants to achieve the vision of real time, data-driven marketing laid out by the unholy trinity:

• Spotty data with big gaps for swathes of consumers or swathes of the consumer experience (e.g., consumer behavior in iOS mobile apps is a black box, since Apple holds that data close)
• A churning and unpredictable privacy landscape, with legislative shifts threatening disruption
• Agency rosters that are fragmented and highly resistant to the kind of collaboration and change needed to actually move with real-time speed in an integrated way for much of the marketing mix
• Client-side decision making processes, structures and skill gaps that prevent executing as the vision would have it

Unless you are: a small, nimble company, a company that grew up online (e.g., Amazon) or a rare breed of large enterprise that grew up on an analytics culture and has data coursing through its veins (e.g., CapitalOne, Harrah’s), you just aren’t going to be able to overcome these barriers in the near term.

That’s the cynic’s view.  I’m not sure yet how much of it I believe.  I do think it makes sense for companies to start the Big Data marketing journey.  But this feels to me like CRM did 15 years ago–huge promise, results a long time coming.

Here’s where I think it’s safe for marketers to START: use Big Data and analytics to figure out who to STOP marketing to, or when to STOP trying to engage them with a direct mail piece, or yet another email, or yet another plea to join in a Facebook contest.  If Wannamaker was right, half of your marketing dollars are wasted–use analytics to figure out which half, and STOP wasting them.

Your consumers and customers will thank you for it.

And, it’s all in keeping with MLC’s findings on making consumer purchase decisions simpler.

By Ana Lapter

Businesses are once again in the mood to grow revenues.  Unlike the pre-recession era, the source of growth, however, will no longer come from streamlining and automating processes, or from adopting systems for better management of structured data.  Since the majority of businesses have been improving processes and data management for some time, there aren’t too many gains to be had there.  Rather, the next era of growth will likely come from from understanding changing customer preferences and acting quickly on those insights.  In other words, your company needs to get smarter about using information, as compared to processes, to more effectively drive customer insight and quickly translate that knowledge into usable plans and strategies.

It’s not that organizations don’t have this data; you do. But the problem is how to effectively use all the information that companies gather about markets and customer preferences.  Over the past few weeks, I heard the words “analytics” and “customer insight” in separate conversations with eight senior marketers, while discussing the key analytic competencies that their teams need to strengthen or develop to move forward.

Here is my list of things to avoid when building an analytics-driven Marketing organization:

Confusing analytical maturity with the need for absorbing more data and information. The “Information at your fingerprints” phenomenon can easily motivate Marketing to become more analytical.  The real problem is not about getting more data and information, but, rather, differentiating between real customer insight and mere sets of facts.  A recent cross-functional survey of knowledge workers from our sister program, the CIO Executive Council, shows that Marketing relies excessively on data analysis, especially at the management level. Meanwhile, only 44% of marketers have high confidence in information from their own function, and only 34% have high confidence in information from other functions.

Interestingly, the survey indicates that despite excessive reliance on data and information, Marketing has a low degree of analytical maturity, as compared to other functions.  As a result, marketers miss opportunities to improve performance in areas such as new market identification, market strategy development, targeting customer segments and demand forecasting.

Over-relying on data as a basis for strategic planning decisions. The CIO survey suggests that progressive organizations use a combination of judgment and data when making strategic decisions.  This means that data should not be a substitute, but a complementary tool supporting experience, judgment and instinct.  From the perspective of organizational capability, Marketing should balance investments in getting better at data collection and analysis with the need to leverage the full potential of personality characteristics like creativity and imagination to drive effective strategic decisions.

Considering analytical tools as a solution to problems. Tools are as good as the people who use them.  A better approach to driving an information-driven culture is to consider tools as a means to solve a particular problem, improve a workflow or support a specific, narrowly-defined objective.  In addition, a decision to select a specific analytics tool should take into consideration the degree of organizational maturity and user involvement in performing diverse activities, ranging from basic reporting and visualizing information, to aggregating data, conducting pattern recognition and making predictions based on trends.

We will be exploring the topic of analytic capabilities as part of our upcoming research into marketing talent.  I would love to get your thoughts on the topic, so please contact me at: abostan@executiveboard.com.

Enter Agent-Based Modeling (ABM) – a technique best known for predicting disease outbreaks by simulating interactions between individuals and organizations. This approach tests different marcomm mixes using computer simulations.  The “Agent” is the simulated consumer – programmed to act like the brand’s target population in terms of media consumption and purchase habits. By simulating the likely impact of various touchpoints on consumer purchase behavior (rather than relying purely on historical spend/sales data), ABM can predict the influence of emerging touchpoints.  And since ABM takes all touchpoints into account, it can shed light on cross-touchpoint synergies.  Further, by modeling the behavior of individual consumers, ABM can account for differences between segments.

This technique won’t (and shouldn’t) replace mix modeling or in-market testing, but can be used in conjunction to gain deeper understanding of how the mix influences consumer purchase behavior.

How Agent-Based Modeling Works

Step 1: Recreate the target population’s media consumption and purchase habits (ABM vendors, such as ThinkVine or DemandROMI have expertise here).

Step 2: Simulate the impact of proposed marcomm plans on consumers.

Step 3: Validate the model against actual sales data and refine over time.

What You Can Model:

• Different touchpoint mixes, including emerging media and word of mouth
• Different messages/creative
• Price changes
• Changes in distribution
• Competitors’ actions
• Impact on multiple segments

Benefits

• Behavioral Insights: ABM explains how the mix works (e.g., what actions each touchpoint drives, how different segments react), not just that it works.
• Holistic Understanding: ABM takes touchpoint interactions into account and also isolates the impact of both emerging and traditional media.

Drawbacks

• Cost: An annual subscription with ThinkVine will set you back $150,000, versus about $85,000 for a one-off advanced mix model.
• Culture Change: Shifting from “last click” attribution of sales to a more accurate, multi-touchpoint attribution can be threatening to direct marketers who may feel the impact of their work is diminished.

Agent-Based Modeling in Practice

ThinkVine – an agency specializing in Agent-Based Modeling – shared a few case studies with us.

• One company wanted to optimize digital media investments in the mix and planned to eliminate Print in order to boost digital media spend. ThinkVine’s agent-based model revealed that print drove a significant portion of digital traffic (after seeing print ads, consumers would search online).  As such, the company decided to reduce Print spend only slightly and increase creative alignment between Print and digital media to capitalize on touchpoint synergies.
• Another company used ThinkVine to understand the impact of Hispanic-focused marketing campaigns on non-Hispanics. While traditional regressions only look at one segment at a time, agent-based models can test knock-on effects more effectively.

MLC members, to learn more about agent-based modeling please attend our 2011 meeting series on Simplifying Consumer Purchase Decisions.  For a comparison of marketing measurement methods (Mix Modeling, In-Market Testing, Market Contact Audit and ROMI), please click here.