It’s old news that CEOs tend to have misgivings about their CMOs. One recent study, for example, showed 80% of CEOs didn’t trust their CMO. That’s a stunning number, and a scary one for marketers. And while I always attributed a significant portion of that distrust to poor communication of how marketing metrics produce business ROI, new data suggests that a large portion of CMOs deserve some of that mistrust.
A recent survey of CMO and CIO attitudes, CMOs vs CIOs: The Fight for Mobile Web Strategy, focused mostly on their ongoing turf wars in mobile. Mobile customer tools more commonly fall under the CIO, while these same tools are of ever-increasing importance to the CMO.
Testing? Who Needs It?
One data point in this study had nothing to do with turf battles, though, and goes a long way to explaining why so much marketing is ineffective. Specifically, the study found that only half of all CMOs surveyed considered testing to be very important for improving customer experience.
While I suppose a glass-half-full optimist might take solace in seeing that half the CMOs did consider testing very important, I take no comfort from that fact. To me, this statistic says that a big portion of the CMOs out there are shooting from the hip, or assuming that they or their staff know best when it comes to designing for optimal customer experience.
I’ve spoken with dozens of conversion experts, and if there’s any common theme that has emerged from those conversations it is that testing is essential. Even individuals with years of experience and loads of expertise fail to correctly predict customer behavior. These experts earn their living by improving web and mobile results, and you’ll never find one who will say, “Don’t bother testing, I know this will work.” CMOs, apparently, are much more confident.
It’s worth noting that the CEO survey found one group of CMOs was an exception to the absurdly high level of distrust: performance marketers. This group works to deliver specific business results, like ecommerce sales or leads, and is known for an emphasis on continuous testing.
One might also suggest a brighter side: perhaps the CMOs don’t value testing much, but the people working at the customer experience design level are doing the real work, including frequent testing with real customers. I’m not persuaded by that argument, either. Budgets and deadlines are cruel taskmasters, and when push comes inevitably to shove, the stuff the boss doesn’t care about will get pushed to the side.
About the only ray of hope that I found was in the modest size of the survey. The study was conducted by NetBiscuits and polled “300 CIOs, CMOs and other C-level executives” from a variety of companies with at least 100 employees. Maybe they simply picked a bad sample.
Somehow, though, I doubt that sample size caused the data to be wildly distorted. In small, medium, and large companies, I see people sitting in conference rooms discussing how to make websites, mobile apps, etc. better for their customers. The debates rage on, but more often than not there is no user testing to inform the decision-makers. If any user testing is done, it happens when the site or app is nearly ready to launch and when only minor changes are possible.
Why don’t companies place a higher value on testing and do more of it? One big reason is the common perception that user experience testing is complicated, expensive, time consuming, and requires specialized staff and equipment. While user testing can be all of those things, it can be far simpler but still effective.
Steve Krug, author of the best-selling Web design classic, Don’t Make Me Think, more recently authored Rocket Surgery Made Easy: The Do-It-Yourself Guide to Finding and Fixing Usability Problems. In it, Krug shows how very simple testing with minimal hardware and just a few subjects will identify your worst user experience problems, even early in the design process. If you think user testing is a complex, time-consuming nightmare, Rocket Surgery is a good starting point to get useful business results with minimal resources.
I asked Steve what he thought about this depressing statistic, and he commented,
I’m pretty skeptical of how much impact the idea of UX being important is really having. I know awareness has increased 1000-fold since the iPhone hit, but I still find it hard to imagine that UX is a budget line-item with the same importance as, say, the marketing budget. I still feel like it’s what will get shortchanged (or thrown overboard) first in lean times. Not “lean: in the newfangled sense, of course.
It’s hard to argue with any of that, unfortunately. This survey certainly supports a somewhat pessimistic view of how much value companies place on user or customer experience.
Actually, I don’t think half of all CMOs are dumb or are know-it-alls. I think those who undervalue testing as a way of improving customer experience just don’t realize how simple and effective it can be.
I do find it interesting that fully 74% of CIOs found testing to be at least “very” important. That’s either an optimistic sign or an indication that CIOs are more in touch with what customers want than the executives who should be the experts: CMOs.
Does your organization test enough? Is there support for testing at all levels of management? Has the value of testing been proven by important findings? Share your thoughts and experience in a comment.
Roger Dooley is the author of Brainfluence: 100 Ways to Persuade and Convince Consumers with Neuromarketing (Wiley, 2011). Find Roger on Twitter as @rogerdooley and at his website, Neuromarketing.