Integrating Text Mining with NPS: The Story Behind the Score at Sage Software

A Commitment to Customer Centricity

Sage North America provides customizable business-management software and services for accounting, customer relationship management (CRM), human resources, merchant services and time tracking, as well as specialized solutions for customers in construction, distribution, healthcare, manufacturing, nonprofit and real estate, among other industries. About 2.9 million small and medium-sized businesses rely on our products, and their needs vary depending on applications and industries. We are dedicated to providing superior customer experience to the entire range of our clients, and that requires staying attuned to their unique and changing needs and desires.

This is one of the key goals of our Customer Relationship Program. We strive to contact and survey 200,000 customers quarterly, to ensure we stay in touch with what they need and how they feel about Sage North America's products.

When the Customer Relationship Program was launched three years ago, it was exclusively quantitative data on which we based our primary metrics for trending, ranking and assessing customer feedback. But the open-ended comments that were captured in the surveys could not be efficiently processed. The general managers for Sage CRM Solutions and our other products had to manually read and categorize this qualitative information. The process was complex and time-consuming, and it also was nearly impossible to accurately link customer loyalty metrics to customer behaviors.

This issue was certainly not unique to Sage North America at the time. In 2006, when we started the Customer Relationship Program, CDC Respond reported that, while 95 percent of surveyed companies collected feedback in some form from their customers, only 35 percent used the data in some way. Furthermore, only about 10 percent made changes in response to the feedback—and just 5 percent told their customers what they had done in response to the suggestions.

At least we were leveraging the NPS scores and other quantitative data that we were collecting. And we always understood the potential value of the qualitative, textual feedback gathered in our surveys; we just didn't have a good, efficient way to convert that feedback into dependable, timely business intelligence.

Implementing CEM and Text Mining

For years at Sage North America, we've felt that data are like garbage—you don't want to collect it unless you know what to do with it. CEM and text mining are what we now do with the open-ended responses we receive in combination with our customers' NPS scores.

We implemented Clarabridge's Content Mining Solution service in the fourth quarter of 2007. Clarabridge's text mining solution augments NPS by illuminating the "why" behind a customer's scores.

Text mining can capture and analyze valuable customer intelligence found in various text-based feedback channels and indirect sources of consumer opinions, such as call center notes, qualitative survey feedback, Web 2.0 content, online consumer forums, social networking sites and even customer warranty forms. Aberdeen Group in July 2007 estimated sources of unstructured data such as these to account for 85 percent of all of the data available to a company.

Our text-based customer feedback is automatically categorized, and associated sentiment is determined. The qualitative feedback is integrated with quantitative via a single report system, and this allows us to efficiently rank the issues that customers are talking about in relation to their NPS scores and identify the aspects of our business that cause our customers to recommend us to others. It is a fully automated, end-to-end approach to CEM that enables us to make more intelligent business decisions on behalf of our customers.

Measurable Impact

Since implementing text mining and CEM, we have incorporated 21 product lines into our survey program. Hundreds of thousands of verbatim responses have been analyzed. The positive impact is far-reaching. The insight we receive from our customers allows us to develop short-term plans to maximize our "wins" and longer-term plans to address the "pain points."

Because text mining avails our product general managers to real-time customer intelligence, it serves as a valuable, early warning system to identify burgeoning concerns and address issues (such as software bugs and customer service issues) before they reach a broader base of customers. This ability helps us keep small problems from growing—avoiding potential damage to our reputation for quality among customers, as well as the costs of dealing with large-scale support issues.

Not only does our expanded Customer Relationship Program enable Sage North America to make short-term corrections or enhancements, it also provides guidance to our general managers so that they can develop well-founded, longer-term strategic plans. Our product general managers use our "Category Influencer Report" rankings—leveraging the business intelligence harvested via text mining and CEM—to assist them in the development of their annual action plans.

Creating a Competitive Gap

Customer experience is the gap that industry leaders are exploiting to distance themselves from their competition.

For "The Customer Experience Index, 2008," Forrester asked 4,564 U.S. consumers about the usefulness, ease of use and enjoyment of their experiences with 113 companies across 12 industries. Only 11 percent of the companies received an "excellent" rating among the 4,564 consumers surveyed; 38 percent were rated "poor" or "very poor." In a March 2008 report, "The Business Impact Of Customer Experience," Bruce D. Temkin wrote, .".. Our analysis shows that good customer experience correlates highly to loyalty—especially when it comes to consumers' plans for making additional purchases. When we examined how this might affect the annual revenue of individual companies, we found that customer experience quality could cause a swing of $242 million for a large bank and $184 million for a large retailer ..."