Closing the Feedback Loop at Scale: Automating Customer Sentiment Analysis for a Global Apparel Brand.

About the Client

Company O is one of the world’s largest suppliers of athletic footwear and apparel. It collected customer feedback across nearly every transaction channel — its website, in-store surveys, call centers, and email surveys — as part of a broader focus on customer experience, brand positioning, and brand image.

Industry Context

Net Promoter Score, developed in 2003 by Fred Reichheld with Bain & Company and Satmetrix, had become one of the most widely adopted customer loyalty metrics across consumer-facing industries, valued for its simplicity but traditionally calculated from small survey samples rather than a company’s full base of customer feedback. Attensity, whose Analyze platform anchored this project, was among the leading vendors in text analytics and customer experience management. For large consumer brands generating well over a million comments a month, the real challenge was not collecting feedback but making genuine use of nearly all of it — moving from a small, manually reviewed sample to full-population analysis was a defining shift many customer-experience-focused enterprises were making around this same period.

Current Situation

Company O wanted to use its voice-of-the-customer data to quickly identify and respond to problems, improve the performance of its internal operations, strengthen its brand positioning and messaging, and enhance its brand image. However, with more than one million customer comments per month, the volume was far too large to read and analyze manually, so Company O purchased Attensity’s Analyze software and asked Iknow to configure the product and set up the data analysis methodology.

• An unmanageable volume of customer feedback. Company O collected more than one million customer comments per month, far too large a volume to read and analyze manually.
• Most customer feedback going unread and unused. Before this project, Company O could read and respond to only a fraction of the feedback it collected, leaving most customer input unused.
• A key loyalty metric based on an unrepresentative sample. Company O’s Net Promoter Score was calculated using only a small sample of its actual customer feedback, limiting its accuracy and usefulness.
• No consistent way to classify and route feedback. Company O needed a way to consistently classify and route enormous volumes of unstructured, verbatim text comments to the right internal teams for action.

• Configure Attensity’s Analyze software and establish a data analysis methodology for Company O’s voice-of-the-customer program.
• Develop a comprehensive set of categories for classifying customer comments.
• Automate the processing, classification, and sentiment analysis of Company O’s full volume of customer comments.
• Establish an internal reporting process routing results to the executives responsible for addressing them.

How Iknow Structured the Work

Iknow structured the engagement around building a category taxonomy from a representative sample, automating classification at full scale, and validating and routing the results — reflecting Iknow’s automated content classification methodology for turning an unmanageable volume of unstructured feedback into a trusted, actionable reporting process.

Key Activities & Decisions

• Category taxonomy development. Iknow reviewed a representative data subset and, working with Company O’s marketing, digital commerce, and business intelligence staff, developed the categories for classifying comments; more than 130 categories were ultimately selected.
• Automated classification via learn-by-example. Iknow used the Analyze learn-by-example (LBE) capability to automate the processing of text comments, reading and analyzing each sentence to determine its meaning and sentiment, and classifying it into one of the defined categories.
• Testing and validation. Iknow worked with Company O to test and validate the results from the automated classification.
• Internal reporting process. Iknow established the internal reporting process, routing results to specific executives throughout the company so they could address problems within their spans of control.
• Full-scale processing and NPS calculation. Iknow processed and analyzed more than five million comments, categorized them, and calculated the Net Promoter Score.

Stakeholders & Collaboration

Iknow served as a subcontractor to Attensity, the engagement’s prime contractor, working directly with Company O’s marketing, digital commerce, and business intelligence staff throughout category development, testing, and reporting design.

Turning a Flood of Unstructured Comments Into Something Classifiable

More than a million largely unstructured, verbatim comments a month risked becoming an unmanageable flood of text rather than a usable signal. Iknow addressed this by first reviewing a representative data subset and working directly with Company O’s marketing, digital commerce, and business intelligence staff to build a genuinely comprehensive, business-relevant taxonomy of more than 130 categories before attempting any automation.

Trusting Fully Automated Classification at Massive Scale

Classifying nuanced, sentiment-laden customer language automatically at scale, without human review of every comment, required real confidence in the underlying model. Iknow addressed this by rigorously testing and validating the learn-by-example classification results against Company O’s own data before relying on it for reporting, and by routing results into a structured internal process so specific executives could verify and act on findings within their own areas.

Quantitative Outcomes

• Feedback volume: More than one million customer comments collected monthly across Company O’s website, in-store surveys, call centers, and email surveys.
• Classification taxonomy: More than 130 categories developed for classifying customer comments.
• Comments processed: More than five million comments processed, analyzed, and categorized during the engagement.
• Feedback coverage achieved: Coverage moved from a small sample to 100 percent of customer comments and feedback.
• Engagement duration: Six-month assignment.

Qualitative Outcomes

This project implemented an ongoing process for collecting, analyzing, and acting on voice-of-the-customer data and for calculating the Net Promoter Score. Before this project, Company O read and responded to only a fraction of the feedback it collected, and its Net Promoter Score was calculated using only a small sample of data; Company O now electronically reads and reports on 100 percent of the customer comments and feedback it receives. Company O achieved significant improvements across its operations by closing the feedback loop between its customers and the internal departments and functions that serve them.

Timeline to Impact

Within the roughly six-month engagement, Iknow moved Company O from manual, sample-based feedback review to a fully automated, continuously operating process covering 100 percent of incoming customer comments, with results reaching the executives accountable for acting on them.

Core Skills

• Automated text classification and sentiment analysis
• Voice-of-the-customer program design
• Net Promoter Score analytics
• Learn-by-example model configuration

Methods & Frameworks

• Representative sampling and category taxonomy development
• Automated classification testing and validation
• Executive-routed internal reporting design

Technologies & Tools

• Attensity Analyze (customer analytics and engagement application)
• Learn-by-example (LBE) automated text classification