Chattermill Insights

Get our latest insights right in your inbox

by Jack Miller on 4 Oct 2016

Most customer surveys are too long, and don't actually end up changing anything at a company. Here's how to change this.

Customer Feedback Data Analysis
Customer Feedback Data Analysis

Right now in my inbox I have at least 5 emails from companies asking for my feedback as a customer. I may open one of them later on if I have 60 seconds to spare. The rest I will delete. Working in a CX startup, that may come as a surprise; but fundamentally if a company’s survey mechanism is not inviting or well made then I would doubt them using my feedback in any meaningful way at all.

Response rates are the first indicator a company can use to check how engaged their customers are. If they’re low and stay low, your customers are telling you something — you don’t respect their time. The reasons for this usually fall into 2 buckets:

You survey is too long - this is such a big problem that we dedicated a whole blog post to it. In a nutshell, your surveys should be quick and easy to complete. If it takes anything longer than 2 minutes, expect a low turnout. More importantly, your sample will probably be biased and the customers who do finish the survey, will feel less excited about your product by the end.

No follow-up action on issues raised - More often than not, companies are overwhelmed by the feedback they collect. Only 23% of CMO’s said they track or measure customer feedback. So it will sit in a spreadsheet or data warehouse somewhere and be forgotten about.

Ultimately, poorly executed feedback setups do more harm than good. Low quality data leaves companies with little insight into customer experiences and customers feeling ignored. It shouldn’t be like this. Customers should be able to tell you about their experiences in their own words. Companies with a relentless focus on this gain better insights into customer behaviour and make better decisions.

Example

Consider the scenario — a travel company implements a post-trip feedback system to gain insights into what customers find are the most important factors in their trip. Trying out 2 different types of surveys, they get vastly different results.

Survey 1:

Long Survey

There are so many problems with this approach. It’s far too long (and I’ve seen much longer), response rates will be under 5% and of the customers who do respond, they’ll get survey exhaustion and click through the survey to get to the end meaning any useful insight is really hard to extract.

Yes, you can quickly calculate the correlation between the accommodation score and the overall score (or better yet the actual re-booking rate) to estimate it’s effect but a 3 for one person is not the same as the 3 for another. Moreover, you have no idea what weight does each customer assign to accommodation. Maybe they gave you a 5 but did not care one way or another.

Understandably, a company will want as much feedback as possible but it is about the quality of insight, not quantity of questions that tries to cover all the aspects of experiences you can think of. You’ll never be able to cover every single thing a customer experiences into a survey.

Plus, why would you want to? The richest data sets are those from your customers in their own words on the most important factors of their experiences. Not the questions that the Product team want answering, and the questions that Marketing need to figure out.

Survey 2

Gif for blog

By keeping it short and sweet, customers are more likely to take the opportunity to give you their feedback in their own words and mention the most important things about their trip. Using the latest advances in deep learning we can have the machine automatically analyse customer verbatims. This will give you an amazingly rich data set that you can analyse to your heart’s content, observing sentiment and theme trends to uncover why customers feel certain factors impact their customer experience.

The Deep Learning approach to processing feedback can ‘learn’ a bespoke theme structure based on the comments it analyses, catching positive and negative sentiment tags and sub-themes of customer comments that looks something like this:

final theme structure

By now you should be convinced that asking customers for feedback in their own words is the only way to build feedback systems and ultimately make customer surveys worth the effort they take to build. Following the key guidelines above will help you make use of your customer feedback, discover the ‘unknown unknowns’ of the drivers of customer experience and grow the company.

At Chattermill, we use cutting edge Deep Learning approaches to solve these problems and provide companies with complete understanding of their customer experiences. Get in touch to find out how we do it.


  • img

    Jack Miller

    As a CX Analyst, I work with companies designing their customer journeys to ensure they are both comprehensive and accurate. Applying cross sector best practices and an understanding of each business’s unique relationship with their customer base, I focus on identifying what elements need to be put in place to take each organisation from their current level of customer understanding to a best in class understanding of all customers.