Analyse Customer Sentiment on Zendesk Ticket Comments

Understanding Customer Attitudes

Understanding customer opinions, attitudes, and emotions is a very challenging aspect of contact centre management. Manually completing this task can be close to impossible when dealing with thousands of interactions daily. An ability to identify happy or frustrated customers can provide insight to guide decision making and lead to improvements in customer satisfaction and net promoter scores (NPS).

At EdgeTier, we use natural language processing (NLP) and “sentiment analysis” techniques to assist our clients in gaining insight into their customers, in particular on our WatchTower product. In this blog we are going to discuss sentiment analysis and perform some simple sentiment analysis on comments from tickets in a Zendesk view using Python, Zenpy and TextBlob. All code can be found on our Github.

What is Sentiment Analysis?

Sentiment analysis is a method of understanding positivity or negativity in text that can be used to gauge opinions or attitude to different topics. 

Customer sentiment analysis involves discovering customer attitudes towards products and brands. It can be performed on customer service data such as chats and emails to enable companies answer questions including: 

• What is making our customers happy or frustrated?
• How can we improve our customer service?
• What can we change to improve our product for customers?

By gaining these insights, companies can better understand what changes need to be made to improve aspects such as customer satisfaction and customer retention.

NLP and machine learning models automate the process of understanding and classifying text into sentiment categories – typically positive, negative or neutral. Recent advancements in text embeddings (representations of text as numbers stored in vectors) mean the semantic meaning of language and the relationship between words can be captured more effectively which has led to improved accuracy in NLP models. 

We are now ready to perform some simple sentiment analysis on comments from tickets in a Zendesk view using Python. Let’s get started!

Download Zendesk Comments from a View using Zenpy in Python

In Zendesk, tickets are the means in which customers communicate with agents. Views provide Zendesk users a way to organise or group tickets based on selected criteria. Each view is assigned an ID which we will need to get tickets for our analysis. The actual text sent by customers is stored in ticket comments which are attached to tickets. 

We are going to use the Zenpy library in Python to retrieve data from the Zendesk API that we can later use to analyse sentiment. Zenpy is a Python library developed by Facetoe that simplifies the process of accessing the Zendesk API using Python. 

To begin, we need to install the Zenpy library using pip and import the Zenpy module. We will then need to pass our Zendesk credentials to the Zenpy class to create a client which can be used to make API requests as in the code snippet below. To generate a Zendesk API token, follow the instructions here.

As we want to retrieve ticket comments for analysing sentiment, we first need to extract all tickets and then extract the comments attached to each ticket. When authenticated, we can retrieve the tickets from our view by passing our view ID to the .views.tickets() method. In the code snippet below we iterate through each ticket, then iterate through the comments in each ticket and append the ticket id and comment body to a list. We can convert this list to a Pandas DataFrame to make it easier to view. We can now see the data in our view and are ready to perform some sentiment analysis!

Customer Service Sentiment Analysis with TextBlob

To detect the sentiment of our ticket comments, we will use the TextBlob library in Python which can be installed using pip. TextBlob is a useful library that provides an API to help with various NLP tasks in Python. In TextBlob, the sentiment function returns polarity scores. A polarity score is a float that falls in the range of [-1,1]  where a score of 1 indicates positive text and a score of -1 indicates negative text. 

After installing and importing TextBlob, we can iterate through our dataframe of comments and use the TextBlob() function to get a polarity score for each comment.

We can see in our output the first sentence “I want to change this booking please” receives a polarity score of 0 indicating a neutral sentence. “This is great customer service” receives a score of 0.8 indicating positivity and “That is very frustrating” receives a score of -0.52 indicating negativity. 

In just a few lines of code, we now have a polarity score for all comments in our view that can help us get insights into our customers.

Customer Emotion Detection at EdgeTier

At EdgeTier, we tested multiple sentiment models on customer chats and emails with the aim of supporting our clients to better understand their customers. We quickly realised that the standard approaches to sentiment analysis are not the best option in the area of customer service. Often, certain phrases such as “I want to cancel” or “there is an error on the website” are considered to have negative sentiment, but this does not necessarily mean the customer’s attitude is negative. 

To overcome this, we began to focus on emotion detection, a branch of sentiment analysis used to detect human emotions such as happiness or sadness. Rather than focusing on a univariate score to evaluate customer attitudes, individual emotion scores provide a more complete view of the different aspects of interactions. Our data science team built a bespoke multilingual customer service emotion detection model using a transformer-based neural network architecture to detect three key emotions that provide a proxy to customer satisfaction: praise, gratitude and frustration.

This emotion system is embedded in our real-time alerting and exploration system, WatchTower.

Emotion detection is proving a lot more useful in the area of customer service than standard off-the-shelf models, since the phrases mentioned earlier such as “I want to cancel” are classified as neutral whereas a phrase like “I want to cancel this booking as I am really annoyed with your company” is classified as frustrated. The model is multilingual and can detect emotion in a broad range of languages without requiring translation technology.

The output of EdgeTier’s emotion models are highly correlated with actual customer satisfaction scores provided by customers. The more frustration detected in a customer service query, the lower the eventual customer satisfaction score, while the more gratitude and praise, the higher the score. Of course, a huge advantage of EdgeTier’s emotion detection models is that they can be run on every customer query, whereas in reality customers only fill in surveys less than 20% of the time. Emotion detection helps fill the gaps in customer understanding that often exist in customer service centres.

If you are interested in gaining insights on the emotions of the customers reaching out to your contact centre or would like to learn more about our approach to emotion detection and how it can help you, please get in touch with us.