[This post is also available in Spanish]
Automatic sentiment analysis is one more step towards translating human emotions into data. The immediacy and spontaneity of opinions in the social media allow for these sentiments to be more meaningful and to preserve their emotional content.
Sentiment analysis of non-structured data, which is also known as opinion mining, adds more features to automatic extraction of topics, entities and concepts: polarity identification (is the expression positive or negative?), intensity (how strong are emotions?), and subjectivity (is the source biased or objective?). The precision of sentiment analysis, and how challenging it is, have been topics discussed in this blog before. We have also mentioned how important semantics is throughout the process. Now we are dealing with some new perspectives and applications.
Social media analysis has stimulated the interest in sentiment analysis. However, we must note that opinion mining is not solely related to the social media. Analyzable information comes from sources which are external and autonomous (social networks, blogs/microblogs, online forums, the mass media…), but also internal or company-owned (interactions stored in the CRM, transcriptions of conversations registered in the incident management system, surveys of customers or employees…).
Another interesting question is the degree of formality and accuracy of the language (grammar and lexicon) in which the contents to analyze are expressed. It ranges from a careful and skilful use of language, which is common among conventional media, to the Twitter’s jargons full of ambiguities and abbreviations. This feature has an enormous influence on the precision and results from the automatic analysis performed.
The most widespread criticism of these techniques is that irony and sarcasm are impossible to handle by automatic analysis, and no doubt it is sometimes a difficult task even for the brain of ordinary people. It may explain why irony or sarcasm, compared to strong criticism or insults, are not so frequently employed.
According to the application, a sentiment analysis tool can incorporate rules for identifying expressions containing sarcasm. Next it can rule them out, as long as this does not produce a biased result, or leave them aside so as to be handled by human experts. This may be possible provided that our company is not affiliated to the Sarcasm Society. If so, we should better master this type of communication (by the way, the Society’s slogan is “We would LOVE to hear what you think”).
Not long ago, the outcome of the analysis was a single score for the sentiment classification at a document-level (post, tweet…). This value is obtained by adding/subtracting the scores associated with different positive/negative expressions from the text. However, it can lead to an aggregate value which is neutral and results from different extreme and opposite polarities on different aspects. This fact hides very interesting information about the strengths and weaknesses of our company or products.
To avoid that, the latest techniques pursue the identification of those targets on which opinions have been expressed, such as entities and its components and attributes. This is a feature-based sentiment analysis, where information is more fine-grained and exploitable.
Sentiment analysis is a field in constant growth. Many emerging scenarios show the applicability of these techniques and how helpful they are in producing specific results:
- Find out which are the weak (or strong) points within the different areas of the products, services and brands of your company. Act immediately and find solutions. Sentiment analysis is performed in order to discover which opinions are truly negative or positive. These opinions are expressed by opinion leaders and current (or potential) customers, and recorded both inside (i.e. incident management systems) and outside the company (i.e. online forums on product ratings and reviews).
- Prevent customer churn. What poses a risk of losing a present customer can be identified in the negative opinions which are interpreted as rejection. This information comes from both internal and external sources.
- Learn how you compare against your competitors. Evaluate the opinion on their brand/company/products. Compare it with that on your business on the basis of the aggregation of direct judgments and the analysis of comparative expressions from mainly external sources.
- Measure the level of satisfaction of the company’s employees and the labor climate. Open questions give the most valuable information in surveys conducted to listen to the “voice of the employee”. Within big companies, opinion processing allows for more efficient management of questionnaires (internal sources).
- Study the electorate’s opinion. Tracking the social media (external sources), political parties and other associations can learn about the people’s positions and trends, by listening to the “voice of the voter”.
- Predict a share price. The market sentiment is a common term within the financial investments vocabulary. Opinion mining over the mass media, social networks and online forums (external sources), together with the analytical treatment of internal data, provide us with predictors of the price of shares.
- Assess the impact on corporate reputation. Contrary to some analytical tool providers’ belief, reputation does not equal opinion. Corporate reputation is not assessed by adding polarities from the tweets mentioning the company during the last hour. Reputation is a multidimensional asset resulting from the accumulated effect of multiple interactions. Analyzing how a set of external opinions affects reputation is not a trivial issue and has to do with the different indicators and dimensions of reputation, and with the authority of the opinion issuer. We will discuss this further in our next post.
Capturing sentiments at the Point of Emotion
Finally, let’s reflect on how sentiment analysis is being applied to the social media. Traditionally, opinion analysis (for instance, market studies or political surveys) is based on recall. However, this approach is rather incomplete as people are required to remember past events outside the context where they were experimented.
On the contrary, the “Point of Emotion”, as A. Jeavons calls it here, is the common distinguishing feature for the majority of people who share online content, write posts and give opinions through the social media. In other words, they are emotionally invested in the product or experience about which they are opining. The immediacy and spontaneity of the social media, plus the permanent connection that cell phones offer, provide “micro-surveys” taking place where and when the experience is happening. In this way, opinion is expressed and published at the same time, and there is no delay in the middle.
Emotions are misleading depending on the context. But in order to know what consumers, influencers or voters really think, and to make predictions about their behavior, we need to find them at the Point of Emotion.
[Would you like to know how Semantic and Natural Language Processing Technologies allow analyzing and exploiting opinions, ideas... and other user-generated contents? Visit the Daedalus website and find out how we are helping both conventional and social media monitoring and analysis companies].