Sentiment analysis is one of the oldest tasks in natural language processing. In this note we will introduce some examples and terminology, some key problems in the field and a simple model that we can understand by just knowing Backpropagation Log Linear Models and the Softmax Function.
We say:
- Polarity: the orientation of the sentiment.
- Subjectivity: if it expresses personal feelings.
See demo
Some applications:
Businesses use sentiment analysis to understand if users are happy or not with their product. It’s linked to revenue: if the reviews are good, usually you make more money. But companies can’t read every review, so they want automatic methods.
Other applications are opinion mining: if you like a politician or not. Spam detection. Recommender systems.
A simple solution: MLP
Traditionally you define a set of features and then try to log linear model to get a result. This is a complicated pipeline to get this result.
A surprising result from Iyyer (2015) says we just need a bag of words model and an MLP to get good result. Over 90% accuracy they say!.
With this solution you just:
- Pool the embeddings of the input sequence tokens.
- Pass this to MLP layers
- Softmax at the end. Currently, we can have the same thing asking ChatGPT and it also works quite well. (Prof. Cotterell says it could have been trained on this kind of tasks).
With 2-3 layers it works the best! And it seems the network is very sentitive to the sentiment of the kind of word that is used e.g. (okay, good, terrible etc…).