Long Short Term Memory Networks for Sentiment Analysis
"Hope you are having an excellent day!" - Positive Sentiment
WHAT IS SENTIMENT ANALYSIS?
If you have ever bought a product on an E-commerce site, or read about movie reviews for an upcoming movie, or thought about what people think of a brand or even a twitter post, you already know what sentiment analysis is about.
A sentiment tells us about what a person 'feels' towards something. This sentiment is usually expressed in the form of textual data.
Sentiment analysis deals with understanding and trying to model the sentiment of a person given the piece of text written by him/her. In other words, sentiment analysers try to predict how a person 'feels' with respect to something.
Is it possible to predict how a person feels towards a product given his/her review? Yes!
SEQUENCE MODELLING
Consider this piece of text: "Dog bites man" This sentence is written in a particular sequence and if written in a different sequence would mean something completely different: "Man bites dog." Thus we can see that not only the words but the 'sequence' of the words is also extremely important in our task of sentiment analysis.
The Fibonacci Sequence
Can we use traditional neural networks to do sentiment analysis? To a certain extent we can. These networks may be able to represent the words but they have a hard time dealing with the 'sequence'. For such tasks of sequence modelling we need to use different types of neural networks - Recurrent Neural Networks.
LEARNING ABOUT RNN's
In all my previous projects, I had never worked with sequence data. Hence for this task of sentiment analysis, I had to learn about sequence modelling. To do this, I took a free course on Coursera - https://www.coursera.org/learn/nlp-sequence-models?
This course enabled me with the right knowledge to start working with sequence models.
BUILDING THE SEQUENCE MODEL
Before starting to build the model, I first had to get some data on which the model would train. For this task I used a small portion(~9K reviews) of a massive dataset(~35 million reviews) The reason for this was the computational constraints while building the model. I hence extracted ~9K product reviews from Amazon's Electronics section of product reviews.
A Brief Snapshot from the Dataset
For training the model I used a Bidirectional LSTM(Long Short Term Memory) network with several Dropout layers to prevent overfitting. I also simplified the training process to output:
Negative Sentiment - for reviews with rating 1 and 2
Positive Sentiment - for reviews with rating 4 and 5
Neutral Sentiment - for reviews with rating 3
The overall model size was 4.5 million parameters! The model's final accuracy on the training set was ~95% and that on the test was ~80%.
TESTING THE MODEL
After training the model for several hours on Google Colab's free GPU, I wanted to test the model on some unseen data. I went to Amazon and searched for some product reviews for headphones.
Below are the results of the testing:
Text Input: 'Awesome awesome awesome...just awesome :)I m doubtful when I m booking.. Amazon delivery is very fast.,The sound quality is awesome for dis price 499/- very cheap price for awesome sound experience.,Very very soft headphone..I could not feel headphone in my head..:) that much soft..Before I used JBL 350 series it comes 799- that one broken..I just tried Sony.. doubtful! But now I m big fan of SONY!!!BECAUSE FOR DIS PRICE 499/- GIVING AWESOME SOUND QUALITY.. ALSO VERY SAFE FOR EARS..VERY SOFT AND SMOOTH..FOR THE PRICE RANGE.. I GIVEN DIS FEEDBACK!!'
As we can see the model outputs a positive sentiment
2. Text Input: 'Okay product. Sound quality is not great so I would have preferred some other product. But it is okay @₹500.'
As we can see the model outputs a neutral sentiment
3. Text Input: 'Of about fifty products that I have so far bought from Amazon over the last three years, this is the only item that I did not use for more than 2 weeks. This headphone is fake and did not work. I had to trash it. I lost my money.'
As we can see the model outputs a negative sentiment.
FURTHER STEPS
Obviously this model is not perfect and it also fails sometimes.
Text Input: 'Guys, guys do not buy this product its a scam this product things to start malfunctioning after a few days the voice clarity is not good you will not appreciate this headphone if you are a bass lover and problem is that the sounds are not clear'
The model outputs a neutral sentiment instead of negative sentiment. However, I think that given more training data and compute power, RNN's will be able to do even better sentiment analysis.
CONCLUDING THOUGHTS
It was an extremely fun journey of learning about sequence models to finally being able to implement one in the form of a sentiment analyser. I hope you enjoyed reading this blog as much as I did writing it and working on the project.
If you have a positive sentiment for this blog I hope you would like, comment and share it as well.