Tai Xiang's Random Projects

We’ve reached a point in our technological continuum in which coffee makers are connected to WiFi, fridges have apps, and hairbrushes are Bluetooth connectable. With these tremendous innovations in how we utilize technology, it would be unnatural for massive companies and large-scale corporations to rely on bothersome surveys and questionnaires to aggregate user perceptions on a product or a practice. Enter sentiment analysis, a manner of data mining and user opinion aggregation that a company can perform without disturbance to the user experience and without the bothersome need for excess data processing.

Our opinions and thoughts on products and services are forever memorialized nowadays, showing themselves in our social media posts, our tweets, our pictures. This concept is apparent to the human eye with a quick use of the Twitter search feature. If I look up ‘In-n-out’ and scroll through some of the most recent tweets, I see pictures and text praising and pledging allegiance to this fast food chain: generally positive it seems. If I look up ‘Taco Bell’ and do the same, I see comparisons of the food to fecal matter, jokes relating Taco Bell and diarrhea, and much time spent on the toilet after eating Taco Bell: generally negative. From these two very limited data sets, I may extrapolate the idea that people have a more positive perception of In-n-out in comparison to Taco Bell. Sentiment analysis essentially does what I just did, although it works on massive amounts of usually social-media related data sets at a much quicker speed.

Sentiment analysis, in a broad sense, involves ascribing some sort of humanized perception or intent (positive, negative, neutral) to our digitized manners of communication. If little Johnny tweets that he “hates the new Lego helicopter set”, then perhaps a sentiment analysis performed on that tweet will reveal a ‘negative’ perception of Lego’s new product. If that same process is run on tens of thousands of tweets, then Lego can receive a broad sense of how their new product is seen by the general public without any sort of disturbance our irritation to their consumers.

Now perhaps Lego isn’t satisfied with only knowing little Johnny hated their product, they want a more in depth look as to what aspect little Johnny hated about the product. By applying certain keywords during sentiment analysis, the scope of data that is fed into the process is decreased, and analysis will yield results much more specific to that aspect. If a company such as Starbucks wants to understand how their consumers feel about the price of a large coffee, they retain that similar process of sentiment analysis, but only filter in tweets that contain messages relating to price. If the majority of related outputs are negative, perhaps they may choose to decrease their prices.

This all seems like a lovely magical process, but how does an unfeeling block of metal and plastic understand human intent? Huge data sets and lots of training. To build a sentiment analysis model, one must first feed it lots of pre-labeled data to ensure it can receive a sense of how this weird human language thing all works. Here is a simplified approach: Suppose I take the sentences “I love my computer so much”, “This computer is ok” and “This absolute piece of garbage substituting for a computer is terrible”: I assign labels to each of the important words in the sentence and ascribe a certain value that corresponds to how the author feels. Lets say we give “absolute piece of garbage” a value of 0, “ok” a value of 2, and “love” a value of 4 (with 0 being negative, 2 being neutral, and 4 being positive): through these pre-labeled words and phrases, we gently guide the computer into understanding what human language is representative of. By doing this with millions of words and sentences, the trained model is gradually able to learn and understand how humans feel based off their words and statements, albeit in a mechanical manner. When we feed it a new tweet it has never seen before, it is able to draw from all the other million we pre-labeled and spit out a value of 0, 2, or 4, informing us of the perception of the tweet.

Now of course this fancy technique is not without its flaws. Human language is a complex and convoluted thing full of odd intricacies that a computer simply cannot understand. Sarcasm is especially hard to reconcile: If I say “My computer showed up at my door with three bullet holes and no screen, what a lovely company”, when my model takes in this tweet, it might just see “what a lovely company” and decide this intent is positive. Ambiguity in wording is also difficulty to process, with words meaning so many different things in different contexts. If I say “this food tastes funny” and “this show is funny” the word “funny” is drastically altered in regards to positive and negative perceptions. Once again, it is very difficult for a model to remedy this.

These issues still need to be ironed out, but they do not change the fact that sentiment analysis is a powerful tool. Despite its popularity and widespread use, this technology is still in a relatively infantile state. There is much work being done to improve the accuracy and speed of execution, with researchers already finding success with bypassing sources of errors with tools such as deep learning and improved algorithms. With an average individual outputting so much data throughout their lifespan and the accessibility of sentiment analysis data sets and libraries, there is seemingly endless potential for how this technology is used – whether it be for further extraction of profit for a massive corporation or a guy in a basement simply trying to see how people feel about the latest single off an upcoming album.