Computational Humor: What’s At Stake

With recent advance in machine learning and data science, more and more fields that were once sealed off from computational approaches are opening up for the touch of artificial intelligence. Humor is one such field. Notoriously difficult to analyze, assess and rate even for the human agent, it provides a challenge for the adventurous computer scientists who dare to take it on. Literature that provides analysis of the roots and causes of humor has been circulating at least since ancient Greece, with the following major theories presented to date[10]:

1. Superiority theory. The superiority theory of humor states that the feeling of self containment associated with a sense of superiority in our status or well being over others is what makes us laugh. In other words, we laugh when we are better off than a fellow human, and our assertion of this fact. Notable philosophers proposed and bolstered this assumption, an act which did not help the status of comedians and humor: they were deemed as agents of evil mockery and pride. Among the famous supporters of this view were Plato, Descartes, Hobbes and the Bible.

2. Incongruity theory. Scholars endorsed a more modern and positive approach to humor was endorsed mostly since the renaissance. This reading of humor claims that what creates a comedic effect is a “benign violation of expectation”[8]. Recent research have shown that many types of laugh outbursts could be explained by their surprising nature. Whenever an unexpected event happened, that turned out to be harmless – amusement was afoot. Two conditions had to be met to create humor:
(a) an incongruity, which first took an audience by surprise, perhaps even a tense, fearful one.
(b) a resolution of the contradiction of expectation had to happen, followed by relief, which had a vocal manifestation, of the form of “Ha Ha”, accompanied with an upward twitch of the lips.

Among the supporters of this understanding are James Beattie, Immanuel Kant, Arthur Schopenhauer and Søren Kierkegaard.

3. Stress release theory. Along the lines of the relief theme proposed by the “incongruists”, Sigmond Freud proposed a similar reading into humor, with a typical Freudian twist. To Freud, humor was a result of pent up stresses, many of whom related to societally taboo subjects, such as sexuality and violence, that are released when a joke teller refers to them directly or as an innuendo. By “opening for a conversation” a subject the listener has been putting active effort to suppress, a joke can signify a promise of relief, an appeasing statement, an “it’s OK” signal. It tells the listener oth- ers are contemplating and suppressing the same topics, and offers an opportunity to vent some of these accumulated suppressions. Lord Shaftesbury, Herbert Spencer and John Dewey also adopted a biological-psychological reading into humor as a stress reliever.

4. Mock-A ression play theory. From a biological-evolutionary point of view, laughter can be explained by its predecessors in primates. Observations by evolutionary biologists revealed that primates laugh too, usually during mock-aggression activity within a family of chimpanzees. Mock-aggression activity can be seen when members of the same family of monkeys are engaged in playing, that to an outside observer might look like a violent confrontation. When two or more such primates are “fighting” in this way, like children who are engaged in a mock-fight, they seem to be train- ing for actual potential future conflicts, yet need to signify to each other that they are not being truly aggressive. The chosen signals are usually a quick, fast breath of air coming from the diaphragm and a movement of the mouth, in which the front teeth are exposed and the lips are drawn back and upwards. The sound this mock- aggressive signal is making is “ah ah”, which resembles human laughter, only in a backward direction: primates’ diaphragm is set up a little differently than that of humans (or rather the other way around?), to allow for appropriate locomotion. Thus, primates breathe in instead of out, when poking fun at their fellow primates.

Computational Humor

A computational approach to humor is a much more recent development, yet it has already produced its own modest his- tory. Beginning in the 1990s, computer scientists attempted at understanding, and moreover producing humor automatically. Developments in machine learning made it possible to imagine an algorithm that could take in examples of humor and to try uncover their inner pattern. HAHAcronym[14] was one of the first examples of the development of a humor-generation focused system, which aimed to produce humorous acronyms. In the mid 2000s, Twitter proved to be a useful platform for such investigations, due to its ease of access and its inclusion of ready-made initial classifications (e.g. hash-tags). Barbieri and Saggion utilized the popular social network to detect irony[3], while Yishay Raz focused on automatic classification of types of humor[12], such as anecdotes, fantasy, insult, irony, etc. Some of these computational approaches referenced the voluminous traditional thinking of humor, presented above[2]. There was also much work done for the understanding and assessment of visual humor. In 2015, Shahaf et al. joined Bob Mankoff, cartoon editor of the New Yorker, to build a system that would be able to predict which one of the +5,000 weekly submissions to the news- paper’s cartoon caption competition is the funniest one[13]. These attempts and more have all been interesting and illuminating, but proved to have a limited amount of success. There is clearly much more to be done to achieve a more accurate understanding of what makes things funny and how we can “teach” humor to computers.

Increments to our knowledge of humor are gradual and modest, being a difficult task as it is, and yet there are more and more indications as to the potential of such investigations. In an attempt to add to the existing corpus of computational humor literature and experimentation, this thesis focuses on generating literal humor, through an examination of Google’s word2vec word embedding. In particular, we will examine humorous 4-word analogies generated based on the embedding’s representation of words.

Word Embeddings

For this literal approach to the composition of humorous 4-word analogies, we took advantage of the existing and relatively new technology of neural word embeddings, and in particular Google’s word2vec, which opened to public usage in 2013119. Using a neural net trained on instances of Google news articles containing 3 billion running words, Google’s machine learning engineers were able to create an embedding of 3 billion words mapped into vec- tors of 300 dimensions. The vectors are learned representations of the words in the training text corpus, which were crafted using continuous bag-of-words and skip-gram architectures. A neural net is trained to predict a word for the words that appear close by in the text, and the parameters learned by it are used as these word representations. In fact, a semantic field is created, such that words that tend to appear closer across the training texts appear closer to each other in this multidimensional space as well. The resulting vectors thus capture relations between words in the underlying training data, such as which words are similar to each other (and thus are ’neighbors’ in the semantic space) and allow us to complete analogies that capture the relations between pairs of words (such as Paris:France::Rome:Italy). These resulting vectors can therefore be used as features when training models in various natural language processing and machine learning applications. For example, through word2vec, Bolukbasi et al. uncovered gender biases in the underlying embedding6. In this study, analogies such as Man : Computer programmer :: Woman : homemaker were automatically created after taking the cross product and Euclidean distance measures of vector representation of single words, pairs and quadruples. We decided to utilize this approach to study the humorous nature of association of words. If word embeddings can uncover gender biases, why can they not uncover funniness of words and combinations of phrases?

Linear regression and SVM classification

This paper had three main goals: to generate humorous analogies, to predict ratings of humorous analogies and to perform a Turing test to assess our results. In order to achieve them, it focuses on binary classification of words and analogies into “funny” and “not funny” categories, and on linear regression to generate prediction of “funniness” scores, based on ratings given by Amazon Mechanical Turk users. Following is a brief explanation of these two methods, meant to dispel the magical nature of machine learning as a “buzz word.”

SVM classification

Once we had our data organized as numerical values, that rep- resent features of different objects (provided by word2vec, in our case), we could train a classification model. Our approach required 4 different classifiers, which created somewhat of a “cascading” effect. SVM classifiers fit a linear classification separator between groups of data that have certain labels. In most tasks of binary classifica- tion, they are used to separate positive from negative examples. Our case was no different – we tried to separate positive funny examples from unfunny examples. The separator is fitted such that the least data points will be misclassified (having the opposite label than the desired one). SVMs, or support vector machines, are unique in their definition of a decision boundary which has a desired margin. In a classification problem treated with an SVM, we look at the points that are closest to the hyperplane as support vector (hence, the name). The certainty of a classification of a data point can be determined by its distance from the hyperplane (the farthest it is, the more certain the classification).[5] Thus, the best possible hyperplane we could fit is the one that maximizes the distance of the closest points (or SVs) to the hyperplane. A basic hyperplane can be defined as




The goal is to fit the classifier with lowest loss rate, where labels of the classification itself will be described as 1 if h(x; w, w0) > 0 or -1 otherwise, and in our case as funny if h(x; w, w0) > 0 or unfunny if h(x; w, w0) < 0. When determining the weight vector w we will try to maximize the distance of the closest points, the support vec- tors, from the hyperplane, on both sides, while still maintaining a correct classification. To find the optimal hyperplane we will need to minimize the following expression:




Linear Regression (Ridge)

The regression task was done with ridge regression, where a linear function was fitted to predict the scores (y axis) to match a data point, represented by the numerical value associated with a 4-tuple. The following is the definition of the regression[1]:



And the loss function we aim to minimize is defined as:



Where X is our features matrix (aka the featurized data), w is the weight chosen for the production of the regression line, w0 is the included bias terms and λ is the ridge regularization parameter. h is therefore the function for a prediction generation. The generation and rating process discussed in this thesis uses these two ideas from machine learning theory[5].

The implementation was made possible through the python library scikit-learn7, which offers great support in putting machine learning theories into practice, and into actual predictions and classification models.

Contribution

In this project we furthered the understanding of humor and the capabilities of producing it “on demand”. There are various benefits to the development of artificial humor capabilities:

1. Allowing the creation of smoother, more fun interfaces to use, which will surely play an even greater role in our lives in the coming years. Systems which include humorous components could be more congenial: making queries, tasks and warnings less repetitive, statements of ignorance more acceptable and error mes- sages less patronizing[4].

2. Facilitating a better understanding of humor itself, by asserting or disproving notions of what makes things funny.

3. Overcoming yet another interesting artificial intelligence challenge posed to computer science researchers.

To the best of our knowledge, there has been no attempt to use our newly gained understanding of word embeddings in the field of computational humor. Furthermore, there has been limited success in previous attempts of humor generation tasks. This is yet another attempt at providing a proof of concept, for future research. This paper shows that computers can not only construct a humorous structure, but also recognize humorous themes relatively well, implying that it might have implications on a fuller understanding of what makes things funny.