SMASH Research Group

Datasets & Tools

Datasets, tools, and tutorial material from SMASH’s research over the years — kept here for anyone who still needs them, even though they’re not all from recent projects.

HaHackathon (SemEval-2021 Task 7)

Linking Humor and Offense Across Different Age Groups

Humor, like most figurative language, poses interesting linguistic challenges to NLP, due to its emphasis on multiple word senses, cultural knowledge, and pragmatic competence. Humor appreciation is also a highly subjective phenomenon, with age, gender and socio-economic status known to have an impact on the perception of a joke. In this task, we collected labels and ratings from a balanced set of age groups from 18-70. Our annotators also represented a variety of genders, political stances and income levels.

Tasks

  • Task 1 emulates previous humor detection tasks in which all ratings were averaged to provide mean classification and rating scores.
    • Task 1a: predict if the text would be considered humorous (for an average user). This is a binary task.
    • Task 1b: if the text is classed as humorous, predict how humorous it is (for an average user). The values vary between 0 and 5.
    • Task 1c: if the text is classed as humorous, predict if the humor rating would be considered controversial, i.e. the variance of the rating between annotators is higher than the median. This is a binary task.
  • Task 2 aims to predict how offensive a text would be (for an average user) with values between 0 and 5.
    • Task 2a: predict how generally offensive a text is for users. This score was calculated regardless of whether the text is classed as humorous or offensive overall.

Organizers

Join our mailing list: hahackathon@googlegroups.com

View the results · Download the data

Mazajak — an online Arabic sentiment analyser

Sentiment analysis is one of the most useful natural language processing applications. There are many papers and systems addressing this task, but most of the work is focused on English. Therefore, we present Mazajak, an online system for Arabic sentiment analysis. The system is based on a deep learning model, which achieves state-of-the-art results on many Arabic dialect datasets including SemEval 2017 and ASTD. The system provides three-way sentiment classification into one of the classes (Positive, Negative, Neutral).

Mazajak provides many features such as sentiment analysis for a sentence, a file, or you can submit a Twitter account and get an analysis of the user, plus an online API.

Mazajak was created by Ibrahim Abu Farha and Dr. Walid Magdy at the ILCC, part of the School of Informatics, the University of Edinburgh. This project was funded by The Alan Turing Institute, UK. Please cite: Mazajak: An Online Arabic Sentiment Analyser. Ibrahim Abu Farha and Walid Magdy. In Proceedings of the Fourth Arabic Natural Language Processing Workshop (WANLP). 2019.

Access the online tool

Urban Dictionary Embeddings

This resource contains a set of English-language word embeddings trained on the entirety of Urban Dictionary (urbandictionary.com) as of October 16, 2019. All terms, definitions, examples, and tags were treated as running text and embeddings were learned using the fasttext framework (fasttext.cc) with a window size of 5, a negative sampling rate of 10, and a word-level dimensionality of 300.

These embeddings perform competitively on a range of word-level semantics tasks, and were also useful initializations for classifiers trained for sentiment and sarcasm detection. If you are working in a domain that uses a high degree of slang or nonstandard English and want representations that better capture slang meanings, give them a try.

If you use this resource, please cite: Wilson, S. R., Magdy, W., McGillivray, B., Garimella, K., & Tyson, G. Urban Dictionary Embeddings for Slang NLP Applications. In Proceedings of the Language Resources and Evaluation Conference (LREC). Marseille, France, May 2020.

Download the embeddings (4.1 GB)

Tutorial: Detection and Characterization of Stance on Social Media

Stance detection involves identifying the position of a piece of text or a user towards a target such as a topic, entity, or claim. A growing body of research in the ICWSM and Social Computing community on performing and using stance detection shows its importance for a variety of applications, including properly analyzing the attitudes of online users.

This tutorial teaches participants how to perform and use stance detection: a general introduction to the concept of stance and how it differs from sentiment analysis; recent methodologies for stance detection on social media including supervised, semi-supervised, and unsupervised methods; and various applications of stance detection on social media, concluding with open challenges and future directions.

Organizers

  • Abeer AlDayel — PhD student at the School of Informatics, University of Edinburgh. Her work is on stance detection on social media. Website: abeeraldayel.github.io
  • Kareem Darwish — Principal scientist at Qatar Computing Research Institute, HBKU University. Website: kareemdarwish.com
  • Walid Magdy — Associate professor at the School of Informatics, University of Edinburgh, and faculty fellow at the Alan Turing Institute. Website: homepages.inf.ed.ac.uk/wmagdy

Slides

Other materials