There is an abundance of health-related data on social networks, including chatter about therapies for health conditions. These therapies include but are not limited to medication, behavioral, and physical therapies. Social media subscribers who discuss such therapies often express their sentiments associated with the therapies. In this task, the focus will be to build a system that can automatically classify the sentiment associated with a therapy into one of three classes—positive, negative, and neutral. The annotated dataset for this task has been drawn from multiple preidentified Twitter cohorts (chronic pain, substance use disorder, migraine, chronic stress, long-COVID, and intimate partner violence). Thus, there is a high possibility that the therapies are being mentioned by people who are actually receiving/consuming them. The dataset consists of English Tweets containing mentions of a variety of therapies manually labeled as positive, negative, or neutral with the following approximate distribution: 50%, 15%, and 35%, respectively. The evaluation metric for this task is the micro-averaged F1-score over all 3 classes. The data include annotated collections of posts on Twitter which will be shared in csv files. There are 4 fields in the csv files: tweet_id, therapy, text, label. The training data is already prepared and will be available to the teams registering to participate. The testing data will be released when the evaluation phase starts.
- Training data: TBA
- Validation data: TBA
- Testing data: TBA
- Evaluation metric: micro-averaged F1-score
Contact: Yuting Guo, Emory University, USA (yuting.guo@emory.edu)
Data Examples:
| tweet_id | therapy | text | label |
| 15309 | meditation | Did you know meditation can be one of the most rewarding important things you do in your life? Did you also know it’s impossible to not be able to meditate? For people that believe your mind must somehow go blank you’re wrong unless you’re dead. | positive |
| 15262 | acupuncture | abt to get acupuncture for my migraines for the first time ever & i am terrified | neutral |
Submission Format
Please use the format below for submission. Submissions should contain tweet_id and label separated by tabspace in the same order as below.
| tweet_id | label |
| 15309 | positive |
| 15262 | negative |