Identifying personal mentions of COVID19 symptoms requires distinguishing personal mentions from other mentions such as symptoms reported by others and references to news articles or other sources. The classification medical symptoms from COVID-19 Twitter posts presents two key issues: First, there is plenty of discourse around news and scientific articles that describe medical symptoms. While this discourse is not related to any user in particular, it enhances the difficulty of identifying valuable user-reported information. Second, many users describe symptoms that other people experience, instead of their own, as they are usually caregivers or relatives of people presenting the symptoms. This makes the task of separating what the user is self-reporting particularly tricky, as the discourse is not only around personal experiences.
This task is considered a three-way classification task where the target classes are:
(1) self-reports,
(2) non-personal reports, and
(3) literature/news mentions.
- Training data: 9,567 tweets
- Test data: 6,500 tweets
Register your team here : https://forms.gle/1qs3rdNLDxAph88n6
After registration approval, you will be invited to join the Google group for the task. Link to the dataset is available in the Google groups banner. If you do not receive the invite please request to join the Google group with team name using the link below.
Google groups : https://groups.google.com/g/smm4h21-task-6
Link to Codalab : https://competitions.codalab.org/competitions/28766
Evaluation Period for Task 6 :
| Test Dataset Release | 28th Feb 2021 12:00am UTC |
| Predictions Due | 2nd Mar 2021 11:59pm UTC (3:59pm PST) |
Submission format: Please use the format below for submission. Submissions should contain tweet_id and label separated by tabspace in the same order as below. Predictions for each task should be contained in a single .tsv (tab separated values) file. This file (and only this file) should be compressed into a .zip file. Please upload this zip file as submission.
| tweet_id | label |
| 24556 | Lit-News_mentions |
| 31068 | Self_reports |
| 24621 | Lit-News_mentions |
| 57164 | Self_reports |
| 245678 | Nonpersonal_reports |
| 85662 | Nonpersonal_reports |
Evaluation Metric : Micro F1-score.
Contact information: Juan Banda (juan@jmbanda.com)