Task 6 : Classification of COVID19 tweets containing symptoms

Identifying personal mentions of COVID19 symptoms requires distinguishing personal mentions from other mentions such as symptoms reported by others and references to news articles or other sources. The classification medical symptoms from COVID-19 Twitter posts presents two key issues: First, there is plenty of discourse around news and scientific articles that describe medical symptoms. While this discourse is not related to any user in particular, it enhances the difficulty of identifying valuable user-reported information. Second, many users describe symptoms that other people experience, instead of their own, as they are usually caregivers or relatives of people presenting the symptoms. This makes the task of separating what the user is self-reporting particularly tricky, as the discourse is not only around personal experiences. 

This task is considered a three-way classification task where the target classes are:
(1) self-reports,
(2) non-personal reports, and
(3) literature/news mentions.

  • Training data: 9,567 tweets
  • Test data: 6,500 tweets

Register your team here : https://forms.gle/1qs3rdNLDxAph88n6
After registration approval, you will be invited to join the Google group for the task. Link to the dataset is available in the Google groups banner. If you do not receive the invite please request to join the Google group with team name using the link below.
Google groups : https://groups.google.com/g/smm4h21-task-6
Link to Codalab : https://competitions.codalab.org/competitions/28766

Evaluation Period for Task 6 :

Test Dataset Release28th Feb 2021 12:00am UTC
Predictions Due2nd Mar 2021 11:59pm UTC (3:59pm PST)
All submissions are automated and time limits are enforced by Codalab. No extensions will be provided.

Annotation examples

Submission format: Please use the format below for submission. Submissions should contain tweet_id and label separated by tabspace in the same order as below. Predictions for each task should be contained in a single .tsv (tab separated values) file. This file (and only this file) should be compressed into a .zip file. Please upload this zip file as submission.

tweet_idlabel
24556Lit-News_mentions
31068Self_reports
24621Lit-News_mentions
57164Self_reports
245678Nonpersonal_reports
85662Nonpersonal_reports
Submission format for Task 6

Evaluation Metric : Micro F1-score. 

Contact information: Juan Banda (juan@jmbanda.com)