Dear all
We’re excited to invite you to take part in AdabEval 2026, a shared task on politeness classification in Arabic social media posts, hosted at OSACT7 and co-located with LREC 2026.
The website for the shared task is: https://sites.google.com/view/adabeval2026/home
Description: Participants will develop systems to perform one of the following two subtasks, which investigate politeness in Arabic social media posts:
Politeness Classification (Subtask A): This subtask focuses on building and evaluating models that automatically assess the politeness level of Arabic social-media posts. Given a post’s text, participants must classify it into one of three categories: Polite, Neutral, or Impolite. Systems will be compared using accuracy as well as macro-averaged precision, recall, and F1-score on the test data.
Subtask A Summary - Input: Arabic post - Output: One of 3 predefined category labels (polite, neutral or impolite) - Metric: Accuracy, macro-averaged precision, recall, and F1-score
Category Prediction (Subtask B): In this subtask, the objective is to assess systems’ ability to identify multiple pragmatic functions in Arabic social-media posts. Each text may express one or more categories drawn from nine culturally grounded functions, including criticism, insult, respect, prayer, greeting, and hospitality. The task is formulated as a multi-label classification problem, requiring systems to predict all applicable categories for each instance. The official evaluation metric is the macro-averaged F1 score computed across the nine categories.
Subtask B Summary - Input: Arabic post - Output: All applicable categories for each instance. - Metric:Macro-averaged F1-score
Dataset Information: The dataset is an annotated corpus of Arabic social‑media posts. Each record contains an identifier, raw text, a primary label (polite, neutral or impolite), and up to three category-keyword pairs. Categories capture pragmatic functions such as Criticism, Insult, Disparagement, Prayers, Greetings, Admiration, Respect, Felicitation and Hospitality & generosity. Keywords highlight words or phrases that motivated the annotation.
EXAMPLE 1: --------------- Sentence: مبدعون ومميزين بطرحكم وضيوفكم. جزاكم الله خير الجزاء Source: YouTube Label: Polite Criteria 1: Appreciation & Love (الإعجاب والحب) — Keywords: مبدعون، مميزين Criteria 2: Thanks & Gratitude (الشكر والامتنان) — Keywords: جزاكم الله خير الجزاء
EXAMPLE 2: --------------- Sentence: من كان بتخيل دكتور معتز ومقالاته قبل ثوره ٢٥ يناير يتحول لحذاء في أيدي العسكر ويصبح مجرد دلدول ويتبع اُسلوب منحط ووضيع كده Source: Tweet Label: Impolite Criteria 1: Insult (شتيمة) — Keywords: منحط، وضيع
How to Participate? 1. Registration is required, please complete the registration formhttps://docs.google.com/forms/d/e/1FAIpQLSezmjuMBJBi6KHxk6hdEF5JQwX8w3TwdXq6VDV1dVsGJR8LLA/viewform. 2. Join the AdabEval2026 at slackhttps://adabeval2026.slack.com/?redir=%2Farchives%2FC09TZFH47FZ%3Fname%3DC09TZFH47FZ.
System Description Papers All participating teams are encouraged to submit a short system description paper. Papers will be included in the workshop proceedings and do not require high leaderboard ranking. We welcome creative approaches, analysis, and lessons learned.
Contact For questions or clarifications, please contact the organising team at adabeval2026@gmail.com. We look forward to your participation and contributions!
Best regards, The AdabEval 2026 Organizing Team
________________________________
Disclaimer: This communication is intended for the above named person and is confidential and / or legally privileged. Any opinion(s) expressed in this communication are not necessarily those of KSU (King Saud University). If it has come to you in error you must take no action based upon it, nor must you print it, copy it, forward it, or show it to anyone. Please delete and destroy the e-mail and any attachments and inform the sender immediately. Thank you. KSU is not responsible for the political, religious, racial or partisan opinion in any correspondence conducted by its domain users. Therefore, any such opinion expressed, whether explicitly or implicitly, in any said correspondence is not to be interpreted as that of KSU. KSU may monitor all incoming and outgoing e-mails in line with KSU business practice. Although KSU has taken steps to ensure that e-mails and attachments are free from any virus, we advise that, in keeping with best business practice, the recipient must ensure they are actually virus free.