# Call for Participation

We are delighted to announce SemEval-2026 Task 5: Rating Plausibility of Word Senses in Ambiguous Sentences through Narrative Understanding.

Word Sense Disambiguation tasks commonly assume that only one word sense is "correct", but this does not always reflect how humans perceive meaning. Ambiguity, underspecification and subjective factors such as individual linguistic experience can influence which word senses are perceived as plausible in a given context. For these reasons, human intuitions and model predictions may frequently diverge.

This shared task addresses this challenge using the AmbiStory dataset. The dataset consists of five-sentence English short stories, each containing a lexically ambiguous sentence with human plausibility judgements. Surrounding context may provide clues or influence how plausible different word senses appear.

Example:

"The detectives followed the track." Depending on the given context, either the rail track or the piece of evidence sense may be perceived as plausible.

## Task

Given a short story with a lexically ambiguous sentence and a pair of candidate word senses, participating systems must predict the plausibility of each sense on a scale from 1 (implausible) to 5 (highly plausible).

Systems will be evaluated based on two metrics:  * Accuracy within standard deviation (SD): the proportion of predictions that fall within one standard deviation of the average human judgment.  * Ranked correlation: the correlation between model predictions and aggregated human judgments.

## Participation

Website and Data: https://nlu-lab.github.io/semeval.html Code: https://github.com/Janosch-Gehring/semeval26-05-scripts Registration and Submission: https://www.codabench.org/competitions/10877/

We invite researchers and students of all experience levels to participate in this task and help us disambiguate the world, story by story!

Do you have any questions? Please join our Google group: https://groups.google.com/g/semeval2026-task5-participants

## Important Dates

Development phase: now -- Jan 10, 2026 Evaluation phase: Jan 10 -- Jan 31, 2026

Feb 2026 (tentative): Shared task papers due Mar 2026 (tentative): Notification of acceptance Apr 2026 (tentative): Camera-ready submission Summer 2026: SemEval workshop

## Organizing Committee

Janosch Gehring, University of Technology Nuremberg Selina Meyer, University of Technology Nuremberg Michael Roth, University of Technology Nuremberg