ESSLI 2026 Workshop: Human Label Variation in Discourse and Pragmatic Phenomena Prague, August 10-14, 2026

Call for Papers

Workshop description: Over the past 20 years, the NLP community has given steadily-increasing attention to the idea of regarding human annotation disagreement not as a nuisance but as a potential asset that can be exploited for understanding the respective task better. In practice, this means that multiple annotations are not just adjudicated or averaged (and the individual annotations then thrown away) but taken as a spectrum that constitutes a "complex ground truth", for instance in the form of probability distributions over labels attached to items. While some of the pioneering work on appreciating human label variation (HLV) came from research in discourse tasks such as anaphoric reference and discourse structure, the great majority of the recent work in the HLV community has concentrated on tasks in which variation arises from subjective differences, such as humour or offensive language detection, whereas the study of variation in pragmatics and discourse has seen relatively little progress. This is on the one hand surprising - pragmatics and discourse are highly prone to interesting variation in human judgement - but may well be an effect of the relative complexity of the annotation tasks, which make treatments of HLV difficult. In this workshop, we aim to bring together experts on HLV and on discourse phenomena, and to identify ways forward. We welcome contributions on the following topics, for example (the list is not meant to be exhaustive): • Discourse phenomena that can benefit from incorporating HLV • Technical approaches to handling HLV (ideally with an eye on discourse): ◦ Soft-label vs. perspectivist approaches ◦ Implementation of machine learning regimes that handle HLV ◦ Evaluation of both automatic and manual annotation in the face of HLV ◦ Underspecification in representation formalism • Case studies

Submission guidelines: Abstracts should be at most two pages in 12pt font (plus up to two pages for references and appendix material). We primarily aim to promote participation and active discussion, and no proceedings will be published. Therefore, workshop submissions are not limited to unpublished work. We welcome proposals for both long (30 min. + discussion) and short (15 min. + discussion) presentations as well as poster presentations. Be sure to indicate your target type on the abstract. Subject to discussion at the end of the workshop, we envisage the possibility of a follow-up publication in the form of a journal special issue (with a new call for papers and reviewing process).

Submission deadline: April 24, 2026. Submission is via email (see below)

Workshop organizers: Massimo Poesio (Utrecht Univ. and Queen Mary University), Manfred Stede (Univ. of Potsdam)

Workshop website: https://hlv-dp.github.io

Contact (and submission) email: hlvdp@proton.me

Important dates: Deadline for abstract submission:  April 24, 2026 Notification of acceptance: May 8, 2026 Workshop dates: August 10-14, 2026 (one 90min slot per day)