RetroEval 2026: Symposium on Natural Language Generation Evaluations ============================================ 1-2 June 2026, Aberdeen, Scotland, United Kingdom https://retroeval.github.io/ ============================================
Evaluation in the field of Natural Language Generation (NLG) has changed considerably over the past several decades. This special symposium in honour of Prof. Ehud Reiter’s retirement provides a forum for academic and industry researchers to look back on the topic of how evaluations in the field of NLG have changed and to explore unaddressed challenges. The two day symposium will be held in-person at the Sir Duncan Rice Library in the historic University of Aberdeen, June 1-2, 2026. For this symposium, we welcome submissions of long papers, short papers, and extended abstracts.
*** Workshop Theme ***
Ehud Reiter has been a leading light in Natural Language Generation (NLG) research throughout the four decades he worked in this area, in both academia (Aberdeen) and industry (CoGenTech; Arria NLG). Ehud’s influence extends to all aspects of NLG, but the areas in which it has arguably been the strongest is evaluation of NLG systems. On the occasion of his retirement, this workshop, which is held in his honour, will therefore focus on evaluation of NLG systems, highlighting in particular some of the topics that Ehud has tended to emphasize (see below) such as the importance of reproducibility and the risks of data contamination.
*** Topics of Interest ***
Topics of interest include (but are not limited to) the following:
-- The aims of NLG and NLG evaluation -- Intrinsic versus extrinsic NLG evaluation -- Evaluation of NLG systems in the real world -- Impact assessment of NLG systems and LLMs -- New evaluation challenges arising from the use of LLMs -- Hallucination annotation and its role in NLG evaluation -- Statistical analysis for NLG evaluations -- Data contamination in NLG evaluation -- LLMs as evaluators: opportunities and pitfalls -- The role of LLMs in the development of evaluation metrics -- Reproduction and reproducibility of human evaluation experiments -- Publication bias: What to do with negative results? -- Pre-publication of research hypotheses in NLG evaluation -- NLG evaluation versus psycholinguistic experimentation: what can we learn from each other? -- Disciplinary cultures and evaluation methods -- Evaluating NLG systems/LLMs for assistive technology
*** Submission Types ***
The workshop accepts the following submission types: • Long Papers (archival) • Short Papers (archival) • Extended Abstracts (non-archival)
Accepted contributions will be presented as oral or poster presentations.
*** Archival Submissions ***
• Long papers: • Up to 8 pages (excluding references) • Unlimited references • Up to 2 appendix pages • 1 additional page in the final version to address reviewer comments • Short papers: • Up to 4 pages (excluding references) • Unlimited references • Up to 1 appendix page • 1 additional page in the final version for reviewer comments
*** Non-Archival Submissions ***
• Extended abstracts: • Up to 2 pages including references • 1 additional appendix page for tables/figures • Selection based on the symposium fit
*** Submission Format ***
• Two-column ACL 2026 format • LaTeX template only • PDF submissions only • Submissions via OpenReview
*** Important Dates ***
Note: All deadlines are 23:59 UTC-12.
• ARR commitment deadline (archival): 16 March, 2026 • Direct paper submission deadline (archival): 24 April, 2026 • Direct paper submission deadline (non-archival): 1 May, 2026 • Notification of acceptance: 8 May, 2026 • Camera-ready deadline: 22 May, 2026 • Symposium dates: 1-2 June, 2026
*** Review Policy ***
Long and short papers will follow ACL double-blind review policies. Submissions must be anonymized, including self-references and links. Papers violating anonymity requirements will be rejected without review. Demo descriptions are exempt from anonymization.
Contact and Information
• Website: https://retroeval.github.io/ • Email: retroeval@googlegroups.com mailto:retroeval@googlegroups.com
Workshop Organisers: Saad Mahamood (Shopware), David Howcroft (University of Aberdeen), Kees van Deemter (Utrecht University), Albert Gatt (Utrecht University), Simone Balloccu (TU Darmstadt), Margaret Mitchell (Hugging Face), Alberto Bugarín Diz (CiTIUS & University of Santiago de Compostela), Jose María Alonso-Moral (CiTIUS & University of Santiago de Compostela), Adarsa Sivaprasad (University of Aberdeen), Chenghua Lin (Manchester University), and Alexandra Johnstone (University of Aberdeen).