CfP : Evaluation lab for generative language model quality - Topical quiz generation shared task at CLEF 2026
We hereby invite you to participate in the shared task on Topical Quiz Generation, one of the four tasks of the ELOQUENT track at CLEF.
Can your generative language model, given a textual stimulus, generate relevant, diverse, and pedagogically meaningful quiz questions and answers?
The task focuses on generating Question–Answer items inspired by the PISA framework, targeting students aged 10–15 The goal is simple: produce questions that meaningfully engage with the text and reflect different levels of understanding.
This is an open and accessible task: use your favorite model, improve prompting strategies, and submit multiple experimental variations to explore what improves your model’s performance.
Submit multiple experimental variations to discover what works best! Submissions will be evaluated in cooperation with the OECD, organiser of the PISA test.
And there is more! There are three other tasks in ELOQUENT!
- Topical quiz question scoring: can your generative language model score responses to topical quiz questions (such as the ones generated in this task)?
- Voight-Kampff: does your language model produce human-like text? Can it fool a classifier? Generate text to be matched against human-authored texts in a classification experiment! (You may also submit classifiers through the PAN lab, also at CLEF)
- Cultural Robustness and Diversity: how does your language model take cultural context into account? Will it give the same response in every language?
Details ======= Find out more: [ https://eloquent-lab.github.io/ | https://eloquent-lab.github.io/ ] Register for the tasks: Contact: [ mailto:eloquent-clef2026-organizers@googlegroups.com | eloquent-clef2026-organizers@googlegroups.com ]
Important Dates ===============
2026, April 23: Registration closes - register now if you haven't already! 2026, May 07: Experiment submission deadline 2026, May 20: Participant paper submission 2026, June 2: Notification of acceptance 2026, June 23: Paper Camera Ready Submission 2026, Sept 21-24: CLEF Conference in Jena, Germany with Eloquent Workshop
Organisers of the Topical quiz generation task
====================================================
Sarah Bouaraba, Said Ettejjari, Diandra Fabre, Lorraine Goeuriot, Jussi Karlgren, Philippe Mulhem, Mario Piacentini, Didier Schwab, Katherina Thomas, Markarit Vartampetian, Luis Francisco Vargas Madriz