Third Run of the Automatic Minuting Shared Task - AutoMin @ SIGDIAL 2025
A great opportunity to assess LLMs' ability to summarize and retrieve information from long contexts
We propose AutoMin 2025, the third instance of bi-annual shared task on meeting summarization into structured meeting minutes. We build upon our past experience from two previous AutoMins and add a new challenge to facilitate personalized access: answering questions about meeting content.
As in the previous editions, the tasks are run in English and Czech while the new question-answering task will be run monolingually (English-only) and cross-lingually (Czech questions about English meetings).
This challenge is highly relevant for assessing LLMs' ability to summarize and retrieve information from long contexts, equivalent to the length of an hour-long meeting.
A dedicated workshop is scheduled for August 28th, 2025, as part of SIGDIAL 2025: https://2025.sigdial.org.
Challenge Website
https://ufal.github.io/automin-2025/
Tasks
We propose two main tasks.
Task A (minuting): The task consists of automatically creating minutes from multiparty meeting transcripts. The generated minute would be evaluated both via automatic and manual metrics.
Task B (question answering): The task consists in answering questions based on meeting transcripts. For this challenge, participants will be tasked with answering these questions either monolingually (English questions on English transcripts) or cross-lingually (Czech questions on English transcripts). Answers will be evaluated using a LLM-as-a-judge metric.
Participant registration
All participants should register using the following link
Important Dates
Publication
All teams will be required to submit a brief technical report describing their method. Details will be given on the challenge website soon.
Organizers
Laurent Besacier, Naver Labs Europe, France
Ondřej Bojar, UFAL, Charles University, Czech Republic
Tirthankar Ghosal, Oak Ridge National Laboratory, USA
Marie Hledıkova, UFAL, Charles University, Czech Republic
Thibaut Thonet, Naver Labs Europe, France
Contact
The organizers can be contacted via the following email address: automin@ufal.mff.cuni.cz