Third Run of the Automatic Minuting Shared Task - AutoMin @ SIGDIAL 2025 A great opportunity to assess LLMs' ability to summarize and retrieve information from long contexts We propose AutoMin 2025, the third instance of bi-annual shared task on meeting summarization into structured meeting minutes. We build upon our past experience from two previous AutoMins and add a new challenge to facilitate personalized access: answering questions about meeting content. As in the previous editions, the tasks are run in English and Czech while the new question-answering task will be run monolingually (English-only) and cross-lingually (Czech questions about English meetings). This challenge is highly relevant for assessing LLMs' ability to summarize and retrieve information from long contexts, equivalent to the length of an hour-long meeting. A dedicated workshop is scheduled for August 28th, 2025, as part of SIGDIAL 2025: https://2025.sigdial.org https://2025.sigdial.org/.
Challenge Website https://ufal.github.io/automin-2025/
Tasks We propose two main tasks. Task A (minuting): The task consists of automatically creating minutes from multiparty meeting transcripts. The generated minute would be evaluated both via automatic and manual metrics. Task B (question answering): The task consists in answering questions based on meeting transcripts. For this challenge, participants will be tasked with answering these questions either monolingually (English questions on English transcripts) or cross-lingually (Czech questions on English transcripts). Answers will be evaluated using a LLM-as-a-judge metric.
Participant registration All participants should register using the following link https://docs.google.com/forms/d/e/1FAIpQLSeMTa2DkXtZCMbBpTo5c853nOr3yq2N1dXExPim0sMZlWyzKQ/viewform
Important Dates Registration and CfP From March 1st, 2025 Training and Dev Data Available March 1st, 2025 Test Data Available April 15th, 2025 System Run and Output Submission April 29th, 2025 System Paper Submission May 5th, 2025 Result Announcement May 22d, 2025 Acceptance Notification of System Papers June 2025 (TBA) Camera Ready for System Papers TBA Event Date August 28th, 2025
Publication All teams will be required to submit a brief technical report describing their method. Details will be given on the challenge website soon.
Organizers Laurent Besacier, Naver Labs Europe, France Ondřej Bojar, UFAL, Charles University, Czech Republic Tirthankar Ghosal, Oak Ridge National Laboratory, USA Marie Hledıkova, UFAL, Charles University, Czech Republic Thibaut Thonet, Naver Labs Europe, France
Contact The organizers can be contacted via the following email address: automin@ufal.mff.cuni.cz