Third Run of the Automatic Minuting Shared Task - AutoMin @ SIGDIAL 2025

A great opportunity to assess LLMs' ability to summarize and retrieve information from long contexts

We propose AutoMin 2025, the third instance of bi-annual shared task on meeting summarization into structured meeting minutes. We build upon our past experience from two previous AutoMins and add a new challenge to facilitate personalized access: answering questions about meeting content.

As in the previous editions, the tasks are run in English and Czech while the new question-answering task will be run monolingually (English-only) and cross-lingually (Czech questions about English meetings).

This challenge is highly relevant for assessing LLMs' ability to summarize and retrieve information from long contexts, equivalent to the length of an hour-long meeting.

A dedicated workshop is scheduled for August 28th, 2025, as part of SIGDIAL 2025: https://2025.sigdial.org.

 

Challenge Website

https://ufal.github.io/automin-2025/ 

 

Tasks

We propose two main tasks.

  • Task A (minuting): The task consists of automatically creating minutes from multiparty meeting transcripts. The generated minute would be evaluated both via automatic and manual metrics.

  • Task B (question answering): The task consists in answering questions based on meeting transcripts. For this challenge, participants will be tasked with answering these questions either monolingually (English questions on English transcripts) or cross-lingually (Czech questions on English transcripts). Answers will be evaluated using a LLM-as-a-judge metric.


Participant registration

All participants should register using the following link


Important Dates

Registration and CfP

From March 1st, 2025

Training and Dev Data Available

March 1st, 2025

Test Data Available

April 15th, 2025

System Run and Output Submission

April 29th, 2025

System Paper Submission

May 5th, 2025

Result Announcement

May 22d, 2025

Acceptance Notification of System Papers

June 2025 (TBA)

Camera Ready for System Papers

TBA

Event Date

August 28th, 2025

 

Publication

All teams will be required to submit a brief technical report describing their method. Details will be given on the challenge website soon.

 

Organizers

Laurent Besacier, Naver Labs Europe, France

Ondřej Bojar, UFAL, Charles University, Czech Republic

Tirthankar Ghosal, Oak Ridge National Laboratory, USA

Marie Hledıkova, UFAL, Charles University, Czech Republic

Thibaut Thonet, Naver Labs Europe, France


Contact

The organizers can be contacted via the following email address: automin@ufal.mff.cuni.cz