Shared task on Multilingual Grammatical Error Correction (MultiGEC-2025)

We invite you to participate in the shared task on Multilingual Grammatical Error Correction, MultiGEC-2025, covering over 10 languages, including Czech, English, Estonian, German, Icelandic, Italian, Latvian, Slovene, Swedish and Ukrainian.

The results will be presented on March 5 (or 2), 2025, at the NLP4CALL workshop, colocated with the NoDaLiDa conference (https://www.nodalida-bhlt2025.eu/conference) to be held in Estonia, Tallinn, on 2--5 March 2025.

The publication venue for system descriptions will be the proceedings of the NLP4CALL workshop. Official system evaluation will be carried out on CodaLab.

* TASK DESCRIPTION

In this shared task, your goal is to rewrite learner-written texts to make them grammatically correct or both grammatically correct and idiomatic, that is either adhering to the "minimal correction" principle or applying fluency edits.

For instance, the text

My mother became very sad, no food. But my sister better five months later.

can be corrected minimally as

My mother became very sad, and ate no food. But my sister felt better five months later.

or with fluency edits as

My mother was very distressed and refused to eat. Luckily, my sister recovered five months later.

For fair evaluation of both approaches to the correction task, we will provide two evaluation metrics, one favoring minimal correction, one suited for fluency-edited output (read more under Evaluation).

We particularly encourage development of multilingual systems that can process all (or several) languages using a single model, but this is not a mandatory requirement to participate in the task.

* DATA

We provide training, development and test data for each of the languages. The training and development dataset splits will be made available through Github. Evaluation will be performed on a separate test set. See website for more detailed information: https://github.com/spraakbanken/multigec-2025/

* EVALUATION

During the shared task, evaluation will be based on cross-lingually applicable automatic metrics, primarily:

- GLEU score (reference-based) - Scribendi score (reference-free)

For comparability with previous results, we will also provide F0.5 scores.

After the shared task, we also plan on carrying out a human evaluation experiment on a subset of the submitted results.

* TIMELINE (preliminary)

- June 18, 2024 - first call for participation - September 20, 2024 - second call for participation - October 20, 2024 - third call for participation. Training and validation data released, CodaLab opens for team registrations - October 30, 2024 - reminder. Validation server released online - November 13, 2024 - test data released - November 20, 2024 - system submission deadline (system output) - November 29, 2024 - results announced - December 20, 2024 - paper submission deadline with system descriptions - January 20, 2025 - paper reviews sent to the authors - February 7, 2025 - camera-ready deadline - March 5 (or March 2), 2025 - presentations of the systems at the NLP4CALL workshop

* PUBLICATION

We encourage you to submit a paper with your system description to the NLP4CALL workshop special track. We follow the same requirements for paper submissions as the NLP4CALL workshop, i.e. we use the same template and apply the same page limit. All papers will be reviewed by the organizing committee. Upon paper publication, we encourage you to share models, code, fact sheets, extra data, etc. with the community through GitHub or other repositories.

* ORGANIZERS

- Arianna Masciolini, University of Gothenburg, Sweden - Andrew Caines, University of Cambridge, UK - Orphee De Clecrq, Ghent university, Belgium - Murathan Kurfali, Stockholm University, Sweden - Ricardo Muñoz Sánchez, University of Gothenburg, Sweden - Elena Volodina, University of Gothenburg, Sweden - Robert Östling, Stockholm University, Sweden

* DATA PROVIDERS (more languages to come)

- Czech: Alexandr Rosen, Charles University, Prague - English: Andrew Caines, University of Cambridge - Estonian: -- Mark Fishel, University of Tartu, Estonia -- Kais Allkivi-Metsoja, Tallinn University, Estonia -- Kristjan Suluste, Eesti Keele Instituut, Estonia - German: -- Torsten Zesch, Fernuniversität in Hagen, Germany -- Andrea Horbach, Fernuniversität in Hagen, Germany - Icelandic: Isidora Glisič, University of Iceland - Italian: Jennifer-Carmen Frey, Eurac Research Bolzano, Italy - Latvian: - Roberts Darģis, University of Latvia - Ilze Auzina, University of Latvia - Slovene: Špela Arhar Holdt, University of Ljubljana, Slovenia - Swedish: Arianna Masciolini, University of Gothenburg, Sweden - Ukrainian: -- Oleksiy Syvokon, Microsoft and -- Mariana Romanyshyn, Grammarly

* CONTACT

Please join the MultiGEC-2025 Google group (https://groups.google.com/g/multigec-2025) in order to ask questions, hold discussions and browse for already answered questions.