[Due to many requests, we have extended the deadlines.]

We invite proposals for tasks to be run as part of RANLP 2025 (Recent Advances in Natural Language Processing): https://ranlp.org/ranlp2025/.

RANLP is one of the most influential and competitive NLP conferences. RANLP 2025 will take place in September 2025 at the Black Sea city of Varna. For the first time in RANLP history, we are organising a shared task campaign as part of the main conference and inviting task organisers to submit their task proposals. Researchers and practitioners from all areas of Natural Language Processing and related communities are invited to submit task proposals.

For RANLP 2025, we welcome any task that can evaluate an automatic system for natural language processing. We especially encourage tasks for languages other than English, multi-lingual tasks, and tasks that develop novel applications of natural language processing.

We strongly encourage proposals based on already published datasets, as this can provide concrete examples and help minimise the challenges of organising the shared task. In the event of receiving many proposals, preference will be given to proposals based on already published datasets.

If you are unsure whether a task is suitable, please contact the shared task chairs to discuss your idea.

Task Selection

Task proposals will be reviewed by at least two reviewers, and the reviews will serve as the basis for acceptance decisions. Task proposals will be evaluated on:

Novelty - Is the task based on a new problem that has not been explored much in the community? If similar tasks have been organised before, does this task cover new languages/ domains?
Data – Is the data available and published already? Do annotations have meaningfully high inter-annotator agreements? Have all appropriate licenses for the use and re-use of the data been secured?
Evaluation—Is the evaluation methodology sound? Is there an automated platform for the evaluation (e.g., CodaLab, Kaggle)?

Task Organisation

We specifically welcome task proposals from early career researchers. However, we strongly encourage tasks that have a diverse team of organisers as that will ease the task organisation. Apart from providing a dataset, task organisers are expected to:

Verify data quality in terms of annotator agreement.
Verify licenses for the data to allow its use in the competition.
Provide task participants with baseline systems.
Create a CodaLab or other similar evaluation platform for the task and manage automatic evaluation.
Promote the task within the target research community.
Manage and organise review process of participants’ submissions of system description papers.
Write a task description paper to be included in RANLP proceedings.
Contribute to the tasks overview paper written by shared task chairs and other task organisers which will also be included in RANLP proceedings.
Register and present the shared task description paper at RANLP 2025 on either 11th or 12th September 2025 (the exact date will be confirmed later)

Important Dates

Task proposals due - November 20, 2024
Task selection notification – November 25, 2024

Recommended Timeline for the Tasks

Sample data and task website ready - December 1, 2024
Training data ready - December 15, 2024
Evaluation data ready - March 1, 2025
Evaluation starts – March 10, 2025
Evaluation end - March 31, 2025 (latest date; task organisers may choose an earlier date)
Paper submission due – April 20, 2025
Notification to authors – May 16, 2025
Task overview paper due – May 25, 2025
Camera-ready due - May 31, 2025
Shared task presentation co-located with RANLP 2025 – September 11 and September 12, 2025

Tasks that do not meet critical deadlines, such as those for launching the task, setting up the CodaLab website, and uploading samples, training, and evaluation data, may be cancelled at the discretion of the shared task chairs.

Submission Details

The task proposal should be a self-contained document of no longer than 2 pages (plus additional pages for references). All submissions must be in PDF format, following the RANLP 2023 template available at https://ranlp.org/ranlp2023/index.php/submissions/

Each proposal should contain the following:

Overview

Summary of the task – What is the goal of the task
Expected number of participants and justification

Data & Resources

How the training/testing data will be produced. Discuss whether the dataset is already published
Details of license, so that the data can be used by the research community
How much data will be produced
How data quality will be ensured and evaluated
An example of what the data would look like

Evaluation

The evaluation methodology to be used, including clear evaluation criteria -
The evaluation platform (i.e. CodaLab, Kaggle etc.)

Task organisers

Names, affiliations, email addresses
brief description of relevant experience or expertise

The submissions should be done via START - https://softconf.com/ranlp25/papers/user/scmd.cgi?scmd=submitPaperCustom&pageid=3

Proceedings

Tasks overview paper, task description papers and participant papers will be published as part of RANLP 2025 proceedings in ACLAnthology. Task organisers and participants are expected to attend RANLP 2025 on September 11 and September 12, 2025, and present their work in order to include it in the proceedings.

Shared Task Chairs

Dr Tharindu Ranasinghe, Lancaster University, UK

Dr Saad Ezzini, Lancaster University, UK

RANLP 2024 Chairs

Programme Committee Chair: Prof Dr Ruslan Mitkov, Lancaster University, UK

Organising Committee Chair: Prof Dr Galia Angelova, Bulgarian Academy of Sciences, Bulgaria

Best Regards

Dr Tharindu Ranasinghe | Lecturer in Security and Protection Science
School of Computing and Communications | Lancaster University
Contact me on Teams
www.lancaster.ac.uk