- SIGUL - ELRA lists

[CfP] Third LLM4Eval@SIGIR 2025
by Guglielmo Faggioli 21 Mar '25

21 Mar '25

The Third Workshop on LLM4Eval progresses the discussion from the previous series. These earlier events investigated the potential and challenges of using LLMs for search relevance evaluation, automated judgments, and retrieval-augmented generation (RAG) assessment. As modern IR systems integrate search, recommendations, conversational interfaces, and personalization, new evaluation challenges arise beyond basic relevance assessment. These applications create personalized rankings, explanations, and adapt to user preferences over time, requiring new evaluation methods. Overview Recent advancements in Large Language Models (LLMs) have significantly impacted evaluation methodologies in Information Retrieval (IR), reshaping the way relevance, quality, and user satisfaction are assessed. Initially demonstrating the potential for query-document relevance judgments, LLMs are now being applied to more complex tasks, including relevance label generation, assessment of retrieval-augmented generation systems, and evaluation of the quality of text-generation systems. As IR systems evolve toward more sophisticated and personalized user experiences, integrating search, recommendations, and conversational interfaces, new evaluation methodologies become necessary. Building upon the success of our previous workshops, this third iteration of the LLM4Eval workshop at SIGIR 2025 seeks submissions exploring new opportunities, limitations, and hybrid approaches involving LLM-based evaluations. Important Dates - Paper submission deadline: April 23, 2025 (AoE) - Notification of acceptance: May 21, 2025 (AoE) - Workshop date: July 17, 2025 Topics of interest We invite submissions on topics including, but not limited to: - LLMs for query-document relevance assessment - Evaluating conversational IR and recommendation systems with LLMs - Hybrid evaluation frameworks combining LLM and human annotations - Identifying failure modes and limitations of LLM annotations - Prompt engineering strategies for improving LLM annotation quality - Standardizing protocols for reliable LLM-based evaluations - Bias, fairness, and ethical considerations in LLM evaluations - LLM annotation robustness, reliability, and reproducibility - User-centric evaluations, personalization, and subjective assessments with LLMs - Case studies and lessons from industry applications of LLM-based evaluations Submission guidelines - Papers must follow SIGIR format and should not exceed 9 pages, excluding references. - We accept full papers (published or unpublished), position papers, and demo papers. - All papers will be peer-reviewed (double-blind) by the program committee and judged by their relevance to the workshop themes and potential to generate discussion. - Previously published studies can be submitted in their original format and will be reviewed solely for their relevance to this workshop. - All submissions must be in English (PDF format). - Submission through EasyChair: https://easychair.org/conferences/?conf=llm4evalsigir25. Publication options Authors can choose between archival and non-archival options for their submissions: - Archival: Papers will be included in the workshop proceedings. - Non-archival: Papers may be uploaded to arXiv.org, allowing submission elsewhere as they will be considered non-archival. The workshop’s website will maintain a link to the arXiv versions of the papers.

1 0

RANLP 2025: Third Call for Papers and Extended Submission Deadline
by Ranasinghe, Tharindu 17 Mar '25

17 Mar '25

RANLP 2025 RECENT ADVANCES IN NATURAL LANGUAGE PROCESSING Hotel “Cherno More” Varna, Bulgaria https://ranlp.org/ranlp2025/ Summer School on Deep Learning and LLMs for NLP: 3-5 September 2025 (Wednesday-Friday) Tutorials: 6-7 September 2025 (Saturday-Sunday) Main Conference: 8-10 September 2025 (Monday-Wednesday) Workshops and shared tasks: 11-13 September 2025 (Thursday-Saturday) We are pleased to announce that the 15th biennial RANLP conference will take place in September 2025 at the Black Sea city of Varna. In addition to the conference programme of competitively peer-reviewed papers reporting on the recent advances of a wide range of Natural Language Processing (NLP) topics, the conference features keynote talks by leading experts in NLP. Poster and demo sessions will be held at the conference exhibition area. The conference will be preceded by three days of summer school on Deep Learning and LLMs for NLP (3-5 September 2025) and two days of tutorials (6-7 September 2025). Post-conference specialised workshops as well as shared tasks covering timely NLP topics will be held on 11-13 September 2025. A Student Research Workshop will run in parallel to the main conference. The Student Research Workshops (now the 9th edition) have become active discussion fora for young researchers. As from RANLP 2009, the papers accepted at RANLP and the associated workshops are included in the ACL Anthology. The RANLP proceedings are indexed by SCOPUS and DBLP. The SCOPUS SJR of RANLP proceedings is 0,299 (2023). After 2017, all accepted papers have DOI numbers. CHAIR OF THE PROGRAMME COMMITTEE Ruslan Mitkov (University of Lancaster, UK and University of Alicante, Spain) CHAIR OF THE ORGANISING COMMITTEE Galia Angelova (Bulgarian Academy of Sciences, Bulgaria) The Programme Committee members are distinguished NLP experts from all over the world. The list of PC members will be announced on the conference website in due course. INVITED SPEAKERS The list of keynote speakers at RANLP 2025 and tutorial lecturers as well as summer school lecturers and teaching assistants includes (more names will be announced in the coming weeks): KEYNOTE SPEAKERS: * Eneko Agirre (University of the Basque Country, Spain) * Roberto Navigli (Sapienza University of Rome, Italy) * Anna Rogers (IT University of Copenhagen, Denmark) TUTORIAL LECTURERS: * Burcu Can (University of Sterling, UK) * Anna Rogers and Max Müller-Eberstein (IT University of Copenhagen, Denmark) SUMMER SCHOOL LECTURERS and TEACHING ASSISTANTS (in alphabetical order): * Maram Alharbi (University of Lancaster, UK) * Isuri Nanomi Arachchige (University of Lancaster, UK) * Burcu Can (University of Sterling, UK) * Salmane Chafik (Mohammed VI Polytechnic University, Morocco) * Ernesto Luis Estevanell (University of Alicante, Spain) * Hansi Hettiarachchi (University of Lancaster, UK) * Andrei Mikheev (Daxtra Technologies, UK) * Damith Dola Mullage (University of Lancaster, UK) * Max Müller-Eberstein (IT University of Copenhagen, Denmark) * Henry Oldroyd (University of Lancaster, UK) * Tharindu Ranasinghe (University of Lancaster, UK) WORKSHOPS and SHARED TASKS: The RANLP 2025 workshops and shared tasks will be held on 11-13 September 2025. The first workshops have already been selected (see below) and further proposals are expected by 31 March 2025. * The first Interdisciplinary Workshop on Observations of Misunderstood, Misguided and Malicious Use of Language Models (OMMM 2025), organised by Piotr Przybyła, Matthew Shardlow, Clara Colombatto and Nanna Inie * The first Workshop on Ethical Concerns in Training, Evaluating and Deploying Large Language Models (EthicalLLMs 2025), organised by Damith Premasiri, Tharindu Ranasinghe and Hansi Hettiarachchi. * Natural Language Processing and Language Models for Digital Humanities organised by Isuri Nanomi Arachchige, Francesca Frontini, Ruslan Mitkov and Paul Rayson * From rules to language models: comparative evaluation of NLP methods organised by Alicia Picazo-Izquierdo, Ernesto Luis Estevanell-Valladares, Ruslan Mitkov and Raúl García Cerdá * Advancing NLP for Low-Resource Languages organised by Ernesto Luis Estevanell-Valladares, Alicia Picazo-Izquierdo and Tharindu Ranasinghe The following SHARED TASKS have been accepted and Call for Participation has been distributed: * PolyHope-M: Bridging Hope Speech Detection Across Multiple Languages, organised by Fazlourrahman Balouchzahi, Sabur Butt, Maaz Amjad, Luis Jose Gonzalez-Gomez, Abdul Gafar Manuel Meque, Helena Gomez-Adorno, Bharathi Raja Chakravarthi, Grigori Sidorov, Thomas Mandl, Ruba Priyadharshini and Saranya Rajiakodi Task website - https://www.codabench.org/competitions/5635/ * Multilingual Coreference Resolution, organised by Vijay Sundar Ram, Pattabhi RK Rao and Sobha Lalitha Devi Task website - https://www.codabench.org/competitions/5759/ * Sentiment Analysis on Arabic Dialects in the Hospitality Domain: A Multi-Dialect Benchmark, organised by Maram I. Alharbi, Salmane Chafik, Ruslan Mitkov and Saad Ezzini Task website - https://ahasis-42267.web.app/ * Multi-Domain Detection of AI-Generated Text (M-DAIGT), organised by Salima Lamsiyah, Saad Ezzini, Abdelkader El Mahdaouy, Hamza Alami, Abdessamad Benlahbib, Samir El Amrany, Salmane Chafik and Hicham Hammouchi Task website - https://ezzini.github.io/M-DAIGT/ * Identification of the severity of the depression in forum posts, organised by Isuri Anuradha, Hasintha Hewawasam, Deshan Koshala Sumanathilaka, Ruslan Mitkov, Paul Rayson and Saad Ezzini Task website - https://www.codabench.org/competitions/5894/ SUBMISSION OF PAPERS, POSTERS, DEMOS The submissions will be maintained by the conference management software START. For further instructions, please follow the submission information at the conference website at https://ranlp.org/ranlp2025/. The reviewing process will be anonymous. Double submission is acceptable, but authors will be asked to declare it at the time of submission. Submissions will be reviewed by at least three members of the Programme Committee. Authors of accepted papers will receive guidelines regarding how to produce camera-ready versions of their papers for inclusion in the proceedings. All RANLP papers have DOI numbers assigned. The full conference proceedings will be uploaded on the ACL Anthology. RANLP publishes Regular papers 8 pages (with 30 min oral presentation), Short papers 6 pages (with 20 min oral presentation), and Poster/Demo papers 4 pages (with presentation in a poster or demo session). Additional pages are allowed for references only. RANLP-2025 aims to provide early notification of acceptance to authors and presenters who need visa to enter Bulgaria. We invite early submissions of authors’ names and paper abstracts, in order to plan quick reviewing. Access to the conference management software will be available as from 1 April 2025. IMPORTANT DATES Deadline for submission of workshop proposals: 15 March 2025 31 March 2025 Workshop selection: 22 March 2025 2 April 2025 Conference abstracts submission: 11 May 2025 (strongly recommended, to facilitate review planning) Conference papers submission: 25 May 2025 Conference papers acceptance notification: 4 July 2025 Camera-ready versions of the conference papers: 31 July 2025 Workshop paper submission deadline (suggested): 6 July 2025 Workshop paper acceptance notification (suggested): 31 July 2025 Workshop paper camera-ready versions (suggested): 30 August 2025 Workshop camera-ready proceedings ready (suggested): 8 September 2025 RANLP Summer School on Deep Learning in NLP: 3-5 September 2025 RANLP tutorials: 6-7 September 2025 (Saturday-Sunday) RANLP conference: 8-10 September 2025 (Monday-Wednesday) RANLP workshops and Shared Tasks presentations: 11-13 September 2025 (Thursday-Saturday) VENUE RANLP 2025 will be held at the conference facilities of Hotel “Cherno More” (http://www.chernomorebg.com<http://www.chernomorebg.com/> ) in Varna, the largest city on the Bulgarian Black Sea Coast. The event venue is centrally located at the entrance of the Sea Garden and offers excellent conference facilities. The city is a major tourist destination with flights to/from the Varna International Airport. It is also known for its Archaeological Museum, which features the oldest gold treasure in the world (https://en.wikipedia.org/wiki/Varna_Necropolis). The conference organisers plan to arrange a visit to Provadia-Solnitsata, the oldest salt-production and urban centre in Europe (5600 - 4350 BC, https://provadia-solnitsata.com/en/ ) which is located 50 km from Varna. THE TEAM BEHIND RANLP-25 Galia Angelova, Bulgarian Academy of Sciences, Bulgaria (Chair Organising Committee) Ruslan Mitkov, University of Lancaster, UK and University of Alicante, Spain (Chair Programme Committee) Nikolai Nikolov, Bulgarian Association for Computational Linguistics, Bulgaria Tharindu Ranasinghe, Lancaster University, UK (Workshops Chair and Shared tasks Co-Chair) Saad Ezzini, KFUPM, Saudi Arabia (Sponsorship Chair and Shared tasks Co-Chair) Maria Kunilovskaya, Saarland University, Germany (Publication Chair) Preslav Nakov, MBZUAI, Abu Dhabi, UAE Ivelina Nikolova, Bulgarian Academy of Sciences, Bulgaria Kiril Simov, Bulgarian Academy of Sciences, Bulgaria (Workshops Co-Chair) Petya Osenova, Bulgarian Academy of Sciences, Bulgaria (Workshops Co-Chair)

1 0

Job Opening for Data Scientist (with a focus on natural language processing)
by Menno Van Zaanen 17 Mar '25

17 Mar '25

Job Opening for Data Scientist (with a focus on natural language processing) Application link: https://bit.ly/4iUSSAN Application deadline: 20 April 2025 As a Data Scientist at the South African Centre for Digital Language Resources (SADiLaR) you will have the opportunity to initiate and lead projects focusing on Human Language Technology and Digital Humanities stemming from your own research interests. You will work closely together with a team of researchers as part of SADiLaR’s extended network, both on your own and commissioned projects. Dissemination of project results at national and international conferences will be encouraged and supported. This position is crucial for research and development in Human Language Technology and Digital Humanities, fields that form the essence of SADiLaR, which is a national Research Infrastructure supported by the Department of Science and Innovation. Read more at https://www.sadilar.org. KEY RESPONSIBILITIES: * Research: Research in the area of Human Language Technology and Digital Humanities. * Project work: Initiating and contributing to Human Language Technology and Digital Humanities projects. * Teaching: Teaching in the area of Human Language Technology and Digital Humanities. * Mentorship: Mentorship of researchers in the field of Human Language Technology and Digital Humanities. Minimum requirements * A PhD (NQF level 10) in one of the following fields: Computational Linguistics, Natural Language Processing, Human Language Technology, Digital Humanities, Data Science, Computer Science, Information Technology, Artificial Intelligence, or related fields. The PhD should have a focus on computational aspects of linguistics. * A minimum of (five) 5 years’ experience in the use of Python (other programming languages used within the computational linguistics or Digital Humanities domain can also be considered). * Evidence of peer-reviewed academic publications. * A minimum of (three) 3 years’ experience as a supervisor/co- supervisor of students or playing a mentorship/supervising role for individuals. * A minimum of (three) 3 years’ experience with using and/or developing computational tools. * A minimum of (three) 3 years experience related to research within the domain of Language Technology or Digital Humanities. * A minimum of (one) 1 year experience related to teaching or training within the domain of Language Technology or Digital Humanities. ADDED ADVANTAGE * Membership with Academic subject communities. Functional / Technical Competencies (Knowledge and Skills) * Advanced computer literacy. * Ability to lead research projects. * Experience in the presentation of research-based results at national and international conferences. * Experience with writing research reports. * Evidence of acquiring research funding. KEY BEHAVIOURAL COMPETENCIES: * Ability to work independently or as part of a team. * Ability to effectively liaise and communicate with public, students, colleagues, and other stakeholders at various levels and from diverse backgrounds. * Demonstration of language proficiency in order to function optimally in the various multilingual environments of SADiLaR. * Strong interest in the advancement of under-resourced South African languages. REMUNERATION The annual total remuneration package will be commensurate with the level of appointment as advertised and in line with the NWU policy guidelines. ENQUIRIES REGARDING JOB CONTENT MAY BE DIRECTED TO: Prof. Menno van Zaanen, E-mail: Menno.VanZaanen(a)nwu.ac.za ENQUIRIES REGARDING RECRUITMENT PROCESS MAY BE DIRECTED TO: Mr. Byron Louw, Tel No. 018 285 2304 CLOSING DATE: 20 April 2025 PLANNED COMMENCEMENT OF DUTIES: As soon as possible Kindly take note: applications must be submitted online through the official nwu vacancy website. Incomplete applications and those submitted through any other platform will not be considered. Application link: https://bit.ly/4iUSSAN -- Prof Menno van Zaanen menno.vanzaanen(a)nwu.ac.za Professor in Digital Humanities South African Centre for Digital Language Resources https://www.sadilar.org ________________________________ NWU PRIVACY STATEMENT: http://www.nwu.ac.za/it/gov-man/disclaimer.html DISCLAIMER: This e-mail message and attachments thereto are intended solely for the recipient(s) and may contain confidential and privileged information. Any unauthorised review, use, disclosure, or distribution is prohibited. If you have received the e-mail by mistake, please contact the sender or reply e-mail and delete the e-mail and its attachments (where appropriate) from your system. ________________________________

1 0

Call for Participation - RANLP 2025 Shared Tasks
by Ranasinghe, Tharindu 14 Mar '25

14 Mar '25

We invite researchers to participate in RANLP 2025 shared tasks. RANLP is one of the most influential and competitive NLP conferences. RANLP 2025 will take place in September 2025 at the Black Sea city of Varna. For the first time in RANLP history, we are organising a shared task campaign as part of the main conference. Researchers and practitioners from all areas of natural language processing and related communities are invited to participate. The following tasks have been accepted for RANLP 2025. 1. PolyHope-M: Bridging Hope Speech Detection Across Multiple Languages Organisers - Fazlourrahman Balouchzahi, Sabur Butt, Maaz Amjad, Luis Jose Gonzalez-Gomez, Abdul Gafar Manuel Meque, Helena Gomez-Adorno, Bharathi Raja Chakravarthi, Grigori Sidorov, Thomas Mandl, Ruba Priyadharshini and Saranya Rajiakodi Task website - https://www.codabench.org/competitions/5635/ 1. Multilingual Coreference Resolution Organisers - Vijay Sundar Ram, Pattabhi RK Rao and Sobha Lalitha Devi Task website - https://www.codabench.org/competitions/5759/ 1. Multi-Domain Detection of AI-Generated Text (M-DAIGT) Organisers - Salima Lamsiyah, Saad Ezzini, Abdelkader El Mahdaouy, Hamza Alami, Abdessamad Benlahbib, Samir El amrany, Salmane Chafik and Hicham Hammouchi Task website - https://ezzini.github.io/M-DAIGT/ 1. Identification of the severity of the depression in forum post Organisers - Isuri Anuradha, Hasintha Hewawasam, Deshan Koshala Sumanathilaka, Ruslan Mitkov, Paul Rayson and Saad Ezzini Task website - https://www.codabench.org/competitions/5894/ 1. Sentiment Analysis on Arabic Dialects in the Hospitality Domain: A Multi-Dialect Benchmark Organisers - Maram I. Alharbi, Salmane Chafik, Ruslan Mitkov, Tharindu Ranasinghe, Hansi Hettiarachchi and Saad Ezzini Task website - https://ahasis-42267.web.app/ For inquiries regarding the individual shared tasks, please contact the corresponding task organisers. Shared Task Proceedings At least one member from each team needs to register and attend the RANLP 2025 shared task (which will take place on 11, 12 or 13 September 2025) and present their work in order to include it in the proceedings. More information about the RANLP main conference and its co-located events are available at https://ranlp.org/ranlp2025/ Shared Task Chairs Dr Tharindu Ranasinghe, Lancaster University, UK Dr Saad Ezzini, King Fahd University of Petroleum and Minerals, Saudi Arabia RANLP 2024 Chairs Programme Committee Chair: Prof Dr Ruslan Mitkov, Lancaster University, UK Organising Committee Chair: Prof Dr Galia Angelova, Bulgarian Academy of Sciences, Bulgaria Dr Tharindu Ranasinghe Lancaster University

1 0

Fwd: 1st Call For Participation: Automatic Minuting Shared Task - AutoMin @ SIGDIAL 2025
by Laurent Besacier 14 Mar '25

14 Mar '25

> Third Run of the Automatic Minuting Shared Task - AutoMin @ SIGDIAL 2025 > A great opportunity to assess LLMs' ability to summarize and retrieve information from long contexts > We propose AutoMin 2025, the third instance of bi-annual shared task on meeting summarization into structured meeting minutes. We build upon our past experience from two previous AutoMins and add a new challenge to facilitate personalized access: answering questions about meeting content. > As in the previous editions, the tasks are run in English and Czech while the new question-answering task will be run monolingually (English-only) and cross-lingually (Czech questions about English meetings). > This challenge is highly relevant for assessing LLMs' ability to summarize and retrieve information from long contexts, equivalent to the length of an hour-long meeting. > A dedicated workshop is scheduled for August 28th, 2025, as part of SIGDIAL 2025: https://2025.sigdial.org <https://2025.sigdial.org/>. > > Challenge Website > https://ufal.github.io/automin-2025/ > > Tasks > We propose two main tasks. > Task A (minuting): The task consists of automatically creating minutes from multiparty meeting transcripts. The generated minute would be evaluated both via automatic and manual metrics. > Task B (question answering): The task consists in answering questions based on meeting transcripts. For this challenge, participants will be tasked with answering these questions either monolingually (English questions on English transcripts) or cross-lingually (Czech questions on English transcripts). Answers will be evaluated using a LLM-as-a-judge metric. > > > Participant registration > All participants should register using the following link <https://docs.google.com/forms/d/e/1FAIpQLSeMTa2DkXtZCMbBpTo5c853nOr3yq2N1dX…> > > Important Dates > Registration and CfP > From March 1st, 2025 > Training and Dev Data Available > March 1st, 2025 > Test Data Available > April 15th, 2025 > System Run and Output Submission > April 29th, 2025 > System Paper Submission > May 5th, 2025 > Result Announcement > May 22d, 2025 > Acceptance Notification of System Papers > June 2025 (TBA) > Camera Ready for System Papers > TBA > Event Date > August 28th, 2025 > > Publication > All teams will be required to submit a brief technical report describing their method. Details will be given on the challenge website soon. > > Organizers > Laurent Besacier, Naver Labs Europe, France > Ondřej Bojar, UFAL, Charles University, Czech Republic > Tirthankar Ghosal, Oak Ridge National Laboratory, USA > Marie Hledıkova, UFAL, Charles University, Czech Republic > Thibaut Thonet, Naver Labs Europe, France > > Contact > The organizers can be contacted via the following email address: automin(a)ufal.mff.cuni.cz > >

1 0

Re: SpeD 2025
by Laurent Besacier 13 Mar '25

13 Mar '25

bonjour Corneliu oui ça va bien j espère que toi aussi ! merci pour la sollictiation, j accepte avec plaisir ! et relire 2 articles est aussi ok pour moi ! amicalement et bon succès dans l’organisation de cet évènement Laurent > Le 11 mars 2025 à 16:56, Corneliu BURILEANU <corneliu.burileanu(a)yahoo.com> a écrit : > > Cher Laurent, > > J'espère que tu vas bien, ça faisait longtemps qu'on ne s'était pas vu... > > Nous continuons notre conférence SpeD. Cette année, ce sera du 19 au 22 octobre à Cluj-Napoca (pub.sped.ro) > > Nous serions très heureux si vous acceptiez de continuer à faire partie du Comité Scientifique de la Conférence ! > > Si vous êtes d’accord, nous pourrons également vous demander de procéder à 2 ou 3 évaluations des articles soumis à la Conférence. > > Bien sûr, je serais très heureux de vous voir à cette occasion, mais nous en parlerons d'ici là. > > Avec beaucoup d'amitié, > Corneliu >

1 0

Call for participation in "Machine Translation Metrics for Indigenous languages" shared task, AmericasNLP 2025
by ali.marashian＠colorado.edu 25 Feb '25

25 Feb '25

The AmericasNLP 2025 Shared Task on machine translation metrics for Indigenous languages is a competition intended to motivate the advancement of automatic evaluation metrics for machine translation, with a focus on translation into Indigenous languages. Participants will build metrics to evaluate the quality of translations from Spanish into Guarani, Bribri, and Nahuatl. Many Indigenous languages of the Americas have linguistic characteristics that are uncommon in the languages typically studied in natural language processing (NLP). For example, Indigenous languages can often be polysynthetic, and they frequently lack orthographic standardization, both of which pose challenges for, e.g., the widely used BLEU and ChrF scores. The goal of the AmericasNLP 2025 shared task on machine translation metrics for Indigenous languages is to encourage researchers to tackle the challenge of creating automatic evaluation metrics for MT systems with a focus on Indigenous languages. Shared task description: https://turing.iimas.unam.mx/americasnlp/2025_st_3.html Registration: https://docs.google.com/forms/d/e/1FAIpQLSfmV9gyab03pGGzzDJ9DohOf9AkDaZf_94… AmericasNLP will be held in NAACL 2025, Albuquerque, New Mexico. The workshop date is May 4, 2025. Thanks, Ali, on behalf of the AmericasNLP team

1 0

Third Call for Papers, Participation and Demo - IWSLT 2025
by Atul K. Ojha 17 Feb '25

17 Feb '25

Apologies for cross-posting. ---------------------------------------- *The International Conference on Spoken Language Translation* *ACL – 22nd IWSLT 2025 – **Third** Call for Participation* *31 July-1 August 2025 - Vienna, Austria* http://iwslt.org The International Conference on Spoken Language Translation (IWSLT) <https://iwslt.org/> is the premier annual conference for all aspects of Spoken Language Translation. Every year, the conference organises and sponsors open evaluation campaigns around key challenges in simultaneous and consecutive translation, under real-time/low latency or offline conditions and under low-resource or multilingual constraints. System descriptions and results from participants’ systems and scientific papers related to key algorithmic advances and best practices are presented. IWSLT is the venue of the SIGSLTs <https://iwslt.org/sigslt/>, the Special Interest Group on Spoken Language Translation <https://iwslt.org/sigslt/> of ACL <https://www.aclweb.org/portal/>, ISCA <https://www.isca-speech.org/> and ELRA <https://www.elra.info/>. With a track record of 21 years, IWSLT benchmarks and proceedings serve as reference for all researchers and practitioners working on speech translation and related fields. The 22nd edition of IWSLT will be run as a hybrid ELRA <https://www.elra.info/>/ACL <https://www.aclweb.org/portal/> event, co-located with ACL 2025 <https://2025.aclweb.org/> from 31 July to 1 August 2025. *Important Dates* *January 1, 2025*: Release of shared task training and dev data *March 15, 2025*: Scientific paper submission deadline *Apr 1-15, 2025*: Evaluation period *April 21, 2025*: System description paper and demo submission deadline *May 15, 2025*: Notification of acceptance *June 1, 2025*: Camera-ready deadline (all paper) *July 31-Aug 1*, *2025*: IWSLT conference *Evaluation* The IWSLT 2025 features shared tasks <https://iwslt.org/2025/#shared-tasks> that address the following focus areas: - High-resource ST: Offline track, Simultaneous track, Subtitling track, Model compression track - Low-resource ST: Low-resource and Indic (multilingual) tracks - Instruction-following Speech Processing track: Technical domain ST, ASR, Summarization, and QA Training and development data for each shared task will be prepared and released by the respective organisers (for further information on this initiative, please refer to the IWSLT website <https://iwslt.org/2025/>). Participants will receive instructions about how to submit their runs. In addition, participants have the opportunity to present their work through a system paper that will be published in the ACL Proceedings. *Conference* IWSLT also invites submissions of scientific papers to be published in the ACL Proceedings and presented either in oral or poster format. The conference selects high-quality, original contributions on theoretical and practical issues of spoken language translation research, technologies and applications. Submissions will be accepted directly through the IWSLT submission site (to be announced on the website <https://iwslt.org/2025/>). We will also accept commitments of submissions with reviews from the ACL Rolling Review. Additionally, to foster cross-pollination of ideas, the conference also invites the presentation of papers on speech translation recently published elsewhere. Please note that this is for non-archival presentation of papers relevant to speech translation already published in other venues (e.g., Findings for the *ACL, speech, NLP or MT conferences). Submissions for this category will be accepted through a dedicated form (to be announced on the website <https://iwslt.org/2025/>). Papers will be checked for relevance to IWSLT, and assigned either oral or poster presentation slots if selected. *Demo Session* We invite researchers, practitioners, and industry professionals to participate in an engaging demo session highlighting innovative systems, tools, and component technologies that advance the field of speech translation. The session will include live and interactive system demonstrations to foster discussion and knowledge exchange among participants across the field. For more information, please see our Call for Demos <https://iwslt.org/2025/call-for-demos>. *Contact* Please email iwslt-evaluation-campaign(a)googlegroups.com if you have any questions related to the shared tasks. Thanks, Marine, Marcello, Alex, Jan, Sebastian, Elizabeth, Atul (IWSLT organisers)

1 0

Third and Final CFP: 21st Workshop on Multiword Expressions (MWE 2025) @NAACL2025
by Atul K. Ojha 29 Jan '25

29 Jan '25

Apologies for cross-postings ******************************************************************************** Final Call for Papers 21st Workshop on Multiword Expressions (MWE 2025) Organized, sponsored and endorsed by SIGLEX, the Special Interest Group on the Lexicon of the ACL Full-day workshop collocated with NAACL 2025, Albuquerque, New Mexico, U.S.A., May 3 or 4, 2025 Hybrid (on-site & on-line) Submission deadline:* February 13, 2025* MWE 2025 website: <https://multiword.org/mwe2022/> https://multiword.org/mwe2025/ ******************************************************************************** Multiword expressions (MWEs), i.e., word combinations that exhibit lexical, syntactic, semantic, pragmatic, and/or statistical idiosyncrasies (Baldwin and Kim, 2010), such as “by and large”, “hot dog”, “make a decision” and “break one's leg” are still a pain in the neck for Natural Language Processing (NLP). The notion encompasses closely related phenomena: idioms, compounds, light-verb constructions, phrasal verbs, rhetorical figures, collocations, institutionalized phrases, etc. Given their irregular nature, MWEs often pose complex problems in linguistic modeling (e.g. annotation), NLP tasks (e.g. parsing), and end-user applications (e.g. natural language understanding and Machine Translation), hence still representing an open issue for computational linguistics (Constant et al., 2017). For more than two decades, modelling and processing MWEs for NLP has been the topic of the MWE workshop organised by the MWE section <https://multiword.org/> of ACL-SIGLEX <http://www.siglex.org/> in conjunction with major NLP conferences since 2003. Impressive progress has been made in the field, but our understanding of MWEs still requires much research considering their need and usefulness in NLP applications. This is also relevant to domain-specific NLP pipelines that need to tackle terminologies most often realised as MWEs. Following previous years, for this 21st edition of the workshop, we identified the following topics on which contributions are particularly encouraged: - MWE processing to enhance end-user applications. MWEs gained particular attention in end-user applications, including Machine Translation (MT) (Zaninello and Birch, 2020), simplification (Kochmar et al., 2020), language learning and assessment (Paquot et al., 2020), social media mining (Pelosi et al., 2017), and abusive language detection (Zampieri et al. 2020). We believe that it is crucial to extend and deepen these first attempts to integrate and evaluate MWE technology in these and further end-user applications. - MWE processing and identification in the general language, as well as in specialized languages and domains: Multiword terminology extraction from domain-specific corpora (Lossio-Ventura et al, 2014) is of particular importance to various applications, such as MT (Semmar and Laib, 2017), or for the identification and monitoring of neologisms and technical jargon (Chatzitheodorou and Kappatos, 2021). - MWE processing in low-resource languages: The PARSEME shared tasks (2017 <https://multiword.sourceforge.net/PHITE.php?sitesig=CONF&page=CONF_05_MWE_2…>, 2018 <https://multiword.sourceforge.net/PHITE.php?sitesig=CONF&page=CONF_04_LAW-M…>, 2020 <https://multiword.sourceforge.net/PHITE.php?sitesig=CONF&page=CONF_02_MWE-L…>) among others, have fostered significant progress in MWE identification, providing datasets that include low-resource languages, evaluation measures, and tools that now allow fully integrating MWE identification into end-user applications. There are continuous efforts in this direction (Diaz Hernandez, 2024) and a few of them have also explored methods for the automatic interpretation of MWEs (Bhatia et al., 2018), and their processing in low-resource languages (Eder et al., 2021). Resource creation and sharing should be pursued in parallel with the development of multilingual benchmarks for MWE identification (Savary et al., 2023). - MWE identification and interpretation in LLMs: Most current MWE processing is limited to their identification and detection using pre-trained language models, but we still lack understanding about how MWEs are represented and dealt with therein (Garcia et al., 2021), how to better model the compositionality of MWEs from semantics (Phelps et al., 2024). Now that NLP has shifted towards end-to-end neural models like BERT, capable of solving complex tasks with little or no intermediary linguistic symbols, questions arise about the extent to which MWEs should be implicitly or explicitly modelled (Shwartz and Dagan, 2019). - New and enhanced representation of MWEs in language resources and computational models of compositionality as gold standards for formative intrinsic evaluation. Through this workshop, we will bring together and encourage researchers in various NLP subfields to submit their MWE-related research, We also intend to consolidate the converging results of previous joint workshops LAW-MWE-CxG 2018 <http://multiword.sourceforge.net/lawmwecxg2018/>, MWE-WN 2019 <http://multiword.sourceforge.net/mwewn2019/> and MWE-LEX 2020 <http://multiword.sourceforge.net/mwelex2020/>, the joint MWE-WOAH panel in 2021 <https://multiword.org/mwe2021/#program>, the MWE-SIGUL 2022 joint session <https://multiword.org/mwe2022/>, and the MWE-UD 2024 <https://multiword.org/mweud2024/>, extending our scope to MWEs in e-lexicons, and WordNets, MWE annotation, as well as grammatical constructions. Correspondingly, we call for papers on research related (but not limited) to MWEs and constructions in: - Computationally-applicable theoretical work in psycholinguistics and corpus linguistics; - Annotation (expert, crowdsourcing, automatic) and representation in resources such as corpora, treebanks, e-lexicons, WordNets, constructions (also for low-resource languages); - Processing in syntactic and semantic frameworks (e.g. CCG, CxG, HPSG, LFG, TAG, UD, etc.); - Discovery and identification methods, including for specialized languages and domains such as clinical or biomedical NLP; - Interpretation of MWEs and understanding of text containing them; - Language acquisition, language learning, and non-standard language (e.g. tweets, speech); - Evaluation of annotation and processing techniques; - Retrospective comparative analyses from the PARSEME shared tasks; - Processing for end-user applications (e.g. MT, NLU, summarisation, language learning, etc.); - Implicit and explicit representation in pre-trained language models and end-user applications; - Evaluation and probing of pre-trained language models; - Resources and tools (e.g. lexicons, identifiers) and their integration into end-user applications; - Multiword terminology extraction; - Adaptation and transfer of annotations and related resources to new languages and domains including low-resource ones. Submission formats: The workshop invites two types of submissions: - archival submissions that present substantially original research in both long paper format (8 pages + references) and short paper format (4 pages + references). - non-archival submissions of abstracts describing relevant research presented/published elsewhere which will not be included in the MWE proceedings. Paper submission and templates Papers should be submitted via the workshop's submission page <https://openreview.net/group?id=aclweb.org/NAACL/2025/Workshop/MWE> ( https://openreview.net/group?id=aclweb.org/NAACL/2025/Workshop/MWE). Please choose the appropriate submission format (archival/non-archival). Archival papers with existing reviews will also be accepted through the ACL Rolling Review. Submissions must follow the ACL stylesheet <https://github.com/acl-org/acl-style-files>. Important Dates Paper Submission Deadline: February 13, 2025 Notification of acceptance: March 8, 2025 Camera-ready papers due: March 17, 2025 Workshop: May 3 or 4, 2025 All deadlines are at 23:59 UTC-12 (Anywhere on Earth). Organizing Committee Verginica Barbu Mititelu, Voula Giouli, Grazina Korvel, A. Seza Doğruöz, Alexandre Rademaker, Atul Kr. Ojha, Mathieu Constant Anti-harassment policy The workshop follows the ACL anti-harassment policy <https://www.aclweb.org/adminwiki/index.php?title=Anti-Harassment_Policy>. Contact For any inquiries regarding the workshop, please send an email to the Organizing Committee at mwe2025workshop(a)gmail.com.

1 0

Third and final CFP: LoResMT 2025 at NAACL 2025
by Atul K. Ojha 29 Jan '25

29 Jan '25

Apologies for cross-posting. --------------------------------------------------------------------------- *The Eighth Workshop on Technologies for Machine Translation of Low-Resource Languages (LoResMT 2025)* *https://www.loresmt.org/ <https://www.loresmt.org/>* *@ NAACL 2025 (May 3–4, 2025)* *Albuquerque, New Mexico, U.S.A.* *SUBMISSION* * <https://openreview.net/group?id=aclweb.org/ACL/2024/Workshop/LoResMT>https://openreview.net/group?id=aclweb.org/NAACL/2025/Workshop/LoResMT <https://openreview.net/group?id=aclweb.org/NAACL/2025/Workshop/LoResMT>* *TIMELINE* *Paper submission due:* *February 13, 2025* (Anywhere on Earth) *Pre-reviewed (ARR) submission deadline:* *February 27, 2025* *Notification of acceptance:* March 8, 2025 *Camera-ready papers due:* March 17, 2025 (Anywhere on Earth) *Pre-recorded video due (hard deadline):* April 8, 2025 *Workshop dates at NAACL 2025:* May 3–4, 2025 *SCOPE* Based on the success of past low-resource machine translation (MT) workshops at AMTA 2018, MT Summit 2019, AACL-IJCNLP 2020, AMTA 2021, COLING 2022, EACL 2023, ACL 2024, we introduce LoResMT 2025 workshop at NAACL 2025. The workshop provides a discussion panel for researchers working on MT systems/methods for low-resource and under-represented languages in general. We would like to help review/overview the state of MT for low-resource languages and define the most important directions. We also solicit papers dedicated to supplementary NLP tools that are used in any language and especially in low-resource languages. Overview papers of these NLP tools are very welcome. It will be beneficial if the evaluations of these tools in research papers include their impact on the quality of MT output. *TOPICS* We are highly interested in (1) original research papers, (2) review/opinion papers, and (3) online systems on the topics below; however, we welcome all novel ideas that cover research on low-resource languages. - Neural machine translation (NMT) for low-resource languages - Use of LLMs (large language models) for low-resource MT systems - COVID-related corpora, their translations and corresponding NLP/MT systems - Work that presents online systems for practical use by native speakers - Word tokenizers/de-tokenizers for specific languages - Word/morpheme segmenters for specific languages - Alignment/Re-ordering tools for specific language pairs - Use of morphology analyzers and/or morpheme segmenters in MT - Multilingual/cross-lingual NLP tools for MT - Corpora creation and curation technologies for low-resource languages - Review of available parallel corpora for low-resource languages - Research and review papers on MT methods for low-resource languages - MT systems/methods (e.g. rule-based, SMT, NMT) for low-resource languages - Pivot MT for low-resource languages - Zero-shot MT for low-resource languages - Fast building of MT systems for low-resource languages - Re-usability of existing MT systems for low-resource languages - Machine translation for language preservation *SUBMISSION INFORMATION* We are soliciting two types of submissions: (1) research, review, and position papers and (2) system demonstration papers. For research, review and position papers, the length of each paper should be at least four (4) and not exceed eight (8) pages, plus unlimited pages for references. For system demonstration papers, the limit is four (4) pages. Submissions should be formatted according to the official ACL style templates (Overleaf). Please refer to the NAACL submission guideline for further information <https://2025.naacl.org/calls/papers/#paper-submission-details>. Accepted papers will be published at ACL Anthology in the NAACL 2025 and will be presented at the conference. Submissions must be anonymized and should be done using the provided submission system. Scientific papers that have been or will be submitted to other venues must be declared as such and must be withdrawn from the other venues if accepted and published at LoResMT. The review will be double-blind. Authors of an accepted paper should present their paper in person at NAACL 2025. Papers should be submitted in PDF to the LoResMT Open Review <https://openreview.net/group?id=aclweb.org/NAACL/2025/Workshop/LoResMT>. We would like to encourage authors to cite papers written in ANY language that are related to the topics, as long as both original bibliographic items and their corresponding English translations are provided. Registration is handled by the main conference (https://2025.naacl.org/). *ORGANIZING COMMITTEE (LISTED ALPHABETICALLY)* Atul Kr. Ojha, University of Galway Chao-Hong Liu, Potamu Research Ltd Ekaterina Vylomova, University of Melbourne, Australia Jonathan Washington, Swarthmore College Nathaniel Oco, National University (Philippines) Flammie Pirinen, UiT The Arctic University of Norway, Tromsø Xiaobing Zhao, Minzu University of China *PROGRAM COMMITTEE (LISTED ALPHABETICALLY)* Abigail Walsh, ADAPT Centre, Dublin City University, Ireland Alberto Poncelas, Rakuten, Singapore Ali Hatami, University of Galway Alina Karakanta, Fondazione Bruno Kessler (FBK), University of Trento Anna Currey, AWS AI Labs Aswarth Abhilash Dara, Walmart Global Technology Atul Kr. Ojha, University of Galway & Panlingua Language Processing LLP Bogdan Babych, Heidelberg University Chao-hong Liu, Potamu Research Ltd Constantine Lignos, Brandeis University, USA Daan van Esch, Google Dana Moukheiber, Massachusetts Institute of Technology Ekaterina Vylomova, University of Melbourne, Australia Eleni Metheniti, CLLE-CNRS and IRIT-CNRS Flammie Pirinen, UiT Norgga árktalaš universitehta Gaurav Negi, University of Galway Jinliang Lu, Institute of automation, Chinese Academy of Sciences John Philip McCrae, University of Galway Jonathan Washington, Swarthmore College Koel Dutta Chowdhury, Saarland University Majid Latifi, UPC University Maria Art Antonette Clariño, University of the Philippines Los Baños Milind Agarwal, George Mason University Mathias Müller, University of Zurich Nathaniel Oco, De La Salle University Pavel Rychlý, Masaryk University and Lexical Computing Pengwei Li, Meta Rashid Ahmad, International Institute of Information Technology, Hyderabad Rico Sennrich, University of Zurich Santanu Pal, Wipro Sangjee Dondrub, Qinghai Normal University Sardana Ivanova, University of Helsinki Sourabrata Mukherjee, Charles University Thepchai Supnithi, National Electronics and Computer Technology Center Timothee Mickus, University of Helsinki Valentin Malykh, Huawei Noah’s Ark lab and Kazan Federal University Wen Lai, LMU Munich Xuebo Liu, Harbin Institute of Technolgy, Shenzhen Yalemisew Abgaz, Dublin City University Yasmin Moslem, Bering Lab Zhanibek Kozhirbayev, National Laboratory Astana, Nazarbayev University *CONTACT* Please email loresmt(a)googlegroups.com if you have any questions/comments/suggestions.

1 0