Call-for-Participation: TalentCLEF Shared Task (CLEF) - Skill and Job Title Intelligence for Human Capital Management
(* apologies for cross-posting *)
Call for Participation TalentCLEF Shared Task (CLEF 2025)
Skill and Job Title Intelligence for Human Capital Management
https://talentclef.github.io/talentclef/
TalentCLEF is an initiative to advance Natural Language Processing (NLP) in Human Capital Management (HCM). It aims to create a public benchmark for model evaluation and promote collaboration to develop fair, multilingual, and flexible systems that improve Human Resources (HR) practices across different industries.
Key information:
-
Web: https://talentclef.github.io/talentclef/ -
Data: https://doi.org/10.5281/zenodo.14002665 -
Registration: https://clef2025-labs-registration.dei.unipd.it/
Motivation In today’s rapidly changing socio-technological landscape, industries and workplaces are transforming quickly. Technological advancements, such as task automation and Artificial Intelligence (AI), are reshaping the labor market by creating new roles that demand specialized skills, often difficult to source. The rise of remote hiring, fueled by technological innovation, has expanded the labor market to a global and multilingual scale. Simultaneously, social progress is narrowing ethnic and gender disparities within companies, fostering more inclusive workplaces.
Integrating Natural Language Processing (NLP) into Human Capital Management (HCM) enhances key areas such as sourcing and hiring, onboarding and training, strategic workforce planning, and career development. Despite these benefits, challenges persist in managing multilingual information, ensuring fair AI models, and developing systems flexible enough to work across industries.
The TalentCLEF organizers expect that participation in the shared task will contribute to establishing a public benchmark for multilingual job title matching and skill prediction, enabling the evaluation and comparison of different approaches. This initiative will also provide a foundation for measuring gender bias in job-related NLP tasks and lay the groundwork for future benchmarks in other areas of Human Capital Management, fostering fairness, transparency, and adaptability in AI-driven workforce analysis.
The inaugural TalentCLEF shared-task aims to tackle these challenges through two key tasks:
-
Task A - Multilingual Job Title Matching: Participants will develop systems to identify and rank job titles most similar to a given one. For each job title in a test set, systems must generate a ranked list of similar titles from a predefined knowledge base. Evaluation will be conducted in English, Spanish, German, and Chinese, covering both monolingual and cross-lingual (between English and the other languages) matching. -
Task B - Job Title-Based Skill Prediction: This task focuses on retrieving relevant skills associated with a given job title. Participants will develop systems that predict and extract key skills based on job titles. The evaluation will be conducted in English.
Schedule
-
20th January 2025 - Training data available for Tasks A and B -
17th February 2025 – Start of Task A with the release of the development data -
17th March 2025 – Start of Task B with the release of the development data -
21st April 2025 – Test set release -
21st April - 5th May 2025 – Evaluation period of Task A and B -
7th May 2025 – Publication of Official Results -
30th May 2025 – Submission of CLEF 2025 Participant Working Notes (CEUR-WS) -
27th June 2025 - Notification of Acceptance for Participant Papers
Publications and CLEF 2025 workshop Teams participating in TalentCLEF will be invited to submit a system description paper for the CLEF 2025 Working Notes proceedings, published on CEUR-WS. Additionally, they will have the opportunity to present a brief overview of their approach at the CLEF 2025 workshop, which will take place in Madrid, Spain, from September 9th to 12th, 2025.
Main Organizers
-
Luis Gascó, Avature, Spain -
Hermenegildo Fabregat, Avature, Spain -
Laura García-Sardiña, Avature, Spain -
Daniel Deniz Cerpa, Avature, Spain -
Paula Estrella, Avature, Spain -
Álvaro Rodrigo, Universidad Nacional de Educación a Distancia (UNED), Spain -
Rabih Zbib, Avature, Spain
Scientific Committee
-
Eneko Agirre - Full Professor of the University of the Basque Country UPV/EHU - ACL Fellow -
David Camacho - Full Professor of the Technical University of Madrid (UPM) -
Debora Nozza - Assistant Professor of Bocconi University -
Jens-Joris Decorte - Lead AI Scientist at TechWolf -
David Graus - Lead Data Scientist at Randstad Group -
Mesutt Kayaa - Postdoctoral Researcher at Jobindex A/S and IT University Copenhagen -
Jan Luts - Senior Data Scientist at NTT Data & ESCO -
Elena Montiel-Ponsoda - Professor at the Technical University of Madrid (UPM) - AI4Labour project -
Javier Huertas Tato - Assistant Professor of the Technical University of Madrid (UPM) -
Patricia Martín Chozas - Postdoctoral Researcher at the Ontology Engineering Group (UPM) - AI4Labour project
*Call for Tracks* *FIRE 2025: 17th meeting of the Forum for Information Retrieval Evaluation*
Indian Institute of Technology (BHU) Varanasi 17th - 20th December Website: fire.irsi.org.in http://fire.irsi.org.in/
*Call for Tracks - Deadline extended to April 6th*
We invite proposals for offering evaluation tracks at FIRE 2025.
FIRE 2025 is the 17th edition of the annual meeting of Forum for Information Retrieval Evaluation (fire.irsi.org.in). Since its inception in 2008, FIRE had a strong focus on shared tasks similar to those offered at Evaluation forums like TREC, CLEF, and NTCIR. The shared tasks focus on solving specific problems in the area information access and, more importantly help in generating evaluation datasets for the research community.
It is not required for the tasks to focus on a specific language, and they can broadly cover any problem in the fields related (but not limited) to IR, NLP, multi-modal information access, and ML. However, the organizers especially encourage proposals for tracks related to South Asian, African, and Middle Eastern languages. In the past, FIRE has hosted tracks from Arabic, Persian, German, Russian, and Urdu languages besides several Indian languages. We aim to continue these efforts and include more language groups from these regions. For knowing more about tracks in past FIRE meetings, you can visit fire.irsi.org.in http://fire.irsi.org.in
Informal inquiries can also be sent to the track chairs.
Please include the following details in your proposal:
1. Track name 2. Track description 3. Use case/s 4. Target Audience and number of expected submissions 5. Data(*) (Fair Details) 6. Evaluation plan 7. Timeline: Please try to align with the FIRE conference dates as given below 8. Organizer/s Details 9. Prior experience in organizing shared task/workshop at relevant venues
*Tentative Timeline*
*6th **April 2025* Track proposals due *15th April, 2025* Track acceptance notification *15th May, 2025* Open track websites and release of training data *15th June, 2025* Test data release *30th June, 2025* Run submission deadline *15th July, 2025* Track results declaration *30th August, 2025* Working notes due *30th September, 2025* Camera-ready copies of working notes and overview paper due *17th December, 2025* FIRE Conference
Please send these details in a pdf format to clia@isical.ac.in with a copy to fire2025@itbhu.ac.in , kripa.ghosh@gmail.com and mandl@uni-hildesheim.de
(*) We require that after FIRE, the data should be made publicly available through Information Retrieval Society of India. In case, data can not be distributed publicly (e.g., Twitter data), a unique identifier that can be used to recreate the original corpus can be provided (e.g., tweet ids in case of Twitter data). This disbursal will be governed by a copyright form, which the users have to sign before getting the data. A sample form is available at ( fire.irsi.org.in/fire/static/data http://fire.irsi.org.in/fire/static/data ).
-
Luis Gasco -
Thomas Mandl