The 2nd LLMs4Subjects Shared Task: LLM-based Subject Tagging for the TIB Technical Library's Digital Catalog

Theme: The Development of Energy- and Compute-Efficient LLM Systems

Organized as part of the German Evaluation (GermEval 2025) Shared Task Series

10. - 12. September, 2025

Hildesheim, Germany

(co-located with KONVENS 2025 - Conference on Natural Language Processing)

2nd LLMs4Subjects Shared Task: https://sites.google.com/view/llms4subjects-germeval/

Join the Codabench Competition: https://www.codabench.org/competitions/8373/

KONVENS 2025: https://konvens-2025.hs-hannover.de/about/

Task Overview

LLMs4Subjects challenges the research community to develop cutting-edge LLM-based solutions for subject tagging of technical records from Leibniz University's Technical Library (TIBKAT). Participants are tasked with leveraging large language models (LLMs) to tag technical records using the GND taxonomy. The task involves bilingual language modeling, as systems must process technical documents in both German and English. Successful solutions may be integrated into the operational workflows of TIB, the Leibniz Information Centre for Science and Technology.

With the rapid advancements in LLMs, the focus is shifting toward making these models more energy- and compute-efficient while maintaining high performance. Recent innovations, such as the DeepSeek series, have demonstrated how techniques like mixture-of-experts (MoE) and model distillation can significantly reduce computational costs without sacrificing effectiveness.

The 2nd LLMs4Subjects shared task highlights the importance of efficiency in LLMs, encouraging participants to explore strategies that enhance model performance while optimizing for energy consumption and inference speed. We welcome approaches (but not limited to) that leverage model compression, quantization, efficient fine-tuning, and adaptive computation techniques to push the boundaries of sustainable AI development.

Subtasks

The 2nd LLMs4Subjects shared task organizes the following two subtasks:

Subtask 1 - Multi-Domain Classification of Library Records

Subtask 2 - Large-scale Multilabel Subject Indexing of Library Records

Important Dates

* Release of training data: March 8, 2025 * Release of testing data: May 30, 2025 * Deadline for system submissions: June 2, 2025 * Evaluation end: June 27, 2025 * Paper submission deadline: July 7, 2025 * Notification of acceptance: June 28, 2025 * Camera-ready paper due: August 15, 2025 * Workshop/KONVENS: September 10 - 12, 2025 (TBA)

Note: Submit your system outputs on our Codabench live leaderboard at https://www.codabench.org/competitions/8373/