Libraries are at the heart of our society and education, so we believe in driving technological innovation in these age-old institutions as well as modern digital libraries when we can! If you're passionate about bringing AI and LLMs into library workflows, join computer science researchers and AI enthusiasts worldwide in the LLMs4Subjects shared task!
**We are excited to announce the 3rd Call for Participation to the LLMs4Subjects Shared Task organized as part of SemEval 2025.**
Calling all researchers interested to explore research objectives or new research questions around which to test LLMs. Questions such as: To what extent do LLMs pass semantic subject comprehension tests given a technical context? How effectively do they function in bilingual contexts, specifically German and English? What strategies can enhance their capability as bilingual semantic comprehension models?
If these questions spark your interest, we invite you to participate in the LLMs4Subjects shared task at SemEval 2025.
**TL;DR Overview – LLMs4Subjects SemEval 2025 Shared Task:**
LLMs4Subjects marks the first shared task of its kind, challenging participants to develop innovative LLM-based solutions for subject tagging of technical records from Leibniz University’s Technical Library (TIBKAT). The task requires systems to process documents in both German and English using the GND taxonomy. Successful implementations may be integrated into TIB’s operational workflows, enhancing the Leibniz Information Centre for Science and Technology.
For more details, visit our website: https://sites.google.com/view/llms4subjects/
Why Participate? • Explore critical LLM research areas like prompting, fine-tuning, RAG techniques, agentic workflows, and more! 🔍 • The SemEval meeting is held is conjunction with premier NLP conferences like ACL, EMNLP, or COLING. Hence participants will have the opportunity to publish a paper, present and discuss your work at one of these venues! 🗣️ • Your solutions could be integrated into the workflows of the TIB Leibniz Information Centre! 🏛️
How to Participate: 1. please submit your interest to participate using our online form (https://forms.gle/YQzupcoySAyJi45c6), 2. sign up to the shared task Google Groups (https://groups.google.com/u/6/g/llms4subjects) for FAQs, news, and announcements, and, 3. last but not the least, download the datasets (https://github.com/jd-coderepos/llms4subjects/) to begin development.
Participants have a month remaining to develop solutions, and we welcome hackathon-style proofs-of-concept to showcase promising directions. The official evaluation phase begins in January 2025.
Don’t wait – get started today! And please, share this with your network if you think it might interest others.