November 2024 - Corpora

Text2Story@ECIR'25 Call for Papers
by Hugo Sousa 21 Nov '24

21 Nov '24

*** Apologies for cross-posting *** ++ CALL FOR PAPERS ++ **************************************************************************** Eighth International Workshop on Narrative Extraction from Texts (Text2Story'25) Held in conjunction with the 47th European Conference on Information Retrieval (ECIR'25) April 10th, 2025 – Lucca, Italy Website: https://text2story25.inesctec.pt **************************************************************************** ++ Important Dates ++ - Submission Deadline: January 24th, 2025 - Acceptance Notification: March 3rd, 2025 - Camera-ready copies: March 17th, 2025 - Workshop: April 10th, 2025 ++ Overview ++ For seven years, the Text2Story Workshop series has fostered a vibrant community dedicated to understanding narrative structure in text, resulting in significant contributions to the field and developing a shared understanding of the challenges in this domain. While traditional methods have yielded valuable insights, the advent of Transformers and LLMs have ignited a new wave of interest in narrative understanding. In the eighth edition of the Text2Story workshop, we propose to go deeper into the role of LLMs in narrative understanding exploring the issues involved in using LLMs to unravel narrative structures, while also examining the characteristics of narratives generated by LLMs. By fostering dialogue on these emerging areas, we aim to identify the wide-ranging issues related to the narrative extraction task and continue the workshop's tradition of driving innovation in narrative understanding research. ++ List of Topics ++ Research works submitted to the workshop should advance the scientific understanding of all aspects of narrative extraction from texts. This includes, but is not limited to, topics such as narrative information extraction, formal representation of narratives, narrative analysis and generation, development of datasets and evaluation protocols, as well as ethics and bias in narratives, and narrative applications. We encourage the submission of high-quality and original submissions covering the following topics and contributions focused on low and medium-resource languages. Narrative Information Extraction - Identification of Participants, Events and Temporal Expressions - Identification of Participants, Events and Temporal Expressions - Temporal Reasoning and Ordering of Events - Causality Detection - Big Data Applied to Narrative Extraction - LLMs for Narrative Extraction Narrative Representation - Annotation Protocols - Narrative Representation Models - Lexical, Syntactic, and Semantic Ambiguity in Narrative Representation - LLM-learned Representation Narrative Analysis and Generation - Discourse and Argument Structure Analysis - Narrative analysis of LLM generated text - Multilingual and Cross-lingual Narrative Analysis - Story Evolution and Shift Detection - Automatic Timeline Generation - Generative Language Models for Narrative Generation Datasets and Evaluation Protocol - Evaluating LLM-Generated Narratives - Evaluation of Multimodal Narrative Models - Annotated datasets - Narrative Resources - Using LLMs for Data Creation and Augmentation Ethics and Bias in Narratives - Identifying and Mitigating Bias in Generated Narratives - Ethical and Fair Narrative Generation - Misinformation and Fact Checking - Bias in LLM-generated narratives Narrative Applications - Narrative-focused Search in Text Collections - Narrative Summarization - Narrative Q&A - Multimodal Narrative Summarization - Multimodal Narrative-focused Search - Sentiment and Opinion Detection in Narratives - Social Media Narratives - Narrative Text Simplification - Narrative-based Text Anonymization - Personalization and Recommendation of Narratives - Storyline Visualization (including multimodal) and Narrative Structures ++ Submission Guidelines ++ We solicit the following types of contributions: -> Full papers (up to 8 pages + references): Original and high-quality unpublished contributions to the theory and practical aspects of the narrative extraction task. Full papers should introduce existing approaches, describe the methodology and the experiments conducted in detail. Negative result papers to highlight tested hypotheses that did not get the expected outcome are also welcomed. -> Short papers (up to 5 pages + references): Unpublished short papers describing work in progress; position papers introducing a new point of view, a research vision or a reasoned opinion on the workshop topics; and dissemination papers describing project ideas, ongoing research lines, case studies or summarized versions of previously published papers in high-quality conferences/journals that is worthwhile sharing with the Text2Story community, but where novelty is not a fundamental issue. -> Demos | Resource Papers (up to 5 pages + references): Unpublished papers presenting research/industrial demos; papers describing important resources (datasets or software packages) to the text2story community; Submissions will be refereed through a single-blind peer-review process by three reviewers with final acceptance decisions made by all the workshop organizers. The accepted papers will appear in the proceedings published at CEUR workshop proceedings (indexed in Scopus and DBLP) as long as they don't conflict with previous publication rights. ++ Workshop Format ++ Participants of accepted papers will be given 15 minutes for oral presentations. ++ Invited Speakers ++ Jochen L. Leidner, Coburg University of Applied Sciences, Germany Mirella Lapata, University of Edinburgh, Scotland ++ Organizing committee ++ Ricardo Campos (INESC TEC; University of Beira Interior, Covilhã, Portugal) Alípio M. Jorge (INESC TEC; University of Porto, Portugal) Adam Jatowt (University of Innsbruck, Austria) Sumit Bhatia (Media and Data Science Research Lab, Adobe) Marina Litvak (Shamoon Academic College of Engineering, Israel) ++ Proceedings Chair ++ João Paulo Cordeiro (NOVA Lincs & University of Beira Interior, Covilhã, Portugal) Conceição Rocha (INESC TEC, Portugal) ++ Web and Dissemination Chair ++ Hugo Sousa (INESC TEC & University of Porto, Portugal) Behrooz Mansouri (University of Maine, USA) ++ Program Committee ++ Ali Salehi (University at Buffalo) Arian Pasquali (Faktion AI) Andreas Spitz (University of Konstanz) Antoine Doucet (Université de La Rochelle) António Horta Branco (University of Lisbon) Bart Gajderowicz (University of Toronto) Behrooz Mansouri (Rochester Institute of Technology) Brenda Santana (Federal University of Rio Grande do Sul) Brucce dos Santos (Computational Intelligence Laboratory (LABIC) - ICMC/USP) Bruno Martins (IST & INESC-ID, University of Lisbon) David Semedo (Universidade NOVA de Lisboa) Dennis Aumiller (Cohere) Dhruv Gupta (Norwegian University of Science and Technology) Evelin Amorim (INESC TEC) Sérgio Matos (University of Aveiro) Florian Boudin (Nantes University) Henrique Lopes Cardoso (LIACC & University of Porto) Irina Rabaev (Shamoon College of Engineering) Ismail Altingovde (Middle East Technical University) Junbo Huang (University of Hamburg) Jakub Piskorski (Polish Academy of Sciences) João Paulo Cordeiro (Nova lincs & University of Beira Interior) Jin Zhao (Brandeis University) Luca Cagliero (Politecnico di Torino) Ludovic Moncla (INSA Lyon) Luis Filipe Cunha (INESC TEC & University of Minho) Marc Finlayson (Florida International University) Marc Spaniol (Université de Caen Normandie) Moreno La Quatra (Kore University of Enna) Nianwen Xue (Brandeis University) Nuno Guimarães (INESC TEC & University of Porto) Paulo Quaresma (Universidade de Évora) Paul Rayson (Lancaster University) Purificação Silvano (CLUP & University of Porto) Ross Purves (University of Zurich) Sérgio Nunes (INESC TEC & University of Porto) Sriharsh Bhyravajjula (University of Washington) Udo Kruschwitz (University of Regensburg) Valentina Bartalesi (ISTI-CNR, Italy) Yangyang Chen (Brandeis University) ++ Contacts ++ Website: https://text2story25.inesctec.pt For general inquiries regarding the workshop, reach the organizers at: text2story2025(a)easychair.org

1 0

Shared Tasks on AI-Generated Content (text and image) Detection @ DEFACTIFY - AAAI 2025
by Amitava Das 20 Nov '24

20 Nov '24

Apologies for the multiple postings. ------------------------------------- *Motivation:* A report by the European Union Law Enforcement Agency predicts that by 2026, up to 90% of online content could be synthetically generated, raising concerns among policymakers, who cautioned that ”*Generative AI could act as a force multiplier for political disinformation. The combined effect of the generative text, images, videos, and audio may surpass the influence of any single modality*”. In response, California’s Bill AB 3211 mandates the watermarking of AI-generated images, videos, and audio. However, concerns remain regarding the vulnerability of invisible watermarking techniques to tampering and the potential for malicious actors to bypass them entirely. Therefore, AI-generated content detection has become an essential research problem. https://defactify.com/ Shared Task 1: CT2: AI-Generated Text Detection Colab: https://codalab.lisn.upsaclay.fr/competitions/20330 A snapshot of the data can be viewed here <https://huggingface.co/datasets/gsingh1-py/train>. Shared Task 2: CT2: AI-Generated Image Detection Colab: https://codalab.lisn.upsaclay.fr/competitions/20331 A snapshot of the data can be viewed here <https://huggingface.co/datasets/NasrinImp/Defactify4_Train> Please take part in the shared tasks if you are interested in AI-generated content detection. Thanks, A *Prof. (Dr.) Amitava Das* Research Associate Professor Artificial Intelligence Institute of the University of South Carolina <http://aiisc.ai> Web <http://www.amitavadas.com/> | LinkedIn <http://:%20https:/www.linkedin.com/in/prof-dr-amitava-das-78348022/> | Google Scholar <https://scholar.google.com/citations?hl=en&user=HYpfhaEAAAAJ> Advisory Scientist Wipro AI Lab <https://wiprodigital.com/> Bangalore, India [image: Mailsuite] Sent with Mailsuite · Unsubscribe <https://mailtrack.io/en/privacy/opt-out/unsubscribe/646a9eaa87f3fb95b71d0ae…> 20/11/24, 12:42:56 pm

1 0

PerAnsSumm 2025 Shared Task - CL4Health Workshop @ NAACL 2025 - Call for Participation
by sagarw38＠uic.edu 20 Nov '24

20 Nov '24

Introducing the PerAnsSumm: Perspective-aware Healthcare answer summarization Shared Task at the CL4Health Workshop colocated with NAACL 2025! Website: peranssumm.github.io Motivation: Healthcare community question-answering (CQA) forums are rich with diverse user perspectives ranging from personal experiences and factual advice to thoughtful suggestions. However, traditional summarization methods focus on a single "best" answer, overlooking this diversity in perspectives. Tasks: - Task A: Identifying and classifying perspective-specific spans in answers. - Task B: Generating structured, perspective-specific summaries for question-answer threads. Key Dates: - Registration and Training Data Live now! - Release of test data: 25th January, 2025 - Submission of Results: 1st February, 2025 Learn More & Join the Shared Task: - More information: peranssumm.github.io - CodaBench Registration page: https://www.codabench.org/competitions/4312/ Contact us via email at sagarw38(a)uic.edu

1 0

Data Statements schema version 3 & guide now available
by Emily M. Bender 20 Nov '24

20 Nov '24

Dear all, A Guide for Creating and Documenting Language Datasets with Data Statements Schema Version 3 (Angelina McMillan-Major and Emily M. Bender, 2024) is now available! You can find the Guide along with templates for data statements and other resources here: https://techpolicylab.uw.edu/data-statements/ Data Statements schema version 3 builds on McMillan-Major's 2023 dissertation Language Dataset Documentation Design: Learning from Deaf and Indigenous Communities <http://hdl.handle.net/1773/50854>. The updated guide serves as both both as a resource for writing data statements and as an aid for planning data collection. Emily -- Emily M. Bender (she/her) Department of Linguistics Thomas L. and Margo G. Wyckoff Endowed Professor University of Washington Fedi: @emilymbender@dair-community.social

1 0

First call for papers: The 10th Workshop on Noisy and User-generated Text (WNUT 2025)
by Rob van der Goot 20 Nov '24

20 Nov '24

The WNUT Workshop will be collocated with NAACL 2025 (Albuquerque, New Mexico). The website for the workshop is at: http://noisy-text.github.io/ The WNUT workshop focuses on core NLP tasks (e.g., POS/NER tagging and translation; not computational social science) over user-generated text, such as that found on social media, web forums, online reviews, digital health records, or language learner essays. We seek submissions of long and short papers on original and unpublished work (same format and page limit as NAACL main conference). All accepted submissions will be presented as posters. Additionally, selected submissions will be presented orally. There will be best paper awards for both short and long papers. Topics of interest include but are not limited to: * NLP of noisy text, e.g. POS, NER tagging, Parsing * Text normalization and error correction * Paraphrase identification and semantic similarity of short text or noisy text * Extracting user demographics, profiles, and major life events * Machine translation and Multilingual NLP over noisy text * Information extraction from noisy text, global and regional trend detection, and event extraction * Colloquial language, e.g. idiom detection * Domain adaptation to user-generated text * Detecting rumors, contradictory information, sarcasm and humor on social media * Sentiment analysis * Temporal aspects of user-generated content (resolving time expressions, concept drift, etc...) * Representing and mining language variation in user-generated content * Processing of automatically generated data [IMPORTANT DATES] * Submission Deadline: January 30, 2025 (anytime on earth; dual-submission allowed) * Final ARR Submission Date: February 15, 2025 * ARR Commitment Date: February 20, 2025 * Acceptance Notification: March 1, 2025 * Camera-Ready Deadline: March 10, 2025 * Workshop Day: May 3 or 4, 2025 (TBD) [INVITED SPEAKERS] * Su Lin Blodgett * Verena Blaschke [ORGANIZERS] * JinYeong Bak (SungKyunKwan University) * Wei Xu (Georgia Institute of Technology) * Alan Ritter (Georgia Institute of Technology) * Rob van der Goot (IT University of Copenhagen) * Hyeju Jang (Indiana University) * Weerayut Buaphet (Vidyasirimedhi Institute of Science and Technology) [SUBMISSION] Submissions should conform to the ACL style guidelines. Long and short paper submissions must be anonymized. Please submit your papers via OpenReview: https://openreview.net/group?id=aclweb.org/NAACL/2025/Workshop/WNUT or commit them through ARR: https://openreview.net/group?id=aclweb.org/NAACL/2025/Workshop/WNUT_ARR_Com…

1 0

Call for Abstracts: FOM@Play Conference 2–5 September 2025 on Migration, Identity, and Transnational Discourses
by PASCUAL FRANCISCO PEREZ PAREDES 20 Nov '24

20 Nov '24

Call for Abstracts: FOM@Play Conference on Migration, Identity, and Transnational Discourses Dear Colleagues, We are excited to announce the FOM@Play Conference: Migration, Identity, and Transnational Discourses, to take place 2–5 September 2025 in Granada, Spain. Organized by the University of Granada, this conference offers a platform for interdisciplinary dialogue on migration, identity, and the lived experiences of transnational communities in Europe and beyond. The conference is an activity of the FOM@Play project,<http://www.um.es/fomatplay> funded by the European Commission, which aims to shed light on the complexities of migration, the concept of freedom of movement, and how these shape collective and individual identities. The discussions will focus on the intersections of migration policies, societal narratives, and personal journeys, fostering a deeper understanding of transnational discourses in an era of globalization and socio-political change. FOM@Play is not just a conference but a forum for meaningful exchange among researchers, practitioners, and policymakers. Attendees will have opportunities to network, present cutting-edge research, and engage in thought-provoking discussions about the future of migration and identity in Europe. We invite scholars from diverse fields, including but not limited to: - Migration studies - Discourse analysis and sociolinguistics - Political science and international relations - Cultural studies and anthropology - Sociology and psychology Topics of interest include (but are not limited to): - Narratives of migration in media and politics - The role of freedom of movement in shaping transnational identities - The impact of migration policies on social cohesion and inclusion - The representation of migrants in European cultural production - Methodological advancements in studying migration and identity (e.g., corpus linguistics, critical discourse analysis) Important Dates - Abstract submission deadline: 21 March 2025 - Notification of acceptance: 7 April 2025 - Early registration deadline: 7 May 2025 For detailed submission guidelines and further information, please visit our website: https://www.granadacongresos.com/fomaplay For any inquiries, feel free to contact fomatplay(a)granadacongresos.com<mailto:fomatplay@granadacongresos.com> We look forward to your contributions and hope to see you in Granada! Warm regards, Prof Pascual Pérez-Paredes On behalf of the FOM@Play Organising Committee

1 0

Working Group on Metadata in Learner Corpus Research
by Magali Paquot 20 Nov '24

20 Nov '24

Dear colleagues, As announced at LCR2024, we are pleased to confirm the formation of a new Working Group on Metadata in Learner Corpus Research, established under the aegis of the Learner Corpus Association (LCA) and in collaboration with the CLARIN K-centre for Learner Corpora (https://uclouvain.be/en/research-institutes/ilc/clarin-knowledge-centre-for…). This initiative follows the successful release of the second version of the Core Metadata Schema for Learner Corpora (LC-meta), a milestone that reflects our collective commitment to advancing the field. To understand the rationale behind LC-meta, we invite you to read our recently published open-access article in the International Journal of Learner Corpus Research: https://doi.org/10.1075/ijlcr.24010.paq. Alternatively, if you prefer listening to a lecture, you can tune into a talk given by M. Paquot at the Corpus Linguistics and Applied Linguistics 2023 online seminar series: https://www.um.es/languagecorpora/2023/09/14/the-core-metadata-schema-for-l…. The schema itself is downloadable at: https://doi.org/10.14428/DVN/AAUEM2. The Working Group will focus on evaluating and refining the schema, promoting its adoption, and addressing metadata-related challenges in learner corpus research. It will also work towards integrating the schema into relevant infrastructures and developing user-friendly interfaces that enhance accessibility. We believe that by working together, we can significantly enhance the quality, accessibility, and impact of learner corpora. As a first step, we invite interested colleagues to express their interest in joining the working group by contacting us at kc-l2corpora(a)uclouvain.be<mailto:kc-l2corpora@uclouvain.be> by January 20. We extend our thanks to everyone who has already reached out and will be in touch shortly. Due to busy schedules over the coming months, we have decided to schedule the first WGT meeting for early spring 2025. This additional time will allow us to better organize the event and ensure we can share updates on accessibility and user-friendliness. In the meantime, we would greatly appreciate hearing from colleagues who have already attempted to use the schema in new corpus compilation projects. If that is the case, we would be delighted to discuss your experiences with you! We look forward to your active participation and to collectively driving forward this important aspect of learner corpus research. Best regards, Magali Paquot, Jennifer-Carmen Frey, Alexander König, and Egon Stemle

2 1

[Call For Paper] COLING-2025 - FinNLP-FNP-LLMFinLegal Workshop
by CHUNG-CHI CHEN 20 Nov '24

20 Nov '24

Dear colleague, (We apologize if you receive this email multiple times from different mailing lists.) We are excited to invite you to submit your paper to the FinNLP-FNP-LLMFinLegal Workshop at COLING-2025 in Abu Dhabi, UAE! This event brings together three workshops in the fields of financial technology, narrative processing, and the emerging role of large language models in finance and legal domains. The submission deadline is Nov. 25th, 2025. Below are more details of the call for papers. Please visit our website for more details ( https://sites.google.com/nlg.csie.ntu.edu.tw/finnlp-fnp-llmfinlegal/). We have 5 shared tasks and will have 5 invited speakers for this event. Looking forward to your participation. Best Regards, FinNLP-FNP-LLMFinLegal Organizers Chung-Chi Chen, Antonio Moreno-Sandoval, Qianqian Xie, Jimin Huang, Sophia Ananiadou, Hsin-Hsi Chen *Topics of Interest*We invite submissions of original contributions on methods, theories, applications, and systems on artificial intelligence, machine learning, natural language processing & understanding, big data, statistical learning, data analytics, and deep learning, with a focus on knowledge discovery in the financial services domain. The scope of the workshop includes, but is not limited to, the following areas: - Language modeling on financial corpora, including tabular and numerical data, and multi-modal modeling - Graph representation learning and mining on financial data - Multi-source knowledge integration and fusion - Synthetic and genuine financial datasets and benchmarks - Transfer learning applications for financial data - Financial search and question answering systems - Event discovery and impact on organizational equity price - ESG event discovery, evaluation, and impact assessment - Compliance monitoring - Cross-disciplinary LLM-based methodologies for financial and legal domains - Applications of LLMs in financial auditing and regulatory reporting - Ethical implications and bias mitigation in AI applications - Hallucination mitigation and evaluation in LLMs - Privacy concerns and data protection strategies - Enhancing interpretability and explainability of LLM models - Responsible AI practices and governance - Methods, evaluation metrics, benchmarks, and datasets for LLMs in finance and law *Submission Details* - *Submission Deadline: November 25th, 2024 (Time zone: Anywhere On Earth (AOE)) * - *Submission System: https://softconf.com/coling2025/FinNLP25/ <https://softconf.com/coling2025/FinNLP25/>* - Paper Notification: December 5th, 2024 - Camera-Ready Deadline: December 13th, 2024 - FinNLP-FNP-LLMFinLegal-2024: Jan 19-20, 2025 (Two-Day Workshop) Format - The ACL Template MUST be used for your submission(s).* Accepted papers proceedings will be published at ACL Anthology.* - Long Paper: May consist of up to 8 pages of content, plus unlimited pages for references and appendix. - Short Paper and Demo Paper: May consist of up to 4 pages of content, plus unlimited references and appendix. Policies - The reviewing process will be double-blind for Long and Short Paper, and single-blind for Demo Paper. Submissions must be in electronic form using the FinNLP-FNP-LLMFinLegal-2024 paper submission link above. - At least one author of each accepted paper should register and present their work in FinNLP-FNP-LLMFinLegal-2024.

1 0

Faculty Search at UT Austin Computational Linguistics
by Jessy Li 19 Nov '24

19 Nov '24

Dear all, We are hiring for two faculty positions in UT Austin Linguistics, in the area of computational linguistics. One assistant professor position, the other can be either assistant or associate. Deadline: December 1. See the ad here: https://apply.interfolio.com/158280 and below. Feel free to reach out to me or my colleague (and search chair) Jessy Li jessy(a)austin.utexas.edu<mailto:jessy@austin.utexas.edu> if you have questions! *************************************************** ad: https://apply.interfolio.com/158280 Description The Department of Linguistics at The University of Texas at Austin invites applications for two positions in computational linguistics to begin Fall of academic year 2025-26. One position will be at the Assistant Professor rank, and one can be at the rank of Assistant Professor or Associate Professor with tenure. We seek candidates whose work develops computational methods in any area of linguistics and specifically connects linguistics to artificial intelligence, with particular interest in candidates who can interface with any of our current major areas of research in syntax, semantics, pragmatics, sounds, psycholinguistics, cognitive science, language documentation, historical linguistics, sign language linguistics, or sociolinguistics. Duties will include research, service, and teaching linguistics courses at all levels (lower- and upper-division undergraduate, and graduate), and training the next generation of researchers working in AI, natural language processing and machine learning, and computational methods for language. Qualifications Applicants must hold a Ph.D. in linguistics, computer science, cognitive science, or a related field by the time of appointment and have demonstrated a strong research and teaching trajectory. Application Instructions Applicants must upload a letter of application, a CV, a statement describing their research program, a statement of teaching interests (including a list of courses that the applicant would be prepared to teach), evidence of past teaching performance or teaching potential, three letters of recommendation, and three writing samples. The search committee will begin reviewing applications on December 1, 2024, and continue until the position is filled. Salary is competitive and commensurate with experience and qualifications. Position funding is subject to budget availability. For further information, please contact: Prof. Jessy Li, Search Committee Chair Email:jessy@austin.utexas.edu<mailto:Email%3Ajessy@austin.utexas.edu>

1 0

[CFP] SemEval 2025 Task 5 - LLMs4Subjects - 2nd Call for Shared Task Participation
by Jennifer D'Souza 19 Nov '24

19 Nov '24

Libraries are at the heart of our society and education, so we believe in driving technological innovation in these age-old institutions as well as modern digital libraries when we can! If you're passionate about bringing AI and LLMs into library workflows, join computer science researchers and AI enthusiasts worldwide in the LLMs4Subjects shared task! *We are excited to announce the 2nd Call for Participation to the LLMs4Subjects Shared Task organized as part of SemEval 2025.* *Overview:* As the first of its kind, LLMs4Subjects invites the research community to *develop cutting-edge LLM-based semantic solutions for the subject tagging of the Leibniz University's Technical Library's open-access collection*. The shared task provides an opportunity for the research community to creatively utilize LLMs for subject tagging of technical records. *Systems need to demonstrate bilingual language modeling in understanding technical documents in both German and English.* Moreover, successful solutions may be directly integrated into the operational workflows of the TIB Leibniz Information Centre for Science and Technology University Library. *What we provide to participants:* a human-readable form of a subject's taxonomy (this is the *GND* or *Gemeinsame Normdatei*, the integrated authority file used for cataloging in German-speaking countries) and a large collection of technical records tagged with these subjects from the TIB's open-access collection called *TIBKAT*. More details on the task website: https://sites.google.com/view/llms4subjects/ *LLMs4Subjects defines the following three tasks:* - Task 1: Learn the GND - Task 2: Align subject tagging to the TIBKAT collection - (Optional and Fun) Task 3: Develop Elegant Frontend Interfaces for Subject Tagging *LLMs4Subjects will have three separate evaluations:* - Evaluation 1: Quantitative Metrics-based Evaluations - Evaluation 2: Qualitative Evaluations by the Human Subject Specialists - (Optional) Evaluation 3: HCI evaluations for subject indexing interfaces submitted *To participate in the LLMs4Subjects shared task,* 1. please submit your interest to participate using our online form ( https://forms.gle/YQzupcoySAyJi45c6), 2. sign up to the shared task Google Groups ( https://groups.google.com/u/6/g/llms4subjects) for FAQs, news, and announcements, and, 3. last but not the least, download the datasets ( https://github.com/jd-coderepos/llms4subjects/) to begin development. *Dates* Training and validation datasets available:October 2, 2024 Test data available/Evaluation starts: January 10, 2025 Evaluation ends: January 31, 2025 Participant paper submissions due: February 28, 2025 Notification to authors: March 31, 2025 Camera ready due: April 21, 2025 SemEval workshop: TBD *Task Organizers* Jennifer D'Souza, Sameer Sadruddin, Holger Israel, Mathias Begoin et al. All organizers are affiliated with the TIB Leibniz Information Centre for Science and Technology - Germany (https://www.tib.eu/en/) *We look forward to having you on board!* *Contact: *llms4subjects [at] gmail.com

1 0

2025

2024

2023

2022

Corpora November 2024