- Corpora - ELRA lists

Postdoc position in Cambridge (deadline 25/1)
by A. Vlachos 09 Jan '24

09 Jan '24

Dear all, We are looking for a Research Assistant/Associate to work on the project FEVER-IT: Fact Extraction and VERification with Images and Text funded by the Alan Turing Institute and DSO labs, Singapore. The successful candidate will be based in the Natural Language and Information Processing group (http://www.cl.cam.ac.uk/research/nl/) at the Department of Computer Science and Technology. The project will focus on constructing a dataset and developing approaches enabling the verification of claims which require both text and images as evidence. Particular focus will be paid to accompanying the verdicts with suitable justifications. Candidates will have completed a Ph.D. (or be close to completing it) in a relevant field such as NLP, Information Retrieval, Artificial Intelligence or Machine Learning and be able to demonstrate a strong track record of independent research and high-quality publications. Essential skills include excellent programming (Python), NLP techniques (multimodal NLP in particular), Machine Learning, and proven communication skills. Candidates must provide the names and contact details of two referees who are familiar with their work in the relevant field whom we can contact for a reference before the interviews, which are expected to take place in January 2024. We are looking to start the project in February 2024, or as soon as possible after that. Enquiries concerning this position should be directed to Prof. Andreas Vlachos (av308(a)cam.ac.uk), and applicants are encouraged to contact him regarding the position. Apply using this link: https://www.jobs.cam.ac.uk/job/44672/ Thanks, Andreas

1 0

Call for Participation: Learner Corpus Symposium (Feb 3, Kobe, Japan)
by Shin ISHIKAWA 09 Jan '24

09 Jan '24

Dear Colleagues We are happy to announce that Learner Corpus Studies in Asia and the World (LCSAW) 6th will be held in Kobe, Japan, on February 3, 2024. We have 24 presentations. Participation at the event is free of charge, but prior registration is needed. Program https://language.sakura.ne.jp/icnale/lcsaw/lcsaw6_program.pdf Venue Centennial Hall of Kobe University https://x.gd/N1ji1 Transportation https://language.sakura.ne.jp/icnale/lcsaw/2019/lcsaw_travel_guide/pdf Registration https://forms.office.com/r/nU132uW5tc NB: FCFS policy will be applied. Shin LCSAW Organizing Committee ICNALE Development Team Dr. Shin ISHIKAWA (he/his) Kobe University iskwshin(a)gmail.com

1 0

ArabicNLP 2024 - Second Call for Shared Task Proposals
by Salam Khalifa 09 Jan '24

09 Jan '24

*** Apologies for Cross-Posting *** The Second Arabic Natural Language Processing Conference (ArabicNLP 2024) Co-located with ACL 2024 in Bangkok, Thailand, August 16, 2024. (Hybrid Mode). We invite proposals for shared tasks related to Arabic NLP to be part of the ArabicNLP 2024 conference. <https://arabicnlp2024.sigarab.org/> The proposals should provide an overview of the proposed task, motivation, data/resource collection and creation, task description, pilot run details (if available), a tentative timeline that matches the submission dates below, and task organizers (name, email, affiliation). Proposals in PDF format can be up to 4 pages. Shared Task Proposal Submission URL: https://shorturl.at/eCJOS Selection Process The proposals will be reviewed by the organizing committee and selected based on multiple factors such as the novelty of the task, the expected interest from the community, how convincing the data collection plans are, the soundness of the evaluation method, and the expected impact of the task. Task Organization Upon acceptance, the task organizers are expected to verify that the task organization and data delivery to participants are happening in a timely manner, provide the participants with all needed resources related to the task, create a mailing list, and maintain communication and support to participants, create and manage CodaLab or similar competition website, manage submissions to CodaLab, write a task description paper, manage participants submissions of system description papers, and review and maintain the quality of submitted system description papers. Important Dates for Shared Task Proposals - January 23, 2024: Submission of shared tasks proposals due date - February 6, 2024: Notification of acceptance of shared tasks - Proposals should target the following dates when planning their calls - April 29, 2024: Shared task papers due date - June 4, 2024: Notification of acceptance - June 24, 2024: Camera-ready papers due - August 16, 2024: ArabicNLP conference All deadlines are 11:59 pm UTC -12h <https://www.timeanddate.com/time/zone/timezone/utc-12> (“Anywhere on Earth”). For any questions, please contact the Shared Task Chair: arabicnlp-shared-task-chair(a)sigarab.org The ArabicNLP 2024 Organizing Committee -- Salam Khalifa

1 0

PhD vacancy in Formal Methods for Natural Language Processing at Leiden University, The Netherlands
by Wijnholds, G.J. (Gijs) 09 Jan '24

09 Jan '24

** Apologies for cross-posting ** Applications are invited for a fully funded PhD candidature in at Leiden Institute of Advanced Computer Science (LIACS), The Netherlands, on the use of Formal Methods to enhance the efficiency, transparency and the understanding of Transformer-based language models. While large language models (LLMs) have proven successful in many areas of Natural Language Processing, they suffer from high data and resource usage, and display limited generalization capacity in tasks that humans excel at. In this PhD project you will have the opportunity to investigate how formal methods can help in developing more efficient and more transparent models for Natural Language Understanding. Specifically, you will investigate the use of implicit or explicit structural bias in Transformer-based language models to reduce training data and model parameters; additionally, you will look at novel techniques for evaluating models for their generalization capabilities on Natural Language Understanding tasks such as Natural Language Inference, possibly in a multilingual and multimodal setting. The specific project content is to be decided between the applicants’ interest and the expertise of the supervisor, dr. Gijs Wijnholds. Topics include (but are not limited to): - Using logical methods to define task-relevant constraints on LLM finetuning; - Incorporating structured representations in regularized training of smaller language models; - Assessing the generalization capacity of Transformer-based models in the context of formal language theory, model probing, Natural Language Inference; - Evaluation of Natural Language Understanding models in the presence of ambiguity and/or annotator disagreement; - Understanding multilingual Natural Language Inference in Vision-Language Models; For the official vacancy text, please see https://www.universiteitleiden.nl/en/vacancies/2023/qw4/23-81914335phd-cand… The application deadline is January 13, 2024. The ideal starting date is in March 2024, but can be negotiated depending on circumstances. For further information feel free to reach out to dr. Gijs Wijnholds (g.j.wijnholds(a)liacs.leidenuniv.nl<mailto:g.j.wijnholds@liacs.leidenuniv.nl>). dr. Gijs Wijnholds Assistant Professor in Natural Language Processing Text Mining and Retrieval Group<https://tmr.liacs.nl/> Leiden Institute of Advanced Computer Science https://gijswijnholds.github.io

1 0

Call for Tutorial Proposals at ICWSM 2024 (International AAAI Conference on Web and Social Media)
by Indira Sen 08 Jan '24

08 Jan '24

Hi everyone! Sharing the call for ICWSM-2024’s Tutorials Track! This is a great venue for tutorials at the intersection of NLP, Computational Social Science, Social Media Analysis, and Social Computing topics. *Tutorial Submission Deadline*: January 26, 2024 *Tutorial Acceptance Notification*: February 9, 2024 *ICWSM-2024 Tutorial Day*: June 3, 2024 *Submit to*: tutorials(a)icwsm.org ICWSM-2024 invites proposals for Tutorials Day at the 18th International AAAI Conference on Web and Social Media (ICWSM). ICWSM-2024 is seeking proposals for tutorials on topics related to the computational analysis and understanding of social phenomena in the following formats: - *Lecture-style*: Traditional tutorials to teach concepts, methodologies, tools, and software packages. Tutorials on novel and fast growing directions and significant applications are highly encouraged. The conference is paying particular attention to themes around new perspectives in social theories, as well as computational algorithms for analyzing new forms of social media. Lecture-style tutorials on these themes are highly encouraged. - *Hands-on*: Interactive, in-depth, hands-on training on cutting edge systems and tools (with a proven track record in the community), targeted at novice as well as moderately skilled users, with a focus on providing an engaging experience. The pace of the tutorial should be set such that beginners can follow along comfortably. - *Translation*: Tutorials that aim to translate concepts between disciplines. For example, such a tutorial could introduce social science concepts to computer scientists, or computational concepts to social scientists. Thus, these tutorials should be geared towards a beginner audience. - *Case study*: Focused tutorials that emphasize real world applications of ICWSM work. These tutorials should walk the audience through how research insights and tools were applied in practice. We welcome submissions from practitioners in industry, government, local communities, and non-governmental organizations (NGOs) in addition to academics. - *Free-style*: We also welcome proposals for creative and unconventional training sessions, such as hackathons, competitions/challenges, etc., as long as participants can learn practical skills and participate in an active way. We welcome tutorials of various lengths (1, 2, 4, or up to 8 hours). We are looking for contributions from experts in both the social and computational sciences, in industry, academia, and beyond. For a list of tutorials from previous years, we encourage you to visit the tutorials page for 2018 <https://www.icwsm.org/2018/program/tutorial/>, 2019 <http://www.icwsm.org/2019/program/tutorial/>, 2020 <https://www.icwsm.org/2020/index.html#tutorials_schedule>, 2021 <https://www.icwsm.org/2021/#tutorials_schedule>, 2022 <https://www.icwsm.org/2022/index.html/#tutorials-schedule>, and 2023 <https://icwsm.org/2023/index.html/#tutorials-schedule>. We especially encourage applications from first-time proposers and scholars with research communities beyond ICWSM. We also welcome tutorials on obtaining data from understudied platforms, the use of large language models (LLMs) for and their impact on computational social science and social computing, mixed methods approaches, and other topics of cutting-edge and enduring interest. *Tutorial Proposal Content and Format* Proposals for tutorials should be no more than three (3) pages in length. Proposal submissions should include the following information: - *Title*. A concise title. - *Abstract*. A short description (200 words) of the main objective of the tutorial, to be published on the main ICWSM website. - *Type*. The type of tutorial you are proposing: lecture-style, hands-on, translation, case study, or free-style. - *Names, affiliations, emails, and personal websites of the tutorial organizers*. A main contact author should be specified. A typical proposal should include no more than three presenters (more people can be involved in the organization). - *Duration*. A short timeline description of how you plan to break down the material over the proposed duration (1, 2, 4, or 8 hours). Please mention here the proposed duration, but keep in mind that the Tutorial Chairs might conditionally accept a proposal and suggest a different duration to best fit the organization of the whole event. - *Tutorial schedule and activities*. A description of the proposed tutorial format, a schedule of the proposed activities (e.g., presentations, interactive sessions) along with a *detailed* description for each of them. - *Target audience, prerequisites and outcomes*. A description of the target audience, the prerequisite skill set for the attendee (if any) as well as a brief list of goals for the tutors to accomplish by the end of the tutorial. - *Materials*. The organizers of accepted tutorials will be required to set up a web page containing all the information for the tutorial attendees before the tutorial day (roughly 2 weeks before the tutorial day). The proposal should contain the list of materials that will be made available on the website. - *Past precedent (when available)*. A list of other tutorials held previously at related conferences, if any, together with a brief statement on how the proposed tutorial relates to previous events. If the authors of the proposal have organized other tutorials in the past, pointers to the relevant material (e.g., slides, videos, web pages, code) should be provided. - *Additional info for hands-on tutorials*: 1. Operating system and required installed tools on attendees’ devices. 2. List of software licenses required for the tools. 3. Setup instructions for attendees. (The setup should not take more than 1 hour to complete.) If you have questions on any of the submission requirements or for pre-submission feedback/questions, please reach out to the tutorial chairs (Joshua Uyheng, Indira Sen, and Carlos Toxtli) at the address tutorials(a)icwsm.org. *Joshua Uyheng, Indira Sen, and Carlos Toxtli* (ICWSM-2024 Tutorial Chairs | tutorials(a)icwsm.org)

1 0

36th International Conference on Advanced Information Systems Engineering (CAiSE 2024): Second Call for Tutorial Proposals
by Announce 08 Jan '24

08 Jan '24

*** Second Call for Tutorial Proposals *** 36th International Conference on Advanced Information Systems Engineering (CAiSE'24) June 3-7, 2024, 5* St. Raphael Resort and Marina, Limassol, Cyprus https://cyprusconferences.org/caise2024/ (*** Submission Deadline: 28th February, 2024 AoE ***) CAiSE'24 invites proposals for tutorials on advanced topics in the field of Information Systems Engineering. Tutorials should aim at offering new insights, knowledge, and skills to professionals, educators, researchers, and students seeking to gain a better understanding either about methods of broad interest in the field, or emergent paradigms that are ripe for practical adoption or that require further research to reach maturity. Proposals emphasizing the special theme of the CAISE'24 conference “Information Systems in the Age of Artificial Intelligence” are encouraged, but proposals on other new or long-standing topics in information systems engineering are also welcome. Tutorials should be focused on principles, concepts, and methods. Commercial or sales-oriented presentations are not allowed and will not be accepted. Tutorials are intended to provide a pedagogic introduction to or overview of a topic of relevance. Potential presenters should keep in mind that there may be a heterogeneous audience, including novice graduate students, experienced practitioners, and specialized researchers. Tutorial speakers should be prepared to cope with this diversity in the audience. Tutorials will be 90 minutes long and organized in parallel with the technical sessions of the main conference and participants of the conference will have free access to all of them. Potential proposers are free to contact the tutorial chairs via e-mail to validate their idea prior to the submission. SELECTION CRITERIA The tutorial chairs will review each proposal and select a subset of them based on the following criteria: 1. relevance to the field of IS engineering; 2. anticipated appeal to the conference audience; 3. timeliness and importance for the conference audience; 4. past experience and qualifications of the instructor(s). The tutorial chairs will also consider the complementarity of the proposal w.r.t. the conference program and other tutorial proposals. SUBMISSION GUIDELINES Tutorial proposals should be submitted to Easy Chair using the conference submission site (https://easychair.org/conferences/?conf=caise2024) and then selecting the “CAiSE 2024 Tutorials” track. The proposal (length up to 1500 words) should cover the following points: • Title • Presenters and affiliation • Goal: The overall goal of the tutorial. • Scope: Intended audience, level (basic or advanced), and prerequisites. • Topic relevance and novelty: Specifically indicate the relevance to the scope of CAiSE, the relevance to practice, the novel aspects that would make this tutorial beneficiary and appealing to CAiSE participants. • Structure of contents: Here you should provide a structured overview of your planned tutorial, organized into numbered sections and subsections. For each subsection, you should sketch its contents in a few sentences or bullet points. • References: Provide references to papers, books, etc. that your tutorial builds on. Please specify previous venues at which similar tutorials have been presented by you and indicate the difference between the proposed tutorial and previous ones. CAiSE usually does not accept tutorials that have been presented in other venues. • Sample Slides: Include at least 5 sample slides of the presentation you plan to give if your tutorial is accepted. Select slides that are typical of your presentation style. These slides have to be submitted in a separate PDF file. Services provided to tutorialists • A 2-page tutorial abstract will be published in the CAiSE LNCS proceedings • Tutorials will benefit from the local organizational infrastructure (registration, badges, refreshments, beamers, screens, etc.). • Advertisement of the tutorial on CAISE 2024 homepage and mailings. • The conference fee will be waived for tutorial presenters (one fee per tutorial). IMPORTANT DATES • Submission of Tutorial Proposals: 28th February, 2024 (AoE) • Notification of Acceptance: 15th March, 2024 • Camera-ready Abstracts: 5th April, 2024 • Tutorial Presenters Registration Deadline: 8th April, 2024 TUTORIAL CHAIRS • Adela del Rio Ortega, University of Seville, Spain (adeladelrio(a)us.es) • Tiago Prince Sales, University of Twente, The Netherlands (t.princesales(a)utwente.nl) Other Committee Members https://cyprusconferences.org/caise2024/committees/

1 0

CFP 1st Workshop on Natural Scientific Language Processing and Research Knowledge Graphs
by Georg Rehm 08 Jan '24

08 Jan '24

1st Workshop on Natural Scientific Language Processing and Research Knowledge Graphs (NSLP 2024) 26 or 27 May 2024 (tbc) Hersonissos, Crete, Greece (co-located with ESWC 2024) https://nfdi4ds.github.io/nslp2024/ Scientific research is almost exclusively published in unstructured text formats, which are not readily machine-readable. While technological approaches can help to get this flood of scientific information and new knowledge under control, the development of such technologies is very complex in practice and hinders the creation of infrastructures and systems to track research and assist the scientific community with applications such as dedicated scientific search engines and recommender systems. The 1st Workshop on Natural Scientific Language Processing and Research Knowledge Graphs (NSLP) aims to bring together researchers working on the processing, analysis, transformation and making-use-of scientific language and RKGs including all relevant sub-topics. NSLP 2024 is a full-day workshop co-located with ESWC 2024 <https://2024.eswc-conferences.org/> to be held in Crete, Greece, in May 2024. The workshop will consist of two keynote speakers and two shared tasks, as well as presentations and posters of accepted papers. Topics of interest include, but are not limited to - Research/Scientific Knowledge Graphs (RKGs/SKGs) and other forms of Structured Scientific Knowledge Representation - Information Extraction for RKGs/SKGs - Question Answering over RKGs/SKGs - Scientific LLMs: LLMs for Natural Scientific Language Processing - Natural Scientific Language Processing (monolingual, cross-lingual, multilingual) - Language Resources and Language Technologies for Natural Scientific Language Processing - Information Extraction from Scholarly Publications - Classification of Scholarly Publications (document collections, individual documents, parts of documents) - Summarisation of Scholarly Articles - Scholarly Information Retrieval and Scientific Search Engines - Digital Libraries of Scholarly Information - Metadata and Cataloging - Bibliometrics and Scientometrics - Domain-specific Adaptation of Natural Language Processing (NLP) methods for NSLP purposes - Micropublications and Nanopublications Important dates - Deadline for submissions: March 7, 2024 - Notification of acceptance: April 4, 2024 - Deadline for camera-ready papers: April 18, 2024 Submissions The NSLP workshop invites submissions of regular long papers (up to 15 pages, Springer LNCS style), position papers, and short papers (up to 8 pages, Springer LNCS style) presenting negative results, in-progress projects, and demos. We especially encourage submissions from junior researchers and students from diverse backgrounds. We’ll attempt to publish the proceedings of the workshop in an Open Access book volume. Shared tasks The workshop will offer two shared tasks: - FoRC: Field of Research Classification of Scholarly Publications - SOMD: Software Mention Detection in Scholarly Publications The NSLP 2024 website <https://nfdi4ds.github.io/nslp2024/> provides more information on the shared tasks. Confirmed keynote speakers - Natalia Manola, OpenAIRE, Greece - Francesco Osborne, Open University, UK Organisers - Georg Rehm, DFKI, Germany - Sonja Schimmler, TU Berlin & Fraunhofer FOKUS, Germany - Stefan Dietze, GESIS & HHU Düsseldorf, Germany - Frank Krüger, Wismar University, Germany Contact - Georg Rehm <georg.rehm(a)dfki.de> -- [image: DFKI] <https://www.dfki.de/> *Prof. Dr. Georg Rehm <http://georg-re.hm/>* Principal Researcher and Research Fellow, DFKI Adjunct Professor, Humboldt-Universität zu Berlin DFKI GmbH <https://www.dfki.de/>, Alt-Moabit 91c, 10559 Berlin, Germany georg.rehm(a)dfki.de Deutsches Forschungszentrum für Künstliche Intelligenz GmbH Firmensitz: Trippstadter Strasse 122, D-67663 Kaiserslautern Geschäftsführung: Prof. Dr. Antonio Krüger (Vorsitzender), Helmut Ditzer Vorsitzender des Aufsichtsrats: Dr. Ferri Abolhassan Amtsgericht Kaiserslautern, HRB 2313

1 0

[ECAI-2024] Deadline for Workshop Proposals approaching (and more!)
by luis.magdalena＠gmail.com 08 Jan '24

08 Jan '24

Would you like to organise a workshop at ECAI-2024? The 27th European Conference on Artificial Intelligence will be held in the beautiful city of Santiago de Compostela during 19-24 October 2024, with workshops taking place during the first two days. Proposals from all subfields of AI and organisers of all levels of seniority are welcome. The deadline is fast approaching: Monday, 15 January 2024. Full details here: https://www.ecai2024.eu/calls/workshops Check the website for further opportunities to participate: submitting a tutorial proposal, submitting a paper to either the main conference or PAIS, submitting a demo paper, or taking part in the doctoral consortium. -- Luis Magdalena Publicity Chair of the European Conference on Artificial Intelligence (ECAI-2024)

1 0

1st CFP: 8th Workshop on Cognitive Aspects of the Lexicon
by Emmanuele Chersoni 08 Jan '24

08 Jan '24

*The 8th International Workshop on Cognitive Aspects of the Lexicon* https://sites.google.com/view/cogalex-viii-2024 *Co-located with LREC-COLING 2024* https://lrec-coling-2024.org/ <https://www.aacl2022.org/home> Turin, Italy, 20-25 May 2024 * Important Dates* Submission deadline: February 23, 2024 Date of notification: March 20, 2024 Camera ready deadline: March 29, 2024 CogALex workshop: May 20, 2024 *Meeting Description* The way we look at the lexicon has changed dramatically over the last few decades. While in the past being considered as an appendix to grammar, the lexicon has now moved to the center stage. Indeed, there is hardly any task in NLP that can be conducted without it. Also, many new proposals have emerged during the last few years. Living in a fast-moving world, it is hard for anyone to stay on top of the wave. Hence the reason for organizing events like this. The goal of this workshop is to provide builders and users of lexical resources (researchers in NLP, psychologists, computational lexicographers) a forum to share their knowledge and needs concerning the construction, organization, and use of a lexicon by people (lexical access) and machines (NLP, IR, data mining). Like in the past, we invite researchers to address unsolved problems concerning the lexicon, by considering this time however also Large Language Models (LLMs). More precisely, we would like to explore their potential for building and using lexical resources as well as their ability to shed new light on the cognitive aspects of the lexicon. We solicit contributions including, but not limited to, the topics listed below, topics, which can be considered from any of the following points of view: - traditional-, computational- or corpus linguistics, - neuro- or psycholinguistics (tip of the tongue problem, word associations), - mathematics (embedding-based approaches, graph theory, small-world problems), etc. *Submissions* Possible submission topics are: - The potential of Large Language Models for the creation and use of lexical resources; - Organization, i.e., structure of the lexicon; - The meaning of words and how to reveal it; - Analysis of the conceptual input given by a dictionary user; - Methods for crafting dictionaries or indexes; - Creation of new types of dictionaries; - Dictionary access (navigation and search strategies), interface issues Short papers can be up to 4 pages in length and long papers up to 8 pages. Both submission formats can have an unlimited number of pages for references. To create your document, please follow the guidelines defined by COLING using their style sheets ( https://lrec-coling-2024.org/authors-kit/). The submissions must be anonymous and they will be peer-reviewed by our program committee. The peer review is double blinded. Accepted papers will also be given an additional page to address the reviewers’ comments. Notice that at least one of the authors of an accepted paper must register for the main conference and present the paper. *Submission Page* https://softconf.com/lrec-coling2024/cogalex2024/ *Dual Submission Policy* Papers may not be submitted to the workshop if they are or will be concurrently submitted to another meeting or publication. *Invited Speaker* Prof. Gilles-Maurice de Schryver (Ghent University, Belgium) https://tshwanedje.com/members/gmds/cv.html *Workshop organizers* - Michael Zock (CNRS, LIS, Aix-Marseille University, Marseille, France) - Emmanuele Chersoni (Department of Chinese and Bilingual Studies, The Hong Kong Polytechnic University, Hong Kong, China) - Yu-Yin Hsu (Department of Chinese and Bilingual Studies, The Hong Kong Polytechnic University, Hong Kong, China) - Simon de Deyne (University of Melbourne / School of Psychological Sciences, University of Adelaide)

1 0

2nd Call for Participation: SemRel SemEval Shared Task 1
by Nedjma OUSIDHOUM 08 Jan '24

08 Jan '24

Dear corpora-list members, We are announcing the first SemEval shared task on Semantic Textual Relatedness (STR): A shared task on automatically detecting the degree of semantic relatedness (closeness in meaning) between pairs of sentences. The semantic relatedness of two language units has long been considered fundamental to understanding meaning (Halliday and Hasan, 1976; Miller and Charles, 1991), and automatically determining relatedness has many applications such as evaluating sentence representation methods, question answering, and summarization. Two sentences are considered semantically similar when they have a paraphrasal or entailment relation. On the other hand, relatedness is a much broader concept that accounts for all the commonalities between two sentences: whether they are on the same topic, express the same view, originate from the same time period, one elaborates on (or follows from) the other, etc. For instance, for the following sentence pairs: - Pair 1: a. There was a lemon tree next to the house. b. The boy enjoyed reading under the lemon tree. - Pair 2: a. There was a lemon tree next to the house. b. The boy was an excellent football player. Most people will agree that the sentences in pair 1 are more related than the sentences in pair 2. In this task, new textual datasets will be provided for Afrikaans <https://en.wikipedia.org/wiki/Afrikaans>, Algerian Arabic <https://en.wikipedia.org/wiki/Algerian_Arabic>, Amharic <https://en.wikipedia.org/wiki/Amharic>, English, Hausa <https://en.wikipedia.org/wiki/Hausa_language>, Hindi <https://en.wikipedia.org/wiki/Hindi>, Indonesian <https://en.wikipedia.org/wiki/Indonesian_language>, Kinyarwanda <https://en.wikipedia.org/wiki/Kinyarwanda>, Marathi <https://en.wikipedia.org/wiki/Marathi_language>, Moroccan Arabic <https://en.wikipedia.org/wiki/Moroccan_Arabic>, Modern Standard Arabic <https://en.wikipedia.org/wiki/Modern_Standard_Arabic>, Punjabi <https://en.wikipedia.org/wiki/Punjabi_language>, Spanish <https://en.wikipedia.org/wiki/Spanish_language>, and Telugu <https://en.wikipedia.org/wiki/Telugu_language>. Data Each instance in the training, development, and test sets is a sentence pair. The instance is labeled with a score representing the degree of semantic textual relatedness between the two sentences. The scores can range from 0 (maximally unrelated) to 1 (maximally related). These gold label scores have been determined through manual annotation. Specifically, a comparative annotation approach was used to avoid known limitations of traditional rating scale annotation methods This comparative annotation process (which avoids several biases of traditional rating scales) led to a high reliability of the final relatedness rankings. Further details about the task, the method of data annotation, how STR is different from semantic textual similarity, applications of semantic textual relatedness, etc. can be found in this paper: https://aclanthology.org/2023.eacl-main.55.pdf Tracks Each team can provide submissions for one, two or all of the tracks shown below: Track A: Supervised Participants are to submit systems that have been trained using the labeled training datasets provided. Participating teams are allowed to use any publicly available datasets (e.g., other relatedness and similarity datasets or datasets in any other languages). However, they must report additional data they used, and ideally report how impactful each resource was on the final results. Track B: Unsupervised Participants are to submit systems that have been developed without the use of any labeled datasets pertaining to semantic relatedness or semantic similarity between units of text more than two words long in any language. The use of unigram or bigram relatedness datasets (from any language) is permitted. Track C: Cross-lingual Participants are to submit systems that have been developed without the use of any labeled semantic similarity or semantic relatedness datasets in the target language and with the use of labeled dataset(s) from at least one other language. Note: Using labeled data from another track is mandatory for submission to this track. Deciding which track a submission should go to: - If a submission uses labeled data in the target language: submit to Track A - If a submission does not use labeled data in the target language but uses labeled data from another language: submit to Track C - If a submission does not use labeled data in any language: submit to Track B ** Here ‘labeled data’ refers to labeled datasets pertaining to semantic relatedness or semantic similarity between units of text more than two words long. Evaluation The official evaluation metric for this task is the Spearman rank correlation coefficient, which captures how well the system-predicted rankings of test instances align with human judgments. You can find the evaluation script for this shared task on our Github page <https://github.com/semantic-textual-relatedness/Semantic_Relatedness_SemEva…> . Helpful Links - Competition Website: https://codalab.lisn.upsaclay.fr/competitions/15704 - Task Website: <https://afrisenti-semeval.github.io/> https://semantic-textual-relatedness.github.io - Twitter X: <https://twitter.com/AfriSenti2023> https://twitter.com/SemRel2024 - Contact organisers semrel-semeval-organisers(a)googlegroups.com - Google group for participants semrel -semeval-participants(a)googlegroups.com Important Dates - Training data ready: 11 September 2023 - Evaluation Starts: 10 January 2024 - Evaluation End: 31 January 2024 - System Description Paper Due: February 2024 - SemEval workshop: Summer 2024 - (co-located with NAACL 2024) NB. We will organise a mentorship session in January and a system description writing tutorial in February for all participants, especially students and junior researchers. References - Shima Asaadi, Saif Mohammad, Svetlana Kiritchenko. 2019. Big BiRD: A Large, Fine-Grained, Bigram Relatedness Dataset for Examining Semantic Composition. Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. - M. A. K. Halliday and R. Hasan. 1976. Cohesion in English. London: Longman. - George A Miller and Walter G Charles. 1991. Contextual Correlates of Semantic Similarity. Language and Cognitive Processes, 6(1):1–28 - Mohamed Abdalla, Krishnapriya Vishnubhotla, and Saif Mohammad. 2023. What Makes Sentences Semantically Related? A Textual Relatedness Dataset and Empirical Study. In Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics, pages 782–796, Dubrovnik, Croatia. Association for Computational Linguistics. Task Organizers Nedjma Ousidhoum Shamsuddeen Hassan Muhammad Mohamed Abdalla Krishnapriya Vishnubhotla Vladimir Araujo Meriem Beloucif Idris Abdulmumin Seid Muhie Yimam Nirmal Surange Christine De Kock Sanchit Ahuja Oumaima Hourrane Manish Shrivastava Alham Fikri Aji Thamar Solorio Saif M. Mohammad

1 1

2026

2025

2024

2023

2022

Corpora