May 2024 - Corpora - ELRA lists

Call for Participation: NLPerspectives @ LREC-COLING
by Abercrombie, Gavin 08 May '24

08 May '24

The 3rd Workshop on Perspectivist approaches to NLP (NLPerspectives) Co-located with LREC-COLING, Turin, Italy or online. May 21st 2024 (full day) Keynote presentation: Barbara Plank (LMU Munich) “From Human Label Variation and Model Uncertainty to Error Detection (and Back)” Panel discussion: Barbara Plank (LMU Munich) Alicia Parish (Google) Massimo Poesio (QMU London) Plus lightning talks and poster presentations of 22 perspectivist NLP papers. Registration: https://lrec-coling-2024.org/ https://nlperspectives.di.unito.it/w/3rd-workshop-on-perspectivist-approach… ________________________________ Founded in 1821, Heriot-Watt is a leader in ideas and solutions. With campuses and students across the entire globe we span the world, delivering innovation and educational excellence in business, engineering, design and the physical, social and life sciences. This email is generated from the Heriot-Watt University Group, which includes: 1. Heriot-Watt University, a Scottish charity registered under number SC000278 2. Heriot- Watt Services Limited (Oriam), Scotland's national performance centre for sport. Heriot-Watt Services Limited is a private limited company registered is Scotland with registered number SC271030 and registered office at Research & Enterprise Services Heriot-Watt University, Riccarton, Edinburgh, EH14 4AS. The contents (including any attachments) are confidential. If you are not the intended recipient of this e-mail, any disclosure, copying, distribution or use of its contents is strictly prohibited, and you should please notify the sender immediately and then delete it (including any attachments) from your system.

1 0

1st CfP: StaGe-Shared Task @ KONVENS 2024
by Regina Stodden 08 May '24

08 May '24

We invite the community to participate in a text labelling Shared Task regarding the segmentation of statements in German Easy Language (STaGE), co-located at KONVENS 2024 [1] in Vienna, Austria. For more information, visit: https://german-easy-to-read.github.io/statements Motivation: Assessing the complexity of sentences is still an object of ongoing research. One aspect of sentence complexity is the number of statements. Knowing the different statements conveyed in a sentence is important for numerous NLP tasks, such as extracting the different statements to further simplify the original sentence by separating it into statement-reduced sentences. Another use case is in-depth fact-checking of the isolated statements or the readability evaluation of the text in accordance with Easy Languages guidelines. However, for German, there exists no implementation to extract statements automatically. Our shared task aims to analyze and annotate the number of statements in German Easy language (DE: "Leichte Sprache") texts. We have decided on German Easy Language, since this language variety recommends the usage of sentences with a reduced number of statements. Therefore, it profits from the results and automated analysis implemented in our task. Important dates: * 09.03.2024: Trial data ready * 14.04.2024: Train data ready * 18.05.2024: Test data ready * 06.2024: Evaluation * 07.2024: Paper submission due * 13.09.2024: Workshop date Feel free to contact us via statements(a)soc.cit.tum.de We are looking forward to your participation! Best regards, Miriam Anschütz, Thorben Schomacker & Regina Stodden Links: ------ [1] https://konvens-2024.univie.ac.at/

1 0

Call For Papers : NoDaLiDa/Baltic-HLT 2025
by Sara Stymne 08 May '24

08 May '24

***Apologies for possible cross-posting *** The two major conferences in the Baltic and Nordic regions, NoDaLiDa, organized by The Northern European Association for Language Technology (NEALT) and Baltic HLT are joining forces to organize NoDaLiDa/Baltic-HLT 2025 – The Joint 25th Nordic Conference on Computational Linguistics and 11th Baltic Conference on Human Language Technologies, to be held in Tallinn, Estonia, on March 2–5, 2025. https://www.nodalida-bhlt2025.eu/conference SUBMISSIONS NoDaLiDa/Baltic-HLT 2025 addresses all aspects of natural language processing, speech recognition and synthesis, and computational linguistics, including work in closely related neighboring disciplines (such as, for example, machine learning, linguistics, digital humanities, or psychology) that is sufficiently formalized or applied to bear relevance to speech and language technologies. We invite paper submissions of three types: * regular papers on substantial, original, and unpublished research, including empirical evaluation results, where appropriate; * short papers on smaller, focused contributions, work in progress, negative results, surveys, or opinion pieces; and * demonstration papers on software or resource demonstrations, e.g. of systems, interfaces, infrastructures, data collections, or annotations. We particularly encourage submission of papers on completed or ongoing work, where the first author is a Master's or PhD student. This should be indicated at submission time. Papers accepted for presentation at the conference will be included in the NoDaLiDa/Baltic-HLT 2025 proceedings, which are published in the ACL Anthology and the NEALT Proceedings Series at DSpace at Tartu University Library (negotiations for indexation are ongoing and expected to be in place at publication time) SCHEDULE * Monday, October 21, 2024: Submission of Papers * Monday, December 9, 2024: Notification of Acceptance * Monday, January 13, 2025: Camera-Ready Manuscripts * Monday and Tuesday, March 3–4, 2025: Main Conference The main conference will be held on-site only, without an online option, in order to facilitate networking. SUBMISSION FORMATS All submissions must follow the NoDaLiDa 2025 style files, which will be available for LaTeX (preferred) and MS Word. Submissions must be anonymous, i.e. not reveal author(s) on the title page or through self-references. Papers must be submitted digitally, in PDF, and uploaded through the online conference system. Paper submissions that violate either of these requirements will be returned without review. The page limits for submissions are: up to eight pages for regular papers and up to four pages for short papers and demo papers. For all three submission types, these page limits do not include additional pages with bibliographic references. We do not allow any extra pages for appendices. DOUBLE SUBMISSION and PRE-PUBLICATION Papers that have been or will be submitted to other venues must indicate this at submission time and must be withdrawn from the other venues if accepted to NoDALiDa/Baltic-HLT 2025. Authors of papers accepted for presentation at NoDALiDa/Baltic_HLT must notify the program chairs by the camera-ready deadline as to whether the paper will be presented. We will not accept for publication or presentation papers that overlap significantly in content or results with papers that will be (or have been) published elsewhere. SUBMISSION MANAGEMENT Submissions to the conference must be uploaded electronically, obeying the above requirements, and no later than (end of day, anywhere on earth): Monday, October 21, 2024. NoDaLiDa/Baltic-HLT 2025 uses the OpenReview conference management system for the submission, reviewing, and preparation of proceedings. PROGRAM COMMITTEE General Chair * Sara Stymne, Uppsala University, Sweden Program Chairs * Mark Fišel, University of Tartu, Estonia * Daniel Hershcovich, University of Copenhagen, Denmark * Jenna Kanerva, University of Turku, Finland * Pierre Lison, Norwegian Computing Centre, Norway * Inguna Skadiņa, University of Latvia, Lativa * Andrius Utka, Vytautas Magnus University, Lithuania Workshop chairs * Normunds Grūzītis, University of Latvia, Latvia * Samia Touileb, University of Bergen, Norway Publication chair * Richard Johansson, Chalmers Technical University, Sweden Social media chair * Mike Zhang, Aalborg University, Denmark To inquire about the submission and reviewing process or the scientific program of the conference, please email ‘nodalida_baltichlt_2025-pc(a)googlegroups.xn--com-to0a. Local Chairs * Helen Kaljumäe, Institute of the Estonian Language, Estonia * Kadri Vare, Institute of the Estonian Language, Estonia * Merily Remma, Institute of the Estonian Language, Estonia For all practical inquiries, please email ‘nodalida_baltichlt_2025-loc(a)eki.xn--ee-o2t. Follow us on X: https://twitter.com/NoDaLiDa Web page: https://www.nodalida-bhlt2025.eu/conference När du har kontakt med oss på Uppsala universitet med e-post så innebär det att vi behandlar dina personuppgifter. För att läsa mer om hur vi gör det kan du läsa här: http://www.uu.se/om-uu/dataskydd-personuppgifter/ E-mailing Uppsala University means that we will process your personal data. For more information on how this is performed, please read here: http://www.uu.se/en/about-uu/data-protection-policy

1 0

BCS SEARCH INDUSTRY AWARDS 2024
by Tony Russell-Rose 08 May '24

08 May '24

BCS SEARCH INDUSTRY AWARDS 2024 We are delighted to announce this year's Search Industry Awards, celebrating the best search innovations of 2024. Presented by the Information Retrieval Specialist Group of the BCS, these awards recognize people, projects, and organisations around the world that have excelled in the design of search and information retrieval products and services. If you know of any people, projects, or products that deserve recognition, let us know by submitting a nomination. Alternatively, if you're involved with something special yourself, you can submit an application today. CATEGORIES This year we are offering four awards: 1. Best search project recognises the most impactful implementation of search technology or methodology in solving a specific problem or need. Previous winners include: * Wikiframe Visual Graph, a search capability for Special Collections data stored on Wikidata (wikidata.org) * CiteSeerX, one of the largest open source academic search engines with over 10 million documents * LexisNexis for their work on Open question answering on Lexis+ 2. Search professional of the year is made to an individual who has made a significant contribution through their work and professionalism. Previous winners include: * Amey Porobo Dharwadker, Machine Learning Tech Lead Manager at Meta * Adam Tocock, Library Assistant at NHS * Stuart Mackie, Lead Data Scientist at BiP Solutions 3. Most promising start up (or new enterprise) recognises the innovative and disruptive potential of a business model, technology, or solution. Previous winners include: * batteryincluded.ai, First BI Product Discovery Framework incl. 3 pillars for highest relevancy within global product listings * Giotto AI, an all-in-one platform to automatize, digitalize, and standardize the data collection, analysis and writing of a Clinical Evaluation Report * Resolute.AI, an AI driven platform to search major FDA databases in the public domain in a federated way 4. Best paper / presentation (at Search Solutions). Previous winners include: * Charlie Hull, OSC: “Pragmatic AI-powered Search – Keeping it Simple, not Stupid”. * Filip Radlinski, Google: “Challenges with Really Understanding Natural Language in Conversational Recommendation” * Olivia Foulds, University of Strathclyde: “Crossing the 49th Parallel in Data and Information Science” The last award is open only to presenters at Search Solutions, and will be judged on the day of the event. For all others, apply today! JUDGING PANEL Winners will be selected by our panel of judges (details to be announced shortly). AWARDS CEREMONY The awards ceremony will take place during Search Solutions 2024. APPLY We’ve designed the application process to be simple to complete: https://forms.gle/W54kx6t5jPVp8dkQ7 If you are unsure which category to apply for, or have questions about the application process, contact us via the address below. For further details, see: https://www.bcs.org/membership-and-registrations/member-communities/informa… Nominations will remain open until 31st October. CONTACT If you have any questions on the above, please contact the IRSG Events Organiser at tgr2uk+irsg(a)gmail.com ABOUT IRSG The IRSG is a Specialist Group of BCS. Its mission is to provide a focus for the European IR community, facilitate communication between researchers and practitioners and promote the adoption of IR research within industry. We host a major European conference (ECIR) and provide an associated programme of workshops, seminars and events. The IRSG is free to join via the BCS website, which provides access to further IR articles, events and resources. BCS is the industry body for IT professionals. With members in over 100 countries around the world, BCS is the leading professional and learned society in the field of computers and information systems.

1 0

Job Opening for Data Scientist with a focus on natural language processing
by Menno Van Zaanen 08 May '24

08 May '24

Job Opening for Data Scientist with a focus on natural language processing Application link: https://bit.ly/3QAkC1M Application deadline: 31 May 2024 The South African Centre for Digital Language Resources (SADiLaR) is looking for a data scientist with a focus on natural language processing (permanent position). As a Data Scientist at the South African Centre for Digital Language Resources (SADiLaR) you will have the opportunity to initiate and lead projects focusing on Human Language Technology and Digital Humanities stemming from your own research interests. You will work closely together with a team of researchers as part of SADiLaR's extended network, both on your own and commissioned projects. Dissemination of project results at national and international conferences will be encouraged and supported. This position is crucial for research and development in Human Language Technology and Digital Humanities, fields that form the essence of SADiLaR, which is a national Research Infrastructure supported by the Department of Science and Innovation. Read more about SADiLaR at https://www.sadilar.org. Key responsibilities: - Research: Research in the area of Human Language Technology and Digital Humanities. - Project work: Initiating and contributing to Human Language Technology and Digital Humanities projects. - Teaching: Teaching in the area of Human Language Technology and Digital Humanities. - Mentorship: Mentorship of researchers in the field of Human Language Technology and Digital Humanities. Minimum requirements: - A PhD (NQF level 10) in one of the following fields: Computational Linguistics, Natural Language Processing, Human Language Technology, Digital Humanities, Data Science, Computer Science, Information Technology, Artificial Intelligence, or related fields. The PhD should have a focus on computational aspects of linguistics. - A minimum of (five) 5 years' experience in the use of Python (other programming languages used within the computational linguistics or Digital Humanities domain can also be considered). - Evidence of peer-reviewed academic publications. - A minimum of (three) 3 years' experience as a supervisor/co-supervisor of students or playing a mentorship/supervising role for individuals. - A minimum of (three) 3 years' experience with using and/or developing computational tools. - A minimum of (three) 3 years experience related to research within the domain of Language Technology or Digital Humanities. - A minimum of (one) 1 year experience related to teaching or training within the domain of Language Technology or Digital Humanities. More information can be found at the application link. For informal inquiries please contact: Menno van Zaanen <menno.vanzaanen(a)nwu.ac.za> Menno will be attending LREC-COLING, so please feel free to connect with him for a discussion. Application link: https://bit.ly/3QAkC1M -- Prof Menno van Zaanen menno.vanzaanen(a)nwu.ac.za<mailto:menno.vanzaanen@nwu.ac.za> Professor in Digital Humanities South African Centre for Digital Language Resources https://www.sadilar.org<https://www.sadilar.org/> ________________________________ NWU PRIVACY STATEMENT: http://www.nwu.ac.za/it/gov-man/disclaimer.html DISCLAIMER: This e-mail message and attachments thereto are intended solely for the recipient(s) and may contain confidential and privileged information. Any unauthorised review, use, disclosure, or distribution is prohibited. If you have received the e-mail by mistake, please contact the sender or reply e-mail and delete the e-mail and its attachments (where appropriate) from your system. ________________________________

1 0

AMTA 2024 - Third and Final Call for Papers
by derickofajardo＠gmail.com 07 May '24

07 May '24

We are pleased to announce the Third and Final Call for Papers, Presentations, Tutorials, Workshops and Best Thesis Award Submissions for AMTA 2024. https://amtaweb.org/amta-2024-call-for-proposals/ AMTA 2024 will be a three-day hybrid event under the auspices of the Association for Machine Translation in the Americas to be held on Monday, 30 September 2024 through Wednesday, 2 October 2024 at the Renaissance Schaumburg Convention Center in Chicago, Illinois, USA. Like all AMTA events, AMTA 2024 will bring together researchers, practitioners, and providers of MT and related cross-lingual technology from academia, industry, and government. It will include keynote talks, panel discussions, individual presentations, and demonstrations from technology providers. The organizing committee of AMTA 2024 is seeking proposals for papers and presentations on all topics related to research, development, application, and evaluation of Machine Translation and cross-lingual technologies. Our goal is to have a broad and engaging program that appeals to the various constituents of the MT community (researchers, developers, users, and language professionals). Accepted papers and presentations will be presented as 20-minute talks (15-minute presentations, plus 5 minutes for questions). Please visit https://amtaweb.org/amta-2024-call-for-proposals/ for a sample list of topics of interest, additional conference details, and complete submission guidelines. Workshops and tutorials Traditionally, workshops and tutorials associated with the AMTA conference were held on a pre- and post-conference day. AMTA 2024, however, will feature a day of workshops and tutorials held completely virtually ahead of the main conference. For information about submitting a proposal for a workshop or tutorial, please visit https://amtaweb.org/amta-2024-cfp-workshops-tutorials/. Best Thesis Award The AMTA Board of Directors is also pleased to sponsor the first edition of the AMTA Best Thesis Award, which aims to highlight the achievements of a recent PhD graduate whose thesis has focused on a topic or topics related to machine translation. This award includes a $1000 USD prize, free conference registration and AMTA membership, among other benefits. For more information about eligibility and submission details, please visit https://amtaweb.org/amta-2024-cfp-best-thesis-award/. Important dates Submission deadline: 6 June 2024, 23:59 Anywhere on Earth (UTC-12) Notification of acceptance for papers, presentations and tutorials: 19 July 2024 Notification of Best Thesis Award: 31 July 2024 Final “camera-ready” papers for proceedings and materials for tutorials: 16 August 2024 Please direct any questions you may have to organizing.committee(a)amtaweb.org. We look forward to receiving your submission! AMTA Board of Directors

1 0

[Deadline Extension] TextGraphs-17: Graph-based Methods for Natural Language Processing
by Yanjun Gao 07 May '24

07 May '24

Venue: ACL 2024 Location: Bangkok, Thailand Date: August 15, 2024 Papers Due: May 14, 2024 (23:59 AOE) Website: https://sites.google.com/view/textgraphs2024 OpenReview Site: https://openreview.net/group?id=aclweb.org/ACL/2024/Workshop/TextGraphs-17<https://openreview.net/group?id=aclweb.org/ACL/2024/Workshop/TextGraphs-17#…> Workshop Description For the past seventeen years, the workshops in the TextGraphs series have published and promoted the synergy between the field of Graph Theory (GT) and Natural Language Processing (NLP). The mix between the two started small, with graph-theoretical frameworks providing efficient and elegant solutions for NLP applications. Graph-based solutions initially focused on single-document part-of-speech tagging, word sense disambiguation, and semantic role labeling. They became progressively larger to include ontology learning and information extraction from large text collections. Nowadays, graph-based solutions also target Web-scale applications such as information propagation in social networks, rumor proliferation, e-reputation, multiple entity detection, language dynamics learning, and future events prediction, to name a few. We plan to encourage the description of novel NLP problems or applications that have emerged in recent years, which can be enhanced with existing and new graph-based methods. We widen the workshop topics beyond the familiar graph domain, encompassing a broader range of less examined structured data domains as well. The seventeenth edition of the TextGraphs workshop aims to extend the focus on exploring rising topics of large language models (LLMs) prompting from the unique perspective of GT. Therefore, our workshop aims to foster stronger, mutually advantageous connections between NLP and structured data, tackling key challenges inherent in each field. TextGraphs-17 invites submissions on (but not limited to) the following topics: * Knowledge Graphs Meet LLMs. A proper utilization of graph-based methods for reasoning over a Knowledge Graph (KG) is a prospective way to overcome critical limitations of the existing LLMs which lack interpretability and factual knowledge and are prone to the hallucination problem. Vice versa, the incorporation of LLM knowledge learnt from large textual collections may help many graph-related tasks, such as KG completion and graph representation learning. Thus, we are highly interested in novel research on the joint use of KG and LLM for an improved processing of either the NLP or graph domain (preferably both). * Chain Prompting of LLMs. Recent studies show that prompting strategies like Chain-of-Thought and Graph-of-Thought enhance language understanding and generation tasks compared to the traditional few-shot methods. We welcome submissions developing advanced prompting schemes and software for LLMs and other pre-trained machine learning models. * Learning from Structured Data. We greet novel efforts to build a bridge between NLP and various structured data formats including relational and non-relational databases, as well as standardized data formats (such as XML, JSON, RDF, etc.) * Interpretability of NLP Systems. The question of interpretability poses a fundamental challenge for the practical application of NLP methods. We invite researchers to adopt structured data and employ graph-based methods to shed light on decision-making and logic behind modern LLMs. Any work on applying a KG or any other structured knowledge to explore and evaluate factual awareness, treating the interpretability problem from the GT perspective, or any other topic that utilizes graphs and other structured data to make LLMs more understandable, is met with appreciation. Important dates - Papers due: May 14, 2024 - Notification of acceptance: June 15, 2024 - Camera-ready papers due: July 1, 2024 - Conference date: August 15, 2024 Submission We invite submissions of up to eight (8) pages maximum, plus bibliography for long papers and four (4) pages, plus bibliography, for short papers. The ACL 2024 templates must be used; these are provided in LaTeX and also Microsoft Word format. Submissions will only be accepted in PDF format. This year, TextGraph submission is managed through OpenReview. Submit papers by the end of the deadline day (timezone is UTC-12; AoE) via the submission link on our site: https://openreview.net/group?id=aclweb.org/ACL/2024/Workshop/TextGraphs-17<https://openreview.net/group?id=aclweb.org/ACL/2024/Workshop/TextGraphs-17#…> Shared Task We invite participation in the task of Knowledge Graph Question Answering (KGQA). We will ask the participants to analyze candidate answers with text and graph features. For each query-answer candidate, a graph characterizing paths in Wikidata from entity from the query to the answer entity will be given. Contact Please direct all questions and inquiries to our official e-mail address (textgraphsOC(a)gmail.com<mailto:textgraphsOC@gmail.com>) or contact any of the organizers via their individual emails. Also you can join us on Facebook: https://www.facebook.com/groups/900711756665369. Organizers - Dmitry Ustalov, JetBrains - Arti Ramesh, Binghamton University - Alexander Panchenko, Skolkovo Institute of Science and Technology - Yanjun Gao, University of Wisconsin-Madison - Andrey Sakhovskiy, Skolkovo Institute of Science and Technology - Elena Tutubalina, Artificial Intelligence Research Institute - Gerald Penn, University of Toronto - Marco Valentino, Idiap Research Institute

1 0

Intern Position at Ulster University
by muskaan singh 07 May '24

07 May '24

Exciting Opportunity for Summer 2024 NLP Internship at Ulster University! Are you passionate about Natural Language Processing (NLP) and eager to join a cutting-edge team at Ulster University? We're thrilled to announce an internship opportunity for our Core NLP team, and we're looking for enthusiastic individuals to join us. Key Details: 1. Duration: 2.5 months, remote mode, from mid-May to July 2024. Open to graduates from both 2024 and 2025. 2. Focus: You'll be involved in building data pipelines, cleaning, processing, and analyzing diverse datasets. 3. Requirements: Prior hands-on experience with Machine Learning frameworks and models is essential. 4. Responsibilities: You'll contribute to the design, training, integration, testing, and optimization of NLP (LLMs) and ML models. 5. Bonus Skills:Experience with LLM finetuning, RAG-based methods, and deployments is highly desirable. How to Apply: If you're excited about research and scientific dissemination and meet the qualifications above, we'd love to hear from you! Please apply directly through the link https://forms.gle/Unah7Nt522KcDYoC6

1 0

Postdocs in ML, NLP and CV at U Amsterdam
by Ivan Titov 07 May '24

07 May '24

The University of Amsterdam invites applications for postdoctoral positions on the intersection of ML, NLP, and Computer Vision. The research is funded by NWO (Dutch Science Foundation) grant of Ivan Titov ( http://ivan-titov.org/). The postdocs and PhD students will be employed by the University of Amsterdam and will be members of the Institute for Logic, Language, and Computation <https://www.illc.uva.nl/> and the Faculty of Science. The collaboration is envisaged with researchers at the University of Amsterdam (e.g., Efstratios Gavves, and Wilker Aziz), as well as at the University of Edinburgh (e.g., Edoardo Ponti, Hakan Bilen, Sidharth N., Pasquale Minervini and Kenny Smith). The research will focus primarily on the following directions (or their intersections): 1) Modular and decentralized learning in multimodal settings 2) Learning from language in grounded settings: exploiting knowledge embedded in language and language models to help solve decision-making applications and produce generalizable and interpretable models for these tasks 3) Emergent communication and collaboration: developing agents, which learn to communicate with each other to solve problems, while maintaining transparency to humans and improving with human feedback. *Application deadline: May 31, 2024* https://www.illc.uva.nl/NewsandEvents/News/Positions/newsitem/14960/Two-Pos… <https://www.illc.uva.nl/NewsandEvents/News/Positions/newsitem/14960/Two-Pos…> For informal enquiries please contact: Ivan Titov (titov(a)uva.nl) Ivan is attending ICLR, so please feel free to connect with him for a discussion. <https://www.illc.uva.nl/NewsandEvents/News/Positions/newsitem/14960/Two-Pos…> <https://www.illc.uva.nl/NewsandEvents/News/Positions/newsitem/14960/Two-Pos…>

1 0

Call for Participation: HumEval 2024 @ LREC-COLING 2024
by Simone Balloccu 06 May '24

06 May '24

Call for Participation: The 4th Workshop on Human Evaluation of NLP Systems (HumEval’24) Date: 21 May 2024 (full day) Venue: Lingotto Conference Centre, Turin, Italy Registration Registration is mandatory for attending the workshop. Find more info about registration on the LREC-COLING 2024 website: https://lrec-coling-2024.org/ Workshop description Human evaluation plays a central role in NLP, from the large-scale crowd-sourced evaluations to the much smaller experiments routinely encountered in conference papers. Yet there is growing unease about how human evaluations are conducted in NLP. Researchers have pointed out the less-than-perfect experimental and reporting standards that prevail (van der Lee et al., 2019 <https://aclanthology.org/W19-8643/>; Gehrmann et al., 2023 <https://www.jair.org/index.php/jair/article/view/13715/26927>), and that low-quality evaluations with crowdworkers may not correlate well with high-quality evaluations with domain experts (Freitag et al., 2021 <https://aclanthology.org/2021.tacl-1.87>). Only a small proportion of papers provide enough detail for reproduction of human evaluations, and in many cases the information provided is not even enough to support the conclusions drawn (Belz et al., 2023 <https://aclanthology.org/2023.insights-1.1>). The HumEval workshop (previously at EACL 2021, ACL 2022, and RANLP 2023) aim to create a forum for current human evaluation research and future directions, a space for researchers working with human evaluations to exchange ideas and begin to address the issues human evaluation in NLP faces in many respects, including experimental design, meta-evaluation and reproducibility. Programme Find the detailed programme on the workshop website: https://humeval.github.io/2024/programme Invited speakers Mark Diaz (Google Research) Sheila Castilho (ADAPT/DCU) Organising Committee Simone Balloccu, Charles University, CZ Anya Belz, ADAPT Centre, Dublin City University, Ireland Rudali Huidrom, Dublin City University, Ireland Ehud Reiter, University of Aberdeen, UK João Sedoc, New-York University Craig Thomson, ADAPT Centre, Dublin City University, Ireland For questions and comments regarding the workshop please contact Simone Balloccu at balloccu(a)ufal.mff.cuni.cz and humeval.ws(a)gmail.com. -- Kind regards, Simone Balloccu.

1 0