***Third Call for Papers***

**Overview** - Submission of long and short papers: December 18, 2023 (no deadline extension possible) - Submission page: https://softconf.com/eacl2024/LAW-XVIII/ - Website: https://sigann.github.io/LAW-XVIII-2024/

**Workshop Description** LAW-XVIII will be the 18th annual meeting endorsed by the ACL Special Interest Group for Annotation (SIGANN). It will take place in March 2024 at EACL in St. Julians, Malta. Linguistic annotation of natural language corpora is the backbone of supervised methods in both statistical and neural natural language processing. Annotated corpora are also a major supporting source of information for unsupervised methods, multitask learning, and evaluation of both NLP tools and theories about language within and outside of linguistics. The LAW-XVIII will provide a forum for presentation and discussion of innovative research on all aspects of linguistic annotation, including creation/evaluation of annotation schemes, methods for automatic and manual annotation, use and evaluation of annotation software and frameworks, representation of linguistic data and annotations, semi-supervised “human in the loop” methods of annotation, crowd-sourcing approaches, and more. The LAW will also provide a forum for annotation researchers to work towards standardization, best practices, and interoperability of annotation information and software.

In line with the EACL main conference, LAW will be hybrid, allowing both in-person and virtual presentations.

**Special Theme** The special theme of LAW-XVIII is “Annotation in the Age of Large Language Models (LLMs).” In addition to LAW’s general topics, we specifically invite submissions on the following topics: - Comparison of linguistically annotated datasets vs. datasets created using large language models. Potential topics include: - Comparison of models that have been trained on the respective datasets - Impact of data size of manually annotated resources already available prior to dataset creation with LLMs - Is synthetic dataset creation a viable option for non-standard domains, e.g., the medical domain, where expert knowledge is required? - Non-performance-related considerations of manual vs. synthetic dataset creation (e.g., explainability) - Impact and prevention of test dataset contamination in LLM training - Usefulness of LLMs for linguistic research (in relation to annotation). - Any other topics related to the special theme.

**Submissions** We accept both direct submissions and commitments from ACL Rolling Review (ARR).

We welcome submissions of long and short papers, posters, and demonstrations relating to the special theme or any aspect of linguistic annotation, including: - Annotation procedures - Innovative automated and manual strategies for annotation - Machine learning and knowledge-based methods for automation of corpus annotation - Creation, maintenance, and interactive exploration of annotation structures and annotated data - Annotation evaluation - Inter-annotator agreement and other evaluation metrics and strategies - Qualitative evaluation of linguistic representations - Innovative means to evaluate annotation quality - Annotation access and use - Representation formats/structures for annotations of different phenomena, especially annotations at multiple levels, and means to explore/manipulate them - Linguistic considerations for merging annotations of distinct phenomena - Annotation schemes, guidelines and standards - New and innovative annotation schemes, comparison of annotation schemes - Methodologies and resources for annotation scheme development - Best practices for annotation procedures and/or development and documentation of annotation schemes - Interoperability of annotation formats and/or frameworks among different systems as well as different tasks, frameworks, modalities, and languages - Results from the application and evaluation of standards for linguistic annotation - Annotation software and frameworks - Development, evaluation and/or innovative use of annotation software frameworks

Submissions should report original and unpublished research on topics of interest to the workshop. We also invite substantiated position papers, in particular with regard to our special theme. Accepted papers are expected to be presented at the workshop and will be published in the workshop proceedings. They should emphasize obtained results rather than intended work, and should indicate clearly the state of completion of the reported results. A paper accepted for presentation at the workshop must not be or have been presented at any other meeting with publicly available proceedings. Long/short paper submissions must use the official ACL style templates. Long papers must not exceed eight (8) pages of content. Short papers and demonstration papers must not exceed four (4) pages of content. References do not count against these limits.

Note: The supplementary material does not count towards page limit and should not be included in the paper, but should be submitted separately using the appropriate field on the submission website. All submissions must be in PDF format. Reviewing of papers will be double-blind. Therefore, the paper must not include the authors' names and affiliations or self-references that reveal the authors’ identity--e.g., "We previously showed (Smith, 1991) ..." should be replaced with citations such as "Smith (1991) previously showed ...". Papers that do not conform to these requirements will be rejected without review. Authors of papers that have been or will be submitted to other meetings or publications must provide this information to the workshop co-chairs ( law-xviii-2024@googlegroups.com). Authors of accepted papers must notify the program chairs within 10 days of acceptance if the paper is withdrawn for any reason. We follow previous and current ACL policy to establish an anonymity period (from submission to author notification) during which non-anonymous posting of preprints is not allowed. Also included in that policy are instructions to reviewers to not rate papers down for not citing recent preprints. Authors are asked to cite published versions of papers instead of preprint versions when possible.

Papers can be submitted at https://softconf.com/eacl2024/LAW-XVIII/. If you have any questions, please feel free to contact the program co-chairs via e-mail or check the workshop website ( https://sigann.github.io/LAW-XVIII-2024/) for updates.

**Dates** (All submission deadlines are 11:59 p.m. UTC-12:00 “anywhere on Earth” and will not be extended) Anonymity period starts: November 18, 2023 Submission of long and short papers: December 18, 2023 ARR Commitment deadline: January 17, 2024 Notification of acceptance: January 20, 2024 Camera-ready papers due: January 30, 2024 Workshop: March 21 or 22, 2024

**Workshop Organizers** Manfred Stede (Program Co-Chair) Sophie Henning (Program Co-Chair) Amir Zeldes (ACL SIGANN President) Ines Rehbein (ACL SIGANN Secretary)