- Corpora - ELRA lists

Paper DL extension: The 4th Workshop on Evaluation and Comparison of NLP Systems (Eval4NLP 2023)
by Juri Opitz 24 Aug '23

24 Aug '23

Dear colleagues, due to popular demand, the deadline for regular paper submissions to the 4th Workshop on Evaluation and Comparison for NLP systems (Eval4NLP) at AACL 2023 will be extended by 1 week. The new submission deadline is **September 1**. Please note that Eval4NLP also allows submission of pre-reviewed papers *together* with their reviews and explanation of possible refinements. These can be submitted up to one month later. For details, please see below or visit the Eval4NLP webpage: https://eval4nlp.github.io/2023/cfp.html. ----------------------------------------------------------------------------------------- The 4th Workshop on Evaluation and Comparison for NLP systems (Eval4NLP), co-located at the 2023 Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics (AACL 2023), invites the submission of long and short papers, with a theoretical or experimental nature, describing recent advances in system evaluation and comparison in NLP. ** Important Dates ** All deadlines are 11.59 pm UTC -12h (“Anywhere on Earth”). - Direct submission to Eval4NLP: ** September 1 ** - Submission of pre-reviewed papers to Eval4NLP (see below for details) : September 25 - Notification of acceptance: October 2 - Camera-ready papers due: October 10 - Workshop day: November 1 Please see the Call for Papers for more details: https://eval4nlp.github.io/2023/cfp.html. ** Special topic of this year’s workshop ** This year's edition of the Eval4NLP workshop puts a focus on the evaluation of and through large language models (LLMs). Notably, the workshop will feature a shared task on LLM evaluation and specifically encourages the submission of LLM evaluation focused papers. Other submissions that fit the general scope of Eval4NLP are of course also welcome. See below for more details. ** Shared Task ** This year’s version will come with a shared task on explainable evaluation of generated language (MT and summarization) with a focus on LLM prompts. Please find more information on the shared task page: https://eval4nlp.github.io/2023/shared-task.html. ** Topics ** Designing evaluation metrics: Proposing and/or analyzing metrics with desirable properties, e.g., high correlations with human judgments, strong in distinguishing high-quality outputs from mediocre and low-quality outputs, robust across lengths of input and output sequences, efficient to run, etc.; Reference-free evaluation metrics, which only require source text(s) and system predictions; Cross-domain metrics, which can reliably and robustly measure the quality of system outputs from heterogeneous modalities (e.g., image and speech), different genres (e.g., newspapers, Wikipedia articles and scientific papers) and different languages; Cost-effective methods for eliciting high-quality manual annotations; and Methods and metrics for evaluating interpretability and explanations of NLP models. Creating adequate evaluation data: Proposing new datasets or analyzing existing ones by studying their coverage and diversity, e.g., size of the corpus, covered phenomena, representativeness of samples, distribution of sample types, variability among data sources, eras, and genres; and Quality of annotations, e.g., consistency of annotations, inter-rater agreement, and bias check. Reporting correct results: Ensuring and reporting statistics for the trustworthiness of results, e.g., via appropriate significance tests, and reporting of score distributions rather than single-point estimates, to avoid chance findings; reproducibility of experiments, e.g., quantifying the reproducibility of papers and issuing reproducibility guidelines; and Comprehensive and unbiased error analyses and case studies, avoiding cherry-picking and sampling bias. ** Submission Guidelines ** The workshop welcomes two types of submission -- long and short papers. Long papers may consist of up to 8 pages of content, plus unlimited pages of references. Short papers may consist of up to 4 pages of content, plus unlimited pages of references. Please follow the ACL ARR formatting requirements, using the official templates: https://github.com/acl-org/acl-style-files. Final versions of both submission types will be given one additional page of content for addressing reviewers’ comments. The accepted papers will appear in the workshop proceedings. The review process is double-blind. Therefore, no author information should be included in the papers and the (optional) supplementary materials. Self-references that reveal the author's identity must be avoided. Papers that do not conform to these requirements will be rejected without review. ** The submission sites on Openreview ** Standard submissions: https://openreview.net/group?id=aclweb.org/AACL-IJCNLP/2023/Workshop/Eval4N… Pre-reviewed submissions: https://openreview.net/group?id=aclweb.org/AACL-IJCNLP/2023/Workshop/Eval4N… See below for more information on the two submission modes. ** Two submission modes: standard and pre-reviewed ** Eval4NLP features two modes of submissions. Standard submissions: We invite the submission of papers that will receive up to three double-blind reviews from the Eval4NLP committee, and a final verdict from the workshop chairs. Pre-reviewed: To a later deadline, we invite unpublished papers that have already been reviewed, either through ACL ARR, or recent AACL/EACL/ACL/EMNLP/COLING venues (these papers will not receive new reviews but will be judged together with their reviews via a meta-review; authors are invited to attach a note with comments on the reviews and describe possible revisions). Final verdicts will be either accept, reject, or conditional accept, i.e., the paper is only accepted provided that specific (meta-)reviewer requirements have been met. Please also note the multiple submission policy. ** Optional Supplementary Materials ** Authors are allowed to submit (optional) supplementary materials (e.g., appendices, software, and data) to improve the reproducibility of results and/or to provide additional information that does not fit in the paper. All of the supplementary materials must be zipped into one single file (.tgz or .zip) and submitted via Openreview together with the paper. However, because supplementary materials are completely optional, reviewers may or may not review or even download them. So, the submitted paper should be fully self-contained. ** Preprints ** Papers uploaded to preprint servers (e.g., ArXiv) can be submitted to the workshop. There is no deadline concerning when the papers were made publicly available. However, the version submitted to Eval4NLP must be anonymized, and we ask the authors not to update the preprints or advertise them on social media while they are under review at Eval4NLP. ** Multiple Submission Policy ** Eval4NLP allows authors to submit a paper that is under review in another venue (journal, conference, or workshop) or to be submitted elsewhere during the Eval4NLP review period. However, the authors need to withdraw the paper from all other venues if they get accepted and want to publish in Eval4NLP. Note that AACL and ARR do not allow double submissions. Hence, papers submitted both to the main conference and AACL workshops (including Eval4NLP) will violate the multiple submission policy of the main conference. If authors would like to submit a paper under review by AACL to the Eval4NLP workshop, they need to withdraw their paper from AACL and submit it to our workshop before the workshop submission deadline. ** Best Paper Awards ** We will optionally award prizes to the best paper submissions (subject to availability; more details to come soon). Both long and short submissions will be eligible for prizes. ** Presenting Published Papers ** If you want to present a paper which has been published recently elsewhere (such as other top-tier AI conferences) at our workshop, you may send the details of your paper (Paper title, authors, publication venue, abstract, and a link to download the paper) directly to eval4nlp(a)gmail.com. We will select a few high-quality and relevant papers to present at Eval4NLP. This allows such papers to gain more visibility from the workshop audience and increases the variety of the workshop program. Note that the chosen papers are considered as non-archival here and will not be included in the workshop proceedings. ----------------------------------------------------------------------------------------- Best wishes, Eval4NLP organizers Website: https://eval4nlp.github.io/2023/index.html Email: eval4nlp(a)gmail.com

1 0

CfP: NLP4DH & IWCLUL 2023 in Tokyo (no publication fees)
by Mika Hämäläinen 23 Aug '23

23 Aug '23

NLP4DH 2023 & IWCLUL 2023 The Joint 3rd International Conference on Natural Language Processing for Digital Humanities and 8th International Workshop on Computational Linguistics for Uralic Languages will be held in Tokyo, Japan. The proceedings will be published in the ACL anthology<https://aclanthology.org/>. The workshop will take place on December 1-3 2023. https://rootroo.com/en/joint-nlp4dh-iwclul-2023/ Submission deadline: October 1, 2023 Registration/publication fees: 0€! The focus of NLP4DH is on applying natural language processing techniques to digital humanities research. The topics can be anything of digital humanities interest with a natural language processing or generation aspect. A list of suitable NLP4DH topics include but are not limited to: -Text analysis and processing related to humanities using computational methods -Dataset creation and curation for NLP (e.g. digitization, digitalization, datafication, and data preservation). -Research on cultural heritage collections such as national archives and libraries using NLP -NLP for error detection, correction, normalization and denoising data -Generation and analysis of literary works such as poetry and novels -Analysis and detection of text genres In addition, IWCLUL solicits papers that focus on NLP methods for Uralic languages. Many of these languages are endangered and call for innovative NLP approaches that can deal with small amounts of data. A list of suitable IWCLUL topics include but are not limited to: -Parsers, analysers and processing pipelines of Uralic languages -Lexical databases, electronic dictionaries -Finished end-user applications aimed at Uralic languages, such as spelling or grammar checkers, machine translation or speech processing -Evaluation methods and gold standards, tagged corpora, treebanks We solicit original and unpublished work related to digital humanities and natural language processing (NLP4DH) or NLP methods for Uralic languages (IWCLUL). Short papers can be up to 4 pages in length (5 for camera-ready version). Short papers can report on work in progress or a more targeted contribution such as software or partial results. Long papers can be up to 8 pages in length (9 for camera-ready version). Long papers should report on previously unpublished, completed, original work. Lightning talks submitted as 750-word abstracts. Lightning talks are suited for discussing ideas or presenting work in progress. The abstracts will not be published or indexed and will only be made available on the conference website. All submission formats can have an unlimited number of pages for references. All submissions must follow the ACL stylesheet<https://2023.aclweb.org/calls/style_and_formatting/>. The submissions must be anonymous and they will be peer-reviewed by our program committee. The peer review is double blinded. Papers must be submitted using the conference submission system by the deadline. At least one of the authors of an accepted paper must attend the event and present their paper. Accepted papers (short and long) will be published in the joint proceedings that will appear in the ACL Anthology. Accepted papers will also be given an additional page to address the reviewers’ comments. The length of a camera ready submission can then be 5 pages for a short paper and 9 for a long paper with an unlimited number of pages for references. The authors of the accepted papers will be invited to submit an extended version of their paper to a special issue in the Journal of Data Mining & Digital Humanities<https://jdmdh.episciences.org/>. Important dates -Paper submission (full and short): October 1, 2023 -Notification of acceptance: November 3, 2023 -Camera ready deadline: November 17, 2023 -NLP4DH & IWCLUL in Tokyo: December 1-3, 2023

1 0

Extension of Deadline and Final CfP: Computational approaches to metaphor and figurative language
by Sebastian Michael Reimann 23 Aug '23

23 Aug '23

Final Call for Abstracts Computational approaches to metaphor and figurative language Date: Feb 28-Mar 1, 2024 Location: Ruhr-University Bochum, Germany Organizers: Stefanie Dipper, Tatjana Scheffler, Sebastian Reimann, Adam Roussel (Ruhr-University Bochum) Homepage: https://www.ruhr-uni-bochum.de/metaphor-dgfs2024/ Call deadline: September 1, 2023 Meeting description: Workshop at the Annual Conference of the German Linguistic Society (DGfS 2024) https://dgfs.de/ Call for Papers: Figurative and non-literal language, such as metaphor, metonymy, or personification, poses a special challenge for computational analysis, since these expressions are not used with their usual, ordinary meanings, as represented in corpus data or recorded in lexical resources, but with different, derived meanings. We aim to bring together computational linguists working on the automatic analysis of non-literal language. A special focus will be on non-conventionalized usages, such as novel metaphors or innovative forms of metonymy. In addition, we are particularly interested in approaches applicable to languages other than English, for example low-resource languages or domains. We welcome contributions dealing with the identification of (specific forms of) non-literal language, with the interpretation of figurative expressions, or with the relation between literal and non-literal meanings in distributional semantic representations. Further, we are interested in discussing how the meaning of figurative expressions is negotiated, for example in social media, and how distributional semantic representations can be enriched in order to reflect figurative meaning. Possible topics include, but are not limited to: * non-conventionalized or innovative figurative language * computational approaches to metaphors, allegory, or metonymy * features for automatically detecting figurative language * the interpretation of non-literal expressions * semantic divergence of non-literal expressions * figurative expressions in low-resource languages * meaning negotiation in social media * semantic representations for figurative meaning Invited speakers: Sabine Schulte im Walde (University of Stuttgart), tba Submission details: Authors should submit anonymous abstracts of maximally 2 pages (including references) in a 12 point font (e.g., Times New Roman). References should be formatted according to the APA guidelines. The workshop language is English for both abstracts and talks. According to DGfS regulations, speakers can only present a paper in one workshop. Please submit your abstracts as a (anonymized) pdf document to the following address: metaphor-dgfs2024(a)ruhr-uni-bochum.de , and include the author information in the body of the email. Upon acceptance, participants will be requested to submit a de-anonymized shortened version of the abstract for the conference booklet. Important dates: Abstract submission deadline: September 1, 2023 Notification of acceptance: September 8, 2023 Workshop: Feb 28-Mar 1, 2024 --- Sebastian Reimann (he/him) Wissenschaftlicher Mitarbeiter Department for German Language & Literature / CRC 1475: Metaphors of Religion Universitätsstraße 90a 44799 Bochum Germany

1 0

[2nd CFP] 1st International Conference on Recent Advances in Robust Argumentation Machines (RATIO24)
by Philipp Cimiano 21 Aug '23

21 Aug '23

Apologies for cross-postings ====================== 2nd CFP: The 1st International Conference on Robust Argumentation Machines (RATIO-24) will take place from June 5th-7th, 2024, in Bielefeld, Germany. https://ratio-conference.net <https://ratio-conference.net/> In recent years, we have witnessed significant advances in our ability to develop approaches that support the automated analysis, summarization, aggregation, retrieval and ranking of arguments exchanged “in the wild” at large scale. By "in the wild" we mean arguments exchanged on the web in debate portals or other online formats where users share opinions and viewpoints on topics relevant to them. Argument analysis methods have indeed reached a level of maturity and robustness that make them applicable to the analysis of real online debates, to find the main arguments exchanged, to summarize and group arguments, or even to automatically generate arguments to present different viewpoints and perspectives. We call for submissions of original research work on the following topics: automatic semantic analysis of arguments, including tasks such as stance detection, keypoint identification, attack/support classification, etc. analysis of arguments in discourse and dialogue automatic synthesis and generation of arguments summarization of arguments argument retrieval methods for predicting argument quality ranking of arguments according to, e.g., quality methods for rephrasing and repurposing arguments inferring the frame, viewpoint or perspective of an argument common sense knowledge in the automated analysis of arguments scalable reasoning methods for arguments applications of argument analysis in domains such as political discourse, law, science, education, finance, social sciences, etc. Papers will be peer-reviewed and published by Springer in the LNCS series. Two types of papers will be accepted: Long Papers (up to 15 pages including references): Description of substantive and original research. Short Papers (up to 8 pages including references): Description of work in progress or original research contribution of limited scope. Papers should be submitted via Easychair: https://easychair.org/conferences/?conf=ratio24 <https://easychair.org/conferences/?conf=ratio24> Important dates: Abstract submission deadline: October 27th, 2023 Full paper submission deadline: November 10th, 2023 Notification of Acceptance: January 26th, 2024 Camera-ready version: February 23rd, 2024 Conference Chairs: Philipp Cimiano (CITEC, Bielefeld University) Anette Frank (University of Heidelberg) Michael Kohlhase (University of Erlangen-Nürnberg) Benno Stein (Bauhaus University Weimar) Jürgen Ziegler (University Duisburg - Essen) Invited Speakers: Elena Cabrio (Université Côte d’Azur, Inria <http://www.unice.fr/>) Yufang Hou (IBM Research Europe) Henning Wachsmuth (Institute for Artificial Intelligence, Leibniz University of Hannover) Venue: The conference will be held in Bielefeld, Germany at the Cognitive Interaction Technology Center (CITEC). All questions about submissions should be emailed to Philipp Cimiano: cimiano(a)cit-ec.uni.bielefeld.de <mailto:cimiano@cit-ec.uni.bielefeld.de> Prof. Dr. Philipp Cimiano AG Semantic Computing Coordinator of the Cognitive Interaction Technology Center (CITEC) Co-Director of the Joint Artificial Intelligence Institute (JAII) Universität Bielefeld Tel: +49 521 106 12249 Fax: +49 521 106 6560 Mail: cimiano(a)cit-ec.uni-bielefeld.de Office CITEC-2.307 Universitätsstr. 21-25 33615 Bielefeld, NRW Germany

1 0

CfP - The Praxis of Digital Humanities
by Mai Zaki 21 Aug '23

21 Aug '23

Dear colleagues, Hope you all had a great summer and happy new academic year. Kindly find below the CfP for our conference on Digital Humanities and Arabic/Islamic/literary/media studies next year. We hope to welcome many of you at AUS in March. https://info.aus.edu/ara-2024 Mai Zaki On behalf of the organising committee ******************* *Mai Zaki, Ph.D. د. مي زكي* Associate Professor of Linguistics أستاذ مشارك في اللسانيات CAS | Department of Arabic and Translation Studies قسم دراسات اللغة العربية والترجمة American University of Sharjah, UAE الجامعة الأمريكية في الشارقة، الإمارات العربية المتحدة *http://www.aus.edu <http://www.aus.edu/>* Follow me on ResearchGate <https://www.researchgate.net/profile/Mai_Zaki2> Follow me on Academia.edu <https://aus.academia.edu/DrMaiZaki> Certified mentor/trainer @ ALDESD <https://aldesd.org/certified-bootcamp-mentors-2/> Recent publications: Zaki, M. (2022) The Metadiscourse of Arabic Academic Abstracts: A corpus-based Study. *Research in Corpus Linguistics*, 10(2), 113-146. (open access <https://ricl.aelinco.es/index.php/ricl/article/view/221>) Al-Kaisi, M. and Zaki, M. (2022) Arab influencers and social media: A Sociolinguistic Study of a New Culture. In Al-Rashdi, F. and Mehta, S. (eds.)* Language and Identity in the Arab World*, London: Routledge, pp. 153-173.

1 0

Employment opportunities: Research Assistant Position for corpora and data mining at Hong Kong Baptist University
by janice pan 21 Aug '23

21 Aug '23

*Hong Kong Baptist University* *Research Assistant (Full-time) * We invite applications for a full-time Research Assistant position for the project "Re-testing the Universals: Mining Interpreting Data through Re-Engineered Mega-Size Corpora". This pioneering project is at the intersection of corpus-based interpreting studies, natural language processing, and text mining. The project involves working with three significant corpora, namely the Chinese/English Political Interpreting Corpus (CEPIC), the European Parliament Interpreting Corpus (EPIC), and the European Parliament Interpreting Corpus Ghent (EPICG). The successful candidate will be involved in the analysis of these corpora in languages including Chinese, English, Italian, French, Dutch. *Responsibilities:* - Primary focus on data cleaning, preparation, and initial analysis of the CEPIC, EPIC, and EPICG corpora across multiple languages - Collaborative work on hypothesis and methodology development - Application of natural language processing techniques and text mining to analyse and draw insights from the corpora *Requirements:* - A bachelor's degree or above in a related field such as Translation Studies, Linguistics, Data Science, or Computational Linguistics. Recent university graduates are also encouraged to apply. - Proficiency in text mining and programming languages such as Python and R, or tools such as SPSS, for data processing and analysis. - Basic competence in at least two of the languages used in the project: Chinese, English, Italian, French and Dutch. - Prior experience or familiarity with the CEPIC, EPIC, and EPICG corpora will be a significant advantage. - A strong background in computational linguistics or a related field is preferred. *Benefits and Training:* The Research Assistant will be trained and supervised by the project investigators. The initial appointment will be made on a fixed-term contract for one year, with the possibility of further extension. Salary will be competitive, in the range of 20,000-24,000 HKD per month, commensurate with qualifications and experience. For further information, please contact Dr. Janice Pan at janicepan(a)hkbu.edu.hk. *Application Procedures:* Applicants are invited to submit their CV, detailing their relevant experience to the project to Dr. Janice Pan at janicepan(a)hkbu.edu.hk. Review of applications is ongoing until the position is filled.

1 0

CFP: Special Issue on Explainability of NLP models, TAL journal
by Marianna Apidianaki 18 Aug '23

18 Aug '23

Call for Papers Special Issue of the TAL Journal: Explainability of NLP models https://tal-64-3.sciencesconf.org/ ** Deadline for submission: 15 October 2023 ** ** Guest Editors ** Marianna Apidianaki, University of Pennsylvania Guillaume Wisniewski, Université Paris Cité, LLF Neural models build language representations without direct supervision, and have contributed to the important advances of language and speech processing systems in recent years. The use of the representations built by pre-trained language models (such as BERT) allows the development of systems for many languages and domains. However, model decisions are not interpretable. It is often not possible to know why a system makes a specific decision, and the reasons behind the good performance of state-of-the-art models remain mostly unknown. An increasing number of studies attempt to provide answers to these questions by addressing the explicability of NLP systems, and exploring questions related to: • model analysis and interpretability, aimed at identifying the information encoded in neural representations [1] ; • the development of methods aimed at describing and justifying the reasoning steps that allowed systems to reach a specific answer, for example with chain-of-thought prompting (model explicability) [2] ; • the development of "safe" systems, capable of self-justification (model accountability). This special issue of the TAL journal aims to address explanation and analysis methods that have been proposed for NLP systems, as well as our current understanding of the linguistic capabilities of neural models and their limitations. We welcome submissions on the following topics (non-exclusively): • Methods aimed at explaining the decisions of neural networks (identification of important elements, explanation in textual format, …), and specifically methods that allow to establish causality relations between a prediction and the input (or part of the input); • Probing methods aimed at identifying linguistic and world knowledge encoded in neural representations; • Methods that allow to distinguish information that is encoded by neural networks, from information that is actually used by the models for making a decision (correlation versus causality); • Methods inspired by analysis methodology from related domains (experimental linguistics, computer vision, psychology, …); • Bias identification in neural language models; • Prompting methods to generate explanations with text [2] or structured representations [4]; • Study of artificial languages or of linguistically motivated examples; • Evaluation of explanation methods and especially methods aimed at evaluating the faithfulness of explanations. [1] Interpretability and Analysis in Neural NLP (Belinkov et al., ACL 2020) [2] Chain-of-Thought Prompting Elicits Reasoning in Large Language Models (Wei et al., NeurIPS 2022) [3] Towards Faithful Model Explanation in NLP: A Survey (Lyu et al., arxiv 2023) [4] Causal Reasoning of Entities and Events in Procedural Texts (Zhang et al., EACL Findings 2023) ** Important dates ** • Submission deadline: 15 October 2023 • Notification to the authors after first review: December 2023 • Notification to the authors after second review: February 2024 • Publication : April 2024 ** Submission format ** The length of the papers must be between 20 and 25 pages. Style sheets are available on the journal's website (https://www.atala.org/content/instruction-authors-style-files-0). Authors are invited to submit their paper on this platform: https://tal-64-3.sciencesconf.org/ To do so, authors will need to first create an account by clicking on "Create account" (Créer un compte) next to the “Login" (Connexion) button at the top of this page. To submit a paper, authors can connect to their account and upload their submission in "My Space" > "My submissions”. The articles can be written in English or in French. Papers in English need to include a title and abstract in French. The TAL journal has a double-blind review process. It is necessary to anonymize the article and the name of the file, and to avoid self-references. Each article is evaluated by three reviewers, two external reviewers and a member of the editorial board of the journal TAL. ** TAL Journal ** TAL (Traitement Automatique des Langues / Natural Language Processing) is an international journal published by ATALA (French Association for Natural Language Processing, http://www.atala.org) since 1959 with the support of CNRS (National Centre for Scientific Research). It has moved to an electronic mode of publication, with printing on demand. The TAL journal is open-access. Paper submission, publication and access are free of charge. Papers published in the TAL journal will be made available on the ATALA website and on ACL Anthology.

1 0

Registration for AIRiAL 2023 is now open
by Voss, Erik 17 Aug '23

17 Aug '23

Artificial Intelligence Research in Applied Linguistics (AIRiAL) 2023 Conference Dear Colleagues, Registration for *AIRiAL 2023* is now open. The conference will be held in person on Friday and Saturday, September 29-30, 2023 at Teachers College in New York City. A banquet has been planned for Saturday evening. The early registration discount will be available through *Thursday, August 24, 2023.* AIRiAL Conference site <https://sites.google.com/tc.columbia.edu/airialconference/home?pli=1> *AIRiAL 2023 r*egistration page <https://sites.google.com/tc.columbia.edu/airialconference/airial-2023/regis…> We are looking forward to seeing you in New York City! Best, The AIRiAL 2023 Organizing Committee -- Erik Voss, Ph.D. Assistant Professor, Applied Linguistics & TESOL program Language & Technology Specialization Department of Arts & Humanities Teachers College, Columbia University TC Faculty Profile <https://www.tc.columbia.edu/faculty/ev2449/>, Linkedin Profile <https://www.linkedin.com/in/erik-voss-ph-d-941a3ab9>, Google Scholar <https://scholar.google.com/citations?user=FMnVdjcAAAAJ&hl=en> ALTESOL Language & Technology Research Group <https://sites.google.com/tc.columbia.edu/al-tesol-language-technology/home> AIRiAL 2023 Conference <https://sites.google.com/tc.columbia.edu/airialconference/home> *Latest Publications* Voss, E. (2022). Argument-based validation in the time of the COVID-19 pandemic <https://www.taylorfrancis.com/chapters/edit/10.4324/9781003221463-6/argumen…>. (Ch. 5) Routledge. Voss, E. (2023). Proctoring remote language assessments <https://www.routledge.com/Fundamental-Considerations-in-Technology-Mediated…>. (Ch. 12) Routledge.

1 0

Two-year post doc position at the University of Genoa (Italy) - Deadline August 31st 2023
by mauro.giacomini＠unige.it 17 Aug '23

17 Aug '23

Dear all, we are looking for a Research Assistant to work on the project Digital Health Solutions in Community Medicine (DHEAL-COM), financed by the NextGeneration EU program through the Italian Health Ministry. The position is for two years. The successful candidate will be based in the Health Informatic Section of the Bioengineering Laboratory at the Department of Informatics, Bioengineering, Robotics and System Engineering (DIBRIS - https://dibris.unige.it/) of the University of Genova - Italy. The overall objective of the Digital Health Solutions in Community Medicine project (DHEAL-COM) Project is the strengthening and development of a Life Science Hub at national level (DHEAL-COM HUB) for the co-creation of digital health services where the person’s needs are the starting point. The DHEAL-COM HUB will act as a central integrator, to bring together and coordinate three satellite national Competence Centre Nodes that will focus on: Digital Health Model/Services, Digital Health Technologies, and Digital Health Usability/Accessibility. Specifically, the title of the call is: Electronic Clinical Data Management: Developing a machine learning-based system for extracting meaningful information from clinical records to fuel a growing data lake. The goal of the two-year research is to collect high-quality data to support the construction and growth of the DHEAL-COM data lake. For this purpose, it will be necessary to automatically remove or replace information that directly identifies the patient, in order to process large amounts of data. Secondly, it will be necessary to extract fundamental information for diagnosis and treatment from data, both structured and unstructured, and codify them according to international standards (eg: ICD11) to allow sharing of the same across several national and international centres. These activities require a strong knowledge of NLP techniques which will have to be widely applied to achieve the purposes of the call. The official call and is available here: https://concorsi.unige.it/home/procedure/4125 Deadline for submission is: August 31st Constraint: the person must have a PhD or obtain it during the first year of the contract Kind regards Mauro

1 0

Second Call for Papers: The 4th Workshop on Evaluation and Comparison of NLP Systems (Eval4NLP 2023)
by Juri Opitz 15 Aug '23

15 Aug '23

Dear colleagues, the 4th Workshop on Evaluation and Comparison for NLP systems (Eval4NLP), co-located at the 2023 Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics (AACL 2023), invites the submission of long and short papers, with a theoretical or experimental nature, describing recent advances in system evaluation and comparison in NLP. ****_New_****: This year's edition of the Eval4NLP workshop puts a focus on the evaluation of and through large language models (LLMs). Notably, the workshop will feature a shared task on LLM evaluation and specifically encourages the submission of LLM evaluation focused papers. Other submissions that fit the general scope of Eval4NLP are of course also welcome. See below for more details. ** Important Dates ** All deadlines are 11.59 pm UTC -12h (“Anywhere on Earth”). - Direct submission to Eval4NLP deadline: August 25 - Submission of pre-reviewed papers to Eval4NLP (see below for details) : September 25 - Notification of acceptance: October 2 - Camera-ready papers due: October 10 - Workshop day: November 1 Please see the Call for Papers for more details: https://eval4nlp.github.io/2023/cfp.html. ** Shared Task ** This year’s version will come with a shared task on explainable evaluation of generated language (MT and summarization) with a focus on LLM prompts. Please find more information on the shared task page: https://eval4nlp.github.io/2023/shared-task.html. ** Topics ** Designing evaluation metrics: Proposing and/or analyzing metrics with desirable properties, e.g., high correlations with human judgments, strong in distinguishing high-quality outputs from mediocre and low-quality outputs, robust across lengths of input and output sequences, efficient to run, etc.; Reference-free evaluation metrics, which only require source text(s) and system predictions; Cross-domain metrics, which can reliably and robustly measure the quality of system outputs from heterogeneous modalities (e.g., image and speech), different genres (e.g., newspapers, Wikipedia articles and scientific papers) and different languages; Cost-effective methods for eliciting high-quality manual annotations; and Methods and metrics for evaluating interpretability and explanations of NLP models. Creating adequate evaluation data: Proposing new datasets or analyzing existing ones by studying their coverage and diversity, e.g., size of the corpus, covered phenomena, representativeness of samples, distribution of sample types, variability among data sources, eras, and genres; and Quality of annotations, e.g., consistency of annotations, inter-rater agreement, and bias check. Reporting correct results: Ensuring and reporting statistics for the trustworthiness of results, e.g., via appropriate significance tests, and reporting of score distributions rather than single-point estimates, to avoid chance findings; reproducibility of experiments, e.g., quantifying the reproducibility of papers and issuing reproducibility guidelines; and Comprehensive and unbiased error analyses and case studies, avoiding cherry-picking and sampling bias. ** Submission Guidelines ** The workshop welcomes two types of submission -- long and short papers. Long papers may consist of up to 8 pages of content, plus unlimited pages of references. Short papers may consist of up to 4 pages of content, plus unlimited pages of references. Please follow the ACL ARR formatting requirements, using the official templates: https://github.com/acl-org/acl-style-files. Final versions of both submission types will be given one additional page of content for addressing reviewers’ comments. The accepted papers will appear in the workshop proceedings. The review process is double-blind. Therefore, no author information should be included in the papers and the (optional) supplementary materials. Self-references that reveal the author's identity must be avoided. Papers that do not conform to these requirements will be rejected without review. ** The submission sites on Openreview ** Standard submissions: https://openreview.net/group?id=aclweb.org/AACL-IJCNLP/2023/Workshop/Eval4N… Pre-reviewed submissions: https://openreview.net/group?id=aclweb.org/AACL-IJCNLP/2023/Workshop/Eval4N… See below for more information on the two submission modes. ** Two submission modes: standard and pre-reviewed ** Eval4NLP features two modes of submissions. Standard submissions: We invite the submission of papers that will receive up to three double-blind reviews from the Eval4NLP committee, and a final verdict from the workshop chairs. Pre-reviewed: To a later deadline, we invite unpublished papers that have already been reviewed, either through ACL ARR, or recent AACL/EACL/ACL/EMNLP/COLING venues (these papers will not receive new reviews but will be judged together with their reviews via a meta-review; authors are invited to attach a note with comments on the reviews and describe possible revisions). Final verdicts will be either accept, reject, or conditional accept, i.e., the paper is only accepted provided that specific (meta-)reviewer requirements have been met. Please also note the multiple submission policy. ** Optional Supplementary Materials ** Authors are allowed to submit (optional) supplementary materials (e.g., appendices, software, and data) to improve the reproducibility of results and/or to provide additional information that does not fit in the paper. All of the supplementary materials must be zipped into one single file (.tgz or .zip) and submitted via Openreview together with the paper. However, because supplementary materials are completely optional, reviewers may or may not review or even download them. So, the submitted paper should be fully self-contained. ** Preprints ** Papers uploaded to preprint servers (e.g., ArXiv) can be submitted to the workshop. There is no deadline concerning when the papers were made publicly available. However, the version submitted to Eval4NLP must be anonymized, and we ask the authors not to update the preprints or advertise them on social media while they are under review at Eval4NLP. ** Multiple Submission Policy ** Eval4NLP allows authors to submit a paper that is under review in another venue (journal, conference, or workshop) or to be submitted elsewhere during the Eval4NLP review period. However, the authors need to withdraw the paper from all other venues if they get accepted and want to publish in Eval4NLP. Note that AACL and ARR do not allow double submissions. Hence, papers submitted both to the main conference and AACL workshops (including Eval4NLP) will violate the multiple submission policy of the main conference. If authors would like to submit a paper under review by AACL to the Eval4NLP workshop, they need to withdraw their paper from AACL and submit it to our workshop before the workshop submission deadline. ** Best Paper Awards ** We will optionally award prizes to the best paper submissions (subject to availability; more details to come soon). Both long and short submissions will be eligible for prizes. ** Presenting Published Papers ** If you want to present a paper which has been published recently elsewhere (such as other top-tier AI conferences) at our workshop, you may send the details of your paper (Paper title, authors, publication venue, abstract, and a link to download the paper) directly to eval4nlp(a)gmail.com. We will select a few high-quality and relevant papers to present at Eval4NLP. This allows such papers to gain more visibility from the workshop audience and increases the variety of the workshop program. Note that the chosen papers are considered as non-archival here and will not be included in the workshop proceedings. ------------------------------------------------- Best wishes, Eval4NLP organizers Website: https://eval4nlp.github.io/2023/index.html Email: eval4nlp(a)gmail.com

2 3

2026

2025

2024

2023

2022

Corpora