- SIGUL - ELRA lists

Fwd: Call for applications (all areas of linguistics): fully funded conference, ‘Exploring the Dark Side of Future Language Technologies: Linguistic (In)security, Ethics, and Privacy in the Human-Machine Era’, UCLouvain, Belgium, 2-3 Sept 2024
by Maite Melero 08 May '24

08 May '24

---------- Forwarded message --------- From: Dave Sayers <dave.sayers(a)cantab.net> Date: Tue, 7 May 2024, 20:24 Subject: Call for applications (all areas of linguistics): fully funded conference, ‘Exploring the Dark Side of Future Language Technologies: Linguistic (In)security, Ethics, and Privacy in the Human-Machine Era’, UCLouvain, Belgium, 2-3 Sept 2024 To: <LITHME(a)jiscmail.ac.uk> ‘Language in the Human-Machine Era’ (https://lithme.eu/) welcomes everyone interested in the impact of new and emerging language technologies that integrate with human senses. Whether you are a tech developer who wants to learn more about linguistics, or a linguist who wants to know more about tech, we want to hear from you! You can find out more about our themes of interest from our published forecast report ( https://doi.org/10.17011/jyx/reports/20210518/1) and our animations ( https://lithme.eu/animations). Our 4th annual conference will be held at UCLouvain (Université catholique de Louvain, Louvain-la-Neuve, Belgium) on 2-3 September 2024. This year's theme is: ‘Exploring the Dark Side of Future Language Technologies: Linguistic (In)security, Ethics, and Privacy in the Human-Machine Era’. The call for papers is now open, and we warmly encourage submissions from any eligible researcher or practitioner who is interested in exploring these timely topics. We welcome experienced developers, but no technological expertise is required, only an interest in exploring the possible effects of these near-future advances in language technology. Presentations can address any of the topics that fall within the interests of LITHME. Selection for funded places will be made by the conference scientific committee. The deadline for submitting your abstract is June 8th, 2024. More details of funding eligibility, the conference theme, and a link to the abstract submission form are on our website: https://lithme.eu/conference2024/ Please forward this message to anyone who may be interested, and please repost the social media announcements here: https://twitter.com/LgHumanMachine/status/1787799682338390457 https://bsky.app/profile/lghumanmachine.bsky.social/post/3krvwfjaqpk2f We hope to see you at the conference! All the best, Dave (on behalf of the organising committee) __________________ Dr. Dave Sayers, ORCID no. 0000-0003-1124-7132 Senior Lecturer & Docent, Dept Language & Communication Studies, U. Jyväskylä, Finland | www.jyu.fi Chair, EU COST Action CA19102 'Language in the Human-Machine Era' | www.lithme.eu Founder & Moderator, TeachLing | https://www.jiscmail.ac.uk/teachlingdave.sayers@cantab.net | https://jyu.academia.edu/DaveSayers ------------------------------ To unsubscribe from the LITHME list, click the following link: https://www.jiscmail.ac.uk/cgi-bin/WA-JISC.exe?SUBED1=LITHME&A=1

1 0

NeTTT’2024 Call for Last minute results’ submissions
by Tharindu Ranasinghe 06 May '24

06 May '24

International Conference ‘New Trends in Translation and Technology’ (NeTTT’2024) Varna, Bulgaria, 3-6 July 2024 (https://nettt-conference.com/) Call for ‘Last minute results’ submissions In view of the special track of the NeTTT'24 event on Future of Translation Technology in the Era of LLMs and Generative AI and the latest dynamic developments with LLMs, we would like to call on researchers and users/companies to submit ‘‘Last minute results” of ongoing studies in the form of short 4-to-page submissions (The conference will not consider and evaluate abstracts only). The idea is to fast-track the reviewing process for these submissions so that the results presented at the event are as up-to-date as possible. The presentations can be either in oral or poster format. Submission deadline: 5 June 2024 Notification: 12 June 2024 Submission is done via the Softconf START conference management system at https://softconf.com/n/nettt2024. We invite the authors to comply with the Springer format, following the templates: * LaTeX<https://resource-cms.springernature.com/springer-cms/rest/v1/content/192386…>, * Overleaf<https://nettt-conference.com/wp-content/uploads/2024/03/Overleaf_Springer_C…>, * Word<https://nettt-conference.com/wp-content/uploads/2024/03/Word_splnproc2311.p…>. Registration Conference registration is open on https://nettt-conference.com/fees-registration/ Venue The conference will take place at Conference Hotel Cherno More<https://www.chernomorebg.com/en/conference-centre.html>, Varna, situated only 200 m away from the fine sandy Black Sea beach. Further information and contact details The conference website is https://nettt-conference.com<https://nettt-conference.com/> and will be updated on a regular basis. For further information, please contact us at nettt2024(a)nettt-conference.com<mailto:nettt2024@nettt-conference.com>

1 0

Deadline extension: 1st UniDive Training School 2024
by Atul K. Ojha 02 May '24

02 May '24

======================================== 1st UniDive Training Summer School 2024 ======================================== ===== LATEST NEWS AND CLARIFICATIONS ===== - Due to recently *extended budget*, UniDive can now fund more than 50 trainees. - The *funding* covers travel and stay for 6 nights - The project submission deadline is extended to *Monday 6 May* (or until budget exhaustion) - One does not have to be a UniDive member to apply. - Gradate research master students are eligible - Eligibility for funding depends on the *affiliation* (not the nationality). The eligible countries are: * COST countries <https://www.cost.eu/about/members/> * Near-Neighbor Countries (Algeria, Armenia, Azerbaijan, Belarus, Egypt, Jordan, Kosovo, Lebanon, Libya, Morocco, Palestine, Syria, and Tunisia) - Please, *circulate this call* to all potential candidates to help us enhance the coverage of low-resourced languages ====================================== Dates: *8 — 12 July 2024* Location: *Technical University of Moldova*, Chișinău, Moldova Coordinating Project: UNIDIVE <https://unidive.lisn.upsaclay.fr/doku.php?id=start> (Universality, Diversity and Idiosyncrasy in Language Technology) Website: https://unidive.lisn.upsaclay.fr/doku.php?id=meetings:other-events:1st_unid… Cost: *Participants selected on the basis of their application will be reimbursed, details are below.* *Apply by:* *May 06, 2024* ====================================== CALL FOR APPLICATIONS We are happy to announce the 1st edition of UNIDIVE Summer School on Universality, Diversity and Idiosyncrasy in Language Technology. It is dedicated mainly (but not exclusively) to young researchers and investigators. Researchers working on low-resourced languages, dialects and varieties are particularly welcome SUMMER SCHOOL ACTIVITIES - Annotation of Universal Dependencies treebank for a new language - a course by Sylvain Kahane (Université Paris Nanterre, France) - Annotation of multiword expressions in a new language - course by Verginica Mititelu (Romanian Academy) and Voula Giouli (Aristotle University of Thessaloniki and ILSP, ATHENA RC, Greece) - Corpus annotation infrastructure- a course by Daniel Zeman (Charles University, Czechia), Bruno Guillaume (LORIA, France) and Agata Savary (Université Paris-Saclay, France) - A brainstorming hackathon on topics submitted by the trainees - Poster sessions APPLICATIONS AND SUBMISSION GUIDELINES Each applicant should *submit a project* for a construction of a resource related to the topics of the training school (e.g. a new/enhanced UD treebank, a new PARSEME corpus, a resource adding a new annotation layer on top of a UD/PARSEME corpus, etc.). The length of the application should be 2 pages (excluding references). The application should contain: - The title - Applicant's name and affiliation (including the country) - A list of 3-4 key-words - Description of a resource related to the topics of the training school - Explanation how the participation in the training school will be useful for the project - Open questions related to the project which could be addressed during the brainstorming hackathon - Short statement of the project phase (planning, started, in the process of creation) The projects are to be submitted via the OpenReview <https://openreview.net/group?id=UniDive/2024/Training_School> portal. TRAINEE'S SELECTION CRITERIA We can fund at least 40 trainees, the selection criteria include: - Trainee's country: trainees only from COST countries[1] <https://mail.math.md/?_task=mail&_caps=pdf%3D1%2Cflash%3D0%2Ctiff%3D0%2Cweb…> and Near-Neighbour Countries can be funded. See here <https://www.cost.eu/about/members/> and here <https://www.cost.eu/about/strategy/international-collaboration/>. - Age: Young Researchers and Investigators, i.e. under the age of 40, are promoted - Gender and geographical balance (notably between Inclusiveness Target Countries and others COST countries) - Relevance and quality of the project submitted by the trainee - Status of the language on which the trainee intends to work (low-resourced languages, dialects or varieties are promoted) *If you are not selected on the basis of these criteria and you can find other financial sources to cover your travel, accommodation and meals, you are also welcome to participate. * *The authors of the selected projects may optionally present them in a poster session during the Training School. * IMPORTANT DATES Deadline for project submission: May 6, 2024 Notification of acceptance: May 23, 2024 Summer school: July 8-12, 2024 For any inquiry, please contact the organisers at: victoria.bobicev(a)ia.utm.md Looking forward to seeing you in Moldova, Organizing Committee

1 0

Second CFP: 1st UniDive training Summer School 2024
by Atul K. Ojha 26 Apr '24

26 Apr '24

1st UniDive Training Summer School 2024 Dates: *8 — 12 July 2024* Location: *Technical University of Moldova*, Chișinău, Moldova Coordinating Project: UNIDIVE <https://unidive.lisn.upsaclay.fr/doku.php?id=start> (Universality, Diversity and Idiosyncrasy in Language Technology) Website: https://unidive.lisn.upsaclay.fr/doku.php?id=meetings:other-events:1st_unid… Cost: *Participants selected on the basis of their application will be reimbursed, details are below.* *Apply by:* *May 01, 2024* CALL FOR APPLICATIONS We are happy to announce the 1st edition of UNIDIVE Summer School on Universality, Diversity and Idiosyncrasy in Language Technology. It is dedicated mainly (but not exclusively) to young researchers and investigators. Researchers working on low-resourced languages, dialects and varieties are particularly welcome SUMMER SCHOOL ACTIVITIES - Annotation of Universal Dependencies treebank for a new language - a course by Sylvain Kahane (Université Paris Nanterre, France) and Francis Tyers (Indiana University, USA) - Annotation of multiword expressions in a new language - course by Verginica Mititelu (Romanian Academy) and Voula Giouli (Aristotle University of Thessaloniki and ILSP, ATHENA RC, Greece) - Corpus annotation infrastructure (annotation platforms, format validators, Git etc.) - a course by Daniel Zeman (Charles University, Czechia), Bruno Guillaume (LORIA, France) and Agata Savary (Université Paris-Saclay, France) - A brainstorming hackathon on topics submitted by the trainees - Poster sessions APPLICATIONS AND SUBMISSION GUIDELINES Each applicant should *submit a project* for a construction of a resource related to the topics of the training school (e.g. a new/enhanced UD treebank, a new PARSEME corpus, a resource adding a new annotation layer on top of a UD/PARSEME corpus, etc.). The length of the application should be 2 pages (excluding references). The application should contain: - The title - Applicant’s name and affiliation (including the country) - A list of 3-4 key-words - Description of a resource related to the topics of the training school - Explanation how the participation in the training school will be useful for the project - Open questions related to the project which could be addressed during the brainstorming hackathon - Short statement of the project phase (planning, started, in the process of creation) The projects are to be submitted via the OpenReview <https://openreview.net/group?id=UniDive/2024/Training_School> portal. TRAINEE’S SELECTION CRITERIA We can fund at least 40 trainees, the selection criteria include: - Trainee’s country: trainees only from COST countries[1] <#m_5787468061823227072_m_-5162026155145348429__ftn1> and Near-Neighbour Countries can be funded. See here <https://www.cost.eu/about/members/> and here <https://www.cost.eu/about/strategy/international-collaboration/>. - Age: Young Researchers and Investigators, i.e. under the age of 40, are promoted - Gender and geographical balance (notably between Inclusiveness Target Countries and others COST countries) - Relevance and quality of the project submitted by the trainee - Status of the language on which the trainee intends to work (low-resourced languages, dialects or varieties are promoted) *If you are not selected on the basis of these criteria and you can find other financial sources to cover your travel, accommodation and meals, you are also welcome to participate. * *The authors of the selected projects may optionally present them in a poster session during the Training School. * IMPORTANT DATES Deadline for project submission: May 01, 2024 Notification of acceptance: May 15, 2024 Summer school: July 8-12, 2024 For any inquiry, please contact the organisers at: victoria.bobicev(a)ia.utm.md Looking forward to seeing you in Moldova, Organizing Committee ------------------------------ [1] <#m_5787468061823227072_m_-5162026155145348429__ftnref1> COST members include 3 categories: COST Full Members, COST Cooperating Members (Israel), COST Partner Members (South Africa).

1 0

Important Update: KI 2024 Conference Abstract and Full Paper Submission Deadline Extended
by Janna Omeliyanenko 25 Apr '24

25 Apr '24

Dear KI 2024 Conference Participants, as the 47th German Conference on Artificial Intelligence (KI 2024) approaches, we have some exciting news to share! Extension of Abstract and Full Paper Submission Due to overwhelming interest and numerous requests, we are pleased to announce that the deadline for abstract and full paper submissions has been extended. This extension provides you with additional time to refine your research and contribute to the vibrant discussions at KI 2024. Here are the revised submission deadlines: • Abstract Submission: New deadline is *May 3rd*, 2024 • Full Paper Submission: You now have until *May 12th*, 2024 to submit your full papers. Summary of Already Accepted Papers at other AI-Conferences as Extended Abstracts We are delighted to inform you that the deadline for submitting summaries of papers already accepted at other AI conferences, as extended abstracts, has been set for *June 2nd*. We encourage all researchers, academics, and industry professionals interested in artificial intelligence to take advantage of this opportunity. KI 2024 is a premier German AI conference that brings together experts from various domains, fostering collaboration and knowledge exchange. For more details about the conference program, venue, and submission guidelines, please visit the official KI 2024 website <https://www.informatik.uni-wuerzburg.de/ki24/> ( https://www.informatik.uni-wuerzburg.de/ki24/). Thank you for your continued support, and we look forward to your valuable contributions at KI 2024! Best regards, KI 2024 Team

1 0

[CFP] Deadline extended - 2024 SIGIR First Workshop on Large Language Models, LLMs, for Evaluation in Information Retrieval
by Guglielmo Faggioli 23 Apr '24

23 Apr '24

The first workshop on evaluating IR systems with Large Language Models (LLMs) is accepting submissions that describe original research findings, preliminary research results, proposals for new work, and recent relevant studies already published in high-quality venues. Topics of interest We welcome both full papers and extended abstract submissions on the following topics, including but not limited to: - LLM-based evaluation metrics for traditional IR and generative IR. - Agreement between human and LLM labels. - Effectiveness and/or efficiency of LLMs to produce robust relevance labels. - Investigating LLM-based relevance estimators for potential systemic biases. - Automated evaluation of text generation systems. - End-to-end evaluation of Retrieval Augmented Generation systems. - Trustworthiness in the world of LLMs evaluation. - Prompt engineering in LLMs evaluation. - Effectiveness and/or efficiency of LLMs as ranking models. - LLMs in specific IR tasks such as personalized search, conversational search, and multimodal retrieval. - Challenges and future directions in LLM-based IR evaluation. Submission guidelines We welcome the following submissions: - Previously unpublished manuscripts will be accepted as extended abstracts and full papers (any length between 1 - 9 pages) with unlimited references, formatted according to the latest ACM SIG proceedings template available at http://www.acm.org/publications/proceedings-template. - Published manuscripts can be submitted in their original format. All submissions should be made through Easychair: https://easychair.org/conferences/?conf=llm4eval All papers will be peer-reviewed (single-blind) by the program committee and judged by their relevance to the workshop, especially to the main themes identified above, and their potential to generate discussion. For already published studies, the paper can be submitted in the original format. These submissions will be reviewed for their relevance to this workshop. All submissions must be in English (PDF format). All accepted papers will have a poster presentation with a few selected for spotlight talks. Accepted papers may be uploaded to arXiv.org, allowing submission elsewhere as they will be considered non-archival. The workshop’s website will maintain a link to the arXiv versions of the papers. Important Dates - Submission Deadline: April 25th May 2nd, 2024 (AoE time) - Acceptance Notifications: May 31st, 2024 (AoE time) - Workshop date: July 18, 2024 Website For more information, visit the workshop website: https://llm4eval.github.io/ Contact For any questions about paper submission, you may contact the workshop organizers at llm4eval(a)easychair.org

1 0

NeTTT’2024 Last Call for Papers
by t.ranasinghe＠aston.ac.uk 17 Apr '24

17 Apr '24

International Conference ‘New Trends in Translation and Technology’ (NeTTT’2024) Varna, Bulgaria, 3-6 July 2024 https://nettt-conference.com/ *** Last Call for Papers (Submission deadline approaching: 30 April 2024) *** # The conference The second edition of the forthcoming International Conference ‘New Trends in Translation and Technology’ (NeTTT’2024) will take place in Varna, Bulgaria, 3-6 July 2024. Continuing the tradition of the first edition of the NeTTT conference and HiT-IT events series, the objective of the conference is (i) to bridge the gap between academia and industry in the field of translation and interpreting by bringing together academics in linguistics, translation studies, machine translation and natural language processing, developers, practitioners, language service providers and vendors who work on or are interested in different aspects of technology for translation and interpreting, and (ii) to be a distinctive event for discussing the latest developments and practices. NeTTT’2024 invites all professionals who would like to learn about the new trends, present the latest work or/and share their experience in the field, and who would like to establish business and research contacts, collaborations and new ventures. The conference will take the form of presentations (peer-reviewed research and user presentations, keynote speeches), and posters; it will also feature panel discussions. The accepted papers will be published as open-access conference e-proceedings. # Conference topics Contributions are invited on any topic related to latest technology and practices in machine translation, translation, subtitling, localisation and interpreting. NeTTT’2024 will feature a Special Theme Track "Future of Translation Technology in the Era of LLMs and Generative AI". The conference topics include but are not limited to: ## CAT tools - Translation Memory (TM) systems - NLP and MT for translation memory systems - Terminology extraction tools - Localisation tools ## Machine Translation - Latest developments in Neural Machine Translation - MT for under-resourced languages - MT with low computing resources - Multimodal MT - Integration of MT in TM systems - Resources for MT ## Technologies for MT deployment - MT evaluation techniques, metrics and evaluation results - Human evaluations of MT output - Evaluating MT in a real-world setting - Quality estimation for MT - Domain adaptation ## Translation Studies - Corpus-based studies applied to translation - Corpora and resources for translation - Translationese - Cognitive effort and eye-tracking experiments in translation ## Interpreting studies - Corpus-based studies applied to interpreting - Corpora and resources for interpreting - Interpretese - Resources for interpreting and interpreting technology applications - Cognitive effort and eye-tracking experiments in interpreting ## Interpreting technology - Machine interpreting - Computer-aided interpreting - NLP for dialogue interpreting - Development of NLP based applications for communication in public service settings (healthcare, education, law, emergency services) ## Emerging Areas in Translation and Interpreting - MT and translation tools for literary texts and creative texts - MT for social media and real-time conversations - Sign language recognition and translation ## Subtitling - NLP and MT for subtitling - Latest technology for subtitling ## User needs - Analysis of translators’ and interpreters’ needs in terms of translation and interpreting technology - User requirements for interpreting and translation tools - Incorporating human knowledge into translation and interpreting technology - What existing translators’ (including subtitlers’) and interpreters’ tools do not offer - User requirements for electronic resources for translators and interpreters - Translation and interpreting workflows in larger organisations and the tools for translation and interpreting employed ## The business of translation and interpreting - Translation workflow and management - Technology adoption by translators and industry - Setting up translation /interpreting / language provider company ## Teaching translation and interpreting - Teaching Machine Translation - Teaching translation technology - Teaching interpreting technology - Latest AI developments in the syllabi of translation and interpreting curricula ## Ethical issues in translation and technology - Bias and fairness in MT - Privacy and security in cloud MT systems - Transparency and explainability of MT systems - Environmental impact on MT systems # Special Theme Track - Future of Translation Technology in the Era of LLMs and Generative AI We are excited to share that NeTTT’2024 will have a special theme with the goal of stimulating discussion around Large Language Models, Generative AI and the Future of Translation and Interpreting Technology. While the new generation of Large Language Models such as CHATGPT and LLAMA showcase remarkable advancements in language generation and understanding, we find ourselves in uncharted territory when it comes to their performance on various Translation and Interpreting Technology tasks with regards to fairness, interpretability, ethics and transparency. The theme track invites studies on how LLMs perform on Translation and Interpreting Technology tasks and applications, and what this means for the future of the field. The possible topics of discussion include (but are not limited to) the following: - Changes in the translators and interpreters’ professions in the new AI era especially as a result of the latest developments in LLMSs and Generative AI - Generative AI and translation - Generative AI and interpreting - Augmenting machine translation systems with generative AI - Domain and terminology adaptation with Large Language Models - Literary translation with Large Language Models - Improving Machine Translation Quality with Contextual Prompts in Large Language Models - Prompt engineering for translation - Generative AI for professional translation - Generative AI for professional interpreting # Keynote speakers We are delighted to announce the NeTTT’2024 keynote speakers - Helena Moniz (University of Lisbon and Unbabel), President of the European Association of Machine Translation - Carla Parra Escartín (RWS Language Weaver) # Tutorial (3 July 2024) - Tharindu Ranasinghe (Aston University), Quality Estimation for Machine Translation # Programme Committee The Programme Committee of NeTTT’2024 is listed https://nettt-conference.com/26844-2/. # Conference Chairs - Ruslan Mitkov (Lancaster University) - Gloria Corpas Pastor (University of Malaga) # Programme Chairs - Constantin Orasan (University of Surrey) - Tharindu Ranasinghe (Aston University) # Sponsorship Chair - Vilelmini Sosoni (Ionian University) # Publication Chair - Maria Kunilovskaya (University of Saarland) # Organising Committee - Organising Committee of NeTTT’2024 is listed https://nettt-conference.com/organisers/ # Submissions and publication NETTT’2024 invites the following types of submissions: User papers – for industry and practitioners. References to related work are optional. Allowed paper length: between 1 and 4 pages. Academic submissions, in three different categories (have to follow formatting requirements, references to related work are required): • (academic) full papers – describing original completed research. Allowed paper length: maximum 12 pages + unlimited references. • (academic) work-in-progress papers/posters – describing work in progress, late breaking research, papers at a more conceptual stage, and other types of papers that do not fit in the ‘full’ papers category. Allowed paper length: maximum 7 pages + unlimited references. • (academic) demo papers – describing working systems. Allowed paper length: maximum 5 pages + unlimited references. In addition to the papers, the authors will be expected to demonstrate the systems at the workshop. The conference will not consider and evaluate abstracts only. Each submission will be reviewed by three members of the Programme Committee. Submission is organised via Softconf START conference management system at https://softconf.com/n/nettt2024. For submitting the papers, we invite the authors to comply with the Springer format, following the templates: • LaTeX: https://resource-cms.springernature.com/springer-cms/rest/v1/content/192386…, • Overleaf: https://www.overleaf.com/latex/templates/springer-lecture-notes-in-computer…, • Word: https://resource-cms.springernature.com/springer-cms/rest/v1/content/192387…. The accepted papers will be published in the conference proceedings and made available online on the conference website. Authors of accepted papers will receive guidelines regarding how to produce camera-ready versions of their papers. The final version of the accepted papers will be published in e-proceedings with assigned ISBN and DOI. All accepted papers will be included in the conference e-proceedings which will be available at the conference website. # Schedule - Submission deadline: 30 April 2024 - Notification: 5 June 2024 - Final version due: 20 June 2024 All deadlines are valid for 23.59 Anywhere on Earth. # Registration Conference registration is open on https://nettt-conference.com/fees-registration/ The promotional early registration fee has been extended to 17 March 2024. # Venue The conference will take place at https://www.chernomorebg.com/en/conference-centre.html, Varna, situated only 200 m away from the fine sandy Black Sea beach. # Sponsors We are proud to announce the conference sponsors: OONA - Diamond Sponsor Pangeanic – Gold Sponsor MITRA Translations – Silver Sponsor Juremy – Bronze Sponsor # Further information and contact details The conference website is https://nettt-conference.com and will be updated on a regular basis. For further information, please contact us at nettt2024(a)nettt-conference.com Dr Tharindu Ranasinghe Lecturer in Computer Science School of Informatics and Digital Engineering Birmingham, B4 7ET, UK aston.ac.uk

1 0

[CFP] 2024 SIGIR First Workshop on Large Language Models (LLMs) for Evaluation in Information Retrieval
by Guglielmo Faggioli 12 Apr '24

12 Apr '24

The first workshop on evaluating IR systems with Large Language Models (LLMs) is accepting submissions that describe original research findings, preliminary research results, proposals for new work, and recent relevant studies already published in high-quality venues. Topics of interest We welcome both full papers and extended abstract submissions on the following topics, including but not limited to: - LLM-based evaluation metrics for traditional IR and generative IR. - Agreement between human and LLM labels. - Effectiveness and/or efficiency of LLMs to produce robust relevance labels. - Investigating LLM-based relevance estimators for potential systemic biases. - Automated evaluation of text generation systems. - End-to-end evaluation of Retrieval Augmented Generation systems. - Trustworthiness in the world of LLMs evaluation. - Prompt engineering in LLMs evaluation. - Effectiveness and/or efficiency of LLMs as ranking models. - LLMs in specific IR tasks such as personalized search, conversational search, and multimodal retrieval. - Challenges and future directions in LLM-based IR evaluation. Submission guidelines We welcome the following submissions: - Previously unpublished manuscripts will be accepted as extended abstracts and full papers (any length between 1 - 9 pages) with unlimited references, formatted according to the latest ACM SIG proceedings template available at http://www.acm.org/publications/proceedings-template. - Published manuscripts can be submitted in their original format. All submissions should be made through Easychair: https://easychair.org/conferences/?conf=llm4eval All papers will be peer-reviewed (single-blind) by the program committee and judged by their relevance to the workshop, especially to the main themes identified above, and their potential to generate discussion. For already published studies, the paper can be submitted in the original format. These submissions will be reviewed for their relevance to this workshop. All submissions must be in English (PDF format). All accepted papers will have a poster presentation with a few selected for spotlight talks. Accepted papers may be uploaded to arXiv.org, allowing submission elsewhere as they will be considered non-archival. The workshop’s website will maintain a link to the arXiv versions of the papers. Important Dates - Submission Deadline: April 25th, 2024 (AoE time) - Acceptance Notifications: May 31st, 2024 (AoE time) - Workshop date: July 18, 2024 Website For more information, visit the workshop website: https://llm4eval.github.io/ Contact For any questions about paper submission, you may contact the workshop organizers at llm4eval(a)easychair.org

1 0

1st UniDive training Summer School 2024
by Atul K. Ojha 11 Apr '24

11 Apr '24

*1st UniDive training Summer School 2024* Dates: *8 — 12 July 2024* Location: *Technical University of Moldova*, Chișinău, Moldova Coordinating Project: UNIDIVE <https://unidive.lisn.upsaclay.fr/doku.php?id=start> (Universality, Diversity and Idiosyncrasy in Language Technology) Website: https://unidive.lisn.upsaclay.fr/doku.php?id=meetings:other-events:1st_unid… Cost: *Participants selected on the basis of their application will be reimbursed, details are below.* *Apply by:* *April 29, 2024* CALL FOR APPLICATIONS We are happy to announce the 1th edition of UNIDIVE Summer School on Universality, Diversity and Idiosyncrasy in Language Technology. It is dedicated mainly (but not exclusively) to young researchers and investigators. Researchers working on low-resourced languages, dialects and varieties are particularly welcome SUMMER SCHOOL ACTIVITIES - Annotation of Universal Dependencies treebank for a new language - a course by Sylvain Kahane (Université Paris Nanterre, France) and Francis Tyers (Indiana University, USA) - Annotation of multiword expressions in a new language - course by Verginica Mititelu (Romanian Academy) and Voula Giouli (Aristotle University of Thessaloniki and ILSP, ATHENA RC, Greece) - Corpus annotation infrastructure (annotation platforms, format validators, Git etc.) - a course by Daniel Zeman (Charles University, Czechia), Bruno Guillaume (LORIA, France) and Agata Savary (Université Paris-Saclay, France) - A brainstorming hackathon on topics submitted by the trainees - Poster sessions APPLICATIONS AND SUBMISSION GUIDELINES Each applicant should *submit a project* for a construction of a resource related to the topics of the training school (e.g. a new/enhanced UD treebank, a new PARSEME corpus, a resource adding a new annotation layer on top of a UD/PARSEME corpus, etc.). The length of the application should be 2 pages (excluding references). The application should contain: - The title - Applicant’s name and affiliation (including the country) - A list of 3-4 key-words - Description of a resource related to the topics of the training school - Explanation how the participation in the training school will be useful for the project - Open questions related to the project which could be addressed during the brainstorming hackathon - Short statement of the project phase (planning, started, in the process of creation) The projects are to be submitted via the OpenReview <https://openreview.net/group?id=UniDive/2024/Training_School> portal. TRAINEE’S SELECTION CRITERIA We can fund at least 40 trainees, the selection criteria include: - Trainee’s country: trainees only from COST countries[1] <#m_-2062973076706449294_m_-5162026155145348429__ftn1> and Near-Neighbour Countries can be funded. See here <https://www.cost.eu/about/members/> and here <https://www.cost.eu/about/strategy/international-collaboration/>. - Age: Young Researchers and Investigators, i.e. under the age of 40, are promoted - Gender and geographical balance (notably between Inclusiveness Target Countries and others COST countries) - Relevance and quality of the project submitted by the trainee - Status of the language on which the trainee intends to work (low-resourced languages, dialects or varieties are promoted) *If you are not selected on the basis of these criteria and you can find other financial sources to cover your travel, accommodation and meals, you are also welcome to participate. * *The authors of the selected projects may optionally present them in a poster session during the Training School. * IMPORTANT DATES Deadline for project submission: April 29, 2024 Notification of acceptance: May 15, 2024 Summer school: July 8-12, 2024 For any inquiry, please contact the organisers at: victoria.bobicev(a)ia.utm.md Looking forward to seeing you in Moldova, Organizing committee ------------------------------ [1] <#m_-2062973076706449294_m_-5162026155145348429__ftnref1> COST members include 3 categories: COST Full Members, COST Cooperating Members (Israel), COST Partner Members (South Africa).

1 0

[CfP] SIGIR Workshop on Large Language Model for Evaluation in IR (LLM4EVAL)
by Guglielmo Faggioli 21 Mar '24

21 Mar '24

Overview The first workshop on evaluating IR systems with Large Language Models (LLMs) is accepting submissions that describe original research findings, preliminary research results, proposals for new work, and recent relevant studies already published in high-quality venues. The workshop will have both an in-person and virtual component, and submissions are welcome even for researchers who cannot attend in person, as they will present their work in the virtual component. Topics of interest We welcome both full papers and extended abstract submissions on the following topics, including but not limited to: - LLM-based evaluation metrics for traditional IR and generative IR. - Agreement between human and LLM labels. - Effectiveness and/or efficiency of LLMs to produce robust relevance labels. - Investigating LLM-based relevance estimators for potential systemic biases. - Automated evaluation of text generation systems. - End-to-end evaluation of Retrieval Augmented Generation systems. - Trustworthiness in the world of LLMs evaluation. - Prompt engineering in LLMs evaluation. - Effectiveness and/or efficiency of LLMs as ranking models. - LLMs in specific IR tasks such as personalized search, conversational search, and multimodal retrieval. - Challenges and future directions in LLM-based IR evaluation. Submission guidelines We welcome the following submissions: - Previously unpublished manuscripts will be accepted as extended abstracts and full papers (any length between 1 - 9 pages) with unlimited references, formatted according to the latest ACM SIG proceedings template available at http://www.acm.org/publications/proceedings-template. - Published manuscripts can be submitted in their original format. All submissions should be made through Easychair: https://easychair.org/conferences/?conf=llm4eval All papers will be peer-reviewed (single-blind) by the program committee and judged by their relevance to the workshop, especially to the main themes identified above, and their potential to generate discussion. For already published studies, the paper can be submitted in the original format. These submissions will be reviewed for their relevance to this workshop. All submissions must be in English (PDF format). Please note the workshop will have an in-person (to be held with SIGIR 2024) and virtual component (to be held at a later date on SIGIR VF). During submission, the authors should select their preferred component. All accepted papers will have a poster presentation with a few selected for spotlight talks. Accepted papers may be uploaded to arXiv.org, allowing submission elsewhere as they will be considered non-archival. The workshop’s website will maintain a link to the arXiv versions of the papers. Important Dates - Submission Deadline: April 25th, 2024 (AoE time) - Acceptance Notifications: May 31st, 2024 (AoE time) - Workshop date: July 18, 2024 Website and Contact More details are available at https://llm4eval.github.io/cfp/. For any questions about paper submission, you may contact the workshop organizers at llm4eval(a)easychair.org RispondiInoltra

1 0

2025

2024

2023

2022

2021

2020

2019

2018

2017

SIGUL