March 2024 - Corpora

[CIKM 2024] - Call for AnalytiCup Competition Proposals
by antonela.tommasel＠isistan.unicen.edu.ar 28 Mar '24

28 Mar '24

* We apologize if you receive multiple copies of this CfP * * For the online version of this Call, visit: https://cikm2024.org/call-for-analyticup-competition-proposals/ =============== CIKM 2024: 33rd ACM International Conference on Information and Knowledge Management Boise, Idaho, USA October 21–25, 2024 =============== CIKM 2024 AnalytiCup is an open competition including compelling data challenges aimed at members of the industry and academia interested in information and knowledge management. The challenges will be rolled out progressively and last for several weeks. The final solutions will be presented at CIKM 2024 AnalytiCup which is to be held in conjunction with the CIKM conference during October 2024. -------------------------- Key Dates -------------------------- * Proposal due: May 31 2024 * Notification: June 7 2024 * Competition Kickoff: June 24 20224 * Competition Ends: August 30, 2024 (All deadlines are at 11:59 pm AOE) -------------------------- Proposals Submissions -------------------------- We invite proposals from practitioners across industry and academia who are interested in the areas of information retrieval, databases, and knowledge management. The best fit proposal should include a well-motivated goal with a positive social impact, a novel and challenging task, a fair setup with stable evaluation approach, and adequate amount of real-world data for the competition. * A well-motivated goal: A goal of the proposed competition should be solving a challenging real-world problem at the same time impacting the research and other communities positively. A good competition is where the output of the competition should lead to a greater good of everyone, such proposals are encouraged. * A challenging task: The task should be challenging in the sense that there is enough room for improvement from the basic solutions, and novel ideas are required to succeed in the competition. At the same time the task should be manageable in about 2 months’ time. * A fair setup: The organizers should guarantee the availability of the data and the confidentiality of the test set. The evaluation metrics should be both meaningful for the application in-hand and statistically sound for the objective comparison. The baseline should be established to show that non-trivial results can be achieved. * A real-world dataset: A proposal should clearly explain what data will be provided for competition and the source of the data. Also, explain how/why the provided data is sufficient for the competition. A proposal should cover all the important details such as dates, submission and evaluation of results, etc. and describe the competition rules clearly. Please provide following details with your proposal: * Title: The title of your challenge. * Problem Description: Describe the problem clearly in detail. Explain the importance of the problem and its impact. Discuss different scenarios for the problem with its challenges and limitations. Share the simple data samples and explain the data clearly. If the proposed competition includes more than one track, please describe each track clearly and show unique value for each track. * Evaluation: Describe how you plan to evaluate the submission. Select the evaluation method which is fair and statistically robust. * Suggested Participants: Provide a list of suggested participants in the challenge. * Timeline. Dates for expected start of the competition, user registration, team formation, submission, evaluation, and notification. * Awards. Specify the type and form of the awards you want to share with the winners. * Host information: Names, affiliations, email addresses, and short biographies of the organizers. -------------------------- Chairs Contact Information -------------------------- For more information, contact the AnalytiCup Chairs at: CIKM2024-analyticup [at] easychair [dot] org Vachik Dave, Walmart Global Tech Carl Yang, Emory University

1 0

1st CFP: EvalLAC 2024 – AIED 24 workshop
by Shiva Taslimipoor 28 Mar '24

28 Mar '24

***** *1st Workshop on Automated Evaluation of Learning and Assessment Content* AIED 2024 workshop | Recife (Brazil) & Hybrid | 8-12 July 2024 https://sites.google.com/view/eval-lac-2024/ ***** We are happy to announce the first edition of the Workshop on Automated Evaluation of Learning and Assessment Content will be held in Recife (Brazil) & online during the AIED 2024 conference. *About the workshop* The evaluation of learning and assessment content has always been a crucial task in the educational domain, but traditional approaches based on human feedback are not always usable in modern educational settings. Indeed, the advent of machine learning models, in particular Large Language Models (LLMs), enabled to quickly and automatically generate large quantities of texts, making human evaluation unfeasible. Still, these texts are used in the educational domain -- e.g., as questions, hints, or even to score and assess students -- and thus the need for accurate and automated techniques for evaluation becomes pressing. This hybrid workshop aims to attract professionals from both academia and the industry, and to to offer an opportunity to discuss which are the common challenges in evaluating learning and assessment content in education. Topics of interest include but are not limited to: - Question evaluation (e.g., in terms of alignment to learning objectives, factual accuracy, language level, cognitive validity, etc.). - Estimation of question statistics (e.g., difficulty, discrimination, response time, etc.). - Evaluation of distractors in Multiple Choice Questions. - Evaluation of reading passages in reading comprehension questions. - Evaluation of lectures and course material. - Evaluation of learning paths (e.g., in terms of prerequisites and topics taught before a specific exam). - Evaluation of educational recommendation systems (e.g., personalised curricula). - Evaluation of hints and scaffolding questions, as well as their adaptation to different students. - Evaluation of automatically generated feedback provided to students. - Evaluation of techniques for automated scoring. - Evaluation of bias in educational content and LLM outputs. Human-in-the-loop approaches are welcome, provided that there is also an automated component in the evaluation and there is a focus on the scalability of the proposed approach. Papers on generation are also very welcome, as long as there is an extensive focus on the evaluation step. *Important dates* Submission deadline: May 17, 2024 Notification of acceptance: June 4, 2024 Camera ready: June 11, 2024 Workshop: 8 July or 12 July 2024 *Submission guidelines* Authors are invited to submit short papers (5 pages, excluding references) and long papers (10 pages, excluding references), formatted according to the workshop style available on the website. Submissions should contain mostly novel work, but there can be some overlap between the submission and work submitted elsewhere (e.g., summaries, focus on the evaluation phase of a broader work). Each of the submissions will be reviewed by the members of the Program Committee, and the proceedings volume will be submitted for publication to CEUR Workshop Proceedings. *Organisers* Luca Benedetto (1), Andrew Caines (1), George Dueñas (2), Diana Galvan-Sosa (1), Anastassia Loukina (3), Shiva Taslimipoor (1), Torsten Zesch (4) (1) ALTA Institute, Dept. of Computer Science and Technology, University of Cambridge (2) National Pedagogical University, Colombia (3) Grammarly, Inc. (4) FernUniversität in Hagen

1 0

Reminder: Professor of Language Technology at the Department of Swedish, Multilingualism, Language Technology; University of Gothenburg [new deadline]
by Gerlof Bouma 28 Mar '24

28 Mar '24

The Department of Swedish, Multilingualism, Language Technology at the University of Gothenburg is inviting applications for the position of Professor of Language Technology. The new professor's main duties will be to lead and develop research, education, and outreach in the field of language technology at the department, in particular within its Språkbanken Text group. A detailed description of the position and the application requirements can be found in University of Gothenburg's job application portal, at the link below. This detailed description is only available in Swedish, as proficiency in Swedish or another Scandinavian language is required for the position. Applications must be submitted no later than 6 May 2024. https://web103.reachmee.com/ext/I005/1035/job?site=6&lang=SE&validator=3038… <https://web103.reachmee.com/ext/I005/1035/job?site=6&lang=SE&validator=3038…> -- GERLOF BOUMA Universitetslektor GÖTEBORGS UNIVERSITET Institutionen för svenska, flerspråkighet och språkteknologi Språkbanken Text https://spraakbanken.gu.se/om/personal/gerlof

1 0

2nd CfP - MILLing (Update: Invited Speakers)
by William Noble 28 Mar '24

28 Mar '24

https://gu-clasp.github.io/MILLing Multimodality and Interaction in Language Learning (MILLing) will bring together researchers in linguistics and computational linguistics to discuss learning through linguistic interaction, from the perspectives of both human language acquisition and machine learning. We encourage contributions from the fields of theoretical linguistics, experimental linguistics, pragmatics, computational linguistics, artificial intelligence, and cognitive science. The conference is organised by the Centre for Linguistic Theory and Studies in Probability (CLASP, <https://gu-clasp.github.io/>), University of Gothenburg. The conference will be held between October 14 and 15 in Gothenburg, Sweden. Important dates ---- - Submission deadline: May 31, 2024, anywhere on Earth - Notification of acceptance: Aug 30, 2024, anywhere on Earth - Camera ready: Sep 20, 2024, anywhere on Earth - Conference: Oct 14--15, 2024, University of Gothenburg, Sweden Invited speakers ---- - Catherine Pelachaud <http://chronos.isir.upmc.fr/~pelachaud/>, Director of Research at CNRS in the laboratory LTCI, TELECOM ParisTech - Charles Yang <https://www.ling.upenn.edu/~ycharles/> Professor of Linguistics and Computer Science at the University of Pennsylvania - Napoleon Katsos <https://sites.google.com/site/napoleonkatsos/home> Professor of Experimental Pragmatics at the Section of Theoretical and Applied Linguistics at the University of Cambridge and Fellow at Trinity College, Cambridge Topics of interest ---- We hope to see innovative work that considers language learning from different perspectives, and we hope to cultivate discussion that reaches across traditionally disparate disciplines. Papers are invited on topics in these and closely related areas, including (but not limited to) the following: - Language acquisition: formal, statistical, experimental, and machine learning- based work - Language learning through dialogue in humans and machines - Multi-modality and figurativeness in language learning and dialogue - Linguistic variation, adaptation, and audience design - Low-resource and ecologically plausible language modelling (e.g., BabyLM) - Cognitive architectures for language learning - Information state update in humans and machines - Cognitive aproaches to second language acquisition - Dialogue systems for language learning - Online, reinforcement and curriculum learning in NLP - Atypical development and language learning - Ethical considerations in AI-assisted language learning Submission Requirements ---- MILLing will feature two types of submissions: long papers and short papers. Long papers must describe original research, and they must not exceed 8 pages excluding references (position papers are also accepted and should be formatted in the same way). Short papers present work in progress, or they describe systems and/or projects. They must not exceed 4 pages excluding references. All types of papers will be published in the 2024 ACL Anthology as a CLASP Conference Proceedings. Papers should be electronically submitted via the softconf system at: <https://softconf.com/n/MILLing2024/>. Submissions should be PDF files and use the LaTeX or Word templates provided for ACL submissions (<https://github.com/acl-org/acl-style-files>). Submissions have to be anonymous. Please make sure that you select the right track when submitting your paper. Contact the organisers if you have problems using softconf. Concurrent Submissions ---- Papers that have been or will be submitted to other conferences or publications must indicate this at submission time using a footnote on the title page of the submissions. Authors of papers accepted for presentation at MILLing must notify the program chairs by the camera-ready deadline as to whether the paper will be presented. All accepted papers must be presented at the conference to appear in the proceedings. We will not accept publications or presentation papers that overlap significantly in content or results with papers that will be (or have been) published elsewhere. Camera Ready Versions ---- Camera ready versions should follow the same guidelines with respect to style and page numbers as the initial submission, i.e. there are no additional pages allowed in the final submission. Please submit the camera ready version by Sep 20, 2024. About CLASP ---- MILLing is organised by the Centre for Linguistic Theory and Studies in Probability (CLASP, <https://gu-clasp.github.io/>) at the Department of Philosophy, Linguistics and Theory of Science (FLoV), University of Gothenburg. CLASP focuses its research on the application of probabilistic and information theoretic methods to the analysis of natural language. CLASP is concerned both with understanding the cognitive foundations of language and developing efficient language technology. We work at the interface of computational linguistics/natural language processing, theoretical linguistics, and cognitive science.

1 0

Extended Deadline: Submission of Extended Abstracts - CLARIN Annual Conference 2024
by t.kontino＠uu.nl 28 Mar '24

28 Mar '24

Call for Submission of Extended Abstracts CLARIN Annual Conference 2024 CLARIN ERIC is pleased to announce the CLARIN Annual Conference 2024 and calls for the submission of extended abstracts. CLARIN is the European research infrastructure that makes digital language resources available to scholars, researchers, students and citizen scientists from a wide range of disciplines, coordinates the collection of language resources and tools, and offers advanced tools to explore, exploit, annotate, analyse or combine such datasets, regardless of their location. New in this year's call is the topic of Education and Training with CLARIN tools. Submission deadline: 26 April 2024 (Extended) Location The conference will take place in Barcelona, Spain. The event will be hosted and organised by CLARIN ERIC in collaboration with CLARIAH-ES and the Basque Center for Language Technology (HiTZ). Important Dates 24 January 2024: First call published on CLARIN website, disseminated, and submission system open 12 February 2024: Second call for abstracts disseminated 29 March 2024: Third call for abstracts disseminated 26 April 2024: Submission deadline (Extended) 17 June 2024: Notification of acceptance 2 September 2024: Camera-ready version deadline (that will be extended) 15-17 October 2024: CLARIN Annual Conference Conference Aims The CLARIN Annual Conference is organised for the wider Humanities and Social Sciences (SSH) community in order to exchange experiences and best practices in working with the CLARIN infrastructure and to share plans for future developments. The programme will cover a range of topics, including the design, construction and operation of the CLARIN infrastructure, the data, tools and services that it contains or should contain, its actual use by researchers, teachers or interested parties, its relation to other infrastructures and projects, and the CLARIN Knowledge Infrastructure. Keynote Speakers To be confirmed. Conference Topics We invite submissions describing CLARIN-related work addressing the following aspects: Use of the CLARIN Infrastructure: Use of the CLARIN infrastructure in SSH research and beyond Usability studies and evaluations of CLARIN services Analysis of the CLARIN infrastructure usage and impact studies/use cases Identification and analysis of user audiences and developer communities, including digital humanities, libraries, computer science, information science, cognitive science and human-centred AI Showcases, demonstrations and research projects that are relevant to CLARIN Design and Construction of the CLARIN Infrastructure: Recent tools and resources added to the CLARIN infrastructure Metadata and concept registries, cataloguing and browsing Persistent identifiers and citation mechanisms Access, including single sign-on authentication and authorisation Search functions, including Federated Content Search Web applications, web services and workflows Standards and solutions for interoperability of language resources, tools and services Models for the sustainability of the infrastructure, including curation, migration financing and cooperation Legal and ethical issues in operating the infrastructure. CLARIN Knowledge Infrastructure and Dissemination: User assistance (help desks, user manuals, FAQs) CLARIN portals and outreach to users Videos, screencasts, recorded lectures Knowledge centres. CLARIN vis-à-vis other Infrastructures and Initiatives: SSH research infrastructures, such as DARIAH and CESSDA and the collaboration under the umbrella of the SSH Open Cluster, etc. Generic infrastructural initiatives, such as EOSC, Europeana, Language Data Space, etc. Projects such as ATRIUM, EOSC Focus, ERIC Forum, EOSC Future, FAIRCORE4EOSC, OSCARS, OSTrails National and regional initiatives. Education and Training Using CLARIN language resources and services in teaching and training activities targeting audiences from different sectors (academia, GLAM, industry) and lessons learnt The impact of the DH Course Registry (e.g. development of the DH curricula, student exchange programmes) Guidelines and best practices for using CLARIN in the university curricula Developing new courses reusing existing materials from the CLARIN Learning Hub (e.g. UPSKILLS) FORMAT OF THE PROGRAMME SESSIONS The programme of the conference will include oral presentations and posters, and may also include demos. Due to limits in the time schedule, the number of oral presentations is limited. Authors can select if they prefer a poster presentation. If not, papers are allocated a presentation format based on the suitability of the paper for a session as decided by the programme committee. Authors of accepted submissions will be offered the opportunity to demo their work in addition to their presentation. SUBMISSIONS The language of the conference is English and presentations will be made in English. Proposals for oral, poster or demo presentations must be submitted as extended abstracts (length: 3 to 4 pages A4, including references) in PDF format, in accordance with the template (ZIP-archive, Overleaf template). Authors can choose whether to submit on an anonymous or non-anonymous basis. Extended abstracts should address one or more topics that are relevant to CLARIN’s activities, resources, tools or services. This relevance should be explicitly articulated in the submission, as well as in the presentation at the conference. Contributions addressing desiderata for the CLARIN infrastructure that are currently not in place are also eligible. Authors are not required to be or have been directly involved in national or cross-national CLARIN projects. Extended abstracts must be submitted through the EasyChair submission system and will be reviewed by the Programme Committee. All proposals will be reviewed on the basis of the following criteria: Appropriateness: The contribution must pertain to the CLARIN infrastructure or be relevant for it (e.g. its use, design, construction, operation, exploitation, illustration of possible applications, etc.), and this relevance should be explicitly articulated in the submission. Soundness and correctness: The content must be technically and factually correct and methods must be scientifically sound, according to best practice, and preferably evaluated. Meaningful comparison: The abstract must indicate that the author is aware of alternative approaches, if any, and highlight relevant differences. Substance: Concrete work and experiences will be given preference over ideas and plans. Impact: Contributions with a higher impact on the research community and society more broadly will be given preference over papers with lower impact. Clarity: The abstract should be clearly written and well structured. Timeliness and novelty: The work must convey relevant new knowledge to the audience at this event. ATTENDANCE For each accepted abstract, CLARIN ERIC offers one author free access, free accommodation and meals. Travelling costs are not covered by CLARIN ERIC. Authors are encouraged to reach out to their national consortium, to their home institution or to third party funds to cover travel costs. PROCEEDINGS Accepted submissions will be published in the online conference Book of Extended Abstracts, ISSN: 2773-2177. After the conference, the author(s) of accepted submissions will be invited to submit full papers (10-12 pages) to be reviewed according to the same criteria as the abstracts. Accepted full papers will be published in a digital conference proceedings volume after the conference: Linköping Electronic Conference Proceedings (peer reviewed) ISSN: 1650-3686 (print), 1650-3740 (online) https://ep.liu.se/en/conferences.aspx CONFERENCE PROGRAMME COMMITTEE The Programme Committee for the conference consists of the following members: Vincent Vandeghinste, Instituut voor de Nederlandse Taal (Dutch Language Institute), the Netherlands & KU Leuven, Belgium -- chair Starkaður Barkarson, Árni Magnússon Institute for Icelandic Studies, Iceland Lars Borin, University of Gothenburg, Sweden António Branco, University of Lisbon, Portugal Tomaž Erjavec, Jožef Stefan Institute, Slovenia Cristina Grisot, University of Zurich and at the Swiss National Center for Data & Services for the Humanities DaSCH Eva Hajičová, Charles University Prague, Czech Republic Marianne Hundt, University of Zurich, Switzerland Krister Lindén, University of Helsinki, Finland Monica Monachini, Institute of Computational Linguistics ‘A. Zampolli’, Italy Karlheinz Mörth, Austrian Academy of Sciences, Austria Costanza Navarretta, University of Copenhagen, Denmark Gijsbert Rutten, Leiden University, the Netherlands Maciej Piasecki, Wrocław University of Science and Technology, Poland Stelios Piperidis, ILSP, Athena Research Center, Greece German Rigau, HiTZ, the Basque Center for Language Technology, Spain Kiril Simov, IICT, Bulgarian Academy of Sciences, Bulgaria Inguna Skadiņa, Institute of Mathematics and Computer Science, University of Latvia, Latvia National Coordinator Norway Marko Tadić, University of Zagreb, Croatia Jurgita Vaičenonienė, Vytautas Magnus University, Lithuania Tamás Váradi, Research Institute for Linguistics, Hungarian Academy of Sciences, Hungary Joshua Wilbur, Center of Estonian Language Resources, Estonia Andreas Witt, University of Mannheim, Germany Friedel Wolff, South African Centre for Digital Language Resources, North-West University, South Africa Martin Wynne, University of Oxford, United Kingdom

1 0

Two post-doctoral positions in an ERC-funded project at the University of Helsinki, Finland
by Hiippala, Tuomo J 28 Mar '24

28 Mar '24

Dear all, I'm currently looking to hire two post-doctoral researchers for my upcoming project funded by the European Research Council: https://www.helsinki.fi/en/researchgroups/multimodality/a-foundation-for-em… The project develops novel methods and resources for studying multimodality, or how human communication naturally combines multiple 'modes' of expression. These methods and resources are used to develop empirically founded theories of multimodal communication in the domain of everyday cultural artefacts. The data studied in the project includes, for example, school textbooks, user-generated explanation videos on social media, news broadcasts, instruction manuals and online newspapers. Please find descriptions for both positions below – if you have any questions, please contact me at tuomo.hiippala(a)helsinki.fi. For instructions on how to apply, please see the links to the full job announcements below. Best, Tuomo Hiippala --- POST-DOCTORAL RESEARCHER IN DIGITAL HUMANITIES --- This is a fixed-term position for 36 months, starting in September 2024 or as agreed, based in the Department of Languages at the University of Helsinki, Finland. RESPONSIBILITIES: The post-doctoral researcher is responsible for developing structured, machine-readable representations of multimodality for the page-based and audiovisual media studied in the project. As the descriptions of multimodality are collected using crowdsourcing, the post-doctoral researcher is also expected to participate in designing and implementing crowdsourcing pipelines. QUALIFICATIONS: The appointee must hold a doctoral degree in digital humanities, humanities computing, language technology, computer science or a related field, and have a keen interest in multimodality. Previous experience of developing structured representations for diverse forms of data is essential. Previous experience or interest in volunteer-based or paid crowdsourcing or citizen science is considered an asset. For more information, see the full announcement here: https://jobs.helsinki.fi/job/Helsinki-Post-doctoral-researcher-in-digital-h… --- POST-DOCTORAL RESEARCHER IN MULTIMODAL COMMUNICATION --- This is a fixed-term position for 48 months, starting in September 2024 or as agreed, based in the Department of Languages at the University of Helsinki, Finland. RESPONSIBILITIES: The post-doctoral researcher is responsible for developing large corpora that describe the multimodal structure of the diverse media studied in the project. To scale up the corpus size, the data will be described by crowdsourced non-expert workers. The post-doctoral researcher will lead the development of crowdsourcing tasks that generate descriptions that can be then compiled into systematic corpus annotations. The appointee will also participate in developing new methods for analysing large multimodal corpora. QUALIFICATIONS: The appointee must hold a doctoral degree in linguistics, semiotics, media studies or related field with a specific focus on multimodality. The appointee must have a deep understanding of contemporary theories of multimodality relevant to the domain of the project. Previous experience of designing and implementing annotation schemes for multimodal corpora is essential. Knowledge of programming languages such as Python and mark-up languages such as XML and JSON for managing research data is considered an advantage. For more information, see the full announcement here: https://jobs.helsinki.fi/job/Helsinki-Post-doctoral-researcher-in-multimoda… --- Dr. Tuomo Hiippala | Associate Professor in English Language and Digital Humanities Department of Languages | University of Helsinki, Finland | +358 50 377 33 66 http://www.helsinki.fi/~thiippal/ | https://www.helsinki.fi/multimodality

1 0

[Iber AuTexTification@IberLEF2024] Training set released
by Francisco Manuel Rangel Pardo 28 Mar '24

28 Mar '24

Event Notification Type: Training set released Website: <https://sites.google.com/view/autextification> https://sites.google.com/view/iberautextification TRAINING SET RELEASED! Iber AuTexTification Automated Text Identification on Languages of the Iberian Peninsula Dear All, The Training dataset for the Iber AuTexTification 2024 shared task has been released. It can be found on the shared task website ( https://sites.google.com/view/iberautextification/data <https://sites.google.com/view/iberautextification/data?authuser=0>), in Genaios Github repository (https://github.com/Genaios/IberAuTexTification) and Zenodo (https://zenodo.org/records/10853560). Please, remember that registration is on a per-team basis, meaning only one member from each team needs to sign up. Once registered and requested the training set on one of the previous links, you will receive the download permission and a password to decompress the data within 24 hours. Please, make sure to write your email address correctly, since we will send passwords, as well as future notifications to that address. Please reach out to the organizers <organizers.autextification(a)gmail.com> or join the Slack <https://join.slack.com/t/iberautextification/shared_invite/zt-2c28ezgwy-lHH…> workspace to connect with the other participants and organizers. Best regards on behalf of Iber AuTexTification shared tasks organizers

1 0

[NTCIR-18] Final Call for Participation
by CHUNG-CHI CHEN 28 Mar '24

28 Mar '24

Dear colleagues, We are glad to share the information of NTCIR-18 with you. This year, we have 9 interesting tasks, that cover the (1) Modern IR Tasks, (2) Access to Non-Digitized Archive Material, (3) Large Language Model Evaluation, and (4) NLP Tasks in Financial and Medical Domains. We will have a kickoff event within 24 hours. The task organizers will introduce their tasks. Below are more details, and please register for the kickoff event. We will send a Zoom link to you soon. Feel free to let us know if you have any questions. Best Regards, Qingyao Ai, Chung-Chi Chen, Shoko Wakamiya NTCIR-18 Program Co-Chairs *Kick-Off Event* - Date/Time: March 29th (Friday) 2024, From 16:00 To 17:30 p.m. - Onsite: Room 1902-1903, National Institute of Informatics - Online: Zoom (The link will be available for participants before the event.) - Fees: Free - Registration: https://docs.google.com/forms/d/e/1FAIpQLScumfe8q3HDe2Vul2RUhTxNacSt9KujvqM… - Zoom link will be sent to the registered email address before the event. - For more details, please refer to our site: https://research.nii.ac.jp/ntcir/ntcir-18/kickoffcfp.html *Kick-Off Event Tentative Program* - Overview of NTCIR - Overview of NTCIR-18 - Introduction of each task - How to participate in NTCIR-18 - Q & A *NTCIR-18 Tasks* *CORE TASKS* 1. AEOLLM: Automatic Evaluation of LLMs 2. FairWeb-2: The Second Fair Web Task 3. FinArg-2: Temporal Inference of Financial Arguments 4. MedNLP-Rad: Medical Natural Language Processing for Radiology 5. MedNLP-CHAT: Medical Natural Language Processing for AI Chat 6. Transfer-2: The Resource Transfer Based Dense Retrieval Task *PILOT TASKS* 1. HIDDEN-RAD: Hidden Causality Inclusion in Radiography Report Generation 2. SUSHI: Searching Unseen Sources for Historical Information 3. UFO 2.0: Understanding of Non-Financial Objects in Financial Reports

1 0

CFP - SIGIR 2024 Workshop on eCommerce
by Surya Kallumadi 28 Mar '24

28 Mar '24

Call For Papers - SIGIR eCom'24 - https://sigir-ecom.github.io/ The SIGIR Workshop on eCommerce will serve as a platform for publication and discussion of Information Retrieval, NLP and Vision research relative to their applications in the domain of eCommerce. This workshop will bring together practitioners and researchers from academia and industry to discuss the challenges and approaches to product search and recommendation in eCommerce. The deadline for paper submission is April 25, 2024 (11:59 P.M. AoE) The special theme of this year's workshop is eCommerce Search in the Age of Generative AI and LLMs. The workshop will also include a data challenge. This year we will collaborate with TREC on a product search data challenge ( https://trec-product-search.github.io/index.html). The overarching goal is to study how end-to-end retrieval systems can be built and evaluated given a large set of products. The data challenge provides a corpus of products and a set of user intents (queries): the goal is to find the product that suits the user’s needs. SIGIR eCom is a full day workshop taking place on Thursday, July 18, 2024 in conjunction with SIGIR 2024. SIGIR eCom'24 will be an in-person workshop. ________________ Important Dates: Paper submission deadline - April 25, 2024 (11:59 P.M. AoE) Notification of acceptance - May 23, 2024 Camera Ready Version of Papers Due - June 24, 2024 SIGIR eCom Full day Workshop - July 18, 2024 We invite quality research contributions, position and opinion papers addressing relevant challenges in the domain of eCommerce. We invite submission of both papers and posters. All submitted papers and posters will be single-blind and will be peer reviewed by an international program committee of researchers of high repute. Accepted submissions will be presented at the workshop. Topics: Topics of interest include, but are not limited to: - eCommerce search in the age of Generative AI and LLMs (2024 special theme) - Ranking and Whole Page Relevance - Optimization for IR and business metrics - Diversity in product search and recommendations - Relevance models for multi-faceted entities - Relevance vs. revenue - Deterministic sorts (e.g. price low to high) - Temporal dynamics and seasonality - Query and Document Understanding - Query intent, query suggestions, and auto-completion - Strategies for resolving low or zero recall queries - Converting across modalities (e.g., text, structured data, images) - Categorization and facets - Reviews and sentiment analysis - Recommendation and Personalization - Personalization & contextualization, including the use of personal facets such as age, gender, location - Privacy, bias and ethics in eCommerce IR - Blending recommendations and search results - Representations and Data - Semantic representation of products, queries, and customers - Construction and use of knowledge graphs for eCommerce - IR Fundamentals for eCommerce - Unified and universal search and recommendations - Cross-lingual search and machine translation - Indexing and search in rapidly changing environments (e.g., auction sites) - Experimentation techniques including AB testing and multi-armed bandits - Visual Search in ecommerce - Large-scale Visual Search Challenges and Solutions - Multimodal Search and combining visual and textual information - Combining Vision and language models - Explainable AI for Visual Search - Other challenges - Trust, transparency, and fairness in eCommerce - UX for eCommerce - The role of search in trust and security for marketplaces - Question answering and chatbots for eCommerce Data/Resource Track: In order to promote academic research in the eCommerce domain, we plan to accept a small number of high quality dataset contributions. These submissions should be accompanied by a clear and detailed description of the dataset, some potential questions and applications that arise from it. Preliminary empirical investigations conveying any insight about the data will increase the quality of the submission. Submission Instructions: All papers will be peer reviewed (single-blind) by the program committee and judged by their relevance to the workshop, especially to the main themes identified above, and their potential to generate discussion. Submissions must describe work that is not previously published, not accepted for publication elsewhere, and not currently under review elsewhere. All submissions must be in English. The workshop follows a single-blind reviewing process, i.e. author names must be on the papers. We do not accept anonymized submissions. At least one of the authors of each accepted paper must register for the workshop and present the paper. All submissions must be in PDF formatted according to the latest CEUR single column format; the short (8-page) and long (15-page) limits are extended to account for this. For instructions and LaTeX/Overleaf/docx templates, see: https://ceur-ws.org/HOWTOSUBMIT.html#CEURART Read up to and including the “License footnote in paper PDFs” section. Please Use Emphasizing Capitalized Style for Paper Titles. Submit your paper PDF through the SIGIR eCom’24 Easychair: https://easychair.org/conferences/?conf=sigirecom24 Long paper limit: 15 pages. References are not counted in the page limit. Short paper limit: 8 pages. References are not counted in the page limit. The deadline for paper submission is April 25, 2024 (11:59 P.M. AoE) https://sigir-ecom.github.io/

1 0

Meta KDD Cup 2024: Comprehensive RAG Challenge
by Kai Sun 28 Mar '24

28 Mar '24

Hello Colleagues, Imagine using an AI assistant to get the latest updates on your favorite sports team, but instead of receiving latest scores, you get last year’s results. Or when asking about a niche movie you really like, but couldn't get meaningful answers. These are classic examples of “hallucination” where LLMs provide outdated or incorrect information. The Meta Comprehensive RAG (CRAG) Benchmark Challenge aims to address these challenges. Participate to improve how Large Language Models (LLMs) can keep-up with the ever-evolving reality and provide accurate responses by leveraging Retrieval Augmented Generation (RAG). Why RAG Matters Despite the advancements of LLMs, the issue of hallucination persists as a significant challenge; that is, LLMs may generate answers that lack factual accuracy or grounding. Retrieval-Augmented Generation (RAG) has recently emerged as a promising solution to alleviate LLM’s deficiency in lack of knowledge and attracted a lot of attention from both academia research and industry. Introducing the Meta Comprehensive RAG (CRAG) Benchmark Challenge CRAG is a factual question answering benchmark that covers 5 domains and 8 question types, and provides a practical set-up to evaluate RAG systems. Different from existing benchmarks, CRAG is designed to include questions with a wide variety of domains and types. In particular, it includes questions with answers that change from over seconds to over years; it considers entity popularity and covers not only head, but also torso and tail facts; it contains simple-fact questions as well as 7 types of complex questions such as comparison, aggregation and set questions to test the reasoning and synthesis capabilities of RAG solutions. Why This Challenge Is a Game-Changer Addressing "hallucination" and outdated information is critical to enhancing the reliability of LLM-powered question-answering systems. RAG proposes a solution by integrating the external data into its responses. The CRAG Benchmark is a comprehensive test to evaluate these advanced systems' effectiveness across various domains and question types, challenging them with scenarios that require immediate data and those that explore less popular "tail" facts. What's unique about this challenge? * Tasks Designed to Improve QA Systems: The three tasks focus on web-based retrieval summarization, knowledge graph and web augmentation, and an end-to-end RAG challenge, each built on top of the previous one. * Rich Dataset Across Diverse Domains: The CRAG dataset covers domains from finance to music, to address questions that mirror real-world variability and complexity. * Prizes For Winners: Compete for a chance to win a part of the $31,500 prize pool, with the top performing teams in each task winning up to $4,000. Challenge Timeline * Website Online and Registration Begin: 20th March, 2024 23:55 UTC * Phase 1 Start Date: 1st April, 2024 23:55 UTC * Phase 1 End Date: 20th May, 2024 23:55 UTC * Phase 2 Start Date: 22nd May, 2023 23:55 UTC * Registration and Team Freeze Deadline: 31st May, 2024 23:55 UTC * Phase 2 End Date: 20th Jun, 2023 23:55 UTC * Winner Notification: 15th July, 2024 * Winner Announcement: 26th August, 2024 (KDD Cup Winners) 👉 Engage Now: Begin this journey by delving into the challenge details at https://www.aicrowd.com/challenges/meta-comprehensive-rag-benchmark-kdd-cup…. Join a community of innovative thinkers, share ideas, and engage in this exciting challenge. Connect with us on our Community Forum and Discord Server for support and collaboration. We're eager to see the innovations you'll bring to life. Refer to: https://www.aicrowd.com/challenges/meta-comprehensive-rag-benchmark-kdd-cup… for the competition rules. All the best, Team AIcrowd *NO PURCHASE NECESSARY TO ENTER/WIN. Open to individuals who are 18 + age of majority and meet the full eligibility requirements in the full Rules. Open 3/20/2024 10:00:01 AM thru 6/15/2024 23:59:59 PM PT. Void where prohibited. Subject to full Rules at https://www.aicrowd.com/challenges/meta-comprehensive-rag-benchmark-kdd-cup…. See Rules for prize details and values. Sponsor: Meta Platforms, Inc. 1 Hacker Way, Menlo Park, California 94025 (for US entrants) or Meta Ireland Limited, 4 Grand Canal Square, Dublin 2, Ireland (for all other entrants).

1 0