tl;dr:
-
submission deadline for research track paper via Softconf: December 18th 2023 -
submission deadline for research track submissions already reviewed via ARR: January 17th 2024
-
submission deadline for shard task systems: January 20th 2024 -
submission deadline for shard task system descriptions via SoftConf: January 26th 2024
https://sites.google.com/view/dialogue-evaluation/
Call for Papers
The aim of this workshop is to bring together experts working in the area of open-domain dialogue. In this speedily advancing research area many challenges still exist, such as learning information from conversations, engaging in realistic and convincing simulation of human intelligence, reasoning, and so on.
SCI-CHAT follows previous workshops on open domain dialogue, but with a focus on the simulation of intelligent conversation, including the ability to follow a challenging topic over a multi-turn conversation, the ability to posit questions, refuting and reasoning with live human evaluation employed as the primary mechanism for evaluating models. The workshop will include a research track and shared task:
SCI-CHAT's research track aims to explore recent advances and challenges in open-domain dialogue research. Researchers working on all aspects of open-domain dialogue are invited to submit papers on recent advances, resources, tools, analysis, evaluation, and challenges on the broad theme of open-domain dialogues.
The topics of the workshop include but are not limited to the following:
-
Intelligent conversation, chit-chat, open-domain dialogue; -
Automatic and human evaluation of open-domain dialogue; -
Limitations, risks and safety in open-domain dialogue; -
Instruction-tuned and instruction-enabled models; -
Any other topic of interest to the dialogue community.
SCI-CHAT's shared task will focus on simulating intelligent conversations; participants will be asked to submit (access to the APIs of) automated dialogue agents with the aim of carrying out nuanced conversations over multiple dialogue turns. Participating systems will be interactively evaluated in a live human evaluation. All data acquired within the context of the shared task will be made public, providing an important resource for improving metrics and systems in this research area.
Submission guidelines:
Authors are invited to submit their unpublished work that represents novel research through either direct submission or ARR commitment. Papers should consist of up to 8 pages of content, plus unlimited pages for references and appendix. Authors should make use of the EACL Latex Template https://2023.eacl.org/calls/styles/ alongside supplementary materials, including technical appendices, links to source code, datasets, and multimedia appendices.
Papers can also be submitted as non-archival, so that their content can be reused for other venues by adding "(NON-ARCHIVAL)" to the title of their submission. Previously published work can also be submitted as non-archival in the same way, with the additional requirement to state such on the first page.
-
Direct paper submissions must be submitted through SoftCon submission link: https://softconf.com/eacl2024/SCI-CHAT-2024/
Multiple submissions of the same paper to more EACL workshops are forbidden.
All papers will be double-blind peer-reviewed, by at least 2 program committee members. As such, all submissions, including the main paper and its supplementary materials, should be fully anonymized. For more information on formatting and anonymity guidelines, please refer to EACL guidelines https://eacl.org/index.html.
Organizers
-
Yvette Graham (Trinity College Dublin, Ireland) -
Qun Liu (Huawei Noah's Ark Lab, China) -
Gerasimos Lampouras (Huawei Noah's Ark Lab,UK) -
Ignacio Iacobacci (Huawei Noah's Ark Lab, UK) -
Sinead Madden (Trinity College Dublin, Ireland) -
Haider Khalid (Trinity College Dublin, Ireland) -
Rameez Qureshi (Trinity College Dublin, Ireland)
Important Dates
Regarding Research Track:
-
Research paper via Softconf: December 18th 2023 -
Pre-reviewed ARR commitment deadline: January 17th 2024 -
Notification of research paper acceptance: January 20th, 2024 -
Camera-ready papers due: January 30th 2024
Regarding Shared Task:
-
Release of training and development data: November 9th 2023 -
Release of baseline systems: November 9th 2023 -
Preliminary System submission deadline: January 13th 2024 (optional - if you want help testing your API, please submit early) -
System submission (API) deadline: January 20th 2024 -
System description paper via SoftConf: January 26th 2024 -
Camera-ready papers due: January 30th 2024
Overview of results at one-day workshop: March 21 or 22, 2024
CONTACT: sci-chat@adaptcentre.ie