CALL FOR NTCIR-18 TASK PARTICIPATION

NTCIR-18 tasks: http://research.nii.ac.jp/ntcir/ntcir-18/tasks.html NTCIR-18 registration: http://research.nii.ac.jp/ntcir/ntcir-18/howto.html ____________________________________________________________ NTCIR-18 (June 10-13, 2025, Tokyo, Japan) now calls for task participation of anyone interested in research on information access technologies and their evaluation, such as retrieval from a large amount of document collections, question answering and natural language processing. We welcome students, young researchers, professors who supervise students, researchers working for a company, and anyone who is interested in informatics.

== What is NTCIR? == Development of Information Access technologies based on techniques of Information Retrieval, Natural Language Processing, and Database Management becomes increasingly more important for many applications, (e.g., providing effective access to Web resources and text archives, and analyzing big data obtained from various kinds of sensors). It is indispensable for developing such technologies to experimentally evaluate them by using test collections constructed under collaborations of many researchers. Over the past 20 years, NTCIR has been formulating the infrastructure for the evaluation, and contributing to development of the Information Access technologies. A total of over 80 “evaluation tasks” have been organized, attracting over 1,000 participant research groups worldwide so far. Furthermore, over 4,600 research groups have signed up to use the NTCIR test collections in their research. Consequently, NTCIR has been a major forum for researchers to intensively discuss the evaluation methodology of emerging information access technologies.

== NTCIR-18 Tasks == NTCIR-18 Program Committee has selected 10 tasks. The overview slide of each task can be available at the kick-off page: https://research.nii.ac.jp/ntcir/ntcir-18/kickoffcfp.html For more details, please visit the websites of each task: Task Overview page: http://research.nii.ac.jp/ntcir/ntcir-18/tasks.html ---------------------------------------------------------------------------- NTCIR-18 Tasks Core Tasks AEOLLM: Automatic Evaluation of LLMs https://aeollm.github.io/

FairWeb-2: The Second Fair Web Task sakailab.com/fairweb2/

FinArg-2: Temporal Inference of Financial Arguments https://sites.google.com/nlg.csie.ntu.edu.tw/ntcir-18-finarg-2/finarg-2

Lifelog-6: Personal Lifelog Organisation & Retrieval Task http://lifelogsearch.org/ntcir-lifelog/

RadNLP: Natural Language Processing for Radiology https://sociocom.naist.jp/radnlp-2024

MedNLP-CHAT Medical Natural Language Processing for AI Chat https://sociocom.naist.jp/mednlp-chat

Transfer-2: Resource Transfer Based Dense Retrieval https://github.com/ntcirtransfer/transfer2/discussions

Pilot Tasks HIDDEN-RAD: Hidden Causality Inclusion in Radiology Report Generation https://sites.google.com/view/ntcir-18-hidden-rad/hidden-rad

SUSHI: Searching Unseen Sources for Historical Information https://sites.google.com/view/ntcir-sushi-task/

U4: Unifying, Understanding, and Utilizing Unstructured Data in Financial Reports https://sites.google.com/view/ntcir18-u4/

== How to Participate == 1. Please carefully read “How to Participate to NTCIR-18 Task(s)“: http://research.nii.ac.jp/ntcir/ntcir-18/howto.html 2. Please register at http://ntcir.nii.ac.jp/index.php/ntcir-18-registration-form/ The datasets of each task will be delivered to the team after registration (the date may vary depending on the task). Registration Due (Depend on the task): Nov. 1st, 2024 AEOLLM: Automatic Evaluation of LLMs FairWeb-2: The Second Fair Web Task RadNLP: Natural Language Processing for Radiology MedNLP-CHAT: Medical Natural Language Processing for AI Chat HIDDEN-RAD: Hidden Causality Inclusion in Radiology Report Generation Dec. 15th, 2024 Transfer-2: Resource Transfer Based Dense Retrieval SUSHI Searching Unseen Sources for Historical Information U4: Unifying, Understanding, and Utilizing Unstructured Data in Financial Reports Jan 8th, 2025 FinArg-2: Temporal Inference of Financial Arguments Lifelog-6: Personal Lifelog Organisation & Retrieval Task == Schedule == (*Schedule can be different in different tasks. Please visit webpages of each task for the details.) May 2024 Dataset release* Jun-Dec 2024 Dry run* Sep 2024-Feb 2025 Formal run* Feb 1, 2025 Evaluation results return Feb 1, 2025 Task overview release (draft) Mar 1, 2025 Submission due of participant papers (draft) May 1, 2025 Camera-ready participant paper due Jun 10-13, 2025 NTCIR-18 Conference (At the NTCIR-18 Conference, an online presentation will be available)

== Questions? == For information regarding the task specifications etc., please contact the NTCIR Program Chairs: ntc18-pcc@nii.ac.jp or task organizers. For information regarding the online registration and previous NTCIR test collections, please contact the NTCIR office: ntc-secretariat@nii.ac.jp

We are looking forward to your participation!

NTCIR-18 Program Co-Chairs: Qingyao Ai (Tsinghua University, China) Chung-Chi Chen (AIST, Japan) Shoko Wakamiya (NAIST, Japan)

NTCIR-18 General Co-Chairs: Charles Clarke (University of Waterloo, Canada) Noriko Kando (National Institute of Informatics, Japan) Makoto P. Kato (University of Tsukuba, Japan) Yiqun Liu (Tsinghua University, China)