[apologies if you receive multiple copies of this call]
NTCIR-18 tasks: https://research.nii.ac.jp/ntcir/ntcir-18/tasks.html NTCIR-18 registration: https://research.nii.ac.jp/ntcir/ntcir-18/howto.html
____________________________________________________________
NTCIR-18 (June 10-13, 2025, Tokyo, Japan) now calls for task participation of anyone interested in research on information access technologies and their evaluation, such as retrieval from a large amount of document collections, question answering and natural language processing. We welcome students, young researchers, professors who supervise students, researchers working for a company, and anyone who is interested in informatics.
== What is NTCIR? == Development of Information Access technologies based on techniques of Information Retrieval, Natural Language Processing, and Database Management becomes increasingly more important for many applications, (e.g., providing effective access to Web resources and text archives, and analyzing big data obtained from various kinds of sensors). It is indispensable for developing such technologies to experimentally evaluate them by using test collections constructed under collaborations of many researchers. Over the past 20 years, NTCIR has been formulating the infrastructure for the evaluation, and contributing to development of the Information Access technologies. A total of over 80 "evaluation tasks" have been organized, attracting over 1,000 participant research groups worldwide so far. Furthermore, over 4,600 research groups have signed up to use the NTCIR test collections in their research. Consequently, NTCIR has been a major forum for researchers to intensively discuss the evaluation methodology of emerging information access technologies.
== NTCIR-18 Tasks == NTCIR-18 Program Committee has selected 10 tasks. The overview slide of each task is available at the kick-off page: https://research.nii.ac.jp/ntcir/ntcir-18/kickoffcfp.html For more details, please visit the websites of each task: Task Overview page: https://research.nii.ac.jp/ntcir/ntcir-18/tasks.html ----------------------------------------------------------------------------
NTCIR-18 Tasks
Core Tasks
AEOLLM: Automatic Evaluation of LLMs
-
FairWeb-2: The Second Fair Web Task
-
sakailab.com/fairweb2/
FinArg-2: Temporal Inference of Financial Arguments
-
https://sites.google.com/nlg.csie.ntu.edu.tw/ntcir-18-finarg-2/ https://sites.google.com/nlg.csie.ntu.edu.tw/ntcir-18-finarg-2/finarg-2
Lifelog-6: Personal Lifelog Organisation & Retrieval Task
-
http://lifelogsearch.org/ntcir-lifelog/
RadNLP: Natural Language Processing for Radiology
-
https://sociocom.naist.jp/radnlp-2024
MedNLP-CHAT Medical Natural Language Processing for AI Chat
-
https://sociocom.naist.jp/mednlp-chat
Transfer-2: Resource Transfer Based Dense Retrieval
-
https://github.com/ntcirtransfer/transfer2/discussions
Pilot Tasks
HIDDEN-RAD: Hidden Causality Inclusion in Radiology Report Generation
-
https://sites.google.com/view/ntcir-18-hidden-rad/hidden-rad
SUSHI: Searching Unseen Sources for Historical Information
-
https://sites.google.com/view/ntcir-sushi-task/
U4: Unifying, Understanding, and Utilizing Unstructured Data in Financial Reports
-
https://sites.google.com/view/ntcir18-u4/
== How to Participate == 1. Please carefully read "How to Participate to NTCIR-18 Task(s)": https://research.nii.ac.jp/ntcir/ntcir-18/howto.html
2. Please register at https://ntcir.nii.ac.jp/index.php/ntcir-18-registration-form/
The datasets of each task will be delivered to the team after registration (the date may vary depending on the task).
Registration Due (Depend on the task): Nov. 1st, 2024 (Passed)
-
FairWeb-2: The Second Fair Web Task
Nov. 15th, 2024
-
SUSHI Searching Unseen Sources for Historical Information
Dec. 1st, 2024
-
RadNLP: Natural Language Processing for Radiology
Dec. 16th, 2024
-
Transfer-2: Resource Transfer Based Dense Retrieval
Dec. 28th, 2024
-
U4: Unifying, Understanding, and Utilizing Unstructured Data in Financial Reports
Jan 8th, 2025
-
HIDDEN-RAD: Hidden Causality Inclusion in Radiology Report Generation
Jan 15th, 2025
-
AEOLLM: Automatic Evaluation of LLMs -
FinArg-2: Temporal Inference of Financial Arguments -
Lifelog-6: Personal Lifelog Organisation & Retrieval Task
Jan 17th, 2025
-
MedNLP-CHAT: Medical Natural Language Processing for AI Chat
== Schedule == (*Schedule can be different for different tasks. Please visit webpages of each task for the details.)
Sep 2024-Feb 2025 Formal run*
Feb 1, 2025 Evaluation results return
Feb 1, 2025 Task overview release (draft)
Mar 1, 2025 Submission due of participant papers (draft)
May 1, 2025 Camera-ready participant paper due
Jun 10-13, 2025 NTCIR-18 Conference
(At the NTCIR-18 Conference, an online presentation will be available)
== Questions? == For information regarding the task specifications etc., please contact the NTCIR Program Chairs: ntc18-pcc@nii.ac.jp or task organizers.
For information regarding the online registration and previous NTCIR test collections, please contact the NTCIR office: ntc-secretariat@nii.ac.jp
We are looking forward to your participation!
NTCIR-18 Program Co-Chairs:
Qingyao Ai (Tsinghua University, China)
Chung-Chi Chen (AIST, Japan)
Shoko Wakamiya (NAIST, Japan)
NTCIR-18 General Co-Chairs: Charles Clarke (University of Waterloo, Canada)
Noriko Kando (National Institute of Informatics, Japan)
Makoto P. Kato (University of Tsukuba, Japan)
Yiqun Liu (Tsinghua University, China)