Dear colleagues and friends, We would like to invite you to participate in the TREC 2025 BioGen Track, which focuses on evaluating the reliability and transparency of large language models (LLMs) in the biomedical domain.
Tasks Building on the TREC 2024 pilot task, this year’s track introduces two key challenges: Reference Attribution – Identify source documents that support LLM-generated responses to biomedical questions.
Answer Grounding – Cite references for each assertion in an answer to a biomedical question to ensure factual accuracy.
These tasks aim to reduce hallucinations and promote the generation of evidence-based answers in biomedical applications.
Track Website https://trec-biogen.github.io/docs/
Timeline Dataset Release: June 2, 2025 Baseline and Starter Kit Release: June 2, 2025 Results Submission Deadline: August 15, 2025 Evaluation Results Release: Late September, 2025 Notebook Paper Due: Late October, 2025 TREC 2025 Conference: November 17-21
Registrations Please follow the TREC 2025 registration guidelines from their Call for Participation.
Communication Join our Google Group for important updates! If you have any questions, ask in our Google Group or email us.
Organizers Deepak Gupta - National Library of Medicine, NIH Dina Demner-Fushman - National Library of Medicine, NIH Bill Hersh - Oregon Health & Science University Steven Bedrick - Oregon Health & Science University Kirk Roberts - University of Texas, Houston
Thank you