The 9th Biomedical Linked Annotation Hackathon (BLAH9)

13 - 17 January, 2025

Tachikawa, Tokyo, Japan

Submission due of project proposals : 18 Oct., 2024

SPECIAL THEME

Ensuring Robustness in LLM-based Research: Reproducibility, Interoperability, and Reliable Evaluation.

INTRODUCTION

BLAH (Biomedical Linked Annotation Hackathon) represents a series of annual hackathon events, specifically designed to foster open collaboration. The goal is to achieve a breakthrough in the sharing and linking of various resources for biomedical literature annotation and mining. By enhancing the interoperability of these resources, the initiative aims to substantially increase both the productivity and the impact within the community.

The 9th edition of BLAH (BLAH9) will be held under the special theme "Ensuring Robustness in LLM-based Research: Reproducibility, Interoperability, and Reliable Evaluation."
Reproducibility and reliable evaluation are key to ensure that research remains robust and trustworthy. However, with the recent surge in research using large language models (LLMs), these important principles have become largely unclear. Interoperability, a vital component for fostering robust collaboration and promoting open science, has similarly faced challenges as LLM-based research expands. Now, two years into the surge of LLM-based research, it is an opportune moment to reassess and prioritize these critical aspects of research and development to ensure long-term sustainability and rigor in the field.

CALL FOR PROJECT PROPOSALS

We are seeking project proposals from individuals and teams interested in advancing biomedical literature annotation and mining, with a particular focus this year on enhancing reproducibility, interoperability, and reliable evaluation in the context of using large language models (LLMs). Proposals should be structured to achieve measurable outcomes through collaboration during the hackathon, with clearly defined objectives that can lead to meaningful insights by the end of the event.

Suggested proposal topics may include, but are not limited to:

- Enhancing interoperability in LLM-based annotation and mining

- Developing reliable evaluation frameworks for LLM-based annotation and mining

- Improving reproducibility in LLM-based annotation and mining

- ...

Submission due of project proposals is 18 Oct., 2024

TRAVEL SUPPORT

Those who submit project proposals are eligible to apply for travel support. See the homepage for detailed information.

PROGRAM COMMITTEE

- Jin-Dong Kim - DBCLS, ROIS-DS

- Fabio Rinaldi - IDSIA

- Zhiyong Lu - NCBI, NLM

- Lars Juhl Jensen - Univ. Copenhagen