Dear all,

The Regulations Challenge aims to push the boundaries of LLMs in understanding, interpreting, and applying regulatory knowledge in the finance industry. In this challenge, participants will participate in 9 tasks to explore key issues, including, but not limited to, regulatory complexity, ethical considerations, domain-specific terminology, industry standards, and interpretability. We welcome students, researchers, and practitioners who are passionate about finance and LLMs. We encourage participants to develop solutions that advance the capabilities of LLMs in addressing the challenges of financial regulations and industry standards.

These tasks assess the LLM's ability to handle different types of questions within the regulatory domain which follow: - Abbreviation Recognition Task: Goal: Match an abbreviation with its expanded form. Input Template: "Expand the following acronym into its full form: {acronym}. Answer:"

- Definition Recognition Task: Goal: Correctly define a regulatory term or phrase. Input Template: "Define the following term: {regulatory term or phrase}. Answer:"

- Named Entity Recognition (NER) Task: Goal: Ensure the output correctly identifies entities and places them into groups that the user specifies. Input Template: "Given the following text, only list the following for each: specific Organizations, Legislations, Dates, Monetary Values, and Statistics: {input text}."

- Question Answering Task: Goal: Ensure the output matches the correct answer to a detailed question about regulatory practices or laws. Input Template: "Provide a concise answer to the following question: {detailed question}? Answer:"

- Link Retrieval Task: Goal: Ensure the link output matches the actual law. Input Template: "Provide a link for ____ law, Write in the format of ("{Law}: {Link}" or "{Law}: Not able to find a link for the law")"

- Certificate Question Task: Goal: Select the correct answer choice to a question that may be based on additional context. Input Template: "(This context is used for the question that follows: {context}). Please answer the following question with only the letter and associated description of the correct answer choice: {question and answer choices}. Answer:"

- XBRL Analytics Task: Goal: Ensure the output strictly matches the correct answer to a detailed question about financial data extraction and application tasks via XBRL filings. These standardized digital documents contain detailed financial information. Input Template: "Provide the exact answer to the following question: {detailed question}? Answer:"

- Common Domain Model (CDM) Task: Goal: Deliver precise responses to questions about the Fintech Open Source Foundation's (FINOS) Common Domain Model (CDM). Input Template: "Provide a concise answer to the following question related to Financial Industry Operating Network's (FINO) Common Domain Model (CDM): {detailed question}? Answer:"

- Model Openness Framework (MOF) Licenses Task: Goal: Deliver precise responses to questions concerning the requirement of license under the Model Openness Framework. Input Template: "Provide a concise answer to the following question about MOF's licensing requirements: {detailed question}? Answer:"

The final score is determined by the weighted average of metrics for 9 tasks. We assign the weight of 10% to Task 1-5 each, 20% to Task 6, and 10% to Task 7-8 each.

Important Dates Training Set Release: September 15, 2024 Training Data Details: Summary of Question Dataset Validation Set Release: October 30, 2024 Systems Submission: November 7, 2024 Release of Results: November 12, 2024 Paper Submission Deadline: November 25, 2024 Notification of Acceptance: December 5, 2024 Camera-ready Paper Deadline: December 13, 2024 Workshop Date: January 19-20, 2025

Task Organizers Keyi Wang, Columbia University, Northwestern University Lihang (Charlie) Shen, Columbia University Haoqiang Kang, Columbia University Xingjian Zhao, Rensselaer Polytechnic Institute Namir Xia, Rensselaer Polytechnic Institute Christopher Poon, Rensselaer Polytechnic Institute Jaisal Patel, Rensselaer Polytechnic Institute Andy Zhu, Rensselaer Polytechnic Institute Shengyuan Lin, Rensselaer Polytechnic Institute Daniel Kim, Rensselaer Polytechnic Institute Jaswanth Duddu, Rensselaer Polytechnic Institute Matthew Tavares, Rensselaer Polytechnic Institute Shanshan Yang, Stevens Institute of Technology Sai Gonigeni, Stevens Institute of Technology Kayli Gregory, Stevens Institute of Technology Katie Ng, Stevens Institute of Technology Andrew Thomas, Stevens Institute of Technology Dong Li, FinAI

Supervisors Yanglet Xiao-Yang Liu, Rensselaer Polytechnic Institute, Columbia University Steve Yang, School of Business at Stevens Institute of Technology Kairong Xiao, Roger F. Murray Associate Professor of Business at Columbia Business School Matt White, Executive Director, PyTorch Foundation. GM of AI, Linux Foundation Cailean Osborne, University of Oxford Wes Turner, Rensselaer Center for Open Source (RCOS), Rensselaer Polytechnic Institute Neha Keshan, Rensselaer Polytechnic Institute Luca Borella, PM of AI Strategic Initiative, FINOS Ambassador, Linux Foundation Karl Moll, Technical Project Advocate, FINOS, Linux Foundation

For more details, please visit https://coling2025regulations.thefin.ai/ or contact colingregchallenge2025@gmail.com

Best regards, Jimin Huang The Fin AI (https://thefin.ai)