2nd Call for Shared Task Participation on Visually Grounded Story Generation

Introducing the Visually Grounded Story Generation (VGSG https://vgsg2024.github.io/) Challenge, a shared task at the Generation Challenges workshop at INLG 2024 https://inlg2024.github.io/in Tokyo, Japan!

This task invites enthusiasts in vision-and-language integration to push the boundaries of vision-based story generation. This shared task aims to generate stories grounded on a sequence of images. Participants will explore the intricate relations between visual prompts and textual stories, fostering innovations in techniques for coherent, visually grounded story generation. This task is particularly challenging because: 1) the output story should be a narratively coherent text with multiple sentences, and 2) the protagonists in the generated stories need to be grounded in the images. In addition, we also elicit submissions to fine-tune existing multimodal models efficiently, which is exciting for both industry and academics.

Major progress has been made by the industry in vision-based text generation in the last few years. While generating descriptions from images or videos is widely available, grounded story generation is not evaluated with a benchmark. Investigating vision-based story generation requires several capabilities including commonsense reasoning and content planning.

We're calling for submissions that demonstrate novel approaches to generating narratives that are not only coherent but also visually grounded. We particularly welcome participants to explore the following aspects of multimodal story generation:

*

Retrieval Augmented Generation

*

Active Learning or Curriculum Learning

*

Human-in-the-loop Generation

*

Psycholinguistics-Driven Methods

*

Parameter-Efficient Fine-Tuning

*

Self-Supervised Learning

*

Reinforcement Learning

*

Generative Adversarial Network and Diffusion Models

Both automatic and human evaluations will be used. Participants can submit a 4-page report, which will be peer-reviewed and included in the INLG 2024 proceedings if accepted. Participants will present their systems in a poster session at the Generation Challenges workshop.

Whether it's through advanced computational models, innovative narrative techniques, or new data processing methods, your contribution can help illuminate the paths toward more intuitive, human-like story generation from visual inputs. Join us in this endeavor to bridge the gap between visual prompts and story generation, and help set the stage for the next leap in vision and language research.

Important dates:

Apr 15, 2024: Release of Data (see website for download)

June 15, 2024: Team Registration Deadline

June 15, 2024: System Submission for Validation Open

June 23, 2024: System Submission for Test Open

July 1, 2024: System Submission Deadline

July 8, 2024: System Report Submission Deadline

July 23-Aug 8, 2024: Evaluation Period

Aug 8, 2024: System Report Review Notification

Aug 15, 2024: Camera Ready Deadline for System Reports

Sep 23-24: Presentation at INLG 2024

To participate, please fill in this Google Form https://forms.gle/W4rrXDJVRoNkFDQs9(https://forms.gle/W4rrXDJVRoNkFDQs9 https://forms.gle/W4rrXDJVRoNkFDQs9). For more information, visit the VGSG website https://vgsg2024.github.io/(https://vgsg2024.github.io/ https://vgsg2024.github.io/) or read our shared task proposal https://aclanthology.org/2023.inlg-genchal.3.pdf(https://aclanthology.org/2023.inlg-genchal.3.pdf https://aclanthology.org/2023.inlg-genchal.3.pdf).