We offer a 3-year postdoctoral position in NLP at the University of Oslo, Norway, on the topic "Evaluating large language models - model architectures, training regimes and data selection". The application deadline is April 14, 2024. This position is funded by the DSTrain program (https://www.uio.no/dscience/english/dstrain/).

In the past years, (generative) large language models have become the core foundation models for a wide range of traditional NLP tasks, and they have also seen widespread adoption by the general public. At the same time, little is known about the specific training setups of commercial models, and some design decisions (in terms of model architecture, training regimes, and data selection) are based on traditions rather than empirical or theoretical considerations. Moreover, most current LLMs rely heavily on English training and evaluation data, and their performance on non-English languages remains difficult to assess. Potential candidates are expected to formulate their research project within the broad area of LLM evaluation. Examples of research topics are given below: - Compare fine-tuning external pre-trained LLMs with training language-specific LLMs from scratch. - Compare encoder-decoder LLMs with decoder-only LLMs. - Evaluate generative LLMs on various text generation tasks, such as summarization, simplification, text normalization. - Assess the multilingual (e.g. machine translation) and cross-lingual capabilities (cross-lingual transfer) of LLMs. - Investigate how closely related low-resource languages are best accommodated in LLMs. - Implement benchmarking datasets for LLM evaluation.

Applicants are expected to submit a research project that fits in the proposed research theme (Evaluaing large language models). Prospective applicants are encouraged to discuss their application with the contact person (me) to explore scientific focus and cooperation possibilities.

The application process for the DSTrain call is described here: https://www.uio.no/dscience/english/dstrain/guide-for-applicants/application...

This is the relevant research theme description: https://www.uio.no/dscience/english/dstrain/research-areas/informatics/evalu...

Please apply here: https://www.jobbnorge.no/en/available-jobs/job/255679/dstrain-msca-postdocto...

Contact: Yves Scherrer, LTG, University of Oslo yves.scherrer@ifi.uio.no