The Department of Computer Science at the IT University of Copenhagen is offering a Postdoc position in Natural Language Processing/Computational Linguistics*,* with a start date of *1 September 2024* or as soon as possible. The *application deadline is 31* *May** 2024.* Applications for the position can be submitted via ITU job portal https://candidate.hr-manager.net/ApplicationInit.aspx?cid=119&ProjectId=181688&DepartmentId=3439&MediaId=5 .
*Proposed project title: *Efficiency and Robustness in Language Model Pre-training
*Proposed project description.* Recent generative systems based on pre-trained language models are remarkably fluent, but this is achieved by extreme volumes of computation and training data. This means not only high energy costs, but also training on data that is problematic in various ways: copyright, harmful social stereotypes, non-representative sampling, misinformation, junk SEO texts, pornography, and contamination with NLP datasets used for evaluation.
This project will create an ambitious resource for research on transfer learning, in which pre-training data is held constant, and evaluation takes into account how much similar data was observed in training, and in what ways it was similar. This resource will encourage the development of more efficient and robust approaches, since it will not be possible to improve benchmark scores by simply training on more data.
The ideal candidate will have a strong background in Computational Linguistics/Natural Language Processing and experience developing NLP resources, as well as core skills in programming in Python and machine learning.
The position is funded for 1 year, and it is our intention to find additional funding to extend this postdoc to a 2- or 3-year position. Besides research, the postdoc will gain experience with organization of an international workshop and shared task and build up their international network. For those interested in pursuing an academic career, the following is also possible (but entirely optional):
- gain experience in applying for external funding with professional support (either for the continuation of the postdoc’s own position, e.g. Marie Curie postdoctoral fellowship, or by contributing to PI’s grant proposals); - supervise Master students solo, and/or assist in supervising a PhD student; - undertake a formal teacher training program, including teaching guest lectures in the relevant data science courses at the ITU computer science department.
The successful candidate will be a member of the national Pioneer Centre for Artificial Intelligence https://aicentre.dk/, a 5-university Danish research endeavor, and of the NLPnorth https://nlpnorth.github.io/research group at the IT University’s Computer Science Department. Both the centre and research group are highly international and well-funded, working on a broad range of research topics.
The project will be supervised by Associate Professor Anna Rogers https://annargrs.github.io/ (arog@itu.dk), to whom inquiries about the project can be directed.