We invite you to take part in *CLEF 2023 LongEval: Longitudinal Evaluation
of Model Performance shared task.*
*Updated Timelines:*
--------------------------------
Deadline for registration: 21st May 2023
Test Data release: 4th May 2023
Submissions Due: 22nd May 2023
*You can now directly register on the Codalab website: *
https://codalab.lisn.upsaclay.fr/competitions/12762
*Join the Slack channel*:
https://join.slack.com/t/longeval/shared_invite/zt-1tmql633q-w20e3oNq157bkh…
The CLEF 2023 LongEval lab is motivated by recent research showing that the
performance of information retrieval and text classification models drops
as the test data becomes more distant in time from the training data.
*LongEval-Classification:* The goal of Task 2 is to propose a temporal
persistence classifier which can mitigate performance drop over short and
long periods of time compared to a test set from the same time frame as
training.
Organisers
------------------------------------------------------------
Rabab Alkhalifa, Iman Bilal, Hsuvas Borkakoty, Jose Camacho-Collados,
Romain Deveaud, Alaa El-Ebshihy, Luis Espinosa-Anke, Gabriela
Gonzalez-Saez, Petra Galuščáková, Lorraine Goeuriot, Elena Kochkina, Maria
Liakata, Daniel Loureiro, Harish Tayyar Madabushi, Philippe Mulhem, Florina
Piroi, Martin Popel, Christophe Servan, Arkaitz Zubiaga.
Dear all
The *Journal of Research Design and Statistics in Linguistics and
Communication Science* is hereby soliciting submissions of papers as
well as proposals for guest-edited special issues and is encouraging
all corpus linguists working with quantitative methods to check out
our call published on the journal website at
<https://journal.equinoxpub.com/JRDS/announcement/view/312>. There,
you will also find more info on the range of topics/fields we're
particularly interested in as well as contact information for the
editors.
Cheers,
STG
UC Santa Barbara & JLU Giessen
Fully funded PhD positions in recommendation and search
Come and join us to work as a PhD student! At the IRLab of the University Amsterdam we have two fully funded, four-year PhD positions on the following topics:
* Full page personalization in recommender systems; this is a position in the TAIM Lab, a collaborative research lab with RTL, Maastricht University, and the University of Amsterdam
* Truthful generative methods for news content; this is a position in the REM Lab, a collaborative research lab with DPG Media, the Jheronymus Academy of Data Science, and the University of Amsterdam.
The TAIM and REM Lab are part of the national Innovation Center for AI (ICAI). To find out more about these positions and to apply, please check out https://staff.fnwi.uva.nl/m.derijke/two-phd-positions/
--
Maarten de Rijke
Distinguished University Professor AI & IR
University of Amsterdam
http://staff.fnwi.uva.nl/m.derijke
Third Workshop on Human Evaluation of NLP Systems (HumEval’23)
###############################################################
https://humeval.github.io/
RANLP’23, Varna, Bulgaria, 7 or 8 September 2023
First Call for Papers
++++++++++++++++++++++
The Third Workshop on Human Evaluation of NLP Systems (HumEval’23) invites
the submission of long and short papers on substantial, original, and
unpublished research on all aspects of human evaluation of NLP systems with
a focus on NLP systems which produce language as output. We welcome work on
any quality criteria relevant to NLP, on both intrinsic evaluation (which
assesses systems and outputs directly) and extrinsic evaluation (which
assesses systems and outputs indirectly in terms of its impact on an
external task or system), on quantitative as well as qualitative methods,
score-based (discrete or continuous scores) as well as annotation-based
(marking, highlighting).
Important dates
----------------
Workshop paper submission deadline: 10 July 2023
Workshop paper acceptance notification: 5 August 2023
Workshop paper camera-ready versions: 25 August 2023
Workshop camera-ready proceedings ready: 31 August 2023
All deadlines are 23:59 UTC-12.
Topics
-------
We invite papers on topics including, but not limited to, the following:
Experimental design and methods for human evaluations
Reproducibility of human evaluations
Work on inter-evaluator and intra-evaluator agreement
Ethical considerations in human evaluation of computational systems
Quality assurance for human evaluation
Crowdsourcing for human evaluation
Issues in meta-evaluation of automatic metrics by correlation with
human evaluations
Alternative forms of meta-evaluation and validation of human evaluations
Comparability of different human evaluations
Methods for assessing the quality and the reliability of human
evaluations
Role of human evaluation in the context of Responsible and Accountable
AI
We welcome work from any subfield of NLP (and ML/AI more generally), with a
particular focus on evaluation of systems that produce language as output.
ReproNLP shared task
---------------------
The workshop will also host a shared Task on Reproducibility of Evaluations
in NLP (ReproNLP) -- more details coming soon.
Papers
------
Long papers
- - - - - -
Long papers must describe substantial, original, completed and unpublished
work. Wherever appropriate, concrete evaluation and analysis should be
included. Long papers may consist of up to eight (8) pages of content, plus
unlimited pages of references. Final versions of long papers will be given
one additional page of content (up to 9 pages) so that reviewers’ comments
can be taken into account. Long papers will be presented orally or as
posters as determined by the programme committee. Decisions as to which
papers will be presented orally and which as posters will be based on the
nature rather than the quality of the work. There will be no distinction in
the proceedings between long papers presented orally and as posters.
Short papers
- - - - - - -
Short paper submissions must describe original and unpublished work. Short
papers should have a point that can be made in a few pages. Examples of
short papers are a focused contribution, a negative result, an opinion
piece, an interesting application nugget, a small set of interesting
results. Short papers may consist of up to four (4) pages of content, plus
unlimited pages of references. Final versions of short papers will be given
one additional page of content (up to 5 pages) so that reviewers’ comments
can be taken into account. Short papers will be presented orally or as
posters as determined by the programme committee. While short papers will
be distinguished from long papers in the proceedings, there will be no
distinction in the proceedings between short papers presented orally and as
posters.
Multiple submission policyPermalink
HumEval’23 allows multiple submissions. However, if a submission has
already been, or is planned to be, submitted to another event, this must be
clearly stated in the submission form.
Submission procedure and templates
-----------------------------------
To submit, go directly to the workshop page at the Softconf START system
https://softconf.com/ranlp23/HumEval/
The papers should follow the format of the main conference, described at
the RANLP website, Submissions page.
http://ranlp.org/ranlp2023/index.php/submissions/
Organisers
Anya Belz, ADAPT Centre, Dublin City University, Ireland
Maja Popović, ADAPT Centre, Dublin City University, Ireland
Ehud Reiter, University of Aberdeen, UK
João Sedoc, New-York University
Craig Thomson, University of Aberdeen, UK
For questions and comments regarding the workshop please contact the
organisers at humeval.ws(a)gmail.com.
*** Fifth Call for Submissions ***
10th European Conference On Service-Oriented And Cloud Computing (ESOCC 2023)
October 24-26, 2023, Golden Bay Beach Hotel, Larnaca, Cyprus
https://cyprusconferences.org/esocc2023/
(Proceedings to be published in Springer LNCS;
Journal Special Issue with Springer Computing)
AIM AND SCOPE
Nowadays, Service-Oriented and Cloud Computing are the primary approaches to build
large-scale distributed systems and deliver software services to end users. Cloud-native
software is pervading the delivery of enterprise applications, as they are composed of
(micro)services that can be independently developed and deployed by exploiting multiple
heterogeneous technologies. Resulting applications are polyglot service compositions that can
then be shipped in serverful or serverless platforms (e.g., using virtualization technologies).
These characteristics make Service-Oriented and Cloud Computing the natural answers for
fulfilling the industry’s need for flexibly scalable and maintainable enterprise applications, to
be delivered through state-of-the-art methodologies, like DevOps. To further support this,
researchers and practitioners need to create methods, tools and techniques to support
cost-effective and secure development as well as use of dependable devices, platforms,
services and service-oriented applications in the Cloud, now also considering the Cloud-IoT
computing continuum to exploit widespread adoption of smart connected things and the
increasing growth of their computing capabilities.
The European Conference on Service-Oriented and Cloud Computing (ESOCC) is the premier
conference on advances in the state-of-the-art and practice of Service-Oriented Computing
and Cloud Computing in Europe. ESOCC aims to facilitate the exchange between researchers
and practitioners in the areas of Service-Oriented Computing and Cloud Computing, as well as
to explore the new trends in those areas and foster future collaborations in Europe and beyond.
TOPICS OF INTEREST
ESOCC 2023 seeks original, high-quality contributions related to all aspects of Service-Oriented
and Cloud computing. Specific topics of interest include, but are not limited to: • Applications for Service-Oriented and Cloud Computing, e.g., big data, commerce, energy,
finance, health, scientific computing, smart cities • Blockchains for Service-Oriented and Cloud Computing • Business aspects of Service-Oriented and Cloud Computing, e.g., business models,
brokerage, marketplaces, costs, pricing • Business processes, e.g., service-based workflow deployment and management • Cloud interoperability, service and Cloud standards, • Cloud-IoT computing continuum, e.g., edge computing, fog computing, mobility computing,
next generation services/IoT • Cloud-native architectures and paradigms, e.g., microservices and DevOps • Cloud service models, e.g., IaaS, PaaS, SaaS, DBaaS, FaaS, etc. • Deployment, composition, and management of applications in Service-Oriented and Cloud
Computing • Foundations and formal methods for Service-Oriented and Cloud Computing • Enablers for Service-Oriented and Cloud Computing, e.g., service discovery, orchestration,
matchmaking, monitoring, and analytics • Model-Driven Engineering for Service-Oriented and Cloud Computing • Multi-Cloud, cross-Cloud, and federated Cloud solutions • Requirements engineering, design, development, and testing of applications in
Service-Oriented and Cloud Computing • Semantic services and service mining • Service and Cloud middlewares and platforms • Software/service adaptation and evolution in Service-Oriented and Cloud Computing • Storage, computation and network Clouds • Sustainability and energy issues in Service-Oriented and Cloud Computing • Quality aspects (e.g., governance, privacy, security, and trust) of Service-Oriented and Cloud
Computing • Quality of Service (QoS) and Service-Level Agreement (SLA) for Service-Oriented and Cloud
Computing • Social aspects of Service-Oriented and Cloud Computing, e.g., crowdsourcing services, social
and crowd-based Clouds • Virtualization for Service-Oriented and Cloud Computing, e.g., serverless, container-based
virtualization, VMs
IMPORTANT DATES
• Submission of abstracts: June 4th, 2023 (AoE) • Submission of full papers: June 11th, 2023 (AoE) • Notification to authors: July 14th, 2023 (AoE) • Camera-ready versions due: July 31st, 2023 (AoE)
TYPES OF SUBMISSIONS
ESOCC 2023 invites submissions of the following kinds: • Regular Research Papers (15 pages including references, for the technical and special tracks) • PhD Symposium (12 pages including references) • Projects and Industry Reports (Projects and Industry Reports (1 to 6 pages including
references, describing an ongoing EU or national project, or providing industrial perspectives
on innovative applications, technologies, or methods in ESOCC’s scope)
We only accept original papers, not submitted for publication elsewhere. The papers must be
formatted according to the proceedings guidelines of Springer’s Lecture Notes in Computer
Science (LNCS) series (http://www.springer.com/lncs).
They must be submitted to the EasyChair site at: https://easychair.org/conferences/?conf=esocc2023 by selecting the right track.
Accepted papers from all tracks will be published in the main conference proceedings by
Springer in the LNCS series. For publication to happen, at least one author of each accepted
paper is expected to register and present the work at the conference.
The best papers accepted will be invited to submit extended versions for a Journal Special
Issue to be published by Springer Computing.
ORGANISATION
General Chair
• George A. Papadopoulos, University of Cyprus, CY
(george at ucy.ac.cy)
Program Chairs
• Florian Rademacher, University of Applied Sciences and Arts Dortmund, DE
(florian.rademacher at fh-dortmund.de) • Jacopo Soldani, University of Pisa, IT
(jacopo.soldani at unipi.it)
Steering and Program Committee
https://cyprusconferences.org/esocc2023/committees/
The 1st Workshop on Computational Terminology in NLP and Translation
Studies (ConTeNTs)
Varna, 7th-8th September, 2023
In conjunction with RANLP 2023 - International Conference "Recent
Advances in Natural Language Processing"
Second call for papers
Computational Terminology and new technologies applied to translation
studies have attracted the interest of researchers with very different
multidisciplinary backgrounds and motivations. Those fields cover a
range of areas in Natural Language Processing (NLP) such as information
retrieval, terminology extraction, question-answering systems, ontology
building, machine translation, computer-aided translation, automatic or
semi-automatic abstracting, text generation, etc.
Terminological identification, extraction and coinage of new terms are
essential for knowledge mining from texts, both in high and low
resources languages. Quick evolutions and new developments in
specialised domains require efficient and systematic automatic term
management. New terms need to be coined and translated to ensure the
equitable development of domains in all languages.
During the last decade, deep learning and neural methods have become the
state of the art for most NLP applications. Those applications were
shown to outperform previous methods on various tasks, including
automatic term extraction, language mining, assessment of quality in
machine translation, accessibility of terminology, etc. On the one hand,
NLP and computational linguistics try to improve the work of translators
and interpreters by developing Computer-Assisted Translation (CAT)
tools, Translation Memories (TMs), terminological databases and
terminology extraction tools, etc. On the other hand, the NLP field
still needs the efforts and knowledge of translators, interpreters and
linguists to provide better services and tools based on the real
necessities of those language professionals.
The aim of this workshop is to promote new insights into the ongoing and
forthcoming developments in computational terminology by bringing
together NLP experts, as well as terminologists and translators. By
uniting researchers with such diverse profiles, we hope to bridge some
of the gaps between these disciplines and inspire a dialogue between
various parties, thus paving the way to more artificial intelligence
applications based on mutual collaboration between language and
technology.
Topics of Interest
The ConTeNTs workshop invites the submission of papers reporting on
original and unpublished research on topics related to Computational
Terminology in NLP and Translation Studies, including but not limited
to:
* Automatic term extraction: monolingual and multilingual extraction
of terms from parallel and comparable corpora, including single and
multiword expressions;
* Extraction and acquisition of semantic relations between terms;
* Extraction and generation of domain specific definitions and
disambiguation of terms;
* Representation of terms, management of term variation and the
discovery of synonym terms or term clusters and its relation to NLP
applications;
* Extraction of terminological context, through the use of comparable
and parallel corpus;
* Accessibility of terminology in certain domains, relevant to
non-experts or to laypersons, and its relevance to NLP applications such
as, chatbots, automatic email generation or spoken language interface;
* The impact of terminology on MT (applying terminology constraints,
evaluation of MT in domain-specific settings, etc.);
* The creation of domain ontologies, thesaurus, terminological
resources in specialised domains;
* The use of new technologies in translation studies and research and
the use of terminological resources in specialised translation;
* Identification of key problems in terminology and new technologies
used in translation studies;
* Evaluation of terminological resources in various NLP applications
and the impact of these resources have on the performance of the
automatic systems;
* Emerging language technologies: how the increased reliance on
real-time language technologies would change the structure of language;
* Corpus based studies applied to translation and interpreting: the
use of parallel and comparable corpora for translating phraseological
units;
* Phraseology and multiword expressions in cross-linguistic studies;
* Translation and interpreting tools, such as translation memories,
machine translation and alignment tools;
* User requirements for interpreting and translation tools.
SUBMISSION GUIDELINES
Submissions must consist of full-text papers and should not exceed 7
pages excluding references, they should be a minimum of 5 pages long.
The accepted papers will be published as ConTeNTs workshop e-proceedings
with ISBN, will be assigned a DOI and will be also available at the time
of the conference. The papers should be in English.
Authors of accepted papers will receive guidelines regarding how to
produce camera-ready versions of their papers for inclusion in the
proceedings.
Each submission will be reviewed by at least two programme committee
members. Accepted papers will be presented orally as part of the
programme of the workshop.
Submissions
Link to START system: https://softconf.com/ranlp23/ConTeNTS
Website of the workshop: https://contents2023.kulak.kuleuven.be/
Should you require any assistance with the submission, please do not
hesitate to contact us at amalhaddad(a)ugr.es and
ayla.rigoutsterryn(a)kuleuven.be.
Important Dates
Deadline for paper submission: 10 July 2023
Acceptance notification: 5 August 2023
Final camera-ready version: 25 August 2023
Workshop camera-ready proceedings ready: 31 August 2023
ConTeNTs workshop: 7/8 September 2023
Workshop Chairs & Organising Committee
Ayla Rigouts Terryn, Katholieke Universiteit Leuven, Belgium
Amal Haddad Haddad, Universidad de Granada, Spain
Ruslan Mitkov, University of Wolverhampton, United Kingdom
Programme Committee
* Sophia Ananiadou (University of Manchester)
* Maria Andreeva Todorova (Bulgarian Academy of Sciences)
* Silvia Bernardini (University of Bologna)
* Melania Cabezas García (Universidad de Granada)
* Rute Costa (Universidade Nova de Lisboa)
* Esther Castillo Pérez (Universidad de Granada)
* Patrick Drouin (Université de Montréal)
* Pamela Faber (Universidad de Granada)
* Mercedes García de Quesada (Universidad de Granada)
* Dagmar Gromann (Centre for Translation Studies - University of
Vienna)
* Tran Thi Hong Hanh (L3i Laboratory, University of La Rochelle)
* Rejwanul Haque (National College of Ireland)
* Amir Hazem (Nantes University)
* Kyo Kageura (University of Tokyo)
* Barbara Karsch (BIK Terminology - USA)
* Dorothy Kenny (Dublin City University)
* Miloš Jakubíček (Sketch Engine)
* Hendrik Kockaert (KU Leuven)
* Philipp Koehn (Johns Hopkins University)
* Maria Kunilovskaya (Saarland University)
* Marie-Claude L'Homme (Université de Montréal)
* Hélène Ledouble (Université de Toulon)
* Pilar León-Araúz (Universidad de Granada)
* Rodolfo Maslias (former Head of TermCoord, European Parliament)
* Silvia Montero Martínez (Universidad de Granada)
* Emmanuel Morin (LS2N-TALN)
* Rogelio Nazar (Pontificia Universidad Católica de Valparaíso)
* Sandrine Peraldi (University College Dublin)
* Silvia Piccini (Italian National Research Council)
* Thierry Poibeau (CNRS)
* Senja Pollak (Jožef Stefan Institute)
* Maria Pozzi Pardo (El Colegio de México)
* Tharindu Ranasinghe (Aston University)
* Arianne Reimerink (Universidad de Granada)
* Andres Repar (Jožef Stefan Institute)
* Christophe Roche (Université Savoie Mont-Blanc)
* Antonio San Martín Pizarro (Université du Québec à Trois-Rivières)
* Beatriz Sánchez Cárdenas (Universidad de Granada)
* Vilelmini Sosoni (Ionian University)
* Irena Spasic (Cardiff University)
* Elena Isabelle Tamba (Romanian Academy, Iași Branch)
* Rita Temmerman (Vrije Universiteit Brussel)
* Jorge Vivaldi Palatresi (Universitat Pompeu Fabra)
We are happy to announce that we have extended the deadline for the
workshop on Individual Differences in Pragmatics and Discourse (IndiPRAG).
Extended deadline: 8th May 2023
Notification date: 5th June 2023
Workshop dates: 18th September (all day) and 19th September (morning) 2023
Workshop venue: Saarland University, Saarbrücken, Germany (the workshop is
collocated with XPRAG in Paris, 20th-23rd September)
**
Call for submissions:
Experimental research in pragmatics and discourse processing has
consistently found that not all comprehenders behave the same: while some
seem to draw rich pragmatic inferences, others respond in a way that is
more consistent with a literal interpretation (Fairchild & Papafragou,
2021; Mayn & Demberg, 2022). Similarly for discourse inference,
experiments have found differences with respect to the sensitivity to
discourse cues and the readiness for discourse predictions between
participants (Scholman, Demberg & Sanders, 2020; Tskhovrebova, Zufferey &
Gygax, 2022).
This workshop aims to bring together researchers interested in exploring
individual differences at the level of pragmatics and discourse, as well
as methods for relating those differences to cognitive properties, and
approaches for modelling the mechanism driving the individual differences
effects.
IndiPRAG Workshop invites submissions of abstracts addressing the
following questions:
- To what extent do pragmatic processing and discourse inferences differ
between individuals?
- How consistent are interpretation biases across different types of
pragmatic implicatures?
- What individual difference measures are particularly suitable for
measuring IDs related to pragmatic processing?
- How can we computationally model individual differences in discourse and
pragmatics?
- What statistical methods are best suited to identifying latent groups of
participants and relating ID measures to task performance?
**
Formatting guidelines:
The abstracts must not exceed 1000 words for the text (excl. captions),
10000 characters for references, 2 figures. Abstracts should be submitted
in PDF format, with 2.54 cm margins on all sides and 12 point font size,
single-spaced. Please indicate up to three appropriate keywords for your
abstract, which will be used for session planning.
Abstracts must be written in English and should include a title but no
information revealing the author(s).
We welcome submissions for work that is being considered by other
conferences, workshops, or journals.
Submissions should be handed in via easychair:
https://easychair.org/conferences/?conf=indiprag2023
**
We will have invited talks by:
Kirsten Abbot-Smith, University of Kent
Morten Christiansen, Cornell University
Craig Hedge, Aston University
Petra Hendriks, University of Groningen
Antje Meyer & Florian Hintz, MPI for Psycholinguistics Nijmegen
**
IndiPRAG is being organised by: Vera Demberg, Jia Loy, Alexandra Mayn,
Dongqi Pu, Margarita Ryzhova, Merel Scholman, Sebastian Schuster
You can contact us at: indiprag(a)lst.uni-saarland.de
Call for Papers: The 1st International Workshop on Implicit Author
Characterization from Texts for Search and Retrieval (IACT’23)
The workshop will be held in conjunction with the 46th International ACM
SIGIR Conference on Research and Development in Information Retrieval
Workshop website: https://en.sce.ac.il/news/iact23
July 27, 2023. Taipei, Taiwan.
Paper submission deadline extended to May 9, 2023, AoE
Submission link: https://easychair.org/conferences/?conf=iact23
To bring the research community's attention to the limitations of current
models in recognizing and characterizing AI vs. human authors, we organize
the first edition of IACT workshops under the umbrella of the SIGIR
conference. Research works submitted to the workshop should foster
scientific advances in all aspects of author characterization.
Organizing Committee:
- Marina Litvak - marinal(a)ac.sce.ac.il; Shamoon College of Engineering
Beer Sheva; Israel
- Irina Rabaev - irinar(a)ac.sce.ac.il; Shamoon College of Engineering
Beer Sheva; Israel
- Alípio Mário Jorge - amjorge(a)fc.up.pt; University of Porto; Porto,
Portugal
- Ricardo Campos - ricardo.campos(a)ipt.pt; Polytechnic Institute of Tomar
INESC TEC, Portugal; Porto, Portugal
- Adam Jatowt - adam.jatowt(a)uibk.ac.at; University of Innsbruck;
Innsbruck, Austria
Invited Speakers:
- Prof. Mark Last - Ben-Gurion University of the Negev, Israel
- Prof. Dr. Valia Kordoni - Humboldt-Universität Berlin, Germany
IACT’23 proceedings will be published at CEUR workshop proceedings (indexed
in Scopus and DBLP) as long as they do not conflict with previous
publication rights.
Contact:
- Dr. Marina Litvak: litvak.marina(a)gmail.com
- Dr. Irina Rabaev: irinar(a)ac.sce.ac.il
--
Best regards,
Marina Litvak