• Event title: PARSEME/UniDive annotation campaign (task 1.2)
• Dates: September 2023 – September 2025
• Co-leaders:
  • Voula Giouli, Aristotle University of Thessaloniki, Greece
  • Stella Markantonatou, Language and Speech Processing/ATHENA RC, Athens, Greece
  • Carlos Ramisch, Aix-Marseille Université, France
  • Agata Savary, Université Paris-Saclay, France
  • Sara Stymne, Uppsala University, Sweden

The UniDive COST action (task 1.2) and the PARSEME community are carrying on a multilingual corpus annotation campaign dedicated to multiword expressions (MWEs).

Three past PARSEME annotation campaigns were dedicated exclusively to verbal MWEs (VMWEs) and resulted in 4 editions of the PARSEME corpus, which jointly covers 26 languages. Three PARSEME shared tasks on automatic identification of VMWEs have been organized on the basis of this corpus and set the state of the art in the task.

The current annotation campaign covers MWEs of all syntactic types (including nominal, adjectival, adverbial and functional MWEs). It follows the spirit of universality. Namely, the annotation guidelines are unified across all participating languages, whenever possible, still leaving room for truly language-specific phenomena. This approach is expected to promote meaningful cross-language comparisons. The resulting corpus will be used in a PARSEME/UniDive shared task on identifying and understanding MWEs, submitted as a proposal for SemEval 2026.

Each language should be annotated by a team on native annotators (except when this is not possible, e.g. in the case of extinct languages like Ancient Greek or Egyptian). A language team should consist of at least 2 annotators (including the Language Leader), for the sake of inter-annotator agreement estimation. It is possible to start annotating alone and recruit more annotators at a later stage (May 2025 at latest). See the language teams from past and present annotation campaigns.

Each language team should have at least one Language Leader. See the call for Language Leaders.

For the languages already present in the PARSEME corpus, the agenda is to:

• Re-annotate |the existing corpus with MWEs other than verbal. Annotating only part of the existing corpus is an option. In this case we recommend a minimum of 2000 annotated MWEs (so that each selected text is exhaustively annotated for all syntactic types of MWEs). A lower number of annotations can do but the system results are expected not to be representative.
• Add some new texts annotated from scratch (to counterbalance language model contamination from previously published data)

For new languages, corpora will be annotated for all syntactic types at once.

Conversions from other MWE annotation schemes are fine, if curated so as to fit the PARSEME guidelines.

• [language leaders: 27 February] Expression of interest from Language Leaders
• [task leaders: late-February] Creating FLAT accounts
• [language leaders: mid-March] Reading guidelines, reading the Language Leader guide, filling in MWE examples, recruiting annotators, selecting corpora
• [all: March] Pilot annotation
• [shared task leaders: 31 March] SEMEVAL shared task proposal
• [language teams: April-May] Annotation (including a double-annotated sample for inter-annotator agreement estimation)
• [SEMEVAL: 19 May] Notification about the selected shared task
• [language leaders: June] Consistency checks and inter-annotator agreement estimation
• [shared task leaders: 15 July] Sample data ready
• [task leaders: July-August] Consolidating and splitting the corpora
• [WG3 shared task leaders: 1 September] Training data for SEMEVAL

• PARSEME/UniDive annotation campaign master document
• PARSEME corpus wiki
• PARSEME annotation guidelines 2.0 and what’s new in version 2.0
• Language Leader's guide
• FLAT annotation platform and FLAT User's Guide
• Minutes from task 1.2 co-leaders’ meetings
• Minutes from Language Leaders' meetings