The fourth WG meeting (online) was mainly devoted to reports from two subgroups, followed by discussion.
Multilingual Evaluation Campaigns
Omer Goldman made a progress report on the morpho-syntactic analyso-parsing task and presented a data definition document that will be shared with WG members shortly. [ Worksheet ]
Discussion points:
How do the representations relate to current UD guidelines? They contradict UD guidelines in some respects (notably with respect to words as central units) but should remain compatible with them in most other respects.
What tools can be used to create the representations? Udapi was recommended as a good tool for manipulating UD resources.
What evaluation metrics should be used? MLAS is a metric used in the CoNLL shared tasks that includes both morphology and syntax, but it may have to be adapted.
Adriana Pagano and Ilan Kernerman presented plans for a subgroup on defining multilingual and cross-lingual tools in the context of UniDive [ Slides ].
Discussion points:
Similar issues are being discussed in WG4, where a survey of diversity measures is planned.
What tasks will be in focus? Still to be decided.
How does this task relate to the documentation task? This task focuses on defining the concepts, rather than documenting existing tools, but there is a need to coordinate the activities of the two subgroups.
List of Participants (from Zoom)
Joakim Nivre
Louis Estève
Tanja Samardzic
Vladimir Cvetkoski
Martin Benjamin
Atul Kr. Ojha
Maria Giagkou (ILSP, Athena RC)
Csilla Horváth
Radovan Garabík
Omer Goldman
Leonie Weissweiler
Reut Tsarfaty
Verena Blaschke
Doğukan Arslan
Fatih Bektaş
Tuğba Pamay Arslan
Lucía Amorós-Poveda (University of Murcia, Spain)
Inguna Skadiņa
Olha Kanishcheva
Nataliia Cheilytko (Friedrich Schiller University Jena, Germany)
Rob van der Goot
Sviatlana Karpava
Evelin Amorim
Tunga Gungor
Miryam de Lhoneux
Dan Zeman
Vivian Stamou
Adriana Pagano
Harald Hammarström
Ilan Kernerman
Teresa Lynn
Danka Jokić
Enriketa Sogutlu
Roberto A. Díaz Hernández
Nina Böbel
Ranka Stanković
Federica Gamba
Rusudan Makhachashvili
Agata Savary
Gosse Bouma
Luka Terčon
Mücahit Altıntaş
Verginica Mititelu
Serge Sharoff
Artur Kulmizev