Differences

This shows you the differences between two versions of the page.

--- wg3:wg3_meeting_2023-09-08 [2023/09/20 16:46] – [Documentation of Multilingual Language Technology Tools] joakim.nivre
+++ wg3:wg3_meeting_2023-09-08 [2023/09/26 10:22] (current) – [Multilingual Evaluation Campaigns] joakim.nivre
@@ Line 1: / Line 1: @@
 ===== WG3 2nd Meeting Minutes – 2023-09-08 =====
-The second WG meeting was mainly devoted to the two tasks/deliverables outlined in the MoU, documentation of multilingual language tools and multilingual evaluation campaigns. In addition, we discussed ways of working in the WG.
+The second WG meeting was mainly devoted to the two tasks/deliverables outlined in the MoU, documentation of multilingual language tools and organization of multilingual evaluation campaigns. In addition, we discussed ways of working in the WG.
 ==== Documentation of Multilingual Language Technology Tools ====
@@ Line 25: / Line 25: @@
          * Do we include only strictly multilingual tools, or do we want to document monolingual tools for many languages?
       * What languages do we include?
-         * ELG focuses on languages in Europe, but could easily be extended.
+         * ELG focuses on languages in Europe, but can easily be extended.
          * In practice, we may be limited by the expertise available within the action.
          * How do we deal with dialects and language variants?
@@ Line 32: / Line 32: @@
       * Should we try to integrate typology resources? How?
+The WG leaders will call a meeting with the task leaders to discuss next steps. Anyone else who would like to be involved in this task is welcome to contact the task leaders. The goal is to select a platform and define the scope of the task before the first general meeting in 2024.
+==== Multilingual Evaluation Campaigns ====
+The afternoon session was organized by the WG3 leaders and featured short invited presentations from WG members who had submitted ideas for evaluation campaigns:
+   * D. Zeman: UD Shared Task Inside Out ({{ :wg3:EvalCampaign_Dan.pdf|abstract}})
+   * A. Wróblewska: NLPre Benchmarking Platform ({{ :wg3:EvalCampaign_Alina.pdf|abstract}})
+   * O. Goldman, L. Weissweiler and R. Tsarfaty: Morpho-Syntactic Analyso-Parsing ({{ :wg3:EvalCampaign_Omer.pdf|abstract}}, {{ :wg3:EvalCampaign_Omer_Slides.pdf|slides}})
+   * T. Samardzic: Subword Pooling ({{ :wg3:EvalCampaign_Tanja.pdf|slides}})
+The proposals were discussed in breakout groups, followed by a general plenary discussion where several other ideas for evaluation campaigns were proposed as well. The WG leaders will call a meeting with the proposers to discuss next steps. Anyone else who would like to be involved in this task is welcome to contact the WG leaders. The goal is to organize at least one evaluation campaign during 2024 and a different one during 2025.
+==== Ways of Working ====
+In a short session after lunch, we discussed how to organize the work going forward. Two decisions were made:
+   * Following other WGs, we will organize regular online meetings for exchange of information, where task groups can report on their progress.
+   * The importance of information flow between different WGs was emphasised, and the WG leaders were tasked with proposing a system of WG liaisons to the other WG leaders. The role of WG liaisons is to report on work in other WGs at our meetings (and vice versa).