Differences

This shows you the differences between two versions of the page.

--- wg3:wg3_meeting_2023-03-17 [2023/03/21 16:51] – gulsen.eryigit
+++ wg3:wg3_meeting_2023-03-17 [2023/09/20 10:28] (current) – [WG3 1st Meeting Minutes -- 2023-03-17] joakim.nivre
@@ Line 1: / Line 1: @@
-===== WG3 1st Meeting Minutes - 2023-03-17 =====
+===== WG3 1st Meeting Minutes – 2023-03-17 =====
 ==== Session 1 ====
-.45–11.00 	Introduction to WG3 (slides)
+.45–11.00 	Introduction to WG3 ({{ :wg3:WG3_1stMeeting_Slides.pdf |slides}})
 .00–11.30	Brainstorming on ideas and expectations
@@ Line 63: / Line 63: @@
+==== Session 2 ====
+.30–13.35	Recap of Session 1 (for new participants)
+.35–14.20	Initial discussion on evaluation campaigns
+== Background on goals and previous shared tasks ==
+({{ :wg3:WG3_1stMeeting_Slides.pdf |slides}})
+== Brainstorming – define a novel shared task/evaluation campaign: ==
+  * How is the task defined?
+  * What are the evaluation metrics?
+  * What kind of data is needed?
+  * Which languages should be included?
+== Ideas: ==
+  * Task = provide resources for shared tasks (eval metrics, test sets)
+  * Instead of a shared task, build a dynamic leaderboard for LMs
+  * Compare “traditional methods” to LMs on UD and MWE data
+  * UD parsing with only surprise test languages, minimize training data
+  * NLP tasks on top of UD data using linguistically defined embeddings
+  * Distinguish similar languages or dialects (for example, using MWEs)
+  * Objective: make every language appear at the center of the world
+  * Collect idiom data using LLMs, evaluate on gold data
+.20–14.30	Next steps
+  * Next WG3 meeting in Istanbul, September 8, 2023
+  * We will focus on documentation of tools
+  * Two tasks in preparation for the meeting:
+  - A taxonomy of multi- and cross-lingual language technology
+  - An infrastructure for multi- and cross-lingual language technology
+Volunteers for these tasks are encouraged to contact WG leaders by email
+.30–14.45	Presentation of the European Language Equality project ({{ :wg3:WG3_1stMeeting_Slides_ELE.pdf |slides}})