Differences

This shows you the differences between two versions of the page.

--- wg3:wg3_meeting_2023-03-17 [2023/03/21 16:51] – gulsen.eryigit
+++ wg3:wg3_meeting_2023-03-17 [2023/03/21 16:53] – gulsen.eryigit
@@ Line 63: / Line 63: @@
+==== Session 2 ====
+.30–13.35	Recap of Session 1 (for new participants)
+.35–14.20	Initial discussion on evaluation campaigns
+		== Background on goals and previous shared tasks (slides) ==
+		Brainstorming – define a novel shared task/evaluation campaign:
+  * How is the task defined?
+  * What are the evaluation metrics?
+  * What kind of data is needed?
+  * Which languages should be included?
+		Ideas:
+  * Task = provide resources for shared tasks (eval metrics, test sets)
+  * Instead of a shared task, build a dynamic leaderboard for LMs
+  * Compare “traditional methods” to LMs on UD and MWE data
+  * UD parsing with only surprise test languages, minimize training data
+  * NLP tasks on top of UD data using linguistically defined embeddings
+  * Distinguish similar languages or dialects (for example, using MWEs)
+  * Objective: make every language appear at the center of the world
+  * Collect idiom data using LLMs, evaluate on gold data
+.20–14.30	Next steps
+		Next WG3 meeting in Istanbul, September 8, 2023
+		We will focus on documentation of tools
+		Two tasks in preparation for the meeting:
+A taxonomy of multi- and cross-lingual language technology
+An infrastructure for multi- and cross-lingual language technology
+		Volunteers for these tasks are encouraged to contact WG leaders by email
+.30–14.45	Presentation of the European Language Equality project (slides)