wg3:wg3_meeting_2023-03-17
Differences
This shows you the differences between two versions of the page.
Both sides previous revisionPrevious revisionNext revision | Previous revisionNext revisionBoth sides next revision | ||
wg3:wg3_meeting_2023-03-17 [2023/03/21 16:51] – gulsen.eryigit | wg3:wg3_meeting_2023-03-17 [2023/03/21 16:56] – gulsen.eryigit | ||
---|---|---|---|
Line 63: | Line 63: | ||
+ | ==== Session 2 ==== | ||
+ | 13.30–13.35 Recap of Session 1 (for new participants) | ||
+ | |||
+ | 13.35–14.20 Initial discussion on evaluation campaigns | ||
+ | |||
+ | == Background on goals and previous shared tasks (slides) == | ||
+ | |||
+ | == Brainstorming – define a novel shared task/ | ||
+ | |||
+ | * How is the task defined? | ||
+ | * What are the evaluation metrics? | ||
+ | * What kind of data is needed? | ||
+ | * Which languages should be included? | ||
+ | |||
+ | == Ideas: == | ||
+ | |||
+ | * Task = provide resources for shared tasks (eval metrics, test sets) | ||
+ | * Instead of a shared task, build a dynamic leaderboard for LMs | ||
+ | * Compare “traditional methods” to LMs on UD and MWE data | ||
+ | * UD parsing with only surprise test languages, minimize training data | ||
+ | * NLP tasks on top of UD data using linguistically defined embeddings | ||
+ | * Distinguish similar languages or dialects (for example, using MWEs) | ||
+ | * Objective: make every language appear at the center of the world | ||
+ | * Collect idiom data using LLMs, evaluate on gold data | ||
+ | |||
+ | |||
+ | 14.20–14.30 Next steps | ||
+ | |||
+ | Next WG3 meeting in Istanbul, September 8, 2023 | ||
+ | We will focus on documentation of tools | ||
+ | Two tasks in preparation for the meeting: | ||
+ | - A taxonomy of multi- and cross-lingual language technology | ||
+ | - An infrastructure for multi- and cross-lingual language technology | ||
+ | Volunteers for these tasks are encouraged to contact WG leaders by email | ||
+ | |||
+ | |||
+ | 14.30–14.45 Presentation of the European Language Equality project (slides) | ||
wg3/wg3_meeting_2023-03-17.txt · Last modified: 2023/09/20 10:28 by joakim.nivre