Název: Porozumění mezijazykovým vlastnostem ve velkých vícejazyčných jazykových modelech
Překlad názvu: Understanding cross-lingual abilities in large multilingual language models
Autoři: Del Valle Girón, José Jacobo ; Libovický, Jindřich (vedoucí práce) ; Limisiewicz, Tomasz (oponent)
Typ dokumentu: Diplomové práce
Rok: 2023
Jazyk: eng
Abstrakt: Cross-lingual abilities have been evident in large multilingual language models over the past few years. However, understanding why and under what circumstances they work is not entirely clear. In this work, we work towards a better understanding of these aspects in a specific subset of multilingual models, namely modular multilingual models with cross-lingual transfer learning abilities. We try to quantify claims in Pfeiffer et al. [2022] regarding their proposed model, X-MOD, as it was tested in a very specific setting which may not align with common low-resource settings. Specifically, we evaluate how the following factors may affect downstream performance: the amount of available pre- training data; hyperparameters such as number of training steps, checkpoint selection criteria, available overlapping lexicon. With the help of our findings, we also aim to provide guidelines on how to best use X-MOD, especially from a low-resource perspective. 1
Klíčová slova: transfer learning|cross-lingual learning|low-resource|language models; transfer learning|cross-lingual learning|low-resource|language models

Instituce: Fakulty UK (VŠKP) (web)
Informace o dostupnosti dokumentu: Dostupné v digitálním repozitáři UK.
Původní záznam: http://hdl.handle.net/20.500.11956/184175

Trvalý odkaz NUŠL: http://www.nusl.cz/ntk/nusl-534514


Záznam je zařazen do těchto sbírek:
Školství > Veřejné vysoké školy > Univerzita Karlova > Fakulty UK (VŠKP)
Vysokoškolské kvalifikační práce > Diplomové práce