Název:
Empirické modely pro indické jazykové kontinuum
Překlad názvu:
Empirical Models for an Indic Language Continuum
Autoři:
Bafna, Niyati ; Žabokrtský, Zdeněk (vedoucí práce) ; Zeman, Daniel (oponent) Typ dokumentu: Diplomové práce
Rok:
2022
Jazyk:
eng
Abstrakt: Empirical Models for an Indic Language Continuum Niyati Bafna July 20, 2022 Many Indic languages and dialects of the so-called "Hindi Belt" and surrounding re- gions in the Indian subcontinent, spoken by more than 100 million people, are severely under-resourced and under-researched in NLP, individually and as a dialect continuum. We first collect monolingual data for 26 Indic languages and dialects, 16 of which were previously zero-resource, and perform exploratory character, lexical and subword cross- lingual alignment experiments for the first time on this linguistic system. We present a novel method for unsupervised cognate/borrowing identification from monolingual cor- pora designed for low and extremely low resource scenarios, based on combining noisy se- mantic signals from joint bilingual spaces with orthographic cues modelling sound change; to the best of our knowledge, this is the first work to do so, especially in a (truly) low- resource setup. We create bilingual evaluation lexicons against Hindi for 20 of the lan- guages, and show that our method outperforms both traditional orthography baselines as well as EM-style learnt edit distance matrices, showing that even noisy bilingual em- beddings can act as good guides for this task. We release our crawled data in a new collection called...
Klíčová slova:
vícejazyčná data|jazykové kontinuum|zpracování přirozeného jazyka; multilingual data|language continuum|Natural Language Processing