|
|
Multi-Target Machine Translation
Ihnatchenko, Bohdan ; Bojar, Ondřej (advisor) ; Kocmi, Tom (referee)
In international and highly-multilingual environments, it often happens, that a talk, a document, or any other input, needs to be translated into a massive number of other languages. However, it is not always an option to have a distinct system for each possible language pair due to the fact that training and operating such kind of translation systems is computationally demanding. Combining multiple target languages into one translation model usually causes a de- crease in quality of output for each its translation direction. In this thesis, we experiment with combinations of target languages to see, if a specific grouping of them can lead to better results than just randomly selecting target languages. We build upon a recent research on training a multilingual Transformer model without any change to its architecture: adding a target language tag to the source sentence. We trained a large number of bilingual and multilingual Transformer models and evaluated them on multiple test sets from different domains. We found that in most of the cases grouping related target languages into one model caused a better performance compared to models with randomly selected languages. However, we also found that a domain of the test set, as well as domains of data sampled into the training set, usu- ally have a more...
|
|
|
Exploring Benefits of Transfer Learning in Neural Machine Translation
Kocmi, Tom ; Bojar, Ondřej (advisor) ; van Genabith, Josef (referee) ; Cuřin, Jan (referee)
Title: Exploring Benefits of Transfer Learning in Neural Machine Translation Author: Tom Kocmi Department: Institute of Formal and Applied Linguistics Supervisor: doc. RNDr. Ondřej Bojar, Ph.D., Institute of Formal and Applied Linguistics Keywords: transfer learning, machine translation, deep neural networks, low-resource languages Abstract: Neural machine translation is known to require large numbers of parallel train- ing sentences, which generally prevent it from excelling on low-resource lan- guage pairs. This thesis explores the use of cross-lingual transfer learning on neural networks as a way of solving the problem with the lack of resources. We propose several transfer learning approaches to reuse a model pretrained on a high-resource language pair. We pay particular attention to the simplicity of the techniques. We study two scenarios: (a) when we reuse the high-resource model without any prior modifications to its training process and (b) when we can prepare the first-stage high-resource model for transfer learning in advance. For the former scenario, we present a proof-of-concept method by reusing a model trained by other researchers. In the latter scenario, we present a method which reaches even larger improvements in translation performance. Apart from proposed techniques, we focus on an...
|
|
|
Deep contextualized word embeddings from character language models for neural sequence labeling
Lief, Eric ; Pecina, Pavel (advisor) ; Kocmi, Tom (referee)
A family of Natural Language Processing (NLP) tasks such as part-of- speech (PoS) tagging, Named Entity Recognition (NER), and Multiword Expression (MWE) identification all involve assigning labels to sequences of words in text (sequence labeling). Most modern machine learning approaches to sequence labeling utilize word embeddings, learned representations of text, in which words with similar meanings have similar representations. Quite recently, contextualized word embeddings have garnered much attention because, unlike pretrained context- insensitive embeddings such as word2vec, they are able to capture word meaning in context. In this thesis, I evaluate the performance of different embedding setups (context-sensitive, context-insensitive word, as well as task-specific word, character, lemma, and PoS) on the three abovementioned sequence labeling tasks using a deep learning model (BiLSTM) and Portuguese datasets. v
|
guest ::
