National Repository of Grey Literature 125 records found  1 - 10nextend  jump to record: Search took 0.00 seconds. 
Practical neural dialogue management using pretrained language models
Šafář, Jaroslav ; Dušek, Ondřej (advisor) ; Bojar, Ondřej (referee)
Task-oriented dialogue systems pose a significant challenge due to their complexity and the need to handle components such as language understanding, state tracking, action selection, and language generation. In this work, we explore the improvements in dialogue management using pretrained language models. We propose three models that incorporate pretrained language models, aiming to provide a practical approach to designing dialogue systems capable of effectively addressing the language understanding, state tracking, and action selection tasks. Our dialogue state tracking model achieves a joint goal accuracy of 74%. We also identify limitations in handling complex or multi- step user requests in the action selection task. This research underscores the potential of pretrained language models in dialogue management while highlighting areas for further improvement. 1
Methods of User-Assisted Summarization of Meetings
Kmječ, František ; Bojar, Ondřej (advisor) ; Kasner, Zdeněk (referee)
Automated minuting, or meeting summarization, is the task of accurately capturing the contents of a meeting in a short text or in bullet points. Recently, a lot of progress has happened in this area, largely due to the rise of the large language models. However, most fully automated approaches have severe limitations; either their outputs are vague or they are prone to hallucinations. We explore the possibility of user-assisted minuting to provide factual accuracy as well as coverage. We introduce a novel open-source tool, Minuteman, integrated with JitSi Meet to explore the methods by which users can interact with summarization models. We then analyze data gathered from multiple experiments with users and show how similar means of interaction can be of use in increasing summary quality. 1
German Compounds in Transformer Models
Neumannová, Kristýna ; Bojar, Ondřej (advisor) ; Zeman, Daniel (referee)
German is known for its highly productive word formation processes, particularly in the area of compounding and derivation. In this thesis, we focus on German nominal compounds and their representation in machine translation (MT) outputs. Despite their importance in German text, commonly used metrics for MT evaluation, such as BLEU, do not adequately capture the usage of compounds. The aim of this thesis was to investigate the generation of German compounds in Transformer models and to explore the conditions that lead to their production. Our analysis revealed that MT systems tend to produce fewer compounds than humans. However, we found that due to the highly productive nature of German compounds, it is not feasible to identify them based on a fixed list. Therefore, we manually identified novel compounds, and even then, human translations still contained more compounds than MT systems. We trained our own Transformer model for English-German translation and conducted experiments to examine various factors that influence the production of compounds, in- cluding word segmentation and the frequency of compounds in the training data. Addi- tionally, we explored the use of forced decoding and the impact of providing the model with the first words of a sentence during translation. Our findings highlight the...
Methods of Input Segmentation for Simultaneous Speech Translation
Ryšlink, Václav ; Bojar, Ondřej (advisor) ; Polák, Peter (referee)
Segmentation methods are an essential part of the simultaneous machine translation process because, in the ideal case, they split the input into chunks whose translation is independent of any forthcoming context. Furthermore, the optimal splitting should also ensure that the segments with the previous characterization have minimal lengths. However, there is still no agreement about the rules that should produce such an optimal splitting. Therefore, we started with the annotation of the ESIC dataset by simulating a perfect human interpreter with an infinite amount of time and resources. Then we proposed multiple segmentation methods that we compared to each other in terms of segments' lengths, counts, and statistics of the most frequently split types of words. Apart from the segmentation methods, we also implemented and analyzed two variants of neural machine translation models - one trained solely on complete sentences and the other finetuned with partial translations. Finally, we evaluated the translation quality and delay of segments produced by splitting methods with the SLTev evaluation toolkit and discussed the effect of both machine translation models on the results.
Adapting Pretrained Models for Machine Translation
Kurniawan, Aditya ; Bojar, Ondřej (advisor) ; Variš, Dušan (referee)
Pre-trained language models received extensive attention in recent years. However, it is still challenging to incorporate a pre-trained model such as BERT into natural language generation tasks. This work investigates a recent method called adapters as an alternative to fine-tuning the whole model in machine translation. Adapters are a promising approach that allows fine-tuning only a tiny fraction of a pre-trained network. We show that with proper initialization, adapters can help achieve better performance than training models from scratch while training substantially fewer weights than the original model. We further show that even with randomly set weights used as the base models for fine-tuning, we can achieve similar performance to one of the baseline models, bypassing the need to train hundreds of millions of weights in the pre-training phase. Furthermore, we study the effectiveness of adapters in the Transformer model for machine translation. We put adapters either in the encoder or the decoder only, and we also attempt to down-scale the pre-trained model size to decrease GPU memory demands. We found that incorporating adapters in the encoder alone matches the setup's performance when we include the adapters on both the encoder and decoder. Finally, our down-scaling study found that using only half...
Rich Features in Phrase-Based Machine Translation
Kos, Kamil ; Bojar, Ondřej (advisor) ; Žabokrtský, Zdeněk (referee)
In this thesis we investigate several methods how to improve the quality of statistical machine translation (MT) by using linguistically rich information. First, we describe SemPOS, a metric that uses shallow semantic representation of sentences to evaluate the translation quality. We show that even though this metric has high correlation with human assessment of translation quality it is not directly suitable for system parameter optimization. Second, we extend the log-linear model used in statistical MT by additional source-context model that helps to better distinguish among possible translation options and select the most promising translation for a given context.
Classifier for semantic patterns of English verbs
Kríž, Vincent ; Holub, Martin (advisor) ; Bojar, Ondřej (referee)
The goal of the diploma thesis is to design, implement and evaluate classifiers for automatic classification of semantic patterns of English verbs according to a pattern lexicon that draws on the Corpus Pattern Analysis. We use a pilot collection of 30 sample English verbs as training and test data sets. We employ standard methods of machine learning. In our experiments we use decision trees, k-nearest neighbourghs (kNN), support vector machines (SVM) and Adaboost algorithms. Among other things we concentrate on feature design and selection. We experiment with both morpho-syntactic and semantic features. Our results show that the morpho-syntactic features are the most important for statistically-driven semantic disambiguation. Nevertheless, for some verbs the use of semantic features plays an important role.
Language Modelling for German
Tlustý, Marek ; Bojar, Ondřej (advisor) ; Hana, Jiří (referee)
The thesis deals with language modelling for German. The main concerns are the specifics of German language that are troublesome for standard n-gram models. First the statistical methods of language modelling are described and language phenomena of German are explained. Following that suggests own variants of n-gram language models with an aim to improve these problems. The models themselves are trained using the standard n-gram methods as well as using the method of maximum entropy with n-gram features. Both possibilities are compared using corelation metrics of hand-evaluated fluency of sentences and automatic evaluation - the perplexity. Also, the computation requirements are compared. Next, the thesis presents a set of own features that represent the count of grammatical errors of chosen phenomena. Success rate is verified on ability to predict the hand-evaluated fluency. Models of maximum entropy and own models that classify only using the medians of phenomena values computed from training data are used.

National Repository of Grey Literature : 125 records found   1 - 10nextend  jump to record:
Interested in being notified about new results for this query?
Subscribe to the RSS feed.