|
|
Domain Specific Data Crawling for Language Model Adaptation
Gregušová, Sabína ; Švec, Ján (referee) ; Karafiát, Martin (advisor)
The goal of this thesis is to implement a system for automatic language model adaptation for Phonexia ASR system. System expects input in the form of source that, which is analysed and appropriate terms for web search are chosen. Every web search results in a set of documents that undergo cleaning and filtering procedures. The resulting web corpora is mixed with Phonexia model and evaluated. In order to estimate the most optimal parameters, I conducted 3 sets of experiments for Hindi, Czech and Mandarin. The results of the experiments were very favourable and the implemented system managed to decrease perplexity and Word Error Rate in most cases.
|
|
|
Word prediction using language models
Koutný, Michal ; Popel, Martin (advisor) ; Novák, Michal (referee)
The thesis utilizes ngram language models to improve text entry with QWERTY keyboard by the means of word prediction. Related solutions are briedly introduced. Then follows theoretical background for the work. The analysis in the next part divides problems into four tasks: language model training, incorporating model for word prediction, GUI component and evaluation framework. The realization combines Python and C++. The used corpora come from Czech (19\,M words) and (84\,M words) English Wikipedia articles. A small corpus of Czech educative texts was used to test domain adaptation. The quality metrics are defined and various configuration are measured. The best solutions reduced keystrokes per character to 0.44, resp. 0.55 for English, resp. Czech on testing data.
|
|
|
OCR Trained with Unanotated Data
Buchal, Petr ; Dobeš, Petr (referee) ; Hradiš, Michal (advisor)
The creation of a high-quality optical character recognition system (OCR) requires a large amount of labeled data. Obtaining, or in other words creating, such a quantity of labeled data is a costly process. This thesis focuses on several methods which efficiently use unlabeled data for the training of an OCR neural network. The proposed methods fall into the category of self-training algorithms. The general approach of all proposed methods can be summarized as follows. Firstly, the seed model is trained on a limited amount of labeled data. Then, the seed model in combination with the language model is used for producing pseudo-labels for unlabeled data. Machine-labeled data are then combined with the training data used for the creation of the seed model and they are used again for the creation of the target model. The successfulness of individual methods is measured on the handwritten ICFHR 2014 Bentham dataset. Experiments were conducted on two datasets which represented different degrees of labeled data availability. The best model trained on the smaller dataset achieved 3.70 CER [%], which is a relative improvement of 42 % in comparison with the seed model, and the best model trained on the bigger dataset achieved 1.90 CER [%], which is a relative improvement of 26 % in comparison with the seed model. This thesis shows that the proposed methods can be efficiently used to improve the OCR error rate by means of unlabeled data.
|
|
|
Neural Network for Autocomplete in the Browser
Kubík, Ján Jakub ; Zemčík, Pavel (referee) ; Kolář, Martin (advisor)
The goal of this thesis is to create and train a neural network and use it in a web browser for English text sequence prediction during writing of text by the user. The intention is to simplify the writing of frequent phrases. The problem is solved by employing a recurrent neural network that is able to predict output text based on the text input. Trained neural network is then used in a Google Chrome extension. By normalized ouput of the neural network, text choosing by sampling decoding algorithm and connecting, the extension is able to generate English word sequences, which are shown to the user as suggested text. The neural network is optimized by selecting the right loss function, and a suitable number of recurrent layers, neurons in the layers, and training epochs. The thesis contributes to enhancing the everyday user experience of writing on the Internet by using a neural network for English word sequence autocomplete in the browser.
|
|
|
Neural Language Model Acceleration
Labaš, Dominik ; Černocký, Jan (referee) ; Beneš, Karel (advisor)
This work adresses the topic of neural language model acceleration. The aim of this work is to optimize model of a feed-forward neural network. In accelerating of the neural network we used a change of activation function, pre-calculation of matrices for calculationg the hidden layer, implementation of the model's history cache and unnormalized model. The best-performing model was accelerated by 75.3\%.
|
|
|
Neural Language Models with Morphology for Machine Translation
Musil, Tomáš ; Bojar, Ondřej (advisor) ; Straková, Jana (referee)
Language models play an important role in many natural language processing tasks. In this thesis, we focus on language models built on artificial neural net- works. We examine the possibilities of using morphological annotations in these models. We propose a neural network architecture for a language model that explicitly makes use of morphological annotation of the input sentence: instead of word forms it processes lemmata and morphological tags. Both the baseline and the proposed method are evaluated on their own by perplexity, and also in the context of machine translation by the means of automatic translation quality evaluation. While in isolation the proposed model significantly outperforms the baseline, there is no apparent gain in machine translation. 1
|
|
|
Dynamic Decoder for Speech Recognition
Veselý, Michal ; Glembek, Ondřej (referee) ; Schwarz, Petr (advisor)
The result of this work is a fully working and significantly optimized implementation of a dynamic decoder. This decoder is based on dynamic recognition network generation and decoding by a modified version of the Token Passing algorithm. The implemented solution provides very similar results to the original static decoder from BSCORE (API of Phonexia company). Compared to BSCORE this implementation offers significant reduction of memory usage. This makes use of more complex language models possible. It also facilitates integration the speech recognition to some mobile devices or dynamic adding of new words to the system.
|
|
|
Deciphering tool for puzzlehunt games
Hlásek, Filip ; Mareček, David (advisor) ; Rosa, Rudolf (referee)
The thesis focuses on substitution ciphers used at puzzlehunt games. First, we collect samples of the language which is used in the texts, that we are interested in. Furthermore, we propose a language model specially designed for working with sparse data. After that, we will explore the ways of searching for probable solutions and we will present a straightforward algorithm. Then we will improve it and make it more efficient. A significant part of the project is a console application, which serves as a deciphering tool. It is able to solve more than 15 % out of the ciphers, on which it has been tested. The result can be improved if the user inputs his current geography location. In this case, the program will search just for nearby places. That will allow the deciphering tool to explore more options and achieve a better precision. Powered by TCPDF (www.tcpdf.org)
|
|
|
Word prediction using language models
Koutný, Michal ; Popel, Martin (advisor) ; Novák, Michal (referee)
The thesis utilizes ngram language models to improve text entry with QWERTY keyboard by the means of word prediction. Related solutions are briedly introduced. Then follows theoretical background for the work. The analysis in the next part divides problems into four tasks: language model training, incorporating model for word prediction, GUI component and evaluation framework. The realization combines Python and C++. The used corpora come from Czech (19\,M words) and (84\,M words) English Wikipedia articles. A small corpus of Czech educative texts was used to test domain adaptation. The quality metrics are defined and various configuration are measured. The best solutions reduced keystrokes per character to 0.44, resp. 0.55 for English, resp. Czech on testing data.
|
|
|
STATISTICAL LANGUAGE MODELS BASED ON NEURAL NETWORKS
Mikolov, Tomáš ; Zweig, Geoffrey (referee) ; Hajič,, Jan (referee) ; Černocký, Jan (advisor)
Statistické jazykové modely jsou důležitou součástí mnoha úspěšných aplikací, mezi něž patří například automatické rozpoznávání řeči a strojový překlad (příkladem je známá aplikace Google Translate). Tradiční techniky pro odhad těchto modelů jsou založeny na tzv. N-gramech. Navzdory známým nedostatkům těchto technik a obrovskému úsilí výzkumných skupin napříč mnoha oblastmi (rozpoznávání řeči, automatický překlad, neuroscience, umělá inteligence, zpracování přirozeného jazyka, komprese dat, psychologie atd.), N-gramy v podstatě zůstaly nejúspěšnější technikou. Cílem této práce je prezentace několika architektur jazykových modelůzaložených na neuronových sítích. Ačkoliv jsou tyto modely výpočetně náročnější než N-gramové modely, s technikami vyvinutými v této práci je možné jejich efektivní použití v reálných aplikacích. Dosažené snížení počtu chyb při rozpoznávání řeči oproti nejlepším N-gramovým modelům dosahuje 20%. Model založený na rekurentní neurovové síti dosahuje nejlepších publikovaných výsledků na velmi známé datové sadě (Penn Treebank).
|
guest ::
