|
|
Visualizing Neural Network Used as a Language Model
Ryšánek, Jakub ; Černocký, Jan (oponent) ; Beneš, Karel (vedoucí práce)
Long short-term memory (LSTM) network is a type of neural network designed to analyze sequence data. The advantage of LSTM over the simple recurrent neural network is the ability to store long-term dependencies, which allows them to reach higher accuracy when performing tasks such as speech recognition or language modeling. However, due to their complexity, the internal processes that lead to these results are still not fully understood. To explore their inner workings, I created three visualization methods. These methods focus on the pattern of the behavior of the single unit present in the model or the behavior of the whole model when processing words with similar syntactic or semantic meanings.
|
|
|
Fast Discriminative Neural Networks for Text Correction
Chupáč, Sebastián ; Beneš, Karel (oponent) ; Kohút, Jan (vedoucí práce)
The goal of this work is to propose and implement a fast discriminating neural network with only one forward pass, to detect and correct mistakes in text data. Multiple architectures were implemented for detection and correction separately. These models make use of convolution layers, LSTM layers and CTC loss function. Models were trained and evaluated on datasets made from three different text corpora. Experiments and evaluation present the ability of these models to detect and correct mistakes on character level with only one, fast forward pass.
|
|
|
Bilingual Dictionary Based Neural Machine Translation
Tikhonov, Maksim ; Beneš, Karel (oponent) ; Kesiraju, Santosh (vedoucí práce)
The development in the recent few years in the field of machine translation showed us that modern neural machine translation systems are capable of providing results of outstanding quality. However, in order to obtain such a system, one requires an abundant amount of parallel training data, which is not available for most languages. One of the ways to improve the quality of machine translation of low-resource languages is data augmentation. This work investigates the task of Bilingual dictionary-based neural machine translation (BDBNMT), the basis of which is the use of the augmentation technique that allows the generation of noised data based on bilingual dictionaries. My aim was to explore the capabilities of BDBNMT systems on different language pairs and under different initial conditions and then compare the obtained results with those of traditional neural machine translation systems.
|
|
|
Advanced Visualization of Neural Network Training
Kuchta, Samuel ; Kesiraju, Santosh (oponent) ; Beneš, Karel (vedoucí práce)
This work aims to propose visualization methods and analyze with them the phenomena arising during the training of neural networks, based on which new knowledge regarding deep learning could be discovered. In this work, a program was created that tests the impact of training using different techniques and visualizes the training results using different methods. This work presents two methods of visualization of the training process. The first method displays the area around the path of the trained model by averaging the path's points weighted by their distance from the displayed point. The second method is to display step sizes during learning. The result of the work is represented by graphs and a discussion of the phenomena captured by the visualizations.
|
|
|
Transformer Neural Networks for Handwritten Text Recognition
Vešelíny, Peter ; Beneš, Karel (oponent) ; Kohút, Jan (vedoucí práce)
This Master's thesis aims to design a system using the transformer neural network and perform experiments with this proposed model in the task of handwriting text recognition. In this thesis, a multilingual dataset with predominate Czech texts is used. The experiments examine the influence of basic hyperparameters, such as network size, convolutional encoder type, and the use of different text tokenizers. In this work, I also use text corpora of the Czech language which is used to train the network decoder. Furthermore, I experiment with the usage of additional textual information during the decoding process. This information comes from the previous line of the transcribed image. The transformer achieves a character recognition error rate of 3.41 % on the test data set which is 0.16 % worse performance than the recurrent neural network achieves. To compare this model with other transformer-based models from available articles, the network was trained on the IAM dataset, where it achieved an error of 2.48 % and therefore outperformed other models in handwriting text recognition task.
|
|
|
Uspořádání fragmentů textu s pomocí jazykového modelu
Holubec, Michael ; Kocour, Martin (oponent) ; Beneš, Karel (vedoucí práce)
Cílem této práce je sestrojit a experimentálně ověřit účinnost jazykového modelu při identifikaci posloupnosti čtení (Reading Order). K tomuto účelu byl sestrojen jazykový model využívající rekurentní neuronovou síť LSTM. Práce dále navrhuje a implementuje celkem tři metody, jazykovou analýzu, prostorovou analýzu a kombinovanou analýzu, pomocí kterých je posloupnost čtení identifikována. Jazyková a kombinovaná analýza ke své činnosti přímo používají vytvořený jazykový model. Úspěšnost identifikace posloupnosti prostřednictvím všech tří metod byla změřena na třech datasetech obsahující novinové články s různým rozložením. Jazyková analýza dosahuje úspěšnosti 57,6 %, prostorová analýza dosahuje 91,6 %. Nejlepších výsledků dosahuje kombinovaná analýza, která vykazuje úspěšnost 92,9 %. Práce ukazuje, že jazykový model lze pro identifikaci posloupnosti čtení použít, avšak výsledky experimentů naznačují, že je vhodné zpracování odhadu posloupnosti doplnit o další informace, jako je to například v kombinované analýze, která pracuje jak s jazykovým modelem, tak s prostorovými informacemi.
|
|
|
Detekce stresu v řeči
Šoltés, Samuel ; Beneš, Karel (oponent) ; Grézl, František (vedoucí práce)
Stres vplýva na človeka viacerými spôsobmi a môže viesť k poklesnutiu kvality výkonu či kritickým chybám. Detekcia stresu v reči sleduje ako sa prejavujú vplyvy stresu na reči. Cieľom tejto bakalárskej práce je priblížiť vplyvy stresu, zvoliť vhodné parametre rečového signálu, na ktorých by sa vplyvy prejavili, implementovať výpočet týchto parametrov a porovnať ich úspešnosť detekcie stresu. V práci je opísaný stres a vplyv stresorov na človeka; glotálny pulz, spektrum, základná frekvencia a formanty ako parametre rečového signálu vhodné na analýzu a detekciu stresu; návrh a implementácia výpočtu týchto parametrov a dosiahnuté výsledky výpočtov parametrov na dvoch rôznych databázach.
|
|
|
Neural Network Training Progress Visualization
Němcová, Silvie ; Baskar, Murali Karthick (oponent) ; Beneš, Karel (vedoucí práce)
This work studies a neural network model during its training. The aim of this thesis is to visualize the training of the model and to examine the training. To achieve this goal I choose to implement a set of tools in Python language. The implementation successfully reproduces the linear path experiment, the identification of robust and ambient layers and the visualization of the loss surface. In addition the quadratic path experiment is presented in this thesis as novel method for analyzing the neural network training progress visualization.
|
|
|
Gaussian Processes Based Hyper-Optimization of Neural Networks
Coufal, Martin ; Landini, Federico Nicolás (oponent) ; Beneš, Karel (vedoucí práce)
The goal of this thesis is to create a lightweight toolkit for artificial neural network hyper-parameter optimisation. The optimisation toolkit has to be able to optimise multiple, possibly correlated hyper-parameters. I solved this problem by creating an optimiser that uses Gaussian processes to predict the influence of the hyper-parameters on the resulting neural network accuracy. Based on the experiments on multiple benchmark functions, the toolkit is able to provide better results than random search optimisation and thus reduce the number of necessary optimisation steps. The random search optimisation provided better results only in the first few optimisation steps before Gaussian process optimisation creates sufficient model of the problem. However the experiments on MNIST dataset show that random optimisation achieves almost always better results than used GP optimiser. These differences between the experiments results are probably caused by insufficient complexity of the benchmarks or by selected parameters of the implemented optimiser.
|
|
|
Chatbot Capable of Information Search
Ďurista, Michal ; Beneš, Karel (oponent) ; Černocký, Jan (vedoucí práce)
''Chatbot'' is a very popular term in today's artificial intelligence era. Chatbots can be seen in business solutions more a more nowadays. The main goal of this thesis is to create an algorithm that is capable of information retrieval and implement it into a chatbot. The information resides on a real customer's web pages. The thesis also provides an overview of current chatbot situation along with the Microsoft technologies used for the development. The technological background of these technologies, mostly natural language processing techniques, is covered too. The thesis also describes the implementation of the algorithm and the chatbot itself as well as the real industrial environment testing process.
|
host ::
RSS.