|
|
Recurrent Neural Network for Text Classification
Myška, Vojtěch ; Kolařík, Martin (referee) ; Povoda, Lukáš (advisor)
Thesis deals with the proposal of the neural networks for classification of positive and negative texts. Development took place in the Python programming language. Design of deep neural network models was performed using the Keras high-level API and the TensorFlow numerical computation library. The computations were performed using GPU with support of the CUDA architecture. The final outcome of the thesis is linguistically independent neural network model for classifying texts at character level reaching up to 93,64% accuracy. Training and testing data were provided by multilingual and Yelp databases. The simulations were performed on 1200000 English, 12000 Czech, German and Spanish texts.
|
|
| |
|
|
Time series analysis using deep learning
Hladík, Jakub ; Kolařík, Martin (referee) ; Uher, Václav (advisor)
The aim of the thesis was to create a tool for time-series prediction based on deep learning. The first part of the work is a brief description of deep learning and its comparison to classical machine learning. In the next section contains brief analysis of some tools, that are already used for time-series forecasting. The last part is focused on the analysis of the problem as well as on the actual creation of the program.
|
|
|
Segmentation of multiple sclerosis lesions using deep neural networks
Sasko, Dominik ; Myška, Vojtěch (referee) ; Kolařík, Martin (advisor)
Hlavným zámerom tejto diplomovej práce bola automatická segmentácia lézií sklerózy multiplex na snímkoch MRI. V rámci práce boli otestované najnovšie metódy segmentácie s využitím hlbokých neurónových sietí a porovnané prístupy inicializácie váh sietí pomocou preneseného učenia (transfer learning) a samoriadeného učenia (self-supervised learning). Samotný problém automatickej segmentácie lézií sklerózy multiplex je veľmi náročný, a to primárne kvôli vysokej nevyváženosti datasetu (skeny mozgov zvyčajne obsahujú len malé množstvo poškodeného tkaniva). Ďalšou výzvou je manuálna anotácia týchto lézií, nakoľko dvaja rozdielni doktori môžu označiť iné časti mozgu ako poškodené a hodnota Dice Coefficient týchto anotácií je približne 0,86. Možnosť zjednodušenia procesu anotovania lézií automatizáciou by mohlo zlepšiť výpočet množstva lézií, čo by mohlo viesť k zlepšeniu diagnostiky individuálnych pacientov. Našim cieľom bolo navrhnutie dvoch techník využívajúcich transfer learning na predtrénovanie váh, ktoré by neskôr mohli zlepšiť výsledky terajších segmentačných modelov. Teoretická časť opisuje rozdelenie umelej inteligencie, strojového učenia a hlbokých neurónových sietí a ich využitie pri segmentácii obrazu. Následne je popísaná skleróza multiplex, jej typy, symptómy, diagnostika a liečba. Praktická časť začína predspracovaním dát. Najprv boli skeny mozgu upravené na rovnaké rozlíšenie s rovnakou veľkosťou voxelu. Dôvodom tejto úpravy bolo využitie troch odlišných datasetov, v ktorých boli skeny vytvárané rozličnými prístrojmi od rôznych výrobcov. Jeden dataset taktiež obsahoval lebku, a tak bolo nutné jej odstránenie pomocou nástroju FSL pre ponechanie samotného mozgu pacienta. Využívali sme 3D skeny (FLAIR, T1 a T2 modality), ktoré boli postupne rozdelené na individuálne 2D rezy a použité na vstup neurónovej siete s enkodér-dekodér architektúrou. Dataset na trénovanie obsahoval 6720 rezov s rozlíšením 192 x 192 pixelov (po odstránení rezov, ktorých maska neobsahovala žiadnu hodnotu). Využitá loss funkcia bola Combo loss (kombinácia Dice Loss s upravenou Cross-Entropy). Prvá metóda sa zameriavala na využitie predtrénovaných váh z ImageNet datasetu na enkodér U-Net architektúry so zamknutými váhami enkodéra, resp. bez zamknutia a následného porovnania s náhodnou inicializáciou váh. V tomto prípade sme použili len FLAIR modalitu. Transfer learning dokázalo zvýšiť sledovanú metriku z hodnoty približne 0,4 na 0,6. Rozdiel medzi zamknutými a nezamknutými váhami enkodéru sa pohyboval okolo 0,02. Druhá navrhnutá technika používala self-supervised kontext enkodér s Generative Adversarial Networks (GAN) na predtrénovanie váh. Táto sieť využívala všetky tri spomenuté modality aj s prázdnymi rezmi masiek (spolu 23040 obrázkov). Úlohou GAN siete bolo dotvoriť sken mozgu, ktorý bol prekrytý čiernou maskou v tvare šachovnice. Takto naučené váhy boli následne načítané do enkodéru na aplikáciu na náš segmentačný problém. Tento experiment nevykazoval lepšie výsledky, s hodnotou DSC 0,29 a 0,09 (nezamknuté a zamknuté váhy enkodéru). Prudké zníženie metriky mohlo byť spôsobené použitím predtrénovaných váh na vzdialených problémoch (segmentácia a self-supervised kontext enkodér), ako aj zložitosť úlohy kvôli nevyváženému datasetu.
|
|
|
Optimization of aluminium casting process using numerical simulation
Kolařík, Martin ; Lána, Ivo (referee) ; Krutiš, Vladimír (advisor)
The master’s thesis deals with the analysis of casting technology of the selected aluminium casting. It is a casting of a part of CNC milling machine and it is cast by gravity casting into a permanent mold. The defects which are the cause of a high percentage of nonconforming production were analyzed. Furthermore, the master’s thesis includes a complete analysis of filling and solidification of this casting in the ProCast simulation program. Numerical simulation results are verified and improved. Then the causes of problematic casting defects are proven on several calculated variants. Measures are proposed to minimize the tendency to produce castings with defects leading to nonconforming production.
|
|
|
Email spam filtering using artificial intelligence
Safonov, Yehor ; Uher, Václav (referee) ; Kolařík, Martin (advisor)
In the modern world, email communication defines itself as the most used technology for exchanging messages between users. It is based on three pillars which contribute to the popularity and stimulate its rapid growth. These pillars are represented by free availability, efficiency and intuitiveness during exchange of information. All of them constitute a significant advantage in the provision of communication services. On the other hand, the growing popularity of email technologies poses considerable security risks and transforms them into an universal tool for spreading unsolicited content. Potential attacks may be aimed at either a specific endpoints or whole computer infrastructures. Despite achieving high accuracy during spam filtering, traditional techniques do not often catch up to rapid growth and evolution of spam techniques. These approaches are affected by overfitting issues, converging into a poor local minimum, inefficiency in highdimensional data processing and have long-term maintainability issues. One of the main goals of this master's thesis is to develop and train deep neural networks using the latest machine learning techniques for successfully solving text-based spam classification problem belonging to the Natural Language Processing (NLP) domain. From a theoretical point of view, the master's thesis is focused on the e-mail communication area with an emphasis on spam filtering. Next parts of the thesis bring attention to the domain of machine learning and artificial neural networks, discuss principles of their operations and basic properties. The theoretical part also covers possible ways of applying described techniques to the area of text analysis and solving NLP. One of the key aspects of the study lies in a detailed comparison of current machine learning methods, their specifics and accuracy when applied to spam filtering. At the beginning of the practical part, focus will be placed on the e-mail dataset processing. This phase was divided into five stages with the motivation of maintaining key features of the raw data and increasing the final quality of the dataset. The created dataset was used for training, testing and validation of types of the chosen deep neural networks. Selected models ULMFiT, BERT and XLNet have been successfully implemented. The master's thesis includes a description of the final data adaptation, neural networks learning process, their testing and validation. In the end of the work, the implemented models are compared using a confusion matrix and possible improvements and concise conclusion are also outlined.
|
|
|
Automatic 3D segmentation of brain images
Bafrnec, Matúš ; Dorazil, Jan (referee) ; Kolařík, Martin (advisor)
This bachelor thesis describes the design and implementation of the system for automatic 3D segmentation of a brain based on convolutional neural networks. The first part is dedicated to a brief history of neural networks and a theoretical description of the functionality of convolutional neural networks. It represents a fast introduction to the problematics and provides theoretical basics needed for the understanding and creation of the system. Individual layers of the neural network and principles of their functionality and mutual relations are also described in this part. The second part of the thesis is about problem analysis, designing of a solution and a comparison between neural networks and other solutions. The result of a magnetic resonance imaging of the head is a series of black-and-white images representing a 3D scan. The task is to tag a brain and to remove unnecessary information in the form of surrounding tissues. The final image of the brain can be utilized in a volumetry or during a diagnostic of neurodegenerative diseases. The advantage of neural networks in comparison with deterministic systems is their flexibility. They allow an adaptation to other segmentation problems just by changing the training dataset, without a need of changes in the architecture. One of the systems performing fully automatic 3D segmentation is called U-Net – its name comes from the similarity of the architecture with the letter U. Three real solutions, the first implementation of U-Net, extended U-Net and recurrent U-Net were presented. The first version of U-Net has been very memory-demanding, it required a training on a processor instead of a graphic card and has not allowed data processing in full resolution. The extended U-Net has resolved these problems by loading data in overlaying series of three images. In addition to the possibility of a training on a graphic card with related decrease in learning time, the accuracy was increased by adding interconnections to the internal architecture of the network. The last version, recurrent U-Net, aims for the optimization of extended U-Net based on the reusage of existing levels. This brings a decrease in a time and resource difficulty. The number of parameters of the network was lowered to less than 20%, without any increase in case of further level addition. This network is one of first recurrent networks used on the problem of 3D segmentation and provides a foundation to further research. The last part focuses on the evaluation of results and the comparison of accuracy, speed and requirements between particular networks. The accuracy of human and machine segmentation is also compared. The extended and recurrent U-Net have surpassed their human opponent, which in real case could save a lot of doctors time and prevent human mistakes. The result of this work is a theoretical basis providing an introduction to the problematics of convolutional neural networks and segmentation, fully working systems for automatic 3D segmentation and the foundation for further research in the field of recurrent networks.
|
|
|
Automatic quality control of painted metal parts production using neural networks
Ježek, Štěpán ; Kolařík, Martin (referee) ; Burget, Radim (advisor)
This thesis is focused on the problem of visual quality control during painted metal parts fabrication. The main problem of the thesis is the design of automatic quality control method based on modern artificial intelligence and computer vision techniques. Quality control is an important part of a large number of industrial production processes, in which it is necessary to ensure compliance with a number of quality requirements for manufactured products. Until now, quality control is carried out mainly by specialized staff, who are subject to a number of expertise requirements. Currently known methods of visual quality control based on artificial intelligence are characterized by high demands on the size of the training data set and low tolerance for a significant change in position and rotation of the inspected objects relative to the scanning device. As a result of these shortcomings, the use of automated visual quality control in many current industrial applications is impossible. The main contribution of this thesis is the design of a new method for quality control, which shows a strong ability to function reliably even in cases where the above mentioned phenomena of change in position, rotation of objects and lack of training data occur during manufacturing. The accuracy of the method proposed in this thesis is experimentally verified on a data set based on the issue of quality control of painted metal parts. According to the measurement results of defect detection accuracy, the proposed method outperformed other, currently known methods by 10, 25 % using the AUROC metric.
|
|
|
Image segmentation using graph neural networks
Boszorád, Matej ; Kolařík, Martin (referee) ; Myška, Vojtěch (advisor)
This diploma thesis describes and implements the design of a graph neural network usedfor 2D segmentation of neural structure. The first chapter of the thesis briefly introduces the problem of segmentation. In this chapter, segmentation techniques are divided according to the principles of the methods they use. Each type of technique contains the essence of this category as well as a description of one representative. The second chapter of the diploma thesis explains graph neural networks (GNN for short). Here, the thesis divides graph neural networks in general and describes recurrent graph neural networks(RGNN for short) and graph autoencoders, that can be used for image segmentation, in more detail. The specific image segmentation solution is based on the message passing method in RGNN, which can replace convolution masks in convolutional neural networks.RGNN also provides a simpler multilayer perceptron topology. The second type of graph neural networks characterised in the thesis are graph autoencoders, which use various methods for better encoding of graph vertices into Euclidean space. The last part ofthe diploma thesis deals with the analysis of the problem, the proposal of its specific solution and the evaluation of results. The purpose of the practical part of the work was the implementation of GNN for image data segmentation. The advantage of using neural networks is the ability to solve different types of segmentation by changing training data. RGNN with messaging passing and node2vec were used as implementation GNNf or segmentation problem. RGNN training was performed on graphics cards provided bythe school and Google Colaboratory. Learning RGNN using node2vec was very memory intensive and therefore it was necessary to train on a processor with an operating memory larger than 12GB. As part of the RGNN optimization, learning was tested using various loss functions, changing topology and learning parameters. A tree structure method was developed to use node2vec to improve segmentation, but the results did not confirman improvement for a small number of iterations. The best outcomes of the practical implementation were evaluated by comparing the tested data with the convolutional neural network U-Net. It is possible to state comparable results to the U-Net network, but further testing is needed to compare these neural networks. The result of the thesisis the use of RGNN as a modern solution to the problem of image segmentation and providing a foundation for further research.
|
|
|
Deep Learning for Text Classification
Kolařík, Martin ; Harár, Pavol (referee) ; Povoda, Lukáš (advisor)
Thesis focuses on analysis of contemporary machine learning methods used for text classification based on emotion and testing several deep neural nework architectures. Outcome of this thesis is a neural network architecture, which is tuned for using with text data and which had the best result of 79,94 percent. Proposed method is language independent and it doesn’t require as precisely classified training datasets as current methods. Training and testing datasets were consisted of short amateur movie reviews in Czech and in English. Thesis contains also overview of theoretical basics for convolutional neural networks and history of neural networks and language processing Scripts were written in Python, neural networks were simulated using Keras library and Theano framework. We used CUDA for better performance.
|
guest ::
RSS feed.