|
|
Diagnosing anxiety and depression from brain electroencephalogram (EEG) signals
Osvald, Martin ; Jaroš, Marta (oponent) ; Malik, Aamir Saeed (vedoucí práce)
Mental disorders represent inevitable emotions in our society. These psychological states affect the cognitive, emotional and behavioural functioning of individuals. Common men- tal disorders fall into two main diagnostic categories: depressive disorders and anxiety disorders. The aim of this work is to find a new method for detecting whether a given patient suffers from anxiety or depression using EEG classification. In this work, we use a combination of genetic algorithms and models from deep learning.
|
|
|
Aligning pre-trained models for spoken language translation
Sedláček, Šimon ; Beneš, Karel (oponent) ; Kesiraju, Santosh (vedoucí práce)
In this work, we investigate a novel approach to end-to-end speech translation (ST) by leveraging pre-trained models for automatic speech recognition (ASR) and machine translation (MT) and connecting them with a small connector module (Q-Former, STE). The connector bridges the gap between the speech and text modalities, transforming the ASR encoder embeddings into the latent representation space of the MT encoder. During training, the foundation ASR and MT models are frozen, and only the connector parameters are tuned, optimizing for the ST objective. We train and evaluate our models on the How2 English to Portuguese ST dataset. In our experiments, aligned systems outperform our cascade ST baseline while utilizing the same foundation models. Additionally, while keeping the size of the connector module constant and small in comparison (10M parameters), increasing the size and capability of the ASR encoder and MT decoder universally improves translation results. We find that the connectors can also serve as domain adapters for the foundation models, significantly improving translation performance in the aligned ST setting, compared even to the base MT scenario. Lastly, we propose a pre-training procedure for the connector, with the potential for reducing the amount of ST data required for training similar aligned systems.
|
|
|
Search in speech recordings based on semantic vectors
Boboš, Dominik ; Karafiát, Martin (oponent) ; Schwarz, Petr (vedoucí práce)
In the current era of information overload, efficient methods for information retrieval are crucial. This thesis summarises methods for obtaining vector representations for text and audio, also known as semantic vectors. We took a deeper look at joint-representation models such as SpeechT5 and SeamlessM4T, which transform these various forms of input into one shared vector space. Based on these models, we built a system which allows us to search in data regardless of the modality. In order to evaluate the proposed solution on semantic search tasks, apart from standard keyword spotting tasks, we labelled a dataset to capture similar semantic meanings of the keywords or phrases. Finally, we conducted several experiments, where we explored the possibilities of the models used by limiting the context seen during finetuning or involving text-to-speech (TTS) systems to improve overall performance.
|
|
|
Automated Representation Learning for Cartesian Genetic Programming Using Neural Networks
Koči, Martin ; Mrázek, Vojtěch (oponent) ; Sekanina, Lukáš (vedoucí práce)
This master's thesis addresses the integration of neural networks and Cartesian Genetic Programming (CGP). It explores the use of neural networks for automated representation creation for CGP and their application to improve the evolutionary process in CGP. The study covers basic concepts of machine learning, including various types of learning and neural network models. It also touches on evolutionary algorithms with an emphasis on their basic principles, general algorithms, and types of representations. This work also includes principles of representation learning and two fundamental architectures for their creation. It describes the subsequent use of representation learning in genetic programming. The solution design includes data acquisition and preprocessing, representation creation processes, and the utilization of the resulting representations. The thesis also implements two new approaches for creating representations for Cartesian genetic programs. It further explores their use in two new mutation operators, where one is based on direct modification of the vector representation and the other on the selection of genes for mutation based on their similarity. The last of the explored areas is predicting the suitability of candidate solutions using newly emerged representations.
|
|
|
Human web browsing simulation
Doležal, Jáchym ; Setinský, Jiří (oponent) ; Hranický, Radek (vedoucí práce)
This work introduces a promising tool for automated web navigation and achieving specific goals based on decisions made by a Large Language Model using information from a current page. The results of the simulator with model GPT 4 Turbo demonstrate the tool’s effectiveness, achieving over 80% success in completing predefined goals. The results show the usability of this tool in real use cases.
|
|
|
Generating Documentation to Source Code in Python
Novosád, Juraj ; Nosko, Svetozár (oponent) ; Smrž, Pavel (vedoucí práce)
The aim of this work is to adapt selected language models on domain data and to develop a system that would allow their use on commonly available hardware. The models have been adapted to generate documentation for undocumented source code in the Python progra- mming language to follow the Google Style convention. A prerequisite of model adaptation was to obtain domain data and process it appropriately for the purpose of model fine-tuning. This work focuses on fine-tuning models with fewer than one billion parameters, for the sake of enabling inference even on commonly available hardware. Part of the work was to objectively evaluate the quality of the adapted models. For this reason, I developed a tool that evaluates the quality of the generated documentation on a selected corpus of models. The evaluation of the adapted models showed that they achieve comparable performance to multiply larger models for general tasks, such as gpt-3.5-turbo-0125. The result of this work is a server capable of horizontal scaling that integrates the capabilities of more than just the adapted models through an easy-to-use API.
|
|
| |
|
|
Commented translation of a text on science and technology
Karzel, Vítězslav ; Smutný, Milan (oponent) ; Krhutová, Milena (vedoucí práce)
The goal of this Bachelor thesis is to translate and analyze a specialized text which focuses on transformers. The whole work is divided into two parts. The first part concentrates on the translation and overall preparation of the basis for the analysis which follows in the second part of the semestral thesis. The second part concentrates on the analysis of the text and all linguistic means which were used and the main task is to comment properly on selected phenomena that the analysis reveals.
|
|
|
Klasifikace vztahů mezi pojmenovanými entitami v textu
Ondřej, Karel ; Doležal, Jan (oponent) ; Smrž, Pavel (vedoucí práce)
Tato diplomová práce se zabývá extrakcí vztahů mezi pojmenovanými entitami v textu. V teoretické části práce je rozebrána problematika reprezentace přirozeného jazyka pro strojové zpracování. Následně jsou definovány dvě dílčí úlohy extrakce vztahů, a to rozpoznávání pojmenovaných entit a klasifikace vztahu mezi nimi, včetně shrnutí dnešních nejmodernějších řešení. V praktické části práce je navržen systém pro automatickou extrakci vztahů mezi pojmenovanými entitami ze stažených webových stránek. Model pro klasifikaci vztahů mezi entitami je založen na předtrénovaném modelu sítě typu transfomers. V práci jsou porovnány čtyři předtrénované modely, a to BERT, XLNet, RoBERTa a ALBERT.
|
|
|
Multilingual Open-Domain Question Answering
Slávka, Michal ; Dočekal, Martin (oponent) ; Fajčík, Martin (vedoucí práce)
This thesis explores automatic Multilingual Open-Domain Question Answering. In this work are proposed approaches to this less explored research area. More precisely, this work examines if: (i) utilization of an English system is sufficient, (ii) multilingual models can benefit from a translated question into other languages (iii) or avoiding translation is a better choice. English system based on the T5 model that uses a machine translation is compared to natively multilingual systems based on the multilingual MT5 model. The English system with machine translation only slightly outperforms its monolingual counterparts in multiple tasks. Compared to multilingual models, the English system was trained on a much larger dataset, but the results were comparable. This shows that the use of natively multilingual systems is a promising approach for future research. I also present a method of retrieving multilingual evidence using the BM25 ranking algorithm and compare it with English retrieval. The use of multilingual evidence seems to be beneficial and improves the performance of the systems.
|
host ::
RSS.