|
|
Semi-Supervised Speech-to-Text Recognition with Text-to-Speech Critic
Baskar, Murali Karthick ; Manohar, Vimal (oponent) ; Trmal, Jan (oponent) ; Burget, Lukáš (vedoucí práce)
Sequence-to-sequence automatic speech recognition (ASR) models require large quantities of training data to attain good performance. For this reason, unsupervised and semi-supervised training in seq2seq models have recently witnessed a surge in interest. This work builds upon recent results showing notable improvements in semi-supervised training using cycle-consistency and related techniques. Such techniques derive training procedures and losses able to leverage unpaired speech and/or text data by combining ASR with text-to-speech (TTS) models. This thesis first proposes a new semi-supervised modelling framework combining an end-to-end differentiable ASR->TTS loss with TTS->ASR loss. The method is able to leverage unpaired speech and text data to outperform recently proposed related techniques in terms of word error rate (WER). We provide extensive results analysing the impact of data quantity as well as the contribution of speech and text modalities in recovering errors and show consistent gains across WSJ and LibriSpeech corpora. The thesis also discusses the limitations of the ASR<->TTS model in out-of-domain data conditions. We propose an enhanced ASR<->TTS (EAT) model incorporating two main features: 1) the ASR->TTS pipeline is equipped with a language model reward to penalize the ASR hypotheses before forwarding them to TTS; and 2) speech regularizer trained in unsupervised fashion is introduced in TTS->ASR to correct the synthesized speech before sending it to the ASR model. Training strategies and the effectiveness of the EAT model are explored and compared with augmentation approaches. The results show that EAT reduces the performance gap between supervised and semi-supervised training by absolute WER improvement of 2.6% and 2.7% on LibriSpeech and BABEL respectively.
|
|
|
Extensions to Probabilistic Linear Discriminant Analysis for Speaker Recognition
Plchot, Oldřich ; Fousek, Petr (oponent) ; McCree,, Alan (oponent) ; Burget, Lukáš (vedoucí práce)
This thesis deals with probabilistic models for automatic speaker verification. In particular, the Probabilistic Linear Discriminant Analysis (PLDA) model, which models i--vector representation of speech utterances, is analyzed in detail. The thesis proposes extensions to the standard state-of-the-art PLDA model. The newly proposed Full Posterior Distribution PLDA models the uncertainty associated with the i--vector generation process. A new discriminative approach to training the speaker verification system based on the~PLDA model is also proposed. When comparing the original PLDA with the model extended by considering the i--vector uncertainty, results obtained with the extended model show up to 20% relative improvement on tests with short segments of speech. As the test segments get longer (more than one minute), the performance gain of the extended model is lower, but it is never worse than the baseline. Training data are, however, usually available in the form of segments which are sufficiently long and therefore, in such cases, there is no gain from using the extended model for training. Instead, the training can be performed with the original PLDA model and the extended model can be used if the task is to test on the short segments. The discriminative classifier is based on classifying pairs of i--vectors into two classes representing target and non-target trials. The functional form for obtaining the score for every i--vector pair is derived from the PLDA model and training is based on the logistic regression minimizing the cross-entropy error function between the correct labeling of all trials and the probabilistic labeling proposed by the system. The results obtained with discriminatively trained system are similar to those obtained with generative baseline, but the discriminative approach shows the ability to output better calibrated scores. This property leads to a better actual verification performance on an unseen evaluation set, which is an important feature for real use scenarios.
|
|
|
ASL Fingerspelling Recognition Using Slow Feature Analysis
Winkler, Martin ; Hradiš, Michal (oponent) ; Burget, Lukáš (vedoucí práce)
This work describes the process of testing slow feature analysis as a method of extracting rhobust features from complex image data of american sign language. For purposes of testing a system in python is created that facilitates test runs and offers rich scale of changable specifications to allow the user run various tests in order to determine how viable the method is for classification and recognition of hand shapes. The theoretical part introduces the slow feature analysis, discusses the structure of the system and describes the dataset on which the method is to be observed. In practical part the method was subjected to performance analysis on seen and unseen speakers, its viability with higher number of gestures and some interesting input data formatting in attempt to improve the performance.
|
|
|
Paralelní trénování neuronových sítí pro rozpoznávání řeči
Veselý, Karel ; Fousek, Petr (oponent) ; Burget, Lukáš (vedoucí práce)
Tato diplomová práce je zaměřena na paralelizaci trénování neuronových sítí pro rozpoznávání řeči. V rámci této diplomové práce byly implementovány a porovnány dvě strategie paralelizace. První strategií je paralelizace dat s využitím rozdělení trénování do několika POSIX vláken. Druhou strategií je paralelizace uzlů s využitím platformy pro obecné výpočty na grafických kartách CUDA. V případě první strategie bylo dosaženo 4x urychlení, v případě využití platformy CUDA bylo dosaženo téměř 10x urychlení. Pro trénování byl použit algoritmus Stochastic Gradient Descent se zpětným šířením chyb. Po krátkém úvodu následuje druhá kapitola práce, která je motivační a zasazuje probém do kontextu rozpoznávání řeči. Třetí kapitola práce je teoretická a diskutuje neuronové sítě a metodu trénování. Následující kapitoly jsou zaměřené na návrh a implementaci a popisují iterativní vývoj tohoto projektu. Poslední obsáhlá kapitola popisuje testovací systém a uvádí výsledky provedených experimentů. V závěru jsou krátce zhodnoceny dosažené výsledky a nastíněna perspektiva dalšího vývoje projektu.
|
|
|
Optimization of Gaussian Mixture Subspace Models and Related Scoring Algorithms in Speaker Verification
Glembek, Ondřej ; Brummer, Niko (oponent) ; Campbell,, William (oponent) ; Burget, Lukáš (vedoucí práce)
This thesis deals with Gaussian Mixture Subspace Modeling in automatic speaker recognition. The thesis consists of three parts. In the first part, Joint Factor Analysis (JFA) scoring methods are studied. The methods differ mainly in how they deal with the channel of the tested utterance. The general JFA likelihood function is investigated and the methods are compared both in terms of accuracy and speed. It was found that linear approximation of the log-likelihood function gives comparable results to the full log-likelihood evaluation while simplyfing the formula and dramatically reducing the computation speed. In the second part, i-vector extraction is studied and two simplification methods are proposed. The motivation for this part was to allow for using the state-of-the-art technique on small scale devices and to setup a simple discriminative-training system. It is shown that, for long utterances, while sacrificing the accuracy, we can get very fast and compact i-vector systems. On a short-utterance(5-second) task, the results of the simplified systems are comparable to the full i-vector extraction. The third part deals with discriminative training in automatic speaker recognition. Previous work in the field is summarized and---based on the knowledge from the earlier chapters of this work---discriminative training of the i-vector extractor parameters is proposed. It is shown that discriminative re-training of the i-vector extractor can improve the system if the initial estimation is computed using the generative approach.
|
|
|
Rozpoznávání řeči pro leteckou komunikaci
Žmolíková, Kateřina ; Burget, Lukáš (oponent) ; Veselý, Karel (vedoucí práce)
Tato bakalářská práce se zabývá rozpoznáváním řeči. Jejím cílem je postavit systém rozpoznávání řeči založený na neuronových sítích a otestovat jej na nahrávkách letecké komunikace. Výsledný akustický model bude použit v projektu A-PiMod. Postavený systém dosáhl na testovacích datech úspěšnost 29.5% WER. Dalším úkolem práce byly experimenty s neuronovými sítěmi, které jsou součástí akustického modelu. První experimenty zkoumaly možnost jejich zjednodušení a urychlení a dopad na úspěšnost rozpoznávání. Další se zabývaly aktivační funkcí rectifier a také konvolučními neuronovými sítěmi. V experimentech s konvolučními neuronovými sítěmi bylo dosáhnuto 1.5% zlepšení a dosáhly tak o 0.4% lepšího výsledku než plně propojená neuronová síť se stejnou architekturou.
|
|
|
Fixed-point implementace rozpoznávače řeči
Král, Tomáš ; Černocký, Jan (oponent) ; Burget, Lukáš (vedoucí práce)
Táto diplomová práca sa zaoberá problematikou automatického rozpoznávania reči na systémoch s obmedzenými hardwarovými prostriedkami - embedded systems. Cieľom projektu je navrhnúť a implementovať systém rozpoznávania reči na embedded systémy, ktoré nedisponujú floating-point výpočetnými jednotkami. V prvom rade bola zvolená vhodná hardwarová architektúra a s ohľadom na dostupné prostriedky, ktorými vybraná architektúra disponuje, bolo navrhnuté riešenie rozpoznávania reči. Jednotlivé časti systému rozpoznávania boli následne v priebehu vývoja optimalizované do takej podoby, aby mohli byť nasadené na zvolený HW. Výsledkom práce je dosiahnutie rozpoznávania českých čísloviek na embedded systéme.
|
|
|
Intersession Variability Compensation in Language and Speaker Identification
Hubeika, Valiantsina ; Burget, Lukáš (oponent) ; Matějka, Pavel (vedoucí práce)
Varibiality in the channel and session is an important issue in the text-independent speaker recognition task. To date, several techniques providing channel and session variability compensation were introduced in a number of scientic papers. Such implementation can be done in feature, model and score domain. Relatively new and powerful approach to remove channel distortion is so-called eigenchannel adaptation for Gaussian Mixture Models (GMM). The drawback of the technique is that it is not applicable in its original implementation to different types of classifiers, eg. Support Vector Machines (SVM), GMM with different number of Gaussians or in speech recognition task using Hidden Markov Models (HMM). The solution can be the approximation of the technique, eigenchannel adaptation in feature domain. Both, the original eigenchannel adaptation and eigenchannel adaptation on features in task of speaker recognition are presented. After achieving good results in speaker recognition, contribution of the same techniques was examined in acoustic language identification system with $14$ languages. In this task undesired factors are channel and speaker variability. Presented results are presented on the NIST Speaker Recognition Evaluation 2006 data and NIST Language Recognition Evaluation 2007 data.
|
|
|
Discovering Acoustic Units from Speech: a Bayesian Approach
Ondel, Lucas Antoine Francois ; Häb-Umbach, Reinhold (oponent) ; Glass, Jim (oponent) ; Burget, Lukáš (vedoucí práce)
From an early age, infants show an innate ability to infer linguistic structures from the speech signal long before they learn to read and write. In contrast, modern speech recognition systems require large collections of transcribed data to achieve a low error rate. The relatively recent field of Unsupervised Speech Learning has been dedicated to endow machines with a similar ability. As a part of this ongoing effort, this thesis focuses on the problem of discovering a set of acoustic units from a language given untranscribed audio recordings. Particularly, we explore the potential of Bayesian inference to address this problem. First, we revisit the state-of-the-art non-parametric Bayesian model for the task of acoustic unit discovery and derive a fast and efficient Variational Bayes inference algorithm. Our approach relies on the stick-breaking construction of the Dirichlet Process which allows expressing the model as a Hidden Markov Model-based phone-loop. With this model and a suitable mean-field approximation of the variational posterior, the inference is made with an efficient iterative algorithm similar to the Expectation-Maximization scheme. Experiments show that this approach performs a better clustering than the original model while being orders of magnitude faster. Secondly, we address the problem of defining a meaningful a priori distribution over the potential acoustic units. To do so, we introduce the Generalized Subspace Model, a theoretical framework that allows defining distributions over low-dimensional manifolds in high-dimensional parameter space. Using this tool, we learn a phonetic subspace - a continuum of phone embeddings-from several languages with transcribed recordings. Then, this phonetic subspace is used to constrain our system to discover acoustic units that are similar to phones from other languages. Experimental results show that this approach significantly improves the clustering quality as well as the segmentation accuracy of the acoustic unit discovery system. Finally, we enhance our acoustic units discovery model by using a Hierarchical Dirichlet Process prior instead of the simple Dirichlet Process. By doing so, we introduce a Bayesian bigram phonotactic language model to the acoustic unit discovery system. This approach captures more accurately the phonetic structure of the target language and consequently helps the clustering of the speech signal. Also, to fully exploit the benefits of the phonotactic language model, we derive a modified Variational Bayes algorithm that can balance the preponderance of the role of the acoustic and language model during inference.
|
|
|
Finite-state based recognition networks for forward-backward speech decoding
Hannemann, Mirko ; AD, Ralf Schlüter, (oponent) ; Novák,, Miroslav (oponent) ; Burget, Lukáš (vedoucí práce)
Many tasks can be formulated in the mathematical framework of weighted finite state transducers (WFST). This is also the case for automatic speech recognition (ASR). Nowadays, ASR makes extensive use of composed probabilistic models -- called decoding graphs or recognition networks. They are constructed from the individual components via WFST operations like composition. Each component is a probabilistic knowledge source that constrains the search for the best path through the composed graph -- called decoding. The usage of a coherent framework guarantees, that the resulting automata will be optimal in a well-defined sense. WFSTs can be optimized with the help of determinization and minimization in a given semi-ring. The application of these algorithms results in the optimal structure for search and the optimal distribution of weights is achieved by applying a weight pushing algorithm. The goal of this thesis is to further develop the recipes and algorithms for the construction of optimal recognition networks. We introduce an alternative weight pushing algorithm, that is suitable for an important class of models -- language model transducers, or more generally cyclic WFSTs and WFSTs with failure (back-off) transitions. We also present a recipe to construct recognition networks, which are suitable for decoding backwards in time, and which, at the same time, are guaranteed to give exactly the same probabilities as the forward recognition network. For that purpose, we develop an algorithm for exact reversal of back-off language models and their corresponding language model transducers. We apply these backward recognition networks in an optimization technique: In a static network decoder, we use it for a two-pass decoding setup (forward search and backward search). This approach is called tracked decoding and allows to incorporate the first pass decoding into the second pass decoding by tracking hypotheses from the first pass lattice. This technique results in significant speed-ups, since it allows to decode with a variable beam width, which is most of the time much smaller than the baseline beam. We also show that it is possible to apply the algorithms in a dynamic network decoder by using the incrementally refining recognition setup. This additionally leads to a partial parallelization of the decoding.
|
host ::
RSS.