|
|
Semi-Supervised Speech-to-Text Recognition with Text-to-Speech Critic
Baskar, Murali Karthick ; Manohar, Vimal (oponent) ; Trmal, Jan (oponent) ; Burget, Lukáš (vedoucí práce)
Sequence-to-sequence automatic speech recognition (ASR) models require large quantities of training data to attain good performance. For this reason, unsupervised and semi-supervised training in seq2seq models have recently witnessed a surge in interest. This work builds upon recent results showing notable improvements in semi-supervised training using cycle-consistency and related techniques. Such techniques derive training procedures and losses able to leverage unpaired speech and/or text data by combining ASR with text-to-speech (TTS) models. This thesis first proposes a new semi-supervised modelling framework combining an end-to-end differentiable ASR->TTS loss with TTS->ASR loss. The method is able to leverage unpaired speech and text data to outperform recently proposed related techniques in terms of word error rate (WER). We provide extensive results analysing the impact of data quantity as well as the contribution of speech and text modalities in recovering errors and show consistent gains across WSJ and LibriSpeech corpora. The thesis also discusses the limitations of the ASR<->TTS model in out-of-domain data conditions. We propose an enhanced ASR<->TTS (EAT) model incorporating two main features: 1) the ASR->TTS pipeline is equipped with a language model reward to penalize the ASR hypotheses before forwarding them to TTS; and 2) speech regularizer trained in unsupervised fashion is introduced in TTS->ASR to correct the synthesized speech before sending it to the ASR model. Training strategies and the effectiveness of the EAT model are explored and compared with augmentation approaches. The results show that EAT reduces the performance gap between supervised and semi-supervised training by absolute WER improvement of 2.6% and 2.7% on LibriSpeech and BABEL respectively.
|
|
| |
|
|
Finite State Grammars and Language Models for Automatic Speech Recognition
Beneš, Karel ; Glembek, Ondřej (oponent) ; Hannemann, Mirko (vedoucí práce)
This thesis deals with the transformation of Context Free Grammars (CFG) into Weighted Finite State Transducers (WFST). A subset of CFG is chosen, that can be transformed exactly. Both the test of whether a CFG fulfills such condition and the algorithm for the following transformation are presented. A tool has been implemented, which performs both these tasks, also its input and output processing are reported. Using this tool, a speech recognition system for aircraft cockpit control has been built. Results are presented which show, that the system based on the transformed grammar outperforms the system based on general-purpose language model.
|
|
|
Finite-state based recognition networks for forward-backward speech decoding
Hannemann, Mirko ; AD, Ralf Schlüter, (oponent) ; Novák,, Miroslav (oponent) ; Burget, Lukáš (vedoucí práce)
Many tasks can be formulated in the mathematical framework of weighted finite state transducers (WFST). This is also the case for automatic speech recognition (ASR). Nowadays, ASR makes extensive use of composed probabilistic models -- called decoding graphs or recognition networks. They are constructed from the individual components via WFST operations like composition. Each component is a probabilistic knowledge source that constrains the search for the best path through the composed graph -- called decoding. The usage of a coherent framework guarantees, that the resulting automata will be optimal in a well-defined sense. WFSTs can be optimized with the help of determinization and minimization in a given semi-ring. The application of these algorithms results in the optimal structure for search and the optimal distribution of weights is achieved by applying a weight pushing algorithm. The goal of this thesis is to further develop the recipes and algorithms for the construction of optimal recognition networks. We introduce an alternative weight pushing algorithm, that is suitable for an important class of models -- language model transducers, or more generally cyclic WFSTs and WFSTs with failure (back-off) transitions. We also present a recipe to construct recognition networks, which are suitable for decoding backwards in time, and which, at the same time, are guaranteed to give exactly the same probabilities as the forward recognition network. For that purpose, we develop an algorithm for exact reversal of back-off language models and their corresponding language model transducers. We apply these backward recognition networks in an optimization technique: In a static network decoder, we use it for a two-pass decoding setup (forward search and backward search). This approach is called tracked decoding and allows to incorporate the first pass decoding into the second pass decoding by tracking hypotheses from the first pass lattice. This technique results in significant speed-ups, since it allows to decode with a variable beam width, which is most of the time much smaller than the baseline beam. We also show that it is possible to apply the algorithms in a dynamic network decoder by using the incrementally refining recognition setup. This additionally leads to a partial parallelization of the decoding.
|
|
|
Analýza videozáznamov správ z oblasti finančných trhov
Mikula, Michal
Tato práce se zabývá analýzou videozáznamu zpráv z oblasti finančních trhů. Mnoho médií z finanční sféry stále frekventovaněji zveřejňuje informace prostřednictvím videa, či dokonce tento formát v některých případech upřednostňuje. Manuální analýza těchto videozáznamů je časově velmi náročná. Práce se proto zabývá vytvořením nástroje umožňujícího jejich automatickou analýzu. V práci jsou řešeny dvě hlavní oblasti. První oblastí je automatické rozpoznávání řeči pro získávání přepisů videí a druhou oblastí je zpracování přirozeného jazyka pro provedení textové analýzy daného videa. Textová analýza přepisu videa v sobě zahrnuje analýzu sentimentu, textovou sumarizaci a extrakci klíčových frází.
|
|
|
Out-of-Vocabulary Words Detection and Recovery
Egorova, Ekaterina ; Hannemann, Mirko (oponent) ; Schaaf, Thomas (oponent) ; Černocký, Jan (vedoucí práce)
The thesis explores the field of out-of-vocabulary word (OOV) processing within the task of automatic speech recognition (ASR). It defines the two separate OOV processing tasks - that of detection and recovery - and proposes success metrics for both the tasks. Different approaches to OOV detection and recovery are presented within the frameworks of hybrid and end-to-end (E2E) ASR. These approaches and compared on an open access LibriSpeech database to facilitate replicability. Hybrid approach uses modified decoding graph with phoneme substrings and utilizes full lattice representations for detection and recovery of recurrent OOVs. Recovered OOVs are added to the dictionary and the language model (LM) to improve ASR system performance. The second approach employs inner representations of a word-predicting Listen Attend and Spell architecture (LAS) E2E system to perform OOV detection task. Detection recall and precision rates improved drastically in comparison with the hybrid approach. Recur-rent OOV recovery is performed on a separate character-predicting system with the use of detected time frames and probabilistic clustering.Finally, we propose a new speller architecture with a capability of learning OOV representations together with the word predicting network (WPN) training. The speller forces word embeddings to be spelling-aware during the training and thus not only provides OOV recovery, but also improves the WPN performance.
|
|
|
Finite-state based recognition networks for forward-backward speech decoding
Hannemann, Mirko ; AD, Ralf Schlüter, (oponent) ; Novák,, Miroslav (oponent) ; Burget, Lukáš (vedoucí práce)
Many tasks can be formulated in the mathematical framework of weighted finite state transducers (WFST). This is also the case for automatic speech recognition (ASR). Nowadays, ASR makes extensive use of composed probabilistic models -- called decoding graphs or recognition networks. They are constructed from the individual components via WFST operations like composition. Each component is a probabilistic knowledge source that constrains the search for the best path through the composed graph -- called decoding. The usage of a coherent framework guarantees, that the resulting automata will be optimal in a well-defined sense. WFSTs can be optimized with the help of determinization and minimization in a given semi-ring. The application of these algorithms results in the optimal structure for search and the optimal distribution of weights is achieved by applying a weight pushing algorithm. The goal of this thesis is to further develop the recipes and algorithms for the construction of optimal recognition networks. We introduce an alternative weight pushing algorithm, that is suitable for an important class of models -- language model transducers, or more generally cyclic WFSTs and WFSTs with failure (back-off) transitions. We also present a recipe to construct recognition networks, which are suitable for decoding backwards in time, and which, at the same time, are guaranteed to give exactly the same probabilities as the forward recognition network. For that purpose, we develop an algorithm for exact reversal of back-off language models and their corresponding language model transducers. We apply these backward recognition networks in an optimization technique: In a static network decoder, we use it for a two-pass decoding setup (forward search and backward search). This approach is called tracked decoding and allows to incorporate the first pass decoding into the second pass decoding by tracking hypotheses from the first pass lattice. This technique results in significant speed-ups, since it allows to decode with a variable beam width, which is most of the time much smaller than the baseline beam. We also show that it is possible to apply the algorithms in a dynamic network decoder by using the incrementally refining recognition setup. This additionally leads to a partial parallelization of the decoding.
|
|
|
Neural networks for automatic speaker, language, and sex identification
Do, Ngoc ; Jurčíček, Filip (vedoucí práce) ; Peterek, Nino (oponent)
Název: Neuronové sítě pro automatické rozpoznávání řečníka, jazyka a pohlaví Autorka: Bich-Ngoc Do Katedra: Ústav formální a aplikované lingvistiky Vedoucí práce: Ing. Mgr. Filip Jurek, Ph.D., Ústav formální a aplikované lingvistiky, a Dr. Marco Wiering, Institut umělé inteligence a kognitivních věd, Fakulta matematiky a přírodních věd, Univerzita v Groningenu Abstrakt: Rozpoznávání řečníka je náročný úkol a má využití v mnoha oblastech, například využítí pro autorizaci nebo forenzní vědě. V posledních letech se rozšířil koncept učení hlubokých, především hluboké neuronové sítě, které se ukázaly jako schopná technika strojového učení a dosáhly výborných úspěchů v mnoha oblastech výzkumu zpracování přirozeného jazyka a zpra- cování mluveného slova. Tato práce si dává za cíl prozkoumat možnosti modelu hlubokých neuronových sítí, rekurentních neuronových sítí v úloze rozpoznávání řečníka. Námi navržené systémy byly vyhodnoceny na kor- pusu TIMIT pro úlohu identifikace řečníka. V porovnání s jinými systémy za stejných testových podmínkách náš systém nedosáhl referenčních výsledků kvůli nedostatku validačních dat. Naše experimenty ukázaly, že nejlepší konfigurace systému je...
|
|
|
Vývoj hlasově ovládaných webových her pomocí CloudASR
Milota, Jan ; Jurčíček, Filip (vedoucí práce) ; Vidová Hladká, Barbora (oponent)
Cílem práce je navrhnout a vyvinout software pro výuku jazyků hrou za použití webových technologií a čerstvě vznikající CloudASR knihovny. Běžný uživatel provozuje interakci se svým prohlížečem skoro výhradně prostřednictvím myši a klávesnice. Díky softwaru, který tato práce reprezentuje, má nyní uživatel příležitost zabřednout do někdy ne úplně populární výuky jazyka i za pomoci svého hlasu. Což nabízí zmíněné výuce netušené možnosti, obzvláště stran uživatelské interaktivity. Důraz byl kladen na uživatelskou přívětivost, grafickou fidelitu a na kompetitivní aspekt výuky, využívajíc Facebookovou integraci a bodové hodnotící žebříčky. Powered by TCPDF (www.tcpdf.org)
|
|
|
Finite State Grammars and Language Models for Automatic Speech Recognition
Beneš, Karel ; Glembek, Ondřej (oponent) ; Hannemann, Mirko (vedoucí práce)
This thesis deals with the transformation of Context Free Grammars (CFG) into Weighted Finite State Transducers (WFST). A subset of CFG is chosen, that can be transformed exactly. Both the test of whether a CFG fulfills such condition and the algorithm for the following transformation are presented. A tool has been implemented, which performs both these tasks, also its input and output processing are reported. Using this tool, a speech recognition system for aircraft cockpit control has been built. Results are presented which show, that the system based on the transformed grammar outperforms the system based on general-purpose language model.
|
host ::
RSS.