|
|
Recognition of Multi-Talker Overlapping Speech Using Neural Networks
Hradil, Jaromír ; Švec, Ján (oponent) ; Žmolíková, Kateřina (vedoucí práce)
This work deals with the speech recognition of overlapping speakers using a neural network. It examines the problem of speech recognition from multiple speakers and the ways in which this problem is solved. Specifically, in addition to traditional components such as convolutional neural networks, LSTM, etc., it is also an application of special components: attention mechanism and gated convolution. And also the application of a technique called permutation invariant training. Part of this work is to apply these approaches to assigned training data, which consists of artificially created mixtures of two speakers reading articles from the Wall Street Journal. The next step was to train the respective architectures using the combinations of the elements mentioned above. The models in this work replace the acoustic model. There were two architectures using different types of attention mechanism and one without it. Experiments have shown that architectures using the attention mechanism in this type of task have not surpassed more traditional architecture by suffering from gated convolution. Nevertheless, they showed potential.
|
|
|
Analysis of Polygonal Models Using Neural Networks
Dronzeková, Michaela ; Zemčík, Pavel (oponent) ; Kodym, Oldřich (vedoucí práce)
This thesis deals with rotation estimation of 3D model of human jaw. It describes and compares methods for direct analysis od 3D models as well as method to analyze model using rasterization. To evaluate perfomance of proposed method, a metric that computes number of cases when prediction was less than 30° from ground truth is used. Proposed method that uses rasterization, takes three x-ray views of model as an input and processes it with convolutional network. It achieves best preformance, 99% with described metric. Method to directly analyze polygonal model as a sequence uses attention mechanism to do so and was inspired by transformer architecture. A special pooling function was proposed for this network that decreases memory requirements of the network. This method achieves 88%, but does not use rasterization and can process polygonal model directly. It is not as good as rasterization method with x-ray display, byt it is better than rasterization method with model not rendered as x-ray. The last method uses graph representation of mesh. Graph network had problems with overfitting, that is why it did not get good results and I think this method is not very suitable for analyzing plygonal model.
|
|
|
Phishing Detection Using Deep Learning Attention Techniques
Safonov, Yehor
In the modern world, electronic communication is defined as the most used technologyfor exchanging messages between users. The growing popularity of emails brings about considerablesecurity risks and transforms them into an universal tool for spreading phishing content. Even thoughtraditional techniques achieve high accuracy during spam filtering, they do not often catch up to therapid growth and evolution of spam techniques. These approaches are affected by overfitting issues,may converge into a poor local minimum, are inefficient in high-dimensional data processing andhave long-term maintainability problems. The main contribution of this paper is to develop and trainadvanced deep networks which use attention mechanisms for efficient phishing filtering and text understanding.Key aspects of the study lie in a detailed comparison of attention based machine learningmethods, their specifics and accuracy during the application to the phishing problem. From a practicalpoint of view, the paper is focused on email data corpus preprocessing. Deep learning attention basedmodels, for instance the BERT and the XLNet, have been successfully implemented and comparedusing statistical metrics. Obtained results show indisputable advantages of deep attention techniquescompared to the common approaches.
|
|
|
Analysis of Polygonal Models Using Neural Networks
Dronzeková, Michaela ; Zemčík, Pavel (oponent) ; Kodym, Oldřich (vedoucí práce)
This thesis deals with rotation estimation of 3D model of human jaw. It describes and compares methods for direct analysis od 3D models as well as method to analyze model using rasterization. To evaluate perfomance of proposed method, a metric that computes number of cases when prediction was less than 30° from ground truth is used. Proposed method that uses rasterization, takes three x-ray views of model as an input and processes it with convolutional network. It achieves best preformance, 99% with described metric. Method to directly analyze polygonal model as a sequence uses attention mechanism to do so and was inspired by transformer architecture. A special pooling function was proposed for this network that decreases memory requirements of the network. This method achieves 88%, but does not use rasterization and can process polygonal model directly. It is not as good as rasterization method with x-ray display, byt it is better than rasterization method with model not rendered as x-ray. The last method uses graph representation of mesh. Graph network had problems with overfitting, that is why it did not get good results and I think this method is not very suitable for analyzing plygonal model.
|
|
|
Recognition of Multi-Talker Overlapping Speech Using Neural Networks
Hradil, Jaromír ; Švec, Ján (oponent) ; Žmolíková, Kateřina (vedoucí práce)
This work deals with the speech recognition of overlapping speakers using a neural network. It examines the problem of speech recognition from multiple speakers and the ways in which this problem is solved. Specifically, in addition to traditional components such as convolutional neural networks, LSTM, etc., it is also an application of special components: attention mechanism and gated convolution. And also the application of a technique called permutation invariant training. Part of this work is to apply these approaches to assigned training data, which consists of artificially created mixtures of two speakers reading articles from the Wall Street Journal. The next step was to train the respective architectures using the combinations of the elements mentioned above. The models in this work replace the acoustic model. There were two architectures using different types of attention mechanism and one without it. Experiments have shown that architectures using the attention mechanism in this type of task have not surpassed more traditional architecture by suffering from gated convolution. Nevertheless, they showed potential.
|
host ::
RSS.