|
|
Evolutionary Design of Neural Networks
Kastner, Jan ; Hurta, Martin (referee) ; Sekanina, Lukáš (advisor)
The thesis deals with the implementation of a problem-solving method for the automated design of convolutional neural networks (CNN) architectures. The optimization of two fundamental and often conflicting characteristics, the number of parameters and the quality of CNN classification, is performed using a multi-criteria optimization genetic algorithm (NSGA-II). To encode this problem, the Cartesian genetic programming (CGP) technique is used, which enables the wide range of CNN architectures to be represented, and at the same time, the searched area can be appropriately limited by parameterization. Experiments were performed on the MNIST dataset to understand the effect of population size on the quality of the resulting solution. It is also evident from the results of the experiments that the quality of the architectures found can compete with already established models. This is therefore an alternative approach that does not require human intervention compared to manual design.
|
|
|
Evolutionary Design of Convolutional Neural Networks Utilizing a Supernet
Lamačka, Zbyněk ; Piňos, Michal (referee) ; Sekanina, Lukáš (advisor)
This work explores the possibilities of automated design and optimization of convolutional neural networks (CNNs) using evolutionary algorithms with the concept of Neural Architecture Search (NAS). NAS methods facilitate the work of neural network architects and allow access to neural networks by people who would not normally have access to them. Architectures that are created by automated methods are able to outperform architectures that were created by experienced architects. These methods are not bound by conventional design approaches, and therefore innovative architectures can emerge. The goal of this work is to design and implement a neuroevolutionary method using a supernetwork. The supernetwork concept aims to make the process of automatic network design faster and cheaper. This method will be evaluated based on the architectures it generates. The evaluation of the architectures considers two criteria -- accuracy and complexity of the network. The ImageNet dataset is used for the evaluation.
|
|
|
Evolutionary Algorithms for Neural Networks Learning
Vosol, David ; Rozman, Jaroslav (referee) ; Zbořil, František (advisor)
Main point of this thesis is to find and compare posibilities of cooperation between evolutionary algorithms and neural network learning and their comparison with classical learning technique called backpropagation. This comparison is demonstrated with deep feed-forward neural network which is used for classification tasks. The process of optimalization is via search of optimal values of weights and biases within neural network with fixed topology. We chose three evolutionary approaches. Genetic algorithm, differential evolution and particle swarm optimization algorithm. These three approaches are also compared between each other. The demonstrating program is implemented in Python3 programming language without usage of any third parties libraries focused on deep learning.
|
|
|
Neuroevolution Principles and Applications
Herec, Jan ; Strnadel, Josef (referee) ; Bidlo, Michal (advisor)
The theoretical part of this work deals with evolutionary algorithms (EA), neural networks (NN) and their synthesis in the form of neuroevolution. From a practical point of view, the aim of the work is to show the application of neuroevolution on two different tasks. The first task is the evolutionary design of the convolutional neural network (CNN) architecture that would be able to classify handwritten digits (from the MNIST dataset) with a high accurancy. The second task is the evolutionary optimization of neurocontroller for a simulated Falcon 9 rocket landing. Both tasks are computationally demanding and therefore have been solved on a supercomputer. As a part of the first task, it was possible to design such architectures which, when properly trained, achieve an accuracy of 99.49%. It turned out that it is possible to automate the design of high-quality architectures with the use of neuroevolution. Within the second task, the neuro-controller weights have been optimized so that, for defined initial conditions, the model of the Falcon booster can successfully land. Neuroevolution succeeded in both tasks.
|
|
|
Evolutionary Design of Convolutional Neural Networks
Piňos, Michal ; Vašíček, Zdeněk (referee) ; Sekanina, Lukáš (advisor)
The aim of this work is to design and implement a program for automated design of convolutional neural networks (CNN) with the use of evolutionary computing techniques. From a practical point of view, this approach reduces the requirements for the human factor in the design of CNN architectures, and thus eliminates the tedious and laborious process of manual design. This work utilizes a special form of genetic programming, called Cartesian genetic programming, which uses a graph representation for candidate solution encoding.This technique enables the user to parameterize the CNN search process and focus on architectures, that are interesting from the view of used computational units, accuracy or number of parameters. The proposed approach was tested on the standardized CIFAR-10dataset, which is often used by researchers to compare the performance of their CNNs. The performed experiments showed, that this approach has both research and practical potential and the implemented program opens up new possibilities in automated CNN design.
|
|
|
Neural Networks and Genetic Algorithm
Karásek, Štěpán ; Snášelová, Petra (referee) ; Zbořil, František (advisor)
This thesis deals with evolutionary and genetic algorithms and the possible ways of combining them. The theoretical part of the thesis describes genetic algorithms and neural networks. In addition, the possible combinations and existing algorithms are presented. The practical part of this thesis describes the implementation of the algorithm NEAT and the experiments performed. A combination with differential evolution is proposed and tested. Lastly, NEAT is compared to the algorithms backpropagation (for feed-forward neural networks) and backpropagation through time (for recurrent neural networks), which are used for learning neural networks. Comparison is aimed at learning speed, network response quality and their dependence on network size.
|
|
|
Prediktory přesnosti konvolučních neuronových sítí
Karásek, Daniel ; Mrázek, Vojtěch (referee) ; Piňos, Michal (advisor)
V posledním desetiletí došlo k obrovskému skoku v pokroku neuronových sítí, a to především díky možnosti učit větší sítě než kdy dřív. Pouhé zvětšování velikosti sítí ale není dostatečným prostředkem k jejich dalšímu zefektivnění. Z tohoto důvodu dochází ke komplexnějšímu výzkumu architektur sítí. Jeho velkou slabinou je potřeba natrénovat každou architekturu pro zjištění jejího výkonu na daném problému. To může v některých případech zabírat i dny. Alternativou k učení může být využití prediktoru přesnosti neuronové sítě. Tato práce se zabývá zhodnocením a reimplementací několika vybraných prediktorů určených pro klasifikační konvoluční sítě.
|
|
|
Visualizing Neuroevolution in Neural Network Learning
Bednář, Martin ; Janoušek, Vladimír (referee) ; Zbořil, František (advisor)
This thesis examines options for neural network learning achieved by means of neuroevolution, examines general functioning of neuroevolution, design and implementation of neuroevolution and marginally deals with design and implementation of feed-forward neural networks with fully connected layers. The goal of this thesis is to introduce program, that executes neuroevolutionary algorithm and separate graphic application, which encapsulates this program for easier use and for display of graphic output of the program visualizing problem-solving capabilities of neural networks created by neuroevolution. The end part of the thesis is devoted to experiments done on the created program.
|
|
|
Evolutionary Design of Convolutional Neural Networks
Piňos, Michal ; Vašíček, Zdeněk (referee) ; Sekanina, Lukáš (advisor)
The aim of this work is to design and implement a program for automated design of convolutional neural networks (CNN) with the use of evolutionary computing techniques. From a practical point of view, this approach reduces the requirements for the human factor in the design of CNN architectures, and thus eliminates the tedious and laborious process of manual design. This work utilizes a special form of genetic programming, called Cartesian genetic programming, which uses a graph representation for candidate solution encoding.This technique enables the user to parameterize the CNN search process and focus on architectures, that are interesting from the view of used computational units, accuracy or number of parameters. The proposed approach was tested on the standardized CIFAR-10dataset, which is often used by researchers to compare the performance of their CNNs. The performed experiments showed, that this approach has both research and practical potential and the implemented program opens up new possibilities in automated CNN design.
|
|
|
Evolutionary Algorithms for Neural Networks Learning
Vosol, David ; Rozman, Jaroslav (referee) ; Zbořil, František (advisor)
Main point of this thesis is to find and compare posibilities of cooperation between evolutionary algorithms and neural network learning and their comparison with classical learning technique called backpropagation. This comparison is demonstrated with deep feed-forward neural network which is used for classification tasks. The process of optimalization is via search of optimal values of weights and biases within neural network with fixed topology. We chose three evolutionary approaches. Genetic algorithm, differential evolution and particle swarm optimization algorithm. These three approaches are also compared between each other. The demonstrating program is implemented in Python3 programming language without usage of any third parties libraries focused on deep learning.
|
guest ::
