|
|
Transferring Improved Local Kernel Design in Multi-Source Bayesian Transfer Learning, with an application in Air Pollution Monitoring in India
Nugent, Sh. ; Quinn, Anthony
Existing frameworks for multi-task learning [1],[2] often rely on completely modelled relationships between tasks, which may not be available. Recent work [3], [4] has been undertaken on approaches to fully probabilistic methods for transfer learning between two Gaussian Process (GP) tasks. There, the target algorithm accepts source knowledge in the form of a probabilistic prior from a source algorithm, without requiring the target to model their interaction with the source. These strategies have offered robust improvements on current state of the art algorithms, such as the Intrinsic Coregionalization Model. The Bayesian Transfer Learning algorithm proposed in [4], was found to provide robust, positive\ntransfer. This algorithm was then extended to accommodate knowledge transfer from multiple source modellers [5]. Improved predictive performance was observed from increases in the number of sources. This report reviews the multi-source transfer findings in [5] and applies it to a real world problem of pollution modelling in India, using public-domain data.
|
|
|
Bayesian Optimization of Hyperparameters Using Gaussian Processes
Arnold, Jakub ; Straka, Milan (advisor) ; Vomlelová, Marta (referee)
The goal of this thesis was to implement a practical tool for optimizing hy- perparameters of neural networks using Bayesian optimization. We show the theoretical foundations of Bayesian optimization, including the necessary math- ematical background for Gaussian Process regression, and some extensions to Bayesian optimization. In order to evaluate the performance of Bayesian op- timization, we performed multiple real-world experiments with different neural network architectures. In our comparison to a random search, Bayesian opti- mization usually obtained a higher objective function value, and achieved lower variance in repeated experiments. Furthermore, in three out of four experi- ments, the hyperparameters discovered by Bayesian optimization outperformed the manually designed ones. We also show how the underlying Gaussian Process regression can be a useful tool for visualizing the effects of each hyperparameter, as well as possible relationships between multiple hyperparameters. 1
|
|
| |
|
| |
guest ::
