Use this URL to cite or link to this record in EThOS:
Title: Adaptive Bayesian optimization for dynamic problems
Author: Nyikosa, Favour Mandanji
ISNI:       0000 0004 7966 2727
Awarding Body: University of Oxford
Current Institution: University of Oxford
Date of Award: 2018
Availability of Full Text:
Access from EThOS:
Full text unavailable from EThOS. Please try the link below.
Access from Institution:
This thesis studies the problem of tracking the extremum of an objective function that is latent, noisy and expensive to evaluate. This problem is notable because many large-scale learning systems with complex models operating on non-stationary data have meta-problems whose solutions require the tracking of an evolving extremum. We start by describing dynamic optimization problems and model them using spatiotemporal Gaussian process priors. We construct an intelligent search mechanism that uses the learnt insights to skillfully guide the search by dynamically modifying the feasible search region as a device to keep track of the evolution. We also show that this mechanism induces a natural approximation scheme for cases where the number of samples for the model becomes too expensive for inference. We test the resulting method on synthetic and real-world problems. In the next part of the thesis, we demonstrate the utility of the method on pertinent real-world meta-problems occurring in essential and widely-used large-scale learning systems. We start by proposing intelligent meta-heuristics, which are derived from our resulting method, for the learning rate adaptation meta-problem that is part of many large-scale learning systems trained by stochastic gradient descent. We incorporate hyper-gradients into the model for the meta-problem and use the additional insights to aid the intelligent meta-heuristic search. We test the resulting adaptation scheme on training various types of neural network models to demonstrate its utility through the significant performance improvements. We also exploit the wealth of prior art to construct a configuration oracle that adapts various types of system parameters in online trading algorithms as they interact with noisy and nonstationary financial data. We test the oracle on multiple algorithms operating using widely-accepted trading principles and demonstrate notable performance gains. The importance of this study is that it proposes a mechanism for intelli- gently exploiting vast amounts of prior knowledge that exists for these meta- problems. The intelligent use of this knowledge significantly improves the performance of the primary learning systems, which are vital for solving many practical problems, operating in dynamic environments.
Supervisor: Roberts, Stephen ; Osborne, Michael Sponsor: Not available
Qualification Name: Thesis (Ph.D.) Qualification Level: Doctoral
EThOS ID:  DOI: Not available