Use this URL to cite or link to this record in EThOS:
Title: Reliable confidence measures and well-calibrated probabilistic outputs in classification algorithms
Author: Lamprou, Antonis
ISNI:       0000 0004 8499 0266
Awarding Body: Royal Holloway, University of London
Current Institution: Royal Holloway, University of London
Date of Award: 2016
Availability of Full Text:
Access from EThOS:
Access from Institution:
The Machine Learning research area is widely used in several predictive systems, where observations from the past can be used to create predictions about future events. Machine Learning can be applied to any area where classification or regression is used. Nonetheless, most Machine Learning algorithms do not provide any measures of valid confidence. Conformal Prediction (CP) is a framework which uses underlying Machine Learning algorithms, and can provide valid measures of confidence for predictions. Additionally, the Venn Prediction (VP) framework, which is an extension to the CP framework, provides well-calibrated probabilistic outputs. This thesis explores and provides new methods for valid measures of confidence and probabilistic outputs, based on the Conformal and Venn Prediction frameworks. We introduce a new Conformal Predictor based on Genetic Algorithms and compare our approach with other methods. Additionally, the CP framework is extended for multi-label applications where predictions can contain more than one possible classifications. Furthermore, a new Venn Predictor based on Inductive CP is introduced, which greatly improves the computational eciency of VP. We conduct experiments on our methods and examine their performance and validity. Finally, we examine the applications of osteoporosis risk assessment, the diagnosis of childhood abdominal pain, and the evaluation of the risk of stroke based on ultrasound images of atherosclerotic carotid plaques. Our experimental results on all our methods demonstrate the reliability and usefulness of our conndence and probabilistic outputs.
Supervisor: Not available Sponsor: Not available
Qualification Name: Thesis (Ph.D.) Qualification Level: Doctoral
EThOS ID:  DOI: Not available
Keywords: confidence measures ; conformal prediction ; venn machines ; machine learning