Use this URL to cite or link to this record in EThOS: http://ethos.bl.uk/OrderDetails.do?uin=uk.bl.ethos.728937
Title: Introspective classification for robot perception and decision making
Author: Grimmett, Hugo
Awarding Body: University of Oxford
Current Institution: University of Oxford
Date of Award: 2016
Availability of Full Text:
Access from EThOS:
Full text unavailable from EThOS. Restricted access.
Access from Institution:
Abstract:
In robotics, a classifier is often a core component of the decision-making framework. Precision and recall have been widely adopted as canonical metrics to quantify the performance of a classifier, but for applications involving mission-critical decision making, good performance in relation to these metrics is insufficient. The use of a classification framework which produces scores with inappropriate confidences will ultimately lead to the robot making bad decisions, thereby compromising robot or user safety. In order to select a classifier which will make decisions reflecting the nature of the costs, we should pay careful attention to the ways in which it generates scores. We introduce and motivate the importance of a classifier's introspective capacity: the ability to give an appropriate assessment of confidence with any test case. Classification made confidently must be correct, and mistakes should be made with high uncertainty. A classifier's capacity to do so must remain consistent despite unusual or surprising test cases. We propose that a key ingredient for introspection is a classifier's potential to increase its uncertainty with the distance between a test datum and its training data. We define the ideal introspective behaviour, and derive idealised classifiers which serve to benchmark a number of commonly used classification frameworks in a variety of decision-making tasks. We show that classifiers that offer predictive variance at test-time are more cautious and less over-confident than those which consider a single hypothesis or discriminant. However, in high-cost (or high-risk) decision making, none of the classifiers evaluated in this thesis are sufficiently introspective to prevent all potential catastrophic mistakes. We show that in sequential decision-making, when the mapping from score to class is explicitly stated, a classifier's ability to behave consistently despite non-stationary test data is of primary importance.
Supervisor: Posner, Ingmar ; Newman, Paul Sponsor: European Community's Seventh Framework Programme: VCharge
Qualification Name: Thesis (Ph.D.) Qualification Level: Doctoral
EThOS ID: uk.bl.ethos.728937  DOI: Not available
Share: