Wittgenstein’s later philosophy of mathematics has been widely interpreted to involve Wittgenstein’s making dogmatic requirements of what can and cannot be mathematics, as well as involving Wittgenstein dismissing whole areas (e.g. set theory) as not legitimate mathematics. Given that Wittgenstein promised to ‘leave mathematics as it is’, Wittgenstein is left looking either hypocritical or confused. This thesis will argue that Wittgenstein can be read as true to his promise to ‘leave mathematics as it is’ and that Wittgenstein can be seen to present coherent, careful and non-dogmatic treatments of philosophical problems in relation to mathematics. If Wittgenstein’s conception of philosophy is understood in sufficient detail, then it is possible to lift the appearance of confusion and contradiction in his work on mathematics. Whilst apparently dogmatic and sweeping claims figure in Wittgenstein’s writing, they figure only as pictures to be compared against language-use and not as definitive accounts (which would claim exclusive right to correctness). Wittgenstein emphasises the importance of the applications of mathematics and he feels that our inclination to overlook the connections of mathematics with its applications is a key source of a number of philosophical problems in relation to mathematics. Wittgenstein does not emphasise applications to the exclusion of all else or insist that nothing is mathematics unless it has direct applications. Wittgenstein does question the alleged importance of certain non-applied mathematical systems such as set theory and the logicist systems of Frege and Russell. But his criticism is confined to the aspirations towards philosophical insight that has been attributed to those systems. This is consonant with Wittgenstein’s promises in (PI, §124) to ‘leave mathematics as it is’ and to see ‘leading problems of mathematical logic’ as ‘mathematical problems like any other.’ It is the aim of this thesis to see precisely what Wittgenstein means by these promises and how he goes about keeping them.