The thesis addresses various problems arising in parasite population dynamics through the use of mathematical techniques, and in particular of stochastic processes. Parasite diseases generally fall into two categories, those in which a host has a small number of disease classifications, such as susceptible, infected and immune, and those in which the severity of the infection is an important property of the disease. It is the latter type that is addressed in this thesis. Parasite-host interactions are studied via simple nonlinear stochastic processes describing the dynamics of parasites within hosts. In particular, the effect of parasite-induced host mortality and acquired immunity on the distribution of parasite numbers in hosts is considered. Moment closure techniques for approximating nonlinear stochastic processes are investigated for the models, including an assumption based on a new multivariate negative binomial distribution. Approximate results are compared with exact results where obtainable, and results from stochastic simulations elsewhere. Various stochastic models are proposed for the study of between-host parasite dynamics in a population of immortal hosts. Their solutions and properties are evaluated through the use of systems of differential equations, which lead to varying results according to the host population size and the assumptions made concerning the method of parasite transmission. It is suggested that the structure of a disease transmission process, often implicitly ignored in the modelling process, may have a greater effect on the parasite distributions than currently thought. Throughout the thesis examples of diseases in humans and wildlife are given to illustrate the motivation behind the mathematical models and the discussions.