This thesis is dedicated to studying cosmological inflation, which is a period of accelerated expansion in the very early Universe that is required to explain the observed anisotropies in the cosmic microwave background. Inflation, when combined with quantum mechanics, also provides the over-densities that grow into the structure of the modern Universe. Understanding perturbations during this period of inflation is important, and we study these perturbations in detail in this work. We will assume that inflation is driven by a single scalar field, called the inflaton. When the shape of the potential energy is flat, the inflaton can enter a phase of “ultraslow-roll inflation”. We study the stability of such a period of inflation, and find that it can be stable and long-lived, although it has a dependence on the initial velocity of the inflaton field. This is different to the slow-roll regime of inflation, which is always stable, but has no dependence on the initial velocity. In the second part of this thesis, we use the stochastic formalism for inflation in order to take account of the non-perturbative backreaction of quantum fluctuations during inflation. This formalism is an effective field theory for long wavelength parts of quantum fields during inflation, and hence is only valid on large scales. We use this formalism to study curvature fluctuations during inflation, and we derive full probability distributions of these fluctuations. This allows us to study the statistics of large fluctuations that can lead to the formation of rare objects, such as primordial black holes. In general, we find that when the quantum effects modelled by the stochastic formalism are correctly accounted for, many more primordial black holes can be formed than one would expect if these quantum effects were not taken into account. We finish by summarising our results and discussing future research directions that have opened up as a result of the work we have done. In particular, we mention future applications of the formalisms we develop using the stochastic techniques for inflation, and note that their applications can be broader than primordial black holes and they can be used to, for example, study other rare objects.