This thesis investigates the use of a recirculating optical loop testbed for modelling high speed fibre optic transmission systems and investigating nonlinear effects in optical transmission in wavelength multiplexed (WDM) and optical time division multiplexed (OTDM) systems. The recirculating loop is a device designed to perform distance dependent and cascadability transmission experiments with a limited number of optical amplifiers and a relatively short (compared to the total transmission distance) length of optical fibre. In the first part of the thesis the recirculating loop testbed and its limitations are described. In addition to the standard recirculating loop an advanced dynamically reconfigurable recirculating loop was developed and is described in this thesis; this testbed allows an investigation of the optimal system configuration to be performed by switching the transmitted signal between two devices under test placed in the loop. Next, a number of transmission experiments using the recirculating fibre loop testbed to investigate the impact of nonlinear effects on WDM and OTDM transmission are described. Firstly, the optimum dispersion compensating scheme for 10 Gbit/s 0.4 nm-spaced WDM channels transmitted over standard single-mode fibre compensated with dispersion compensating fibre was investigated. In these experiments pre- and post- compensated schemes were compared and the impact of fibre nonlinearities such as cross-phase modulation and self-phase modulation on the transmitted signal were investigated. To further investigate nonlinear induced spectral broadening a technique incorporating a fibre Fabry-Perot interferometer with the recirculating loop was developed. The 2 pm resolution achieved in the recirculating loop spectral measurements is the highest reported to date. Secondly, the recirculating loop experiments were repeated using a dispersion compensating grating (DCG). The grating was designed to exactly compensate the dispersion of the standard fibre used for previous experiments, and the transmission results for DCG and DCF compensated systems were compared. The advantages and disadvantages of RZ and NRZ formats for both systems were also investigated. To investigate effects related to signal degradation in high speed transmission (20 Gbit/s and higher) the bit rate of the testbed was increased to 40 Gbit/s using the OTDM technique. In this part of the thesis, two types of OTDM pulse sources are compared using the mode locked fibre grating laser and electro-absorption modulators. Finally, 40 Gbit/s OTDM recirculating loop experiments were performed to investigate the impact of intrachannel fibre nonlinearities and broadband dispersion compensation on 40 Gbit/s RZ pulse transmission. It was demonstrated that the alternating polarisation OTDM RZ format allows an increase in the transmission distance by a factor of 1.5 compared to standard OTDM RZ.