After reviewing previously published techniques, a new algorithm is presented for optimising variable parameters in explicitly correlated many-body trial wavefunctions for use in variational quantum Monte Carlo (VMC) and diffusion quantum Monte Carlo (DMC) calculations. The method optimises the parameters with respect to the VMC energy by extending a low-noise VMC implementation of diagonalisation to include parameters which affect the wavefunction to higher than first-order. Similarly to minimising the variance of the local energy by fixed-sampling, accurate results are achieved using a relatively small number of VMC configurations because the optimisation is based on a least-squares fitting procedure. The method is tested by optimising six small examples intended to broadly cover the range of systems and wavefunctions typically treated using VMC and DMC, including atoms, molecules, and extended systems. Least-squares energy minimisation is found to be stable, fast enough to be practical, and capable of achieving lower VMC energies than minimisation of the filtered underweighted variance of the local energy (and the underweighted mean absolute deviation from the median local energy) by fixed-sampling. Least-squares energy minimisation is used to optimise four different wavefunctions for each of the all-electron first row atoms, from lithium to neon: single-determinant Slater-Jastrow wavefunctions with and without backflow transformations, and multi-determinant Slater-Jastrow wavefunctions with and without backflow transformations. The optimisations are more stable and successful than some previous variance minimisations using similar wavefunctions. The DMC energies of the energy-optimised wavefunctions for the atoms from boron to neon are significantly lower than previously published results, and, using the multi-determinant Slater-Jastrow wavefunctions with backflow, the calculations recover at least 90% of the correlation energies for lithium, beryllium, boron, carbon, nitrogen and neon, 97% for oxygen, and 98% for fluorine.