We study orthogonal modular varieties associated with the moduli of generalised Kummer manifolds. We are particularly interested in understanding the singularities that arise in certain toroidal compactifications. Throughout, we place particular emphasis on the application of these results to problems involving the birational classification of moduli spaces.