Magnetic confinement fusion reactors suffer severely from heat and particle losses through turbulent transport, which has inspired the construction of ever larger and more expensive reactors. Numerical simulations are vital to their design and operation, but particle collisions are too infrequent for fluid descriptions to be valid. Instead, strongly magnetised fusion plasmas are described by the gyrokinetic equations, a nonlinear integro-differential system for evolving the particle distribution functions in a five-dimensional position and velocity space, and the consequent electromagnetic field. Due to the high dimensionality, simulations of small reactor sections require hundreds of thousands of CPU hours on cutting-edge High Performance Computing platforms. We develop a Hankel-Hermite spectral representation for velocity space that exploits structural features of the particle streaming, gyroaveraging, and collision terms in the gyrokinetic system. This representation exactly conserves a discrete free energy in the absence of explicit dissipation, while our Hermite hypercollision operator captures Landau damping with as few as ten variables. Calculation of the electromagnetic fields also becomes purely local. This eliminates all inter-processor communication in, and hence vastly accelerates, searches for linear instabilities. We implement these ideas in SpectroGK, an efficient parallel code. Turbulent fusion plasmas may dissipate free energy through linear phase mixing to fine scales in velocity space, as in Landau damping, or through a nonlinear cascade to fine scales in physical space, as in hydrodynamic turbulence. Using SpectroGK to study saturated electrostatic drift-kinetic turbulence in Cartesian geometry, we find that the nonlinear cascade completely suppresses linear phase mixing at energetically-dominant scales, so the turbulence is fluid-like. We use these observations to derive Fourier-Hermite spectra for the electrostatic potential and distribution function, and confirm these spectra with SpectroGK simulations.