Every cell contains several millions of diffusing and reacting biological molecules. The interactions between these molecules ultimately manifest themselves in all aspects of life, from the smallest bacterium to the largest whale. One of the greatest open scientific challenges is to understand how the microscopic chemistry determines the macroscopic biology. Key to this challenge is the development of mathematical and computational models of biochemistry with molecule-level detail, but which are sufficiently coarse to enable the study of large systems at the cell or organism scale. Two such models are in common usage: the reaction-diffusion master equation, and Brownian dynamics. These models are utterly different in both their history and in their approaches to chemical reactions and diffusion, but they both seek to address the same reaction-diffusion question. Here we make an in-depth study into the physical validity of these models under various biological conditions, determining when they can reliably be used. Taking each model in turn, we propose modifications to the models to better model the realities of the cellular environment, and to enable more efficient computational implementations. We use the models to make predictions about how and why cells behave the way they do, from mechanisms of self-organisation to noise reduction. We conclude that both models are extremely powerful tools for clarifying the details of the mysterious relationship between chemistry and biology.