The objective of this thesis is to show how a Bayesian model comparison framework, coupled with the use of a formal mathematical modeling language (ODEs), can assist researchers in the process of modeling dynamic biological systems. The Bayesian approach differs from classical statistics in the way model parameters are treated: our state of knowledge about them can be summarised by probability distributions. All Bayesian inference depends on the data-updated version of these parameter distributions, the posterior densities. Averaging the data likelihood over the posterior results in the model evidence, a measure that very conveniently balances the complexity of a model with the quality of its fit to the data. This is very useful for model comparison. Such a task arises quite often in biological research, where different hypotheses are often available to explain a given phenomenon, and deciding which one is best is difficult. Despite its importance, model suitability is most often judged in an informal way. The main aspects of the Bayesian approach-together with comparisons to classical statistics methods-are described in detail in the first part of this thesis. The most important formalisms for modeling biological systems are also reviewed, and the building blocks of differential equation models are presented. These methods are then applied to a series of synthetic datasets for which the underlying model is known, allowing to illustrate the main features of Bayesian inference. This is followed by the application of the framework to two real systems: a series of network motifs and the Jak/STAT signal transduction pathway. Results show that network motifs are well identifiable given dynamic data and, in the particular case of complex feedforward motif models, the Bayesian framework outperforms the classical methods. The present work also highlights the lack of an appropriate model for the flagella system, and thus a number of novel models are explored. Finally, the Jak/STAT system is analysed. The results are compared to existing models in the literature, and allow discarding some biologically-motivated new models.