Robustness and entropy are widely used concepts in the study of networks in various disciplines. There exist various notions of network robustness and network entropy, because there are many network properties that can be robust to perturbations and there are many network-associated distributions for which one can calculate a notion of entropy. In this dissertation, we explore several notions of robustness and entropy --- and connections between them --- on networks. We investigate the robustness of structural properties of protein-interaction networks to the targeted removal of nodes and propose a test for identifying structural properties that are plausible proxies for measures of performance or viability of a protein-interaction network. Several researchers have suggested that structural redundancy and functional redundancy are important mechanisms by which a biological system can achieve robustness. A connection between redundancy and the mean subsystem entropy of a network with Ornstein--Uhlenbeck dynamics motivates our study of the relationship between the entropy of a multivariate Ornstein--Uhlenbeck process and the structure of an underlying network. We introduce walk graphs as a type of network motif that can help one understand the relationship between network structure and the covariance and entropy of a multivariate Ornstein--Uhlenbeck process. Because of the large number of subsystems in systems with as few as 20 nodes, studying the mean entropy of subsystems computationally is a difficult task. To address this issue, we derive sampling guarantees that indicate that one can calculate a very accurate sample mean of subsystem entropy by sampling very few subsystems. The results in this dissertation offer new approaches to studying performance, viability, and robustness of systems on the basis of network structure and dynamics on networks. They demonstrate the importance of the choice of an appropriate performance measure for the study of network robustness and provide a framework for identifying plausible performance measures or plausible proxies of performance measures. The analysis of walk graphs and their contribution to covariance and entropy demonstrates that it can be useful to consider dynamics on a network (instead of just a network's structure) as a composite entity that one can decompose into many small parts. This decomposition of dynamics on networks provides a framework for studying the mechanisms by which dynamics and network structure contribute to covariance, entropy, redundancy, and other properties of dynamics on networks.