This thesis deals with the stability analysis of linear discrete-time premium-reserve (P-R) systems in a stochastic framework. Such systems are characterised by a mixture of the premium pricing process and the medium- and long- term stability in the accumulated reserve (surplus) policy, and they play a key role in the modern actuarial literature. Although the mathematical and practical analysis of P-R systems is well studied and motivated, their stability properties have not been studied thoughtfully and they are restricted in a deterministic framework. In Engineering, during the last three decades, many useful techniques are developed in linear robust control theory. This thesis is the first attempt to use some useful tools from linear robust control theory in order to analyze the stability of these classical insurance systems. Analytically, in this thesis, P-R systems are first formulated with structural properties such that time-varying delays, random disturbance and parameter uncertainties. Then as an extension of the previous literature, the results of stabilization and the robust H-infinity control of P-R systems are modelled in stochastic framework. Meanwhile, the risky investment impact on the P-R system stability condition is shown. In this approach, the potential effects from changes in insurer's investment strategy is discussed. Next we develop regime switching P-R systems to describe the abrupt structural changes in the economic fundamentals as well as the periodic switches in the parameters. The results for the regime switching P-R system are illustrated by means of two different approaches: markovian and arbitrary regime switching systems. Finally, we show how robust guaranteed cost control could be implemented to solve an optimal insurance problem. In each chapter, Linear Matrix Inequality (LMI) sufficient conditions are derived to solve the proposed sub-problems and numerical examples are given to illustrate the applicability of the theoretical findings.