This thesis is naturally broken down into two main problems, one concerning optimal routing control and the other optimal service control. In the routing control problem the arriving customers must be allocated to one of the ‘K’ possible service stations. We assume that the customers arrive in a single Poisson stream. We take the service at each of the stations to be exponentially distributed, but perhaps with different parameters. The system cost rate is additive across the queues formed at each station. We also have that at each station the holding cost function is increasing convex. Following Whittle’s approach to a class of restless bandit problems, we develop a Lagrangian relaxation of the routing control problem which serves to motivate the development of index heuristics. The index by a particular station is characterised as a fair charge for rejecting the arriving customer at that station. We also consider a policy improvement index for comparison to the heuristic. We develop these indices and report an extensive numerical investigation which exhibits strong performance of the index heuristic for both discounted and average costs. The second problem concerns the optimal service control of a multi-class M/G/1 queuing system in which customers are served non pre-emptively. The system cost rate is additive across classes and increasing convex in the numbers present with each class. We again follow the method prescribed by Whittle when considering a class of restless bandits. Hence we develop a Lagrangian relaxation of the service control problem which motivates the development of a class of index heuristics. For a particular customer class the index is characterised as a fair charge for service of that class. These indices are developed and we again report representative results from an extensive numerical study which again implies a strong performance of the index heuristic for both discounted and average costs.