Agents operating in domains with time budgets shared between planning and execution must carefully balance the need to plan versus the need to act. This is because planning and execution consume the same time resource. Excessive planning can delay the time it takes to achieve a goal, and so reduce the reward attained by an agent. Whereas, insufficient planning will mean the agent creates and executes low reward plans. This thesis looks at three ways to increase the reward achieved by an agent in domains with shared time budgets. The first way is by optimising time allocated to planning, using two different methods -- an optimal plan duration predictor and an online loss limiter. A second is by finding ways to act in a goal-directed manner during planning. We look at using previous plans or new plans generated quickly as heuristics for acting whilst planning. In addition, we present a way of describing actions that are mid-execution to speed the transition between planning and execution. Lastly, this thesis presents a way in which to manage time budgets in multi-agent domains. We use market-based task allocation with deadlines to produce faster task allocation and planning.