Hand-held robots are an exciting new extension to the toolkit of hand tools at our disposal. They allow the benefits associated with robots, such as precision and task specific knowledge, to be integrated into a form factor that is convenient and cost effective. This work isolates two categories of hand-held robots: reduced degree of freedom robots must rely on the human for at least some degrees of freedom at the end effector; and locally capable robots have the facility to completely decouple user motion of the handle of the robot from that of the end effector. A robot of each variety is designed and built for a set of five degree of freedom tasks, and algorithms for generating end effector trajectories are implemented for both. Both types of hand-held robots rely on a close collaboration with the user to be successful. To facilitate information flow from the robot to the user an Augmented Reality (AR) headset is used. The level of visualisation detail required to form an effective collaboration is investigated. It is found that the level of detail is less important to performance when the robot has control of the interaction, though high fidelity visualisation are important when the user is in control of the integration. Which agent is in control of the interaction at a tactical and operational level is also analysed. Reduced degree of freedom robots must share these levels of planning with the user, and this is found to cause conflict with some users. Complete allocation of these levels to an agent is possible with locally capable robots. No strong evidence of conflict was observed when these roles were more clearly allocated to the user or the robot.