Human motion tracking and analysis is currently one of the most active topics in research community. It is driven by a wide range of application domains, such as motion capture for animation, human-computer interaction, security surveillance, and diagnostics for orthopaedic patients. This thesis aims to develop a motion tracking system to track and analyze the upper limb motion of patients who sustain a stroke, to allow rehabilitation exercise in a home environment without the need for the presence of physiotherapists. The first part of this work is focused on developing visual tracking methods for human motion, which should be accurate, cheap, fast and suitable for home based rehabilitation. A proposed method is colour belt based tracking, which makes use of the advantages of both marker and markerless tracking systems. 2D and 3D arm motion are tracked using monocular and stereo views respectively. Tracking performance is promising. The proposed visual tracking methods serve as fundamental units in the second part of this thesis.In the second part of this work, visual and inertial sensors are integrated into a motion tracking system to track human upper limb motion. By combining different sensors into a single system, this compe.nsates for any of their individual shortcomings, thereby improving the system performance. An Extended Kalman Filter (EKF) and. Particle Filter (PF) are used to fuse different data modalities from inertial and visual sensors. Their performances are compared and validated with a further comparison with the ground truth.