Blind Signal Separation (BSS) is a statistical signal processing-based technique and has recently been developed for many potential applications. This thesis aims to investigate model misfits in BSS problems as well as identify and develop efficient solutions for enhancing the performance of signal separation. This research sets out to investigate model misfits associated with finite signal sample size, mixing model, source signal and noise models. The effects of finite signal sample size on several well-known cost functions have been studied and this thesis has identified the most optimal cost function in separating signals with and without the presence of noise. A set of statistical tests is further developed to measure the performance in terms of speed, accuracy and convergence of the tested BSS algorithms. This work further explores the limitations of conventional assumptions of the noiseless and square mixing model which are often violated in practice and result in poor performance in signal separation. The separation of underdetermined mixing models as well as the assumptions of the source signals and noise are also addressed. This thesis presents the development of a Bayesian framework for underdetermined mixtures that produce accurate results in the estimation of mixing matrix and signals corrupted by noise. The proposed algorithm for underdetermined mixtures is capable of modelling a wide variety of signals ranging from unimodal to multimodal and symmetric to nonsymmetric signals. An integrated noise reduction procedure provides robustness against Gaussian noise and the commonly neglected non-Gaussian noise. Results justify the customisation of an algorithm for underdetermined mixtures and demonstrate the efficacy of the proposed algorithm which is three to five times better than existing algorithms. Finally, the work investigates another model misfit in the form of nonlinearly mixed signals and the difficulty of the problem. An algorithm that accurately separates nonlinear mixtures in the presence of noise is proposed. This algorithm features a system that maintains efficient convergence rate while minimising the risk of divergence regardless of the initialised parameters. There is also a mechanism that ameliorates global convergence. Results show that the proposed algorithm outperforms existing algorithms by at least three times with its features that simultaneously address the two crucial issues in the blind separation of nonlinear mixtures.