In this thesis, a bound is constructed for the value of the convergence coefficient and penalty parameter. This will give conditions for convergence and hence enable more accurate and faster converging adaptive algorithms using the penalty method. The penalty is implemented by using the function {max[cᵢ, 0]}, which is computationally efficient. The theoretical bound on the values for the adaptive LMS and FDLMS algorithms with quadratic constraints is established. The performance is significantly improved by using the new established bound as shown by simulation. In the case of using the second order penalty function max(cᵢ, 0)², the quadratic constraints can be approximated using linear constraints. Using these linear constraints rather than the original quadratic constraints will mean that the new objective function will be quadratic, so that the new bound defined can also be suitable for the second order penalty function. Simulation showed that using linear constraints with a second order penalty function formulation converged to a minimum that was very close to the quadratically constrained case. Increasing the convergence rate of the FDLMS algorithm can also be achieved by incorporating the Bin-normalised FDLMS and the conjugate gradient (CG) algorithms. Efficient constrained BN-FDLMS and CG algorithms implemented in the frequency domain are developed and studied. Their performance showed that they have superior convergence rate compared with the constrained FDLMS. The new constrained adaptive filtering algorithms studied and developed here were applied to the problem of sound equalisation in an enclosed room. It is also shown that spatial robustness can be further improved by using frequency-dependent constraints, by utilizing the structure of the spatial correlation in a reverberant sound field. A simulation study of the frequency-dependent constraints verifies the applicability of the new algorithms.