Optical imaging is becoming a method of choice in applications where high resolution images are required non-invasively. Optical imaging technologies are capable of representing the internal structure of the sample investigated across a range of spatial scales. In this project, we worked on two optical imaging systems, including the optical coherence tomography (OCT) and confocal microscopy (CM). Similar to every other imaging system, optical imaging systems have limitations mainly due to scattering and noise. Four separate limitation factors of imaging, including the speckle noise, intensity decay due to tissue absorption, aberrations, and point spread function (PSF) distortion are investigated in this thesis and a number of algorithms are devised to reduce their impact so an enhanced image is achieved. The hardware of the imaging systems is also modified to improve their performances. We have developed two speckle reduction algorithms based on artificial neural network (ANN) and temporal compounding methods. The algorithms are tested successfully on varieties of skin images, retina, larynx, human teeth and also drosophila images with the view to improve the signal-to-noise ratio (SNR) and contrast. An attenuation compensation algorithm is designed based on Beer- Lambert law using a novel skin layer detection method. The algorithm is successfully tested on in-vivo OCT skin images of human fingertip. For aberration correction, a sensor-less adaptive optics system is studied along with a blind optimization algorithm. Three optimization algorithms are tested effectively on a CM system; simulated annealing, genetic algorithm and particle swarm optimization. To eliminate the effect of the distortion of the PSF of the OCT system, a deconvolution technique with Lucy-Richardson algorithm is used. The PSF of the OCT system is estimated from images of a specially designed phantom. The algorithm is successfully evaluated using OCT images of healthy tissue including dorsal skin of hand, basaloid eyelid skin, skin of fingertip, and basaloid larynx tissues. Compared to the original images, the improved images are less blurred with higher contrast. We improve a dynamic focus (DF-) OCT system operated at 830 nm to be able to image at 1300 nm wavelength. Different tissues of skin, larynx, eyelid, and several phantoms are imaged by this system.