Financial time series analysis is a highly empirical discipline concerned with the evolution of the price of an asset. The key feature that distinguishes financial time series from time series of other scientific domains is the element of uncertainty that they contain. The recent financial crisis has tested the capabilities of several existing models and evidenced the need for methods able to deal with the high complexity and the non-stationary characteristics of the data observed in financial markets. The objective of this thesis is to provide a better understanding of financial time series, to enhance the abilities of existing methods, especially their predictive performance but also to develop novel methods which aim to provide inferences in the presence of non-stationarities and reduce the complexity of high dimensional tasks. To this end, the memory in the magnitude and the memory in the sign of logarithmic returns is studied and a novel model is constructed whose fit suggests that long memory might be present in the volatility process and that when memory in the sign increases so does the memory in the magnitude. Additionally, wavelets are employed for that they operate in both the time and frequency domains. Thus, classic time series models and other methods extensively used in the time domain are deployed across different frequency bands to combine knowledge from both domains and provide information that might not be accessible otherwise. In particular, the volatility process is modeled in the time domain after some of the noisy behavior that exists in high frequencies, which might also contain outliers, is neglected. Moreover, the volatility process is modeled directly in the wavelet domain in a scale-by- scale manner in an effort to improve the forecasting performance. Furthermore, we attempt to detect changes in the autocorrelation function of a process, which result in changes in the spectral density function, by monitoring the wavelet variance across different multiresolution scales.