Non-stationary learning (NSL) refers to the process that can learn rules from data, adapt to shifts, and improve the performance of the system with its experience while operating in the non-stationary environments (NSE). While data processing in NSE, a covariate shift is a major challenge wherein the input data distribution may shift during transitioning from training to testing phase. Covariate shift is one of the fundamental challenges in electroencephalogram (EEG) based brain-computer interface (BCI) systems and these can be often found during multiple trials over different sessions of EEG data recording. Due to these covariate shifts, low performance in terms of classification accuracy has been a confounding factor of conventional BCI systems for motor imagery detection. This research proposes three different steps to designing a novel framework of adaptive learning for modelling non-stationary systems. Firstly, a covariate shift detection (CSD) test has been designed based on an exponentially weighted moving average (EWMA) control chart. The CSD test is a fully data-driven method, and it does not require any assumption on the data distribution to detect the covariate shift. Secondly, transductive-inductive learning based covariate shift adaptation (CSA) algorithms have been proposed, which are based on active and passive approaches to non-stationary learning. To estimate the effectiveness of the proposed adaptive algorithms, extensive experiments have been performed on both synthetic and EEG datasets. The proposed methods are benchmarked against the state-of-the-art methods. In this way, the resulting system utilizes unlabelled data for both the CSD and classifier adaptation purposes and correspondingly implements motor imagery-related classification of single-trial EEG. Lastly, an online active approach based CSA algorithm has been proposed for the non- stationary adaptation in EEG signals for neuro-rehabilitation systems. The online BCI paradigm has been tested on 21 healthy subjects. The result for the online adaptive BCI system has shown a statistically significant improvement over the non-adaptive BCI system. The research contributions collectively provide an efficient method for accounting non-stationarity in data during learning in NSEs.