In this thesis, we study a "heuristic approach" that are frequently used for outlier robustness analysis in either the classical or instrumental variables regression. In applied economics, it is a frequent concern whether a tiny set of atypical observations may have invalidated the key empirical findings. To check the robustness of the conclusion especially with respect to outliers, the heuristic approach is to first run least squares regression and remove observations with residuals beyond a chosen cut-off value. Then, re-run regression with selected observations and compare the updated estimate with the original one relative to their standard errors. This procedure can be iterated until the robust result is obtained. The leading purpose of this thesis is to develop asymptotic theory that formally justifies this simple robust procedure. The argument involves a theory of a new class of the weighted and marked empirical processes of residuals. Asymptotics are derived under the null hypothesis that there is no data contamination.