Software testing is an important activity to verify and validate a system. A test criterion is a set of rules that are used for assessing the system under test. The effectiveness of a test criterion is measured in terms of its ability to reveal faults. Two key issues in software testing are: (a) effectiveness of test criteria in detecting faults, and (b) minimisation of test effort. These issues are studied empirically and formally within the framework of fault domain and test hypothesis. Typically, for a given test criterion, more than one test set may satisfy the criterion for a specification and implementation. A new notion of stability of test criteria is defined to assess the variation in effectiveness of test sets for a given test criterion. Experimental evaluation of stability is performed for various types of coverage such as condition coverage, decision condition coverage, full-predicate coverage, modified condition decision coverage and reinforced condition decision coverage. Fault detection effectiveness is also studied using a formal framework, which is applied to identify the conditions for the detection of various fault classes. It is shown that the number of test cases in a test set that can detect all hypothesised faults in an implementation depends on the complexity of the specification. One of the main difficulties with the fault-based testing approach is due to the large number of possible elements in a fault domain or that can be generated on the basis of a test hypothesis. To overcome this problem, various conditions to establish fault hierarchies that help to identify stronger faults are described. Here, the objective is to establish the relationship between faults such that detection of one fault guarantees the detection of another. The analysis of fault hierarchies is also shown to be useful in validating the coupling hypothesis, which states that if a test technique can detect an implementation with one fault, it can also detect the presence of multiple faults. The results obtained from the empirical and formal analysis provide an insight into earlier contradictory results regarding the effectiveness of test criteria. The formal framework helps in the classification of specifications and implementations in order to evaluate the effort required for testing; and the concept of fault hierarchy is useful in reducing test effort.