We consider the computational power of classes of program schemes obtained from extending NPS(1) (program schemes of NPS(1) consist of neurodeterministic while programs with quantifier-free tests, which take as input some finite structure over some fixed signature) by: incorporating built-in relations from successor, linear-order, addition, multiplication; allowing universal quantification; and providing access to a stack. We are motivated by known schematic characterizations, due to Stewart, of the complexity classes NL and PSPACE involving NPS(1) extended with a built-in successor relation and the presence of arrays. We completely classify the relative computational power of the class of program schemes NPS(1) extended with the aforementioned built-in relations, as we do for the relative descriptive power of similar extension of the bounded-variable infinitary logic L^ω_∞ω and its existential fragment. Our proofs involve playing (infinitary) pebble games. NPS(1) is the first class in an infinite hierarchy of classes of program schemes NPS built by interleaving nondeterminism with universal quantification. The class of problems accepted by NPS equates to that defined by the sentences of transitive closure logic. We show that this hierarchy is proper and so obtain an analogous hierarchy result in transitive closure logic. Interestingly we do not use Ehrenfeucht-Fraïssé Games for our inexpressibility results, as is usually the case in finite model theory, but consider computations of program schemes on certain finite structures. We extend NPS with a stack to obtain the NPSS hierarchy; characterize NPSS with path system logic; and obtain an analogous hierarchy in path system logic. We show that there exists a deterministic program scheme with access to a stack that can solve P-complete problem that is over a functional signature. Our proofs concerning programs schemes with stacks lean heavily on Cook's proof that the classes of formal languages accepted by deterministic and nondeterministic pushdown automata coincide.