In order to increase the predictive pmver of electronic structure calculations on atomic and condensed matter systems, this work explores vertex corrections within the framework of many-body perturbation theory. Hedin's GW approximation has, during the last decades, emerged as a powerful technique to calculate single-particle energy excitation levels in both extended and isolated systems, where other current methods usually fail to account properly for screening. GW calculations should in principle be performed in a self-consistent manner, i.e. be taken by several iterations to their solution, starting from a Hartree zeroth-order calculation. If the initial Green's function is constructed from single-particle orbitals with a zeroth-order approximation to the self-energy like DFT, there is in principle a vertex given by the first iteration. This is routinely ignored in standard calculations today, which are typically perfomed with only a single iteration. vVe have investigated the effects of the vertex correction derived from the DFT starting point in single iterations on two systems, the homogenous electron gas, and closed shell atoms. vVe ~nd that a local potential, i.e. depending on the density at one point only, gives a pathological subsequent vertex correction, if it is consistently applied. To'cure these pathologies, we propose nonlocal initial approximations to the starting self-energy. The vertices subsequently derived from these are well-behaved and can capture many physical effects beyond the one-shot G(O)W(O) approach. Finally, we investigate the interplay of these vertices with self-consistency in GW calculations.