The cross section for the photoproduction of events containing three high transverse energy jets with a three-jet invariant mass of M3J > 50 GeV has been measured with ZEUS at HERA. The data are compared with two different types of QCD prediction; O(alphaalpha2S) pQCD calculations which are leading order for the process under consideration and parton shower models which resum leading logarithms to produce multi-parton final states. The magnitude of the three-jet cross section is in good agreement with the O(alphaalpha2S) pQCD calculations; the parton shower models underestimate the cross section by 20 to 40%. The angular distribution of the three jets is inconsistent with a uniform population of the available phase space. In contrast, parton shower models and O(alphaalpha2S) pQCD calculations are able to describe the three-jet dynamics. Within the parton shower model the relative contribution to three-jet production from initial and final state radiation has been studied and the effect of colour coherence on the angular distribution is demonstrated for the first time.