The origins of ultra-high-energy cosmic rays (UHECRs) are one of the open puzzles of astrophysics. A number of plausible candidates, such as active galactic nuclei (AGNs) have been discussed, but no clear consensus has been reached. One way to assess the different hypotheses is by analyzing the UHECR arrival directions. Recently, a small number of studies have begun applying Bayesian methodologies to this problem, forming the first steps in the development of a comprehensive Bayesian framework for the study of UHECRs. In this work, we have developed two Bayesian methods to study this question, and have applied them to UHECRs from the Pierre Auger Observatory (PAO). The first method was a Bayesian approach to studying the catalogue-independent clustering of UHECRs. Previously, this had been difficult as there is no well motivated clustered model that can be used in a Bayesian model comparison. We have resolved this difficulty by developing a multi-step approach that derives such a model from a sub-set of the data. This approach could have broad applications for anisotropy searches in other areas of astronomy. Our results were consistent with both isotropic and clustered models. The second was a Bayesian method that was aimed to find associations between UHECR arrival directions and source catalogues. It was an extension of a previous Bayesian study, but analyzed a greater data set, used a more refined UHECR model, and was generalized to be applicable to a greater variety of source catalogues. Our results were broadly consistent with previous work, with the purely isotropic UHECR models being disfavoured for reasonable parameter ranges. It will be of great interest to apply our methods to samples of greater size. The extended UHECR samples that will be available in the near future should be sufficient for our methods to determine the origins of the UHECRs.