This thesis studies the connection between the limiting distributions of rational points on horocyle flows and the value distribution of incomplete Gauss sums. A key property of the horocycle flow on a finite-area hyperbolic surface is that long closed horocycles are uniformly distributed. In this thesis we embed rational points on such horocycles on the modular surface and investigate their equidistribution properties. We later extend this study to the metaplectic cover of the modular surface. On the other hand, it is well known that the classical Gauss sums can be evaluated in closed form depending on the residue class of the number of terms in the sum modulo 4. This is not the case for the incomplete Gauss sums, where we restrict the range of summation to a sub-interval (both long and short relative to the complete sums) and study their limiting behavior at random argument as the number of terms goes to infinity. The main ingredient in the proof is the equidistribution of rationals on metaplectic horocyles mentioned above. We also establish an analogue of the weak invariance principle for incomplete Gauss sums.