Feasibility of near-unstable cavities for future gravitational wave detectors

Near-unstable cavities have been proposed as an enabling technology for future gravitational wave detectors, as their compact structure and large beam spots can reduce the coating thermal noise of the interferometer. We present a tabletop experiment investigating the behavior of an optical cavity as it is parametrically pushed to geometrical instability. We report on the observed degeneracies of the cavity's eigenmodes as the cavity becomes unstable and the resonance conditions become hyper-sensitive to mirror surface imperfections. A simple model of the cavity and precise measurements of the resonant frequencies allow us to characterize the stability of the cavity and give an estimate of the mirror astigmatism. The significance of these results for gravitational wave detectors is discussed, and avenues for further research are suggested.