Higher order Laguerre-Gauss mode degeneracy in realistic, high finesse cavities

Higher order Laguerre-Gauss (LG) beams have been proposed for use in future gravitational wave detectors, such as upgrades to the Advanced LIGO detectors and the Einstein Telescope, for their potential to reduce the effects of the thermal noise of the test masses. This paper details the theoretical analysis and simulation work carried out to investigate the behavior of LG beams in realistic optical setups, in particular, the coupling between different LG modes in a linear cavity. We present a new analytical approximation to compute the coupling between modes, using Zernike polynomials to describe mirror surface distortions. We apply this method in a study of the behavior of the LG(33) mode within realistic arm cavities, using measured mirror surface maps from the Advanced LIGO project. We show mode distortions that can be expected to arise due to the degeneracy of higher order spatial modes within such cavities and relate this to the theoretical analysis. Finally, we identify the mirror distortions which cause significant coupling from the LG(33) mode into other order 9 modes and derive requirements for the mirror surfaces.