Mitigating differential limb coupling as an RV error term in diffraction-limited spectrographs on large telescopes

Diffraction-limited spectrographs are key instruments for upcoming large telescopes thanks to their compact sizes that do not scale with telescope diameter and stable point spread functions (PSFs) that are conducive to precise radial velocity (RVs) measurements. A challenge to achieving sub meter-per-second RVs with diffraction-limited spectrographs that does not show up in the RV error budgets of their seeing-limited counterparts, is “differential limb coupling”, which we define as a preferential coupling of one side of a star’s limb over the other. This effect can result in a more blue- or red-shifted stellar spectrum being coupled into a single-mode fiber due imperfect centroiding of the star on the fiber that differs between each RV measurement. This effect is exacerbated when the star is faster rotating and more resolved by an optical system, and can become significant when coupling to single-mode fibers due to their sensitivity to source position on the fiber. In this paper we quantify the RV amplitude of stellar differential limb coupling, and show that the effect can have a significant impact on next generation adaptive-optics (AO) fed spectrographs on large aperture telescopes if uncorrected. We apply these calculations to HISPEC on Keck and MODHIS on TMT and estimate that for 1 mas pointing errors, the RV error term for a star 0.5 mas in diameter rotating at 2 kms⁻¹ would be 1 ms⁻¹and 7.7 ms⁻¹, respectively. We present several design strategies that we expect would reduce the differential limb coupling effect to varying degrees.