Climate models have substantial biases in the climatological latitude of the Southern Hemisphere eddy‐driven jet and the timescale of annular mode variability, and disagree on the jet response to climate change. Zonally symmetric dry dynamical cores are often used for idealized modelling of the jet response to forcing and its sensitivity to model setup changes. The limits to which these models represent the key mechanisms that control the jet in complex models or the real world have not been systematically investigated. Here, we show that substantial inter‐model differences in jet latitude and strength can arise from differences in dynamical cores and resolved topography. Including topography and a more realistic surface drag in a dry model substantially alters the jet response to changes in drag strength. Using real‐world maps, enhanced drag over land shifts the jet poleward, whereas enhanced drag over the ocean leads to an equatorward shift. No universal relationship between annular mode timescale and forced response emerges in the dry model with topography. These results suggest that zonally symmetric models with Rayleigh drag lack important mechanisms that control the behaviour of the mid‐latitude jet in coupled climate models. A dry model with topography and quadratic surface drag can fill this gap in the model hierarchy.