Knowledge on natural climate variability is essential for making informed projections about future climate change. Yet, as highlighted by previous studies using paleoclimate reconstructions, climate models (CMs) often underestimate surface temperature variability on multidecadal to millennial timescales, both in the ocean and on land. Here, we enhance the long-term memory of sea surface temperature (SST) in the Alfred Wegener Institute Earth System Model (ESM) by nudging SSTs towards modified values based on estimates from marine paleoclimate archives. SSTs are nudged, while other dynamics and thermodynamics evolve freely, meaning the atmosphere and sea ice adjust to the altered SST variability. We find that enhanced long-term ocean memory leads to increased atmospheric temperature variability, while maintaining realistic decadal to interannual climate variability modes and mean climate state. It further brings model output into closer agreement with ice core–derived temperature reconstructions from Greenland and East Antarctica. These results support the hypothesis that ocean variability shapes long-term land temperature fluctuations and underscores the need for improved representation of long-term ocean memory in CMs and ESMs. Our framework enables exploration of long-term natural variability, likely underestimated by current ESMs, with applications from paleo-data assimilation to detection and attribution and future climate-risk assessment.