Cenozoic climate changes have been linked to tectonic activity and variations in atmospheric CO2 concentrations. Here, we present Miocene and Pliocene sensitivity experiments performed with the climate model COSMOS. The experiments contain changes with respect to paleogeography, ocean gateway configuration, and atmospheric CO2 concentrations, as well as a range of vertical mixing coefficients in the ocean. For the mid-Miocene, we show that the impact of ocean mixing on surface temperature is comparable to the effect of the possible range in reconstructed CO2 concentrations. In combination with stronger vertical mixing, relatively moderate CO2 concentrations of 450 ppmv enable global-mean surface, deep-water, and meridional temperature characteristics representative of mid-Miocene Climatic Optimum (MMCO) reconstructions. The Miocene climate shows a reduced meridional temperature gradient and reduced seasonality. In the case of enhanced mixing, surface and deep ocean temperatures show significant warming of up to 5–10°C and an Arctic temperature anomaly of >12°C. In the Pliocene simulations, the impact of vertical mixing and CO2 is less important for the deep ocean, which we interpret as a different sensitivity dependence on the background state and mixed layer dynamics. We find a significant reduction in surface albedo and effective emissivity for either a high level of atmospheric CO2 or increased vertical mixing. Our mixing sensitivity experiments provide a warm deep ocean via ocean heat uptake. We propose that the mixing hypothesis can be tested by reconstructions of the thermocline and seasonal paleoclimate data indicating a lower seasonality relative to today.