The gradual cooling of the climate during the Cenozoic has generally been attributed to a decrease in CO2 concentration in the atmosphere. The lack of transient climate models and in particular the lack of high-resolution proxy records of CO2, beyond the ice-core record prohibit however a full understanding of for example the inception of the Northern Hemisphere glaciation and mid-Pleistocene transition. Here we elaborate on an inverse modelling technique to reconstruct a continuous CO2 series over the past 20 million year (Myr), by decomposing the global deep-sea benthic d18O record into a mutually consistent temperature and sea level record, using a set of 1-D models of the major Northern and Southern Hemisphere ice sheets. We subsequently compared the modelled temperature record with ice core and proxy-derived CO2 data to create a continuous CO2 reconstruction over the past 20 Myr. Results show a gradual decline from 450 ppmv around 15 Myr ago to 225 ppmv for mean conditions of the glacial-interglacial cycles of the last 1 Myr, coinciding with a gradual cooling of the global surface temperature of 10 K. Between 13 to 3 Myr ago there is no long-term sea level variation caused by ice-volume changes. We find no evidence for a change in the long-term relation between temperature change and CO2, other than the effect following from the saturation of the absorption bands for CO2. The reconstructed CO2 record shows that the Northern Hemisphere glaciation starts once the long-term average CO2 concentration drops below 265 ppmv after a period of strong decrease in CO2. Finally, only a small long-term decline of 23 ppmv is found during the mid-Pleistocene transition, constraining theories on this major transition in the climate system. The approach is not accurate enough to revise current ideas about climate sensitivity.