Alkalinity generation from rock weathering modulates Earth’s climate at geological time scales. Although lithology is thought to dominantly control alkalinity generation globally, the role of other first-order controls appears elusive. Particularly challenging remains the discrimination of climatic and erosional influences. Based on global observations, here we uncover the role of erosion rate in governing riverine alkalinity, accompanied by areal proportion of carbonate, mean annual temperature, catchment area, and soil regolith thickness. We show that the weathering flux to the ocean will be significantly altered by climate warming as early as 2100, by up to 68% depending on the environmental conditions, constituting a sudden feedback of ocean CO2 sequestration to climate. Interestingly, warming under a low-emissions scenario will reduce terrestrial alkalinity flux from mid-latitudes (–1.6 t(bicarbonate) a−1 km−2) until the end of the century, resulting in a reduction in CO2 sequestration, but an increase (+0.5 t(bicarbonate) a−1 km−2) from mid-latitudes is likely under a high-emissions scenario, yielding an additional CO2 sink.