Downcore sediment grain-size records of mineral dust (2–10 μm) can provide key insights into changes in wind strength and source-area characteristics over glacial-interglacial timescales. However, so far, little is known about glacial-interglacial changes of dust grain size in the open Southern Ocean, which are potentially associated with changes in the strength and position of the southern westerly winds. Here, we analyzed the grain-size distributions of subantarctic deep-sea sediments from the Pacific (PS75/056–1) and Atlantic (ODP Site 1090) sectors of the Southern Ocean, downwind of the major Southern Hemisphere dust source regions. Dust mean grain sizes show opposite trends in the two Southern Ocean sectors. Larger glacial grain sizes are observed in the Pacific sector, while finer glacial grain sizes are observed in the Atlantic sector. In the South Pacific, larger mean dust grain sizes parallel higher Fe fluxes during glacials. In contrast, in the South Atlantic record increased glacial Fe fluxes coincide with a decrease in glacial mean dust grain sizes consistent with some Antarctic ice core records. Our results suggest that the opposing grain-size trends are the result of different responses to glacial conditions in the sources and of changing wind and transport patterns. For the South Pacific, a possible explanation of our results could be an intensification of wind strength over Australia enabling emission of larger dust particles. This strengthening would imply a northward shift of the westerlies which facilitated the transport of dust from enhanced and/or more Australian and New Zealand sources. For the Atlantic, the decreased glacial dust grain size could be the consequence of increased glacial activity in the Patagonian Andes, generating and supplying more and finer-grained dust from the exposed continental shelf to the South Atlantic. These findings indicate that more extensive studies of wind-blown sediment properties in the Southern Ocean can provide important insights on the timing and latitudinal extent of climatic changes in the sources and variations of transport to the Southern Ocean by the westerly winds.