Mountains with complex terrain and steep environmental gradients are biodiversity hotspots such as the eastern Tibetan Plateau (TP). However, it is generally assumed that mountain terrain plays a secondary role in plant species assembly on a millennial time-scale compared to climate change. Here, we investigate plant richness and community changes during the last 18,000 years at two sites: Lake Naleng and Lake Ximen on the eastern TP with similar elevation and climatic conditions but contrasting terrain. We applied plant DNA metabarcoding to lake sediments leveraging a new regional reference database for taxa identification. Furthermore, we developed a simplified species dispersal model named SMARC. This was used to simulate species migration along river valleys in response to past climate change at the taxonomic resolution of the sedimentary ancient DNA (sedaDNA) approach. Statistical analyses, including ordination-based ecological trajectory analysis, yielded a significant match between sedaDNA and simulated results at single taxon and community levels including certain site-specific differences. Steep terrain downstream of Lake Naleng enhances connectivity to glacial lowland refugia during postglacial warming. In contrast, gentle terrain over long distances implies weak connectivity to the lowland and thus resulted in a strong migration lag at Lake Ximen. Likewise, terrain differences among our sites defined the different connectivity to alpine refugia during late-Holocene cooling. Our consistent proxy- and model-based results, for the first time, indicate that dispersal related migration lags in complex mountain terrain lead to uneven vegetation trajectories at sites with similar climatic conditions mainly because of differences in connectivity to refugia. Ultimately our results indicate that connectivity to refugia is a first-order factor for species migration in addition to elevation-related climatic conditions shaping the postglacial vegetation trajectory in mountainous terrain. This has hitherto largely been ignored when forecasting mountain vegetation responses to climate change and related risk assessment.