Arctic microbial communities (i.e., protists and bacteria) are increasingly subjected to an intrusion of new species via Atlantification and an uncertain degree of ocean warming. As species differ in adaptive traits, these oceanic conditions may lead to compositional changes with functional implications for the ecosystem. In June 2021, we incubated water from the western Fram Strait at three temperatures (2 °C, 6 °C, and 9 °C), mimicking the current and potential future properties of the Arctic Ocean. Our results show that increasing the temperature to 6 °C only minorly affects the community, while an increase to 9 °C significantly lowers the diversity and shifts the composition. A higher relative abundance of large hetero- and mixotrophic protists was observed at 2 °C and 6 °C compared to a higher abundance of intermediate-sized temperate diatoms at 9 °C. The compositional differences at 9 °C led to a higher chlorophyll a:POC ratio, but the C:N ratio remained similar. Our results contradict the common assumption that smaller organisms and heterotrophs are favored under warming and strongly indicate a thermal limit between 6 °C and 9 °C for many Arctic species. Consequently, the magnitude of temperature increase is a crucial factor for microbial community reorganization and the ensuing ecological consequences in the future Arctic Ocean.