Microbial communities of the Arctic Ocean are poorly characterized in comparison to other aquatic environments as to their horizontal, vertical and temporal turnover. However, the Arctic marine ecosystem harbors unique microbial communities, which are adapted to harsh environmental conditions, such as near-freezing temperatures and extreme seasonality. The gene for the small ribosomal subunit (16S rRNA) is commonly used to study microbial communities in their natural environment. Several primer sets for this marker gene have been extensively tested across various sample-sets, typically originating from low-latitude environments. Yet, an explicit evaluation of their performance in representing the microbial communities of the Arctic Ocean is currently lacking. To select a suitable primer set for studying microbiomes of various Arctic marine habitats (sea ice, surface and deep ocean, sinking aggregates, and deep-sea sediment), we have conducted a performance comparison between two widely used primer sets, targeting different hypervariable regions of the 16S rRNA gene (V3-V4 and V4-V5). We observed that both primer sets were highly similar in representing the total microbial community composition down to a genus rank, which was also confirmed independently by subgroup-specific CARD-FISH counts. The V3-V4 primer set revealed higher internal diversity sensitivity in various taxonomic groups (e.g., Flavobacteriaceae). On the other hand, the V4-V5 primer set provides concurrent coverage of the archaeal domain, a relevant component that comprises 8-17% of sequences of the deep ocean and sediment Arctic microbial communities. Altogether, our comparison suggests that the use of the V4-V5 primer set is more suitable for studying microbial community dynamics of the Arctic marine environment.