Cold-water coral reefs are known to locally enhance the diversity of deep-sea fauna as well as of microbes. Sponges are among the most diverse faunal groups in these ecosystems, and many of them host large abundances of microbes in their tissues. In this study, twelve sponge species from three cold-water coral reefs off Norway were investigated for the relationship between sponge phylogenetic classification (species and family level), as well as sponge type (high versus low microbial abundance), and the diversity of sponge-associated bacterial communities, taking also geographic location and water depth into account. Community analysis by Automated Ribosomal Intergenic Spacer Analysis (ARISA) showed that as many as 345 (79%) of the 437 different bacterial operational taxonomic units (OTUs) detected in the dataset were shared between sponges and sediments, while only 70 (16%) appeared purely sponge-associated. Furthermore, changes in bacterial community structure were significantly related to sponge species (63% of explained community variation), sponge family (52%) or sponge type (30%), whereas mesoscale geographic distances and water depth showed comparatively small effects (<5% each). In addition, a highly significant, positive relationship between bacterial community dissimilarity and sponge phylogenetic distance was observed within the ancient family of the Geodiidae. Overall, the high diversity of sponges in cold-water coral reefs, combined with the observed sponge-related variation in bacterial community structure, support the idea that sponges represent heterogeneous, yet structured microbial habitats that contribute significantly to enhancing bacterial diversity in deep-sea ecosystems.