Coevolution between bacteriophages (phages) and their bacterial hosts occurs through changes in resistance and counter-resistance mechanisms. To assess phage–host evolution in wild populations, we isolated 195 Vibrio crassostreae strains and 243 vibriophages during a 5-month time series from an oyster farm and combined these isolates with existing V. crassostreae and phage isolates. Cross-infection studies of 81,926 host–phage pairs delineated a modular network where phages are best at infecting co-occurring hosts, indicating local adaptation. Successful propagation of phage is restricted by the ability to adsorb to closely related bacteria and further constrained by strain-specific defence systems. These defences are highly diverse and predominantly located on mobile genetic elements, and multiple defences are active within a single genome. We further show that epigenetic and genomic modifications enable phage to adapt to bacterial defences and alter host range. Our findings reveal that the evolution of bacterial defences and phage counter-defences is underpinned by frequent genetic exchanges with, and between, mobile genetic elements.