The relationship between biodiversity and ecosystem functioning (BEF) is a topic of considerable interest to scientists and managers because a better understanding of its underlying mechanisms may help us mitigate the consequences of biodiversity loss on ecosystems. Our current knowledge of BEF relies heavily on theoretical and experimental studies, typically conducted on a narrow range of spatio-temporal scales, environmental conditions, and trophic levels. Hence, whether a relationship holds in the natural environment is poorly understood, especially in exploited marine ecosystems. Using large-scale observations of marine fish communities, we applied a structural equation modelling framework to investigate the existence and significance of BEF relationships across northwestern European seas. We find that ecosystem functioning, here represented by spatial patterns in total fish biomass, is unrelated to species richness—the most commonly used diversity metric in BEF studies. Instead, community evenness, differences in species composition, and abiotic variables are significant drivers. In particular, we find that high fish biomass is associated with fish assemblages dominated by a few generalist species of a high trophic level, who are able to exploit both the benthic and pelagic energy pathway. Our study provides a better understanding of the mechanisms behind marine ecosystem functioning and allows for the integration of biodiversity into management considerations.