Long-standing systematic model errors in both tropics and extratropics of the ECMWF model run at a horizontal resolution typical for climate models are investigated. Based on the hypothesis that the misrepresentation of unresolved scales contributes to the systematic model error, three model refinements aimed at their representation—fluctuating or deterministically—are investigated. Increasing horizontal resolution to explicitly simulate smaller-scale features, representing subgrid-scale fluctuations by a stochastic parameterization, and improving the deterministic physics parameterizations all lead to a decrease in the systematic bias of the Northern Hemispheric circulation. These refinements reduce the overly zonal flow and improve the model’s ability to capture the frequency of blocking. However, the model refinements differ greatly in their impact in the tropics. While improving the deterministic and introducing stochastic parameterizations reduces the systematic precipitation bias and improves the characteristics of convectively coupled waves and tropical variability in general, increasing horizontal resolution has little impact. The fact that different model refinements can lead to reductions in systematic model error is consistent with the hypothesis that unresolved scales play an important role. At the same time, this degeneracy of the response to different forcings can lead to compensating model errors. Hence, if one takes the view that stochastic parameterization should be an important element of next-generation climate models, if only to provide reliable estimates of model uncertainty, then a fundamental conclusion of this study is that stochasticity should be incorporated within the design of physical process parameterizations and improvements of the dynamical core and not added a posteriori.