While in atmosphere models it is already common to define objective metrics to investigate how well an atmospheric model performs compared to observations, this is not too common for ocean models. Here we define a simple metrics encompassing the 3D structure of bias and absolute error to estimate the performance of ocean models and we apply it to the historical CMIP5 simulations from 1950 to 2005. Ocean model 3D temperature and salinity fields are compared to the PHC climatology for the major ocean basins. For each 3D grid point of the PHC dataset bias and absolute error of the model climatology are calculated and then volume- averaged over each ocean basin. An average CMIP5 model error is calculated for each ocean basin and used as a reference when investigating a particular model - similarly as has been done for the atmosphere by Reichler and Kim (2008) for CMIP3 models. Ocean surface temperature is generally reasonably well simulated by CMIP5 models and mean absolute errors amount to around 1 K which is comparable to the interannual variability. But in 500 to 1000 m - depending on the ocean basin and on the model - mean absolute errors of up to 4 K are detected which clearly exceed the interannual variability of generally below 1 K. For salinity mean absolute errors are in all levels clearly higher than the interannual variability. For example at the surface the mean absolute error amounts to up to 1 psu while the interannual variability is below 0.2 psu. Even if investigating biases which allows for cancelling out of errors within a basin instead of the mean absolute error this statement still holds in many cases. This means that there is a lot of scope for improvement of the simulation of the vertical structure of the ocean.