To describe the progress in Earth System Science, a conceptual framework is proposed which includes hypothesis testing, the formulation of models with different complexity as well as expressing discoveries in terms of metaphors. The later approach is demonstrated by the conveyor belt concept in oceanography which influenced the discussion about abrupt climate changes where the ocean circulation may be involved. It is argued that the combination of different methodologies/complexities and independent results is necessary to prevent over-simplistic views in each discipline of Earth System Science. Emphasis is given on typical steps to obtain new ideas for a new discovery. Examples for over-simplistic views are mentioned for past climate information from proxy data. The recorder system of the proxy has to be taken into account, otherwise the climate information can be misinterpreted. It is concluded that in the field of Earth System Science, basic knowledge and true collaborative problem solving is necessary to make scientists aware of the underlying principles, the limitations and open questions. This is furthermore necessary to develop and sharpen our ideas about the complex Earth System.