A Python interface to the Fortran-based Parallel Data Assimilation Framework: pyPDAF v1.0.2


Contact
lars.nerger [ at ] awi.de

Abstract

Abstract. Data assimilation (DA) is an essential component of numerical weather and climate prediction. Efficient implementation of DA algorithms benefits both research and operational prediction. Currently, a variety of DA software programs are available. One of the notable DA libraries is the Parallel Data Assimilation Framework (PDAF) designed for ensemble data assimilation. The DA framework is widely used with complex high-dimensional climate models, and is applied for research on atmosphere, ocean, sea ice and marine ecosystem modelling, as well as operational ocean forecasting. Meanwhile, there are increasing demands for flexible and efficient DA implementations using Python due to the increasing amount of intermediate complexity models as well as machine learning based models coded in Python. To accommodate for such demands, we introduce a Python interface to PDAF, pyPDAF. pyPDAF allows for flexible DA system development while retaining the efficient implementation of the core DA algorithms in the Fortran-based PDAF. The ideal use-case of pyPDAF is a DA system where the model integration is independent from the DA program, which reads the model forecast ensemble, produces an analysis, and updates the restart files of the model, or a DA system where the model can be used in Python. With implementations of both PDAF and pyPDAF, this study demonstrates the use of pyPDAF and PDAF in a coupled data assimilation (CDA) setup in a coupled atmosphere-ocean model, the Modular Arbitrary-Order Ocean-Atmosphere Model (MAOOAM). This study demonstrates that pyPDAF allows for PDAF functionalities from Python where users can utilise Python functions to handle case-specific information from observations and numerical model. The study also shows that pyPDAF can be used with high-dimensional systems with little slow-down per analysis step of only up to 13 % for the localized ensemble Kalman filter LETKF in the example used in this study. The study also shows that, compared to PDAF, the overhead of pyPDAF is comparatively smaller when computationally intensive components dominate the DA system. This can be the case for systems with high-dimensional state vectors.



Item Type
Article
Authors
Divisions
Primary Division
Programs
Primary Topic
Helmholtz Cross Cutting Activity (2021-2027)
Publication Status
Published
Eprint ID
60454
DOI 10.5194/gmd-18-8235-2025

Cite as
Chen, Y. , Nerger, L. and Lawless, A. S. (2025): A Python interface to the Fortran-based Parallel Data Assimilation Framework: pyPDAF v1.0.2 , Geoscientific Model Development, 18 (21), pp. 8235-8252 . doi: 10.5194/gmd-18-8235-2025


Download
[thumbnail of Chen_etal_GMD18_8235_2025_pyPDAF.pdf]
Preview
PDF
Chen_etal_GMD18_8235_2025_pyPDAF.pdf - Other

Download (2MB) | Preview

Share
Add to AnyAdd to TwitterAdd to FacebookAdd to LinkedinAdd to PinterestAdd to Email


Citation

Research Platforms
N/A


Actions
Edit Item Edit Item