Delivering a step-change in the performance and functionality of the Fluidity shallow water solver

Key Personnel

PI/Co-I: Prof. Matthew D. Piggott, Dr. David A. Ham - Imperial College London

Technical: Dr. Christian T. Jacobs - Imperial College London

Relevant Documents

eCSE Technical Report: Delivering a step-change in performance and functionality to the Fluidity shallow water solver through code generation

Project summary

Fluidity (fluidity-project.org) is a fully-featured, open source, computational fluid dynamics (CFD) framework. It comprises several advanced numerical models based on the finite element method as well as a number of novel numerical features (e.g. mesh adaptivity) making it suitable for multi-scale simulations. It is largely unique in its abilities to also solve large-scale geophysical/oceanographic problems. Key examples include marine renewable energy, tsunami simulation and inundation, and palaeo-tidal simulations for hydrocarbon exploration.

The current Fluidity codebase comprises hand-written Fortran code to perform the finite element discretisation. Not only is this hand-written code potentially sub-optimal, it presents issues regarding its maintainability and longevity; should one want to run Fluidity on a newer hardware architecture more suited to larger scale problems in the future, then the entire codebase may have to be re-written. Furthermore, the need for numerical modellers to not only be experts in their field of science, but also be well-versed in parallel programming and code optimisation, is unsustainable in the long-term.

This eCSE project delivers a step-change in the performance and functionality of the shallow water model within Fluidity, accomplished by using the Firedrake framework (firedrakeproject.org) for the automated solution of partial differential equations using code generation techniques. A key aim is to remove Fluidity's existing hand-written Fortran finite element discretisation code and instead generate it automatically from a higher-level model description, thereby hiding complexity through layers of abstraction. This allows the users of the resulting models to focus on the problem specification and the end results of simulations. The Firedrake project achieves all of this in a performance-portable manner using the PyOP2 framework (github.com/OP2/PyOP2) to target and optimise the automatically-generated code for a desired hardware architecture. Moreover, in contrast to traditional, hand-written models, such as Fluidity, the use of code generation techniques has been shown to deliver significantly enhanced performance, as well as improved code maintainability.

Summary of the software

Firedrake-Fluids is available under the open-source GNU GPL licence (version 3). The source code is under Git version control and hosted on GitHub at the following URL: http://github.com/firedrakeproject/firedrake-fluids

The dependencies of Firedrake-Fluids (namely Firedrake itself and associated packages) can be readily loaded on ARCHER using:
export FDRAKE_DIR=/work/y07/y07/fdrake
module use $FDRAKE_DIR/modules

After loading the necessary modules, Firedrake-Fluids can then be run on ARCHER using a PBS submission script such as:
https://github.com/firedrakeproject/firedrake-fluids/blob/master/tools/parallel_scaling/src/submit_archer.pbs