Improving the performance of Nektar++ using communication and I/O masking

Key Personnel

PI/Co-I: Prof Spencer Sherwin, Dr Chris Cantwell, Dr David Moxey - Imperial College London

Technical: Dr Rupert Nash - EPCC; Simon Clifford - freelance

Relevant Documents

eCSE Technical Report: Communication and I/O masking for increasing the performance of Nektar++

Project summary

Nektar++ is an open-source spectral/hp element framework, designed to allow the numerical solution of partial differential equations in complex geometries using high-order discretisations. It is a complete rewrite of the Nektar incompressible Navier-Stokes flow solver and is designed as a flexible and extensible numerical toolkit for a broad range of scientific domains. The framework supports both continuous and discontinuous spectral/hp element methods and multiple element shapes; it is cross-platform in nature, and leverages communication packages from the Nek5000 software.

We are using Nektar++ to study problems in a range of engineering and biomedical areas. In the area of engineering we are investigating vehicle aerodynamics for Formula 1 racing cars, stability of basic flows, and compressible flow dynamics over wings. Biomedical applications include identifying markers for the formation of atherosclerosis through flow modelling and mass transport, and understanding the mechanistic properties of atrial fibrillation using a model of cardiac electrophysiology. Other groups, in the UK and internationally, are also using the code to examine noise generation in turbine engines, study viscoelastic flows and develop shallow water models.

The objective of this eCSE project was to carry out implementation improvements on Nektar++, not only in terms of the parallel efficiency and scalability of the code, but also in terms of the user experience when handling meshes and simulation data.

The improvements to the code allow for existing numerical experiments to be undertaken in a shorter period of time by making more efficient use of available computational resources, and allowing the code to scale to higher core counts. It is now capable of taking full advantage of the latest national HPC systems, and can be considered to be a cutting-edge tool for complex geometry scientific research.

The improvements have benefited a wide range of application areas which Nektar++ supports including, but not limited to, transient aerodynamics simulation (F1, aircraft design and analysis), flow stability analysis and laminar flow control, aero-acoustic simulation (aircraft engines), modelling of patient-specific cardiac electrophysiology (atrial fibrillation) and biological flow simulations for understanding disease formation and development (atherosclerosis). Many of these applications require large-scale simulation capability and, particularly in the medical sector, may be time-critical.

Summary of the software

Nektar++ is a tensor product based finite element package designed to allow one to construct efficient classical low polynomial order h-type solvers (where h is the size of the finite element) as well as higher p-order piecewise polynomial order solvers. The framework comes with a number of solvers and also allows one to construct a variety of new solvers. Capabilities of the code include:

• Representation of one, two and three-dimensional fields as a collection of piecewise continuous or discontinuous polynomial domains.
• Segment, plane and volume domains are permissible, as well as domains representing curves and surfaces (dimensionally-embedded domains).
• Hybrid shaped elements, i.e triangles and quadrilaterals or tetrahedra, prisms and hexahedra.
• Both hierarchical and nodal expansion bases.
• Continuous or discontinuous Galerkin operators.
• Cross platform support for Linux, Mac OS X and Windows.

Example solvers and tutorials on their use are documented in the Nektar++ user guide, which is available along with the software from http://www.nektar.info. The git code repository can be accessed through http://gitlab.nektar.info. The software is open-source (MIT license) and is currently available as a module on ARCHER.