Project Meshes - a HPC geometry library supporting mesh-to-mesh projections

Key Personnel

PI/Co-I: Tobias Weinzierl (University of Durham)

Technical: Konstantinos Krestenitis (University of Durham)

Relevant documents

eCSE Technical Report: Project Meshes - a HPC geometry library supporting mesh-to-mesh projections

Project summary

Many mesh-based high performance applications require users to map their data from one discretised geometric representation (mesh) to another mesh multiple times. Three examples are:

• Codes read in CAD models, to then be projected onto their compute mesh via an immersed boundary method. For this, they need something similar to distance maps that clarify per degree of freedom in their compute mesh how far away the CAD mesh is.
• Codes read in multiple CAD models, to be moved around and collided, i.e. make the collisions determine how the individual small meshes behave, and ultimately to write out CAD format again.
• Codes read in CAD models of two different meshes with a third mesh constructed in-between those meshes where grid entities are close to each other. This is a classic task in fluid-structure interaction, e.g. Whenever either of the two input meshes change, the interface mesh has to be updated, too.

The Delta library is a tool to handle imports/exports of geometries for use cases such as the one above. Realising the above features is usually time-consuming and technical. Our code serves as building block to simplify setting up realistic simulations using realistic geometries.

All routines within Delta are hereby designed to run with low complexity and to exploit modern vector registers. Delta is fast.

Delta Tour

We showcase how Delta works in the hands of a case study where we want to load a wind blade into an immersed boundary code. Delta loads the actual wind blade from a standard CAD file. For this, we create an instance of Delta's main class and invoke the corresponding load operation.

The Delta C++ object holds the CAD format internally in its own plain graph format, but - hidden away from the user - converts it into a flat triangle representation (see [Krestenitis2018_1]) which can be piped through vector registers. It also hides away the fact that it uses an octree on top of the actual data structure. With the octree, it is easier for Delta to answer queries efficiently. If a query affects an empty area of the octree, the code can skip comparisons.

So we have our mesh (blue; left) and overlay it with the CAD model (white). We then loop over our mesh and ask for our mesh entities whether they contact the Delta object. Delta can compare any two meshes and gives us back contact points, i.e. the closest point in-between two meshes as long as this closest point is closer than a prescribed epsilon. This feature allows us to run over each voxel in our compute grid and to try to find out whether we are close enough/overlapping the CAD model. We can voxelise the CAD model (right).

There are two computational stupidities of the above sketch which Delta avoids internally. First, geometric comparisons can be vectorised quite efficiently (see Krestenitis2018 [1]) if we study not one point or mesh triangle at a time but a whole set of close points/mesh entities. Delta thus provides functions for whole geometry sets. Second, it is inefficient to compare for a given mesh each individual cell and thus to pose similar geometric questions over and over again. If one mesh cell is far away from the CAD model, its neighbour likely is far away, too. So Delta internally constructs an octree to cache insight for repeating or similar geometric queries. This octree can either be built up upfront or on-the-fly (see Krestenitis2018 [2]).

Summary of the Software

• The software is all open source and available through (www.peano-framework.org/hpcsoftware/delta).
• The software is hosted by Leibniz Supercomputing Centre (LRZ) in a git repository and thus regularly backed up and available beyond the project lifetime.
• The Delta webpage links to a guidebook, code snapshots, repository access, and autogenerated source code documentation (doxygen) which is updated every night.
• We integrate a repository dashboard with requests/remarks both through a git mailing list and via a newsfeed on the project webpage.
• The code is tested/evaluated on Durham's Hamilton cluster (Broadwell) and the Archer KNL partition.

We are currently re-releasing the code with improved documentation and slimmer interfaces picking up lessons learned from the eCSE project. This re-release follows "publish early, publish often" while the eCSE project's code base remains available through the repository.

Scientific Benefits

Delta did not exist prior to the eCSE programme. Its ideas have been published in various papers and theses, but there has been no common code base, notably not a usable one. The intrinsic value of the eCSE project here stems from the fact that existing ideas and code fragments have joined forces in one software library/tool which now can be used as a building block for upcoming computational science and engineering. The access to state-of-the-art computing facilities - the particular architecture might be discontinued by Intel but ideas alike AVX512 continue to shape computing - allowed us to combine the development of a library that fuses different algorithmic ideas with state-of-the-art performance engineering.

[1] K. Krestenitis, T. Weinzierl and T. Koziara: Fast DEM collision checks on multicore nodes. In R. Wyrzykowski, J. Dongarra, E. Deelman, K. Karczewski (ed.), Parallel Processing and Applied Mathematics. Volume 10777 of Lecture Notes in Computer Science, pp. 123-132, 2018.

[2] K. Krestenitis and T. Weinzierl: A Multi-Core Ready Discrete Element Method With Triangles Using Dynamically Adaptive Multiscale Grids. Concurrency and Computation: Practice and Experience. (in print; gold access)