Calculating Excited States of Extended Systems in LR-TDDFT

Key Personnel

PI: Dr Nicholas Hine - University of Cambridge (now University of Warwick)

Technical: Dr Tim Zuehlsdorff - University of Cambridge

Relevant documents

eCSE Technical Report: Calculating excited states of extended systems in LR-TDDFT

Project summary

ONETEP and other codes for large-scale Density Functional Theory (DFT) are enabling a paradigm shift in our ability to predict ground-state properties for large systems such as complex biomolecules, such that it is now becoming routine and available even to non-expert users. However, prediction of the behaviour of excited states and of spectroscopy of large systems is still a great challenge, and requires further development of novel methodology and associated simulation software. Overcoming this challenge would allow the computation of excited states of systems such as pigment-protein complexes for light-harvesting, and photoactivated biological molecules such as rhodopsin, a key component of light-sensing.

Linear Response Time-Dependent Density-Functional Theory (LR-TDDFT) has been the method of choice for computing optical properties of large systems, due to a good balance between the achievable accuracy and the computational cost. Enhancing, extending and improving the implementation of LR-TDDFT in ONETEP, so as to make it more suitable for widespread use, was thus the main goal of this eCSE project. The investigators had already implemented a highly novel version of the LR-TDDFT method (T. J. Zuehlsdorff, N. D. M. Hine et al., J. Chem. Phys. 139, 064104 (2013)) into the ONETEP code, that is specifically aimed at computing optical absorption spectra of very large systems. The method had been proven to scale linearly with system size, enabling simulations to reach the kind of system sizes that can be probed experimentally. However, the method had not yet been widely applied, optimised, or had its accuracy extensively tested. Code optimisation needed to be carried out to enable very much more efficient calculations on state-of-the-art computational architectures. The eCSE project provided an opportunity to test this functionality and build upon it significantly.

Achievement of objectives

Our original goals were twofold:

1. Benchmark and improve upon the parallel efficiency and scaling of LR-TDDFT in ONETEP, particularly with respect to hybrid OpenMP-MPI parallelism; and
2. Implement the analytic excited state gradient formalism and consequently enable geometry optimisation of systems in excited states.

In respect of the first goal, we have been highly successful, as detailed in the technical report. We have extended the OpenMP parallelism to all of the routines used in the LR-TDDFT functionality, and we have carried out demonstrations and carefully-controlled tests of the methodology in situations where there is a well-known answer provided by experimental spectroscopy. The technical report gives examples of use of the LR-TDDFT code, using OpenMP/MPI hybrid parallelism to scale effectively to over 1000 cores, making excellent use of ARCHER resources. Our success metric was that the LR-TDDFT functionality should scale as well as the orignal code, and this has been achieved.

The numerical results of the testing revealed a crucial issue that forced a change in emphasis for the rest of the project. It turned out that the so-called Tamm-Dancoff approximation (TDA), a simplifying assumption which was made in our earlier work to implement LR-TDDFT, was insufficiently accurate in the biological systems to which we wanted to apply the method. Indeed sometimes it produced qualitatively wrong results. For example, the TDA predicted an optical absorption spectrum for the well-known chlorophyll molecule, which is responsible for the green colouring of plants, which had absorption peaks in the green region of the spectrum and would have made plants appear red! In view of this we modified the second objective, as it was clearly a higher priority to improve upon the Tamm-Dancoff Approximation before accurate excited state forces could be considered.

Therefore the second objective, which took up the final three months of the project and has recently been successfully completed, became to implement a "beyond TDA" approach to LR-TDDFT. The details of how this was achieved are available in the technical report (see link at top of page) and a recent paper (T. J. Zuehlsdorff, N. D. M. Hine, M. C. Payne and P. D. Haynes, J. Chem. Phys. 143, 204107 (2015)), which can be expected to be of considerable interest to the TDDFT community, as well as pointing the way to much higher-accuracy calculations (see T. J. Zuehlsdorff, P. D. Haynes, F. Hanke, M. C. Payne and N. D. M. Hine, J. Chem. Theory Comput. 12(4), 1853-1861 (2016)).

Summary of the Software

Electronic structure theory is a hugely important contributor to understanding properties of materials. Among software packages for such modelling, implementing density functional theory in forms suitable for large-scale HPC, the code central to the current proposal, ONETEP, has several unique advantages. It combines systematically-controllable accuracy with a requirement of computational effort that scales only linearly with system size and it has excellent parallel scaling to thousands of cores. Developed over the last decade by a collaboration of academics from Cambridge, Imperial, Southampton and Warwick, it is now widely used in the UK and internationally both by industry and by academics in a variety of research fields. It can be obtained by license via Cambridge Enterprise or as part of the Materials Studio Package distributed by BIOVIA. See the ONETEP webpage for details of licensing.

For licensed users, the software is available as a module on ARCHER, and examples of job submission scripts and compilation details can be found at: https://www.archer.ac.uk/documentation/software/onetep/. The code is already heavily used on ARCHER: research into the general area of materials science has accounted for 52% of usage of ARCHER in the most recent accounting period, and ONETEP is the fourth heaviest user by CPU hours among codes for density functional theory (note that the first three are codes aimed at bulk materials, while ONETEP is aimed at nanostructures and biomolecules). It has been the tenth most heavily-used code overall among all application areas in 2014-2015, and has accounted for an continuously growing share of total resource usage during the various phases of HECToR and ARCHER dating back to 2009. Users run the code for applications ranging from inorganic nanomaterials such as catalyst nanotubes and amorphous nanoparticles to biological systems such DNA and proteins.