ELPA CMake Wrapper and Inclusion of ELPA 2026

Summary

A modernization of the ELPA shipped with ELSI was overdue. We now include ELPA-2026, but since the pre-processing step we used to apply is fraught with dangers, we copy the ELPA source verbatim. We also compile it on the fly, which minimizes the risk of mismatches. To help with the compilation of ELPA, we automate the “configure/build” steps with a wrapper that can translate CMake variables to autotools options and variables. ELSI can be built and installed as a stand-alone package, or it can be compiled as a subproject.

In more detail, this merge request implements building of newer versions of ELPA via a CMake wrapper around the ELPA configure/autotools build, using the ExternalProject_Add function to create an elpa_autotools build target which is added as a dependency of elsi. An Elpa::elpa imported target is also created, similar to how the FindExternalELPA.cmake script works. A CPU and GPU version (CUDA/NVIDIA and HIP/AMD) is tested and working. I have tested linking ELSI to FHI-aims both via a subproject (the default in FHI-aims) which works without changing anything in FHI-aims, and building ELSI as a top level project and linking to FHI-aims with ELSI as a static library (both from the ELSI build folder and the ELSI install folder) and this also works. Note that in order to do this, I had to modify how elsiConfig.cmake is generated (please see below). ELSI as a shared library has also been tested both as a standalone build linked to FHI-aims, as well as via a subproject where FHI-aims is also compiled as a shared library. Here everything also works.

This is now ready to be merged. @garalb has provided a review and added commits, which I have incorporated into this MR.

Implementation Details

├── external
│   ├── ELPA
│   │   ├── cmake
│   │   │   ├── BuildElpaAutotools.cmake
│   │   │   ├── ElpaAutotoolsDictionary.cmake
│   │   │   ├── ELPAConfig.cmake.in
│   │   │   └── ELPAConfigVersion.cmake.in
│   │   ├── CMakeLists.txt
│   │   ├── ELPA-2020.05.001
│   │   ├── ELPA-2021.11.001
│   │   ├── ELPA-2026.02.002
│   │   └── patches
│   │       └── 2026.02.002

where:

  • BuildElpaAutotools.cmake : This is the main driving script to build ELPA, uses ExternalProject_Add to drive the autotools build from cmake
  • ElpaAutotoolsDictionary.cmake : This translates cmake options into flags for the autotools configure statement.
  • ELPAConfig.cmake.in : Creates an ELPAConfig.cmake file.
  • ELPAConfigVersion.cmake.in : Creates an ELPAConfigVersion.cmake file
  • The ELPA/CMakeLists.txt file is modified to run the BuildElpaAutotools.cmake script
  • src/CMakeLists.txt is modified to add elpa_autotools as a dependency of elsi, and links ELPA to ELSI via TARGET_LINK_LIBRARIES(elsi PUBLIC Elpa::elpa).
  • For the above linking to work when ELSI is compiled as a static library, ELSI needs to be able to find Elpa::elpa since it is not a true CMake target. For this, I had to include a find_dependency(ELPA ... call in elsiConfig.cmake.
  • elsiConfig.cmake is now generated via a elsiConfig.cmake.in file, rather than automatically generated from an EXPORT call in src/CMakeLists.txt. All places where elsiConfig was previously in various CMakeLists.txt files now are instead elsiTargets and an elsiTargets.cmake file is generated which contains all the things that were previously in elsiConfig.cmake, and is included in the new elsiConfig.cmake.in call via an include.. statement.
  • The elsiConfig.cmake.in file should provide more flexibility going forward, and I believe is the modern CMake way to do things.
  • The pre/post2025 ELPA block in src/CMakeLists.txt is modified so that internal ELPA 2026 can be compiled with the correct sources.
  • The ELPA build files are put in an elpa-build sub-directory of the ELSI build directory, and ELPA install files (e.g. libraries and include files) are put in an elpa-install sub-directory of the build directory. When ELSI is installed, these files are also copied to ELSI's install directory.
  • patches is a directory containing .patch files for bug fixes for ELPA not currently available in the upstream version (currently applied just to version 2026.02.002). A README exists inside this folder briefly describing each of the patches, and each of the patch files contain the commit details they correspond to.

Usability Details

CPU

  • I have been testing locally with an intel OneAPI 2025.3 toolchain, using the following elsi cmake Cache file:
###############
# Basic Flags
###############
set(CMAKE_Fortran_COMPILER mpiifx CACHE STRING "" FORCE)
set(CMAKE_Fortran_FLAGS "-O3 -fp-model precise" CACHE STRING "" FORCE)
set(Fortran_MIN_FLAGS "-O0 -fp-model precise" CACHE STRING "" FORCE)
set(LIB_PATHS "$ENV{MKLROOT}/lib/intel64" CACHE STRING "" FORCE)
set(LIBS "-lmkl_scalapack_lp64 -lmkl_intel_lp64 -lmkl_sequential -lmkl_core -lmkl_blacs_intelmpi_lp64" CACHE STRING "" FORCE)


###########
# C Flags 
###########
set(CMAKE_C_COMPILER mpiicx CACHE STRING "" FORCE)
set(CMAKE_C_FLAGS "-O3 -fp-model precise" CACHE STRING "" FORCE)
set(CMAKE_CXX_COMPILER mpiicpx  CACHE STRING "" FORCE)
set(CMAKE_CXX_FLAGS "-O3 -fp-model precise" CACHE STRING "" FORCE)

# ELPA2026 variables
set(INC_PATHS "$ENV{MKLROOT}/include/mkl/intel64/lp64" CACHE STRING "" FORCE)
SET(USE_ELPA_2026 ON CACHE BOOL "ENABLE ELPA 2026")

The only difference from a current ELSI build is the inclusion of the final 2 lines, namely the USE_ELPA_2026 variable, and setting the INC_PATHS to the directory where mkl_service.mod is found, as ELPA uses this during the configure step (with MKL).

I also tested with a GCC + OpenMPI + OpenBLAS toolchain, here everything was also fine, and no INC_PATHS were required.

GPU

  • I have been testing locally with an intel OneAPI 2025.3 + cuda 12.8 toolchain, using the following elsi cmake Cache file:
###############
# Basic Flags
###############
set(CMAKE_Fortran_COMPILER mpiifx CACHE STRING "" FORCE)
set(CMAKE_Fortran_FLAGS "-O3 -fp-model precise" CACHE STRING "" FORCE)
set(Fortran_MIN_FLAGS "-O0 -fp-model precise" CACHE STRING "" FORCE)
SET(LIB_PATHS "$ENV{MKLROOT}/lib/intel64 $ENV{CUDA_HOME}/lib64 $ENV{CUBLAS_HOME}/lib64" CACHE STRING "External library paths")
SET(LIBS "mkl_scalapack_lp64 mkl_blacs_intelmpi_lp64 mkl_intel_lp64 mkl_sequential mkl_core cusolver cublas cublasLt cudart" CACHE STRING "External libraries")

###########
# C Flags 
###########
set(CMAKE_C_COMPILER mpiicx CACHE STRING "" FORCE)
set(CMAKE_C_FLAGS "-O3 -fp-model precise" CACHE STRING "" FORCE)
set(CMAKE_CXX_COMPILER mpiicpx  CACHE STRING "" FORCE)
set(CMAKE_CXX_FLAGS "-O3 -fp-model precise" CACHE STRING "" FORCE)

###########
# GPU Flags 
###########
SET(CMAKE_CUDA_FLAGS "-O3 -arch=sm_120" CACHE STRING "CUDA flags")
SET(USE_GPU_CUDA ON CACHE BOOL "Use CUDA-based GPU acceleration in ELPA")


# Need to get the INC_PATHS set to properly build ELPA. Not sure how to automate this yet
set(INC_PATHS "$ENV{MKLROOT}/include/mkl/intel64/lp64 $ENV{CUDA_HOME}/include $ENV{CUBLAS_HOME}/include" CACHE STRING "" FORCE)
SET(CMAKE_EXE_LINKER_FLAGS "-lstdc++" CACHE STRING "Linker flags")
SET(USE_ELPA_2026 ON CACHE BOOL "ENABLE ELPA 2026")

Similar to the CPU case, one needs to set the include paths to CUDA and CUBLAS. Also, I needed to include the -lstdc++ CMAKE_EXE_LINKER_FLAGS, similar to the intel_external.cmake and intel_external_llvm.cmake toolchains.

User configure statement

  • As well as the automatic CMake -> autotools translation, it is also possible to provide a file with all the flags for configure that one desires. In the initial Cmake Cache, provide a variable:
SET(ELPA_CONFIGURE_COMMAND_FILE "${CMAKE_CURRENT_LIST_DIR}/configure_script_elpa_local_gpu_oneapi2025.3.sh" CACHE STRING "" FORCE)

with an absolute path to a file, in this case configure_script_elpa_local_gpu_oneapi2025.3.sh which contains flags for the configure command, but without the command itself, or --prefix as these automatically get set. The file contents of configure_script_elpa_local_gpu_oneapi2025.3.sh in this case is:

-enable-option-checking=fatal CC=mpiicx FC=mpiifx CXX=mpiicpx CFLAGS="-O3  -I${MKLROOT}/include -I$CUDA_HOME/include -I$CUBLAS_HOME/include " CXXFLAGS="-O3  -I${MKLROOT}/include -I$CUDA    _HOME/include -I$CUBLAS_HOME/include " FCFLAGS="-O3  -I${MKLROOT}/include/mkl/intel64/lp64 -I$CUDA_HOME/include -I$CUBLAS_HOME/include " LDFLAGS="-L${MKLROOT}/lib -lmkl_scalapack_lp64 -l    mkl_intel_lp64 -lmkl_sequential -lmkl_core -lmkl_blacs_intelmpi_lp64 -lpthread -lm -ldl -lstdc++ -Wl,-rpath,${MKLROOT}/lib -L$CUDA_HOME/lib64 -L$CUBLAS_HOME/lib64" --with-mpi=yes --enabl    e-nvidia-gpu-kernels --enable-c-tests=yes --enable-cpp-tests=yes --enable-single-precision=no --enable-gpu-ccl=no --disable-avx --disable-avx2 --disable-avx512 --disable-sse-kernels --di    sable-sse-assembly --with-cuda-path=$CUDA_HOME --with-NVIDIA-GPU-compute-capability=sm_120

Additional Changes

  • ELSI CMake minimum version bumped to 3.7, as pkg_check_modules(...IMPORTED_TARGET) used in ELPAConfig.cmake.in only introduced in CMake version 3.7: https://cmake.org/cmake/help/latest/module/FindPkgConfig.html
  • I have added the routines elsi_get_elpa_real_kernel_name and elsi_get_elpa_complex_kernel_name in elsi_get to query the currently used ELPA2 kernel names.
  • I have added an interface for elsi_get_elpa_api_version inside elsi_interface to allow outside codes to call this cleanly.

To Do

  • Finish the GPU build
  • Test and get the ELSI as a subproject build working.
  • Review the changes, make sure nothing breaks for standard ELSI
  • Review the ELPAConfigVersion.cmake.in and ELPAConfig.cmake.in files, I have not really touched them from Peter's example.
  • Populate more fully the ElpaAutotoolsDictionary.cmake
  • Cleanup the code - make sure everything is in its proper place
  • Add a toolchain for the ELPA2026 build
  • Install the ELPA bin directory to the ELSI install directory
  • Update ELPA to 2026.02.002
  • Check that ELSI as a shared library works
  • Check for CMake minimum version in newly added parts -> Needs to be bumped to 3.7.
  • Add an elsi_get routine to obtain which ELPA kernel is being used. Currently there is no runtime way to access this information.
  • Add interface for elsi_get_elpa_api_version in elsi_interface to publicly expose this for other codes to use.
  • Add a USE_GPU_HIP CMake variable, analagous to USE_GPU_CUDA to be able to use the AMD GPU accelerated parts of ELPA 2026.
  • Documentation
  • Add fixes from @garalb branch: https://gitlab.com/garalb/elsi_interface/-/tree/elpa-cmake-wrapper--ag?ref_type=heads
  • Add patches folder
  • Add INC_PATHS also to CFLAGS and CXXFLAGS in ElpaAutotoolsDictionary.cmake (adding simply to CPPFLAGS doesn't always work).
  • Make autotools a required package if the autogen.sh route is followed

Future Work

During the review by @garalb , it was also suggested that any arbitrary version of ELPA could be selected by a user, by modifying how the ExternalProject_Add command works by providing a URL to the ELPA tarball, which can then be downloaded at CMake configure time, rather than hard copied in the ELSI project like in the MR. I will leave this to future work, before the next release, so that it could be incorporated in addition to the provided ELPA 2026 version. The reasoning is twofold:

  1. Having a hard-copied version of ELPA means that ELSI can be used offline (similar to how the other libraries can be used)
  2. By providing a specified version of ELPA, we are guaranteeing that it works/has been tested. With other versions of ELPA, this guarantee may not be in place. Furthermore, there may be certain configure flags that are only available in certain versions of ELPA, and additional logic may be needed in the translation of CMake to configure flags.
Edited by James Green

Merge request reports

Loading
Loading