Build Failed on Windows with CUDA >= 12
Summary
The compilation process encounters errors on Windows with CUDA version above 12
Exact steps to reproduce
Within a Powershell with CUDA > 12
PS D:\depots\gromacs_gitlab> cmake -S . -B .\build\ -D GMX_GPU=CUDA
PS D:\depots\gromacs_gitlab> cmake --build .\build --parallel 4
Building CUDA object src\gromacs\CMakeFiles\libgromacs.dir\nbnxm\cuda\nbnxm_cuda.cu.obj
FAILED: src/gromacs/CMakeFiles/libgromacs.dir/nbnxm/cuda/nbnxm_cuda.cu.obj
C:\PROGRA~1\NVIDIA~2\CUDA\v12.6\bin\nvcc.exe -forward-unknown-to-host-compiler -ccbin=C:\PROGRA~2\MICROS~3\2019\COMMUN~1\VC\Tools\MSVC\1429~1.301\bin\Hostx64\x64\cl.exe -DGMX_DOUBLE=0 -DHAVE_CONFIG_H -DMUPARSER_STATIC -DNOMINMAX -DTMPI_EXPORTS -DTMPI_USE_VISIBILITY -DUSE_STD_INTTYPES_H -ID:\depots\gromacs_gitlab\src\include -ID:\depots\gromacs_gitlab\build\src\include -ID:\depots\gromacs_gitlab\src -ID:\depots\gromacs_gitlab\src\external\tng_io\include -ID:\depots\gromacs_gitlab\build\tng\include -ID:\depots\gromacs_gitlab\api\legacy\include -ID:\depots\gromacs_gitlab\build\api\legacy\include -ID:\depots\gromacs_gitlab\src\gromacs\math\include -ID:\depots\gromacs_gitlab\src\gromacs\timing\include -ID:\depots\gromacs_gitlab\src\gromacs\utility\include -ID:\depots\gromacs_gitlab\src\gromacs\pbcutil\include -ID:\depots\gromacs_gitlab\src\gromacs\pulling\include -ID:\depots\gromacs_gitlab\src\gromacs\topology\include -ID:\depots\gromacs_gitlab\src\gromacs\linearalgebra\include -ID:\depots\gromacs_gitlab\src\gromacs\simd\include -ID:\depots\gromacs_gitlab\src\gromacs\taskassignment\include -isystem D:\depots\gromacs_gitlab\src\external\thread_mpi\include -isystem D:\depots\gromacs_gitlab\src\external -isystem D:\depots\gromacs_gitlab\src\gromacs\..\external\rpc_xdr -isystem "C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v12.6\include" -isystem D:\depots\gromacs_gitlab\src\external\muparser\include -isystem D:\depots\gromacs_gitlab\src\external\lmfit -isystem D:\depots\gromacs_gitlab\src\external\colvars -D_WINDOWS -Xcompiler=" /GR /EHsc" -Xcompiler=" -Zi -Ob0 -Od /RTC1" -std=c++17 "--generate-code=arch=compute_35,code=[compute_35,sm_35]" "--generate-code=arch=compute_37,code=[compute_37,sm_37]" "--generate-code=arch=compute_50,code=[compute_50,sm_50]" "--generate-code=arch=compute_52,code=[compute_52,sm_52]" "--generate-code=arch=compute_60,code=[compute_60,sm_60]" "--generate-code=arch=compute_61,code=[compute_61,sm_61]" "--generate-code=arch=compute_70,code=[compute_70,sm_70]" "--generate-code=arch=compute_75,code=[compute_75,sm_75]" "--generate-code=arch=compute_80,code=[compute_80,sm_80]" "--generate-code=arch=compute_86,code=[compute_86,sm_86]" "--generate-code=arch=compute_89,code=[compute_89,sm_89]" "--generate-code=arch=compute_90,code=[compute_90,sm_90]" "--generate-code=arch=compute_53,code=[compute_53,sm_53]" "--generate-code=arch=compute_80,code=[compute_80,sm_80]" -Xcompiler=-MDd -Wno-deprecated-gpu-targets -use_fast_math -Xptxas=-warn-double-usage -Xptxas=-Werror -Xcompiler=-openmp -Xcompiler=\"/DWIN32\" -Xcompiler=\"/D_WINDOWS\" -Xcompiler=\"/GR\" -Xcompiler=\"/EHsc\" -diag-suppress=177 -MD -MT src\gromacs\CMakeFiles\libgromacs.dir\nbnxm\cuda\nbnxm_cuda.cu.obj -MF src\gromacs\CMakeFiles\libgromacs.dir\nbnxm\cuda\nbnxm_cuda.cu.obj.d -x cu -c D:\depots\gromacs_gitlab\src\gromacs\nbnxm\cuda\nbnxm_cuda.cu -o src\gromacs\CMakeFiles\libgromacs.dir\nbnxm\cuda\nbnxm_cuda.cu.obj -Xcompiler=-Fdsrc\gromacs\CMakeFiles\libgromacs.dir\libgromacs.pdb,-FS
nvcc fatal : Unsupported gpu architecture 'compute_35'
For developers: Why is this important?
Currently, none Windows users with CUDA >12 can build GROMACS with default options unless the GMX_CUDA_TARGET_SM
option is provided.
This happends because CUDA 12 has dropped support for compute 3.X and how it's managed by GROMACS in gmxManageNvccConfig.cmake
.
For Windows, the nvcc flags test is disabled so by default, GROMACS try to compile for all compute architectures listed:
-- Found CUDAToolkit: C:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v12.6/include (found suitable version "12.6.68", minimum required is "11.0")
-- The CUDA compiler identification is NVIDIA 12.6.68
-- Detecting CUDA compiler ABI info
-- Detecting CUDA compiler ABI info - done
-- Check for working CUDA compiler: C:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v12.6/bin/nvcc.exe - skipped
-- Detecting CUDA compile features
-- Detecting CUDA compile features - done
Possible fixes
In my opinion, the easiest way to fix it is to remove the OR NOT WIN32
in the various if statements in gmxManageNvccConfig.cmake
:
-- Found CUDAToolkit: C:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v12.6/include (found suitable version "12.6.68", minimum required is "11.0")
-- Check for working NVCC/C++ compiler combination with nvcc 'C:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v12.6/bin/nvcc.exe'
-- Check for working NVCC/C++ compiler combination - works
-- Checking if nvcc accepts flags --generate-code=arch=compute_35,code=sm_35
-- Checking if nvcc accepts flags --generate-code=arch=compute_35,code=sm_35 - No
-- Checking if nvcc accepts flags --generate-code=arch=compute_37,code=sm_37
-- Checking if nvcc accepts flags --generate-code=arch=compute_37,code=sm_37 - No
-- Checking if nvcc accepts flags --generate-code=arch=compute_50,code=sm_50
-- Checking if nvcc accepts flags --generate-code=arch=compute_50,code=sm_50 - Success
-- Checking if nvcc accepts flags --generate-code=arch=compute_52,code=sm_52
-- Checking if nvcc accepts flags --generate-code=arch=compute_52,code=sm_52 - Success
-- Checking if nvcc accepts flags --generate-code=arch=compute_60,code=sm_60
-- Checking if nvcc accepts flags --generate-code=arch=compute_60,code=sm_60 - Success
...
-- The CUDA compiler identification is NVIDIA 12.6.68
-- Detecting CUDA compiler ABI info
-- Detecting CUDA compiler ABI info - done
-- Check for working CUDA compiler: C:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v12.6/bin/nvcc.exe - skipped
-- Detecting CUDA compile features
-- Detecting CUDA compile features - done