qe7.0 GPU version pw.x not working for v.21.11 and later
QE 7.0 release, from github.
Mint OS 20.3 Cinnamon, Linux kernal 5.13.0-25-generic
One AMD Ryzen 3900x (12-core, 24 threads CPU), one nvidia tesla K80
NVHPC (HPC_SDK) version 21.11
openMPI 3.1.5, included in NVHPC
nvidia driver 470.86, CUDA Version 11.4
The term 'not work' is very vague, but I cannot find more appropriate word for this situation.
With QEv6.8, everything is fine. GPU version runs several times faster than CPU only openMP serial version.
(./configure --with-cuda=/opt/nvidia/hpc_sdk/Linux_x86_64/2021/cuda/ --with-cuda-runtime=11.5 --with-cuda-cc=35 --enable-openmp --with-scalapack=no)
(with adding curand and -D__GPU_MPI to make.inc file)
There is no problem for configure and compile for QEv7.0 GPU version. I use same configure command with QEv6.8 above.
But, when I start run pw.x, calculation not proceed after line
[[ Estimated max dynamical RAM per process > 461.61 MB
Initial potential from superposition of free atoms
Initial potential from superposition of free atoms
starting charge 12.9973, renormalised to 13.0000]]
For QE6.8, GPU usage rise to 100% as soon as line [starting charge ...] written, but GPU usage stays idle in QE7.0.
Waiting didn't help. It stays same after overnight - even for very simple model, which cost less than a minute for finish calculation.
(mpirun -n 2 --bind-to none -x OMP_NUM_THREADS=6 /QE7.0PATH/pw.x -i ./xxx.in > ./xxx.out)
I tried some different input files - dozens of atoms in unit cell with many kpoints, or unit cell with 1 atom with 1 kpoints - all behave same.
Compile with different cuda version (--with-cuda-runtime=11.0) not help.
Running with [-x -USEGPU=no] (GPU acceleration is NOT ACTIVE.) also not help. It behaves same.
However, CPU only serial version works fine, no problem. (--enable-openmp --disable-parallel, no cuda, use nvfortran from NVHPC for compiling)
There is no other message or error code.
Process viewer shows pw.x running with no problem, even It use minimum and GPU and CPU usage.