Band unfolding memory requirements
Hi,
I have been trying to use the band unfolding calculation part and been running into issues right at unfold.spectral_function(). Looking at the abrupt stop, I guess it is mostly a memory issue. I am running on a 32 core job with close to 191Gb space. It seems that get_spectral_weights() function in unfold part of the code is creating the ruckus. Any insight into this problem would be greatly appreciated.
Options I tried, but in vain
- reducing k points
- increasing memory
- increasing/reducing parallel cores
Thanks a bunch !
Update 1
Drastically reducing the kpoint to 10 seems to work, confirming more that the memory allocation is probably the issue? This is the loop from get_spectral_weights() that is creating the problem even before a single cycle of k points (i.e. even the first iteration of ik fails in 32 core 191 Gb system)
e_Km = np.zeros((Nk, Nb))
P_Km = np.zeros((Nk, Nb))
myk = range(0, Nk)[world.rank::world.size]
for ik in myk:
k = kpoints[ik]
print('kpoint: %s' % k)
K_c, G_c = find_K_from_k(k, self.M)
iK = self.get_K_index(K_c)
e_Km[ik] = self.get_eigenvalues(iK)
P_Km[ik] = self.get_spectral_weights_k(k)
Update 2
Upon further debugging, the problem seems to be from get_rs_wavefunctions_k(calc, iK, spinorbit=False, v_Knm=None):
happening at
if calc.wfs.mode == 'lcao' and not calc.wfs.positions_set:
calc.initialize_positions()
Where the code is being stuck at calc.initialize_positions()
. I am now confused as to how this should be dependent on k-mesh.
Original Code
from gpaw import GPAW,restart
from gpaw.unfold import Unfold, find_K_from_k
from gpaw import GPAW, FermiDirac
import numpy as np
from ase.parallel import parprint
a,calc=restart('licoo2_gs.gpw')
n=3
M=np.array([
[n,0,0],
[0,n,0],
[0,0,n]
])
bp = a.cell.bandpath(npoints=48)
x, X, name = bp.get_linear_kpoint_axis()
Kpts = []
for k in bp.kpts:
K = find_K_from_k(k, M)[0]
Kpts.append(K)
if True:
parprint("running bands calculation")
calc_bands = GPAW('licoo2_gs.gpw',
fixdensity=True,
kpts=Kpts,
symmetry='off',
nbands=-90,
convergence={'bands': -90})
calc_bands.get_potential_energy()
calc_bands.write('bands_3x3_defect.gpw', 'all')
parprint("\n Unfolding......")
unfold = Unfold(name='3x3_defect',
calc='bands_3x3_defect.gpw',
M=M,
spinorbit=False)
unfold.spectral_function(kpts=bp.kpts, x=x, X=X,
points_name=bp.get_linear_kpoint_axis()[2])
parprint("\n Done......!")
Output
>Unfolding
> kpoint: [ 0.26849791 -0.26849791 0.5 ]
> kpoint: [ 0.0894993 -0.0894993 0.5 ]
> kpoint: [ 0.03815291 -0.03815291 0. ]
> kpoint: [ 0.37350489 -0.31324756 0.18675244]
> kpoint: [ 0. -0.35714286 0. ]
> kpoint: [0. 0. 0.5]
> kpoint: [ 0. -0.21428571 0. ]
> kpoint: [0. 0. 0.16666667]
> kpoint: [ 0. -0.07142857 0. ]
> kpoint: [ 0.18675244 -0.40662378 0.09337622]
> kpoint: [ 0.1789986 -0.1789986 0.5 ]
> kpoint: [ 0. -0.5 0. ]
> kpoint: [ 0.04474965 -0.04474965 0.5 ]
> kpoint: [ 0. -0.14285714 0. ]
> kpoint: [ 0. -0.28571429 0. ]
> kpoint: [ 0. -0.42857143 0. ]
> kpoint: [ 0.09337622 -0.45331189 0.04668811]
> kpoint: [ 0.07630583 -0.07630583 0. ]
> kpoint: [0. 0. 0.33333333]
> kpoint: [ 0.15261165 -0.15261165 0. ]
> kpoint: [ 0.31324756 -0.31324756 0.5 ]
> kpoint: [ 0.30522331 -0.30522331 0. ]
> kpoint: [ 0.28012867 -0.35993567 0.14006433]
> Getting EigenValues and Weights
> kpoint: [0. 0. 0.]
> kpoint: [ 0.13424895 -0.13424895 0.5 ]
> kpoint: [ 0.22374825 -0.22374825 0.5 ]
> kpoint: [0. 0. 0.]
> kpoint: [ 0.22891748 -0.22891748 0. ]
> kpoint: [ 0.11445874 -0.11445874 0. ]
> kpoint: [ 0.34337622 -0.34337622 0. ]
> kpoint: [ 0.26707039 -0.26707039 0. ]
> kpoint: [ 0.19076457 -0.19076457 0. ]
>
> mpirun noticed that process rank 11 with PID 201528 on node compt294 exited on signal 9 (Killed).