Skip to content

Enable large grids

Sebastian Ohlmann requested to merge mesh_i8 into develop

Description

Use 8-byte integers throughout the code for global array sizes and indices. This enables grids with more than about a billion points - the barrier was around (2^31-1)/3 (about 700 million) before. I have tested large grids for the following cases:

  1. Si with 1500^3 points (~3.3 billion points):
  • a few SCF iterations
  • a few TD steps
  1. C allelectron run (~2.1 billion points, fft grid 3375^3)
  • a few SCF iterations
  • a few TD steps
  1. Maxwell runs with a few timesteps
  • about 1430^3 = 2.9 billion points
  • about 2000^3 = 8 billion points
  1. Al spin-polarized (~2.6 billion points, fft grid 3465^3)
  • a few SCF iterations
  • a few TD steps
  1. Al spinor (~2.6 billion points, fft grid 3465^3)
  • a few SCF iterations
  • a few TD steps

Important points:

  • these runs need a lot of memory
  • 64bit versions of METIS and ParMETIS required for more than 2 billion points
  • pfft is very useful for the Poisson solver, otherwise the memory requirements are even larger (normal Poisson solvers need a few global functions extra on each core)
  • use only Pardomains for TD runs
  • no LCAO, also needs a lot of memory
  • output takes long, should be kept to minimum
  • output size is big (>600 GB of restart data alone for Si 1500^3)

News snippet

Enable large grids with more than about 1 billion points

Checklist

  • I have checked that my code follows the Octopus coding standards
  • I have added tests for all the new features added in this request.
Edited by Sebastian Ohlmann

Merge request reports