Tags

2.0.0-alpha.8

2.0.0-alpha.6

State before changing the algorithm to compute the optimal eta for the ewald summation and changes of the default width of the Gauss-charges.

2.0.0-alpha.5

State before consistent renaming of output properties and files (charge/chi instead of elec).

workshop

Stable version of RuNNer 2 for use in the internal RuNNer workshop from 2025-10-21 to 2025-10-23.

2.0.0-alpha.4

### Fixed
- Structures/Atoms should be significantly smaller now -> fewer chunks should be needed

### Changed
- The (smooth) overlap matrix can now be specified for each element independently. It
  is also possible to decide if the features are scaled with respect to the range
  over all features of the overlap matrix or with respect to this specific feature.
  I.e. if there are `<num_features>` (or `<num_eval>` for the non-smooth overlap matrix)
  is 120, the scaling factor can either be determined for each of them (120 factors)
  or for all of them (1 factor). The new keywords are:
  `<(smooth_)overlap_matrix> <element_symbol> (<num_features> <grid_length>) <num_eval>
   <num_s_orb> <num_p_orb> <scale_wrt2_all_features T/F> <neighbor_list_idx>`

# DEVLOG
### Changed
- Renamed `DataLoader` to `DataBatcher` everywhere in the code.
- Generalized the `QeqExtraFeatureCalculator` as the `GlobalFeatureCalculator`.
- Feature maps should not store intermediary results on the atom type anymore.
  Instead they should implement their own caching mechanism. Such a mechanism
  is only useful during traning if we calculate feature derivatives towards
  neighbors (force fraction smaller 50% and not all derivatives can be precomputed).
  In this case, the features of the same atom might be derived towards multiple
  neighbors. If some parts of the calculated derivatives are the same, they
  can/should be cached. The feature maps can hence make use of there cache 
  in their `first_derivative_wrt_to_on_neighbor` and/or
  `first_derivative_wrt_to_self` routines to avoid recalculating the same
  intermediate results multiple times.
  To achieve this each feature map is now required to implement
  a `allocate_cache(max_num_atoms_in_strucuture, num_structures_in_batch)` routine
  and a `free_cache()` routine to allocate/free the mentioned chache for each
  structure batch. If the feature map does not benefit from caching, it can of
  course just implement an empty dummy routine. Correspondingly, each routine
  which might call `first_derivative_wrt_to_on_neighbor` and/or
  `first_derivative_wrt_to_self` must make sure that
  `allocate_cache`/`free_cache` are called correspondingly. `allocate_cache`
  shall be called exactly once and before the first call to
  `first_derivative_wrt_to_on_neighbor` and/or `first_derivative_wrt_to_self`.
  The `free_cache` routine shall be called after each structure batch has been
  processed.

### Added
- Unit tests are now also run with `DEBUG=ON` in the CI pipeline.
- Regression tests (only the one within the repo) are now run with `DEBUG=ON`
  in the CI pipeline.

2.0.0-alpha.3

### Fixed
- Fixed issues related to hirshfeld volume training / prediction in the 3G.
  The initialization of the model indices is still not perfect, but will be
  changed anyway with the 4G.
- fixed a severe memory leak in the LAMMPS interface caused by a vector index
  into the `structures` array in the prediction routines. Calls to
  `FeatureCalculator%calculate_...` do not need a list of structures anymore.
- the LAMMPS interface no works when there are no atoms on an MPI process.
- the LAMMPS interface now always updates the number of local atoms on a process,
  even when the total number of atoms does not change.
- Correct mapping to the feature map name, min and max during the check of the feature map range 
- Rewrite of `add_line_breaks` should be correct and much more stable now
- Feature maps which had more than one feature ((smooth) overlap matrix and SF 1)
  were fundamentally broken. This is fixed now.
- fixed a memory bug. During prediction, if features could be precomputed but feature
  derivatives could not, memory was not properly deallocated. This also affected
  the RMSE calculation.
- fixed bug with automatic force_update_scaling: with DEBUG flags, the code
  crashed due to a division by zero.
- force training: if the estimated number of force updates in a structure batch
  is below 50% we now correctly use the towards_atom evaluation route of the
  `FeatureCalculator`. 
- fixed output in `*`.test.nnatoms.out/`*`.train.nnatoms.out files. Previously,
  the mapping was always wrong. Hence, the output was wrong if there was at least
  one test structure.
- fixed output in `*`.train.forces.out and `*`.test.forces.out during MPI training.
- fixed memory management issues related to the feature calculator during force
  training.
- Force training now correctly calculates only the necessary feature derivatives
  if a low force fraction is used, which increases efficiency.
- `max_num_structures_per_chunk_per_process` was too small for the case that number
  of train and test structures are both not dividable across the number of processes
  which lead to allocation of too small result class objects.
- fixed incorrect comparison operator in gradient clipping.
- Fix calculation of feature derivatives with respect to the target atom of a
  force training step. This may have affected training performance for large
  datasets when using a small force fraction.
- The derivatives of the loss functions are now fully consistent with the loss,
  i.e. are divided by the batch size.

### Added
- `epoch`.train/test.energy.out, `epoch`.train/test.forces.out and 
  `epoch`.train/test.nnatoms.out are now written for each `epoch` which is
   divisable by the argument given to `write_frequency` in input.nn and for
   the "optimal" epoch.
- It is reported which epoch is the "optimal" epoch (the one with the lowest
  energy training error).
- New unittest for results sort routine.
- d_chain_d_features is now OMP-parallelized.
- During training, the code now prints the elapsed time since the last RMSE was
  calculated (with default settings, this is the time for a single epoch).
- `feature_minmax_threshold` if functional again: if the range of the features is
  smaller than the threshold, an erro is thrown listing all feature maps which 
  lead to this error.
- More useful error messages if parsing the overlap matrix keywords fails.
- All feature maps, which produce a range of features which is below the set
  threshold, are now report together in the output and the program is only killed
  after all feature maps have been checked.

- 4G charge training (alpha stage):
  - implemented working 4G charge cost calculation and training loop including
    different batching strategies and training with and without fixed hardness. 
  - implemented separate 4G charge prediction routines using the
    `TrainingChargeEquilibrator`.
  - reverse QeQ based on reference charges for electronegativity initialization
    (as long as hardness is fixed). This is done by default if any `precondition`
    or advanced `initialization_method` is chosen.
  - New keyword `model_type_*`  for all possible suffixes. Can be either
    `elemental` or `hdnn`. This is mostly helpful to switch between an elemental
    hardness or environment-dependent hardness.
  - renamed file `hardness.data` to `weightshardness` for consistency.
  - Introduced `scalinghardness.data` file.
  - new activation function $f(x) = x^2$. Key is `sq`. This may be especially
    helpful for scaling electronegativitity, hardness, or charge nodes.
  - if hardness is not read from file, initialization of hardness is possible
    based on CRC handbook Pearson hardness values.
  - added additional printing of electronegativity statistics.
- improved memory scaling for 3G and 4G applications by getting rid of
  intermediate d_property_d_xyz arrays.
- new batchsize option `num_atoms_in_structure_batch`. Specify
  `batchsize_elements num_atoms_in_structure_batch` in your input.nn file.
- completely generalized and rewritten interface bindings based on more modular
  prediction workflows.
  - all `bindc(c)` routines are collected in `interface_bindings`
  - new routine `runner_interface_finalize_step` to handle graceful exit and
    printing of timings.
  - new routine `runner_interface_reinitialize_electrostatics` to reallocate
    charge equlibrators and electrostatics calculators only when necessary (e.g.
    when the number of atoms in the simulation changes).

- training and prediction are fully committee compatible. Specify
  `num_committee_members` in your input.nn. This also requires creating separate
  folders with continuous indices (starting from 1) in the calculation directory.
  These need to contain the weights files for each committee member when parsing
  weights from a file.
  - Separate optimizers can be assigned to each committee member.
    In case of separate optimizers for each atomic NN and committee member,
    `optimizer_comm [optimizer_idx] [committee_idx] [optimizer_type] ...` 
    must be specified, where `[optimizer_idx]` corresponds to the index given to
    `opt_short`. In case of default optimizers for all atomic NNs of
    each committee member, `default_optimizer_comm [committee_idx] [optimizer_type]`
    must be specified.
    Note: Mixing of `default_optimizer` with `default_optimizer` and
    `optimizer` with `optimizer_comm` for identical `[optimizer_idx]` is not
    possible and results in that an error is thrown for atomic NNs being assigned
    with more than one optimizer.

- training on the total energy of the structure (like in n2p2) is now possible by
  setting `pre_calc_energy_norm` to `1_over_num_atoms` and `pre_calc_force_norm`,
  `post_calc_energy_loss_norm`, `post_calc_energy_grads_norm`,
  `post_calc_force_loss_norm` and `post_calc_force_grads_norm` to `none`.
  Good training results are obtained with the non-fading memory (i.e. standard)
  Kalman Filter (`kalman_std`) with its default hyperparameters.
- in 4G charge training, the hardness weights/values are now written.
- when training an elemental hardness model, one may now specify `initialization_method`:
  - `read` for reading from file weightshardness.data
  - `uniform` and `normal` for initialization from a uniform or normal distribution
    whose range can be controlled via the keywords `parameter_min_hardness` and
    `parameter_max_hardness`.
  - `reference` for setting literature values for each element.
  - `reference_uniform` for initialization from a uniform distribution
    centered around each literature value with a range of 30% of that value.
  - `reference_normal` for initialization from a normal distribution
    centered around each literature value with a standard deviation of 10% of
    that value.
  In the case of a 4G hardness model, the literature values are automatically chosen
  to be the Pearson hardness values.

### Changed
- In the 3G prediction it is now neccessary to specify the keywords
  predict_long_range_energy
  and predict_short_range_energy explicitly. Previously, if specifying none of
  the two, both were assumed for convenience.
  Since this prevents predicting only hirshfeld volumes this was changed.
- Renamed `predict_long_range_energy` to `predict_electrostatics` and
  `predict_short_range_energy` to `predict_short_range` which are both `True` by
  default.
- renamed `use_vdw hirshfeld` to `predict_vdw hirshfeld`
- parsing of input.data file has been sped up.
- added committees to all output dimensions of generic interface
  - pair_style takes additional inline argument `total_charge`
- new keyword `constrain_predicted_charges_before_evaluation`. This turns on the
  projection to the total charge constraint during 3G charge training. `False`
  by default.
- changed the STDOUT during all training procedures.
- new keyword `cost_functions` which takes a space-separated list of cost
  function names (`rmse`, `mae`, `mse`). All given cost functions are used
  to calculate the cost for each epoch and print it in a separate line. 
- new keyword `optimal_epoch_property [property] [train|test]` to specify how
  the optimal epoch is determined. The first entry must be the name of the property
  to use and the second, optional column specifies whether train or test set
  are used.
- keywords `force_update_scaling`, `force_Fraction`, and `batchsize_forces` can
  now be specified element-specific, which is respected during training.
- new memory management strategy during training. Either all data is precomputed
  or none.
- descriptors can be shared between models. Specify `feature_map default` in your
  input.nn to exploit this behavior.
- hardness model can be initialized from input.nn
- LAMMPS interface is now built differently (see README.md).

- Elementwise batching and application of fractions during training was much
  improved. Relevant keywords are:
  - `batchsize_forces` or `batchsize_atomic_property` to control how many samples
    are put into one batch. Possible formats are:
    - `batchsize_forces [atomic_number] [value]` to set the batchsize for one
      specific element.
    - `batchsize_forces [value]` to define one global batchsize for all elements.
  - The keywords `force_fraction` or `atomic_property_fraction` to control how many
    samples are selected. They follow the same format as given above.
  - The logical keywords `batch_atomic_properties_by_element` and `batch_forces_by_element`
    as well as `apply_force_fraction_by_element` and `apply_atomic_property_fraction_by_element`.
    These control whether batching and fractions are done on a per-element basis or
    whether atoms are treated completely independently of their element. 
  - `batchsize_elements num_atoms_in_structure_batch` can still be used to make batches dynamically just
    large enough to accomodate all atoms in one structure batch. This is element-specific by default.

- The normalization of loss and its gradients for energy and force batches have been remodeled.
  Here, the reference and predicted quantities can be normalized by the number of atoms in the
  respective structure (`num_atoms`) before the calculation of the loss and its gradients and
  are controlled by the `pre_calc_[energy,force]_norm` keywords. Alternatively, these quantities
  can be multiplied with number of atoms by passing `1_over_num_atoms`, enabling the training on
  the total energy of the structure like in n2p2.
  Additionally, the loss and its gradients can be normalized after their calculation with the
  number of atoms (`num_atoms_batch`) or backward passes (`num_backward_passes_batch`)
  in the respective structure batch and are controlled by the
  `post_calc_[energy,force]_[loss,grads]_norm` keywords.

  A summary of the available keywords is shown here,
  | Keyword | Available values | Default |
  |:-------:|:----------------:|:-------:|
  | `pre_calc_energy_norm`  | `none`, `num_atoms`, `1_over_num_atoms` | `none` |
  | `pre_calc_force_norm`  | `none`, `num_atoms`, `1_over_num_atoms` | `num_atoms` |
  | `post_calc_energy_loss_norm`  | `none`, `num_atoms_batch`, `num_backward_passes_batch` | `num_atoms_batch` |
  | `post_calc_energy_grads_norm`  | `none`, `num_atoms_batch`, `num_backward_passes_batch` | `none` |
  | `post_calc_force_loss_norm`  | `none`, `num_atoms_batch`, `num_backward_passes_batch` | `none` |
  | `post_calc_force_grads_norm`  | `none`, `num_atoms_batch`, `num_backward_passes_batch` | `num_atoms_batch` |

  - `none` means no normalization, i.e. multiplication by 1.
  - The keywords `normalize_loss_energy`, `dont_normalize_gradients_energy`
    `dont_normalize_loss_forces`, `dont_normalize_gradients_forces` and
    `gradient_divisor` are now no longer supported.

2.0.0-alpha.2

### Fixed
- Renamed `AngularSymmetryFunction9` to `AngularSymmetryFunction3` in the code.
  No changes to input.nn are necessary.
- the weight initialization (see "changed" for more details).
- 3G and Hirshfeld prediction via LAMMPS interface are working correctly (again).
- Segfaults in `LayerDense` get_parameters and `set_parameters` for the latest
  ifx and gfortran compilers were fixed by using `present_and_true`.
- 4G stress is now working correctly again. It was previously false for all but
  the first atom for some (rather unusual) training datasets.
- 3G training is using the correct feature calculator setup and weight
  initialization again.
- Fixed calculation of long-range contributions for 3G training.
- 2G training update order was fixed to retain the same behavior as RuNNer 1.3.
- Fixed various compiler warnings with gfortran, ifort, and ifx.
- All 2G and 4G regression tests in the examples repo are now compatible with
  the latest version of RuNNer again. 
- The number of updates during parallel training is now counted correctly.
- Fixed array bound mismatches and an index bug during neighbor list transfer
  in the LAMMPS interface.
- `energy_fraction` keyword is functional again.
- writing of features to a binary dump or an ASCII file has been fixed.
- the calculation of loss and loss gradients during energy and force training
  for `batchsize_structure` > 1 has been fixed.
- A double-counting mistake in `calculate_feature_derivatives_towards_atom` has
  been fixed.

### Added
- This CHANGELOG file.
- Angular SF type 9 according to the RuNNer documentation, called  
  `AngularSymmetryFunction9` in the code and corresponding unit tests.
  Request it in input.nn with e.g. `symfunction_short H 9 O O ...`.
- Add the option to turn `calculate_forces` and/or `calculate_stress` on during
  training. 
  - This enables the user to calculate the force RMSE during energy training
    without using forces for an update.
  - By default, both keywords are turned off. `use_forces` will implicitely
    turn on `calculate_forces`.
  - Turning on `calculate_stress` is possible, but no additional output files
    will be written at this point.
  - If either option is turned on, a warning message will automatically be added
    to the global warning handler.
- `NeighborGrid` type for handling gridwise loops over neighbors in a
  single structure
- application of `NeighborGrid` type in `find_neighbors_grid`, the PPPM solver
  real-space contribution and all Ewald summation real-space contributions.
- LAMMPS interface:
  - 4G prediction
  - unit conversion read from LAMMPS input file. The keywords are `cflength` and
    `cfenergy`. They work exactly as in n2p2.
  - new keyword `dir` for specifying the path to the RuNNer input files. This
    works the same as in n2p2.
- additional CI stage that catches all compiler warnings.
- all regression tests in the examples repo now run as part of the CI pipeline
  (has to be manually triggered and is required for any merge to main).
- New unittest for results sort routine.
- d_chain_d_features is now OMP-parallelized.
- During training, the code now prints the elapsed time since the last RMSE was
  calculated (with default settings, this is the time for a single epoch).

### Changed
- separated the `AbstractOptimizer` and `AbstractMLModel` classes from each other.
- removed the `AbstractMLModel` class
  - makes the hierarchy of the model classes much more flat.
  - The previously type-bound procedures are translated into a set of more
    generic `initialize_array` routines in `mod_initialize_array`.
- modified the `get_parameters`, `set_parameters`, and
  `get_gradients_wrt_parameters` interfaces of the `AbstractPropertyModel` class.
  They now take the `atomic_numbers` and `optimizer_idx` as additional optional
  arguments.
- This allowed us to completely remove the `trainable_models` variable
  (which was messy).
- changed the `update_optimizers` building block accordingly. The interfaces
  should new be much clearer and work for a single and multiple models.
- the optimizers are read from input.nn in a different format. Instead of

  ```
  optimizer_types 4 4
  kalman_nue 0.98
  ...
  ```

  it is now

  ```
  optimizer 1 kalman nue=...
  opt_short 28 1
  ```

  This works for all optimizers. This makes it very easy to train models with
  different combinations of optimizers and models.

- the `gradient_divisor` keyword now also takes the value `num_atoms` in
  input.nn (previously only `num_backward_passes`).
  The default is `num_atoms`, which is the behavior of RuNNer 1.

- many bugs in the weight initilization were fixed. The weight initialization is
  now much more stable and should work for most cases.
- the syntax for the weight initialization in input.nn was changed. Use

  ```
  initialization_method <method for hidden layers> <method for output layer>
  ```
  supported values for `<method for hidden layers>` are:
    - 'read': Read weights from file. Turns of preconditioning.
    - 'xavier' or 'glorot': Xavier initialization
    - 'xavier_modified' or 'glorot_modified': Modified Xavier initialization
    - 'nguyen_widrow': Nguyen-Widrow initialization
    - 'nguyen_widrow_legacy' or 'nguyen_widrow_runner1': Nguyen-Widrow
      initialization as in RuNNer 1
    - 'uniform': Uniform initialization
    - 'normal' or 'gaussian': Normal initialization
    - 'zero': Initialize all weights and biases to zero

  supported values for `<method for output layer>` are:
    - 'eckhoff': As described in J. Chem. Phys. 153, 164107 (2020)
    - 'eckhoff_modified': As described in J. Chem. Phys. 153, 164107 (2020)
    - 'uniform': Draw values from a uniform distribution.
    - 'normal' or 'gaussian': Draw values from a normal distribution.
    - 'nw_last_layer_runner1', 'uniform05' or 'nw_last_layer_legacy':
       Uniform initialization from -0.5 to 0.5 as it was done in RuNNer 1
       if Nguyen-Widrow was applied.
    - 'zero': Initialize all weights and biases to zero
    - 'precondition_anyway': Only allowed if `<method for hidden layers>` is
      `read`. Preconditioning is applied even if the weights are read from file.

- the syntax for the preconditioning in input.nn was changed. Use

  ```
  precondition_method <method>
  ```
  the only currently supported value for `<method>` is `default` which is the
  behavior of RuNNer 1.

- old initialization keywords (use_old_weights, precondition_weights...) were
  removed and will throw an error if used.
  This is more consistent with the other keywords in input.nn.
- energy.out prints an additional column for the vdw energy and for the sum of
  total energy and VDW contribution. This will be changed again in the near
  future.
- Significant improvement of the prediction and training performance, most
  importantly for the LAMMPS interface.
  - FeatureMap interfaces were all changed from functions to subroutines.
  - sorting of features and feature derivatives is now only applied during
    packing or chaining, not directly during the calculation. This dramatically
    improves the ability of the compiler to vectorize the angular symmetry function
    derivative calculations for type 3 and type 9 which results in an almost 300%
    performance increase for predictions through the LAMMPS interface.
  - All loops were checked and possibly rearranged for optimal speedup.
  - Additional bottlenecks in large-scale LAMMPS simulations were identified
    and resolved.
- removed the `DynamicFeatureMapArray` class (which caused an ICE with ifort
  2021.7.0)
- p_initial (Kalman correlation matrix initialization) defaults to 1.0
- The results class has been generalized and allows for storage, MPI collecation
  and summation of arbitrary atomic and structural properties.

2.0.0-alpha.1

### Added

- ACSF & Overlap Matrix Feature Maps with derivatives. Spin-or electric field prefactors.
- calculation and writing of feature maps and their derivatives to binary and ASCII file (mode 1)
- 2G prediction (serial & MPI)
- 2G energy and force training (serial & MPI)
- most fitting and prediction options
- batchwise processing and iteration over chunks of structures
- 3G prediction: charges, long-range electrostatic contributions, short-range
  contributions, and all combined (serial & MPI)
- 3G training of charges
- 3G training of energies and forces after removing long-range electrostatic
  contributions
- 4G prediction: charges, long-range electrostatic contributions, short-range
  contributions with force trick, and all combined (serial & MPI)
- prediction of Hirshfeld volumina
- training of Hirshfeld volumina
- OpenMP parallelization for feature maps and intrinsic parallelization of
  matrix multiplications in forward/backward pass and Kalman filter optimization

### Known Issues

- faulty weight initialization in comparison to RuNNer 1. In consequence poor
  training behavior for large-scale datasets when using a high P initial value
  (use p_initial < 0.01).
- sys_mem test in the CI pipeline fails randomly when other programs are running
- in separate pipelines (just restart the pipeline until it works).

### Work in Progress
- 4G charge training
- 4G energy and force training
- committee prediction