Commit a2df810a authored by Jan Oliver Oelerich's avatar Jan Oliver Oelerich

A bunch of improvements of the Documentation

parent 55759063
......@@ -16,7 +16,7 @@ Highlights
- Speed improvements by increasing the grid sizes to match efficient FFT sizes. Note, that this may result
in a higher simulation grid density than specified in `grating.density` parameter!
- Alternative parallelization scheme, see :ref:`parallelization_scheme`. When appropriate, different MPI procs
- Alternative parallelization scheme, see :ref:`parallelization-scheme`. When appropriate, different MPI procs
now calculate different frozen phonon configurations / defoci in parallel. This reduces the required amount
of communication between the processors.
- Automatic calculation of `center of mass` of the CBEDs for all ADF points. The COMs are calculated when
......
add_custom_target(docs-manual
add_custom_target(docs
COMMAND sphinx-build -c ${CMAKE_CURRENT_SOURCE_DIR} -b html ${PROJECT_SOURCE_DIR}/docs manual
WORKING_DIRECTORY ${CMAKE_CURRENT_BINARY_DIR}
COMMENT "Build html documentation"
......
File formats
============
A *STEMsalabim* simulation is set-up via **input files** and its results are stored in an **output file**. The file for
A STEMsalabim simulation is set-up via **input files** and its results are stored in an **output file**. The file for
configuring a simulation is described in detail at :ref:`parameter-file`. Here, we describe the format of the **crystal
file**, i.e., the atomic information about the specimen, and the **output file**, in which the results are stored.
......@@ -46,42 +46,247 @@ Below is a very brief, artificial example (without custom slicing): ::
Output file format
------------------
All results of a *STEMsalabim* simulation are written to a binary `NetCDF <https://en.wikipedia.org/wiki/NetCDF>`_ file.
All results of a STEMsalabim simulation are written to a binary `NetCDF <https://en.wikipedia.org/wiki/NetCDF>`_ file.
The NetCDF format is based on the `Hierarchical Data Format <https://en.wikipedia.org/wiki/Hierarchical_Data_Format>`_
and there are libraries to read the data for many programming languages.
The structure of NetCDF files is hierarchical and organized in groups. The following groups are written by
*STEMsalabim*:
The structure of NetCDF files can be inspected using the handy tool ``ncdump -h YOUR_FILE.nc`` (don't forget the ``-h``
parameter, otherwise the whole content of the file is dumped!). Here is the output of an example run: ::
runtime
~~~~~~~
This group contains information about the program and the simulation, such as version, UUID and so on.
netcdf TestRUN {
group: AMBER {
dimensions:
atom = 320 ;
elements = 1 ;
spatial = 3 ;
cell_spatial = 3 ;
cell_angular = 3 ;
label = 6 ;
frame = 4 ;
slices = 11 ;
grid_x = 336 ;
grid_y = 336 ;
variables:
char spatial(spatial) ;
char cell_spatial(cell_spatial) ;
char cell_angular(cell_angular, label) ;
float coordinates(frame, atom, spatial) ;
coordinates:unit = "nanometer" ;
float lattice_coordinates(frame, atom, spatial) ;
float cell_lengths(frame, cell_spatial) ;
cell_lengths:unit = "nanometer" ;
float cell_angles(frame, cell_angular) ;
cell_angles:unit = "degree" ;
float radius(frame, atom) ;
radius:unit = "nanometer" ;
float msd(frame, atom) ;
int slice(frame, atom) ;
float slice_coordinates(slices) ;
short element(frame, atom) ;
float system_lengths(cell_spatial) ;
float system_angles(cell_spatial) ;
char atom_types(elements, label) ;
// group attributes:
:Conventions = "AMBER" ;
:ConventionVersion = "1.0" ;
:program = "STEMsalabim" ;
:programVersion = "5.0.0" ;
:title = "sim" ;
} // group AMBER
group: runtime {
// group attributes:
:programVersionMajor = "5" ;
:programVersionMinor = "0" ;
:programVersionPatch = "0" ;
:gitCommit = "5575906360b70117dffa4f5ae31d4eb3af34c5fa" ;
:title = "TestRUN" ;
:UUID = "22720f20-4816-4732-b287-627027bdcaff" ;
:time_start = "2019-01-24 14:23:09" ;
} // group runtime
group: params {
dimensions:
defocus = 2 ;
variables:
float defocus(defocus) ;
float defocus_weights(defocus) ;
// group attributes:
:program_arguments = "--num-threads 8 -p TestRUN.cfg -o TestRUN.nc" ;
:config_file_contents = "..." ;
group: application {
// group attributes:
:random_seed = 2984830537U ;
} // group application
group: simulation {
// group attributes:
:title = "sim" ;
:normalize_always = 0US ;
:bandwidth_limiting = 1US ;
:output_file = "TestRUN.nc" ;
:output_compress = 0US ;
} // group simulation
group: probe {
// group attributes:
:c5 = 5000000. ;
:cs = 2000. ;
:astigmatism_ca = 0. ;
:defocus = -2. ;
:fwhm_defoci = 6. ;
:num_defoci = 2U ;
:astigmatism_angle = 0. ;
:min_apert = 0. ;
:max_apert = 15.07 ;
:beam_energy = 200. ;
:scan_density = 40. ;
} // group probe
group: specimen {
// group attributes:
:max_potential_radius = 0.3 ;
:crystal_file = "crystal.xyz" ;
} // group specimen
group: grating {
// group attributes:
:density = 300. ;
:nx = 336U ;
:ny = 336U ;
:slice_thickness = 0.5431 ;
} // group grating
group: adf {
// group attributes:
:enabled = 1US ;
:x = 0.5, 0.7 ;
:y = 0.5, 0.7 ;
:detector_min_angle = 0. ;
:detector_max_angle = 200. ;
:detector_num_angles = 201U ;
:detector_interval_exponent = 1.f ;
:average_configurations = 0US ;
:average_defoci = 0US ;
:save_slices_every = 0U ;
} // group adf
group: cbed {
// group attributes:
:enabled = 1US ;
:x = 0., 1. ;
:y = 0., 1. ;
:size = 0U, 0U ;
:average_configurations = 1US ;
:average_defoci = 1US ;
:save_slices_every = 0U ;
} // group cbed
group: frozen_phonon {
// group attributes:
:number_configurations = 2U ;
:fixed_slicing = 1US ;
:enabled = 1US ;
} // group frozen_phonon
} // group params
group: adf {
dimensions:
adf_position_x = 9 ;
adf_position_y = 9 ;
adf_detector_angle = 201 ;
adf_defocus = 2 ;
adf_phonon = 2 ;
adf_slice = 1 ;
coordinate_dim = 2 ;
variables:
float adf_intensities(adf_defocus, adf_position_x, adf_position_y, adf_phonon, adf_slice, adf_detector_angle) ;
float center_of_mass(adf_defocus, adf_position_x, adf_position_y, adf_phonon, adf_slice, coordinate_dim) ;
double adf_probe_x_grid(adf_position_x) ;
double adf_probe_y_grid(adf_position_y) ;
double adf_detector_grid(adf_detector_angle) ;
double adf_slice_coords(adf_slice) ;
} // group adf
group: cbed {
dimensions:
cbed_position_x = 43 ;
cbed_position_y = 43 ;
cbed_k_x = 224 ;
cbed_k_y = 224 ;
cbed_defocus = 1 ;
cbed_phonon = 1 ;
cbed_slice = 1 ;
variables:
float cbed_intensities(cbed_defocus, cbed_position_x, cbed_position_y, cbed_phonon, cbed_slice, cbed_k_x, cbed_k_y) ;
double cbed_probe_x_grid(cbed_position_x) ;
double cbed_probe_y_grid(cbed_position_y) ;
double cbed_x_grid(cbed_k_x) ;
double cbed_y_grid(cbed_k_y) ;
double cbed_slice_coords(cbed_slice) ;
} // group cbed
}
The structure of NetCDF files is hierarchical and organized in groups. The following groups are written by
STEMsalabim:
AMBER
~~~~~
This group contains the atomic coordinates, species, displacements, radii, etc. for the complete crystal for each single
calculated frozen lattice configuration, as well as for each calculated defocus value. The AMBER group content is
compatible with the `AMBER specifications <http://ambermd.org/netcdf/nctraj.xhtml>`_. A *STEMsalabim* NetCDF file can
compatible with the `AMBER specifications <http://ambermd.org/netcdf/nctraj.xhtml>`_. A STEMsalabim NetCDF file can
be opened seamlessly with the `Ovito <http://www.ovito.org/>`_ crystal viewer.
.. csv-table::
:file: table_nc_amber.csv
runtime
~~~~~~~
.. csv-table::
:file: table_nc_runtime.csv
params
~~~~~~
All simulation parameters are collected in the ``params`` group as attributes.
.. note:: The ``params`` group contains subgroups with attributes that correspond exactly to the simulation
parameters as written, except
- ``/params/application/random_seed`` is set to the generated random seed
- ``/params/grating/nx`` and ``/params/grating/ny`` contain the simulation grid size used.
.. csv-table::
:file: table_nc_params.csv
adf
~~~
This group contains the simulated ADF intensities, the coordinates of the electron probe beam during scanning, the
detector angle grid that is used, and coordinates of the slices as used in the multi-slice algorithm.
.. csv-table::
:file: table_nc_adf.csv
cbed
~~~~
This group contains the simulated CBED intensities, the coordinates of the electron probe beam during scanning, k-space
grid, and coordinates of the slices as used in the multi-slice algorithm.
.. csv-table::
:file: table_nc_cbed.csv
Reading NC Files
----------------
......
......@@ -7,11 +7,11 @@ General information
Throughout this documentation we assume that you are familiar with the theoretical background behind the scanning
transmission electron microscope (STEM) to some degree. Also, we assume that you have some knowledge about the
UNIX/Linux command line and parallelized computation. *STEMsalabim* is currently not intended to be run on a desktop
UNIX/Linux command line and parallelized computation. STEMsalabim is currently not intended to be run on a desktop
computer. While that is possible and works, the main purpose of the program is to be used in a highly parallelized
multi-computer environment.
We took great care of making *STEMsalabim* easy to install. You can find instructions at :ref:`installing`. However, if
We took great care of making STEMsalabim easy to install. You can find instructions at :ref:`installing`. However, if
you run into technical problems you should seek help from an administrator of your computer cluster first.
.. _simulation-structure:
......@@ -19,7 +19,7 @@ you run into technical problems you should seek help from an administrator of yo
Structure of a simulation
-------------------------
There essence of *STEMsalabim* is to model the interaction of a focused electron beam with a bunch of atoms, typically
There essence of STEMsalabim is to model the interaction of a focused electron beam with a bunch of atoms, typically
in the form of a crystalline sample. Given the necessary input files, the simulation crunches numbers for some time,
after which all of the calculated results can be found in the output file. Please refer to :ref:`running` for notes
how to start a simulation.
......@@ -28,7 +28,7 @@ Input files
~~~~~~~~~~~
All information about the specimen are listed in the :ref:`crystal-file`, which is one of the two required input files
for *STEMsalabim*. It contains each atom's species (element), coordinates, and `mean square displacement
for STEMsalabim. It contains each atom's species (element), coordinates, and `mean square displacement
<https://en.wikipedia.org/wiki/Mean_squared_displacement>`_ as it appears in the `Debye-Waller factors
<https://en.wikipedia.org/wiki/Debye%E2%80%93Waller_factor>`_.
......@@ -39,7 +39,7 @@ microscope, detector, and all required simulation parameters. All these paramete
Output files
~~~~~~~~~~~~
The complete output of a *STEMsalabim* simulation is written to a `NetCDF
The complete output of a STEMsalabim simulation is written to a `NetCDF
<https://www.unidata.ucar.edu/software/netcdf/>`_ file. NetCDF is a binary, hierarchical file format for scientific
data, based on `HDF5 <https://support.hdfgroup.org/HDF5/>`_. NetCDF/HDF5 allow us to compress the output data and store
it in machine-readable, organized format while still only having to deal with a single output file.
......@@ -51,7 +51,7 @@ You can read more about the output file structure at :ref:`output-file`.
Hybrid Parallelization model
----------------------------
*STEMsalabim* simulations can be parallelized both via `POSIX threads <https://en.wikipedia.org/wiki/POSIX_Threads>`_
STEMsalabim simulations is parallelized both via `POSIX threads <https://en.wikipedia.org/wiki/POSIX_Threads>`_
and via `message passing interface (MPI) <https://en.wikipedia.org/wiki/Message_Passing_Interface>`_. A typical
simulation will use both schemes at the same time: MPI is used for communication between the computing nodes, and
threads are used for intra-node parallelization, the usual multi-cpu/multi-core structure.
......@@ -64,22 +64,31 @@ threads are used for intra-node parallelization, the usual multi-cpu/multi-core
Let us assume a simulation that runs on :math:`M` computers and each of them spawns :math:`N` threads.
There is a single, special *master thread* (the thread 0 of the MPI process with rank 0) that orchestrates the simulation,
i.e., manages and distributes work packages. All other threads (:math:`(M\times N)-1`) participate in the simulation. In
MPI mode, each MPI process writes results to its own temporary file, and after each frozen lattice configuration the
results are merged. Merging is carried out sequentially by each individual MPI processor, so that no race condition
is ran into. The parameter :code:`output.tmp_dir` (see :ref:`parameter-file`) should be set to a directory that is local
Depending on the simulation parameters chosen, STEMsalabim may need to loop through multiple frozen phonon configurations
and values of the probe defocus. The same simulation (with differently displaced atoms and different probe defocus) is
therefore typically run multiple times. There are three parallelization schemes implemented in STEMsalabim:
- When :math:`M == 1`, i.e., no MPI parallelization is used, all pixels (probe positions) are distributed among the
:math:`N` threads and calculated in parallel.
- Each MPI processor calculates *all* pixels (probe positions) of its own frozen phonon / defocus configuration, i.e.,
:math:`M` configurations are calculated in parallel. Each of the :math:`M` calculations splits its pixels between
:math:`N` threads (each thread calculates one pixel at a time).
This scheme makes sense when the total number of configurations (`probe.num_defoci` :math:`\times`
`frozen_phonon.number_configurations`) is much larger than or divisible by :math:`M`.
- A single configuration is calculated at a time, and all the pixels are split between all :math:`M \times N` threads.
In order to reduce the required MPI communication
around, only the main thread of each of the :math:`M` MPI processors communicates with the master thread. The master
thread sends a *work package* containing some number of probe pixels to be calculated to an MPI process, which then
carries out all the calculations in parallel on its :math:`N` threads. When a work package is finished, it requests another
work package from the master MPI process until there is no work left. In parallel, the worker threads of the MPI process
with rank 0 also work on emptying the work queue.
In MPI mode, each MPI process writes results to its own temporary file, and after each frozen lattice configuration the
results are merged. Merging is carried out sequentially by each individual MPI processor, to avoid race conditions.
The parameter :code:`output.tmp_dir` (see :ref:`parameter-file`) should be set to a directory that is local
to each MPI processor (e.g., :code:`/tmp`).
A typical *STEMsalabim* simulation is composed of many independent multi-slice simulations that differ only in the
position of the scanning probe. Hence, parallelization is done on the level of these multi-slice simulations, with each
thread performing them independently from other threads. In order to reduce the number of MPI messages being sent
around, only the main thread of each of the :math:`M` MPI processors communicates with the master thread. The master
thread sends a *work package* containing some number of probe pixels to be calculated to an MPI process, which then
carries out all the calculations in parallel on its :math:`N` threads. When a work package is finished, it requests another
work package from the master MPI process until there is no work left. In parallel, the worker threads of the MPI process
with rank 0 also work on emptying the work queue.
.. note:: Within one MPI processor, the threads can share their memory. As the main memory consumption comes from storing
the weak phase objects of the slices in the multi-slice simulation, which don't change during the actual simulation,
this greatly reduces memory usage as compared to MPI only parallelization. You should therefore always aim for
......
.. _reporting:
HTTP Status reporting
=====================
*STEMsalabim* simulations may take a long time, even when running them in parallel on many processors. In order to ease
tracking of the status of running simulations, we built reporting via HTTP POST requests into the program.
In order to use that feature, the `libCURL <https://curl.haxx.se/libcurl/>`_ library has to be installed and
*STEMsalabim* needs to be linked against it.
.. _configure-reporting:
Configure HTTP reporting
------------------------
To configure HTTP reporting, please add the ``http_reporting: {}`` block to your simulations's parameter file,
containing at least ``reporting = true;`` and the url to report to, ``url = "http://my_server_address:port/path";``.
If you want to use `HTTP basic authentication <https://en.wikipedia.org/wiki/Basic_access_authentication>`_,
you may also specify the options ``auth_user = "your_user";`` and ``auth_pass = "your_pass";``. Note, that HTTP basic
auth will be enabled as soon as ``auth_user`` is not empty. You should therefore only fill in that field when you want
to use authentication.
Additional, custom payload for the HTTP requests may be specified in the sub-block ``parameters: {}``. Each key-value
pair in this block is translated into JSON and appended to each request. This allows you to use custom authentication
techniques, such as token-based authentication
etc.
An example configuration block with HTTP basic authentication may look like this:
::
http_reporting: {
reporting = true;
url = "http://my_api_endpoint:8000/stemsalabim-reporting";
auth_user = "my_user";
auth_pass = "my_pass";
parameters: {
simulation_category = "suitable for many Nature papers";
}
}
The status requests
-------------------
In each request that *STEMsalabim* sends, some JSON payload is common. In addition to the JSON values specified in the
parameter file (see :ref:`configure-reporting`), the following parameters are always reported:
::
time: // The currrent date and time
id: // the UUID of the simulation
num_threads: // the number of threads of each MPI processor
num_processors: // the number of MPI processors
num_defoci: // the total number of defoci to calculate
num_configurations: // the total number of frozen phonon configurations to calculate
event: // A code for what event is reported. see below.
The following four different ``event`` codes, each for a different event, are reported:
``START_SIMULATION``: A simulation is started
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
This request is sent at the beginning of a simulation. Additional key/value pairs sent are:
::
event: "START_SIMULATION"
version: // program version
git_commit: // git commit hash of the program version
title: // simulation title
``START_DEFOCUS``: A defocus iteration is started
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
This request is sent at the beginning of a defocus iteration. Additional key/value pairs sent are:
::
event: "START_DEFOCUS"
defocus: // the defocus value in nm
defocus_index: // the index of the defocus, between 0 and num_defoci
``START_FP_CONFIGURATION``: A frozen phonon iteration is started
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
This request is sent at the beginning of a frozen phonon configuration. Additional key/value pairs sent are:
::
event: "START_FP_CONFIGURATION"
defocus: // the defocus value in nm
defocus_index: // the index of the defocus, between 0 and num_defoci
configuration_index: // the index of the configuration, between 0 and num_configurations
``PROGRESS``: Progress report
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
This request is sent during the calculation, typically after each integer percent of the simulation finished.
Additional key/value pairs sent are:
::
event: "START_CONFIGURATION"
defocus: // the defocus value in nm
defocus_index: // the index of the defocus, between 0 and num_defoci
configuration_index: // the index of the configuration, between 0 and num_configurations
progress: // progress between 0 and 1 of this configuration iteration within this defocus iteration
``FINISH``: Simulation finished
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
This request is sent when the simulation finished. Additional key/value pairs sent are:
::
event: "FINISH"
How to process the reports
--------------------------
Obviously, in order to register the requests, an HTTP(S) server needs to be running on the target machine. For example,
a very simple server in python using the `http://flask.pocoo.org/ <flask>`__ package, that only echos the requests,
can be implemented as:
::
#!/usr/bin/env python
from flask import Flask
from flask import request
import json
app = Flask(__name__)
@app.route('/', methods=['POST'])
def echo():
content = request.get_json()
print(json.dumps(content, indent=4))
return ""
if __name__ == "__main__":
app.run()
Run the script and then start a *STEMsalabim* simulation to see requests imcoming.
STEMsalabim
===========
The *STEMsalabim* software aims to provide accurate scanning transmission electron microscopy (STEM) image simulation of
The STEMsalabim software aims to provide accurate scanning transmission electron microscopy (STEM) image simulation of
a specimen whose atomic structure is known. It implements the frozen lattice multi-slice algorithm as described in
great detail in the book `Advanced computing in electron microscopy <http://dx.doi.org/10.1007/978-1-4419-6533-2>`_ by
Earl J. Kirkland.
......@@ -10,7 +10,7 @@ While there are multiple existing implementations of the same technique, at the
suitable for leveraging massive parallelization available on high-performance computing (HPC) clusters, making it
possible to simulate large supercells and parameter sweeps in reasonable time.
The purpose of *STEMsalabim* is to fill this gap by providing a multi-slice implementation that is well parallelizable
The purpose of STEMsalabim is to fill this gap by providing a multi-slice implementation that is well parallelizable
both within and across computing nodes, using a mixture of threaded parallelization and message passing interface (MPI).
......@@ -18,10 +18,13 @@ both within and across computing nodes, using a mixture of threaded parallelizat
:maxdepth: 2
:caption: Getting Started
what
install
usage
visualization
bla
.. toctree::
:maxdepth: 2
:caption: More information
......@@ -29,7 +32,6 @@ both within and across computing nodes, using a mixture of threaded parallelizat
general
parameters
file_formats
http_reporting
faq
whats_new
citing
......@@ -39,7 +41,7 @@ both within and across computing nodes, using a mixture of threaded parallelizat
Contact us!
===========
*STEMsalabim* is a relatively young software package and was not heavily tested outside the scope of our group.
STEMsalabim is a relatively young software package and was not heavily tested outside the scope of our group.
We are glad to help you getting your simulations to run.
Please contact **strl-stemsalabim [at] lists.uni-marburg.de** for support or feedback.
......@@ -47,13 +49,13 @@ Please contact **strl-stemsalabim [at] lists.uni-marburg.de** for support or fee
Credits
=======
* We acknowledge the creators of the supplementary libraries that *STEMsalabim* depends on.
* We acknowledge the creators of the supplementary libraries that STEMsalabim depends on.
* We would also like to acknowledge the creators of `STEMsim <http://dx.doi.org/10.1007/978-1-4020-8615-1_36>`_,
which we used as a reference implementation to test *STEMsalabim*.
which we used as a reference implementation to test STEMsalabim.
* Once again, we would like to highlight the book
`Advanced computing in electron microscopy <http://dx.doi.org/10.1007/978-1-4419-6533-2>`_ by Earl J. Kirkland for
its detailed description of the implementation of multi-slice algorithms.
* *STEMsalabim* was written in the `Structure & Technology Research Laboratory <https://www.uni-marburg.de/wzmw/strl>`_
* STEMsalabim was written in the `Structure & Technology Research Laboratory <https://www.uni-marburg.de/wzmw/strl>`_
of the `Philipps-Universität Marburg <https://www.uni-marburg.de/>`_ with financial support by
the `German Research Foundation <http://www.dfg.de/en/>`_
......
......@@ -3,37 +3,20 @@
Installing STEMsalabim
======================
Downloading the source code
---------------------------
We recommend you download the latest stable release (|release|) from the
`Releases page <https://gitlab.com/STRL/STEMsalabim/tags>`_. If you want the latest features and/or bugfixes,
you can also clone the repository using
::
$ git clone https://gitlab.com/STRL/STEMsalabim.git
$ git checkout devel # only if you want the devel code.
Requirements
------------
The following libraries and tools are needed to successfully compile the code:
* A C++11 compiler (such as `gcc/g++ <https://gcc.gnu.org/>`_ or `intel mkl <https://software.intel.com/en-us/mkl>`_).
* A C++11 compiler (such as `gcc/g++ <https://gcc.gnu.org/>`_ or `intel compiler suite <https://software.intel.com/en-us/compilers>`_).
* `CMake <https://cmake.org/>`_ > 3.3
* `NetCDF <https://www.unidata.ucar.edu/software/netcdf/>`_
* `libConfig <http://www.hyperrealm.com/libconfig/>`_ >= 1.5
* `FFTW3 <http://www.fftw.org/>`_
* `FFTW3 <http://www.fftw.org/>`_ or `Intel's MKL <https://software.intel.com/en-us/mkl>`_
* An MPI implementation (such as `OpenMPI <http://www.open-mpi.de/>`_)
The following libraries are *optional* and are needed only to enable additional features:
* `libCurl <https://curl.haxx.se/libcurl/>`_ (required for HTTP POST status announcements)
.. note:: You may find some of the requirements in the repositories of your Linux distribution, at least the compiler,
CMake, libCurl and OpenMPI. On Debian or Ubuntu Linux, for example, you can simply run the following command
CMake, and OpenMPI. On Debian or Ubuntu Linux, for example, you can simply run the following command
to download and install all the requirements:
::
......@@ -42,14 +25,26 @@ The following libraries are *optional* and are needed only to enable additional
libconfig++-dev \
libfftw3-dev \
libnetcdf-dev \
libcurl4-openssl-dev \
libopenmpi-dev \
openmpi-bin
.. Tip:: As the main work of the STEM image simulations is carried out by the `FFTW3 <http://www.fftw.org/>`_
library, you may reach best performance when you compile the library yourself with all available CPU level
optimizations enabled.
.. Tip:: Most of the computing time is spent calculating Fourier transforms, so it is beneficial for STEMsalabim
to use optimized FFT libraries. Sometimes, compiling FFTW or MKL on the target machine enables
optimizations that are not available in precompiled binaries, so this may be worth a try.
Downloading the source code
---------------------------
We recommend you download the latest stable release (|release|) from the
`Releases page <https://gitlab.com/STRL/STEMsalabim/tags>`_. If you want the latest features and/or bugfixes,
you can also clone the repository using
::
$ git clone https://gitlab.com/STRL/STEMsalabim.git
$ git checkout devel # only if you want the devel code.
Building STEMsalabim
......@@ -99,17 +94,17 @@ You are now ready to execute your first simulation.
Building with Intel MKL, Intel compiler (and Intel MPI)
-------------------------------------------------------
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
It is possible to use the `Intel® Parallel Studio <https://software.intel.com/en-us/parallel-studio-xe>`_
for compilation, which includes the `Intel® Math Kernel Library (MKL) <https://software.intel.com/en-us/mkl>`_
that *STEMsalabim* can use for discrete fourier transforms instead of FFTW3. If the
that STEMsalabim can use for discrete fourier transforms instead of FFTW3. If the
`Intel® MPI Library <https://software.intel.com/en-us/intel-mpi-library>`_ is also available, it can be used
as the MPI implementation in *STEMsalabim*.
for MPI communication.
.. note:: We have tested compiling and running *STEMsalabim* only with Parallel Studio 2017 so far.
.. note:: We have tested compiling and running STEMsalabim only with Parallel Studio 2017 so far.
*STEMsalabim*'s CMake files try to find the necessary libraries themselves, when the folling conditions are true:
STEMsalabim's CMake files try to find the necessary libraries themselves, when the folling conditions are true:
1. Either the environment variable :code:`MKLROOT` is set to a valid install location of the MKL, or
the CMake variable :code:`MKL_ROOT` (pointing at the same location) is specified.
......@@ -122,10 +117,14 @@ For example, let's say the Intel suite is installed in :code:`/opt/intel` and we
$ export PATH=$PATH:/opt/intel/... # mpicxx and icpc should be in the path!
$ LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/opt/gcc-6.3/lib64 \
cmake ../source -DMKL_ROOT=/opt/intel -DCMAKE_CXX_COMPILER=icpc -DGCCDIR=/opt/gcc-6.3
cmake ../source \
-DMKL_ROOT=/opt/intel \
-DCMAKE_CXX_COMPILER=icpc \
-DGCCDIR=/opt/gcc-6.3 \
-D... more CMAKE arguments as described above.
Depending on how your environment variables are set, you may be able to skip the :code:`LD_LIBRARY_PATH=..` part.
When *STEMsalabim* is executed, you may again need to specify the library path of the :code:`libstdc++`, using ::
When STEMsalabim is executed, you may again need to specify the library path of the :code:`libstdc++`, using ::
$ LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/opt/gcc-6.3/lib64 mpirun -np ... /path/to/stemsalabim -p ...
......
This diff is collapsed.
**Dimensions**,
adf_position_x,Number of probe positions in x direction
adf_position_y,Number of probe positions in y direction
adf_detector_angle,Number of stored detector angle bins
adf_defocus,Number of stored defoci (1 when averaged over defoci)
adf_phonon,Number of stored frozen phonon configuration (1 when averaged over configurations)
adf_slice,Number of stored slices
coordinate_dim,"x,y coordinate dimension (2)"
**Variables**,
adf_intensities,ADF intensities of each pixel ``[fraction of beam]``
center_of_mass,Center of mass of each pixel ``[mrad]``
adf_probe_x_grid,Position vector of the probe in x direction ``[nm]``
adf_probe_y_grid,Position vector of the probe in y direction ``[nm]``
adf_detector_grid,Lower angles of the detector bins ``[mrad]``
adf_slice_coords,Coordinates of the stored slices ``[nm]``
**Attributes**,
Conventions,"String ""AMBER"" (required for AMBER)"
ConventionVersion,Version of the AMBER spec.
program,"Program name (""STEMsalabim"")"
programVersion,STEMsalabim's version
title,Simulation title (Param ``simulation.title``)
**Dimensions**,
atom,Number of atoms
elements,Number of different elements
spatial,Number of spatial dimensions (3)
cell_spatial,Number of spatial dimensions (3)
cell_angular,Number of spatial dimensions (3)
label,Character String for element names (6)
frame,Number of frozen phonon configurations * number of defoci
slices,Number of slices in the multi-slice approximation
grid_x,Number of simulation grid points in x direction
grid_y,Number of simulation grid points in y direction
**Variables**,
spatial,"Names of the spatial dimensions (""x,y,z"")"
cell_spatial,"Names of the spatial cell parameters (""a,b,c"")"
cell_angular,"Names of the cell angles (""alpha,beta,gamma"")"
coordinates,Coordinates of the atoms ``[nm]``
lattice_coordinates,Equilibrium coordinates of the atoms (i.e. lattice positions without displacements) ``[nm]``
cell_lengths,Cell lengths (Same for each frame) ``[nm]``
cell_angles,"Cell angles (Same for each frame, always ""90, 90, 90"") ``[degree]``"
radius,Radii of each atom ``[nm]``
msd,Mean square displacement of each atom ``[nm^2]``
slice,Slice id of each atom
slice_coordinates,z-Coordinate of each slice ``[nm]``
element,Element id of each atom (see ``atom_types``)
system_lengths,Cell lengths ``[nm]``
system_angles,Cell angles ``[degree]``
atom_types,Description of atom types
**Dimensions**,
cbed_position_x,Number of probe positions in x direction
cbed_position_y,Number of probe positions in y direction
cbed_k_x,Number of k grid in k_x direction
cbed_k_y,Number of k grid in k_y direction
cbed_defocus,Number of stored defoci (1 when averaged over defoci)
cbed_phonon,Number of stored frozen phonon configuration (1 when averaged over configurations)
cbed_slice,Number of stored slices
**Variables**,
cbed_intensities,cbed intensities of each pixel ``[fraction of beam]``
cbed_probe_x_grid,Position vector of the probe in x direction ``[nm]``
cbed_probe_y_grid,Position vector of the probe in y direction ``[nm]``
cbed_x_grid,Angles of k_x grid ``[mrad]``
cbed_y_grid,Angles of k_y grid ``[mrad]``
cbed_slice_coords,Coordinates of the stored slices ``[nm]``
**Attributes**,
program_arguments,CLI arguments of this run
config_file_contents,Contents of the parameter file of this run.
**Dimensions**,
defocus,Number of defocus (param ``probe.num_defoci``)
**Variables**,