file_formats.rst 10.4 KB
Newer Older
1 2 3
File formats
============

4
A STEMsalabim simulation is set-up via **input files** and its results are stored in an **output file**. The file for
5 6
configuring a simulation is described in detail at :ref:`parameter-file`. Here, we describe the format of the **crystal
file**, i.e., the atomic information about the specimen, and the **output file**, in which the results are stored.
7 8 9 10 11 12

.. _crystal-file:

Crystal file format
-------------------

13
The crystal file is expected to be in `XYZ format <https://en.wikipedia.org/wiki/XYZ_file_format>`_.
14

15
1. The **first line** contains the number of atoms.
16

17
2. The **second line** contains the cell dimension in x,y,z direction as floating point numbers in units of ``nm``,
18 19 20
   separated by a space. Optionally, it can contain the x, y, z dimensions in the .exyz format: ::

      Lattice="lx 0.0 0.0 0.0 ly 0.0 0.0 0.0 lz"
21

22 23
3. The atomic information is given from the **third line onwards**, with each line corresponding to a single atom. Each
   line must have *exactly 5 columns*:
24

25 26 27
    - The atomic species as elemental abbreviation (e.g., ``Ga`` for gallium)
    - the x,y,z coordinates as floating point numbers in units of ``nm``
    - the mean square displacement for the frozen lattice dislocations in units of :math:`nm^2`.
28
    - (**optional**) The id of the slice this atom belongs to. This can be used to do custom slicing.
29

30
Below is a very brief, artificial example (without custom slicing): ::
31 32 33 34 35 36 37 38 39

    5
    1.0 2.0 10.0
    Ga  0.0  0.0   0.0   1e-5
    P   0.2  0.1   0.0   2e-5
    Ga  0.0  0.0   1.0   1e-5
    P   1.2  0.1   0.0   2e-5
    O   1.0  2.0  10.0   0.0

40 41 42 43

.. note:: Atomic coordinates outside of the cell are periodically wrapped in ``x`` and ``y`` and clipped to the
   simulation box in ``z`` direction!

44 45 46 47 48
.. _output-file:

Output file format
------------------

49
All results of a STEMsalabim simulation are written to a binary `NetCDF <https://en.wikipedia.org/wiki/NetCDF>`_ file.
50 51 52
The NetCDF format is based on the `Hierarchical Data Format <https://en.wikipedia.org/wiki/Hierarchical_Data_Format>`_
and there are libraries to read the data for many programming languages.

53 54
The structure of NetCDF files can be inspected using the handy tool ``ncdump -h YOUR_FILE.nc`` (don't forget the ``-h``
parameter, otherwise the whole content of the file is dumped!). Here is the output of an example run: ::
55 56


57
    netcdf out {
58 59 60

    group: AMBER {
      dimensions:
61
        atom = 164140 ;
62 63 64 65 66
        elements = 1 ;
        spatial = 3 ;
        cell_spatial = 3 ;
        cell_angular = 3 ;
        label = 6 ;
67 68 69 70
        frame = 10 ;
        slices = 142 ;
        grid_x = 490 ;
        grid_y = 490 ;
71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95
      variables:
        char spatial(spatial) ;
        char cell_spatial(cell_spatial) ;
        char cell_angular(cell_angular, label) ;
        float coordinates(frame, atom, spatial) ;
            coordinates:unit = "nanometer" ;
        float lattice_coordinates(frame, atom, spatial) ;
        float cell_lengths(frame, cell_spatial) ;
            cell_lengths:unit = "nanometer" ;
        float cell_angles(frame, cell_angular) ;
            cell_angles:unit = "degree" ;
        float radius(frame, atom) ;
            radius:unit = "nanometer" ;
        float msd(frame, atom) ;
        int slice(frame, atom) ;
        float slice_coordinates(slices) ;
        short element(frame, atom) ;
        float system_lengths(cell_spatial) ;
        float system_angles(cell_spatial) ;
        char atom_types(elements, label) ;

      // group attributes:
            :Conventions = "AMBER" ;
            :ConventionVersion = "1.0" ;
            :program = "STEMsalabim" ;
96
            :programVersion = "5.0.0b" ;
97 98 99 100 101 102 103 104
            :title = "sim" ;
      } // group AMBER

    group: runtime {

      // group attributes:
            :programVersionMajor = "5" ;
            :programVersionMinor = "0" ;
105 106 107 108 109 110
            :programVersionPatch = "0b" ;
            :gitCommit = "f1dcc606c9a78b12fc3afda9496f638992b591bf" ;
            :title = "sim" ;
            :UUID = "8dce768e-f1d6-4876-bb20-c301e3e323f8" ;
            :time_start = "2019-02-12 13:25:43" ;
            :time_stop = "2019-02-13 00:06:05" ;
111 112 113 114
      } // group runtime

    group: params {
      dimensions:
115 116
        defocus = 1 ;
        plasmon_energies = 51 ;
117 118 119
      variables:
        float defocus(defocus) ;
        float defocus_weights(defocus) ;
120
        float plasmon_energies(plasmon_energies) ;
121 122

      // group attributes:
123
            :program_arguments = "--params=inp.cfg --num-threads=4 --tmp-dir=/local --output-file=out.nc" ;
124 125 126 127 128
            :config_file_contents = "..." ;

      group: application {

        // group attributes:
129
                :random_seed = 967613772U ;
130 131 132 133 134 135 136 137
        } // group application

      group: simulation {

        // group attributes:
                :title = "sim" ;
                :normalize_always = 0US ;
                :bandwidth_limiting = 1US ;
138
                :output_file = "out.nc" ;
139 140 141 142 143 144 145 146 147
                :output_compress = 0US ;
        } // group simulation

      group: probe {

        // group attributes:
                :c5 = 5000000. ;
                :cs = 2000. ;
                :astigmatism_ca = 0. ;
148
                :defocus = -0. ;
149
                :fwhm_defoci = 6. ;
150
                :num_defoci = 1U ;
151 152 153 154 155 156 157 158 159 160 161
                :astigmatism_angle = 0. ;
                :min_apert = 0. ;
                :max_apert = 15.07 ;
                :beam_energy = 200. ;
                :scan_density = 40. ;
        } // group probe

      group: specimen {

        // group attributes:
                :max_potential_radius = 0.3 ;
162
                :crystal_file = "Si_110_10x10x200_300K.xyz" ;
163 164 165 166 167
        } // group specimen

      group: grating {

        // group attributes:
168 169 170 171
                :density = 90. ;
                :nx = 490U ;
                :ny = 490U ;
                :slice_thickness = 0.76806 ;
172 173 174 175 176 177
        } // group grating

      group: adf {

        // group attributes:
                :enabled = 1US ;
178 179
                :x = 0.5, 0.6 ;
                :y = 0.5, 0.6 ;
180
                :detector_min_angle = 0. ;
181 182
                :detector_max_angle = 150. ;
                :detector_num_angles = 151U ;
183
                :detector_interval_exponent = 1.f ;
184 185 186
                :average_configurations = 1US ;
                :average_defoci = 1US ;
                :save_slices_every = 10U ;
187 188 189 190 191 192
        } // group adf

      group: cbed {

        // group attributes:
                :enabled = 1US ;
193 194
                :x = 0.5, 0.6 ;
                :y = 0.5, 0.6 ;
195 196
                :size = 0U, 0U ;
                :average_configurations = 1US ;
197 198
                :average_defoci = 0US ;
                :save_slices_every = 101U ;
199 200 201 202 203
        } // group cbed

      group: frozen_phonon {

        // group attributes:
204
                :number_configurations = 10U ;
205 206 207
                :fixed_slicing = 1US ;
                :enabled = 1US ;
        } // group frozen_phonon
208 209 210 211 212 213 214 215 216 217 218 219 220

      group: plasmon_scattering {

        // group attributes:
                :enabled = 1US ;
                :simple_mode = 0US ;
                :plural_scattering = 0US ;
                :max_energy = 25.f ;
                :energy_grid_density = 2.f ;
                :mean_free_path = 128.f ;
                :plasmon_energy = 16.9f ;
                :plasmon_fwhm = 4.f ;
        } // group plasmon_scattering
221 222 223 224
      } // group params

    group: adf {
      dimensions:
225 226 227 228 229 230
        adf_position_x = 22 ;
        adf_position_y = 22 ;
        adf_detector_angle = 151 ;
        adf_defocus = 1 ;
        adf_phonon = 1 ;
        adf_slice = 15 ;
231
        coordinate_dim = 2 ;
232
        adf_plasmon_energies = 51 ;
233
      variables:
234 235
        float adf_intensities(adf_defocus, adf_position_x, adf_position_y, adf_phonon, adf_slice, adf_plasmon_energies, adf_detector_angle) ;
        float center_of_mass(adf_defocus, adf_position_x, adf_position_y, adf_phonon, adf_slice, adf_plasmon_energies, coordinate_dim) ;
236 237 238 239 240 241 242 243
        double adf_probe_x_grid(adf_position_x) ;
        double adf_probe_y_grid(adf_position_y) ;
        double adf_detector_grid(adf_detector_angle) ;
        double adf_slice_coords(adf_slice) ;
      } // group adf

    group: cbed {
      dimensions:
244 245 246 247
        cbed_position_x = 22 ;
        cbed_position_y = 22 ;
        cbed_k_x = 327 ;
        cbed_k_y = 327 ;
248 249
        cbed_defocus = 1 ;
        cbed_phonon = 1 ;
250 251
        cbed_slice = 2 ;
        cbed_plasmon_energies = 51 ;
252
      variables:
253
        float cbed_intensities(cbed_defocus, cbed_position_x, cbed_position_y, cbed_phonon, cbed_slice, cbed_plasmon_energies, cbed_k_x, cbed_k_y) ;
254 255 256 257 258 259 260 261 262 263
        double cbed_probe_x_grid(cbed_position_x) ;
        double cbed_probe_y_grid(cbed_position_y) ;
        double cbed_x_grid(cbed_k_x) ;
        double cbed_y_grid(cbed_k_y) ;
        double cbed_slice_coords(cbed_slice) ;
      } // group cbed
    }

The structure of NetCDF files is hierarchical and organized in groups. The following groups are written by
STEMsalabim:
264 265 266 267 268 269

AMBER
~~~~~

This group contains the atomic coordinates, species, displacements, radii, etc. for the complete crystal for each single
calculated frozen lattice configuration, as well as for each calculated defocus value. The AMBER group content is
270
compatible with the `AMBER specifications <http://ambermd.org/netcdf/nctraj.xhtml>`_. A STEMsalabim NetCDF file can
271
be opened seamlessly with the `Ovito <http://www.ovito.org/>`_ crystal viewer.
272

273 274 275 276 277 278 279 280 281 282 283
.. csv-table::
   :file: table_nc_amber.csv


runtime
~~~~~~~

.. csv-table::
   :file: table_nc_runtime.csv


284 285 286
params
~~~~~~

287 288 289 290 291 292 293 294
.. note::  The ``params`` group contains subgroups with attributes that correspond exactly to the simulation
    parameters as written, except

     - ``/params/application/random_seed`` is set to the generated random seed
     - ``/params/grating/nx`` and ``/params/grating/ny`` contain the simulation grid size used.

.. csv-table::
   :file: table_nc_params.csv
295 296 297 298

adf
~~~

299 300
.. csv-table::
   :file: table_nc_adf.csv
301 302 303 304

cbed
~~~~

305 306
.. csv-table::
   :file: table_nc_cbed.csv
307 308 309 310

Reading NC Files
----------------

311 312 313 314
For a detailed view of the structure, we suggest using the `ncdump
<http://www.unidata.ucar.edu/software/netcdf//old_docs/docs_4_1/netcdf/ncdump.html>`_ utility: ``ncdump -h
some_results_file.nc``. As the underlying file format of NetCDF is HDF5, you may use any other HDF5 viewer to have a
look at the results.
315 316 317

There are NetCDF bindings for most popular programming languages.

318 319
1. In MATLAB, we recommend using ``h5read()`` and the `other HDF5 functions
   <https://de.mathworks.com/help/matlab/high-level-functions.html>`_.
320 321
2. For Python, use the
   `netCDF4 <http://unidata.github.io/netcdf4-python/>`_ module.