http_reporting.rst 5.07 KB
Newer Older
1
.. _reporting:
2

3
4
HTTP Status reporting
=====================
5

6
7
*STEMsalabim* simulations may take a long time, even when running them in parallel on many processors. In order to ease
tracking of the status of running simulations, we built reporting via HTTP POST requests into the program.
8

9
10
In order to use that feature, the `libCURL <https://curl.haxx.se/libcurl/>`_ library has to be installed and
*STEMsalabim* needs to be linked against it.
11

12
.. _configure-reporting:
13

14
15
Configure HTTP reporting
------------------------
16

17
18
To configure HTTP reporting, please add the ``http_reporting: {}`` block to your simulations's parameter file,
containing at least ``reporting = true;`` and the url to report to, ``url = "http://my_server_address:port/path";``.
19

20
21
22
23
24
25
26
27
28
If you want to use `HTTP basic authentication <https://en.wikipedia.org/wiki/Basic_access_authentication>`_,
you may also specify the options ``auth_user = "your_user";`` and ``auth_pass = "your_pass";``. Note, that HTTP basic
auth will be enabled as soon as ``auth_user`` is not empty. You should therefore only fill in that field when you want
to use authentication.

Additional, custom payload for the HTTP requests may be specified in the sub-block ``parameters: {}``. Each key-value
pair in this block is translated into JSON and appended to each request. This allows you to use custom authentication
techniques, such as token-based authentication
etc.
29
30
31

An example configuration block with HTTP basic authentication may look like this:

32
33
::

34
35
36
37
38
39
40
41
42
43
    http_reporting: {
        reporting = true;
        url = "http://my_api_endpoint:8000/stemsalabim-reporting";
        auth_user = "my_user";
        auth_pass = "my_pass";
        parameters: {
            simulation_category = "suitable for many Nature papers";
        }
    }

44
45
46
47
48
49
50
The status requests
-------------------

In each request that *STEMsalabim* sends, some JSON payload is common. In addition to the JSON values specified in the
parameter file (see :ref:`configure-reporting`), the following parameters are always reported:

::
51
52
53
54
55
56
57
58
59

    time: // The currrent date and time
    id: // the UUID of the simulation
    num_threads: // the number of threads of each MPI processor
    num_processors: // the number of MPI processors
    num_defoci: // the total number of defoci to calculate
    num_configurations: // the total number of frozen phonon configurations to calculate
    event: // A code for what event is reported. see below.

60
The following four different ``event`` codes, each for a different event, are reported:
61

62
63
``START_SIMULATION``: A simulation is started
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
64
65
66

This request is sent at the beginning of a simulation. Additional key/value pairs sent are:

67
68
::

69
70
71
72
    event: "START_SIMULATION"
    version: // program version
    git_commit: // git commit hash of the program version
    title: // simulation title
73
74
75

``START_DEFOCUS``: A defocus iteration is started
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
76
77
78

This request is sent at the beginning of a defocus iteration. Additional key/value pairs sent are:

79
80
::

81
82
83
    event: "START_DEFOCUS"
    defocus: // the defocus value in nm
    defocus_index: // the index of the defocus, between 0 and num_defoci
84
85
86

``START_FP_CONFIGURATION``: A frozen phonon iteration is started
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
87
88
89

This request is sent at the beginning of a frozen phonon configuration. Additional key/value pairs sent are:

90
91
::

92
93
94
95
96
    event: "START_FP_CONFIGURATION"
    defocus: // the defocus value in nm
    defocus_index: // the index of the defocus, between 0 and num_defoci
    configuration_index: // the index of the configuration, between 0 and num_configurations

97
98
99
100
101
102
103
``PROGRESS``: Progress report
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

This request is sent during the calculation, typically after each integer percent of the simulation finished.
Additional key/value pairs sent are:

::
104
105
106
107
108
109

    event: "START_CONFIGURATION"
    defocus: // the defocus value in nm
    defocus_index: // the index of the defocus, between 0 and num_defoci
    configuration_index: // the index of the configuration, between 0 and num_configurations
    progress: // progress between 0 and 1 of this configuration iteration within this defocus iteration
110
111
112

``FINISH``: Simulation finished
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
113
114
115

This request is sent when the simulation finished. Additional key/value pairs sent are:

116
117
::

118
119
    event: "FINISH"

120
121
122
123
124
125
126
127
How to process the reports
--------------------------

Obviously, in order to register the requests, an HTTP(S) server needs to be running on the target machine. For example,
a very simple server in python using the `http://flask.pocoo.org/ <flask>`__ package, that only echos the requests,
can be implemented as:

::
128
129
130
131
132
133

    #!/usr/bin/env python

    from flask import Flask
    from flask import request
    import json
134

135
    app = Flask(__name__)
136

137
138
139
140
141
142
143
144
145
    @app.route('/', methods=['POST'])
    def echo():
        content = request.get_json()
        print(json.dumps(content, indent=4))
        return ""
        
    if __name__ == "__main__":
        app.run()

146
Run the script and then start a *STEMsalabim* simulation to see requests imcoming.