index.rst 7.19 KB
Newer Older
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
.. skeleton documentation master file, created by
   sphinx-quickstart on Thu May 17 15:17:35 2018.
   You can adapt this file completely to your liking, but it should at least
   contain the root `toctree` directive.


.. HOME SECTION ==================================================

.. Hidden toctree to manage the sidebar navigation.

.. toctree::
  :maxdepth: 1
  :caption: Home
  :hidden:

16
17
  rest_server
  rest_client
18
  rest_api
19
20
21
22
23
24
25

.. COMMUNITY SECTION ==================================================

.. Hidden toctree to manage the sidebar navigation.

.. toctree::
  :maxdepth: 1
26
  :caption: Public API Documentation
27
28
  :hidden:

29
30
31
32
  package/oet/command
  package/oet/features


33
==========================
34
Observation Execution Tool
35
36
37
38
39
==========================

Project description
===================

40
The `ska-oso-oet` project contains the code for the Observation Execution Tool (OET), the
Stewart Williams's avatar
Stewart Williams committed
41
application which provides on-demand Python script execution for the SKA.
42

43
44
Overview
========
Viivi Pursiainen's avatar
Viivi Pursiainen committed
45
46
47
48
49
50
The OET consists of a script execution engine, which loads specified scripts
and runs them in child Python processes, and a REST layer which makes the
API for the script execution engine available via REST over HTTP.

The REST layer is made up of two components that work together to provide the
remote script execution functionality:
51

Viivi Pursiainen's avatar
Viivi Pursiainen committed
52
- The OET :doc:`rest_server` maintains a list of the scripts that have been
53
54
  loaded and their current state. The server implements the interface specified
  by the OET :doc:`rest_api`.
55
56
- The OET :doc:`rest_client` provides a Command Line Interface (CLI) to the
  OET :doc:`rest_server`. It does this by translating and communicating HTTP
57
58
  messages to and from the server.

Viivi Pursiainen's avatar
Viivi Pursiainen committed
59
60
61
62
63
.. note::
   SKA control scripts are not packaged as part of this project. The repository
   of observing scripts executed by the OET can be found in the
   `OET Scripts <https://developer.skatelescope.org/projects/oet-scripts/en/latest/index.html>`_
   project.
64

65
66
67
68
69
70
71
72
73
74
75
76
77
Quickstart
==========
This project is structured to use Docker containers for development and
testing so that the build environment, test environment and test results are
all completely reproducible and are independent of host environment. It uses
``make`` to provide a consistent UI (see `Makefile targets`_).

Build a new Docker image for the OET with:

::

  make build

78
Execute the test suite and lint the project with:
79
80
81

::

Stewart Williams's avatar
Stewart Williams committed
82
83
  make test
  make lint
84
85
86
87
88
89
90
91
92
93
94
95
96
97

Makefile targets
================
This project contains a Makefile which acts as a UI for building Docker
images, testing images, and for launching interactive developer environments.
The following make targets are defined:

+-----------------+------------------------------------------------+
| Makefile target | Description                                    |
+=================+================================================+
| build           | Build a new application image                  |
+-----------------+------------------------------------------------+
| test            | Test the application image                     |
+-----------------+------------------------------------------------+
98
| lint            | Lint the application image                     |
99
+-----------------+------------------------------------------------+
Stewart Williams's avatar
Stewart Williams committed
100
101
| prune           | Delete stale Docker images for this project    |
+-----------------+------------------------------------------------+
102
103
104
105
106
107
108
109
110
111
112
113
114
115
| interactive     | Launch a minimal Tango system (including the   |
|                 | device under development), mounting the source |
|                 | directory from the host machine inside the     |
|                 | container                                      |
+-----------------+------------------------------------------------+
| push            | Push the application image to the Docker       |
|                 | registry                                       |
+-----------------+------------------------------------------------+
| up              | launch the development/test container service  |
|                 | on which this application depends              |
+-----------------+------------------------------------------------+
| down            | stop all containers launched by 'make up' and  |
|                 | 'make interactive'                             |
+-----------------+------------------------------------------------+
116
117
| rest            | start the OET REST server                      |
+-----------------+------------------------------------------------+
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
| help            | show a summary of the makefile targets above   |
+-----------------+------------------------------------------------+

Creating a new application image
--------------------------------
``make build`` target creates a new Docker image for the application based
on the 'ska-python-runtime' image. To optimise final image size and to support
the inclusion of C extension Python libraries such as pytango, the application
is built inside an intermediate Docker image which includes compilers and
cached eggs and wheels for commonly-used Python libraries
('ska-python-builder'). The resulting Python environment from this
intermediate stage is copied into a final image which extends a minimal SKA
Python runtime environment ('ska-python-runtime'), to give the final Docker
image for this application.

Interactive development using containers
----------------------------------------
``make interactive`` launches an interactive session using the application
image, mounting the project source directory at /app inside the container.
This allows the container to run code from the local workspace. Any changes
made to the project source code will immediately be seen inside the container.

Test execution
--------------
``make test`` runs the application test procedure defined in
test-harness/Makefile in a temporary container. The Makefile example for
this project runs 'python setup.py test' and copies the resulting output and
test artefacts out of the container and into a 'build' directory, ready for
inclusion in the CI server's downloadable artefacts.
147
148
149
150

REST server
-----------
``make rest`` starts the OET REST server. Details of the REST API can be
151
152
153
found in :doc:`rest_api`. Instructions on how to use the REST client
can be found here: :doc:`rest_client`.

154
155
Feature flags
-------------
Stewart Williams's avatar
Stewart Williams committed
156
157
158
OET feature flags are configured via environment variables and configuration
files. The configuration file, oet.ini, can be located either in the user's
home directory, or the root of the installation folder.
159

Stewart Williams's avatar
Stewart Williams committed
160
Feature flags are read in this order:
161

Stewart Williams's avatar
Stewart Williams committed
162
163
164
#. environment variable;
#. oet.ini configuration file;
#. default flag value as specified in OET code.
165

Stewart Williams's avatar
Stewart Williams committed
166
Available feature flags are:
167

Stewart Williams's avatar
Stewart Williams committed
168
169
170
171
172
173
174
+----------------------+-------------------+---------+----------------------------+---------+
| environment variable | oet.ini setting   | Type    | Description                | Default |
+======================+===================+=========+============================+=========+
| OET_READ_VIA_PUBSUB  | read_via_pubsub   | Boolean | sets whether pubsub or     | False   |
|                      |                   |         | the alternative, polling,  |         |
|                      |                   |         | is used to read from tango |         |
+----------------------+-------------------+---------+----------------------------+---------+