README.rst 5.94 KB
Newer Older
1
2
3
Arkindex API Client
===================

Erwan Rouchet's avatar
Erwan Rouchet committed
4
``arkindex-client`` provides an API client to interact with Arkindex servers.
5

Erwan Rouchet's avatar
Erwan Rouchet committed
6
7
8
9
.. contents::
   :depth: 2
   :local:
   :backlinks: none
10

Erwan Rouchet's avatar
Erwan Rouchet committed
11
12
13
14
15
16
17
Setup
-----

Install the client using ``pip``::

   pip install arkindex-client

Erwan Rouchet's avatar
Erwan Rouchet committed
18
19
20
21
Usage
-----

To create a client and login using an email/password combo,
Erwan Rouchet's avatar
Erwan Rouchet committed
22
use the ``ArkindexClient.login`` helper method:
Erwan Rouchet's avatar
Erwan Rouchet committed
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42

.. code:: python

   from arkindex import ArkindexClient
   cli = ArkindexClient()
   cli.login('EMAIL', 'PASSWORD')

This helper method will save the authentication token in your API client, so
that it is reused in later API requests.

If you already have an API token, you can create your client like so:

.. code:: python

   from arkindex import ArkindexClient
   cli = ArkindexClient('YOUR_TOKEN')

Making requests
^^^^^^^^^^^^^^^

Erwan Rouchet's avatar
Erwan Rouchet committed
43
44
45
46
47
48
49
To perform a simple API request, you can use the ``request()`` method. The method
takes an operation ID as a name and the operation's parameters as keyword arguments.

You can open ``https://your.arkindex/api-docs/`` to access the API documentation,
which will describe the available API endpoints, including their operation ID and
parameters.

Erwan Rouchet's avatar
Erwan Rouchet committed
50
51
52
53
54
55
56
57
58
59
60
61
.. code:: python

   corpus = cli.request('RetrieveCorpus', id='...')

The result will be a Python ``dict`` containing the result of the API request.
If the request returns an error, an ``apistar.exceptions.ErrorResponse`` will
be raised.

Dealing with pagination
^^^^^^^^^^^^^^^^^^^^^^^

The Arkindex client adds another helper method for paginated endpoints that
Erwan Rouchet's avatar
Erwan Rouchet committed
62
63
deals with pagination for you: ``ArkindexClient.paginate``. This method
returns a ``ResponsePaginator`` instance, which is a classic Python
Erwan Rouchet's avatar
Erwan Rouchet committed
64
65
66
67
68
iterator that does not perform any actual requests until absolutely needed:
that is, until the next page must be loaded.

.. code:: python

Erwan Rouchet's avatar
Erwan Rouchet committed
69
70
   for element in cli.paginate('ListElements', corpus=corpus['id']):
       print(element['name'])
Erwan Rouchet's avatar
Erwan Rouchet committed
71

Erwan Rouchet's avatar
Erwan Rouchet committed
72
73
74
**Warning:** Using ``list`` on a ``ResponsePaginator`` may load dozens
of pages at once and cause a big load on the server. You can use ``len`` to
get the total item count before spamming a server.
Erwan Rouchet's avatar
Erwan Rouchet committed
75

Bastien Abadie's avatar
Bastien Abadie committed
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
A call to ``paginate`` may produce hundreds of sub-requests depending on the size
of the dataset you're requesting. To accommodate with large datasets, and support
network or performance issues, ``paginate`` supports a ``retries`` parameter to
specify the number of sub-request it's able to run for each page in the dataset.
By default, the method will retry 5 times per page.

You may want to allow ``paginate`` to fail on some pages, for really big datasets
(errors happen). In this case, you should use the optional boolean parameter
``allow_missing_data`` (set to ``False`` by default).

Here is an example of pagination on a large dataset, allowing data loss, lowering
 retries and listing the missed pages:

.. code:: python

   elements = cli.paginate(
       'ListProcessElements',
       id='XXX',
       retries=3,
       allow_missing_data=True,
   )
   for element in elements:
       print(element['id'])

   print("Missing pages: {elements.missing}")



Erwan Rouchet's avatar
Erwan Rouchet committed
104
105
106
107
108
109
110
111
112
113
114
115
Using another server
^^^^^^^^^^^^^^^^^^^^

By default, the API client is set to point to the main Arkindex server at
https://arkindex.teklia.com. If you need or want to use this API client on
another server, you can use the ``base_url`` keyword argument when setting up
your API client:

.. code:: python

   cli = ArkindexClient(base_url='https://somewhere')

Erwan Rouchet's avatar
Erwan Rouchet committed
116
117
118
Handling errors
^^^^^^^^^^^^^^^

Erwan Rouchet's avatar
Erwan Rouchet committed
119
APIStar_, the underlying API client we use, does most of the error handling.
Erwan Rouchet's avatar
Erwan Rouchet committed
120
121
122
123
124
125
126
It will raise two types of exceptions:

``apistar.exceptions.ErrorResponse``
  The request resulted in a HTTP 4xx or 5xx response from the server.
``apistar.exceptions.ClientError``
  Any error that prevents the client from making the request or fetching
  the response: invalid endpoint names or URLs, unsupported content types,
127
  or unknown request parameters. See the exception messages for more info.
Erwan Rouchet's avatar
Erwan Rouchet committed
128

Erwan Rouchet's avatar
Erwan Rouchet committed
129
130
131
132
133
Since this API client retrieves the endpoints description from the server
using the base URL, errors can occur during the retrieval and parsing of the
API schema. If this happens, an ``arkindex.exceptions.SchemaError`` exception
will be raised.

Erwan Rouchet's avatar
Erwan Rouchet committed
134
135
136
137
138
139
140
141
142
143
144
145
146
You can handle HTTP errors and fetch more information about them using the
exception's attributes:

.. code:: python

   from apistar.exceptions import ErrorResponse
   try:
       # cli.request ...
   except ErrorResponse as e:
       print(e.title)   # "400 Bad Request"
       print(e.status_code)  # 400
       print(e.result)  # Any kind of response body the server might give

Erwan Rouchet's avatar
Erwan Rouchet committed
147
Note that by default, using ``repr()`` or ``str()`` on APIStar exceptions will
Erwan Rouchet's avatar
Erwan Rouchet committed
148
149
150
not give any useful messages; a fix in APIStar is waiting to be merged. In
the meantime, you can use Teklia's `APIStar fork`_::

151
   pip install git+https://gitlab.com/teklia/apistar.git
Erwan Rouchet's avatar
Erwan Rouchet committed
152
153
154
155

This will provide support for ``repr()`` and ``str()``, which will also
enhance error messages on unhandled exceptions.

Erwan Rouchet's avatar
Erwan Rouchet committed
156
157
158
Examples
--------

Erwan Rouchet's avatar
Erwan Rouchet committed
159
Print all folders
Erwan Rouchet's avatar
Erwan Rouchet committed
160
161
162
163
^^^^^^^^^^^^^^^^^

.. code:: python

Erwan Rouchet's avatar
Erwan Rouchet committed
164
165
   for folder in cli.paginate('ListElements', folder=True):
       print(folder['name'])
Erwan Rouchet's avatar
Erwan Rouchet committed
166
167
168
169
170
171
172
173
174
175

Download full logs for each Ponos task in a workflow
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

.. code:: python

   workflow = cli.request('RetrieveWorkflow', id='...')
   for task in workflow['tasks']:
       with open(task['id'] + '.txt', 'w') as f:
           f.write(cli.request('RetrieveTaskLog', id=task['id']))
Erwan Rouchet's avatar
Erwan Rouchet committed
176
177

.. _APIStar: http://docs.apistar.com/
178
.. _APIStar fork: https://gitlab.com/teklia/apistar
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197

Linting
-------

We use `pre-commit <https://pre-commit.com/>`_ with `black <https://github.com/psf/black>`_ to automatically format the Python source code of this project.

To be efficient, you should run pre-commit before committing (hence the name...).

To do that, run once :

.. code:: shell

   pip install pre-commit
   pre-commit install

The linting workflow will now run on modified files before committing, and will fix issues for you.

If you want to run the full workflow on all the files: `pre-commit run -a`.