README.md 15.5 KB
Newer Older
Corey O'Connor's avatar
Corey O'Connor committed
1
glngn server is an extensible business process application server. Similar to
Corey O'Connor's avatar
Corey O'Connor committed
2
Microsoft Access or Apple FileMaker, but designed for event sourced business services with a code-first philosophy.
Corey O'Connor's avatar
Corey O'Connor committed
3
4
A standalone application and an approachable SDK is provided.  Applications developed with glngn
server can be easily deployed to a kubernetes cluster.
5

Corey O'Connor's avatar
~    
Corey O'Connor committed
6
- Support available from the [DogHeadBone LLC](mailto:support@dogheadbone.com) company
Corey O'Connor's avatar
Corey O'Connor committed
7
- Useful built-in services
8
- Easy event captures
Brett O'Connor's avatar
Brett O'Connor committed
9
- Predictable REST interface with an OpenAPI schema
10
- A [high level extension API](http://docs.glngn.com) for Scala service developers.
11
12
13
  (Plain, typed akka and zio interfaces are provided.)
- Simple persistence features
- Simple Kubernetes (k8s) cluster deployments. In addition to some custom automation, akka
Corey O'Connor's avatar
~    
Corey O'Connor committed
14
  management tools are included. OpenShift is also supported.
15

Corey O'Connor's avatar
Corey O'Connor committed
16
Sound good so far? Excellent! Let's start with basic, interactive usage of the standalone application.
17

Corey O'Connor's avatar
Corey O'Connor committed
18
# Developer SDK
19

Corey O'Connor's avatar
Corey O'Connor committed
20
For a guide to the SDK see:
21

Corey O'Connor's avatar
Corey O'Connor committed
22
23
- [development guide](docs/Customization.md)
- [development error help](docs/CompileErrorHelp.md)
24

Corey O'Connor's avatar
Corey O'Connor committed
25
26
The SDK API is familiar to Scala service developers. To a large degree,
glngn server is a friendly, constrained, pre-configured typed akka plus ZIO server. A few of the
27
niceties included:
28

Corey O'Connor's avatar
Corey O'Connor committed
29
30
31
- Clustering is enabled and established without additional configuration
- Logging and data marshalling are configured appropriately
- Additional error tracking and handling
32

Corey O'Connor's avatar
~    
Corey O'Connor committed
33
# Standalone Application
34

Corey O'Connor's avatar
~    
Corey O'Connor committed
35
prerequisites:
Corey O'Connor's avatar
Corey O'Connor committed
36

Corey O'Connor's avatar
Corey O'Connor committed
37
- java runtime version 8 or above.
Corey O'Connor's avatar
Corey O'Connor committed
38
- agreement to the [Developer License](docs/Developer%20License%20v1.rtf?inline=false) This license is also included in the jar archive.
39
40
41

## Download

Corey O'Connor's avatar
Corey O'Connor committed
42
Prior to downloading, read and agree to the [Developer License agreement](docs/Developer%20License%20v1.rtf?inline=false).
Corey O'Connor's avatar
~    
Corey O'Connor committed
43

jenkins's avatar
jenkins committed
44
- download [the latest release](https://glngn-server-releases.s3.amazonaws.com/assemblies/glngn/glngn-server-assembly_2.12/0.6.8/jars/glngn-server-assembly_2.12-assembly.jar)
45
46

```bash
jenkins's avatar
jenkins committed
47
curl -L -o glngn-server-assembly.jar https://glngn-server-releases.s3.amazonaws.com/assemblies/glngn/glngn-server-assembly_2.12/0.6.8/jars/glngn-server-assembly_2.12-assembly.jar
48
49
50
51
```

## Command Help

Corey O'Connor's avatar
Corey O'Connor committed
52
Run using `java` and pass `--help` to see usage information:
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77

```bash
$ java -jar ./glngn-server-assembly.jar --help
Usage: glngn-server [options] [command] [command options]
  --port  <int?>
  --disable-ops  <bool>

Commands: print-config; print-full-config; run; version

Command: print-config
Usage: glngn-server print-config

Command: print-full-config
Usage: glngn-server print-full-config

Command: run
Usage: glngn-server run
  --state-dir  <string?>
  --exit-immediately  <bool>
  --initial-node-count  <int>
  --heap-dump-on-init  <bool>

Command: version
Usage: glngn-server version

Corey O'Connor's avatar
Corey O'Connor committed
78
Use of this software is agreement with the included Application Developer License.
79
80
81
82
```

## Running

Corey O'Connor's avatar
Corey O'Connor committed
83
84
Note: For consistency the docs always use port 10000. This is requested using the `--port 10000`
command line argument.
85

Corey O'Connor's avatar
Corey O'Connor committed
86
Let's run a server configured to store data at `./starter.state`. In a terminal session:
87
88
89

```bash
$ java -jar ./glngn-server-assembly.jar --port 10000 run --state-dir ./starter.state
90
INFO  use of this software is agreement with the included Developer License.
91
...
Corey O'Connor's avatar
~    
Corey O'Connor committed
92
[lots of output]
93
...
Corey O'Connor's avatar
~    
Corey O'Connor committed
94
INFO  glngn.server.host.ServerApp$  - application is listening at http://localhost:10000
95
96
```

Corey O'Connor's avatar
Corey O'Connor committed
97
The standalone server is initialized and ready.
98

Corey O'Connor's avatar
Corey O'Connor committed
99
See [docs/Deep Dives](https://gitlab.com/glngn/glngn-server-examples/blob/master/docs/DeepDives.md)
Corey O'Connor's avatar
Corey O'Connor committed
100
for technical details on the initialization process.
Corey O'Connor's avatar
Corey O'Connor committed
101

Corey O'Connor's avatar
~    
Corey O'Connor committed
102
Hit control-C, or TERM the process (there is only one), to shut down the server. This will take a bit
Corey O'Connor's avatar
Corey O'Connor committed
103
for a nice, coordinated shutdown.
Corey O'Connor's avatar
~    
Corey O'Connor committed
104

Corey O'Connor's avatar
Corey O'Connor committed
105
## Usage
Corey O'Connor's avatar
Corey O'Connor committed
106

Corey O'Connor's avatar
Corey O'Connor committed
107
The glngn server provides a REST API with an [OpenAPI](https://www.openapis.org/) specification.
Corey O'Connor's avatar
Corey O'Connor committed
108
109
This API supports the built-in features of glngn server as well as any additional features added by
extensions.
Corey O'Connor's avatar
Corey O'Connor committed
110

Corey O'Connor's avatar
Corey O'Connor committed
111
112
This guide walks through the API using `curl`.  If you are looking for a GUI or integration with
another HTTP client [please let us know](mailto:support@dogheadbone.com).
Corey O'Connor's avatar
Corey O'Connor committed
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128

### OpenAPI schema

Let's query for the API schema to see what endpoints there are:

```bash
$ curl http://localhost:10000/openapi.json
{
  "openapi" : "3.0.1",
  "info" : {
    "title" : "dynamic",
    "version" : "0.1"
  },
  "paths": {
    "/healthz": ...
    "/_ops/groups": ...
129
    "/_ops/services": ...
Corey O'Connor's avatar
Corey O'Connor committed
130
131
132
133
134
135
136
137
138
139
[...]
  }
}
[...]
```

A brief overview:

1. The `/healthz` route. We'll try that one below.
2. The `/_ops/groups` route. This is an operations route for the groups in glngn server. All services
140
are placed under a group. The default configuration does not include any groups. This
Corey O'Connor's avatar
Corey O'Connor committed
141
will be the first route we try in the `API Tour`.
142
143
3. The `/_ops/services` route. This is an operations route to query the services that are available
to instantiate under a group. This will be covered after the `/_ops/groups` route.
Corey O'Connor's avatar
Corey O'Connor committed
144
145
146
147
148

### Multiple Representations

The `/healthz` route will only return status 200 OK if the server is able to handle requests.
This is also a nice example to demonstrate how glngn server supports multiple representations:
Corey O'Connor's avatar
Corey O'Connor committed
149
150

```bash
151
$ curl http://localhost:10000/healthz.txt
Corey O'Connor's avatar
Corey O'Connor committed
152
153
154
OK
```

Corey O'Connor's avatar
Corey O'Connor committed
155
156
Note the use of a `.txt` extension to request a text representation. We could have
requested another representation, such as `json`, explicitly or implicitly:
Corey O'Connor's avatar
Corey O'Connor committed
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178

```bash
$ curl -H 'Accept: text/plain' localhost:10000/healthz
OK

$ curl -H 'Accept: application/json' localhost:10000/healthz
{
  "memStats" : {
[...]
  "ok" : true
}
$ curl localhost:10000/healthz.json
{
  "memStats" : {
[...]
  "ok" : true
}
```

This pattern is supported throughout: Endpoints will have suffixed versions that imply the expected
representation.

Corey O'Connor's avatar
Corey O'Connor committed
179
### API Tour
Corey O'Connor's avatar
~    
Corey O'Connor committed
180

Corey O'Connor's avatar
Corey O'Connor committed
181
182
The API conventions enforced by glngn server:

Corey O'Connor's avatar
Corey O'Connor committed
183
184
185
186
- All routes prefixed with a `_` are operations routes. These are provided by glngn server.
- `/_ops/`
    - Deployment `_ops` routes
    - These are always available
Corey O'Connor's avatar
Corey O'Connor committed
187
    - EG: `/_ops/groups`: query and modify the available groups
Corey O'Connor's avatar
Corey O'Connor committed
188
- `/$GROUP/_ops`
Corey O'Connor's avatar
Corey O'Connor committed
189
    - Group `_ops` routes. These require a group with id, `$GROUP`, to be created
Corey O'Connor's avatar
Corey O'Connor committed
190
- `/$GROUP/$SERVICE/_ops`
Corey O'Connor's avatar
Corey O'Connor committed
191
    - Service fragment `_ops` routes, `/$GROUP/$SERVICE/_ops`, require the service fragment to be
Corey O'Connor's avatar
Corey O'Connor committed
192
    instantiated under the group
Corey O'Connor's avatar
Corey O'Connor committed
193
    - automatically generated for each instantiated service fragment
Corey O'Connor's avatar
Corey O'Connor committed
194
- `/$GROUP/$SERVICE/...`
Corey O'Connor's avatar
Corey O'Connor committed
195
    - All other routes under a service are provided by a Service Fragment's `endpoints`
Corey O'Connor's avatar
Corey O'Connor committed
196

197
#### 1. Create a Group
Corey O'Connor's avatar
Corey O'Connor committed
198

199
200
201
The `/_ops/groups` endpoint is used to query the existing groups and create new groups.  All services
are namespaced under a group. Before we can instantiate and use a service we must first create a
group:
202

Corey O'Connor's avatar
Corey O'Connor committed
203
204
205
~~~
$ curl --data '{ "id": "accounts", "name": "Client Accounts" }' localhost:10000/_ops/groups
{
Corey O'Connor's avatar
Corey O'Connor committed
206
  "enabled" : true,
Corey O'Connor's avatar
Corey O'Connor committed
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
  "id" : {
    "txt" : "accounts"
  },
  "name" : "Client Accounts",
  "operators" : null
}
~~~

This creates a group with the logical name `accounts`. The endpoint responds with the
current definition of the group.

Which we can confirm by querying the `/_ops/groups` endpoint again:

~~~
$ curl localhost:10000/_ops/groups
{
  "accounts" : {
Corey O'Connor's avatar
Corey O'Connor committed
224
    "enabled" : true,
Corey O'Connor's avatar
Corey O'Connor committed
225
226
227
    "id" : {
      "txt" : "accounts"
    },
228
    "name" : "Client Accounts",
Corey O'Connor's avatar
Corey O'Connor committed
229
230
231
232
233
234
    "operators" : null
  }
}
~~~

Note the `operators` attribute. Using the premium license server this would contain the users
Corey O'Connor's avatar
Corey O'Connor committed
235
authorized to be operators in that group. The free license server ignores this attribute.
Corey O'Connor's avatar
Corey O'Connor committed
236

Corey O'Connor's avatar
Corey O'Connor committed
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
#### Groups Extend the API

By creating a group we've extended the provided API. Query the `openapi.json` route again
and note the `/accounts/_ops` route:

~~~
$ curl localhost:10000/openapi.json
{
  "openapi" : "3.0.1",
  "info" : {
    "title" : "dynamic",
    "version" : "0.1"
  },
  "paths" : {
    "/accounts/_ops" : {
      "get" : {
        "operationId" : "getAccounts_ops",
        "responses" : {
          "200" : {
            ...
          }
        }
      }
    }
  }
}
~~~

##### Note: Groups Persist

267
268
If you were to restart the `java -jar ./glngn-server-assembly.jar` process note the group
remains. `glngn-server` persists the created group to the provided state storage.
Corey O'Connor's avatar
Corey O'Connor committed
269

270
In the example above the state storage is the `starter-state` directory provided to the
Corey O'Connor's avatar
Corey O'Connor committed
271
272
273
`--state-dir` option.

#### Instantiate a Service Fragment
Corey O'Connor's avatar
~    
Corey O'Connor committed
274

275
276
277
278
279
280
281
282
283
284
285
286
287
Now that a group exists we will instantiate a service under that group. This will:

1. Extend the API to include that service.
2. Persist the data for that service under the group and service namespace.

##### 1. Query for available services

To determine what services fragments we can instantiate query the `/_ops/services` route:

~~~

$ curl localhost:10000/_ops/services.json
[
288
  "glngn.server.services.StringValues$",
289
  "glngn.server.services.IntegerCounters$"
290
291
292
293
294
295
296
]
~~~

This provides the list of logical IDs of service fragments that can be instantiated under a group.
This set can be customized using the developer SDK. We are using the default glngn distribution which
includes a standard set of generic service fragments.

Corey O'Connor's avatar
Corey O'Connor committed
297
298
See also: [DeepDives - service fragment instantiation](DeepDives#service-fragment-instantiation)

299
300
##### 2. Instantiate the service fragment

301
Let's pick the `glngn.server.services.IntegerCounters$` service fragment to instantiate.
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328

After creating the group `accounts` a route was added `/accounts/_ops/services`. According to the
OpenAPI spec this route accepts a `POST` of a `GroupServiceChange` json structure. The schema of
this structure is:

~~~
"GroupServicesChange" : {
    "required" : [
        "serviceId",
        "logicalServiceId"
    ],
    "type" : "object",
    "properties" : {
        "serviceId" : {
            "type" : "string"
        },
        "logicalServiceId" : {
            "type" : "string"
        }
    }
}
~~~

We already know the logical service ID (`logicalServiceId`) from the query to `/_ops/services`. The
`serviceId` is an identifier we can pick ourselves. We will pick `retail` for this example.

~~~
329
$ curl --data '{ "serviceId": "retail", "logicalServiceId": "glngn.server.services.IntegerCounters$" }' localhost:10000/accounts/_ops/services
330
331
{
  "retail" : [
332
    "glngn.server.services.IntegerCounters$"
333
334
335
336
337
338
  ]
}
~~~

Which replies with the new set of services under that group. This reply states the `retail` service
is a composition of a list of service fragments. Which only contains
339
`glngn.server.services.IntegerCounters$` in this example.
Corey O'Connor's avatar
Corey O'Connor committed
340

Corey O'Connor's avatar
Corey O'Connor committed
341
See [docs/Deep Dives](https://gitlab.com/glngn/glngn-server-examples/blob/master/docs/DeepDives.md) for
342
343
details.

Corey O'Connor's avatar
Corey O'Connor committed
344
#### Use a Service
Corey O'Connor's avatar
Corey O'Connor committed
345

346
347
348
With a service fragment instantiated under a group we can note the OpenAPI schema now includes routes
for the `/accounts/retail` service:

349
350
##### The Extended OpenAPI Schema

Corey O'Connor's avatar
Corey O'Connor committed
351
With this fragment instantiated under `/accounts/retail` the service's routes are now visible in the
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
OpenAPI schema:

~~~
$ curl localhost:10000/openapi.json
{
  "openapi" : "3.0.1",
  "info" : {
    "title" : "dynamic",
    "version" : "0.1"
  },
  "paths" : {
    "/accounts/retail/{Entity ID}" : {
      "get" : {
          ...
        }
      }
    },
    "/accounts/retail/{Entity ID}/inc" : {
      "post" : {
          ...
        }
      }
    },
    "/accounts/retail/{Entity ID}/dec" : {
      "post" : {
          ...
        }
      }
    }
  }
}

~~~

- `/accounts/retail/{Entity ID}`
    - A query, `GET`, that returns an integer given an `Entity ID`
- `/accounts/retail/{Entity ID}/inc`
    - A command, `POST`, that returns the new value of a given `Entity ID` when provided an increment amount.
- `/accounts/retail/{Entity ID}/dec`
    - A command, `POST`, that returns the new value of a given `Entity ID` when provided a decrement amount.
Corey O'Connor's avatar
Corey O'Connor committed
392

393
394
##### Note: Entity ID

Corey O'Connor's avatar
Corey O'Connor committed
395
An `Entity ID` is a common path component and identifier in glngn server. `Entity ID`s are:
396

Corey O'Connor's avatar
Corey O'Connor committed
397
- case insensitive
398
399
400
- case preserving
- consist only of characters in `[a-zA-Z0-9\\-_]`: Upper or lower case letters, numbers, '-' and '_'.
- all invalid characters are filtered if provided for an Entity ID
Corey O'Connor's avatar
Corey O'Connor committed
401
- less than 28 characters in length
402

Corey O'Connor's avatar
Corey O'Connor committed
403
404
New service fragments developed with the [SDK](docs/Customization.md)
can constrain the domain of identifiers.
405
406
407
408
409
410
411
412
413
414
415
416

##### Get a Counter

~~~
$ curl localhost:10000/accounts/retail/central-store
{
  "value" : 0
}
~~~

This is a query for the `retail` counter `central-store`. The result is a value of 0.

Corey O'Connor's avatar
Corey O'Connor committed
417
418
419
420
Creating the counter prior to querying was not required in this case. Counters default to 0. 

For the case of counters having a default value is useful. In general, service fragments
can have entities that require a creation command.
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440

##### Increment a Counter

Using the `inc` subroute the counter can be incremented by a given value:

~~~
$ curl --data 10 localhost:10000/accounts/retail/central-store/inc
{
  "value" : 10
}
$ curl --data 12 localhost:10000/accounts/retail/central-store/inc
{
  "value" : 22
}
$ curl localhost:10000/accounts/retail/central-store
{
  "value" : 22
}
~~~

Corey O'Connor's avatar
Corey O'Connor committed
441
##### Events Persist
442
443

If the glngn server application was to be restarted then this data would persist. If the counter
Corey O'Connor's avatar
Corey O'Connor committed
444
was queried after restart the value would be the same as before.
445

Corey O'Connor's avatar
Corey O'Connor committed
446
The server is persisting all events that modified that counter. Each of those `inc` and `dec`
447
used to change a counter is saved. This history is essential to "event sourcing". Having a
Corey O'Connor's avatar
Corey O'Connor committed
448
history, not only the current state, of a service enables a new level of analysis of your
449
450
451
452
453
business activity.

The standalone version persists to the directory provided to the `--state` command line argument.
When deployed to kubernetes the service can also persist to a database such as Amazon's DynamoDB
or Cassandra.
Corey O'Connor's avatar
Corey O'Connor committed
454

Corey O'Connor's avatar
Corey O'Connor committed
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
#### Operator Access

The REST endpoints above are an interface to the service's event protocol. The
service designer defines the REST endpoint as a request event and expected  
response event. How this request event is handled is defined separately.

One purpose of this separation between endpoints and service protocol is to
differentiate "users" of a service from "operators" of that service. Operators
have additional access to services. Including access to the service protocol 
directly. This enables operators, for example, to post requests that are
not part of the exposed endpoints.

This additional access is exposed under the "_ops" endpoints of a service
and group. We've already used a few of the operator endpoints above. Direct
access to the protocol is provided via the:

- `_ops/requests` - POST envelope containing protcol message

Corey O'Connor's avatar
Corey O'Connor committed
473
#### Capturing Events
Corey O'Connor's avatar
Corey O'Connor committed
474

Corey O'Connor's avatar
Corey O'Connor committed
475
If we have a model of our business events we can build and instantiate service fragments that
476
477
478
479
480
reflect that model. What do we do if we don't have a model of our business events? These unknowns are likely
in a number of situations: New business proposals; exceptional business events; unexpected issues. These
will not have a schema or service structure a-priori. This is why glngn server supports arbitrary
events to enable your business to start building a model.

Corey O'Connor's avatar
Corey O'Connor committed
481
482
483
TODO:

1. demo capturing unmodeled events:
Corey O'Connor's avatar
Corey O'Connor committed
484
    1. deployment-wide
Corey O'Connor's avatar
Corey O'Connor committed
485
486
487
    2. group events
    3. service events
2. demo capturing a service modeled event
Corey O'Connor's avatar
Corey O'Connor committed
488

Corey O'Connor's avatar
Corey O'Connor committed
489
490
aka schema on write

Corey O'Connor's avatar
Corey O'Connor committed
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
#### Requests

A request is a unique interaction with a client. Requests are implicitly
created when the endpoints are used. A request can be explicitly requested
by posting an event of type `command` or `query` to the `_ops` endpoints.

TODO:

1. Each request has a unique ID
2. Any event may be associated with a request
3. A request is resolved when a required event is provided
4. Events may still be associated with a request after resolution

#### Event History

All services have an event history.

TODO:

1. demo querying service history

#### Request History

Each request has an event history.

TODO:

1. demo querying request history