tep-1009.adoc 6.42 KB
Newer Older
1 2
= TEP-1009: Transaction Header Syntax

35V LG84's avatar
35V LG84 committed
3 4
Change Transaction header format so that it is more robust, regular 
and less error prone for humans to manipulate.
5

35V LG84's avatar
35V LG84 committed
6 7
== Overview

35V LG84's avatar
35V LG84 committed
8
Currently Tackler's format tries to mimic Ledger's format, and be as much 
9 10
compatible with it as possible. However, ledger's journal format is somewhat fuzzy,
and makes it unnecessarily difficult to detect some error cases.
35V LG84's avatar
35V LG84 committed
11 12 13 14 15 16 17 18

For example, currently following invalid ISO-8601 offset (one extra space)
is recognized as start of txn description, not as an offset.

....
2019-03-01T12:00:00 -04:00 Description
....

19 20 21
Technically it's invalid ISO-8601 timestamp, but it would be nice if this
very probable human error could be spotted by tackler.

35V LG84's avatar
35V LG84 committed
22 23 24 25
Same goes on with various not-so-obvious format errors with code field.
These are easy to handle with automatically generated transactions, but
for humans these are really confusing and error prone.

26 27 28 29
It doesn't help, that tackler won't notice these as an error.

Trying to fix  these with ledger-compatible way leads to overly
complicated grammar rules and some of the resulting rules are ambiguous.
35V LG84's avatar
35V LG84 committed
30 31


35V LG84's avatar
35V LG84 committed
32
=== Solution
35V LG84's avatar
35V LG84 committed
33

34 35
Break compatibility with Ledger as needed, and change the grammar
to be more regular and less error prone.
35V LG84's avatar
35V LG84 committed
36 37

This will cause that there will be following changes to journal format,
38
which are breaking changes with old journal format and data.
35V LG84's avatar
35V LG84 committed
39 40 41 42


Transaction description::

43 44
Transaction description must start with `'` -prefix.
This is breaking change to journal format.
35V LG84's avatar
35V LG84 committed
45 46 47 48 49 50 51 52 53 54 55


Transaction code::

Transaction code can not contain characters: +
`'` `(` `)` `[` `]` `{` `}` `<` `>` +
This is breaking change to journal format, if these characters are used in code field.


Transaction metadata::

56
Transaction metadata must start with `#` + space -prefix, instead of `;:`.
57
This is breaking change to journal format.
35V LG84's avatar
35V LG84 committed
58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91


=== Execution


==== Phase 1

Release version of Tackler  which accepts old version and new version,
but emits only new syntax (equity, identity exports).  Mark old syntax to deprecated
on all documentation etc.


=== Phase 2

Drop support for old syntax.


== Journal file format



Transaction description::

Transaction description must start with `'` -prefix.


Transaction code::

Transaction code can not contain characters: +
`'` `(` `)` `[` `]` `{` `}` `<` `>`


Transaction metadata::

92
Transaction medadata (uuid) must start with ``#`` + space -prefix
35V LG84's avatar
35V LG84 committed
93 94 95 96 97 98 99 100 101 102


== CLI changes

None


== CONF changes

None
35V LG84's avatar
35V LG84 committed
103 104


35V LG84's avatar
35V LG84 committed
105 106 107 108
== Machinery

Changes to machinery

109 110
* [x] Transition
** [x] In phase 1, it must accept old and new format
111
** [x] After phase 2, only new format will be accepted
112 113
* [x] Changes to Grammar and CTX-handlers
* [x] All test and reference vectors have to be updated (also those inside GIT repositories)
35V LG84's avatar
35V LG84 committed
114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137


=== API changes

None


==== Server API changes

None


==== Client API changes

None


=== New dependencies

None


== Reporting

138
This change causes modifications to Register report.
35V LG84's avatar
35V LG84 committed
139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154


=== Balance report

None


=== Balance Group report

None


=== Register report

Register report has to be changed due this change.

155
Without change to register report it would  be impossible to identify following two
35V LG84's avatar
35V LG84 committed
156 157
transactions correctly from register-report.

158
The first one has code '#123', and the second one has txn with description '(#123)'.
35V LG84's avatar
35V LG84 committed
159 160 161 162 163 164 165 166 167 168 169

....
2019-03-01 (#123)
 a 1
 b

2019-03-01 '(#123)
 a 1
 b
....

170 171
With current register report, both will look exactly same,
and it would impossible e.g. to select correct filters.
35V LG84's avatar
35V LG84 committed
172 173 174

=> Change register report so that it prefix description with `'`

175 176 177 178
Register report also prints transaction UUID. For overall consistency,
it would be good to have same syntax for uuid in register report,
as there is with journal.

35V LG84's avatar
35V LG84 committed
179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197
=> Change register report so that will prefix metadata with `#`


== Exporting

Equity and Identity exports must be valid input to tackler, hence both must be changed.

=== Equity export

Change equity export so that it will use `'` for description.


=== Identity export

Change identity export so that it will use `'` for description, and `#` for metadata.


== Documentation

198 199 200 201 202 203
* [x] xref:./readme.adoc[]: Update TEP index
* [x] link:../../CHANGELOG[]: add new item
* [x] Does it warrant own T3DB file?
** [x] update xref:../../tests/tests.adoc[]
** [x] update link:../../tests/check-tests.sh[]
** [x] Add new T3DB file link:../../tests/tests-1009.yml[]
204 205 206
* [x] User docs
** [x] Journal format
** [x] All examples
35V LG84's avatar
35V LG84 committed
207 208 209 210 211 212


== Future plans and Postponed (PP) features

Following characters are reserved for future use in header's first line: `[` `]` `{` `}` `<` `>`

213 214
Extending current metadata data (uuid) to actual metadata header block (e.g. several lines).

35V LG84's avatar
35V LG84 committed
215 216 217

=== Postponed (PP) features

218
Nothing
35V LG84's avatar
35V LG84 committed
219 220 221 222 223 224 225 226


== Tests

Normal, ok-case tests to validate functionality:

==== Phase 1

227 228 229 230 231
* [x] Accepts old syntax
** [x] Txn header without `'`-prefix
** [x] code field with reserved characters
** [x] Txn metadata with `;:`
** [x] All output is on new syntax
35V LG84's avatar
35V LG84 committed
232

233 234 235 236 237 238 239 240 241 242
* [x] Accepts new syntax
** [x] Txn header with `'`-prefix
** [x] Txn metadata with `#` + space
***  [x] Test Txn metadata with `#` + multiple space

* [x] Test vector to validate that old format is still accepted
** [x] description without `'`
** [x] code with special characters
** [x] old `;:` metadata format

35V LG84's avatar
35V LG84 committed
243 244 245 246


==== Phase 2

247 248 249 250
* [x] Rejects old syntax
** [x] Txn header without `'`-prefix
** [x] code field with reserved characters
** [x] Txn metadata with `;:`
35V LG84's avatar
35V LG84 committed
251

252 253 254
* [x] Accepts new syntax
** [x] Txn header with `'`-prefix
** [x] Txn metadata with `#`
35V LG84's avatar
35V LG84 committed
255

256
* [x] Enable all tests ("todo: perr:" and ignored suites)
35V LG84's avatar
35V LG84 committed
257 258


259
=== Errors
35V LG84's avatar
35V LG84 committed
260

261
Error case tests for Phase-1 and Phase-2.
35V LG84's avatar
35V LG84 committed
262 263 264

==== Phase 1

265
* [x] e: incorrect metadata syntax
35V LG84's avatar
35V LG84 committed
266 267 268 269 270 271 272 273 274 275 276 277 278


==== Phase 2

* [ ] Rejects old syntax
** [ ] e: Txn header without `'`-prefix
** [ ] e: code field with reserved characters
** [ ] e: Txn metadata with `;:`



=== Perf

279
No need for new perf tests - change is covered by normal perf suite.
35V LG84's avatar
35V LG84 committed
280 281 282 283 284


=== Feature and Test case tracking

Feature-id::
285 286
* name: Txn Header Syntax
* id:   f61ad04a-34fd-44f2-a721-8d541fb45180
35V LG84's avatar
35V LG84 committed
287 288 289

Feature-id::
* name: code field syntax
290
* id:   bbecb600-37d1-418e-b825-fd8d36634643
35V LG84's avatar
35V LG84 committed
291 292

Feature-id::
293 294
* name: Txn description syntax
* id:   67bf0fd9-b7d9-4138-8a8f-be524ca3cbc5
35V LG84's avatar
35V LG84 committed
295

296
Feature-id::
35V LG84's avatar
35V LG84 committed
297
* name: metadata syntax
298
* id:   be31bd6b-9ece-4f5d-9179-3ca66f057339
35V LG84's avatar
35V LG84 committed
299 300


301
link:../../tests/tests-1009.yml[TEP-1009 T3DB]
35V LG84's avatar
35V LG84 committed
302 303

'''
35V LG84's avatar
35V LG84 committed
304
Tackler is distributed on an *"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND*,
35V LG84's avatar
35V LG84 committed
305
either express or implied. +
35V LG84's avatar
35V LG84 committed
306
See the link:../../LICENSE[License] for the specific language governing permissions
35V LG84's avatar
35V LG84 committed
307
and limitations under the link:../../LICENSE[License].