Extend DLM with advanced transfer features
As Anastassia,
I want to be able to use all the advanced transfer features (import, transfer) directly from the DLM,
So that I do not have to use the Swagger interface for those.
Specifications:
Extend the data upload screen as follow:
Below "Spaces" add an accordion component titled: "More options: Data validations & Embargo" closed by default,
which contains two sub-components:
- "Data validations" using the materialUI "VerifiedUser" icon
- "Embargo (point-in-time) release management" using the materialUI "Update" icon:
old: image
Extend the data transfer screen as follow:
Below "Partial data query" add an accordion component titled: "More options: Data validations & Embargo" closed by default,
which contains two sub-components:
- "Data validations" using the materialUI "VerifiedUser" icon
- "Embargo (point-in-time) release management" using the materialUI "Update" icon:
old: image
Note: This Embargo component instance has additional parameters for "Source version".
Functional notes:
- Default values are: "Favour speed..." and "Live"
- The release date-time has a user-friendly date-time picker (showing "today" as default). Any date-time from current date-time onward can be chosen.
- The "Release date-time" and "Allow for a later emergency restoration..." are only accessible if target "Embargo" is selected and if data validation != "Only run adv. validations without..."
- The question mark icon for data validations should link to the functional doc: https://sis-cc.gitlab.io/dotstatsuite-documentation/using-api/api-main-features/#data-validation-process
Technical notes:
- Only for internal data transfer:
- If source "Live" then set sourceVersion = 0 (default)
- If source "Embargo" then set sourceVersion = 1
- If target "Live" then set targetVersion = 0 (default), PITReleaseDate = null (default), restorationOptionRequired = false (default)
- If target "Embargo" then set targetVersion = 1 and set PITReleaseDate to selected date-time (current date-time by default)
- The PITReleaseDate parameter submitted to the transfer should include the timezone of the client, according to technical standards (see: https://www.w3.org/TR/NOTE-datetime). Example:
1997-07-16T19:20:30.45+01:00
. This ticket implements the required changes of the transfer service to take the timezone shift (compared to the server timezone) into account. - If "Allow for a later emergency restoration..." is checked then set restorationOptionRequired = true
- If "Favour speed..." then set validationType = 0 (default)
- If "Apply advanced validations" then set validationType = 1
- If "Only run adv. validations without..." then:
- Instead of /import/sdmxFile use the /validate/sdmxFile method (set targetVersion according to user selection)
- Instead of /import/excel use the /validate/excel method (set targetVersion according to user selection)
- Instead of /transfer/dataflow use the /validate/transferDataflow method (set sourceVersion and targetVersion according to user selection)
Advanced transfer service method parameters:
- /import/sdmxFile
- /import/excel
- /transfer/dataflow
Name | Type | Description | Requirement |
---|---|---|---|
sourceVersion | integer | Only in /transfer/dataflow method. Indicates which version of the tables should be used as source (Live or PIT). Available values: 0 (Live), 1 (PIT) - Default: 0 (Live) |
Optional |
targetVersion | integer | Indicates which version of the tables should be targeted (LIVE or PIT). In case there is already a PIT release defined on the DSD, all transaction on this DSD have to target PIT release irrespectively to the value of this parameter. Available values : 0, 1 - Default LIVE (0) When target version is LIVE version, PITReleaseDate is set to null and restorationOptionRequired is set to false |
Optional |
PITReleaseDate | string | When provided, this date will be the date and time when the PIT release is activated (i.e. is made available for data queries). When not provided, the PIT release won't be activated automatically. If its value refers to a date and time in the past, the PIT release should be immediately activated and release right at the end of the transaction. Format dd-MM-yyyy HH:mm:ss ex. 17-05-2019 12:30:00 |
Optional |
restorationOptionRequired | boolean | Optional parameter. Indicates if the current LIVE version should be kept for restoration purposes when PIT release becomes active. The value of this parameter is ignored when the targetVersion is not set to PIT. | Optional |
validationType | integer | Optional parameter. Indicates if the import process should do a basic or a full validation of the data (0 = basic validation, 1 = full validation). Default basic validation | Optional |
- /validate/excel
- /validate/sdmxFile
- /validate/transferDataflow
Name | Type | Description | Requirement |
---|---|---|---|
sourceVersion | integer | Only in /transfer/dataflow method. Indicates which version of the tables should be used as source (Live or PIT). Available values: 0 (Live), 1 (PIT) - Default: 0 (Live) |
Optional |
targetVersion | integer | Indicates which version of the tables should be targeted (LIVE or PIT). In case there is already a PIT release defined on the DSD, all transaction on this DSD have to target PIT release irrespectively to the value of this parameter. Available values : 0, 1 - Default LIVE (0) |
Optional |