Support for Point in Time Feature
At this stage, Data Explorer only supports pulling the Live version of data stored in .Stat. For a number of reasons (technical limitations of the Transfer service, and business policy), the ABS keeps all internal data under embargo. Data Explorer is a powerful method of allowing data-owning areas to validate their data, but this would require it to pull the PIT version of the data, not the Live version.
At the moment it's pretty simple to default all calls to get data to PIT. However, the Data Explorer uses ContentConstraints to limit which filters are available, and (naturally) only uses the Live ContentConstraint. This leads to incorrect filters when there is a Live ContentConstraint, and empty filters when there isn't. It's quite possible coming changes will make this "worse" for this specific use-case, given it's not actually a supported use-case.
I'm not sure precisely how this could or should be implemented. Some options could be:
- A toggle or selector in the UI (perhaps with the default value being configurable). A toggle would require DE to perform some logic to determine when the "PIT" was if it existed, though.
- A configuration value so the entire tenant always gets PIT or always gets Live
- A value on the Datasource, so some Datasources could be Live and some PIT
Technical details
- embargo for DSD: https://sis-cc.gitlab.io/dotstatsuite-documentation/using-api/embargo-management/
- embargo at obs level: https://sis-cc.gitlab.io/dotstatsuite-documentation/using-api/confidential-data/
Header name: X-Release
Header values: PIT