Implement the "available content constraint" in SDMX API
The dynamic data availability for a specific data selection needs to be obtained from the NSI web service through the "available content constraint", which is fully specified here: https://github.com/sdmx-twg/sdmx-rest/blob/master/v2_1/ws/rest/docs/4_6_1_other_queries.md
Note:
Eurostat already implemented the necessary changes in the NSI web service, but this now needs to be plugged into the data DB.
Original functional specifications
This constraint indicates the values of dimensions (or attributes) for which any data exists on the server for a Dataflow for a given partial data selection. Sometimes this constraint is also called Dynamic Content Constraint or Dynamic Data Availability.
The constraint can be queried through the following SDMX ws request, which includes the current data selection:
protocol://ws-entry-point/availableconstraint/AGENCYID,DATAFLOWID,VERSION/key/PROVIDERID/COMPONENTID?startPeriod=XXXX&endPeriod=XXXX&references=codelist&mode=exact
It is a query similar than for data queries. The componentId
parameter can be used to retrieve the values for one specific dimension only.
The query string parameters can be complemented with selections for startPeriod
, endPeriod
, and updatedAfter
. The parameter references=codelist
can be used to retrieve also the codelist(s) as references.
The Available Content Constraint is formed as a CubeRegion
. ReferencePeriod
could also be included for temporal coverage. A dedicated Annotation
could be returned to express metrics (like the number of matching series or observations). Its type is sdmx_metrics
, its ID one of series_count
or obs_count
and the title will contain the measure.
The mode=available
parameter instructs the web service to return a ContentConstraint containing value for which data exists for those dimension not part of the submitted data selection. With mode=exact
(default value), the web service includes also values for the dimensions that were part of the submitted data selection. Note that the returned values can exceed the submitted values. For instance, a request to retrieve exact
ly all dimension values for which data exists when FRANCE is selected, includes also AUSTRIA as dimension value if the same data also existed for AUSTRIA.
Examples:
- http://ws-entry-point/availableconstraint/ECB,ECB_EXR1_WEB,1.0/M..EUR.SP00.A/ECB?mode=available
- http://ws-entry-point/availableconstraint/ECB_EXR1_WEB/M..EUR.SP00.A/ECB (which is the same than http://ws-entry-point/availableconstraint/ECB_EXR1_WEB/M..EUR.SP00.A/ECB?mode=exact)
The DE should retrieve the Available Content Constraint for the corresponding dimension before opening a Filter in the Visualisation Page. This happens at first page load when input or default selections were applied or when the user switches/opens another dimension filter.
E.g. If currently the Indicator filter is opened and its GDP value is selected (and suppose that also the FREQUENCY filter had a section of the ANNUAL value) and then the user opens the Country filter (which closes the Indicator filter at the same time), the DE will request the Availability for the Country dimension. If the COUNTRY dimension doesn't have a selection than this query should be used:
http://ws-entry-point/availableconstraint/AGENCY,DATAFLOW,1.0/.GDP.A/AGENCY/REF_AREA?mode=available
If the COUNTRY dimension does have a selection than this query should be used:
http://ws-entry-point/availableconstraint/AGENCY,DATAFLOW,1.0/FRA.GDP.A/AGENCY/REF_AREA?mode=exact
The DE applies the Available Content Constraint to the newly opened dimension filter by greying out and making unselectable all currently displayed values that are not included in the query result. Previously greyed out and unselectable dimension values that are now included in the Available Content Constraint are turned back into normal mode and become again selectable.
Note: In order to guarantee an optimal user experience, the server request and the application of the constraint in the filter must be so fast that they are unnoticeable by the user. Depending on the speed of the SDMX web service, an alternative approach could be used, where each user selection change triggers a background job to retrieve the availability for all other dimensions even before the user switches to another dimension, e.g.
http://ws-entry-point/availableconstraint/AGENCY,DATAFLOW,1.0/.GDP.A/AGENCY?mode=exact
Note: For a full restful syntax of the SDMX web service, please see https://github.com/sdmx-twg/sdmx-rest/blob/master/v2_1/ws/rest/docs/4_6_1_other_queries.md