DE data views - Limit the displayed filter values according to the current data availability
As a .Stat DE user
In order to never end up with an empty data table or chart
I want to only see filter values for which data actually exists.
Scenario 1: The dataflow has some data: Display only filter values for which data exists
Given that I'm using the DE visualisation page
And that the dataflow has data
When the filter content is generated from the SDMX structure of the dataflow
Then the filter includes only items for which data actually exists (by applying actual content constraints on the codelists - see below). Default values (from SDMX annotations) are selected (if they are part of the items with data), unless there are valid selections provided through the URL parameters. In the case that no data can be displayed because of selections for which no data exists, the data table/view is replaced with the text "There is no data for the current selection. Please change the selection.".
Scenario 2: The dataflow has no data: There are no filter values
Given that I'm navigating directly to the DE visualisation page (maybe from a saved link)
And that the dataflow has no data
When the filter content is generated from the SDMX structure of the dataflow
Then by applying actual content constraints on the filters (see below) these filters become empty (without values) and the data table/view is replaced with the text "There is no data available.". In this case selections made through the URL parameters are not applied.
Scenario 3: Display all filter values (whether there are data or not)
Given that I'm navigating from the DLM to the DE visualisation page (without previous search)
When the filter content is generated from the SDMX structure of the dataflow
Then the content constraints are not applied to the filter values. In the case that data are not returned in the data call and:
- there is a current selection: then the data table/view is replaced with the text "There is no data for the current selection. Please change the selection." or
- there is no current selection: then the data table/view is replaced with the text "There is no data available."
Validity dates and types of content constraints to be considered:
- Content constraints are distinguished between Actual and Allowed by the parameter
type
. It is important to check for this parameter and only consider Actual content constraints. - The validity of Actual content constraints can be temporarily limited. This time limitation is provided in the
validFrom
andvalidTo
artefact parameters and it needs to be considered. Only the currently valid Actual content constraint should be used.
Technical notes:
- Applying this data availability can be achieved by matching the content of the codelists with the content of the actual content constraints that are also returned by the SDMX NSI web service when querying a dataflow with all its references.
E.g. http://nsi.qa.core.oecd.redpelicans.com/rest/dataflow/OECD/AIR_EMISSIONS_DF/1.0?references=all returns the ContentConstraint "OECD:CR_AIR_EMISSIONS_DF_2a0adfe25d64458cb2534b9454091c(1.0)" oftype="Actual"
which contains the list of countries ("COU"), pollutants ("POL") and Variables ("VAR") for which data exists. - The content constraint's CubeRegion with property include="true" contains dimension values with available data. Only those dimension values are to be displayed. Extreme case: If any one of the included dimensions is empty (no values) then the whole dataflow has no data (Scenario 2).
- The content constraint's CubeRegion with property include="false" contains dimension values for which no data are available. These dimension values are to be excluded from the display. Extreme case: If for any one of those dimensions all values (as defined in the codelist) are to be excluded then the whole dataflow has no data (Scenario 2).
- Each dimension can only be part of one single CubeRegion definition, thus either use include="true" or use include="false".
- The constraint for TIME_PERIOD dimension range uses the property "isInclusive".
Please note that the TIME_PERIOD range constraint uses datetime variables, but they have to be applied on SDMX time periods like "2015" (means whole of year 2015) or "2015-11" (means whole of November of 2015). Therefor "isInclusive":"true" indicates that this value should be the first (or last) datetime valid for the intended range, e.g. when dealing with an annual frequency, then
"startPeriod": {
"period": "1990-01-01T00:00:00Z",
"isInclusive": true
}
means: All time periods that start earliest 1990-01-01T00:00:00Z are included in the range: 1990, 1991, 1992, .... Any time period that starts before is excluded.
"endPeriod": {
"period": "2016-12-31T23:59:59Z",
"isInclusive": true
}
means: All time periods that end latest on 2016-12-31T23:59:59Z are included in the range: ...,2014, 2015, 2016. Any time period that ends later is excluded.
"isInclusive": "false" indicates that this value should be the last (or first) datetime non-valid for the intended range, e.g.
"startPeriod": {
"period": "1989-12-31T23:59:59Z",
"isInclusive": false
}
means: All time periods that start later than 1989-12-31T23:59:59Z are included in the range: 1990, 1991, 1992, .... Any time period that starts before or exactly at that datetime is excluded.
"endPeriod": {
"period": "2017-01-01T00:00:00Z",
"isInclusive": false
}
means: All time periods that end before 2017-01-01T00:00:00Z are included in the range: ...,2014, 2015, 2016. Any time period that ends exactly at that datetime or later is excluded.
Example
for dataflow with 2 actual content constraints with restricted validity dates: http://nsi-stable-siscc.redpelicans.com/rest/dataflow/OECD/AIR_EMISSIONS_DF/1.0?references=all&detail=referencepartial
This example also contains restrictions on the TIME_PERIOD dimension using the "isInclusive" property.
technical specs
-
SDMXJS add hasData
in dimension and dimension value -
SDMXJS content contraints are not returned by structure parser -
DE remove selector getContentConstraints (properly...) -
DE remove selector getContentConstrainedDimensions (properly...) -
DE renames selector getFilters into getDimensionsWithDataquerySelection and remove reject part -
DE update to handle reject part (use a reduce) -
VISIONS update with a prop displayed
(use a reduce) -
DE update props to add displayed
withhasData