New rules for hierarchical dimension display in the data table
As DE user,
I want to see in the data table all currently selected (row & column) dimension items in a clear tree structure independently from data availability,
So that I can more easily understand the hierarchical structure.
This ticket does not apply for charts because empty observations for parents without data cannot be displayed.
Functional specifications
The related features for selecting items are addressed with these tickets, which are also listed under the epic Hierarchical selection and table display:
With these tickets, the generated SDMX data query always contains all currently selected dimension items, even those selected parents that do not have data. The table-generating engine can so easily know from the SDMX data query, which items (whether having data or not) were selected.
Only dimension items clearly selected in the filters and included in the SDMX data query are to be displayed in the table. Alone the fact of being returned by default by the NSI service is not sufficient to include an item into the table - It must have been manually or automatically selected. There is only one exception in case that there is no selection at all for the dimension, in which case all items returned by the NSI service (whether having data or not) except children without data, which themselves have no children with data, are being considered as selected.
Selected parents without data (without observations) should be displayed in the data table with a separate row. Their value cells should be empty instead of ..
, which is still to be used for observations without observation values.
Rule 1:
NOT_DISPLAYED continues to be applied only to single fixed dimensions or with the (CODE1+CODE2) syntax (like here or here). Those values remain hidden, even if they have parents to be displayed due to the ALWAYS_DISPLAY_PARENTS annotation. Since the child is not to be displayed there is no point in displaying its parents.
Vice-versa, if a parent (of a selected and to be displayed child) is marked as NOT_DISPLAYED but it is selected due to the ALWAYS_DISPLAY_PARENTS annotation or because the user has selected the parent together with the child, then both parent and child are to be displayed, as we are not in the case of a single fixed dimension or the (CODE1+CODE2) syntax anymore.
However, independently from the selections, NOT_DISPLAYED continues to be always applied within combined dimensions (like here).
Rule 2:
Whenever an empty hierarchical parent is to be added (with all value cells empty), then it is added only once and all following dimension cells on that axis (rows or columns) are to be left empty.** To illustrate this, see these 3 cases (tables can be constructed using this dataflow):
HIERARCHICAL_DIMENSION | OTHER_DIMENSION | ...COLUMNS |
---|---|---|
[MISSING_PARENT] | [EMPTY] | [EMPTY] |
· CHILD | A | ...data |
· CHILD | B | ...data |
· CHILD | C | ...data |
OTHER_DIMENSION | HIERARCHICAL_DIMENSION | ...COLUMNS |
---|---|---|
A | [MISSING_PARENT] | [EMPTY] |
A | · CHILD | ...data |
B | [MISSING_PARENT] | [EMPTY] |
B | · CHILD | ...data |
C | [MISSING_PARENT] | [EMPTY] |
C | · CHILD | ...data |
HIERARCHICAL_DIMENSION_1 | HIERARCHICAL_DIMENSION_2 | ...COLUMNS |
---|---|---|
[MISSING_ROOT_PARENT] | [EMPTY] | [EMPTY] |
· [MISSING_LEVEL1_PARENT] | [EMPTY] | [EMPTY] |
· · CHILD | [MISSING_ROOT_PARENT] | [EMPTY] |
· · CHILD | · [MISSING_LEVEL1_PARENT] | [EMPTY] |
· · CHILD | · · CHILD | ...data |
This is valid for the normal data table as well as for hierarchies in the microdata table.
Rule 3:
Also in combined dimensions, whenever an empty hierarchical parent is to be added (with all value cells empty), then it is added only once and all following dimensions in the combination are omitted from that combination.** To illustrate this, see this case:
COMB:X,Y (assuming that IDs and labels are the same)
X hierarchy (ALWAYS_DISPLAY_PARENTS) | Y hierarchy (ALWAYS_DISPLAY_PARENTS) |
---|---|
A (NOT_DISPLAYED, no data) | K (NOT_DISPLAYED, no data) |
· B (no data) | · L (no data) |
· · C | · · M |
· · D | · · N |
· E | · O |
F (NOT_DISPLAYED) | P (NOT_DISPLAYED) |
(X) | (Y) | --> COMB | ...COLUMNS |
---|---|---|---|
A | (A) A | [EMPTY] | |
A | K | [ROW OMITTED] | [ROW OMITTED] |
A | · L | [ROW OMITTED] | [ROW OMITTED] |
A | · · M | [ROW OMITTED] | [ROW OMITTED] |
A | · · N | [ROW OMITTED] | [ROW OMITTED] |
A | · O | [ROW OMITTED] | [ROW OMITTED] |
A | P | [ROW OMITTED] | [ROW OMITTED] |
· B | · (B) B | [EMPTY] | |
· B | K | [ROW OMITTED] | [ROW OMITTED] |
· B | · L | [ROW OMITTED] | [ROW OMITTED] |
· B | · · M | [ROW OMITTED] | [ROW OMITTED] |
· B | · · N | [ROW OMITTED] | [ROW OMITTED] |
· B | · O | [ROW OMITTED] | [ROW OMITTED] |
· B | P | [ROW OMITTED] | [ROW OMITTED] |
· · C | K | · · (C, K) C | [EMPTY] |
· · C | · L | · · · (C, L) C, L | [EMPTY] |
· · C | · · M | · · · · (C, M) C, M | ...data |
· · C | · · N | · · · · (C, N) C, N | ...data |
· · C | · O | · · · (C, O) C, O | ...data |
· · C | P | · · (C, P) C | ...data |
· · D | K | · · (D, K) D | [EMPTY] |
· · D | · L | · · · (D, L) D, L | [EMPTY] |
· · D | · · M | · · · · (D, M) D, M | [ROW OMITTED if combination at leave level doesn't have data] |
· · D | · · N | · · · · (D, N) D, N | ...data |
· · D | · O | · · · (D, O) D, O | ...data |
· · D | P | · · (D, P) D | ...data |
· E | K | · (E, K) E | [EMPTY] |
· E | · L | · · (E, L) E, L | [EMPTY] |
· E | · · M | · · · (E, M) E, M | ...data |
· E | · · N | · · · (E, N) E, N | ...data |
· E | · O | · · (E, O) E, O | ...data |
· E | P | · (E, P) E | ...data |
F | K | (F, K) F | [EMPTY] |
F | · L | · (F, L) L | [EMPTY] |
F | · · M | · · (F, M) M | ...data |
F | · · N | · · (F, N) N | ...data |
F | · O | · (F, O) O | ...data |
F | P | (F, P) F | ...data |
Note that rows of combinations at leave level are omitted if they have no data. The (grand-)parent(s) row without data would also be omitted if there are no child rows to be displayed.
Rule 4:
Concerning duplicated children due to HCLs (like here), the enforcement of the display of a parent through the ALWAYS_DISPLAY_PARENTS annotation and a user selection of a parent has the exact same effect in the table: Both, child and parent, are considered being selected and to be displayed in the table (row section using the concatenated syntax or rows/columns with additional empty rows/columns). If the same child element is displayed at different hierarchical positions (under different parents or at different levels) then all those occurrences should be displayed. If technically possible, only remove exact duplicates (same (grand-)parents and same level).
Note: This ticket here should not change anything on the algorithm that removes fully empty columns at the end of the table. We plan to change the table generation engine in the not too far future to become fully virtualised with continuous scrolling and by removing size limits on the table (to a certain degree). This would be the occasion to review this aspect.
Rule for row sections:
When a hierarchical dimension is displayed on the row section axis, and when a parent of a selected child is also selected, then display this parent as a prefix of the child, in a breadcrumb manner, whether the parent(s) has data or not.
Examples
Example 1: Parent without data auto-selected through ALWAYS_DISPLAY_PARENTS annotation
- Children with data selected (on level 3): Professional, scientific and technical services and Administrative and support services
- Annotation content:
PRODUCT=LEVEL1+LEVEL2
- Auto-selected parent without data on level 2 displayed in separate row: Professional, scientific and technical services; administrative and support services
- When displayed in rows
Source: https://de-qa.siscc.org/vis?lc=en&df[ds]=qa%3Astable&df[id]=DSD_NASU%40DF_SUPPLY_T1500&df[ag]=OECD.SDD.NAD&df[vs]=1.0&av=true&pd=2015%2C2015&dq=A.AUT...TS%2BP1%2BP7%2BP7ADJ%2BP33%2BD21X31%2BOTTM._T.CPA08_A03%2BCPA08_A02%2BCPA08_A01%2B_T%2BCPA08_A%2BCPA08_B%2BCPA08_M%2BCPA08_N%2B_X..B.Y%2BV.&ly[rw]=PRODUCT&ly[cl]=PRICE_BASE&to[TIME_PERIOD]=false&vw=tb&lo=1Before After - When displayed in columns
Source: https://de-qa.siscc.org/vis?lc=en&df%5bds%5d=qa%3Astable&df%5bid%5d=DSD_NASU%40DF_SUPPLY_T1500&df%5bag%5d=OECD.SDD.NAD&df%5bvs%5d=1.0&av=true&pd=2015%2C2015&dq=A.AUT...TS%2BP1%2BP7%2BP7ADJ%2BP33%2BD21X31%2BOTTM._T.CPA08_A03%2BCPA08_A02%2BCPA08_A01%2B_T%2BCPA08_A%2BCPA08_B%2BCPA08_M%2BCPA08_N%2B_X..B.Y%2BV.&ly%5brw%5d=PRICE_BASE&ly%5bcl%5d=PRODUCT&to%5bTIME_PERIOD%5d=false&vw=tb&lo=1Before After
Example 2: No selection
- All children with data are display as well as all parents with and without data. Children without data are not display.
- When displayed in rows
Source: https://de-qa.siscc.org/vis?lc=en&df[ds]=qa%3Astable&df[id]=DSD_NASU%40DF_SUPPLY_T1500&df[ag]=OECD.SDD.NAD&df[vs]=1.0&av=true&pd=2015%2C2015&dq=A.AUT...TS%2BP1%2BP7%2BP7ADJ%2BP33%2BD21X31%2BOTTM._T...B.V.&ly[rw]=PRODUCT&to[TIME_PERIOD]=false&vw=tb
Before After - When displayed in columns
Similar to rows, but in columns
Example 3: All parents (they all have data) auto-selected through ALWAYS_DISPLAY_PARENTS annotation
- Annotation title:
MEASURE
- Parents with data: Environmental and resource productivity; - CO2 Productivity; - Energy productivity; - Natural asset base; - Freshwater resources
- When displayed in rows
Source: https://data-explorer.oecd.org/vis?lc=en&df[ds]=Design&df[id]=DSD_GG%40DF_GREEN_GROWTH&df[ag]=OECD.ENV.EPI&df[vs]=1.0&av=true&pd=2017%2C&dq=AUS.A.ENVRESPROD%2BCO2%2BCO2_PBPROD%2BCO2_PBEMCAP%2BNRG%2BNRG_PROD%2BNRG_INT%2BNATBASE%2BWATER%2BWATER_FWCAP%2BWATER_FW.._T&ly[rw]=MEASURE%2CUNIT_MEASURE&ly[cl]=TIME_PERIOD&to[TIME_PERIOD]=false&vw=tb
No change to now because the parents have data and are already displayed in separate rows.
Note that cells still contain..
because the observations, while present, have no observation value. - When displayed in columns
Similar to rows, but in columns
Example 4: Combined dimensions when parents have or not data on auto-selected through ALWAYS_DISPLAY_PARENTS annotation
- Annotation content: 'COMBINED_TRANSACTION:TRANSACTION,ACCOUNTING_ENTRY'
- Source: https://de-qa.siscc.org/vis?lc=en&df[ds]=qa%3Astable&df[id]=DSD_NASEC10%40DF_TABLE14_HIER&df[ag]=OECD.SDD.NAD&df[vs]=1.0&av=true&pd=%2C&dq=A.NLD.S1...TR211%2BB1GQ%2BTU1%2BP2%2BD21X31%2BP51C%2BP13%2BP12%2BP11%2BP1%2BTR1._Z....V..&ly[cl]=TIME_PERIOD&ly[rw]=ACCOUNTING_ENTRY%2CTRANSACTION&to[TIME_PERIOD]=false&lo=5&lom=LASTNPERIODS&vw=tb
Before After