Full Unit of measure (UoM) display in the data table
As a .Stat DE user
In order to easily understand the basic context of data shown in the data table (according to standard statistical modelling)
I want to see directly in the data table the predefined standard statistical concepts defining a full Unit of measure (UoM) even if they are defined in different attributes.
Reminders
In SDMX, an attribute is a component of the data structure, similar to a dimension, but it does not allow uniquely identifying an observation. It contains additional information about the data characteristics in coded (--> Codelist) or uncoded form (e.g. free text).
Also like for dimensions, an attribute links to the Concept that defines its localised name and semantic. The localised name is used for display purposes (see details below).
Some attributes and some dimensions that play a specific role (e.g. Unit of measure, Unit multiplier, Base period) will be interpreted, displayed differently or used to adapt the display (see the related rules below).
Note:
Preferred scale and Decimals attributes are not part of this ticket, as they have already been implemented.
See configuration (documentation) for:
"prefscale": "PREF_SCALE",
"decimals": "DECIMALS"
Proposed modelling approach
2 pre-defined dataflow-level annotations are used:
- UNIT_MEASURE_CONCEPTS (optional): List of dimension and/or attribute IDs used to define the full "Unit of measure" and which could be used to auto-generate the according labels (unless the UNIT_MEASURE_LABEL annotation is present). By default in case the annotation is not given, it is supposed to contain the following list of concepts: "UNIT_MEASURE,UNIT_MULT,BASE_PER" provided that these concepts are used in the DSD of the displayed data. If the list is empty, then there is no special display of UoM.
<common:Annotation>
<common:AnnotationTitle>UNIT_MEASURE,UNIT_MULT,BASE_PER</common:AnnotationTitle>
<common:AnnotationType>UNIT_MEASURE_CONCEPTS</common:AnnotationType>
</common:Annotation>
- UNIT_MEASURE_LABEL (optional): ID of the attribute those values contain the handcrafted, literal UoM labels to be used in contexts where editorial precision matters. By default, if this annotation is not provided, then UoM labels are auto-generated by concatenating all the (localised) dimension and/or attribute labels in the given order defined in UNIT_MEASURE_CONCEPTS and separated by ", ".
<common:Annotation>
<common:AnnotationTitle>UNIT_MEASURE_LABEL_ATTRIBUTE</common:AnnotationTitle>
<common:AnnotationType>UNIT_MEASURE_LABEL</common:AnnotationType>
</common:Annotation>
In this case, the producer of the dataset is expected to provide the editorially correct unit of measure labels, as a coded attribute. The UNIT_MEASURE_CONCEPTS are still necessary to indicate which components have to be replaced with the content indicated in the UNIT_MEASURE_LABEL attribute.
The attachment level of the full UoM can be automatically obtained by composing the dimensions directly contained in the UNIT_MEASURE_CONCEPTS with the dimensions defined in any of the underlying attachment levels of the attributes contained in the UNIT_MEASURE_CONCEPTS. In case no dimensions are found (UNIT_MEASURE_CONCEPTS has no dimension IDs and all attributes in UNIT_MEASURE_CONCEPTS are attached at dataset level), then the UoM does not vary for the different values of the data table, and are to be displayed in the table header (see below).
Modelling example
See a stylised DSD (inspired by National Accounts) with the following concepts and codelist samples:
The concepts of the dimensions and attributes highlighted above in red are listed in the UNIT_MEASURE_CONCEPTS. The underlying dimension attachment of the UnitMultiplier and BaseYear attributes are supposed to be Measure and Country.
Therefor:
- UNIT_MEASURE_CONCEPTS should thus be composed of the IDs of: Transformation, UnitMultiplier, UnitMeasure, PriceBase, BaseYear and Adjustment
- UNIT_MEASURE_LABEL would not be used here because there is no specific user-provided attribute, e.g. UnitMeasureLabel, that carries the labels. Concatenated labels will be auto-generated by the DE. A typical full UoM label could read in the above defined order: "Non-transformed, Thousands, Australian Dollar, Current Price, Not-applicable, Non-seasonally adjusted".
- The derived attachment level of the full UoM would be: Measure, Country, Transformation, UnitMeasure, PriceBase and Adjustment
Display in the Data Explorer
The following describes the business rules to determine how Units of Measure (UoM) appear in the Data Explorer tables, and depending on the dimensions of the derived attachment level and their distribution over the table (layout). UNIT_MEASURE_CONCEPTS provides the ordered items which need to be concatenated (in that order) into a UoM label unless the label is explicitly provided as UNIT_MEASURE_LABEL annotation attached at the appropriate level (dataset, group/series, observation).
Case 1
If there are no dimensions in the derived attachment level of the full UoM (in case UNIT_MEASURE_CONCEPTS contains only attribute concepts and all these attributes are attached at dataset level) or if all those dimensions are in the table header due to single dimension values: means that the UoM label is the same for all cells in the table.
The UoM label (if auto-generated then including all the labels for dimensions in UNIT_MEASURE_CONCEPTS) is shown in the second table subtitle. The dimensions defined in UNIT_MEASURE_CONCEPTS are not displayed separately in the first table subtitle.
Example:
National Accounts
Frequency: Annual * Country: Total * Measure: GDP
Unit: Non-transformed, Thousands, Australian Dollar, Current Price, Non-seasonally adjusted
Value | ||
---|---|---|
Time period | ||
2000 | 138.6418738 | |
2001 | 113.7983151 | |
2002 | 136.3920959 |
Note: The dimensions UnitMeasure, PriceBase, Adjustment, Transformation had single values, were all to be placed in the table header, and have thus been merged into the UoM label.
Note: "Not-applicable" item labels (having ID "_Z") are excluded from the UoM label, here BaseYear.
Case 2
At least one dimension in the derived attachment level of the full UoM is on the Row axis of the table but none of those dimensions are on the Column axis: the UoM label is the same for all cells of each row of the table.
The UoM labels (if auto-generated then excluding the labels for dimensions in UNIT_MEASURE_CONCEPTS not placed on the Row axis) are displayed in an additional "Unit" column. The dimensions defined in UNIT_MEASURE_CONCEPTS displayed normally in the Row axis are not displayed separately.
Example:
National Accounts
Frequency: Annual * Measure: GDP
Time period | 2001 | 2002 | 2003 | ||
---|---|---|---|---|---|
Country | Unit | ||||
Adjustment: Non-seasonally adjusted | |||||
Australia | Non-transformed, Thousands, Australian Dollar, Current Price | 138.6418738 | 137.1236678 | 136.6536471 | |
Belgium | Non-transformed, Thousands, Euro, Current Price | 113.7983151 | 113.117064 | 113.5936765 | |
Canada | Non-transformed, Thousands, Canadian Dollar, Current Price | 136.3920959 | 134.0008176 | 135.2551795 | |
Adjustment: Seasonally adjusted | |||||
Australia | Non-transformed, Thousands, Australian Dollar, Current Price | 134.4480365 | 134.154608 | 133.8576924 | |
Belgium | Non-transformed, Thousands, Euro, Current Price | 126.1257334 | 126.0543631 | 126.0131707 | |
Canada | Non-transformed, Thousands, Canadian Dollar, Current Price | 138.4802883 | 138.0792699 | 137.6584535 |
Note: The dimension Measure has only one value and is placed in the header. The dimensions Country and UnitMeasure were placed on rows by the user. The dimension Adjustment was placed on row sections by the user. The dimensions PriceBase and Transformation had single values, were normally to be placed in the table header, but have been merged into the UoM label at the lowest level (row).
Note: "Not-applicable" item labels (having ID "_Z") are excluded from the UoM label, here BaseYear.
Case 3
At least one dimension in the derived attachment level of the full UoM is on the Column axis of the table but none of those dimensions are on the Row or Row section axes: the UoM label is the same for all cells of each column of the table.
The UoM labels (if auto-generated then excluding the labels for dimensions in UNIT_MEASURE_CONCEPTS not placed on the Column axis) are displayed in an additional "Unit" row. The dimensions defined in UNIT_MEASURE_CONCEPTS displayed normally on the Column axis are not displayed separately.
Example:
National Accounts
Frequency: Annual
Country | Australia | Australia | Australia | Australia | Belgium | Belgium | Belgium | Belgium | |
---|---|---|---|---|---|---|---|---|---|
Measure | GDP | GDP | Investments | Investments | GDP | GDP | Investments | Investments | |
Unit | Non-transformed, Thousands, Australian Dollar, Current Price, Non-seasonally adjusted | Non-transformed, Thousands, Australian Dollar, Constant Price, Non-seasonally adjusted | Non-transformed, Thousands, Australian Dollar, Current Price, Non-seasonally adjusted | Non-transformed, Thousands, Australian Dollar, Constant Price, Non-seasonally adjusted | Non-transformed, Thousands, Euro, Current Price, Non-seasonally adjusted | Non-transformed, Thousands, Euro, Constant Price, Non-seasonally adjusted | Non-transformed, Thousands, Euro, Current Price, Non-seasonally adjusted | Non-transformed, Thousands, Euro, Constant Price, Non-seasonally adjusted | |
Time period | |||||||||
2001 | 138.641874 | 113.798315 | 136.392096 | 134.448037 | 126.125733 | 138.480288 | 138.641874 | 113.798315 | |
2002 | 137.123668 | 113.117064 | 134.000818 | 134.154608 | 126.054363 | 138.079270 | 137.123668 | 113.117064 | |
2003 | 136.653647 | 113.593676 | 135.255179 | 133.857692 | 126.013171 | 137.658454 | 136.653647 | 113.593676 |
Note: The dimensions Country, Measure, UnitMeasure and PriceBase were placed on columns by the user. The dimensions Adjustment and Transformation had single values, were normally to be placed in the table header, but have been merged into the UoM label.
Note: "Not-applicable" item labels (having ID "_Z") are excluded from the UoM label, here BaseYear.
Case 4
At least one dimension in the derived attachment level of the full UoM is on the Row section axis of the table but none of those dimensions are on the Row or Column axes: the UoM label is the same for all cells of each row section of the table.
The UoM labels (if auto-generated then excluding the labels for dimensions in UNIT_MEASURE_CONCEPTS not placed in the Row section axis) are displayed in an additional "Row section" line. The dimensions defined in UNIT_MEASURE_CONCEPTS displayed normally in the Row section axis are not displayed separately.
Example:
National Accounts
Frequency: Annual * Measure: GDP
Values | ||
---|---|---|
Time period | ||
Country: Australia Unit: Non-transformed, Thousands, Australian Dollar, Current Price, Seasonally adjusted |
||
2000 | 138.6418738 | |
2001 | 113.7983151 | |
2002 | 136.3920959 | |
Country: Australia Unit: Non-transformed, Thousands, Australian Dollar, Current Price, Non-seasonally adjusted |
||
2000 | 134.4480365 | |
2001 | 126.1257334 | |
2002 | 138.4802883 | |
Country: Belgium Unit: Non-transformed, Thousands, Euro, Current Price, Seasonally adjusted |
||
2000 | 138.6418738 | |
2001 | 113.7983151 | |
2002 | 136.3920959 | |
Country: Belgium Unit: Non-transformed, Thousands, Euro, Current Price, Non-seasonally adjusted |
||
2000 | 134.4480365 | |
2001 | 126.1257334 | |
2002 | 138.4802883 |
Note: The dimension Measure had a single selection and is placed in the header. The dimensions Country, UnitMeasure and Adjustment were placed on row sections by the user. The dimensions PriceBase and Transformation had single values, were normally to be placed in the table header, but have been merged into the UoM label.
Note: "Not-applicable" item labels (having ID "_Z") are excluded from the UoM label, here BaseYear.
Case 5
All other cases. It means that at least one dimension in the derived attachment level of the full UoM is on the Column axis of the table and at least one other of those dimensons is either in the Row or Column axes: the UoM label is potentially different for each cells in the table.
The UoM labels (if auto-generated then excluding the labels for dimensions in UNIT_MEASURE_CONCEPTS) are displayed as a footnote.
Example:
National Accounts
Frequency: Annual * Measure: GDP
Adjustment | Non-seasonally adjusted | Seasonally adjusted | |
---|---|---|---|
Country | |||
Australia | (*) 138.641873833379 | (*) 113.798315117276 | |
Belgium | (*) 134.448036539148 | (*) 126.125733415802 |
Note: The dimension Measure had a single selection and is placed in the header. The dimensions Country and UnitMeasure were placed on rows by the user and the dimension Adjustment was placed on columns by the user. The dimensions PriceBase and Transformation had single values, were normally to be placed in the table header, but have been merged into the UoM label.
Note: "Not-applicable" item labels (having ID "_Z") are excluded from the UoM label, here BaseYear.
Charts
Data Explorer charts should behave in the same way as tables for what concerns the construction of UoM for the chart header.
Live example:
- http://nsi-reset-siscc.redpelicans.com/rest/dataflow/OECD/DF_UOM/3.0
- http://de-qa-oecd.redpelicans.com/vis?dataquery=.A.GDP..V%2BQ.SA%2BNSA.NT&period=2000%2C2002&frequency=A&locale=en&dataflow[datasourceId]=staging%3ASIS-CC-reset&dataflow[dataflowId]=DF_UOM&dataflow[agencyId]=OECD&dataflow[version]=3.0&withoutContentConstraints=false&filter=PANEL_CONTENT_CONSTRAINTS
- Data to reproduce:
- Structures: \\dotstatcor-dev1\Temp\Uploads\UOM_Multiple.xml: UOM_Multiple.xml
- Data: <\\dotstatcor-dev1\Temp\Uploads\National Accounts (UoM) data.csv> (Dataflow "National Accounts (UoM)": OECD:DF_UOM(3.0)): National_Accounts__UoM__data.csv