Support irregular time periods and distinct points in time in DE
As Yves or Edgardo,
I want irregular SDMX time periods to be displayed correctly/appropriately in tables and charts,
So that end-users can understand the underlying data.
Functional specifications
-
Irregular "time range" time periods as specified by the SDMX standard https://sdmx.org/wp-content/uploads/SDMX_2-1_SECTION_6_TechnicalNotes.pdf (p. 19ff): 749 4.2.7 Distinct Range 750 In the case that the reporting period does not fit into one of the prescribe periods 751 above, a distinct time range can be used. The value of these ranges is based on the 752 ISO 8601 time interval format of start/duration. Start can be expressed as either an 753 ISO 8601 date or a date-time, and duration is expressed as an ISO 8601 duration. 754 However, the duration can only be positive. … 761 Time Range: Start time and duration (YYYY-MM-DD(Thh:mm:ss)?/<duration>) … 826 <Series REF_PERIOD="2000-01-01T00:00:00/P2M"/>
where
<duration>
is based on https://www.w3.org/TR/xmlschema-2/#duration and is of formatP[nY][nM][nD][T[nH][nM][nS]]
,
but most often it isPnY
,PnM
,PnD
,PTnH
,PTnM
, orPTnS
(n
being a positive number).We do not need to support mixed durations, e.g.
P1Y2M
(sinceP14M
can be used instead).
Support for datetime periods withPnY
orPnM
, e.g. "2000-01-01T00:00:00/P5Y" or "2000-03-01T12:30:00/P1Y", is less priority and could be skipped/moved to other ticket if this would speed up the implementation.The display string can be determined as follow:
- Get
Y
,M
,D
, (h
,m
ands
) of the period start - Get the duration
- Get
Y
,M
,D
, (h
,m
ands
) of the period end - Determine the lowest datetime part required for the display of the period start and the period end:
- if there is a yearly duration and period start is at first day of Jan then: year
- if there is a yearly duration and period start is at first day of any other month then: month
- if there is a yearly duration and period start is at any other day then: day
- if there is a monthly duration and period start is at first day of month then: month
- if there is a monthly duration and period start is at any other day of month then: day
- if there is a daily duration and period start has no time then: day
- if there is a daily, hourly or minutely duration and period start has time without seconds then: minutes
- otherwise: seconds
- To construct the complete display format concatenate the period start string with "
–
" (long dash within two spaces) and with the period end string according to the specified localised monthly timeFormat, e.g.YYYY – YYYY
,YYYY-MM – YYYY-MM
,YYYY MMM – YYYY MMM
,MMM YYYY – MMM YYYY
,YYYY-MM-DD – YYYY-MM-DD
,YYYY-MM-DD hh:mm – YYYY-MM-DD hh:mm
orYYYY-MM-DD hh:mm:ss – YYYY-MM-DD hh:mm:ss
\textcolor{orange}{\text{except}}
:-
\textcolor{orange}{\text{if}}
the lowest datetime part required for the display is monthly,\textcolor{orange}{\text{if}}
the monthlytimeFormat
contains a space,
e.g.MMM YYYY
orYYYY MMM
,\textcolor{orange}{\text{and if}}
the period start and period end years are equal\textcolor{orange}{\text{then}}
merge the years resulting in
e.g.MMM – MMM YYYY
orYYYY MMM – MMM
. -
\textcolor{orange}{\text{if}}
the display includes hours, minutes or seconds\textcolor{orange}{\text{and if}}
the period start and period end dates (year, month, day) are equal\textcolor{orange}{\text{then}}
merge the dates resulting in e.g.YYYY-MM-DD hh:mm – hh:mm
orYYYY-MM-DD hh:mm:ss – hh:mm:ss
.
-
Examples:
- Get
Irregular time period | corresponds to | monthly timeFormat: default ("YYYY-MM") | locale: EN; monthly timeFormat: "YYYY-MMM" | locale: FR; monthly timeFormat: "MMM YYYY" |
---|---|---|---|---|
"2000-01-01/P5Y" | 5-year range 2000 to 2004 | 2000 – 2004 | 2000 – 2004 | 2000 – 2004 |
"2000-07-01/P1Y" | 1-year range Jul 2000 to Jun 2001 | 2000-07 – 2001-06 | 2000-Jul – 2001-Jun | Juil 2000 – Juin 2001 |
"2000-07-10/P1Y" | 1-year range 10 Jul 2000 to 09 Jun 2001 | 2000-07-10 – 2001-06-09 | 2000-07-10 – 2001-06-09 | 2000-07-10 – 2001-06-09 |
"2015-01-01/P3M" | 3-month range Jan 2015 to Mar 2015 | 2015-01 – 2015-03 | 2015-Jan – 2015-Mar | \textcolor{orange}{\text{Jan – Mar 2015}} |
"2015-02-01/P3M" | 3-month range Feb 2015 to Apr 2015 | 2015-02 – 2015-03 | 2015-Feb – 2015-Apr | \textcolor{orange}{\text{Fév – Avr 2015}} |
"2015-11-01/P3M" | 3-month range Nov 2015 to Jan 2016 | 2015-11 – 2016-01 | 2015-Nov – 2016-Jan | Nov 2015 – Jan 2016 |
"2015-11-10/P3M" | 3-month range 10 Nov 2015 to 09 Jan 2016 | 2015-11-10 – 2016-01-09 | 2015-11-10 – 2016-01-09 | 2015-11-10 – 2016-01-09 |
"2015-01-01/P7D" | 7-day range 01 Jan 2015 to 07 Jan 2015 | 2015-01-01 – 2015-01-07 | 2015-01-01 – 2015-01-07 | 2015-01-01 – 2015-01-07 |
"2015-01-30/P7D" | 7-day range 30 Jan 2015 to 05 Feb 2015 | 2015-01-30 – 2016-02-05 | 2015-01-30 – 2016-02-05 | 2015-01-30 – 2016-02-05 |
"2015-12-30/P7D" | 7-day range 30 Dec 2015 to 05 Jan 2016 | 2015-12-30 – 2016-01-05 | 2015-12-30 – 2016-01-05 | 2015-12-30 – 2016-01-05 |
"2000-03-01T12:30:00/P1D" | 1-year range 01 Mar 2000 12:30 to 02 March 2000 12:29 | 2000-03-01 12:30 – 2000-03-02 12:29 | 2000-03-01 12:30 – 2000-03-02 12:29 | 2000-03-01 12:30 – 2000-03-02 12:29 |
"2000-03-01T23:30:00/PT1H" | 1-hour range 01 Mar 2000 23:30 to 02 March 2000 00:29 | 2000-03-01 23:30 – 2000-03-02 00:29 | 2000-03-01 23:30 – 2000-03-02 00:29 | 2000-03-01 23:30 – 2000-03-02 00:29 |
"2000-03-01T12:30:00/PT30M" | 30-minute range 01 Mar 2000 12:30 to 01 March 2000 12:59 | \textcolor{orange}{\text{2000-03-01 12:30 – 12:59}} |
\textcolor{orange}{\text{2000-03-01 12:30 – 12:59}} |
\textcolor{orange}{\text{2000-03-01 12:30 – 12:59}} |
"2000-03-01T12:30:00/PT5S" | 5-second range 01 Mar 2000 12:30:00 to 01 March 2000 12:30:04 | \textcolor{orange}{\text{2000-03-01 12:30:00 – 12:30:04}} |
\textcolor{orange}{\text{2000-03-01 12:30:00 – 12:30:04}} |
\textcolor{orange}{\text{2000-03-01 12:30:00 – 12:30:04}} |
-
Also make sure to support time periods representing a single distinct point in time (rather than representing a time range). The SMDX representation is YYYY-MM-DDThh:mm:ss
(SDMX allows also reporting seconds fractionally - but support is not required). In this case, the period start and the period end are equal. It is to be displayed asYYYY-MM-DD hh:mm:ss
.
Technical aspects
- Irregular SDMX time periods are already supported in transfer and NSI since dotstatsuite-core-common#113 (closed)
- The usage of any SDMX time periods that include hours, minutes or seconds requires a special DSD annotation of type "SUPPORT_DATETIME", otherwise .Stat Core will not store and consider the time part.
"annotations": [{
"type": "SUPPORT_DATETIME"
}]
Examples
- Example for irregular time range:
Example of a table using "moving quarter" time period (taken from stat.ine.cl (v7)):- structure: CL1-DF_DESOC_PROV-1.0-all.xml
- data in .csv: Data_DF_DESOC_PROV_sdmx.csv
- data in EXcel: Tasa_de_desocupacion_por_provincia_-MovQuarter-_Dwnld_V7.xls
View in INE v7 (expected in .Stat DE):
-
Other example for irregular time range:
-
Example for single distinct point in time:
- structure: TEST_Multiple2.xml
- data: DSD_DEBT_TRANS_DF_QUEST_data_-Copy__1.csv