Improve the performance of reading time dimension
Currently when processing data to be imported, there is 1 step that gets the start time and end time from the SDMX time representation (GetPeriod
). This process is repeaded for every observation found in the source data.
1.01 % Fill • 149,193 ms • DotStat.Db.Reader.SdmxObservationReader.Fill(Object[])
0.07 % Attr • 10,480 ms • DotStat.DB.Util.DataflowMetadataBuilder.Attr(String)
0.07 % GetPeriod • 9,917 ms • DotStat.DB.DateUtils.GetPeriod(String, String)
0.05 % ToGregorianPeriod • 7,813 ms • Org.Sdmxsource.Sdmx.Util.Date.ReportingTimePeriod.ToGregorianPeriod(String, String)
0.03 % GetTimeFormatOfDate • 3,910 ms • Org.Sdmxsource.Sdmx.Util.Date.DateUtil.GetTimeFormatOfDate(String)
0.01 % Create • 2,051 ms • Org.Sdmxsource.Sdmx.Util.Date.PeriodicityFactory.Create(TimeFormatEnumType)
0.01 % TryGetValue • 1,892 ms • System.Collections.Generic.Dictionary`2.TryGetValue(TKey, out TValue)
0.01 % 9 functions hidden • 1,197 ms total
0.02 % FormatDate • 2,966 ms • Org.Sdmxsource.Sdmx.Util.Date.DateUtil.FormatDate(Object, Boolean)
0.01 % ToDateTime • 1,738 ms • Org.Sdmxsource.Sdmx.Util.Date.BasePeriodicity.ToDateTime(String, Boolean)
0.01 % GetTimeFormatOfDate • 1,101 ms • Org.Sdmxsource.Sdmx.Util.Date.DateUtil.GetTimeFormatOfDate(String)
0.00 % Create • 80 ms • Org.Sdmxsource.Sdmx.Util.Date.PeriodicityFactory.Create(TimeFormatEnumType)
0.00 % 5 functions hidden • 661 ms total
0.01 % AddPeriod • 940 ms • Org.Sdmxsource.Sdmx.Util.Date.Quarterly.AddPeriod(DateTime)
0.00 % 9 functions hidden • 703 ms total
0.03 % GetObjectFromString • 4,683 ms • DotStat.Db.DbExtensions.GetObjectFromString(Attribute, String, ICodeTranslator, StagingRowActionEnum)
0.03 % TranslateCodeToId • 4,052 ms • DotStat.Db.Util.CodeTranslator.TranslateCodeToId(String, String)
0.02 % GetPrimaryMeasureObjectFromString • 3,355 ms •
-
A quick performance improvement is to store in memory the previously read SDMX time values, so we avoid to extract the information multiple times for the same value. -
Data -
Referential metadata
-
Edited by Pedro Carranza