undertime issueshttps://gitlab.com/anarcat/undertime/-/issues2023-07-21T03:41:37Zhttps://gitlab.com/anarcat/undertime/-/issues/28broader location searches2023-07-21T03:41:37ZAntoine Beauprébroader location searchesthe tzdata database is pretty awesome: it provides us with *lots* of cities and places around the world, and allows us to (generally) reliably tell what time it is over there.
but it's far from an exhaustive list of places. if i ask, fo...the tzdata database is pretty awesome: it provides us with *lots* of cities and places around the world, and allows us to (generally) reliably tell what time it is over there.
but it's far from an exhaustive list of places. if i ask, for example, about the time in Bruges, `undertime` gets confused quickly:
```
anarcat@angela:~$ undertime --timezone Bruges
WARNING: unknown zone, skipping: Bruges
ERROR: No valid time zone found.
anarcat@angela:~[1]$
```
I mean at least it fails *fast* (well, relatively: 200ms, see also #25) and doesn't give a garbage answer. but it's not very satisfying for people that live in Bruges.
(Interestingly, I have no frigging clue what the timeone is in Bruges. Belgium, is that aligned with Paris? Or Berlin? Or Amsterdam? Why do I need to know all of this? I just want to know what time it is in Bruges damnit.)
(Of course the answer is "yes", to all questions, and time is UTC+1 except in the summer (which needs to be defined here) where it is UTC+2, except it Europe unblocks a proposal to [abolish daylight savings](https://en.wikipedia.org/wiki/Summer_time_in_Europe#Future). This is hard folks.)
Anyway. There Must Be A Better Way...
Turns out: there is! I found this thing called [Geonames](https://www.geonames.org/) that has a plethora of Names of Places, whatever that means. It's a gigantic (361MB) database of places, built as basically a TSV (tab-separated value, which some people mysteriously call a CSV). It's almost usable:
```
anarcat@angela:Downloads$ grep Bruges BE.txt | head -1
2783074 Kanaal Brugge-Oostende Kanaal Brugge-Oostende Canal d'Ostende a Bruges,Canal d'Ostende à Bruges,Kanaal Brugge-Oostende,Oostende-Brugge 51.23333 2.93333 H CNL BE VLG VWV 04 Europe/Brussels 2019-12-03
```
that, of course, is not Bruges, it's a canal in Bruges. But, maybe, good enough?
Turns out there's a *lot* of shit in that database. The above is, indeed a canal (`CNL` is the [code](https://www.geonames.org/export/codes.html)), and not the city. For that, you need to restrict your search to a "populated place" (`PPL`):
```
anarcat@angela:Downloads$ grep Bruges BE.txt | grep PPL
2786581 Sint-Michiels Sint-Michiels Saint-Michel,Saint-Michel-lez-Bruges,Sint-Michiels 51.18806 3.21142 P PPL BE VLG VWV 31 31005 12297 7 Europe/Brussels 2020-05-25
2800931 Brugge Brugge Brige,Briuge,Briugė,Briz,Brizh,Briž,Brjuge,Brjugge,Broegge,Brudje,Bruegge,Brugae,Brugeh,Bruges,Brugge,Bruggy,Brugia,Brugo,Bruhhe,Brujas,Bruxas,Bruza,Bruĝo,Brycg,Brygge,Bryugge,Bryz,Brögge,Brügge,beulwiheo,briuge,bruch,bruja,brwj,brwkhh,brwyz,brwz,bu lu he,bu lu ri li shi zhong xin,buruhhe,Μπρυζ,Бриж,Бругэ,Брюгге,Брюге,Բրյուգգե,ברוז,بروج,بروخه,برویز,ब्रूज,บรูช,ბრიუგე,ブルッヘ,布吕赫,布鲁日历史中心,브뤼허 51.208923.22424 P PPL BE VLG VWV 31 31005 118509 13 Europe/Brussels 2023-02-19
```
... but even that doesn't give us the right answer at first! See, only the *second* entry here is the actual city of Bruges...
Obviously, all of those results actually give us the right answer anyways (`Europe/Brussels` is, as it turns out, the IANA zone, which undertime does know about). But that's kind of a coincidence: in the above we're grepping the `BE.txt` dataset... if we'd look around the world, we'd find there's also a [Bruges in France](https://www.geonames.org/3029799/bruges.html) (because of course there fucking is).
It might be easier to have undertime just talk to their web service instead. This, for example, gives us the right answer if we just take the first hit:
https://www.geonames.org/search.html?q=bruges&country=
but this is where the rubber meets the proverbial road, so to speak: then you get into their API thingie and it's unclear if you can just use that for free... at the very least you need to register.
So anyway, that's my research into this... if anyone else knows of a good (free?) way to get name -> TZ information that's broader than just tzdata, i'm all ears!https://gitlab.com/anarcat/undertime/-/issues/27consider switching from pytz to zoneinfo2023-05-16T18:58:54ZAntoine Beaupréconsider switching from pytz to zoneinfoPython 3.9 now has a zoneinfo package, backported up to 3.6+ in [this pypi package](https://pypi.org/project/backports.zoneinfo/).
According to [this blog post from 2018](https://blog.ganssle.io/articles/2018/03/pytz-fastest-footgun.htm...Python 3.9 now has a zoneinfo package, backported up to 3.6+ in [this pypi package](https://pypi.org/project/backports.zoneinfo/).
According to [this blog post from 2018](https://blog.ganssle.io/articles/2018/03/pytz-fastest-footgun.html), we should *really* switch to zoneinfo, *particularly* since we use it to load actual time zones in `guess_zone`... (Actually, they suggest switching to [dateutil](https://github.com/dateutil/dateutil).tz, but that advice is [apparently outdated](https://news.ycombinator.com/item?id=35914039).)
That said, according to [this LWN article from 2020](https://lwn.net/Articles/813691/), the pytz actually planned to make it a wrapper around the standard library, so we might actually be safe.
The task here, I guess, is to add a unit test that tries to reproduce the issues described in the bug report and, if present, fix the issue by switching away from pytz, or similar.
One issue with the switch is that the stdlib implementation is about a milisecond slower, which could be a problem in our quest for perforamnce (#25).https://gitlab.com/anarcat/undertime/-/issues/26switch from python-ephem to python-skyfield?2022-08-15T15:55:26ZVagrant Cascadianswitch from python-ephem to python-skyfield?Thanks for undertime, it looks to be very useful!
Looking at the website for pyephem, https://rhodesmill.org/pyephem/ it suggests that new projects should use skyfield instead:
The Skyfield astronomy library should be preferred ove...Thanks for undertime, it looks to be very useful!
Looking at the website for pyephem, https://rhodesmill.org/pyephem/ it suggests that new projects should use skyfield instead:
The Skyfield astronomy library should be preferred over PyEphem for new projects. Its modern design encourages better
Python code, and uses NumPy to accelerate its calculations.
https://rhodesmill.org/skyfield/
It is developed by the same person.
No idea how complicated a migration would be, if it is maybe overkill, it would also drag in a dependecy on numpy, and is not yet packaged in Debian...https://gitlab.com/anarcat/undertime/-/issues/25profile undertime to go back to sub 100ms performance2023-07-21T03:41:38ZAntoine Beaupréprofile undertime to go back to sub 100ms performanceone of the reasons i wrote undertime is that loading a web page is slow and annoying.
undertime, therefore, must be really snappy. calculating dates is pretty fast and there's no reason why the user should "feel" undertime "thinking".
...one of the reasons i wrote undertime is that loading a web page is slow and annoying.
undertime, therefore, must be really snappy. calculating dates is pretty fast and there's no reason why the user should "feel" undertime "thinking".
in commit 598c211b78c9f400d961a35821a4986cf2532ebf I actually made a detailed analysis of the performance changes over the years, but basically, we had < 100ms (94-98ms, but still) performance in 1.0.0 through 1.5.0, and dateparser through that completely out the window by crashing us back to ~400ms, in 1.6.0.
now, since 97006512be1c78dd6a603e2fbfe8621cd9be4b23 (not released yet), we have implemented delayed imports which make the default `./undertime` much faster (130ms). and in 598c211b78c9f400d961a35821a4986cf2532ebf we promoted parsedatetime back to the first in the parser queue, and it also makes us much faster. but we're still above 100ms.
in the analysis, I outlined `--version` and `--config` as features that might be the case of that extra 30-40 ms we suffer from right now.
the task list here is:
* [x] <del>consider switching the dependency from dateparser back to parsedatime</del>
* [ ] document this mess more clearly
* [ ] run a profiler to see where else we could cut down some timehttps://gitlab.com/anarcat/undertime/-/issues/2Group locations in same timezone2018-12-21T19:58:19ZVaracGroup locations in same timezoneFirst of all, thx for this neat project, it's very helpful for me!
When specifying multiple locations in same timezone (i.e. `Europe/London` and `UTC`), right now only `UTC` will get displayed.
```
» undertime Los_Angeles EST Bogota N...First of all, thx for this neat project, it's very helpful for me!
When specifying multiple locations in same timezone (i.e. `Europe/London` and `UTC`), right now only `UTC` will get displayed.
```
» undertime Los_Angeles EST Bogota New_York Sao_Paulo UTC London Europe/Berlin
WARNING:root:skipping zone America/Bogota with existing offset -5:00:00
WARNING:root:skipping zone America/New_York with existing offset -5:00:00
WARNING:root:skipping zone Europe/London with existing offset 0:00:00
WARNING:root:skipping zone Europe/Berlin with existing offset 1:00:00
┌───────┬─────────────────────┬───────┬───────────────────┬───────┐
│ CET │ America/Los_Angeles │ EST │ America/Sao_Paulo │ UTC │
├───────┼─────────────────────┼───────┼───────────────────┼───────┤
│ 00:00 │ 15:00 │ 18:00 │ 20:00 │ 23:00 │
```
It would be nice if all given locations from the same timezone were grouped and displayed in the title of the colum, like:
```
» undertime Los_Angeles EST Bogota New_York Sao_Paulo UTC London Europe/Berlin
┌───────┬─────────────────────┬───────┬───────────────────┬───────┐
│ CET │ America/Los_Angeles │ EST │ America/Sao_Paulo │ UTC │
│ │ │ │ │London │
├───────┼─────────────────────┼───────┼───────────────────┼───────┤
│ 00:00 │ 15:00 │ 18:00 │ 20:00 │ 23:00 │
```