Skip to content

UTC timezone being appended to every prebuilt datetime object

Summary

Steps to reproduce

  1. Run this method with a datetime object that is timezone aware.
  2. For example, pass a datetime object that has been parsed from this: 2021-07-28T15:08:23+00:00
  3. The response given the input above will be 2021-07-28T15:08:23+00:00+00:00 instead of 2021-07-28T15:08:23+00:00

This particular bug is triggered when a tap is run, and if the records in the tap are already timezone aware. This could happen if before being sent over to sync, the post_process method of the sdk is called by a stream, and that post_process method does some datetime manipulation where it datetime object is already prebuilt.

For example, if there is a post_process method like below in a tap built with the sdk:

def post_process(self, row: dict, context: Optional[dict] = None) -> dict:
  date_value = row.get("updated_at", None)
  if date_value:
     parsed_date = pendulum.parse(date_value, strict=False)
     row["updated_at"] = parsed_date
   return row

What is the current bug behavior?

The current bug behavior is that +00:00 is appended to every datetime object when converted to string, regardless of if it is timezone aware or not

What is the expected correct behavior?

The expected correct behavior is that timezone aware datetime objects do not have the timezone manually appended

Relevant logs and/or screenshots

See this slack thread for more details

Possible fixes

https://gitlab.com/meltano/sdk/-/blob/main/singer_sdk/helpers/_typing.py#L25

A solution discussed in the above linked slack thread was to enforce timezone awareness in the function.