Serialize all arguments for cleaner sidekiq consumption

Problem

Queueing jobs with sidekiq causes the passed arguments to be json-ified. Currently, we're working around that behavior in a variety of places & in different ways. It would be better to handle this holistically - serializing parameters in a consistent way.

Ideally, we should be able to pass ActiveRecord objects, ruby DateTime objects, Hashes, etc to delayed job workers & expect to receive comparable objects as actionable arguments, rather an a string version which needs to be otherwise manipulated.

Context

  • https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/26228/diffs#note_156854929
  • https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/26228/diffs#note_152464507

Proposal

Phase 1) Implementation & Utilization
  • Implement a serializer/parser for making arguments json-safe
  • Utilize serializer in ApplicationWorker - serializing & parsing all args for #perform
  • As all existing code should already use json-safe arguments, including a serializer should have no impact on the existing behavior
  • As this is a change that could touch basically everything, it should be behind a feature flag
Phase 2) Cleanup
  • code with wonky workarounds (like those linked in the "Context" section above) can be cleaned up as-needed
  • all new code can be written utilizing varied argument types - not just primitives

Risks & Alternatives

Making changes to ApplicationWorker is a pretty risky move, as pretty much everything is built on sidekiq. The above approach shouldn't have an impact on any existing workers. But it's still scary.

Alternative 1) Opt-in to the functionality

Workers could utilize the argument serializer on an as-needed basis. For example, if a job is expected to handle dates or active record model instances as arguments, the serialization & parsing could be introduced from the worker & the caller directly.

Alternative 2) Don't do it

Maybe it's not worth it?

Assignee Loading
Time tracking Loading