Explore and document expected workflow for efficient bring-up of entire application

Summary

We should map a flowchart of how we expect components to come up, and the order of those. From there, we can ensure that the codified patterns within the Operator provide the most efficient route to this, in the most performant fashion.

The largest factors to consider are dependency ordering and stateful data.

Details

Spawned in discussion

@pursultani:

This MR introduces a few major changes but the most significant change is the transposition of the order of resource reconciliation (type x component vs. component x type).

@WarheadsSE:

Generally, I agree with @pursultani in order of type x component, rather than component x type. This results in the requirements always being in place for pods as they come up.

That being said, if we always patch & deploy most Deployment / StatefulSet as paused, then it is more viable for us to actually do component x type, and then effectively un-pause in the same order as we would for an upgrade. There would simply never been any post-migrations.

I think this is best solved by mapping a flowchart of how we expect components to come up, and the order of those. Functionally, they will both achieve the final result. We want to create the optimized choice.

Deferring this discussion to future exploration, documentation, and improvement.

Acceptance Critera

  • diagram present added to the documentation, under development
  • operator reconciliation confirmed to result in the desired behavior