Follow-up from "Add endpoint for serialized DAG pipeline"
The following discussion from !31324 (merged) should be addressed:
-
@ayufan started a discussion: (+4 comments) How is the
scheduling_type_stagehandled?
@fabiopitino
No. I mean "present" as JSON attribute being exposed. We are exposing needs (as array) if the scheduling type is dag. If the scheduling type is stage we don't expose needs at all, meaning the JSON won't have any needs attribute.
needs not exposed --> stage scheduled job
needs: [] --> DAG scheduled with empty needs. Runs immediately
needs: [something] --> DAG scheduled with needs.
@ayufan
I don't really like that absence of something indicates stage given that we moved from that design. I can live with that for now, but I would like us to be more explicit :)
We for exactly the same reason introduced scheduling_type as for exactly the same reason we had an absence of something to indicate, which created a problem handling one of the cases.