circuit_diagram_matplotlib draws gates on top of each other when ref_op is the same

Minimal reproducible example:

qubit = "q0"

sched = Schedule("Test schedule")
sched.add(Reset(qubit))

first_gate = sched.add(X(qubit))
sched.add(X(qubit), ref_op=first_gate, rel_time=1e-6)
sched.add(X(qubit), ref_op=first_gate, rel_time=2e-6)

circuit_diagram_matplotlib(sched)

This produces:

Note that the last two X gates are down on top of each other. It seems to me that even if we don't want information about the timing in the circuit_diagram, it matters in the order in which we draw of the gates. Now the gate is drawn right after whatever is set as the ref_op, resulting in two gates being drawn at the same location in case the ref_op is the same.