Revert e3749476

This caused buffers that should not actually be muxed to be muxed. An example of when this could happen would be:

  |  running time + duration  |
                             | sink_idmap + buf duration |
| sink_layer + buf duration |

Here, both either have their bounds inside running time, but the should clearly not be muxed.