Skip to content

realm: Add indirection channel query

Cory Perry US requested to merge cperry/ind_ch into master

Add a query for channels on if they support using a particular memory as an indirection buffer. Additionally, add this information to the serialization paths for remote channels so remote nodes can make ask if a remote channel can support an indirection buffer. This will allow us to support gpu memories as indirection buffers for gpu channels, and allow gather/scatter graph generators to pick more appropriate ib memories, should they need it.

As part of modifying this, re-arrange the GPUIndirectChannel to properly calculate the bandwidth affinity cost that has the effect of deprioritizing the channel in certain cases. These cases have some data corruption issues that need to be resolved in a follow up change.

Merge request reports