Skip to content

legion_prof_rs: Support for merging attached profiles into a single view

Elliott Slaughter requested to merge legion-prof-rs-merge-data into master

Not quite ready yet because I need to merge https://github.com/StanfordLegion/prof-viewer/pull/35. But once that's done and the commit ID is updated, this brings the Legion Prof UI up to date with latest changes like whole-word search and adds support for merging profiles in --attach.

Merging profiles supports two main use cases:

  1. Very large profiles may not fit in the memory of any one machine. Now you can slice those profiles into pieces, --serve them on different nodes, and --attach them all, stitching them back into a single logical profile. It's not perfect, because you're still viewing what are essentially separate profiles stitched together (e.g., if you make 2 slices, you'll get 2 overall utilization summaries that each cover half the machine). Still, it's pretty good for the complexity cost (minimal, especially compared with a "true" distributed Legion Prof implementation), and if you're in a bind it'll get the job done.
  2. When running Legion as a per-rank runtime within MPI, Legion produces N distinct profiles. Then you can use this mode to stitch them back together. Seshu needs this for running Legion as a intra-node runtime on top of MPI jobs with cuNumeric.

One open question is that the command-line syntax should be. Right now, I added --attach-merge, which is an additional flag you must pass to merge attached profiles (i.e., you always call --attach --attach-merge with both flags, not just the one). I don't particularly like the name, and I'm not sure the behavior is intuitive either.

Edited by Elliott Slaughter

Merge request reports