Skip to content

Mass reorganization of the codebase

Jonathon Anderson requested to merge blue42u/reorganization into develop

The organization for our code is in a relatively sorry state. It's about time for a spring cleaning to review where everything is located and put it in more appropriate locations. This MR handles the top-level part of that cleanup.

The goal of this MR is to make it clear what purpose every file has. For instance, src/lib/analysis/ is presumably some "library" for "analysis," but it's unclear what analysis (binary? performance?) and misleading (we don't expose libraries, we build CLI tools). A much better name is src/common/, it's "source" code which is "common" across the tools (src/hpcprof, src/hpcstruct, etc.). Another example is lib/, which colloquially is where source code goes as an alternative to src/. We have both in our repository which is confusing and ambiguous. Better names are src/hpctesttool/ and doc/dtd/.

This MR does the mass moving of directories to give all the files reasonable top-down structure, based on common standards and colloquialisms. For instance, src/ is source code, doc/ is documentation, ci/ is CI-related.

The src/ tree gets special attention here. The code for front-facing tools is in src/hpc<tool>/ and the minimal common code is in src/common/ or src/vendor/. Internal tools are subdirectories of the front-facing tools (e.g. src/hpcstruct/hpcproflm) and many legacy distinctions have been elided (e.g. src/lib/banal/*.h -> src/hpcstruct/*.h). It is now very clear what code is used where even without documentation.

New top-down organization 
hpctoolkit/
├── ci/             # CI-related auxilliary files
├── cmake/          # CMake-related auxilliary scripts
├── doc/            # Documentation
├── meson/          # Meson-related auxilliary files
├── src/            # Source code
│   ├── common/        # Common code shared across the tools
│   │   └── lean/            # Common code that can be safely shared with hpcrun
│   ├── hpcprof/       # Source code for hpcprof and hpcprof-mpi
│   ├── hpcproftt/     # Source code for hpcproftt, development tool
│   ├── hpcrun/        # Source code for hpcrun
│   │   └── fnbounds-server/ # Source code for hpcfnbounds, internal tool for hpcrun
│   ├── hpcstruct/     # Source code for hpcstruct
│   │   └── hpcproflm/       # Source code for hpcproflm, internal tool for hpcstruct
│   ├── hpctesttool/   # Source code for hpctesttool, testing tool
│   ├── hpctracedump/  # Source code for hpctracedump, development tool
│   └── vendor/        # Vendored source code (Valgrind headers)
├── subprojects/    # Meson distribution metadata for external dependencies
├── tests/          # Integration tests
├── LICENSE         # Main project license
├── meson.*         # Meson build scripts
├── README.*        # First-time user information
├── .ci.predeps/    # Scripts for building predeps containers
├── .devcontainer/  # Devcontainer configuration
├── .e4s/           # E4S-related metadata
├── .editorconfig   # Common editor configuration
├── .git*           # Git configuration
├── .gitlab/        # GitLab configuration
├── .gitlab-ci.yml  # GitLab CI configuration
└── .pre-commit-config.yaml  # Linter configuration (pre-commit)

To Demonstrate

Run tree.

Backward Compatibility

  • This MR does not alter the data formats.
  • This MR does not alter any CLIs.

Checklist

  • I have run pre-commit run -a on my local checkout and fixed any reported issues.
  • I have confirmed that existing tests and/or the tests added by this MR cover the changes introduced by this MR.
  • I have confirmed that any added tests will be automatically run in CI.
  • I have updated all relevant documentation:

Merge request reports