Draft: Kokkos RDC Build
I vibe-coded a PETSc build system to enable Kokkos RDC build with CUDA. The strategy is to build three libraries:
libpetsc.socontaining host and device symbols for regular and RDC-disabled CUDA files (in fact, I mandated--with-single-library=0, so this library is split)libpetsckokkos.acontaining host and device symbols for RDC-enabled Kokkos files (*.kokkos.cxx) along with Kokkos static archiveslibpetsckokkosdlink.acontaining CUDA device-link object for Kokkos files
libpetsckokkos.a and libpetsckokkosdlink.a need to be linked exactly once per program, because they contain CUDA registration/device images. Namely, these static archives are not reusable objects. This complicates build process a bit, as all the intermediate shared libraries should not link these objects while any test executables should. If they are linked multiple times per program, you hit CUDA initialization errors.
Device linking should also be performed exactly once per program. So. if a downstream application (like MOOSE) has additional RDC-enabled CUDA codes that need device linking, libpetsckokkosdlink.a should not be linked, and the downstream application should perform its own device linking with just libpetsckokkos.a and link its own device link object with its executable.
I'm not sure if this design makes sense to PETSc. Reviews would be appreciated. This change has been tested with libMesh and MOOSE and has been confirmed to be working. Related changes downstream: