Better GPU representation in Ryax

GPU representation in Ryax

This is necessary to improve UX for users:

As a user I can ask for a specific GPU brand or model, of an amount of RAM so my action work out of the box

But also to make Intelliscale recommendation less specific and thus, let the scheduler take the placement decisions.

GPU in the Infrastructure

Brand: Nvidia and AMD are not providing the API and the user code might be adapted to only one of them.
Model: Different models provides different advantages and inconvenient and users might want a specific one (only provide the recent ones that are available in the cloud providers)
Partitioning:
- partitions: with share of the GPU compute and memory for example
  - 3g.90G => 3/7 compute, 90G memory
  - 4g.90G or
  - 1g.10G with a checkbox for homogeneous

Brand filter models.

GPU resource request

Brand: Nvidia and AMD are not providing the API and the user code might be adapted to only one of them.
Model: Different models provides different advantages and inconvenient and users might want a specific one

And amount of resources:

Amount of GPU: amount of GPU per node
Amount of RAM: total amount of ram available on the GPU

Details

We choose not to get the information from the cloud provider API for now because of the complexity of it. The registration is still declarative.

The catalog of cards should bind, Brand, Model, Max number of partition. This can be taen from here: https://docs.nvidia.com/datacenter/tesla/mig-user-guide/latest/supported-gpus.html

Max number of partition is useful to check that the proposed partition is valid. Also, it might be necessary for Intelliscale to normalize the compute unit (3 means nothing but 3/7 can be normalized).

Questions

Should we propose an Other field for both brand and model to allow users to add custom GPU with custom max partition number, amount of RAM and compute ? This can be a good way to populate the catalog with an API.

How to select the right Compute

To be able to give compute recommendation we need to be able to compare the GPU between them. To do so, we need an ordering between cards and cards partitions. We can use a score for each GPU compute partition. For example a 4g.40Gb for a H100 that as a score of 100 means 4/7 partition of the card.

This seems complicated. Without this score, a simpler UX for the user would be to ask them to select a card among the ones available. This will be a hard constraint. Then, when the recommendation kick-in it provides a partition on this particular card.

Design

Simple choice: Model is mandatory

Because cards cannot be compared easily regarding the compute (see previous section). The resource requests MUST define a GPU model.

A user can provide constraints on the action spec with the GPU model:

spec:
  resources:
    gpu:
      model: A100
      memory_GB: 40G
      compute_percentage: 50

Then the recommendation of Intelliscale should provide the same kind of values: {"memory_GB": 35, "compute_percentage": 60} which will allow to select the right partition on the model.

Advanced: No requirement (requires a GPU compute score)

With a GPU compute score we can leave the brand and the model as constraints and make the recommendation algo only give compute score and memory amount and leave the choice of the model and the brand to the scheduler if there is no constraints.

TODO

The Runner already accepts the advanced request and reads gpu_model, gpu_memory_gb and gpu_compute_fraction off WorkflowActionResources (numbers 5-7) when they arrive; gpu_brand (8) comes with ryax-core!510 (merged). Studio and UI can land in any order; ryax-repository!286 (merged) must land with or after ryax-core!510 (merged), or the old Action Builder fails to build an action that uses the object form.

Merge requests

MR	What
ryax-core!475 (merged)	GPU catalog: brand, model, MIG profiles, `GET /gpu-models`
ryax-core!498 (merged)	Node pools by card, advanced GPU request, placement on memory/compute, first-fit / best-fit
ryax-intelliscale!33 (merged)	Recommends GB + share of a card instead of a profile name
ryax-engine!81 (merged)	Docs and chart values
ryax-core!510 (merged)	Action Builder reads the GPU request object and sends it in `ActionReady`; Runner places on the brand
ryax-repository!286 (merged)	Action spec accepts `gpu: {count, brand, model, memory, compute_fraction}`
ryax-studio!495 (merged)	Studio stores the request, sends it to the Runner, exposes it in the API

RYAX_SCHEDULER_GPU_FIT_POLICY (first_fit default, best_fit) chooses among the node pools that fit a request.

Follow-ups this turned up

Each is filed separately rather than growing this issue further; all three are linked as related.

#1457 — Decide whether an action can take more than one MIG slice. A MIG pool gives one slice per action whatever the count says. May well be correct — MIG instances do not talk to each other the way two cards on NVLink do — so the issue asks the question before proposing code.
#1458 (closed) — Key recommendations by hardware, not by site. A recommendation is only valid for the machine it was measured on, and site is the wrong key: two node pools in one site can hold different cards. Affects CPU as much as GPU. _hardware_profile_id is most of the key already and reaches IntelliScale unused.
#1459 — Schedule GPUs against node pool availability, not just static capacity. Placement matches declared capacity and cannot see what is already running, so a burst of GPU executions is all sent to the same pool. A GPU cannot be over-committed the way CPU can: the extra pods sit Pending.

Other things worth doing, too small to file unless they grow:

Carry the GPU request into the worker's own view if per-partition Kubernetes resources (nvidia.com/mig-1g.10gb) are ever wanted; today Ryax requests nvidia.com/gpu and relies on mig.strategy=single.
node_pools.gpu_free_total is a column with no field and no reader, and NodePool.gpu_num_total is never assigned. Either finish the idea behind them (see #1459) or drop them.

Edited Oct 02, 2026 by Michael Mercier