Skip to content

Spatial upscaling using AMD FSR 1

Thorwin Vogt requested to merge thorwin.vogt/met.3d:clouds-fsr into master

Implemented features

This MR implements the following features in a single go, since they all benefit from each other and somewhat depend on each other.

The MR looks large when looking at the changed lines, though roughly 3800 of those are the AMD FSR code, which doesn't need to be reviewed.

Scene render-pipeline rework

The first one is a general rework of the scene render pipeline, which is now split up different steps.

The rendering pipeline of Met.3D was modified and split up into 5 parts to accommodate the upscaling.

  1. Rendering of the 3D scene into a separate framebuffer, however 2D ui elements such as labels and transfer functions are skipped.
  2. Possible upscaling of the result from step 1 using a compute shader that runs FSR.
  3. Rendering of the full resolution scene onto a quad with a separate frame buffer. Depth values are written to replicate the original scenes depth
  4. Rendering of 2D UI elements such as labels and transfer functions.
  5. Rendering the result from step 4 onto a quad spanning the whole viewport to display render result to the user.

FSR

Since direct volume rendering can be expensive, especially on high resolution screens, the real time performance while interacting with the scene may be too low to be interactable.

This Merge request tries to solve this performance bottleneck by introducing a spatial upscaling algorithm into Met.3D. The chosen algorithm is AMDs FSR 1, since it is open source and does not require temporal data to create the upscaled image.

UI elements are rendered separately, since they don't scale up well. Instead, they are rendered at full resolution in the final step. The upscaling settings can be set in the system settings under rendering for each scene.

As can be seen when using upscaling, the image quality drops somewhat and edges become particularly sharp. This happens, because FSR tries to keep edges when upscaling. This will result in poorer results in images not using anti-aliasing. When enabling the geometry anti-aliasing of Met.3D, the results can get better, but a post-processing anti-aliasing implementation, such as FXAA or TAA would be better. It is worth noting, that implementing TAA would also enable temporal upscaling in the future, but is more complicated to implement.

The drop in image quality for hard-edge-geometry is no big issue though, as the idea for this feature is to use it with DVR and only when interacting with the scene. When you want to create high quality images from the scene using the synchronization control, you can disable FSR temporarily.

Custom resolution rendering (#309 (closed))

In Addition to FSR upscaling, the new render pipeline allows us to render at custom render resolutions. A new setting was introduced to the scene render settings, to allow the user to set a custom render resolution if wanted. In step 3 and 4 of the new render pipeline, the render will use the custom render resolution if enabled.

When rendering at a custom render resolution, the viewport shows the render result so it fits inside the viewport. This might stretch or scale the image accordingly. However, when saving a screenshot, the renderer now doesn't take the viewports framebuffer result. Instead, the result from step 4 is taken, which contains the full resolution final scene image in a separate framebuffer.

Edited by Thorwin Vogt

Merge request reports