Invatare Automata @ UVT: NVIDIA DLSS 3

Deep learning super sampling (DLSS) is a family of real-time deep learning image enhancement and upscaling technologies developed by Nvidia that are exclusive to its RTX line of graphics cards, and available in a number of video games. The goal of these technologies is to allow the majority of the graphics pipeline to run at a lower resolution for increased performance, and then infer a higher resolution image from this that contains the same level of detail as if the image had been rendered at this higher resolution. This allows for higher graphical settings and/or frame rates for a given output resolution. AMD FSR and Intel XeSS are similar, but DLSS has always been the king of the upscalers, as it uses more sophisticated machine learning/AI techniques to piece together more accurate frames.
As of September 2022, the 1st and 2nd generation of DLSS is available on all RTX branded cards from Nvidia in supported titles, while the 3rd generation to generation RTX 4000 series graphics cards.
DLSS 3 augments DLSS 2.0 by making use of an optical-flow frame generation technique. The DLSS frame generation algorithm takes two rendered frames from the rendering pipeline, and generates a new frame that smoothly transitions between them. So for every frame rendered, one additional frame is generated.
DLSS 3.0 makes use of a new generation Optical Flow Accelerator (OFA) included in Ada Lovelace generation RTX GPUs. The new OFA is faster and more accurate than the OFA already available in previous Turing and Ampere RTX GPUs. This results in DLSS 3.0 being exclusive for the RTX 4000 Series.
DLSS also requires and applies its own anti-aliasing method. It operates on similar principles to TAA. Like TAA, it uses information from past frames to produce the current frame. Unlike TAA, DLSS does not sample every pixel in every frame. Instead, it samples different pixels in different frames and uses pixels sampled in past frames to fill in the unsampled pixels in the current frame. DLSS uses machine learning to combine samples in the current frame and past frames, and it can be thought of as an advanced and superior TAA implementation made possible by the available tensor cores.
Nvidia offers deep learning anti-aliasing (DLAA). DLAA provides the same AI-driven anti-aliasing DLSS uses, but without any upscaling or downscaling functionality.

DLSS 3: upscaling and generated frame quality

The author of this article tested DLSS 3, using the RTX 4090, in three games: Cyberpunk 2077, Microsoft Flight Simulator, and F1 22. As expected from the successor to DLSS 2.4, DLSS 3’s upscaling looks impressive: on its Quality and even Balanced modes, upscaled 4K looks just as good as native 3840x2160 across all three games. On its second-fastest Performance mode, there is a noticeable drop in water quality in Microsoft Flight Simulator, but in terms of sharpness it is seemingly very close to native 4k. There’s little to no fudging of fine details, no strange outlines around objects, and the AI-based anti-aliasing remains superior to just about every built-in AA option.

For the most part, generated frames look 99% like a traditionally rendered frame. Here’s a randomly chosen, fully AI-generated frame in F1 22 (upscaled to 4K), followed by a successive rendered frame: