Fast Eye-Tracked 3D Gaussian Rendering for Virtual Reality

Proc. Vision, Modeling and Visualization (VMV), 2026

Abstract

Recent advances in 3D Gaussian Splatting (3DGS) promise interactive exploration of photorealistic reconstructions in virtual reality (VR) applications. Unfortunately, current approaches for 3DGS are unable to meet the high demands of VR rendering, suffering from immersion-breaking artifacts, requiring aggressive pruning of primitives, or failing to reach sufficiently high frame rates for complex scenes. We introduce VR-HTGS, the first fully integrated eye-tracked foveated solution for 3DGS rendering that, in contrast to previous work, dynamically adapts the tile-based 3DGS rasterizer to a heterogeneous tile layout following the gaze position. In comparison to homogeneous layouts, this allows tiles to be rendered at varying resolutions, retaining real-time performance without sacrificing the primitive count. To minimize immersion-breaking inconsistencies during rendering, VR-HTGS builds on top of an efficient view-consistent Gaussian splat renderer to enable dynamic foveation along with further modifications to reduce aliasing, popping, and sky floaters. In combination, these improvements enable fast, high-quality rendering of 3D Gaussians at 120+ Hz on commodity hardware for scenes consisting of millions of Gaussians.

Citation

@article{scholz2026vrhtgs,
    title = {Fast Eye-Tracked 3D Gaussian Rendering for Virtual Reality},
    author = {Scholz, Timon and Hahlbohm, Florian and Eisemann, Martin and Castillo, Susana and Magnor, Marcus},
    booktitle = {Vision, Modeling, and Visualization},
    publisher = {The Eurographics Association},
    year = {2026}
}

Acknowledgements

The authors gratefully acknowledge funding by the DFG under Germany's Excellence Strategy within the Cluster of Excellence PhoenixD (EXC 2122, Project ID 390833453), as well as from the DFG Projects "Real-Action VR" (ID 523421583) and "Increasing Realism of Omnidirectional Videos in Virtual Reality" (ID 491805996). Open Access funding enabled and organized by Projekt DEAL.