VarSplat: Uncertainty-aware 3D Gaussian Splatting for Robust RGB-D SLAM

Department of Computer Science, George Mason University

CVPR 2026

Visualization (ScanNet scene0169_00)

VarSplat learns per-splat appearance variance online and renders differentiable per-pixel uncertainty map alongside input frame via single-pass rasterization, while simultaneously optimizing the map and inserting new Gaussians.

Given RGB-D inputs, each 3D Gaussian jointly learns position, orientation, scale, color, opacity, and appearance variance. During mapping, variance is optimized jointly with other Gaussian parameters.

Abstract

Simultaneous Localization and Mapping (SLAM) with 3D Gaussian Splatting (3DGS) enables fast, differentiable rendering and high-fidelity reconstruction across diverse real-world scenes. However, existing 3DGS-SLAM approaches handle measurement reliability implicitly, making pose estimation and global alignment susceptible to drift in low-texture regions, transparent surfaces, or areas with complex reflectance properties.

To this end, we introduce VarSplat, an uncertainty-aware 3DGS-SLAM system that explicitly learns per-splat appearance variance. By using the law of total variance with alpha compositing, we compute corresponding differentiable per-pixel uncertainty map via efficient, single-pass rasterization. This variance map guides tracking, submap registration, and loop detection toward focusing on reliable regions and contributes to more stable optimization.

Experimental results on Replica (synthetic) and TUM-RGBD, ScanNet, and ScanNet++ (real-world) show that VarSplat improves robustness and achieves competitive or superior tracking, mapping, and novel view synthesis rendering compared to existing studies for dense RGB-D SLAM.

Method

VarSplat method

VarSplat builds upon 3D Gaussian Splatting SLAM systems and introduces three main contributions:

Per-splat Appearance Variance: Each Gaussian explicitly models an appearance variance parameter in addition to color and opacity, capturing regions of photometric uncertainty.
Differentiable Uncertainty Maps: Using the law of total variance integrated with alpha compositing, VarSplat renders differentiable per-pixel uncertainty maps alongside RGB images — no separate network required and can be operated in single-pass rasterization under online setting.
Uncertainty-guided Optimization: The pixel-level uncertainty map is used to weight during camera tracking, submap registration, and loop closure detection, leading to more robust and consistent global alignment.

BibTeX

@inproceedings{tran2026varsplat,
  title   = {VarSplat: Uncertainty-aware 3D Gaussian Splatting for Robust RGB-D SLAM},
  author  = {Tran, Anh Thuan and Kosecka, Jana},
  booktitle = {CVPR},
  year    = {2026}
}