SG-Splatting: Accelerating 3D Gaussian Splatting with Spherical Gaussians Figure 1
ICASSP 20262026-04-21

SG-Splatting: Accelerating 3D Gaussian Splatting with Spherical Gaussians

Yiwen Wang, Siyuan Chen, Ran Yi

Shanghai Jiao Tong University

加速训练

该文针对3D Gaussian Splatting中三阶球谐颜色表示占用大量参数、拖慢实时渲染的问题,提出用球面高斯表示视角相关颜色,并通过正交多SG组织与低阶球谐混合表示兼顾高低频外观。实验表明该方法可降低颜色参数和计算开销、加速渲染并保持或提升视觉质量,且可作为插件接入其他加速框架;具体量化增益在给定文本中未充分说明。

A Survey on 3D Gaussian Splatting Figure 1
ACM Computing Surveys 20262026-04-09

A Survey on 3D Gaussian Splatting

Guikun Chen, Wenguan Wang

The State Key Lab of Brain-Machine Intelligence, Zhejiang University, The State Key Lab of Brain-Machine Intelligence, Zhejiang University, Zhejiang Lab

综述

针对 NeRF 等隐式辐射场训练/渲染开销大、场景编辑不直观的问题,本文系统梳理 3D Gaussian Splatting:以大量可学习三维高斯作为显式表示,并结合可微 splatting 渲染。论文建立分类框架,解析原理、优化与扩展方向,比较主流模型在多类基准任务上的表现,并总结其在实时渲染、VR/AR、重建等应用中的速度与可编辑性优势及开放挑战。

SparseSplat: Towards Applicable Feed-Forward 3D Gaussian Splatting with Pixel-Unaligned Prediction Figure 1
arXiv preprint2026-04-03

SparseSplat: Towards Applicable Feed-Forward 3D Gaussian Splatting with Pixel-Unaligned Prediction

Zicheng Zhang, Xiangting Meng, Ke Wu, Wenchao Ding

Fudan University, ShanghaiTech University

前馈重建稀疏表示

SparseSplat针对前馈3DGS常见的像素/体素对齐导致高斯冗余、难以用于SLAM和机器人等资源受限场景的问题,指出其根源在于分布与感受野不匹配。方法用基于局部熵的概率采样按信息量自适应分配高斯,并以3D KNN局部属性预测替代单像素回归。实验显示其仅用约22%高斯即可达到SOTA渲染质量,1.5%高斯下仍保持可用效果。

GLINT: Modeling Scene-Scale Transparency via Gaussian Radiance Transport Figure 1
arXiv preprint2026-03-27

GLINT: Modeling Scene-Scale Transparency via Gaussian Radiance Transport

Youngju Na, Jaeseong Yun, Soohyun Ryu, Hyunsu Kim, Sung-Eui Yoon, Suyong Yeon

三维高斯泼溅

GLINT针对3D高斯泼溅在玻璃等透明场景中把界面、反射与透射辐射混在一起、导致几何缺失或鬼影的问题,显式将高斯分解为界面、透射和反射成分,并结合光栅化/光追的透明感知辐射传输及视频重光照先验来稳定定位透明区域。实验在新合成基准3D-FRONT-T和真实DL3DV-10K上显示,其外观重建和几何评估均优于已有方法。

Periodic Vibration Gaussian: Dynamic Urban Scene Reconstruction and Real-time Rendering Figure 1
International Journal of Computer Vision2026-01-23

Periodic Vibration Gaussian: Dynamic Urban Scene Reconstruction and Real-time Rendering

Yurui Chen, Chun Gu, Junzhe Jiang, Xiatian Zhu, Li Zhang

School of Data Science, Fudan University, Shanghai, China, School of Data Science, Fudan University, University of Surrey, Guildford, United Kingdom, University of Surrey

自动驾驶其他

面向自动驾驶中的大规模动态城市场景重建,论文指出将静态/动态对象分开建模会依赖标注或光流且效率低。PVG在3D Gaussian Splatting中引入周期振动式时间动态,并配合时间平滑、位置感知自适应控制,以统一表示静态与运动元素。在Waymo和KITTI上,其重建与新视角合成优于既有方法,且无需3D框或昂贵光流,渲染相对最佳替代方法最高加速约900倍。

RaDe-GS: Rasterizing Depth in Gaussian Splatting Figure 1
ACM TOG 20262026-01-16

RaDe-GS: Rasterizing Depth in Gaussian Splatting

Baowen Zhang, Chuan Fang, Rakesh Shrestha, Yixun Liang, Xiao-Xiao Long, Ping Tan

Department of Electrical and Electronics Engineering, The Hong Kong University of Science and Technology, Department of Electrical and Electronics Engineering, The Hong Kong University of Science and Technology, Hong Kong University of Science and Technology, University of Hong Kong, Simon Fraser University, School of Intelligence Science and Technology, Nanjing University, School of Intelligence Science and Technology, Nanjing University, Nanjing University of Science and Technology

网格重建

判断受限于 PDF 文本抽取质量。RaDe-GS针对传统3D Gaussian Splatting虽能实时高质量渲染、但离散无结构高斯难以准确提取几何的问题,提出直接对通用3D高斯栅格化深度图与法线图的做法,在尽量不改写高斯表示的情况下强化表面约束。文中报告其在DTU上Chamfer误差接近NeuraLangelo,在Tanks & Temples上训练与渲染时间接近原始GS。

Semantic Gaussians: Open-Vocabulary Scene Understanding with 3D Gaussian Splatting Figure 1
IEEE Transactions on Circuits and Systems for Video Technology2026-01-01

Semantic Gaussians: Open-Vocabulary Scene Understanding with 3D Gaussian Splatting

Jun Guo, Xiaojian Ma, Yue Fan, Huaping Liu, Qing Li

语言嵌入分割

面向机器人与 AR 中按自然语言理解三维场景的需求,本文将开放词汇语义注入 3D Gaussian Splatting:通过基于空间对应的投影,把 CLIP/OpenSeg 等二维预训练特征映射为每个 Gaussian 的语义分量,无需为新场景额外联合优化,并用 3D 稀疏卷积网络加速预测。实验在 ScanNet 分割和 LERF 目标定位上优于二维/三维基线,还展示了部件、实例、编辑与时空分割应用。

Fast Dynamic 3D Object Generation from a Single-view Video Figure 1
International Journal of Computer Vision2025-12-24

Fast Dynamic 3D Object Generation from a Single-view Video

Zijie Pan, Zeyu Yang, Xiatian Zhu, Li Zhang

School of Data Science, Fudan University, Shanghai, China, School of Data Science, Fudan University, University of Surrey, Surrey, UK, University of Surrey

扩散生成

单视角视频生成动态3D物体受限于4D标注稀缺,直接用SDS监督又慢且难扩展。Efficient4D将问题拆成先生成跨视角、时空一致的合成图像,再用其监督4D Gaussian Splatting重建,并加入不一致感知置信加权损失和轻量SDS以缓解稀疏视角断裂。实验显示其在合成与真实视频上保持新视角质量,同时建模时间约10分钟,相比Consistent4D的120分钟约快10倍,并支持实时渲染。

LEGO-SLAM: Language-Embedded Gaussian Optimization SLAM Figure 1
arXiv preprint2025-11-20

LEGO-SLAM: Language-Embedded Gaussian Optimization SLAM

Sibaek Lee, Seongbo Ha, Kyeongsu Kang, Joonyeol Choi, Seungjun Tak, Hyeonwoo Yu

Sungkyunkwan University, South Korea, Sungkyunkwan University

同步定位与建图语言嵌入机器人分割

LEGO-SLAM针对3DGS SLAM虽能实时重建高保真地图、却缺少开放词汇语义且高维语言特征开销过大的问题,提出在线自适应编码器将语言嵌入压缩到16维,并直接在紧凑空间做查询、语义剪枝和回环检测。实验显示其在保持建图质量与跟踪精度竞争力的同时,以15 FPS运行,且可减少超过60%的高斯数量。

FastGS: Training 3D Gaussian Splatting in 100 Seconds Figure 1
arXiv preprint2025-11-06

FastGS: Training 3D Gaussian Splatting in 100 Seconds

Shiwei Ren, Tianci Wen, Yongchun Fang, Biao Lu

加速训练密度控制动态场景稀疏表示

FastGS针对3DGS训练中自适应密度控制产生大量冗余高斯、拖慢优化的问题,提出以多视角重建质量而非单个高斯属性来判断重要性的增密VCD与剪枝VCP,并结合更紧凑的光栅化区域,无需预算机制即可控制表示规模。实验显示其在静态场景约100秒完成训练,在Mip-NeRF 360上较DashGaussian加速3.29倍且质量接近,在Deep Blending上较原版3DGS加速15.45倍,并能迁移到动态重建、稀疏视角、大规模场景和SLAM等任务。

Diff4Splat: Controllable 4D Scene Generation with Latent Dynamic Reconstruction Models Figure 1
arXiv preprint2025-11-01

Diff4Splat: Controllable 4D Scene Generation with Latent Dynamic Reconstruction Models

Panwang Pan, Chenguo Lin, Jingjing Zhao, Chenxin Li, Yuchen Lin, Haopeng Li, Honglei Yan, Kairun Wen, Yunlong Lin, Yixuan Yuan, Yadong Mu

Peking University, Xiamen University, CUHK, Carnegie Mellon University, Peking University, Xiamen University, Carnegie Mellon University

扩散生成动态场景前馈重建高斯视频虚拟现实

针对单图生成动态3D场景常依赖“先视频、再逐场景重建”而耗时且难控制的问题,Diff4Splat把视频扩散先验与可变形3D Gaussian表示合到前馈模型中,用Video Latent Transformer从受相机轨迹和可选文本约束的潜变量直接回归时变高斯场,并借助大规模4D标注数据监督外观、几何和运动。实验显示其可在约30秒生成可渲染4D场景,在视频生成、新视角合成和几何提取上达到或超过优化式方法,效率显著更高。

SemGauss-SLAM: Dense Semantic Gaussian Splatting SLAM Figure 1
IROS 20252025-10-19

SemGauss-SLAM: Dense Semantic Gaussian Splatting SLAM

Siting Zhu, Renjie Qin, Guangming Wang, Jiuming Liu, Hesheng Wang

Shanghai Jiao Tong University, School of Automation and Intelligent Sensing, Key Laboratory of System Control and Information Processing, Ministry of Education of China, Shanghai Jiao Tong University, School of Automation and Intelligent Sensing, Key Laboratory of System Control and Information Processing, University of Cambridge, United Kingdom, University of Cambridge

同步定位与建图

针对现有 3DGS SLAM 多停留在 RGB 建图、语义 NeRF 跟踪易累积漂移且新视角语义不准的问题,SemGauss-SLAM 将语义特征嵌入每个 3D Gaussian,并用特征级损失优化;同时引入语义感知 BA,利用多帧语义一致性联合优化位姿与高斯地图。在 Replica 和 ScanNet 上,其建图、跟踪、语义分割与新视角合成均优于现有辐射场 SLAM 方法。

Compact 3D Gaussian Splatting For Dense Visual SLAM Figure 1
IROS 20252025-10-19

Compact 3D Gaussian Splatting For Dense Visual SLAM

Tianchen Deng, Yaohui Chen, Jianfei Yang, Shenghai Yuan, Jiuming Liu, Danwei Wang, Weidong Chen

Shanghai Jiao Tong University, Institute of Medical Robotics and School of Automation and Intelligent Sensing, Shanghai, 200240, Shanghai Jiao Tong University, Institute of Medical Robotics and School of Automation and Intelligent Sensing, Nanyang Technological University, School of Electrical and Electronic Engineering, Singapore, Nanyang Technological University, School of Electrical and Electronic Engineering

同步定位与建图

该工作针对3D Gaussian SLAM在小场景也常需数百MB、难部署到嵌入式平台的问题,指出SLAM生成的高斯椭球存在显著几何相似性与冗余。方法以体素锚定表示组织高斯,结合滑动窗口在线掩码删除冗余椭球,并用残差/码本量化压缩属性,同时通过局部到全局BA与ICP损失提升位姿估计。实验显示在保持重建质量接近SOTA的同时,渲染速度提升约226%,内存压缩约2.21倍,并在Jetson平台验证可行性。

Learning Unified Representation of 3D Gaussian Splatting Figure 1
arXiv preprint2025-09-26

Learning Unified Representation of 3D Gaussian Splatting

Yuelin Xin, Yuheng Liu, Xiaohui Xie, Xinke Li

City University of Hong Kong

压缩前馈重建点云分割

本文针对直接把3DGS的位姿、尺度、颜色等原始参数送入网络时存在的非唯一性、数值异质和流形不匹配问题,提出在高斯等概率表面上定义连续子流形场,并离散为彩色点云来学习统一嵌入;结合VAE和基于最优传输的Manifold Distance后,实验显示其在重建质量、跨域泛化、噪声鲁棒性及无监督分割/神经场解码等下游任务上优于参数空间表示。

StylizedGS: Controllable Stylization for 3D Gaussian Splatting Figure 1
IEEE Transactions on Pattern Analysis and Machine Intelligence2025-08-28

StylizedGS: Controllable Stylization for 3D Gaussian Splatting

Dingxi Zhang, Yu-Jie Yuan, Zhuoxun Chen, Fang-Lue Zhang, Zhenliang He, Shiguang Shan, Lin Gao

Beijing Key Laboratory of Mobile Computing and Pervasive Device, Institute of Computing Technology, Chinese Academy of Sciences, Beijing, China, Beijing Key Laboratory of Mobile Computing and Pervasive Device, Institute of Computing Technology, Chinese Academy of Sciences, University of Chinese Academy of Sciences, Beijing, China, University of Chinese Academy of Sciences, Victoria University of Wellington, Wellington, New Zealand, Victoria University of Wellington, Victoria University of Wellington, New Zealand, Key Laboratory of Intelligent Information Processing, Institute of Computing Technology, Chinese Academy of Sciences, Beijing, China, Key Laboratory of Intelligent Information Processing

渲染风格迁移

针对 NeRF 风格化优化慢、几何纹理迁移不准且缺少可控性的痛点,StylizedGS 将风格迁移建立在显式 3D Gaussian Splatting 上,通过滤波去除 floaters、两阶段颜色与 NN 特征匹配联合优化颜色/几何,并用深度保持约束防止内容变形;还支持颜色、尺度和区域控制。实验显示其在多场景多风格下能生成视角一致、笔触细节较好的结果,并显著提升训练与渲染效率。

MeshSplat: Generalizable Sparse-View Surface Reconstruction via Gaussian Splatting Figure 1
arXiv preprint2025-08-25

MeshSplat: Generalizable Sparse-View Surface Reconstruction via Gaussian Splatting

Association for Artificial Intelligence

University Of Science And Technology Of China, Deep Space Exploration Laboratory/School of Information Science and Technology, University of Science and Technology of China, Deep Space Exploration Laboratory/School of Information Science and Technology, University of Science and Technology of China, Space Engineering University

二维高斯前馈重建网格重建

MeshSplat瞄准极稀疏视角下传统逐场景优化难以获得可靠几何、且3D标注昂贵的问题,核心洞察是用更贴合薄表面的2D Gaussian Splatting连接新视角合成监督与网格重建。方法以前馈MVS网络预测像素对齐2DGS,并用加权Chamfer约束跨视角深度位置、用单目法线先验校正高斯朝向。实验显示其在可泛化稀疏视角网格重建和跨数据集测试中优于现有方法。

MBA-SLAM: Motion Blur Aware Dense Visual SLAM with Radiance Fields Representation Figure 1
IEEE Transactions on Pattern Analysis and Machine Intelligence2025-08-08

MBA-SLAM: Motion Blur Aware Dense Visual SLAM with Radiance Fields Representation

Peng Wang, Lingzhe Zhao, Yin Zhang, Shiyu Zhao, Peidong Liu

College of Computer Science and Technology, Zhejiang University, Hangzhou, China, College of Computer Science and Technology, Zhejiang University, College of Computer Science and Technology, Zhejiang University, China, Zhejiang University of Science and Technology, School of Engineering, Westlake University, Hangzhou, China, School of Engineering, Westlake University, School of Engineering, Westlake University, Hangzhou, Zhejiang, China

同步定位与建图

针对NeRF/3DGS稠密SLAM在低光、长曝光等运动模糊输入下位姿跟踪和建图显著退化的问题,MBA-SLAM将物理模糊成像过程嵌入跟踪与建图,在SE(3)中估计曝光期间局部轨迹,并与NeRF或3D Gaussian地图联合优化。实验覆盖合成、真实模糊及常规清晰RGB-D数据,显示其在定位精度和重建质量上优于既有稠密视觉SLAM方法。

ObjectGS: Object-aware Scene Reconstruction and Scene Understanding via Gaussian Splatting Figure 1
ICCV 20252025-07-21

ObjectGS: Object-aware Scene Reconstruction and Scene Understanding via Gaussian Splatting

Ruijie Zhu, Mulin Yu, Linning Xu, Lihan Jiang, Yixuan Li, Tianzhu Zhang, Jiangmiao Pang, Bo Dai

University of Science and Technology of China, Shanghai Artificial Intelligence Laboratory, Beijing Academy of Artificial Intelligence, The Chinese University of Hong Kong, Chinese University of Hong Kong, The University of Hong Kong, University of Hong Kong

分割语言嵌入

ObjectGS针对3DGS重建质量高但缺少物体级语义、现有方法又将重建与分割割裂且连续语义场易混淆的问题,提出以物体ID为锚的高斯生成框架:先用SAM/DEVA获得跨视角一致ID并投票初始化,再动态增删物体感知anchors,并用固定one-hot离散语义和分类损失约束渲染。实验显示其在开放词汇与全景分割上优于已有方法,并可支持物体网格提取和场景编辑。

EasySplat: View-Adaptive Learning makes 3D Gaussian Splatting Easy Figure 1
ICME 20252025-06-30

EasySplat: View-Adaptive Learning makes 3D Gaussian Splatting Easy

Ao Gao, Luosong Guo, Tao Chen, Zhao Wang, Ying Tai, Jian Yang, Zhenyu Zhang

Nanjing University, Nanjing University of Aeronautics and Astronautics, Nankai University

三维重建加速训练密度控制渲染

EasySplat针对3DGS依赖SfM初始化易受纹理缺失和特征误差影响、传统密度控制在稀疏区域增点低效的问题,改用基于视角相似性的自适应分组来利用DUSt3R类pointmap先验估计相机与点云,并用KNN邻域椭球平均形状差异触发高斯拆分。实验显示其在新视角合成质量和训练速度上超过现有方法。

WarpRF: Multi-View Consistency for Training-Free Uncertainty Quantification and Applications in Radiance Fields Figure 1
WACV 20262025-06-27

WarpRF: Multi-View Consistency for Training-Free Uncertainty Quantification and Applications in Radiance Fields

Sadra Safadoust, Fabio Tosi, Fatma Güney, Matteo Poggi

Koç University, Department of Computer Engineering and KUIS AI Center, Istanbul, Turkey, Koç University, Department of Computer Engineering and KUIS AI Center, University of Bologna, Department of Computer Science and Engineering (DISI), Italy, University of Bologna, Department of Computer Science and Engineering (DISI)

三维高斯泼溅

针对 NeRF/3DGS 等辐射场在少视角训练或主动建图中缺少通用不确定性估计的问题,WarpRF 将渲染深度视为几何代理,通过跨视角反向 warp 比较颜色/深度多视图一致性来估计新视角不确定性,无需改模型或再训练。实验显示其在不确定性评估、主动视角选择和主动建图中优于 FisherRF 等特定框架方法,并可迁移到 NeRF、3DGS、SVRaster。

OccGaussian: 3D Gaussian Splatting for Occluded Human Rendering Figure 1
arXiv preprint2025-06-25

OccGaussian: 3D Gaussian Splatting for Occluded Human Rendering

Jingrui Ye, Zhongkai Zhang, Qingmin Liao

Tsinghua Shenzhen International Graduate School, Tsinghua University, Shenzhen, Guangdong, China, Tsinghua Shenzhen International Graduate School, Tsinghua University, Tsinghua–Berkeley Shenzhen Institute

数字人

该文针对单目动态数字人在真实场景中常被物体遮挡、NeRF 类补全方法训练和渲染过慢的问题,提出基于 3D Gaussian Splatting 的 OccGaussian:在规范空间初始化高斯,并对遮挡区域查询、聚合像素对齐特征,经 Gaussian Feature MLP 与遮挡感知损失补偿缺失外观。实验覆盖模拟与真实遮挡,效果与 SOTA 相当或更优,训练约 6 分钟、渲染最高 160 FPS,速度分别提升约 250×/800×。

SplatAD: Real-Time Lidar and Camera Rendering with 3D Gaussian Splatting for Autonomous Driving Figure 1
CVPR 20252025-06-10

SplatAD: Real-Time Lidar and Camera Rendering with 3D Gaussian Splatting for Autonomous Driving

Georg Hess, Carl Lindström, Maryam Fatemi, Christoffer Petersson, Lennart Svensson

Chalmers University of Technology

自动驾驶

SplatAD面向自动驾驶仿真中相机与激光雷达联合渲染速度不足的问题,将3D Gaussian Splatting扩展到动态交通场景的多传感器表示,设计球坐标下的CUDA稀疏点云光栅化,并显式建模滚动快门、雷达强度和ray dropout。三套自动驾驶数据集上,其新视角/重建PSNR较既有方法最高提升约2/3 dB,渲染速度相比NeRF系方法提升一个数量级。

PERSE: Personalized 3D Generative Avatars from A Single Portrait Figure 1
CVPR 20252025-06-10

PERSE: Personalized 3D Generative Avatars from A Single Portrait

Hyunsoo Cha, Inhee Lee, Hanbyul Joo

Seoul National University

数字人

面向VR/AR中数字人不仅要复刻身份、还需可改变发型胡须等外观属性的需求,PERSE从单张人像出发,先自动合成同一身份、多属性且运动一致的2D视频,再用3D Gaussian Splatting学习解耦连续属性潜空间,并用人脸形变插值监督约束平滑性。实验显示其能生成可驱动、身份保持较好的个性化3D头像,并支持未见属性插值;效果提升可能部分来自更大规模合成数据。

GASP: Gaussian Avatars with Synthetic Priors Figure 1
CVPR 20252025-06-10

GASP: Gaussian Avatars with Synthetic Priors

Jack Saunders, Charlie Hewitt, Yanan Jian, Marek Kowalski, Yiye Chen, Darren Cosker, Virginia Estellers, Nicholas Gydé, Vinay P. Namboodiri, Benjamin E. Lundell

University of Bath, Microsoft, Microsoft Research (United Kingdom), Georgia Institute of Technology, Microsoft and University of Bath, Microsoft (United States)

数字人

GASP面向单目照片或短视频难以生成可自由视角渲染数字人的问题,利用带精确标注的合成数据训练高斯头像先验,并通过每高斯语义相关特征和三阶段拟合缓解合成到真实域差,使先验只参与拟合、不进入推理。结果显示其可从有限前视输入生成可动画化、支持360°渲染的头像,并在商用硬件上约70fps实时运行。

GAF: Gaussian Avatar Reconstruction from Monocular Videos via Multi-view Diffusion Figure 1
CVPR 20252025-06-10

GAF: Gaussian Avatar Reconstruction from Monocular Videos via Multi-view Diffusion

Jiapeng Tang, Davide Davoli, Tobias Kirschstein, Liam Schoneveld, Matthias Nießner

Technical University of Munich, Toyota Motor Corporation (United States)

数字人

GAF面向手机等单目视频重建可动画3D高斯数字人,解决视角覆盖不足导致侧脸等未观测区域伪影的问题。其关键是用面向头部的多视角扩散先验补全缺失视图,并以FLAME法线图精确控视角、VAE特征保身份,再用迭代去噪伪真值和潜空间上采样监督高斯优化。实验在NeRSemble和普通设备视频上显示,其新视角与表情合成优于既有方法。

CAP4D: Creating Animatable 4D Portrait Avatars with Morphable Multi-View Diffusion Models Figure 1
CVPR 20252025-06-10

CAP4D: Creating Animatable 4D Portrait Avatars with Morphable Multi-View Diffusion Models

Felix Taubner, Ruihang Zhang, Mathieu Tuli, David B. Lindell

University of Toronto

数字人

CAP4D面向数字人重建中输入条件差异很大的痛点:单图方法保真不足,多视角方法又依赖大量采集。其核心是用3DMM条件化的可变形多视角扩散模型生成跨视角、跨表情的一致人像,再蒸馏到带表情外观建模的3D Gaussian Splatting 4D头像,实现实时动画渲染。实验显示其在单图、少图和多图头像视图合成与重演任务上优于已有方法。

TOGS: Gaussian Splatting with Temporal Opacity Offset for Real-Time 4D DSA Rendering Figure 1
IEEE Journal of Biomedical and Health Informatics2025-06-02

TOGS: Gaussian Splatting with Temporal Opacity Offset for Real-Time 4D DSA Rendering

Shuai Zhang, Huangxuan Zhao, Zhenghong Zhou, Guanjun Wu, Chuansheng Zheng, Xinggang Wang, Wenyu Liu

School of Electronic Information and Communications, Huazhong University of Science and Technology, Wuhan, China, School of Electronic Information and Communications, Huazhong University of Science and Technology, Department of Radiology, Union Hospital, Tongji Medical College, Huazhong University of Science and Technology, Wuhan, China, Department of Radiology, Tongji Medical College, School of Computer Science & Technology, Huazhong University of Science and Technology, Wuhan, China, School of Computer Science & Technology, School of Computer Science &Technology, Huazhong University of Science and Technology, Wuhan, China, School of Computer Science &Technology

医学影像

针对4D DSA临床重建依赖多视角采集、带来辐射负担且现有稀疏视角渲染慢/质量不足的问题,TOGS抓住血管几何基本静止、主要随造影剂流动改变不透明度的特点,为每个3D Gaussian引入时间不透明度偏移表,并配合平滑损失和随机剪枝抑制过拟合、降低存储。实验显示其在相同训练视角下达到SOTA渲染质量,并实现超过300 FPS的实时渲染。

Adaptive Voxelization for Transform coding of 3D Gaussian splatting data Figure 1
arXiv preprint2025-05-30

Adaptive Voxelization for Transform coding of 3D Gaussian splatting data

Chenjunjie Wang, Shashank N. Sridhara, Eduardo Pavez, Antonio Ortega, Cheng Chang

University of Southern California, Los Angeles, CA, University of Southern California, Meta, Menlo Park, CA, Meta, Menlo School

压缩

针对 3DGS 模型高斯数量多、存储与多码率传输成本高的问题,论文把点云变换编码引入 3DGS 压缩,并提出按高斯体积、密度和体素化误差自适应分配空间分辨率:大体积高斯保高精度,密集小高斯降精度并合并,再轻量重调颜色等属性。实验显示该框架在预训练数据上较现有后训练压缩取得更好的率失真表现。

CLIPGaussian: Universal and Multimodal Style Transfer Based on Gaussian Splatting Figure 1
arXiv preprint2025-05-28

CLIPGaussian: Universal and Multimodal Style Transfer Based on Gaussian Splatting

Kornel Howil, Piotr Borycki, Tadeusz Dziarmaga, Marcin Mazur

风格迁移

针对高斯泼溅内容风格迁移多停留在颜色编辑、难兼顾视频时序与3D/4D几何的问题,CLIPGaussian将风格迁移做成可插入GS管线的统一模块,用CLIP/VGG损失直接优化高斯原语的颜色、位置和尺度,支持文本或参考图驱动的2D、视频、3D与4D风格化。实验按模态比较,显示其在风格一致性、几何变化和视频连贯性上优于或接近现有方法,且不增加原始GS模型规模。

Robust Gaussian Splatting Figure 1
Proceedings of the ACM on Computer Graphics and Interactive Techniques2025-05-22

Robust Gaussian Splatting

Xuechang Tu, Lukas Radl, Michael Steiner, Markus Steinberger, Bernhard Kerbl, Fernando de la Torre

Peking University, Peking, China, Peking University, Institute of Visual Computing, Graz University of Technology, Graz, Austria, Institute of Visual Computing, Graz University of Technology, Institute of Visual Computing, Graz University of Technology, Graz, Austria and Huawei Technologies, Graz, Austria, Austria and Huawei Technologies, Carnegie Mellon University, Pittsburgh, Pennsylvania, USA, Carnegie Mellon University

去模糊渲染

该文针对手持采集中的模糊、位姿误差和跨图像颜色漂移会显著破坏 3DGS 重建的问题,将运动模糊建模为相机位姿上的高斯分布,从而联合做位姿细化与去模糊,并用投影协方差补偿散焦、用逐图像颜色变换处理白平衡和光照不一致。在 Scannet++ 与 Deblur-NeRF 上,相比 3DGS、MipNeRF-360 等基线取得稳定提升,同时基本保留 3DGS 的训练和渲染效率。

PG-SAG: Parallel Gaussian Splatting for Fine-Grained Large-Scale Urban Buildings Reconstruction via Semantic-Aware Grouping Figure 1
PFG – Journal of Photogrammetry Remote Sensing and Geoinformation Science2025-05-20

PG-SAG: Parallel Gaussian Splatting for Fine-Grained Large-Scale Urban Buildings Reconstruction via Semantic-Aware Grouping

Tengfei Wang, Xin Wang, Yongmao Hou, Yiwei Xu, Wendi Zhang, Zongqian Zhan

School of Geodesy and Geomatics, Wuhan University, School of Geodesy and Geomatics, Wuhan University, Institute of Geodesy and Geophysics

大规模场景网格重建优化方法

面向大规模城市建筑重建,现有3DGS方法受显存限制常需空间切块和降采样,难以保留建筑边界与细节。PG-SAG用LSA提取建筑语义掩码,并按多视图可见性分组成可并行优化的子区域,在原始分辨率下仅用掩码像素训练;同时引入边界感知法线损失和梯度约束负载均衡损失。实验显示其在多种城市数据上较Metashape、2DGS、SuGaR、GOF等获得更高精度和更清晰边界。

Next Best Sense: Guiding Vision and Touch with FisherRF for 3D Gaussian Splatting Figure 1
ICRA 20252025-05-19

Next Best Sense: Guiding Vision and Touch with FisherRF for 3D Gaussian Splatting

Matthew Strong, Boshu Lei, Aiden Swann, Wen Jiang, Kostas Daniilidis, Monroe Kennedy

University of Pennsylvania, Department of Computer Science, Philadelphia, PA, USA, University of Pennsylvania, Department of Computer Science, Philadelphia University, School of Engineering and Applied Science, University of Pennsylvania, Philadelphia, PA, USA, School of Engineering and Applied Science, Stanford University, Department of Mechanical Engineering, Stanford, CA, USA, Stanford University, Department of Mechanical Engineering, Stanford

其他机器人

该工作针对在线机器人操作中3DGS可用视角少、随机采样重叠冗余的问题,提出联合“下一最佳视角/触觉”的主动采集流程:用SAM2语义深度对齐及深度、法向损失增强少视角重建,并将FisherRF按深度不确定性扩展到相机与触碰位姿选择。实验称在真实机器人在线训练中带来定性和定量提升,但给定PDF片段与题名不一致,细节和增益拆分判断受限。

DreamDrive: Generative 4D Scene Modeling from Street View Images Figure 1
ICRA 20252025-05-19

DreamDrive: Generative 4D Scene Modeling from Street View Images

Jiageng Mao, Boyi Li, Boris Ivanovic, Yuxiao Chen, Yan Wang, Yurong You, Chaowei Xiao, Danfei Xu, Marco Pavone, Yue Wang

NVIDIA Research, Nvidia (United Kingdom)

自动驾驶动态场景前馈重建

DreamDrive针对自动驾驶中生成视频易泛化但缺3D一致性、重建方法一致但依赖标注的问题,将视频扩散生成的街景参考提升为4D时空场景,并用自监督静动态分解与混合高斯表示结合Gaussian Splatting渲染。实验在nuScenes和野外数据上显示其可生成可控、可泛化且几何一致的新视角驾驶视频,视觉质量较既有方法约提升30%,并能辅助感知与规划任务。

FOCI: Trajectory Optimization on Gaussian Splats Figure 1
IROS 20252025-05-13

FOCI: Trajectory Optimization on Gaussian Splats

Mario Gomez Andreu, Victor Klemm, Vaishakh Patil, Jesus Tordesillas, Marco Hutter

ETH Zürich, Robotic Systems Lab, Switzerland, ETH Zürich, Robotic Systems Lab, Comillas Pontifical University, IIT and ICAI, Spain, Comillas Pontifical University

优化方法机器人

针对机器人如何直接利用3D Gaussian Splatting场景进行导航的问题,FOCI将环境与机器人都表示为高斯体,并用高斯间重叠积分构造可微碰撞代价,从而支持带姿态的全身轨迹优化,避免保守包围盒限制。实验在合成与真实3DGS、ANYmal四足机器人上验证,可在数秒内处理数十万高斯并生成无碰轨迹;但避障仍是软代价,偶发碰撞与对3DGS质量敏感仍需改进。

GaussianHead: High-fidelity Head Avatars with Learnable Gaussian Derivation Figure 1
IEEE Transactions on Visualization and Computer Graphics2025-04-17

GaussianHead: High-fidelity Head Avatars with Learnable Gaussian Derivation

Jie Wang, Jiu-Cheng Xie, Xianyan Li, Feng Xu, Chi-Man Pun, Hao Gao

School of Automation and the School of Artificial Intelligence, Nanjing University of Posts and Telecommunications, Nanjing, China, School of Automation and the School of Artificial Intelligence, Nanjing University of Posts and Telecommunications, School of Software and BNRist, Tsinghua University, Beijing, China, School of Software and BNRist, Tsinghua University, Department of Computer and Information Science, University of Macau, Taipa, Macau, Department of Computer and Information Science, University of Macau

数字人

面向单目视频构建可动画化高保真头部数字人的难题,GaussianHead以各向异性3D Gaussian表示动态几何,并用单分辨率tri-plane存储外观;其关键洞察是通过可学习的Gaussian派生生成多个投影“替身”,缓解轴对齐映射导致的特征稀释,同时用继承式初始化加速新增Gaussian训练。实验显示其在重建、跨身份驱动和新视角合成上优于已有方法,模型约12MB。

3DGS.zip: A survey on 3D Gaussian Splatting Compression Methods Figure 1
Computer Graphics Forum 20252025-04-17

3DGS.zip: A survey on 3D Gaussian Splatting Compression Methods

M. T. Bagdasarian, P. Knoll, Y. Li, F. Barthel, A. Hilsmann, P. Eisert, W. Morgenstern

Fraunhofer Institute for Telecommunications, Heinrich Hertz Institute, Fraunhofer Institute for Telecommunications, Heinrich Hertz Institute

压缩

3DGS虽能实时高质量渲染,但原始模型常达GB级,限制移动端、头显和大场景应用。本文的核心贡献不是提出新压缩器,而是把快速并行发展的方法区分为“压缩文件大小”和“压实高斯数量”两类,梳理量化、剪枝、结构化编码、低维特征图等设计取舍,并用统一指标和数据集重整比较结果。主要结果是给出首个系统评测框架和持续更新网站,为后续方法选择、复现与标准化提供基线。

TranSplat: Generalizable 3D Gaussian Splatting from Sparse Multi-View Images with Transformers Figure 1
Proceedings of the AAAI Conference on Artificial Intelligence2025-04-11

TranSplat: Generalizable 3D Gaussian Splatting from Sparse Multi-View Images with Transformers

Chuanrui Zhang, Yingshuang Zou, Zhuoling Li, Minmin Yi, Haoqian Wang

Tsinghua University, 2The University of Hong Kong, 3E-surfing Vision Technology Co., Ltd, Tsinghua University, The University of Hong Kong

前馈重建稀疏表示

TranSplat面向稀疏多视图前馈3DGS重建中依赖精确匹配的问题:遮挡、弱纹理、重复区域和视角不重叠会使深度与高斯中心估计不稳。其核心是用深度置信图引导局部匹配,并引入单目深度先验,经DDMT与Depth Refine U-Net细化深度后并行预测高斯参数。实验在RealEstate10K和ACID上取得G-3DGS最佳结果,同时保持竞争性速度与较强跨数据集泛化。

Rethinking Open-Vocabulary Segmentation of Radiance Fields in 3D Space Figure 1
Proceedings of the AAAI Conference on Artificial Intelligence2025-04-11

Rethinking Open-Vocabulary Segmentation of Radiance Fields in 3D Space

Hyunjee Lee, Youngsik Yun, Jeongmin Bae, Seoha Kim, Youngjung Uh

Yonsei University

语言嵌入分割

这篇论文针对现有开放词汇 NeRF/3DGS 分割只在视角上输出 2D 掩码、难以表示完整三维语义体的问题,重新定义为对 3D 体空间分割:在体渲染前直接监督 3D 点的语言嵌入,并用 SAM 掩码构造尺度无关特征,再将语言场迁移到 3DGS。结果在 3D 与渲染 2D 分割上达到 SOTA,并实现首个开放词汇实时渲染,速度较此前最快方法提升 28 倍。

Modeling uncertainty for Gaussian Splatting Figure 1
IEEE Transactions on Neural Networks and Learning Systems2025-04-08

Modeling uncertainty for Gaussian Splatting

Luca Savant Aira, Diego Valsesia, Enrico Magli

Department of Electronics and Telecommunications, Politecnico di Torino, Turin, Italy, Department of Electronics and Telecommunications

三维高斯泼溅

针对三维高斯泼溅虽能实时高质量新视角合成、却无法给出结果可信度的问题,论文提出随机高斯泼溅(SGS),用变分推断将GS参数置于贝叶斯框架,并把AUSE稀疏化误差面积纳入训练以同时优化重建与不确定性估计。在LLFF实验中,SGS相较现有方法提升了渲染质量和不确定性指标,为机器人等真实场景决策提供可筛除高风险视角的置信信息。

ABC-GS: Alignment-Based Controllable Style Transfer for 3D Gaussian Splatting Figure 1
ICME 20252025-03-28

ABC-GS: Alignment-Based Controllable Style Transfer for 3D Gaussian Splatting

Wenjie Liu, Zhongliang Liu, Xiaoyan Yang, Man Sha, Yang Li

East China Normal University, School of Computer Science and Technology, Shanghai, China, East China Normal University, School of Computer Science and Technology, East China Normal University, School of Software Engineering, Shanghai, China, School of Software Engineering, Shanghai Chinafortune Co., Ltd, Shanghai, China

风格迁移

针对 NeRF/NNFM 式 3D 风格迁移容易只匹配局部特征、缺少全局风格约束且难以精细控制的问题,ABC-GS 转向显式 3D Gaussian Splatting,并通过分割掩码建立内容区域、风格区域与高斯的可控匹配;其 FAST 损失在特征空间对齐渲染分布与参考风格分布,同时用深度损失和高斯正则保几何。实验显示该方法支持单图、组合和语义感知风格化,风格一致性、可控性和多视角一致性优于相关方法,并保持实时渲染能力。

Snap-it, Tap-it, Splat-it: Tactile-Informed 3D Gaussian Splatting for Reconstructing Challenging Surfaces Figure 1
3DV 20252025-03-25

Snap-it, Tap-it, Splat-it: Tactile-Informed 3D Gaussian Splatting for Reconstructing Challenging Surfaces

Mauro Comi, Alessio Tonioni, Jonathan Tremblay, Max Yang, Valts Blukis, Yijiong Lin, Nathan F. Lepora, Laurence Aitchison

University of Bristol, United Kingdom, University of Bristol, Google, USA, Google, Google (United States), NVIDIA, USA, NVIDIA, Nvidia (United States)

渲染

针对纯视觉3DGS在高光、反射等非朗伯表面和少视角条件下几何易失真的问题,本文将触觉传感得到的局部深度接触图并入多视角RGB重建,通过在接触位置约束高斯的透射率并加入深度平滑正则来直接校正表面几何。实验在合成光泽/反射数据和真实金属烤面包机上显示,其几何重建和新视角合成优于基线,少视角时更稳,并声称速度较逆渲染类方法快约10倍。

Rig3DGS: Creating Controllable Portraits from Casual Monocular Videos Figure 1
3DV 20252025-03-25

Rig3DGS: Creating Controllable Portraits from Casual Monocular Videos

Alfredo Rivero, ShahRukh Athar, Zhixin Shu

Stony Brook University, Stony Brook University, Captions, Adobe Research, Adobe Systems (United States)

数字人

Rig3DGS面向普通手机单目人像视频,解决可控数字人中头部姿态、表情与视角在单视角下难解耦、NeRF方案训练和渲染慢的问题。方法将整场景表示为规范空间3D高斯,并用来自3DMM/FLAME的可学习形变先验约束每个高斯的运动,使表情和头姿控制落在合理子空间。实验显示其在新表情、新头姿和新视角合成上优于既有神经人像方法,并因3DGS表示达到约50倍速度提升。

RealmDreamer: Text-Driven 3D Scene Generation with Inpainting and Depth Diffusion Figure 1
3DV 20252025-03-25

RealmDreamer: Text-Driven 3D Scene Generation with Inpainting and Depth Diffusion

Jaidev Shriram, Alex Trevithick, Lingjie Liu, Ravi Ramamoorthi

University of California, San Diego, University of California, University of Pennsylvania, California University of Pennsylvania

扩散生成

RealmDreamer针对文本到3D场景中3D数据稀缺、SDS蒸馏易过饱和且几何粗糙的问题,将预训练2D修复扩散模型改作“场景条件”监督未知视角区域,并用深度扩散进行几何蒸馏,优化3D Gaussian Splatting;无需视频或多视图数据即可生成复杂前向场景,用户研究中相对现有方法获得88–95%的偏好率。

RadSplat: Radiance Field-Informed Gaussian Splatting for Robust Real-Time Rendering with 900+ FPS Figure 1
3DV 20252025-03-25

RadSplat: Radiance Field-Informed Gaussian Splatting for Robust Real-Time Rendering with 900+ FPS

Michael Niemeyer, Fabian Manhardt, Marie-Julie Rakotosaona, Michael Oechsle, Daniel Duckworth, Rama Gosula, Keisuke Tateno, John Bates, Dominik Kaeser, Federico Tombari

Google, Google (United States)

密度控制其他渲染

RadSplat针对NeRF类辐射场质量高但体渲染昂贵、3DGS虽快却依赖脆弱密度控制且大场景易膨胀的问题,将Zip-NeRF等辐射场作为初始化先验与监督来优化高斯表示,并加入最多约10×减点的剪枝和测试时视点可见性过滤。实验显示其在中大规模复杂采集中质量优于3DGS,PSNR最高提升1.87 dB,速度达907 FPS,较3DGS快3.6×、较Zip-NeRF快3000×以上。

LoopSplat: Loop Closure by Registering 3D Gaussian Splats Figure 1
3DV 20252025-03-25

LoopSplat: Loop Closure by Registering 3D Gaussian Splats

Liyuan Zhu, Yue Li, Erik Sandström, Shengyu Huang, Konrad Schindler, Iro Armeni

Stanford University, University of Amsterdam, ETH Zurich

同步定位与建图

LoopSplat针对现有3DGS RGB-D SLAM虽能构建稠密地图、却缺少回环闭合导致轨迹漂移和地图变形的问题,将场景划分为3D Gaussian子图,并直接在3DGS表示上做在线回环检测与配准以生成位姿图约束,避免传统点云配准的额外转换和低效。其在Replica、TUM-RGBD、ScanNet/++上显示出更稳健的跟踪、建图与渲染表现。

LapisGS: Layered Progressive 3D Gaussian Splatting for Adaptive Streaming Figure 1
3DV 20252025-03-25

LapisGS: Layered Progressive 3D Gaussian Splatting for Adaptive Streaming

Yuang Shi, Géraldine Morin, Simone Gasparini, Wei Tsang Ooi

National University of Singapore, IPAL - Image & Pervasive Access Lab (Institute for Infocomm Research, Institute for Infocomm Research, IRIT - University of Toulouse

压缩

面向XR/云游戏中3DGS模型体量大、网络带宽和设备能力动态变化的问题,LapisGS将高斯点组织为可累积的分层渐进表示,用基础层加增强层共享内容,结合动态不透明度优化保持跨层一致性,并用占用图剔除低价值splat以降存储和渲染成本。实验显示在仅约23%原模型大小下,SSIM最高提升50.71%、LPIPS最高提升286.53%,适合带宽自适应3D流式传输。

From Sparse to Dense: Camera Relocalization with Scene-Specific Detector from Feature Gaussian Splatting Figure 1
CVPR 20252025-03-25

From Sparse to Dense: Camera Relocalization with Scene-Specific Detector from Feature Gaussian Splatting

Zhiwei Huang, Hailin Yu, Yichun Shentu, Jin Yuan, Guofeng Zhang

Zhejiang University, State Key Lab of CAD & CG, Zhejiang University, State Key Lab of CAD & CG, SenseTime Research

位姿估计

针对传统稀疏特征在弱纹理场景匹配不足、NeRF/GS 定位又常依赖初始位姿或光度优化的问题,STDLoc 将特征蒸馏到 3D Gaussian 中,先用面向匹配的高斯采样与场景专属检测器获得稀疏 2D-3D 对应和 PnP 初值,再通过查询特征图与高斯特征场的稠密对齐精化位姿。室内外实验显示其在定位精度和召回率上优于现有方法。

EgoGaussian: Dynamic Scene Understanding from Egocentric Video with 3D Gaussian Splatting Figure 1
3DV 20252025-03-25

EgoGaussian: Dynamic Scene Understanding from Egocentric Video with 3D Gaussian Splatting

Daiwei Zhang, Gengyan Li, Jiajie Li, Mickaël Bressieux, Otmar Hilliges, Marc Pollefeys, Luc Van Gool, Xi Wang

ETH Zürich

动态场景

EgoGaussian面向头戴RGB视频中的人—物交互理解,解决传统静态重建忽略物体运动、易产生鬼影且依赖多相机/深度等传感器的问题。其核心是利用3D Gaussian Splatting离散表示便于分离背景与动态物体,并按交互的静/动态片段进行在线式重建与刚体运动跟踪。论文在两个野外第一视角数据集上按新视角合成协议评估,动态物体和背景重建质量均优于现有方法。

Drivable 3D Gaussian Avatars Figure 1
3DV 20252025-03-25

Drivable 3D Gaussian Avatars

Wojciech Zielonka, Timur Bagautdinov, Shunsuke Saito, Michael Zollhöfer, Justus Thies, Javier Romero

Max Planck Institute for Intelligent Systems, Tübingen, Germany, Max Planck Institute for Intelligent Systems, Meta, Codec Avatars Lab, Meta, Codec Avatars Lab

数字人

面向远程通信中的可驱动写真人体,D3GA针对LBS难处理服装滑动、局部表情和复杂非线性形变的问题,将3D Gaussian嵌入四面体 cage,用形变梯度同时更新位置、方向和协方差,并以身体、衣物、脸等多层组合接受不同驱动信号。多视角数据实验显示其在PSNR/SSIM上超过同类SOTA,同时保持较紧凑和实时;但宽松衣物自碰撞、高频纹理仍有限。

360-GS: Layout-guided Panoramic Gaussian Splatting For Indoor Roaming Figure 1
3DV 20252025-03-25

360-GS: Layout-guided Panoramic Gaussian Splatting For Indoor Roaming

Jiayang Bai, Letian Huang, Jie Guo, Wen Gong, Yuanqi Li, Yanwen Guo

Nanjing University

全景重建渲染

针对传统 3D-GS 难以直接处理稀疏室内全景图、球面投影失真以及无纹理平面易产生漂浮物的问题,360-GS 将高斯先投到单位球切平面再映射到球面,并利用房间布局先验进行点云初始化与几何正则。实验显示其在真实室内场景中可实时全景漫游,较 3D-GS 等方法减少伪影并提升新视角合成质量。

Motion Blender Gaussian Splatting for Dynamic Scene Reconstruction Figure 1
arXiv preprint2025-03-12

Motion Blender Gaussian Splatting for Dynamic Scene Reconstruction

Xinyu Zhang, Haonan Chang, Yuhan Liu, Abdeslam Boularias

Rutgers University

动态场景编辑机器人分割

这篇工作针对动态 Gaussian Splatting 多用隐式运动、只能回放而难以编辑和用于机器人规划的问题,提出 MBGS:以稀疏显式运动图(运动树/可变形图)表示物体运动,并用双四元数蒙皮和可学习权重将连杆运动传播到高斯点,随视频端到端优化。实验在 iPhone 数据集优于 Shape-of-Motion、在 HyperNeRF 具竞争力,并展示了新姿态动画、机器人示教合成和视觉规划,但新姿态仍有伪影。

Persistent Object Gaussian Splat (POGS) for Tracking Human and Robot Manipulation of Irregularly Shaped Objects Figure 1
ICRA 20252025-03-07

Persistent Object Gaussian Splat (POGS) for Tracking Human and Robot Manipulation of Irregularly Shaped Objects

Justin Yu, Kush Hari, Karim El-Refai, Arnav Dalal, Justin Kerr, Chung Min Kim, Richard Cheng, Muhammad Zubair Irshad, Ken Goldberg

UC Berkeley (automation.berkeley.edu), The AUTOLab, UC Berkeley (automation.berkeley.edu), The AUTOLab, Berkeley College, Toyota Research Institute, Los Altos, CA, Toyota Research Institute

语言嵌入机器人分割

面向无 CAD 模型、形状不规则且会被人或机器人反复移动的物体,POGS 将 3D Gaussian Splat 扩展为可持续更新的物体中心特征场,融合 CLIP/DINO/Detic 的语言、视觉与分组特征,用单个双目相机在线估计位姿并支持语言驱动操作。实机中物体复位平均误差 2.92 cm,最多连续成功 12 次;工具伺服可从 30° 扰动中恢复,成功率 80%,但跟踪频率仅约 5Hz、遮挡下鲁棒性仍受限。

SplatFace: Gaussian Splat Face Reconstruction Leveraging an Optimizable Surface Figure 1
WACV 20252025-02-26

SplatFace: Gaussian Splat Face Reconstruction Leveraging an Optimizable Surface

Jiahao Luo, Jing Liu, James Davis

University of California, Santa Cruz, University of California, ByteDance Inc

数字人

SplatFace针对少量输入图像下人脸3DGS重建依赖密集多视角或精确深度的问题,引入可优化的通用3DMM表面,与高斯点通过非刚性联合优化协同对齐;其splat-to-surface距离同时考虑高斯位置与协方差,并用表面引导世界空间 densification 以减少漂浮点、捕获细节。实验显示其在新视角合成质量上可与高斯方法竞争,并能生成几何精度较高的人脸网格。

SGD: Street View Synthesis with Gaussian Splatting and Diffusion Prior Figure 1
WACV 20252025-02-26

SGD: Street View Synthesis with Gaussian Splatting and Diffusion Prior

Zhongrui Yu, Haoran Wang, Jinze Yang, Hanzhang Wang, Jiale Cao, Zhong Ji, Mingming Sun

ETH Zürich, Baidu Research, Harbin Institute of Technology, Baidu (China), University of Chinese Academy of Sciences, Tianjin University

扩散生成稀疏表示

面向自动驾驶仿真中的街景新视角合成,SGD指出车载相机视角稀疏导致3DGS在偏离训练轨迹的自由视角下易模糊和伪影。方法用相邻帧与LiDAR深度微调扩散模型,并在3DGS训练时用其约束未见视角,从而引入街景外观与几何先验。在KITTI/KITTI-360上,相比现有方法在稀疏视角和大视角偏移下渲染更稳,同时保持3DGS推理实时性,但训练因扩散去噪明显变慢。

OmniGS: Omnidirectional Gaussian Splatting for Fast Radiance Field Reconstruction using Omnidirectional Images Figure 1
WACV 20252025-02-26

OmniGS: Omnidirectional Gaussian Splatting for Fast Radiance Field Reconstruction using Omnidirectional Images

Longwei Li, Huajian Huang, Sai-Kit Yeung, Hui Cheng

Sun Yat-sen University, China, Sun Yat-sen University, The Hong Kong University of Science and Technology, China, The Hong Kong University of Science and Technology, Hong Kong University of Science and Technology

全景重建渲染

OmniGS针对传统3DGS只能处理无畸变透视图、全景NeRF训练和渲染较慢的问题,推导球面相机模型在高斯溅射中的梯度,并实现GPU加速的全向光栅器,直接将3D高斯投到等距柱状屏幕空间,无需立方体展开或切平面近似。实验在漫游与自我中心场景中显示,其全景辐射场重建质量达到SOTA,并保持较高渲染速度。

GauFRe: Gaussian Deformation Fields for Real-time Dynamic Novel View Synthesis Figure 1
WACV 20252025-02-26

GauFRe: Gaussian Deformation Fields for Real-time Dynamic Novel View Synthesis

Yiqing Liang, Numair Khan, Zhengqin Li, Thu Nguyen-Phuoc, Douglas Lanman, James Tompkin, Lei Xiao

Brown University, John Brown University, Meta

动态场景

面向单目动态场景新视角合成中几何与运动耦合、传统 NeRF 渲染训练慢的问题,GauFRe 将动态部分表示为规范空间中的 3D Gaussian 模板,并用时间条件前向形变场直接生成各时刻高斯;同时加入面向 GS 的静态高斯分量和归纳偏置初始化,让形变主要处理运动区域。实验显示其质量与现有 NeRF/高斯方法竞争,同时真实场景约 20 分钟训练、RTX 3090 上 96 FPS 渲染。

DN-Splatter: Depth and Normal Priors for Gaussian Splatting and Meshing Figure 1
WACV 20252025-02-26

DN-Splatter: Depth and Normal Priors for Gaussian Splatting and Meshing

Matias Turkulainen, Xuqian Ren, Iaroslav Melekhov, Otto Seiskari, Esa Rahtu, Juho Kannala

ETH, Zurich, ETH Zurich, Tampere University, Aalto University

网格重建

DN-Splatter针对室内3DGS在弱纹理、少视角区域缺少几何约束而产生漂浮物和表面噪声的问题,引入深度与法线先验进行优化正则:用基于颜色梯度的边缘感知深度损失约束高斯位置,并用单目法线对齐局部表面,同时支持从高斯表示直接提取网格。实验显示该策略提升新视角合成与深度估计指标,并显著改善室内场景网格的几何一致性和表面质量。

Instruct-4DGS: Efficient Dynamic Scene Editing via 4D Gaussian-based Static-Dynamic Separation Figure 1
arXiv preprint2025-02-04

Instruct-4DGS: Efficient Dynamic Scene Editing via 4D Gaussian-based Static-Dynamic Separation

Joohyun Kwon, Hanbyel Cho, Junmo Kim

KAIST, South Korea, KAIST

动态场景扩散生成渲染编辑

针对现有4D动态场景编辑需逐帧编辑大量2D图像并反复训练、随时间步数扩展性差的问题,Instruct-4DGS利用4D高斯中静态3D Gaussians与Hexplane形变场的可分离性,只编辑静态外观组件并保留运动,再用基于Coherent-IP2P的分数蒸馏细化时序错位与伪影。实验显示其编辑时间较基线减少一半以上,同时指令一致性和视觉质量更好,但不能直接编辑运动且部分编辑依赖分割。

GaussNav: Gaussian Splatting for Visual Navigation Figure 1
IEEE Transactions on Pattern Analysis and Machine Intelligence2025-02-04

GaussNav: Gaussian Splatting for Visual Navigation

Xiaohan Lei, Min Wang, Wengang Zhou, Houqiang Li

MoE Key Laboratory of Brain-inspired Intelligent Perception and Cognition, University of Science and Technology of China, Hefei, China, MoE Key Laboratory of Brain-inspired Intelligent Perception and Cognition, University of Science and Technology of China, MCC Lab, Department of Electronic Engineering and Information Science, University of Science and Technology of China, Hefei, China, MCC Lab, Department of Electronic Engineering and Information Science, Institute of Artificial Intelligence, Hefei Comprehensive National Science Center, Hefei, China, Institute of Artificial Intelligence, Hefei Comprehensive National Science Center

自动驾驶机器人

GaussNav面向实例图像目标导航中“同类干扰物多、视角差异大”的难点,指出传统2D BEV地图缺少三维几何与纹理细节,难以做实例级匹配。论文将3D Gaussian Splatting引入导航,构建融合几何、语义与外观纹理的Semantic Gaussian地图,通过渲染候选实例并与目标图像匹配来定位目标,再转为路径规划。在HM3D上SPL由0.347提升到0.578,显示细粒度可渲染地图对IIN任务有明显收益。

OmniPhysGS: 3D Constitutive Gaussians for General Physics-Based Dynamics Generation Figure 1
arXiv preprint2025-01-31

OmniPhysGS: 3D Constitutive Gaussians for General Physics-Based Dynamics Generation

Yuchen Lin, Chenguo Lin, Jianjin Xu, Yadong Mu

Peking University, 2Carnegie Mellon University, Peking University, Carnegie Mellon University

动态场景物理建模

针对现有物理驱动 4D/动态 3D 生成常把场景材料预设为单一类别、难以处理异质物体交互的问题,OmniPhysGS 将 3DGS 粒子扩展为可学习的本构高斯,并用 12 类专家本构模型的加权组合表示每个高斯材料,再借助视频扩散模型按文本监督材料权重。实验显示其能覆盖弹性、黏弹、塑性和流体等混合动态,在视觉质量与文本对齐指标上较基线提升约 3%–16%。

Lifting by Gaussians: A Simple, Fast and Flexible Method for 3D Instance Segmentation Figure 1
WACV 20252025-01-31

Lifting by Gaussians: A Simple, Fast and Flexible Method for 3D Instance Segmentation

Rohan Chacko, Nicolai Häni, Eldar Khaliullin, Lin Sun, Douglas Lee

语言嵌入分割虚拟现实

针对现有 3DGS 场景分割常需逐场景训练、预处理昂贵且语义易渗漏的问题,LBG 将 SAM/CLIP/DINO 等 2D 掩码与特征直接提升到最大贡献高斯,并用几何与语义重叠增量合并成实例,无需优化语义场。实验显示其在新视角语义渲染和独立 3D 资产提取上优于或持平现有方法,速度约快一个数量级。

Zero-Shot Novel View and Depth Synthesis with Multi-View Geometric Diffusion Figure 1
CVPR 20252025-01-30

Zero-Shot Novel View and Depth Synthesis with Multi-View Geometric Diffusion

Vitor Guizilini, Muhammad Zubair Irshad, Dian Chen, Greg Shakhnarovich, Rares Ambrus

Toyota Research Institute (TRI), Toyota Research Institute, Toyota Technological Institute at Chicago (TTIC), Toyota Technological Institute at Chicago

全景重建扩散生成前馈重建大规模场景点云

针对稀疏位姿图像重建依赖 NeRF/3DGS 等中间三维表示且跨场景泛化受限的问题,MVGD 将新视角 RGB 与深度直接建模为条件扩散生成任务,用 raymap 注入相机几何,并以任务嵌入统一多模态输出;结合 6000 万级多视图样本、尺度归一化和增量微调,在多项新视角合成、立体与视频深度基准上达到 SOTA,但部分收益可能主要来自 scaling / data。

DiffSplat: Repurposing Image Diffusion Models for Scalable Gaussian Splat Generation Figure 1
arXiv preprint2025-01-28

DiffSplat: Repurposing Image Diffusion Models for Scalable Gaussian Splat Generation

Chenguo Lin, Panwang Pan, Bangbang Yang, Zeming Li, Yadong Mu

Peking University, 2ByteDance, Peking University, ByteDance

扩散生成

DiffSplat针对文本/单图到3D生成中高质量3D数据稀缺、两阶段多视图扩散易不一致的问题,将3D Gaussian Splat组织成类似图像的splat grids/latents,微调预训练2D扩散模型直接生成3DGS,并用快速重建模型扩充数据、加入可微渲染损失约束任意视角一致性。实验显示其在文本和图像条件生成及下游应用上优于对比方法,消融支持VAE适配与渲染损失等设计的贡献。

Advancing Extended Reality with 3D Gaussian Splatting: Innovations and Prospects Figure 1
AIXVR 20252025-01-27

Advancing Extended Reality with 3D Gaussian Splatting: Innovations and Prospects

Shi Qiu, Binzhu Xie, Qixuan Liu, Pheng-Ann Heng

The Chinese University of Hong Kong, Department of Computer Science and Engineering, HKSAR, China, The Chinese University of Hong Kong, Department of Computer Science and Engineering, Chinese University of Hong Kong

综述

本文针对3DGS虽常被认为适合XR、但真实XR系统验证不足的问题,系统梳理272篇公开3DGS论文,发现152篇提到XR却多停留在应用展望。其核心贡献是从XR需求出发归纳3DGS相关创新并建立分类,同时分析VR-GS、DualGS、RGCA等少数具体案例。主要结果是给出3DGS推动XR内容创建、交互和实时渲染的研究路线图,但文中未提供统一实验基准或量化增益评估。

Trick-GS: A Balanced Bag of Tricks for Efficient Gaussian Splatting Figure 1
ICASSP 20252025-01-24

Trick-GS: A Balanced Bag of Tricks for Efficient Gaussian Splatting

Anil Armagan, Albert Saà-Garriga, Bruno Manganelli, Mateusz Nowak, M. Kerim Yucel

Samsung R&D Institute UK (SRUK)

加速训练

Trick-GS 面向 3D Gaussian Splatting 在移动端等受限设备上模型过大、训练仍慢的问题,不提出单一新模块,而是系统筛选并组合渐进式分辨率/噪声/尺度训练、基于重要性的 Gaussian 与球谐带剪枝/掩码,以及加速训练框架。三类数据集上在精度基本相当的同时,较原版 GS 最高实现约 2 倍训练加速、40 倍存储压缩和 2 倍渲染加速。

Micro-macro Wavelet-based Gaussian Splatting for 3D Reconstruction from Unconstrained Images Figure 1
Proceedings of the AAAI Conference on Artificial Intelligence2025-01-24

Micro-macro Wavelet-based Gaussian Splatting for 3D Reconstruction from Unconstrained Images

Yihui Li, Chengxin Lv, Hongyu Yang, Di Huang

State Key Laboratory of Complex and Critical Software Environment, Beijing, China School of Computer Science and Engineering, Beihang University, China, State Key Laboratory of Complex and Critical Software Environment, China School of Computer Science and Engineering, Beihang University, School of Artificial Intelligence, Beihang University, China Shanghai Artificial Intelligence Laboratory, Shanghai, China, School of Artificial Intelligence, China Shanghai Artificial Intelligence Laboratory, Beijing Academy of Artificial Intelligence

野外场景

该文面向互联网/野外无约束图像中光照变化、外观差异和短暂遮挡导致的3D重建模糊与伪影问题,提出MW-GS,将高斯特征拆为全局外观、细化外观与固有特征,并用微/宏投影在多尺度锥台内采样、结合小波频域采样和层级残差融合增强局部纹理与高光建模。实验显示其在多场景新视角渲染和重建质量上优于现有NeRF/3DGS野外方法。

HAMMER: Heterogeneous, Multi-Robot Semantic Gaussian Splatting Figure 1
IEEE RA-L 20252025-01-24

HAMMER: Heterogeneous, Multi-Robot Semantic Gaussian Splatting

Javier Yu, Timothy Chen, Mac Schwager

Department of Aeronautics and Astronautics, Stanford University, Stanford, CA, USA, Department of Aeronautics and Astronautics, Stanford University, Stanford, Vaughn College of Aeronautics and Technology

机器人同步定位与建图

HAMMER针对异构多机器人实时构建统一语义3D地图的计算与坐标对齐难题,采用基于ROS的中心服务器架构,将各机器人本地SLAM位姿、RGB-D/点云流一次性对齐到全局坐标,并在线训练带CLIP语义的3D Gaussian Splatting地图,无需已知相对初始位姿。实验显示其在Replica与真实3–4设备场景中达到或优于多机器人基线,计算时间低于十分之一,真实新视角重建MSE较Di-NeRF改善超过40%,并支持“去沙发”等语言导航任务。

Dense-SfM: Structure from Motion with Dense Consistent Matching Figure 1
CVPR 20252025-01-24

Dense-SfM: Structure from Motion with Dense Consistent Matching

JongMin Lee, Sungjoo Yoo

Seoul National University

点云位姿估计

传统SfM依赖稀疏关键点,在弱纹理区域点云密度和位姿精度受限;Dense-SfM将DKM/RoMa等稠密匹配接入SfM,并用Gaussian Splatting估计可见性来延长短轨迹,再结合Transformer与高斯过程的多视图核化匹配和BA细化轨迹。其在ETH3D、Texture-Poor SfM及IMC 2021上较现有SfM取得更高重建精度与点云密度。

3DGS-CD: 3D Gaussian Splatting-based Change Detection for Physical Object Rearrangement Figure 1
IEEE RA-L 20252025-01-24

3DGS-CD: 3D Gaussian Splatting-based Change Detection for Physical Object Rearrangement

Ziqi Lu, Jianbo Ye, John Leonard

Computer Science and Artificial Intelligence Laboratory, Massachusetts Institute of Technology, Cambridge, MA, USA, Computer Science and Artificial Intelligence Laboratory, Massachusetts Institute of Technology

机器人

面向机器人在未持续观测的真实环境中发现物体被移动、移除或插入的问题,3DGS-CD用3D Gaussian Splatting重渲染旧场景,并结合EfficientSAM在同视角比较新旧RGB图像,将2D变化跨视角关联融合为3D物体掩码和6D位姿变化;无需深度、类别或物体模型,少至一张新图约18秒完成检测,在真实数据上较NeRF式方法最高提升14%准确率且快约三个数量级。

GoDe: Gaussians on Demand for Progressive Level of Detail and Scalable Compression Figure 1
arXiv preprint2025-01-23

GoDe: Gaussians on Demand for Progressive Level of Detail and Scalable Compression

Di Sario

Computer Science, University of Turin, Corso Svizzera, Turin, 10149, Italy, University of Turin, National Institute of Informatics

压缩

GoDe针对现有3D Gaussian Splatting压缩方法通常为单码率、需为不同预算分别训练的问题,提出从单个已训练模型出发,用高斯 primitive 的聚合梯度敏感度构建由粗到细的LoD层级,并通过一次量化感知微调生成可渐进解码的统一表示。实验显示其率失真表现接近或优于单码率SOTA,同时支持按内存、带宽和算力动态自适应渲染。

Fast3R: Towards 3D Reconstruction of 1000+ Images in One Forward Pass Figure 1
CVPR 20252025-01-23

Fast3R: Towards 3D Reconstruction of 1000+ Images in One Forward Pass

Jianing Yang, Alexander Sax, Kevin J. Liang, Mikael Henaff, Hao Tang, Ang Cao, Joyce Chai, Franziska Meier, Matt Feiszli

Meta FAIR, University of Michigan, University of Michigan–Ann Arbor

三维重建

Fast3R针对DUSt3R等多视角重建方法依赖两两图像推理与全局对齐、随视角数增长易变慢甚至显存溢出的瓶颈,改用可并行处理无序多图的Transformer点图回归,让所有视角在一次前向中相互注意并避免迭代对齐。实验显示其可扩展到远超训练视角数的输入,在相机位姿与三维重建上保持或提升精度,并在CO3Dv2上15度内位姿准确率达99.7%,相对DUSt3R全局对齐误差降低14倍且推理更快。

Sketch and Patch: Efficient 3D Gaussian Representation for Man-Made Scenes Figure 1
arXiv preprint2025-01-22

Sketch and Patch: Efficient 3D Gaussian Representation for Man-Made Scenes

Yuang Shi, Simone Gasparini, Geraldine Morin, Chenggang Yang, Wei Tsang Ooi

IPAL, IRL2955, National University of Singapore, Singapore, Singapore, National University of Singapore, IPAL, IRL2955, University of Toulouse, IRIT, Toulouse, France, University of Toulouse, National University of Singapore, Singapore, Singapore

密度控制

针对 3D Gaussian Splatting 在人造场景中因边缘/轮廓过度密集化导致存储开销大的问题,本文提出将高斯按作用拆为 Sketch 与 Patch:前者利用 3D 线等参数化先验紧凑编码结构边界,后者通过剪枝、重训练和向量量化保留平滑区域体积外观。在相同模型大小下,PSNR/SSIM/LPIPS 最高分别提升 32.62%、19.12%、45.41%,室内场景可用原模型 2.3% 大小维持视觉质量。

3DGS$^2$: Near Second-order Converging 3D Gaussian Splatting Figure 1
arXiv preprint2025-01-22

3DGS$^2$: Near Second-order Converging 3D Gaussian Splatting

Lei Lan, Tianjia Shao, Zixuan Lu, Yu Zhang, Chenfanfu Jiang, Yin Yang

LEI LAN, University of Utah, USA, University of Utah, TIANJIA SHAO, Zhejiang University, China, Zhejiang University, ZIXUAN LU, University of Utah, USA, YU ZHANG, University of Utah, USA, CHENFANFU JIANG, University of California, Los Angeles, USA, University of California, YIN YANG, University of Utah, USA

优化方法

针对标准 SGD 训练 3DGS 收敛慢、常需数十分钟的问题,论文指出高斯核属性对图像损失较局部且跨图像耦合稀疏,因此将参数按核与属性拆分,解析构造小规模局部 Newton 系统并在 GPU 并行求解,同时用空间采样抑制随机训练过冲。实验显示迭代数减少超过 10 倍,训练时间约降一个数量级,重建质量保持或优于 SGD 基线。

HAC++: Towards 100X Compression of 3D Gaussian Splatting Figure 1
IEEE Transactions on Pattern Analysis and Machine Intelligence2025-01-21

HAC++: Towards 100X Compression of 3D Gaussian Splatting

Yihang Chen, Qianyi Wu, Weiyao Lin, Mehrtash Harandi, Jianfei Cai

Shanghai Jiao Tong University, Shanghai, China, Shanghai Jiao Tong University, Shanghai Jiao Tong University, China, Monash University, Melbourne, VIC, Australia, Monash University, Monash University, Australia

压缩

针对3D Gaussian Splatting中海量、稀疏且无序的高斯/锚点属性难以压缩的问题,HAC++的核心洞察是利用无序锚点与结构化哈希网格之间的互信息来做上下文熵建模,并结合锚点内部上下文、自适应量化和掩码剪枝去除冗余。实验显示其在五个数据集上相对原始3DGS平均压缩超过100倍、相对Scaffold-GS超过20倍,同时保持甚至提升重建保真度。

DARB-Splatting: Generalizing Splatting with Decaying Anisotropic Radial Basis Functions Figure 1
WACV 20262025-01-21

DARB-Splatting: Generalizing Splatting with Decaying Anisotropic Radial Basis Functions

Hashiru Pramuditha, Vinasirajan Viruthshaan, Vishagar Arunan, Saeedha Nazar, Sameera Ramasinghe, Simon Lucey, Ranga Rodrigo

University of Moratuwa, University of Adelaide, The University of Adelaide

渲染

本文针对 3DGS 长期依赖高斯/指数族核、非高斯核因投影积分困难而少被探索的问题,提出以马氏距离定义的衰减各向异性径向基函数(DARBF)统一替代 splat 核,并用校正因子近似保留高斯闭式投影优势。实验显示若干非指数核在 PSNR、SSIM、LPIPS 基本持平下,可带来最高约 34% 训练加速,并降低显存,尤其半余弦和逆多二次核体现效率收益。

Car-GS: Addressing Reflective and Transparent Surface Challenges in 3D Car Reconstruction Figure 1
arXiv preprint2025-01-19

Car-GS: Addressing Reflective and Transparent Surface Challenges in 3D Car Reconstruction

Congcong Li, Jin Wang, Xiaomeng Wang, Xingchen Zhou, Wei Wu, Yuzhi Zhang, Tongyi Cao

网格重建渲染

面向自动驾驶仿真、VR/AR 等对高质量车辆模型的需求,Car-GS 针对车漆反光与车窗透明导致的几何/颜色耦合问题改造 3DGS:用视角相关高斯刻画反射,为深度和法线单独学习几何不透明度,并以质量感知方式引入预训练法线先验修正玻璃表面。实验显示其在车体表面重建精度和鲁棒性上优于已有方法。

GSTAR: Gaussian Surface Tracking and Reconstruction Figure 1
arXiv preprint2025-01-17

GSTAR: Gaussian Surface Tracking and Reconstruction

Chengwei Zheng, Lixin Xue, Juan Zarate, Jie Song

ETH Z¨urich

数字人动态场景网格重建

GauSTAR针对动态场景中表面出现、消失或分裂导致3D Gaussian难以同时保持重建质量与跨帧跟踪的问题,将高斯绑定到网格面形成Gaussian Surface;拓扑稳定区域随网格跟踪,变化区域自适应解绑高斯并重网格化生成新表面,同时用基于表面的scene flow初始化大位移。实验显示其在外观指标上达到或超过SOTA,并能提供可用于编辑、动捕等任务的显式时序网格跟踪。

GS-LIVO: Real-Time LiDAR, Inertial, and Visual Multi-sensor Fused Odometry with Gaussian Mapping Figure 1
arXiv preprint2025-01-15

GS-LIVO: Real-Time LiDAR, Inertial, and Visual Multi-sensor Fused Odometry with Gaussian Mapping

Sheng Hong, Chunran Zheng, Yishu Shen, Changze Li, Fu Zhang, Tong Qin, Shaojie Shen

大规模场景

GS-LIVO针对现有视觉3D-GS SLAM建图更新慢、显存和算力开销高、难以在大规模场景实时部署的问题,将LiDAR、IMU与相机紧耦合到高斯地图中。其关键在于哈希索引八叉树全局高斯地图、融合初始化、滑窗高斯优化,以及基于渲染的IESKF视觉量测。实验显示其显著降低显存并加速高斯优化,在室内外保持有竞争力的里程计精度和渲染质量,并可在Jetson Orin NX上实时运行。

CityLoc: 6 DoF Localization of Text Descriptions in Large-Scale Scenes with Gaussian Representation Figure 1
arXiv preprint2025-01-15

CityLoc: 6 DoF Localization of Text Descriptions in Large-Scale Scenes with Gaussian Representation

Qi Ma, Runyi Yang, Bin Ren, Ender Konukoglu, Luc Van Gool, Danda Pani Paudel

语言嵌入大规模场景

CityLoc面向城市级3D场景中“交通灯旁建筑”等语言描述天然对应多处位置的问题,目标是为机器人/LLM提供基于文本的6DoF位姿分布而非单点定位。方法用CLIP文本/图像特征条件化扩散模型生成候选位姿,并借助3D Gaussian Splatting渲染候选视图,以文本-图像相似度过滤和细化。作者在5个覆盖超10平方公里的大规模真实/仿真场景上优于常规分布估计与相关定位基线,且文本越细致精度越高。

VINGS-Mono: Visual-Inertial Gaussian Splatting Monocular SLAM in Large Scenes Figure 1
arXiv preprint2025-01-14

VINGS-Mono: Visual-Inertial Gaussian Splatting Monocular SLAM in Large Scenes

Ke Wu, Zicheng Zhang, Muer Tie, Ziqing Ai, Zhongxue Gan, Wenchao Ding

大规模场景网格重建同步定位与建图

面向现有 GS-SLAM 依赖深度/LiDAR、难以扩展到户外公里级场景的问题,VINGS-Mono 将单目视觉惯性前端与 2D Gaussian Map 结合,并用采样光栅化、评分管理、单到多位姿细化提升实时建图与定位;其 NVS 回环利用高斯地图自身做检测和一次性校正大量高斯属性,Dynamic Eraser 抑制动态物体伪影。实验显示其定位接近 VIO,渲染和建图质量优于近期 GS/NeRF SLAM,并可用手机相机和低频 IMU 实时生成高质量大规模高斯地图。

Object-Centric 2D Gaussian Splatting: Background Removal and Occlusion-Aware Pruning for Compact Object Models Figure 1
arXiv preprint2025-01-14

Object-Centric 2D Gaussian Splatting: Background Removal and Occlusion-Aware Pruning for Compact Object Models

Marcel Rogge, Didier Stricker

Augmented Vision, University of Kaiserslautern-Landau, Kaiserslautern, Germany, University of Kaiserslautern-Landau, Department of Augmented Vision, Deutsches Forschungszentrum fuer Kuenstliche Intelligenz, Kaiserslautern, Germany, Department of Augmented Vision, University of Koblenz and Landau, University of Kaiserslautern

压缩密度控制编辑

这篇工作针对高斯泼溅通常重建整场景、在只需单个物体时训练和存储浪费的问题,在2DGS中引入由分割掩码监督的背景损失,直接抑制背景高斯,并用遮挡感知剪枝移除对渲染贡献很小的高斯。实验显示其可生成物体中心的高斯与网格模型,模型最多缩小96%、训练最多加速71%,质量仍具竞争力,可直接用于外观编辑和物理仿真。

UnCommon Objects in 3D Figure 1
CVPR 20252025-01-13

UnCommon Objects in 3D

Xingchen Liu, Piyush Tayal, Jianyuan Wang, Jesus Zarzar, Tom Monnier, Konstantinos Tertikas, Jiali Duan, Antoine Toisoul, Jason Y. Zhang, Natalia Neverova, Andrea Vedaldi, Roman Shapovalov, David Novotny

Meta AI, King Abdullah University of Science and Technology, Carnegie Mellon University

三维高斯泼溅

面向真实物体三维学习中数据难兼顾规模、质量与全视角覆盖的问题,uCO3D构建了17万段众包360°物体视频,覆盖1070类,并提供相机、深度/点云、文本描述和3D Gaussian Splatting重建;通过人工视频质检、VGGSfM与基于新视角合成的标注验证提升可用性。作者用其训练LRM、CAT3D及类Instant3D模型,相比MVImgNet/CO3Dv2取得更好重建、多视角生成和文本到3D结果,增益可能主要来自更高质量与更多样数据。

RMAvatar: Photorealistic Human Avatar Reconstruction from Monocular Video Based on Rectified Mesh-embedded Gaussians Figure 1
Graphical Models 20252025-01-13

RMAvatar: Photorealistic Human Avatar Reconstruction from Monocular Video Based on Rectified Mesh-embedded Gaussians

Sen Peng, Weixing Xie, Zilong Wang, Xiaohu Guo, Zhonggui Chen, Baorong Yang, Xiao Dong

tent human model. Traditional methods usually rely on dense

数字人动态场景网格重建单目重建

RMAvatar面向单目视频重建可驱动数字人时,衣物、头发等非刚性细节难由骨架LBS和纯隐式场稳定表达的问题,提出将3D Gaussian嵌入三角网格以获得几何约束与运动初始化,并用姿态相关的Gaussian校正模块补偿位置和协方差细节变形。论文在公开数据集上报告其渲染质量和定量指标达到SOTA,主要收益来自网格约束与细粒度非刚性校正的结合。

Evaluating Human Perception of Novel View Synthesis: Subjective Quality Assessment of Gaussian Splatting and NeRF in Dynamic Scenes Figure 1
arXiv preprint2025-01-13

Evaluating Human Perception of Novel View Synthesis: Subjective Quality Assessment of Gaussian Splatting and NeRF in Dynamic Scenes

Yuhang Zhang, Joshua Maraval, Zhengyu Zhang, Nicolas Ramin, Shishun Tian, Lu Zhang

动态场景

面向VR/AR等应用中NVS最终由人观看的问题,本文针对现有评测偏静态、偏NeRF和合成场景的不足,构建含GS与NeRF的动态真实场景主观质量评测,覆盖360°、前向和单视点路径,并进行无参考与有参考实验。结果显示不同观看路径会影响感知质量判断,现有客观指标与主观评分仍不一致,说明动态场景下NVS质量评估和方法改进仍有明显空间。

3DGS-to-PC: Convert a 3D Gaussian Splatting Scene into a Dense Point Cloud or Mesh Figure 1
arXiv preprint2025-01-13

3DGS-to-PC: Convert a 3D Gaussian Splatting Scene into a Dense Point Cloud or Mesh

Lewis A G Stuart, Michael P Pound

School of Computer Science, University of Nottingham

点云

这篇技术报告针对 3DGS 场景依赖专用渲染器、直接导出高斯中心又过于稀疏的问题,提出将高斯按三维密度概率采样为点云,并用马氏距离抑制离群点;同时通过多视角渲染按像素贡献重估高斯颜色,避免直接使用球谐颜色带来的伪色。方法可从 .ply/.splat 生成稠密彩色点云,并结合 Poisson 重建输出网格,无需重训,主要结果是提升了与常规点云/网格工具的兼容性,但速度和颜色质量仍依赖原始相机位姿。

Synthetic Prior for Few-Shot Drivable Head Avatar Inversion Figure 1
CVPR 20252025-01-12

Synthetic Prior for Few-Shot Drivable Head Avatar Inversion

Wojciech Zielonka, Stephan J. Garbin, Alexandros Lattas, George Kopanas, Paulo Gotardo, Thabo Beeler, Justus Thies, Timo Bolkart

Max Planck Institute for Intelligent Systems, Tübingen, Germany, Max Planck Institute for Intelligent Systems, Google, Google (United States)

数字人动态场景稀疏表示

针对高保真可驱动头部数字人依赖大量真实多视角/视频数据且受隐私合规限制、少样本方法又易在新视角和表情中过拟合的问题,SynShot 用大规模合成头部数据学习先验,再以少量真实图像进行拟合和 pivotal tuning 适配域差;模型以 UV 空间卷积编码器生成 3D Gaussian 参数,并按头部部件控制 primitive 上采样。实验显示,三张输入即可重建可动画化头像,在新视角和新表情合成上优于多种单目和 GAN 基线。

Generalized and Efficient 2D Gaussian Splatting for Arbitrary-scale Super-Resolution Figure 1
ICCV 20252025-01-12

Generalized and Efficient 2D Gaussian Splatting for Arbitrary-scale Super-Resolution

Du Chen, Liyi Chen, Zhengqiang Zhang, Lei Zhang

The Hong Kong Polytechnic University, Hong Kong Polytechnic University

三维高斯泼溅

针对任意尺度超分中 INR 逐像素查询导致表达力和效率受限的问题,论文将高斯泼溅从单场景优化改造成可泛化的前馈 2D 表示:由低分辨率图像预测条件高斯,并用尺度感知 CUDA 光栅化并行渲染任意倍率结果。实验显示 GSASR 在质量上优于或匹配主流 INR 方法,且在高倍率下推理显著更快,但性能仍受高斯数量影响。

F3D-Gaus: Feed-forward 3D-aware Generation on ImageNet with Cycle-Consistent Gaussian Splatting Figure 1
arXiv preprint2025-01-12

F3D-Gaus: Feed-forward 3D-aware Generation on ImageNet with Cycle-Consistent Gaussian Splatting

Yuxin Wang, Qianyi Wu, Dan Xu

Monash University

前馈重建单目重建

F3D-Gaus面向仅有ImageNet这类单目数据时难以学习稳定3D表示、且新视角纹理/几何不一致的问题,采用前馈的像素对齐Gaussian Splatting,并通过自监督cycle-aggregative约束融合多视角高斯以补足遮挡区域,再引入几何引导的视频先验细化大视角细节。实验显示其在单目训练下获得更真实、跨视角一致的3D-aware生成,并提升训练与推理效率。

MEt3R: Measuring Multi-View Consistency in Generated Images Figure 1
CVPR 20252025-01-10

MEt3R: Measuring Multi-View Consistency in Generated Images

Mohammad Asim, Christopher Wewer, Thomas Wimmer, Bernt Schiele, Jan Eric Lenssen

Max Planck Institute for Informatics, Saarland Informatics Campus, Max Planck Institute for Informatics

三维重建扩散生成

多视角/视频扩散生成缺少无需配对真值且能衡量3D一致性的指标,传统FID、PSNR或基于极线的方法难以反映生成视图是否可用于后续重建。MEt3R用DUSt3R从图像对估计稠密点图,在DINO+FeatUp特征空间做跨视角投影与相似度比较,不依赖相机位姿并对光照变化更稳健。实验显示它能区分近似一致与完全一致序列,捕捉随时间变化的细粒度不一致,并用于系统比较多类新视角/视频生成方法及作者的MV-LDM。

Locality-aware Gaussian Compression for Fast and High-quality Rendering Figure 1
arXiv preprint2025-01-10

Locality-aware Gaussian Compression for Fast and High-quality Rendering

Seungjoo Shin, Jaesik Park, Sunghyun Cho

Seoul National University

压缩

针对 3DGS 单场景需存储大量显式高斯属性、压缩后又易损画质或拖慢渲染的问题,LocoGS 利用邻近高斯属性的局部相干性,将其编码到多分辨率哈希网格神经场,并配合稠密初始化、自适应 SH 带宽、剪枝与属性定制量化,保持原 3DGS 渲染管线。实验显示其在真实数据上优于既有紧凑高斯表示,存储压缩 54.6–96.6 倍,渲染较 3DGS 快 2.1–2.4 倍,且比 HAC 平均快 2.4 倍。

Zero-1-to-G: Taming Pretrained 2D Diffusion Model for Direct 3D Generation Figure 1
arXiv preprint2025-01-09

Zero-1-to-G: Taming Pretrained 2D Diffusion Model for Direct 3D Generation

Xuyi Meng, Chen Wang, Jiahui Lei, Kostas Daniilidis, Jiatao Gu, Lingjie Liu

University of Pennsylvania 2Apple

扩散生成

针对直接 3D 生成受限于高质量 3D 数据稀缺、从头训练成本高的问题,Zero-1-to-G 将 Gaussian splats 分解为多视角、多属性的 2D 图像,并在预训练 2D 扩散模型中加入跨视角与跨属性注意力及 VAE 解码器微调,从而单阶段生成 3D 表示。实验显示其在合成和真实图像输入上提升了结构一致性、渲染质量与未见物体泛化能力。

Scaffold-SLAM: Structured 3D Gaussians for Simultaneous Localization and Photorealistic Mapping Figure 1
arXiv preprint2025-01-09

Scaffold-SLAM: Structured 3D Gaussians for Simultaneous Localization and Photorealistic Mapping

Wen Tianci, Liu Zhiang, Lu Biao, Fang Yongchun

IRAIS, tjKLIR, College of Artificial Intelligence, Nankai University, College of Artificial Intelligence, Nankai University, IITRS, Shenzhen Research Institute, Nankai University, Shenzhen Research Institute

同步定位与建图

该工作针对现有 3DGS-SLAM 在单目/双目/RGB-D 场景中结构不一致、外观变化和高频细节建模不足的问题,引入基于 ORB-SLAM3 点云的结构化高斯锚点初始化,并用由相机位姿驱动的 AfME 与频率金字塔正则提升渲染细节。实验显示其在多类公开数据集上显著提升真实感建图质量,同时保持有竞争力的跟踪精度;TUM RGB-D 单目设置下 PSNR 相比 MonoGS 提升约 19.86%。

Consistent Flow Distillation for Text-to-3D Generation Figure 1
arXiv preprint2025-01-09

Consistent Flow Distillation for Text-to-3D Generation

Runjie Yan, Yinbo Chen, Xiaolong Wang

扩散生成

针对SDS在文本到3D蒸馏中偏向最大似然、易牺牲视觉质量和多样性的问题,论文提出Consistent Flow Distillation:从扩散ODE/SDE的梯度采样视角出发,强调跨视角2D概率流应在物体表面保持一致,并用多视角一致高斯噪声低成本实现对应。实验显示其在不同2D扩散模型上较现有蒸馏方法生成更真实且更多样的3D资产,额外开销接近SDS。

Arc2Avatar: Generating Expressive 3D Avatars from a Single Image via ID Guidance Figure 1
CVPR 20252025-01-09

Arc2Avatar: Generating Expressive 3D Avatars from a Single Image via ID Guidance

Foivos Paraperas Papantoniou, Rolandos Alexandros Potamias, Alexandros Lattas, Stefanos Zafeiriou

Imperial College London, UK, Imperial College London

数字人扩散生成

Arc2Avatar面向单图生成可表情驱动3D数字人的难题,针对传统SDS依赖文本导致身份弱、颜色过饱和的问题,引入Arc2Face身份先验并扩展到多视角,用FLAME模板初始化的改造3DGS、连通正则和低guidance蒸馏保持稠密对应与自然外观。实验显示其在身份保持、真实感和表情一致性上优于对比方法,用户研究中获得明显偏好。

GaussianVideo: Efficient Video Representation via Hierarchical Gaussian Splatting Figure 1
ICCV 20252025-01-08

GaussianVideo: Efficient Video Representation via Hierarchical Gaussian Splatting

Andrew Bond, Jui-Hsien Wang, Long Mai, Erkut Erdem, Aykut Erdem

Koç University, Adobe Research, Adobe Systems (United States), Hacettepe University

高斯视频

该文针对神经视频表示中显存占用高、训练慢和时间一致性差的问题,将视频表示为可显式渲染的层次化 3D Gaussian,并用 B-spline 约束场景元素的平滑运动、Neural ODE 联合学习连续相机轨迹,减少对预计算相机和光流/深度等重监督的依赖。实验显示其在多类高/低运动视频上提升重建质量与时间稳定性,同时降低训练时间和内存开销。

FatesGS: Fast and Accurate Sparse-View Surface Reconstruction using Gaussian Splatting with Depth-Feature Consistency Figure 1
Proceedings of the AAAI Conference on Artificial Intelligence2025-01-08

FatesGS: Fast and Accurate Sparse-View Surface Reconstruction using Gaussian Splatting with Depth-Feature Consistency

Han Huang, Yulun Wu, Chao Deng, Ge Gao, Ming Gu, Yu-Shen Liu

Beijing National Research Center for Information Science and Technology (BNRist), Tsinghua University, Beijing, China School of Software, Tsinghua University, Beijing, China, Beijing National Research Center for Information Science and Technology (BNRist), Tsinghua University, China School of Software, School of Software, Tsinghua University, Beijing, China, School of Software

网格重建稀疏表示

FatesGS针对高斯泼溅在稀疏视角下易过拟合少量训练图像、产生漂浮噪声和不完整网格的问题,将3D高斯转为更适合表面的2D椭圆表示,并结合单目深度排序的视图内深度一致性、平滑约束与跨视角投影特征一致性来同时稳定粗几何和恢复细节。在DTU与BlendedMVS上,其稀疏视角重建优于现有方法,且无需大规模预训练,速度提升约60至200倍。

MoDec-GS: Global-to-Local Motion Decomposition and Temporal Interval Adjustment for Compact Dynamic 3D Gaussian Splatting Figure 1
CVPR 20252025-01-07

MoDec-GS: Global-to-Local Motion Decomposition and Temporal Interval Adjustment for Compact Dynamic 3D Gaussian Splatting

Sangwoon Kwak, Joonsoo Kim, Jun Young Jeong, Won-Sik Cheong, Jihyong Oh, Munchurl Kim

Electronics and Telecommunications Research Institute, Chung-Ang University, Korea Advanced Institute of Science and Technology

压缩动态场景

面向动态 3DGS 在真实复杂运动视频中模型体积大、长序列易模糊的问题,MoDec-GS 将运动分解为全局到局部:先用全局锚点变形建模跨时段粗运动,再用局部高斯变形细化,并通过训练中的时间区间自适应分配减少手工分段。实验显示其在保持或提升渲染质量的同时,较动态 3DGS SOTA 平均缩小约 70% 模型体积,iPhone 数据集相对 SC-GS 还报告 +0.7dB PSNR 与 -94% 存储。

DehazeGS: Seeing Through Fog with 3D Gaussian Splatting Figure 1
arXiv preprint2025-01-07

DehazeGS: Seeing Through Fog with 3D Gaussian Splatting

Jinze Yu, Yiqun Wang, Zhengda Lu, Jianwei Guo, Yong Li, Hongxing Qin, Xiaopeng Zhang

College of Computer Science, Chongqing University, College of Computer Science, Chongqing University, School of Artificial Intelligence, University of Chinese Academy of Sciences, School of Artificial Intelligence, University of Chinese Academy of Sciences, School of Artificial Intelligence, Beijing Normal University, Beijing Normal University, MAIS, Institute of Automation, Chinese Academy of Sciences, Institute of Automation, Chinese Academy of Sciences

野外场景渲染

雾天散射与衰减会破坏3DGS/NeRF在野外场景的重建与新视角渲染,且现有NeRF去雾代价高、细节恢复受限。DehazeGS把大气散射模型显式嵌入3D Gaussian,通过高斯深度到透射率映射联合学习大气光、散射系数和清晰场景,并用深度与暗/亮通道先验约束优化;在合成和真实雾天数据上取得优于既有方法的渲染质量与速度。

Pointmap-Conditioned Diffusion for Consistent Novel View Synthesis Figure 1
WACV 20262025-01-06

Pointmap-Conditioned Diffusion for Consistent Novel View Synthesis

Thang-Anh-Quan Nguyen, Laurent Caraffa, Jean-Philippe Tarel, Roland Brémond

Gustave Eiffel University, France, Gustave Eiffel University

扩散生成

面向自动驾驶街景中相机轨迹覆盖有限、NeRF/3DGS难以外推到远视角的问题,PointmapDiff将栅格化3D坐标点图作为扩散模型条件,并结合ControlNet与参考视角注意力,把稀疏LiDAR或深度中的几何对应注入2D生成过程。实验显示其在真实驾驶数据上能生成更符合几何且跨视角一致的外推图像,并可蒸馏增强3DGS重建。

Gaussian Masked Autoencoders Figure 1
arXiv preprint2025-01-06

Gaussian Masked Autoencoders

Jathushan Rajasegaran, Xinlei Chen, Rulilong Li, Christoph Feichtenhofer, Jitendra Malik, Shiry Ginosar

Meta, FAIR1, UC Berkeley2, Toyota Technological Institute at Chicago3, Meta, UC Berkeley2, Toyota Technological Institute at Chicago3

三维高斯泼溅

这篇论文针对 MAE 虽能学到语义表征、却缺少显式空间结构约束的问题,将可微 3D Gaussian Splatting 引入自监督重建,在像素重建前学习动态分配位置、尺度和深度的高斯中间表示。结果显示,GMAE 在分类、检测等语义表征上基本保持 MAE 水平,同时无需微调即可产生前景/背景分割、图层分解和边缘检测等空间理解能力,额外训练开销约 1.5%。

Compression of 3D Gaussian Splatting with Optimized Feature Planes and Standard Video Codecs Figure 1
ICCV 20252025-01-06

Compression of 3D Gaussian Splatting with Optimized Feature Planes and Standard Video Codecs

Soonbin Lee, Fangwen Shu, Yago Sánchez, Thomas Schierl, Cornelius Hellge

Fraunhofer Heinrich-Hertz-Institute (HHI), Germany, Fraunhofer Heinrich-Hertz-Institute (HHI), Fraunhofer Institute for Telecommunications, Heinrich Hertz Institute, Fraunhofer Institute for Telecommunications, Heinrich Hertz Institute

压缩

该文针对3D Gaussian Splatting动辄占用数GB、难以部署到移动/头显等受限设备的问题,提出用渐进式tri-plane特征平面隐式预测高斯属性,并将通道拼成视频帧交给标准HEVC等非可微编码器压缩;关键在于DCT频域熵建模和按通道重要性分配码率,使训练目标更贴近实际码流。实验显示其在保持较高新视角渲染质量的同时,将场景压到数MB级,率失真表现优于已有3DGS压缩方法。

GS-DiT: Advancing Video Generation with Pseudo 4D Gaussian Fields through Efficient Dense 3D Point Tracking Figure 1
arXiv preprint2025-01-05

GS-DiT: Advancing Video Generation with Pseudo 4D Gaussian Fields through Efficient Dense 3D Point Tracking

Weikang Bian, Zhaoyang Huang, Xiaoyu Shi, Yijin Li, Fu-Yun Wang, Hongsheng Li

Multimedia Laboratory, The Chinese University of Hong Kong, Multimedia Laboratory, The Chinese University of Hong Kong, Centre for Perceptual and Interactive Intelligence 3Avolution AI

三维高斯泼溅

为解决视频生成难以在单目真实视频上实现多机位、Dolly Zoom 等 4D 镜头控制的问题,GS-DiT 用密集 3D 点跟踪直接构建无需优化的伪 4D 高斯场,并将其渲染结果作为条件微调预训练 Video DiT。其 D3D-PT 相比 SpatialTracker 精度更高、推理快两个数量级;模型可在保持动态内容的同时改变相机外参/内参并支持物体运动编辑,但真实 4D 表示仍未生成。

VideoLifter: Lifting Videos to 3D with Fast Hierarchical Stereo Alignment Figure 1
arXiv preprint2025-01-03

VideoLifter: Lifting Videos to 3D with Fast Hierarchical Stereo Alignment

Wenyan Cong, Kevin Wang, Jiahui Lei, Colton Stearns, Yuanhao Cai, Dilin Wang, Rakesh Ranjan, Matt Feiszli, Leonidas Guibas, Zhangyang Wang, Weiyao Wang, Zhiwen Fan

加速训练扩散生成

VideoLifter针对单目视频无位姿/内参重建中SfM不稳、逐帧优化慢且易累积漂移的问题,将长视频切成片段,先用MASt3R等学习式3D先验只提取位姿、尺度和点图等必要信息,再通过关键帧引导的树状层级3D Gaussian合并、剪枝与联合优化获得全局一致场景。在Tanks and Temples和CO3D-V2上,相比现有方法训练时间减少超过82%、约5倍加速,并提升新视角渲染质量。

EnerVerse: Envisioning Embodied Future Space for Robotics Manipulation Figure 1
arXiv preprint2025-01-03

EnerVerse: Envisioning Embodied Future Space for Robotics Manipulation

Siyuan Huang, Liliang Chen, Pengfei Zhou, Shengcong Chen, Zhengkai Jiang, Yue Hu, Peng Gao, Hongsheng Li, Maoqing Yao, Guanghui Ren

SJTU 2AgiBot 3Shanghai AI Lab 4CUHK MMLab 5LV-NUS Lab

动态场景机器人

这篇论文试图把视频生成的“预测未来”能力用于机器人操作,弥合通用视频模型与具身3D动态环境之间的表示差距。EnerVerse以chunk自回归扩散和稀疏记忆生成长时未来空间,用多视角预训练形成3D先验,并通过4DGS数据飞轮缓解sim-to-real;接入策略头后在仿真和真实任务达到SOTA,单4090约280ms输出8步动作块。

CrossView-GS: Cross-view Gaussian Splatting For Large-scale Scene Reconstruction Figure 1
arXiv preprint2025-01-03

CrossView-GS: Cross-view Gaussian Splatting For Large-scale Scene Reconstruction

Chenhao Zhang, Yuanping Cao, Lei Zhang

大规模场景优化方法

该文针对航拍、地面等跨视角输入中视角差异过大导致 3DGS 自适应致密化梯度被平滑、重建易退化的问题,提出 CrossView-GS:先按不同视角集合训练多分支 3DGS 作为高斯分布先验,再用梯度感知正则和高斯补充策略进行隐式与显式融合。实验显示其在大规模场景新视角合成上优于多种现有方法。

Cloth-Splatting: 3D Cloth State Estimation from RGB Supervision Figure 1
arXiv preprint2025-01-03

Cloth-Splatting: 3D Cloth State Estimation from RGB Supervision

Bardienus P

KTH Royal Institute of Technology 2Carnegie Mellon University

网格重建渲染

面向机器人布料操作中仅靠2D或深度表示难以稳定获得3D状态的问题,Cloth-Splatting将动作条件动力学预测与网格约束3D Gaussian Splatting结合,把布料网格顶点到RGB渲染建立可微映射,并用图像损失更新状态。实验在仿真和真实场景中优于2D/3D跟踪基线,精度提升约57%、收敛时间减少约85%,但仍非实时且依赖标定多相机与较好初始化。

Splat-Nav: Safe Real-Time Robot Navigation in Gaussian Splatting Maps Figure 1
T-RO 20252025-01-01

Splat-Nav: Safe Real-Time Robot Navigation in Gaussian Splatting Maps

Timothy Chen, Ola Shorinwa, Joseph Bruno, Aiden Swann, Javier Yu, Weijia Zeng, Keiko Nagami, Philip Dames, Mac Schwager

Stanford University, Stanford, CA, USA, Stanford University, Stanford, Temple University, Philadelphia, PA, USA, Temple University, University of California San Diego, San Diego, CA, USA, University of California San Diego

机器人

针对 NeRF 导航难以实时、传统点云/栅格又可能丢失几何细节的问题,Splat-Nav把高斯溅射地图中的椭球几何直接用于机器人导航:Splat-Plan构造可证明安全的多面体走廊并生成 Bézier 轨迹,Splat-Loc仅用机载RGB做PnP式定位,还支持语义GSplat语言目标。仿真中比点云规划更安全,硬件飞行中达到与MoCap/VIO相近的安全和速度,且无需手工坐标系对齐,重规划超过2Hz、定位约25Hz。

On Scaling Up 3D Gaussian Splatting Training Figure 1
arXiv preprint2025-01-01

On Scaling Up 3D Gaussian Splatting Training

Hexu Zhao, Haoyang Weng, Daohan Lu, Ang Li, Jinyang Li, Aurojit Panda, Saining Xie

New York University, New York, USA, New York University, Pacific Northwest National Laboratory, New York, USA, Pacific Northwest National Laboratory

大规模场景

本文针对单 GPU 3DGS 在高分辨率、大规模场景中受显存与负载不均限制的问题,提出分布式训练系统 Grendel:在 Gaussian 维与像素维之间混合并行,利用空间局部性的稀疏 all-to-all 通信和动态负载均衡,并支持多视角批训练及 sqrt(batch_size) 学习率缩放。实验显示其可在 16 GPU 上训练 4040 万高斯,在 4K Rubble 上 PSNR 达 27.28,高于单卡 1120 万高斯的 26.28。

Octree-GS: Towards Consistent Real-time Rendering with LOD-Structured 3D Gaussians Figure 1
IEEE Transactions on Pattern Analysis and Machine Intelligence2025-01-01

Octree-GS: Towards Consistent Real-time Rendering with LOD-Structured 3D Gaussians

Kerui Ren, Lihan Jiang, Tao Lu, Mulin Yu, Linning Xu, Zhangkai Ni, Bo Dai

Shanghai Jiao Tong University and Shanghai AI Laboratory, Shanghai Jiao Tong University, The University of Science and Technology of China and Shanghai AI Laboratory, University of Science and Technology of China, University of Shanghai for Science and Technology, Brown University, John Brown University, Shanghai AI Laboratory, ShangHai JiAi Genetics & IVF Institute, Shanghai Artificial Intelligence Laboratory, The Chinese University of Hong Kong, Chinese University of Hong Kong

大规模场景渲染

针对3D Gaussian Splatting在大规模场景、尤其远景缩放时仍渲染大量细小高斯而导致速度不稳和容量浪费的问题,Octree-GS用八叉树组织多尺度高斯,引入按视角足迹与场景复杂度动态选择LOD的机制,并配合grow-and-prune与渐进训练把细节分配到合适层级。实验显示其在保持重建质量的同时减少渲染原语,在大规模场景可达实时且最高约比现有方法快10倍。

Gaussian Splatting in Style Figure 1
arXiv preprint2025-01-01

Gaussian Splatting in Style

Abhishek Saroha, Mariia Gladkova, Cecilia Curreli, Dominik Muhle, Tarun Yenamandra, Daniel Cremers

Munich Center for Machine Learning, Munich, Germany, Munich Center for Machine Learning, Technical University of Munich, Munich, Germany, Technical University of Munich

风格迁移

这篇论文面向3D场景风格迁移中多视角外观不一致、逐风格重训代价高的问题,提出GSS:以3D Gaussian Splatting为显式场景骨架,用多分辨率哈希网格和轻量MLP在风格码条件下预测高斯颜色,从而在测试时泛化到新风格并保持几何一致性。实验在室内外真实数据上显示其短期/长期一致性和视觉质量优于多类基线,渲染约150FPS,适合AR/VR等实时应用。

Gamba: Marry Gaussian Splatting with Mamba for single view 3D reconstruction Figure 1
IEEE Transactions on Pattern Analysis and Machine Intelligence2025-01-01

Gamba: Marry Gaussian Splatting with Mamba for single view 3D reconstruction

Qiuhong Shen, Zike Wu, Xuanyu Yi, Pan Zhou, Hanwang Zhang, Shuicheng Yan, Xinchao Wang

National University of Singapore, Singapore, National University of Singapore, University of British Columbia, Canada, University of British Columbia, Nanyang Technological University, Singapore, Nanyang Technological University, Singapore Management University, Singapore, Singapore Management University, Research, Skywork AI, Singapore, Research

前馈重建稀疏表示

Gamba面向单图3D重建中SDS优化耗时、NeRF渲染低效和Transformer难以扩展大量3DGS token的问题,将3D高斯生成视为序列预测,提出基于Mamba的GambaFormer,并用多视角掩码导出的径向约束替代点云预热监督。在Objaverse训练、GSO评测中,其质量与现有方法竞争,同时A100上约0.05秒完成重建,显著快于优化式方法。

GIR: 3D Gaussian Inverse Rendering for Relightable Scene Factorization Figure 1
IEEE Transactions on Pattern Analysis and Machine Intelligence2025-01-01

GIR: 3D Gaussian Inverse Rendering for Relightable Scene Factorization

Yahao Shi, Yanmin Wu, Chenming Wu, Xing Liu, Chen Zhao, Haocheng Feng, Jian Zhang, Bin Zhou, Errui Ding, Jingdong Wang

State Key Laboratory of Virtual Reality Technology and Systems, Beihang University, Beijing, China, State Key Laboratory of Virtual Reality Technology and Systems, Beihang University, School of Electronic and Computer Engineering, Peking University, Shenzhen, China, School of Electronic and Computer Engineering, Peking University, VIS, Baidu Inc., Beijing, China, Baidu Inc, Baidu (China)

重光照渲染

GIR针对3DGS虽能实时新视角合成、却难以进行PBR材质分解和重光照的问题,将逆渲染直接建立在显式高斯表示上:用最短特征向量估计高斯法线并配合方向掩码约束,引入卷积式环境光表示和体素化间接光追踪来分离多次反射。实验显示其在重光照与新视角合成上优于近期逆渲染方法,同时保持实时渲染,适合材质编辑等交互应用。

EndoGS: Deformable Endoscopic Tissues Reconstruction with Gaussian Splatting Figure 1
arXiv preprint2025-01-01

EndoGS: Deformable Endoscopic Tissues Reconstruction with Gaussian Splatting

Lingting Zhu, Zhao Wang, Jiahao Cui, Zhenchao Jin, Guying Lin, Lequan Yu

The University of Hong Kong, Hong Kong SAR, China, The University of Hong Kong, University of Hong Kong, The Chinese University of Hong Kong, Hong Kong SAR, China, The Chinese University of Hong Kong, Chinese University of Hong Kong, Sun Yat-sen University, Guangzhou, China, Sun Yat-sen University

医学影像

针对单视角内窥镜视频中组织非刚性形变和器械遮挡导致重建慢、质量不稳的问题,EndoGS将3D Gaussian Splatting引入可变形手术场景,结合时间形变场、深度监督的时空权重掩码及表面对齐正则来优化几何与渲染。在DaVinci手术视频上,其渲染质量优于既有动态辐射场方法,并具备更快的实时重建潜力。

STORM: Spatio-Temporal Reconstruction Model for Large-Scale Outdoor Scenes Figure 1
arXiv preprint2024-12-31

STORM: Spatio-Temporal Reconstruction Model for Large-Scale Outdoor Scenes

Jiawei Yang, Jiahui Huang, Yuxiao Chen, Yan Wang, Boyi Li, Yurong You, Apoorva Sharma, Maximilian Igl, Peter Karkus, Danfei Xu, Boris Ivanovic, Yue Wang, Marco Pavone

自动驾驶动态场景大规模场景

STORM针对动态户外重建依赖逐场景优化、密集观测和运动伪标签导致耗时且泛化差的问题,采用数据驱动Transformer一次前向预测带速度的3D Gaussians,并用自监督场景流将多帧高斯聚合到目标时刻形成amodal表示;motion tokens进一步约束运动并涌现动态分割。在Waymo、NuScenes等数据上,动态区域PSNR较优化法高4.3–6.6,200ms完成大场景重建且场景流指标明显提升。

4D Gaussian Splatting: Modeling Dynamic Scenes with Native 4D Primitives Figure 1
arXiv preprint2024-12-30

4D Gaussian Splatting: Modeling Dynamic Scenes with Native 4D Primitives

Philip H. S

Fudan University, 200433, Shanghai, China, Fudan University, University of Surrey, GU2 7XH, Guildford, UK, University of Surrey, University of Oxford, OX1 2JD, Oxfordshire UK, University of Oxford

压缩动态场景大规模场景

面向动态场景新视角合成中时空相关难建模、实时渲染与存储开销难兼顾的问题,4DGS将场景直接表述为可优化的时空4D体,用各向异性4D高斯和Spherindrical Harmonics联合刻画几何、运动与外观演化,并配套可微splatting渲染。实验显示其在合成/真实、室内、驾驶等场景中提升画质与效率,支持交互式高分辨率渲染,并通过压缩变体缓解内存瓶颈。

MaskGaussian: Adaptive 3D Gaussian Representation from Probabilistic Masks Figure 1
CVPR 20252024-12-29

MaskGaussian: Adaptive 3D Gaussian Representation from Probabilistic Masks

Yifei Liu, Zhihang Zhong, Yifan Zhan, Sheng Xu, Xiao Sun

Shanghai AI Laboratory, Shanghai Artificial Intelligence Laboratory, ShangHai JiAi Genetics & IVF Institute, Beihang University

压缩密度控制

MaskGaussian针对3DGS因数百万高斯带来的显存与速度瓶颈,指出一次性或确定性剪枝会误删后期仍有用的点。其核心是把高斯建模为“存在概率”,并在混合阶段施加mask,使未采样高斯也能获梯度并动态调整重要性。实验在多个真实数据集平均剪除超过60%高斯,PSNR几乎不降,并带来约2–3倍渲染加速。

GSplatLoc: Ultra-Precise Camera Localization via 3D Gaussian Splatting Figure 1
arXiv preprint2024-12-28

GSplatLoc: Ultra-Precise Camera Localization via 3D Gaussian Splatting

Atticus J

Southeast University Chengxian College, Nanjing, China, Southeast University Chengxian College, Southeast University

野外场景点云位姿估计机器人同步定位与建图

传统/NeRF式定位在实时性、特征贫乏或渲染成本上受限,且现有3D Gaussian SLAM多依赖ICP,未充分利用可微渲染。GSplatLoc把已建3D高斯场景中的深度渲染与观测深度对齐,直接对相机位姿做梯度优化并GPU加速。实验在Replica达到0.01 cm内平移误差和近零旋转误差,并在TUM RGB-D复杂运动场景中显示较强鲁棒性。

Dust to Tower: Coarse-to-Fine Photo-Realistic Scene Reconstruction from Sparse Uncalibrated Images Figure 1
arXiv preprint2024-12-27

Dust to Tower: Coarse-to-Fine Photo-Realistic Scene Reconstruction from Sparse Uncalibrated Images

Xudong Cai, Yongcai Wang, Zhaoxin Fan, Deng Haoran, Shuo Wang, Wanting Li, Deying Li, Lun Luo, Minhang Wang, Jintao Xu

School of Information, Renmin University of China, Beijing, China, School of Information, Renmin University of China, Beijing Advanced Innovation Center for Future Blockchain and Privacy Computing, Institute of Artificial Intelligence, Beihang University, Beijing, China, Institute of Artificial Intelligence, Beihang University

位姿估计稀疏表示

针对实际中只有少量且未标定图像时,稀疏视角方法依赖精确位姿、SfM-free 方法又需密集采集的问题,D2T 采用由粗到细的 3DGS 与位姿联合优化:先用快速 MVS 初始化粗模型和相机,再以置信感知深度对齐和扭曲图像引导修复生成新视角监督。实验显示其在新视角合成与位姿估计上达到 SOTA,同时保持较高效率。

DAS3R: Dynamics-Aware Gaussian Splatting for Static Scene Reconstruction Figure 1
arXiv preprint2024-12-27

DAS3R: Dynamics-Aware Gaussian Splatting for Static Scene Reconstruction

Tze Ho Elden

National University of Singapore

三维高斯泼溅

DAS3R面向日常单目视频中大面积运动物体和复杂相机运动会干扰位姿估计、静态背景重建的问题。其关键做法是用成对帧学习动态掩码并聚合到视频级,再以该先验改进全局对齐、初始化3D高斯,并引入可优化的staticness属性在渲染中抑制动态成分。在DAVIS和Sintel上,相比近期去干扰重建方法PSNR提升超过2 dB,同时改善相机位姿估计;局限是静态区域仍可能被误判为动态。

WeatherGS: 3D Scene Reconstruction in Adverse Weather Conditions via Gaussian Splatting Figure 1
arXiv preprint2024-12-25

WeatherGS: 3D Scene Reconstruction in Adverse Weather Conditions via Gaussian Splatting

Chenghao Qian, Yuhu Guo, Wenjing Li, Gustav Markkula

野外场景

针对雨雪等恶劣天气下3DGS会把空中密集粒子和镜头水滴遮挡一起重建、导致场景浑浊的问题,WeatherGS将天气伪影拆分为密集粒子与稀疏镜头遮挡,采用先AEF去粒子、再LED生成遮挡掩码的预处理,并在掩码约束下训练3D高斯。论文还构建含合成与真实雨雪场景的基准,实验显示其相比NeRF/3DGS及相关方法能重建更干净的场景并保持实时渲染。

Gaussian Splatting with NeRF-based Color and Opacity Figure 1
Computer Vision and Image Understanding2024-12-25

Gaussian Splatting with NeRF-based Color and Opacity

Dawid Malarz, Jacek Tabor

College of Economics and Computer Science, WSEI, Kraków, Poland, College of Economics and Computer Science, Krakow University of Economics, Tischner European University, Doctoral School of Exact and Natural Sciences, Jagiellonian University, Kraków, Poland, Doctoral School of Exact and Natural Sciences, Jagiellonian University, Faculty of Mathematics and Computer Science, Jagiellonian University, Kraków, Poland, Department of Engineering, University of Cambridge, Cambridge, United Kingdom, Department of Engineering, University of Cambridge

渲染

该文针对 NeRF 渲染质量高但训练/推理慢、3D Gaussian Splatting 快但难以按视角条件化的问题,提出 VDGS:用高斯表示几何形状,同时引入接收高斯参数与观察方向的 NeRF 式网络调节颜色/不透明度,消融中主要调节不透明度效果更稳。结果显示其基本保持 GS 的训练和推理速度,同时更好刻画阴影、反射与透明物体,并减少视角相关伪影。

GauSim: Registering Elastic Objects into Digital World by Gaussian Simulator Figure 1
arXiv preprint2024-12-23

GauSim: Registering Elastic Objects into Digital World by Gaussian Simulator

Yidi Shao, Mu Huang, Chen Change Loy, Bo Dai

S-Lab Nanyang Technological University, 2Fudan University, S-Lab Nanyang Technological University, Fudan University, The University of Hong Kong, 4Shanghai Artificial Intelligence Laboratory, The University of Hong Kong, Shanghai Artificial Intelligence Laboratory

动态场景物理建模

GausSim面向3D Gaussian表示的真实弹性物体动态模拟,动机是弥合视频生成缺少物理约束、MPM等解析先验依赖理想假设的不足。核心做法是把每个高斯核视为连续介质的质心系统,并用层级CMS进行粗到细传播,同时显式约束质量与动量守恒。实验在合成数据和新建READY多视角真实形变数据上优于物理驱动基线,且将逐核计算减少约95%。

FaceLift: Single Image to 3D Head with View Generation and GS-LRM Figure 1
arXiv preprint2024-12-23

FaceLift: Single Image to 3D Head with View Generation and GS-LRM

Weijie Lyu, Yi Zhou, Ming-Hsuan Yang, Zhixin Shu

数字人前馈重建

单张人脸到完整3D头部的难点在于遮挡视角缺失、身份保持和后脑/头发一致性。FaceLift将问题拆成多视图扩散补全侧后视图与Transformer式GS-LRM融合为3D Gaussian,并通过合成多视图训练、输入视图重建约束和Objaverse预训练缓解域差。实验显示其在真实与合成基准上优于PanoHead、RodinHD等方法,身份保真、细节和渲染质量更好。

CoSurfGS:Collaborative 3D Surface Gaussian Splatting with Distributed Learning for Large Scene Reconstruction Figure 1
arXiv preprint2024-12-23

CoSurfGS:Collaborative 3D Surface Gaussian Splatting with Distributed Learning for Large Scene Reconstruction

Yuanyuan Gao, Yalun Dai, Hao Li, Weicai Ye, Junyi Chen, Danpeng Chen, Dingwen Zhang, Tong He, Guofeng Zhang, Junwei Han

Brain and Artificial Intelligence Lab, Northwestern Polytechnical University, Brain and Artificial Intelligence Lab, Northwestern Polytechnical University, Nanyang Technological University, Zhejiang University, Shanghai AI Lab

大规模场景网格重建

CoSurfGS针对现有3DGS表面重建在大规模场景中显存高、训练慢且几何细节不足的问题,将全局优化拆成多智能体本地区域并行训练,再通过LMC压缩冗余高斯、MAS以自蒸馏聚合并对齐相邻区域几何。实验在Urban3d、MegaNeRF、BlendedMVS上显示其可扩展地获得高保真网格/表面与照片级渲染,并较已有方法降低训练时间和显存开销。

Balanced 3DGS: Gaussian-wise Parallelism Rendering with Fine-Grained Tiling Figure 1
arXiv preprint2024-12-23

Balanced 3DGS: Gaussian-wise Parallelism Rendering with Fine-Grained Tiling

Hao Gui, Lin Hu, Rui Chen, Mingxiao Huang, Yuxin Yin, Jin Yang, Yong Wu

NVIDIA, Beijing, China, NVIDIA

加速训练

这篇论文针对 3DGS 训练中 renderCUDA 因像素与高斯球工作量差异导致的 SM、tile 和训练阶段间负载不均问题,从 CUDA kernel 层面重排并行方式。核心做法是结合跨 block 动态任务分配、Gaussian-wise 并行渲染与细粒度 tiling,并按训练阶段自适应选择 kernel,以减少 warp 内分歧和 SM 空闲。实验报告前向渲染 kernel 最高加速 7.52×,整体训练效率提升,但端到端收益幅度需结合具体场景解读。

ActiveGS: Active Scene Reconstruction using Gaussian Splatting Figure 1
IEEE RA-L 20252024-12-23

ActiveGS: Active Scene Reconstruction using Gaussian Splatting

Liren Jin, Xingguang Zhong, Yue Pan, Jens Behley, Cyrill Stachniss

Center for Robotics, University of Bonn, Bonn, Germany, Center for Robotics, University of Bonn, Center for Robotics, University of Bonn, Germany, MAVLab, TU Delft, Delft, The Netherlands, MAVLab, The Netherlands, MAVLab, TU Delft, the Netherlands, the Netherlands, Delft University of Technology

网格重建机器人同步定位与建图

ActiveGS面向机器人在未知场景中用有限任务时间主动获取RGB-D视角、构建可用于下游任务的高保真地图。其关键做法是将Gaussian Splatting用于细节重建,并并行维护粗体素图来表达未知/空闲空间与支撑避障规划;同时用基于视角分布的高斯置信度识别重建不足表面,引导下一视角。实验显示其较NeRF式方法和GS基线获得更好重建质量,并在无人机实景中验证可用性。

SqueezeMe: Efficient Gaussian Avatars for VR Figure 1
arXiv preprint2024-12-19

SqueezeMe: Efficient Gaussian Avatars for VR

Shunsuke Saito, Stanislav Pidhorskyi, Igor Santesteban, Forrest Iandola, Divam Gupta, Anuj Pahuja, Nemanja Bartolovic, Frank Yu, Emanuel Garbin, Tomas Simon

数字人动态场景

SqueezeMe 面向独立 VR 头显上同时驱动高保真全身 Gaussian 数字人的算力与显存瓶颈,指出逐帧神经解码姿态相关校正是主要开销。方法将高容量 CNN 学到的校正蒸馏为从姿态到 Gaussian 位移、旋转、尺度和外观参数的线性映射,并在 UV 邻域共享低频校正,配合 Vulkan 移动端 splatting 渲染。实验显示解码延迟由 50 ms 降至 0.45 ms,可在 Quest 3 上以 72 FPS 同时运行动画与渲染 3 个头像,质量损失较小。

SolidGS: Consolidating Gaussian Surfel Splatting for Sparse-View Surface Reconstruction Figure 1
arXiv preprint2024-12-19

SolidGS: Consolidating Gaussian Surfel Splatting for Sparse-View Surface Reconstruction

Zhuowen Shen, Yuan Liu, Zhang Chen, Zhong Li, Jiepeng Wang, Yongqing Liang, Zhengming Yu, Jingdong Zhang, Yi Xu, Scott Schaefer, Xin Li, Wenping Wang

Texas A&M University, OPPO US Research Center, Nanyang Technological University, Hong Kong University of Science and Technology, The University of Hong Kong

网格重建稀疏表示

SolidGS面向机器人/AR等常见的稀疏视角场景,指出现有高斯表面重建在alpha混合下会因高斯尾部导致多视角深度不一致、监督不足而产生噪声网格。其核心是在高斯核中引入共享可学习的“solidness”因子,使表示从普通高斯逐步收敛为更实的surfel,并结合虚拟视角几何自监督和单目法线约束。实验在DTU、Tanks-and-Temples和LLFF上显示,仅3张RGB图约3分钟即可获得优于高斯和神经场基线的稀疏视角表面重建与新视角合成结果。

EnvGS: Modeling View-Dependent Appearance with Environment Gaussian Figure 1
CVPR 20252024-12-19

EnvGS: Modeling View-Dependent Appearance with Environment Gaussian

Tao Xie, Xi Chen, Zhen Xu, Yiman Xie, Yudong Jin, Yujun Shen, Sida Peng, Hujun Bao, Xiaowei Zhou

Zhejiang University

光线追踪渲染

EnvGS针对3DGS及环境贴图方法难以重建近场、高频镜面反射的问题,将场景拆为基础2D Gaussian与显式环境Gaussian,并沿反射方向用基于CUDA/OptiX和RT Core的可微光线追踪器渲染反射后再混合颜色。实验在真实与合成数据上显示其能生成更细致反射,并在实时新视角合成中取得领先画质。

Turbo-GS: Accelerating 3D Gaussian Fitting for High-Quality Radiance Fields Figure 1
arXiv preprint2024-12-18

Turbo-GS: Accelerating 3D Gaussian Fitting for High-Quality Radiance Fields

R Venkatesh

Indian Institute of Science, Bangalore 2Brown University, Indian Institute of Science, Bangalore 2Brown University, Cornell University, Samsung R&D Institute India - Bangalore, K images in minutes, significantly outperforming baseline methods. Notably, significantly outperforming baseline methods. Notably

加速训练密度控制

Turbo-GS针对高分辨率3DGS逐场景拟合耗时过长的问题,指出密集像素监督在tile渲染下存在冗余,改为扩张渲染只优化子采样像素,并用收敛感知预算控制与位置-外观联合致密化提升高斯添加效率、缓解弱纹理梯度消失。实验显示其可将4K场景重建从数小时压到约10分钟,同时保持或提升新视角渲染质量。

GAGS: Granularity-Aware Feature Distillation for Language Gaussian Splatting Figure 1
Proceedings of the AAAI Conference on Artificial Intelligence2024-12-18

GAGS: Granularity-Aware Feature Distillation for Language Gaussian Splatting

Yuning Peng, Haiping Wang, Yuan Liu, Chenglu Wen, Zhen Dong, Bisheng Yang

Wuhan University, Hong Kong University of Science and Technology, Xiamen University

语言嵌入分割

GAGS面向3D开放词汇场景理解中2D CLIP特征蒸馏到3D Gaussian时的多视角语义粒度不一致问题。其核心是用相机距离自适应调节SAM提示点密度,并学习无监督粒度因子,在蒸馏中选择更一致的子部件/部件/物体级特征。实验在LERF和Mip-NeRF-360标注集上显示,其在视觉定位与语义分割上更稳定、更准确,查询速度约为基线2倍。

Real-time Free-view Human Rendering from Sparse-view RGB Videos using Double Unprojected Textures Figure 1
CVPR 20252024-12-17

Real-time Free-view Human Rendering from Sparse-view RGB Videos using Double Unprojected Textures

Guoxing Sun, Rishabh Dabral, Heming Zhu, Pascal Fua, Christian Theobalt, Marc Habermann

Saarland Informatics Campus, Max Planck Institute for Informatics, Max Planck Institute for Informatics, EPFL

数字人稀疏表示纹理建模

面向稀疏 RGB 相机下实时自由视角数字人渲染,论文指出现有纹理空间方法常把几何与外观耦合,或未利用图像线索估计几何,导致姿态泛化和细节不稳。DUT 通过两次反投影纹理先做图像条件模板形变,再基于更新几何预测 Gaussian 外观,并用尺度细化缓解形变影响;实验显示其在多个基准上优于既有方法,可在单 GPU 实时生成 4K 结果。

Gaussian Billboards: Expressive 2D Gaussian Splatting with Textures Figure 1
arXiv preprint2024-12-17

Gaussian Billboards: Expressive 2D Gaussian Splatting with Textures

Sebastian Weiss, Derek Bradley

二维高斯纹理建模

这篇论文针对2D Gaussian Splatting每个splat仅用单一颜色、难以表达墙面图案等高频纹理的问题,指出2DGS与传统billboard同属三维空间中的半透明二维片元,并将二者结合:为每个2D高斯引入小型可优化纹理,通过UV双线性插值产生空间变化颜色。实验与消融显示,在固定或自适应primitive数量下,该方法较原始2DGS提升重建清晰度和新视角合成质量,但纹理分辨率受CUDA共享内存限制。

GaussTR: Foundation Model-Aligned Gaussian Transformer for Self-Supervised 3D Spatial Understanding Figure 1
CVPR 20252024-12-17

GaussTR: Foundation Model-Aligned Gaussian Transformer for Self-Supervised 3D Spatial Understanding

Haoyi Jiang, Liu Liu, Tianheng Cheng, Xinjie Wang, Tianwei Lin, Zhizhong Su, Wenyu Liu, Xinggang Wang

Huazhong University of Science & Technology, Huazhong University of Science and Technology

三维高斯泼溅

针对3D语义占据依赖密集体素标注、计算开销大且泛化受限的问题,GaussTR用Transformer前馈预测稀疏3D高斯表示,并通过可微splatting回投到2D与视觉/视觉语言基础模型特征对齐,实现自监督、开放词表占据预测。其在Occ3D-nuScenes零样本达到12.27 mIoU,较既有方法提升约23%,训练时间减少40%。

3DGUT: Enabling Distorted Cameras and Secondary Rays in Gaussian Splatting Figure 1
CVPR 20252024-12-17

3DGUT: Enabling Distorted Cameras and Secondary Rays in Gaussian Splatting

Qi Wu, Janick Martinez Esturo, Ashkan Mirzaei, Nicolas Moenne-Loccoz, Zan Gojcic

NVIDIA

三维高斯泼溅

3DGUT针对传统3DGS依赖EWA光栅化、难以处理畸变相机和滚动快门且不支持二次光线的问题,用无迹变换以sigma点近似高斯粒子并精确通过任意非线性投影,同时将渲染形式对齐到可光追的粒子评估。实验显示其在保持接近3DGS速度和画质的同时,更好适配强畸变相机,并可在同一表示中支持反射、折射等效果。

Wonderland: Navigating 3D Scenes from a Single Image Figure 1
CVPR 20252024-12-16

Wonderland: Navigating 3D Scenes from a Single Image

Hanwen Liang, Junli Cao, Vidit Goel, Guocheng Qian, Sergei Korolev, Demetri Terzopoulos, Konstantinos N. Plataniotis, Sergey Tulyakov, Jian Ren

University of Toronto, University of California, Los Angeles, University of California

前馈重建稀疏表示世界生成

Wonderland针对单图生成大范围3D场景中多视角需求、逐场景优化慢、遮挡几何失真和背景质量差的问题,利用可相机轨迹控制的视频扩散模型提供具3D一致性的潜空间,再以前馈LaLRM从视频latent直接回归3D Gaussian Splatting,并用渐进训练降低内存与计算。实验显示其在RealEstate10K、DL3DV、Tanks-and-Temples等零样本新视角/单视图场景生成上优于既有方法,域外图像表现尤其突出。

PanSplat: 4K Panorama Synthesis with Feed-Forward Gaussian Splatting Figure 1
CVPR 20252024-12-16

PanSplat: 4K Panorama Synthesis with Feed-Forward Gaussian Splatting

Cheng Zhang, Haofei Xu, Qianyi Wu, Camilo Cruz Gambardella, Dinh Phung, Jianfei Cai

Monash University, Australian Regenerative Medicine Institute, ETH Zurich

全景重建前馈重建世界生成

PanSplat面向VR、虚拟游览等对4K全景新视角合成的需求,解决现有方法受球面几何、显存与计算限制而停留在低分辨率的问题。其核心是用Fibonacci球面采样构建多尺度3D Gaussian金字塔,减少极区冗余,并结合层级球面代价体、局部Gaussian head与延迟反传实现单A100上的4K训练。实验显示其在合成和真实数据上达到SOTA画质,并相较NeRF式SOTA最高约70倍推理加速。

MASt3R-SLAM: Real-Time Dense SLAM with 3D Reconstruction Priors Figure 1
CVPR 20252024-12-16

MASt3R-SLAM: Real-Time Dense SLAM with 3D Reconstruction Priors

Riku Murai, Eric Dexheimer, Andrew J. Davison

Imperial College London

三维重建同步定位与建图

针对单目 SLAM 依赖标定、难以在野外视频中同时获得准确位姿和一致稠密地图的问题,MASt3R-SLAM 将两视图 3D 重建先验作为跟踪、建图和重定位的统一基础,并设计点图匹配、局部融合、回环与二阶全局优化等实时机制。在仅假设中心相机、可处理时变相机模型的条件下,系统约 15 FPS 运行;已知标定时在轨迹精度和稠密几何上达到 SOTA。

Deformable Radial Kernel Splatting Figure 1
CVPR 20252024-12-16

Deformable Radial Kernel Splatting

Yi-Hua Huang, Ming-Xian Lin, Yang-Tian Sun, Ziyi Yang, Xiaoyang Lyu, Yan-Pei Cao, Xiaojuan Qi

The University of Hong Kong, University of Hong Kong

优化方法渲染

这篇论文针对 3D Gaussian Splatting 用径向对称、平滑高斯难以表示尖锐边界和非高斯形状、因而需要大量 primitive 的问题,提出可变形径向核 DRK:用可学习角度/尺度的多径向基、L1/L2 混合距离和分段重映射控制形状与锐度,并配套平面核求交、排序和裁剪。实验在 DiverseScene 与 Mip-NeRF 360 上显示其以更少 primitive 获得更高渲染质量和效率。

GaussianProperty: Integrating Physical Properties to 3D Gaussians with LMMs Figure 1
ICCV 20252024-12-15

GaussianProperty: Integrating Physical Properties to 3D Gaussians with LMMs

Xinli Xu, Wenhang Ge, Dicong Qiu, ZhiFei Chen, Dongyu Yan, Zhuoyun Liu, Haoyu Zhao, Hanfeng Zhao, Shunsi Zhang, Junwei Liang, Ying-Cong Chen

Quincy University

语言嵌入机器人

面向机器人抓取、仿真和 AR 中难以从视觉直接标注的材料物性估计,GaussianProperty 将 SAM 的局部分割与 GPT-4V 的开放世界识别结合,在多视角 2D 图像上推理密度、摩擦、硬度等属性,并通过投票投射到 3D Gaussians。论文展示这些物性标注可驱动 MPM 动态仿真和自适应抓取力预测,在材料分割与真实抓取实验中优于固定力基线,但对材质外观歧义仍有限。

SuperGSeg: Open-Vocabulary 3D Segmentation with Structured Super-Gaussians Figure 1
3DV 20262024-12-13

SuperGSeg: Open-Vocabulary 3D Segmentation with Structured Super-Gaussians

Siyun Liang, Sen Wang, Kunyi Li, Michael Niemeyer, Stefano Gasperini, Hendrik P.A. Lensch, Nassir Navab, Federico Tombari

Technical University of Munich, Google, Google (United States), University of Tubingen, University of Tübingen

语言嵌入分割

SuperGSeg针对3DGS开放词汇分割中逐高斯存储高维语言特征显存过高、压缩又损失语义且难处理遮挡的问题,将分割学习与语言蒸馏解耦:先用多视图图像和2D掩码学习几何、实例与层级特征,再通过在线聚类形成稀疏Super-Gaussian,并在其上蒸馏CLIP特征。实验在LERF-OVS和ScanNet上显示,其在开放词汇物体检索与语义分割中获得更完整一致的掩码和较低显存开销。

SplineGS: Robust Motion-Adaptive Spline for Real-Time Dynamic 3D Gaussians from Monocular Video Figure 1
CVPR 20252024-12-13

SplineGS: Robust Motion-Adaptive Spline for Real-Time Dynamic 3D Gaussians from Monocular Video

Jongmin Park, Minh-Quan Viet Bui, Juan Luis Gonzalez Bello, Jaeho Moon, Jihyong Oh, Munchurl Kim

Korea Advanced Institute of Science and Technology (KAIST), Korea Advanced Institute of Science and Technology, Chung-Ang University

动态场景单目重建

针对野外单目视频中动态物体难建模、缺少多视角约束且 COLMAP 相机位姿不可靠的问题,SplineGS 用三次 Hermite 样条为每个动态 3D Gaussian 表示连续轨迹,并通过运动自适应控制点剪枝按复杂度分配自由度,同时联合优化相机参数与高斯属性。实验显示其在动态新视角合成质量上优于现有方法,并实现实时、较基线快数千倍的渲染。

SimAvatar: Simulation-Ready Avatars with Layered Hair and Clothing Figure 1
CVPR 20252024-12-12

SimAvatar: Simulation-Ready Avatars with Layered Hair and Clothing

Xueting Li, Ye Yuan, Shalini De Mello, Gilles Daviet, Jonathan Leaf, Miles Macklin, Jan Kautz, Umar Iqbal

NVIDIA

数字人扩散生成语言嵌入

SimAvatar针对现有文本生成数字人把身体、衣物和头发纠缠为单一几何,难以进行真实布料/头发仿真的问题,采用分层表示:SMPL身体、可仿真衣物网格和发丝先由文本条件模型生成,再绑定3D Gaussian并用扩散先验优化外观。其结果是在保持高保真纹理的同时,可由物理或神经模拟器驱动衣物和头发产生姿态相关动态,动画真实感优于现有方法。

SLAM3R: Real-Time Dense Scene Reconstruction from Monocular RGB Videos Figure 1
CVPR 20252024-12-12

SLAM3R: Real-Time Dense Scene Reconstruction from Monocular RGB Videos

Yuzheng Liu, Siyan Dong, Shuzhe Wang, Yingda Yin, Yanchao Yang, Qingnan Fan, Baoquan Chen

Peking University, The University of Hong Kong, University of Hong Kong, Aalto University

三维重建同步定位与建图

针对单目 RGB 稠密 SLAM 难以同时兼顾重建质量、完整性与实时性的矛盾,SLAM3R 将视频切成重叠窗口,用 I2P 直接回归局部稠密点图,再由 L2W 前馈网络把局部点图增量对齐到全局坐标,绕开显式相机位姿求解和全局优化。多数据集实验显示其在精度与完整性上达到或超过现有方法,并保持 20+ FPS 实时运行。

Feat2GS: Probing Visual Foundation Models with Gaussian Splatting Figure 1
CVPR 20252024-12-12

Feat2GS: Probing Visual Foundation Models with Gaussian Splatting

Yue Chen, Xingyu Chen, Anpei Chen, Gerard Pons-Moll, Yuliang Xiu

Zhejiang University, Westlake University, University of Tübingen, Tübingen AI Center, University of Tübingen, Tübingen AI Center, TH Bingen University of Applied Sciences

渲染世界生成

这篇论文针对现有VFM 3D探测过度依赖深度/法线或稀疏匹配、难以评估纹理与稠密多视一致性的问题,提出用轻量读出层从冻结视觉特征预测3D Gaussian参数,并通过新视角合成分别探测几何与纹理意识。实验比较多类VFM,发现不同训练目标在几何和纹理上存在取舍;基于特征拼接的Feat2GS变体在多数据集NVS上超过InstantSplat,说明该框架既可作诊断工具也可作简单基线。

ResGS: Residual Densification of 3D Gaussian for Efficient Detail Recovery Figure 1
ICCV 20252024-12-10

ResGS: Residual Densification of 3D Gaussian for Efficient Detail Recovery

Yanzhe Lyu, Kai Cheng, Xin Kang, Xuejin Chen

University of Science and Technology of China, MoE Key Laboratory of Brain-inspired Intelligent Perception and Cognition, University of Science and Technology of China, MoE Key Laboratory of Brain-inspired Intelligent Perception and Cognition

密度控制

ResGS针对3D Gaussian Splatting中固定阈值决定split/clone导致的细节模糊、几何缺失与冗余小高斯问题,提出“残差分裂”:为欠优化高斯生成缩小副本并降低原高斯不透明度,同时用图像金字塔监督和渐进高斯选择实现粗到细训练。实验在Mip-NeRF360、Tanks&Temples、DeepBlending上达到SOTA渲染质量,并可稳定提升多种3D-GS变体。

ReCap: Better Gaussian Relighting with Cross-Environment Captures Figure 1
CVPR 20252024-12-10

ReCap: Better Gaussian Relighting with Cross-Environment Captures

Jingzhi Li, Zongwei Wu, Eduard Zamfir, Radu Timofte

Computer Vision Lab, CAIDAS & IFI, University of Würzburg, Germany, Computer Vision Lab, University of Würzburg

重光照

ReCap针对3D Gaussian重光照中反照率与光照难分离、学习环境图缺乏物理对应的问题,利用同一物体在未知多环境下的采集作为多任务监督,共享材质并分别优化光照表示,配合简化的split-sum着色与HDR后处理,使标准HDR图可直接用于重光照;在扩展的漫反射/高光物体基准上优于现有方法,并显示更多训练环境可持续提升效果。

Faster and Better 3D Splatting via Group Training Figure 1
ICCV 20252024-12-10

Faster and Better 3D Splatting via Group Training

Chengbo Wang, Guozheng Ma, Yifei Xue, Yizhen Lao

School of Design, Hunan University, School of Design, Hunan University, Nanyang Technological University

加速训练密度控制优化方法

针对 3DGS 训练中高斯数量膨胀带来的计算瓶颈,论文提出 Group Training:周期性将高斯划分为训练组与缓存组,缓存而非直接剪枝,并用基于不透明度的优先采样保留更关键、渲染更高效的高斯。该插件可接入 3DGS 与 Mip-Splatting,在多场景中最高约 30% 加速收敛,同时提升 PSNR 并减少浮游伪影。

Diffusion-Based Attention Warping for Consistent 3D Scene Editing Figure 1
arXiv preprint2024-12-10

Diffusion-Based Attention Warping for Consistent 3D Scene Editing

Eyal Gomel, Lior Wolf

Tel-Aviv University

扩散生成风格迁移

这篇论文针对2D扩散编辑迁移到3D场景时多视角不一致、同时编辑多视图又易损失细节且计算开销大的问题,提出从单个参考编辑中提取自/交叉注意力特征,并借助Gaussian Splatting的深度、法向与遮挡掩码将其几何对齐地扭曲到其他视角。实验显示该方法在编辑保真度、空间一致性和语义保持上优于多种3D编辑基线,同时降低多视图联合处理需求。

MV-DUSt3R+: Single-Stage Scene Reconstruction from Sparse Views In 2 Seconds Figure 1
CVPR 20252024-12-09

MV-DUSt3R+: Single-Stage Scene Reconstruction from Sparse Views In 2 Seconds

Zhenggang Tang, Yuchen Fan, Dilin Wang, Hongyu Xu, Rakesh Ranjan, Alexander Schwing, Zhicheng Yan

Meta Reality Labs, META Health, University of Illinois Urbana-Champaign

三维重建稀疏表示

针对 DUSt3R 类方法需两两重建再全局优化、在多稀疏视角场景中易累积错配且耗时的问题,MV-DUSt3R+将多视图信息在单次前向中融合,并用多参考视角交叉注意力降低参考视角选择敏感性,还接入 Gaussian splatting 支持新视角合成。其在 HM3D、ScanNet、MP3D 上提升重建、位姿估计和渲染质量,推理较 DUSt3R 快约一个数量级以上。

MAtCha Gaussians: Atlas of Charts for High-Quality Geometry and Photorealism From Sparse Views Figure 1
CVPR 20252024-12-09

MAtCha Gaussians: Atlas of Charts for High-Quality Geometry and Photorealism From Sparse Views

Antoine Guédon, Tomoki Ichikawa, Kohei Yamashita, Ko Nishino

Centre National de la Recherche Scientifique, Laboratoire d'Informatique Gaspard-Monge, Graduate School of Informatics, Kyoto University, Japan, Graduate School of Informatics, Kyoto University

网格重建稀疏表示

针对 NeRF/3DGS 虽能逼真渲染但几何模糊、且常需密集视角的问题,MAtCha 将场景表面表示为图册化的 2D charts,并在其上动态附着 2D Gaussian surfels。方法用单目深度蒸馏高频细节,再通过轻量神经形变和结构损失校正尺度歧义并保留法向/曲率。实验显示其在少量 RGB 视角下数分钟内重建清晰网格,同时新视角渲染质量接近强基线。

Deblur4DGS: 4D Gaussian Splatting from Blurry Monocular Video Figure 1
arXiv preprint2024-12-09

Deblur4DGS: 4D Gaussian Splatting from Blurry Monocular Video

Renlong Wu, Zhilu Zhang, Mingyang Chen, Xiaopeng Fan, Zifei Yan, Wangmeng Zuo

Harbin Institute of Technology

去模糊

该文针对单目动态视频常因相机抖动和物体运动产生模糊、导致现有4D重建监督失效的问题,提出 Deblur4DGS。其核心是用4D Gaussian Splatting的显式运动轨迹,将曝光期内连续动态表示估计转化为曝光时间估计,并加入曝光、多帧/多分辨率一致性正则及模糊感知可变 canonical Gaussians,以处理欠约束和大运动。合成与真实实验显示其在新视角合成、去模糊、插帧和稳像上优于现有4D重建方法。

4D Gaussian Splatting with Scale-aware Residual Field and Adaptive Optimization for Real-time Rendering of Temporally Complex Dynamic Scenes Figure 1
arXiv preprint2024-12-09

4D Gaussian Splatting with Scale-aware Residual Field and Adaptive Optimization for Real-time Rendering of Temporally Complex Dynamic Scenes

Jinbo Yan, Rui Peng, Luyang Tang, Ronggang Wang

School of Electronic and Computer Engineering, Peking University, Shenzhen, China, School of Electronic and Computer Engineering, Peking University, School of Electronic and Computer Engineering, Peking University & Pengcheng Laboratory, Shenzhen, China, Peking University & Pengcheng Laboratory, Peng Cheng Laboratory

密度控制动态场景优化方法

面向动态场景重建中渲染慢、难处理大幅运动及物体出现/消失的问题,SaRO-GS 将高斯基元放在 4D 空间优化,并用尺度感知残差场编码高斯占据区域而非单点特征,再按时间属性自适应调整优化策略。实验覆盖单目和多视角数据,显示其在保持实时渲染的同时达到 SOTA 重建质量,并可基于寿命实现动静分割。

Mirror-3DGS: Incorporating Mirror Reflections into 3D Gaussian Splatting Figure 1
arXiv preprint2024-12-08

Mirror-3DGS: Incorporating Mirror Reflections into 3D Gaussian Splatting

Jiarui Meng, Haijie Li, Yanmin Wu, Qiankun Gao, Shuzhou Yang, Jian Zhang, Siwei Ma

Peking University, School of Electronic and Computer Engineering, China, Peking University, School of Electronic and Computer Engineering, Peking University, School of Computer Science, China, School of Computer Science

渲染

这篇论文针对标准 3DGS 会把镜中虚像当作真实几何重建、导致镜面区域不一致的问题,提出为高斯引入镜面属性,并利用平面镜成像从“镜后视点”联合渲染真实视角与镜像视角。方法通过两阶段训练估计镜面平面,配合平面一致性和深度约束稳定优化。在合成与真实镜面场景中,Mirror-3DGS 相比 Mirror-NeRF 保持或提升镜面区域保真度,同时显著缩短训练并支持实时渲染。

Temporally Compressed 3D Gaussian Splatting for Dynamic Scenes Figure 1
arXiv preprint2024-12-07

Temporally Compressed 3D Gaussian Splatting for Dynamic Scenes

Ahmad Jarrar

CVLab, EPFL, CVLab, EPFL, Swiss Data Science Center

加速训练压缩动态场景

面向 AR/VR、游戏和机器人中动态场景实时渲染,TC3DGS针对 Dynamic 3DGS 随序列长度线性膨胀的存储与渲染成本,利用时间相关性做高斯裁剪,并以梯度敏感度进行混合精度量化,再用改造的 RDP 算法压缩跨帧轨迹。多数据集结果显示其最高可达 67× 压缩,视觉质量几乎不降。

HAHA: Highly Articulated Gaussian Human Avatars with Textured Mesh Prior Figure 1
arXiv preprint2024-12-07

HAHA: Highly Articulated Gaussian Human Avatars with Textured Mesh Prior

David Svitov, Pietro Morerio, Lourdes Agapito, Alessio Del Bue

Italian Institute of Technology, University of Genoa, Department of Computer Science, University College London, London, UK, Department of Computer Science, University College London

数字人

HAHA针对单目视频数字人中纯3D Gaussian表示高斯数量大、手指等高关节部位难以高效动画化的问题,将带纹理的SMPL-X网格作为主体先验,只在头发、宽松衣物等离网格细节处保留高斯,并用无监督透明度正则学习删减。在SnapshotPeople上以不足三分之一高斯达到接近SOTA重建质量,在X-Humans新姿态上定量和视觉均优于既有方法。

Momentum-GS: Momentum Gaussian Self-Distillation for High-Quality Large Scene Reconstruction Figure 1
ICCV 20252024-12-06

Momentum-GS: Momentum Gaussian Self-Distillation for High-Quality Large Scene Reconstruction

Jixuan Fan, Wanhua Li, Yifei Han, Tianru Dai, Yansong Tang

Tsinghua Shenzhen International Graduate School, Tsinghua–Berkeley Shenzhen Institute, Harvard University, Harvard University Press

大规模场景

Momentum-GS针对大规模3D高斯重建中显存/存储开销高、分块独立训练导致跨块不一致且块数受GPU数量限制的问题,引入动量更新的教师Gaussian decoder进行自蒸馏,为各块提供全局一致性约束,并用重建质量自适应调整块权重。实验在多个大场景上优于现有方法,相比CityGaussian以更少分块获得18.7%的LPIPS提升。

Sparse Voxels Rasterization: Real-time High-fidelity Radiance Field Rendering Figure 1
CVPR 20252024-12-05

Sparse Voxels Rasterization: Real-time High-fidelity Radiance Field Rendering

Cheng Sun, Jaesung Choe, Charles Loop, Wei-Chiu Ma, Yu-Chiang Frank Wang

NVIDIA, Cornell University

渲染稀疏表示

针对 3D Gaussian Splatting 深度排序不准导致 popping、体密度定义不清,而传统网格射线投射又较慢的问题,SVRaster 回到体素表示:用多层自适应稀疏体素显式分配细节,并设计随视线方向变化的 Morton 排序光栅化器以保证深度顺序。结果在无神经网络/无高斯的设定下,相比既有无神经体素方法提升超 4dB PSNR、渲染速度超 10 倍,并保持与 SOTA 相近的新视角合成质量。

Pixel-GS: Density Control with Pixel-aware Gradient for 3D Gaussian Splatting Figure 1
arXiv preprint2024-12-05

Pixel-GS: Density Control with Pixel-aware Gradient for 3D Gaussian Splatting

Zheng Zhang, Wenbo Hu, Yixing Lao, Tong He, Hengshuang Zhao

The University of Hong Kong, Pok Fu Lam, Hong Kong, The University of Hong Kong, University of Hong Kong, Tencent AI Lab, Shenzhen, China, Tencent AI Lab, Tencent (China), Shanghai AI Lab, Shanghai, China, Shanghai AI Lab, Shanghai Artificial Intelligence Laboratory

密度控制渲染

Pixel-GS针对3DGS在SfM初始点稀疏区域难以增密、导致模糊和针状伪影的问题,指出原有按可见视角等权平均梯度会低估大高斯的分裂/克隆需求。方法改用高斯在各视角覆盖像素数加权梯度,并按相机距离缩放梯度以抑制近相机floaters。实验在Mip-NeRF 360与Tanks & Temples上提升LPIPS等渲染质量,保持实时速度且对初始点稀疏更鲁棒。

PlanarSplatting: Accurate Planar Surface Reconstruction in 3 Minutes Figure 1
CVPR 20252024-12-04

PlanarSplatting: Accurate Planar Surface Reconstruction in 3 Minutes

Bin Tan, Rui Yu, Yujun Shen, Nan Xue

University of Louisville, University of Louisville Hospital

加速训练渲染

针对室内多视图平面重建依赖2D/3D平面检测、匹配与跟踪且结果粗糙的问题,PlanarSplatting直接优化3D矩形平面基元,通过可微平面splatting渲染到2.5D深度/法线图,并借助CUDA实现加速。方法在ScanNet与ScanNet++数百场景上约3分钟完成单场景重建,几何精度优于基线;与3DGS/2DGS结合还可减少训练时间并提升室内新视角渲染质量。

NeRF and Gaussian Splatting SLAM in the Wild Figure 1
arXiv preprint2024-12-04

NeRF and Gaussian Splatting SLAM in the Wild

Fabian Schmidt, Markus Enzweiler, Abhinav Valada

野外场景综述同步定位与建图

针对NeRF/3DGS等神经SLAM多在室内评测、难以判断其野外适用性的问题,论文在ROVER自然户外数据上比较传统、深度学习、NeRF与3DGS SLAM,关注跟踪精度、环境鲁棒性和计算代价。结果显示神经方法在低照等困难条件下更稳但开销高,传统方法跨季节表现较好却对光照变化敏感。

GaussCtrl: Multi-View Consistent Text-Driven 3D Gaussian Splatting Editing Figure 1
arXiv preprint2024-12-04

GaussCtrl: Multi-View Consistent Text-Driven 3D Gaussian Splatting Editing

Jing Wu, Jia-Wang Bian, Xinghui Li, Guangrun Wang, Ian Reid, Philip Torr, Victor Adrian Prisacariu

University of Oxford, Oxford, England, University of Oxford, Mohamed bin Zayed University of Artificial Intelligence, Abu Dhabi, United Arab Emirates, Mohamed bin Zayed University of Artificial Intelligence

编辑

GaussCtrl针对文本驱动3DGS编辑中多视角不一致、迭代逐帧优化慢且易产生模糊伪影的问题,先渲染多视角图像与深度,再用ControlNet进行深度条件编辑,并通过参考视角的自/跨视角注意力对齐潜变量,使几何和外观同时保持一致。实验显示其在前向和360度场景上比既有方法更快,视觉质量与一致性更好。

Feed-Forward Bullet-Time Reconstruction of Dynamic Scenes from Monocular Videos Figure 1
arXiv preprint2024-12-04

Feed-Forward Bullet-Time Reconstruction of Dynamic Scenes from Monocular Videos

Hanxue Liang, Jiawei Ren, Ashkan Mirzaei, Antonio Torralba, Ziwei Liu, Igor Gilitschenski, Sanja Fidler, Cengiz Oztireli, Huan Ling, Zan Gojcic, Jiahui Huang

NVIDIA, 2University of Cambridge, 3Nanyang Technological University, NVIDIA, University of Cambridge, Nanyang Technological University, University of Toronto, 5MIT, 6Vector Institute, University of Toronto, Vector Institute

动态场景前馈重建单目重建

针对单目动态视频重建通常依赖逐场景优化、难以实时且泛化受限的问题,BTimer将“子弹时间”作为目标时间嵌入,让Transformer从多帧上下文直接聚合并预测该时刻的3D Gaussian场景;该表述统一静态/动态数据训练,并用NTE补强快速运动。实验显示其约150ms完成重建,渲染可实时,在静态和动态新视角合成基准上达到或超过多种优化式方法。

Taming 3DGS: High-Quality Radiance Fields with Limited Resources Figure 1
arXiv preprint2024-12-03

Taming 3DGS: High-Quality Radiance Fields with Limited Resources

Saswat Subhajyoti Mallick, Rahul Goel, Bernhard Kerbl, Markus Steinberger, Francisco Vicente Carrasco, Fernando De La Torre

Carnegie Mellon University, Pittsburgh, United States of America, Carnegie Mellon University, The International Institute of Information Technology, Hyderabad, India, The International Institute of Information Technology, International Institute of Information Technology, Hyderabad, International Institute of Information Technology, Graz University of Technology, Graz, Austria, Graz University of Technology

加速训练密度控制

该文针对原始 3DGS 高斯数量不可控、冗余多、训练和显存开销在受限设备上易失控的问题,提出按目标预算纯增量生长的密度控制:用跨视角损失、显著性和高斯属性打分来选择增密对象,并重写/近似部分梯度、属性更新与反传并行方式。实验显示在预算设置下模型规模和训练时间均降至约 1/4–1/5,质量接近或超过 3DGS;同等规模预算下质量进一步超过原版 3DGS。

Occam's LGS: A Simple Approach for Language Gaussian Splatting Figure 1
arXiv preprint2024-12-02

Occam's LGS: A Simple Approach for Language Gaussian Splatting

Jan-Nico Zaech, Luc Van Gool, Danda Pani Paudel

Johns Hopkins University, INSAIT, Sofia University, Sofia University

加速训练语言嵌入分割

这篇论文针对语言特征融入 3D Gaussian Splatting 时训练慢、需场景级压缩的问题,指出复杂优化管线并非必要;其核心是从 GS 前向渲染的概率形式出发,用加权多视角聚合直接把 2D 视觉语言特征提升到 3D,避免高维特征反复渲染与压缩。实验报告在保持或达到 SOTA 表现的同时提速约两个数量级,512 维特征可在一分钟内完成提升。

HDGS: Textured 2D Gaussian Splatting for Enhanced Scene Rendering Figure 1
arXiv preprint2024-12-02

HDGS: Textured 2D Gaussian Splatting for Enhanced Scene Rendering

Yunzhou Song, Heguang Lin, Jiahui Lei, Lingjie Liu, Kostas Daniilidis

University of Pennsylvania

二维高斯抗锯齿网格重建

HDGS针对2D Gaussian Splatting在任意视角/分辨率下近景纹理细节不足、远景或降采样易混叠的问题,将每个2D surfel与可优化纹理图对齐,并用逐射线深度排序减少popping、Fisher剪枝控制开销;同时以像素视锥采样替代中心射线采样来抗锯齿。实验显示其在标准基准和自建纹理丰富数据集上提升细节保留与抗锯齿效果,但逐射线排序和纹理索引带来更高计算成本。

DynSUP: Dynamic Gaussian Splatting from An Unposed Image Pair Figure 1
arXiv preprint2024-12-01

DynSUP: Dynamic Gaussian Splatting from An Unposed Image Pair

Weihang Li, Weirong Chen, Shenhan Qian, Jiajie Chen, Daniel Cremers, Haoang Li

Technical University of Munich, Munich Center for Machine Learning, The Hong Kong University of Science and Technology (Guangzhou)

动态场景位姿估计

DynSUP针对动态场景中新视角合成常依赖多视图、已知位姿和静态假设的问题,尝试仅用两张无位姿图像训练3D Gaussian。核心做法是把场景分解为分段刚体,通过对象级两视图BA联合估计相机位姿与物体运动,再用每个Gaussian可学习的SE(3)变换场细化运动。合成与真实实验显示其在动态、稀疏、无位姿设置下优于面向静态或已知位姿的基线,但对非刚体形变和初始运动分割较敏感。

Contrastive Gaussian Clustering: Weakly Supervised 3D Scene Segmentation Figure 1
arXiv preprint2024-12-01

Contrastive Gaussian Clustering: Weakly Supervised 3D Scene Segmentation

Myrna Castillo, Mahtab Dahaghin, Matteo Toso, Alessio Del Bue

Italian Institute of Technology

分割

针对3D场景标注稀缺且SAM等自动2D掩码跨视角不一致的问题,本文在3D Gaussian Splatting中为每个高斯学习分割特征,并用对比学习与空间正则直接从不一致2D掩码中约束多视角一致性;随后通过特征聚类实现3D分割、投影生成任意视角2D掩码。实验显示其相较NeRF和3DGS相关方法取得更高精度,掩码IoU较SOTA提升约8%,但训练时间约翻倍。

Speedy-Splat: Fast 3D Gaussian Splatting with Sparse Pixels and Sparse Primitives Figure 1
CVPR 20252024-11-30

Speedy-Splat: Fast 3D Gaussian Splatting with Sparse Pixels and Sparse Primitives

Alex Hanson, Allen Tu, Geng Lin, Vasu Singla, Matthias Zwicker, Tom Goldstein

University of Maryland, College Park, University of Maryland, College Park

加速训练稀疏表示

Speedy-Splat针对3D Gaussian Splatting在边缘设备、VR和多视角流媒体中仍受渲染速度与模型体积限制的问题,指出开销来自每个高斯覆盖的像素数和高斯总数。方法用SnugBox/AccuTile更精确定位tile交集,并将高效Soft/Hard剪枝嵌入训练。实验在三类数据集上平均渲染提速6.71倍、模型缩小10.6倍、训练加速约1.4倍,但画质略有下降。

T-3DGS: Removing Transient Objects for 3D Scene Reconstruction Figure 1
arXiv preprint2024-11-29

T-3DGS: Removing Transient Objects for 3D Scene Reconstruction

Vadim Pryadilshchikov, Alexander Markin, Artem Komarichev, Ruslan Rakhimov, Peter Wonka, Evgeny Burnaev

渲染

真实视频中的行人、车辆等瞬态/半瞬态物体会使 3DGS 重建产生模糊和漂浮伪影。T-3DGS 利用重建训练动态中的不确定性与语义差异做无监督瞬态检测,再结合分割模型和双向视频跟踪细化掩码,提升边界与时序一致性。在稀疏和密集采集数据及作者新数据集上,其重建质量优于现有鲁棒 NeRF/3DGS 方法,尤其能处理慢移动和半瞬态干扰。

GausSurf: Geometry-Guided 3D Gaussian Splatting for Surface Reconstruction Figure 1
arXiv preprint2024-11-29

GausSurf: Geometry-Guided 3D Gaussian Splatting for Surface Reconstruction

Jiepeng Wang, Yuan Liu, Peng Wang, Cheng Lin, Junhui Hou, Xin Li, Taku Komura, Wenping Wang

网格重建

GausSurf针对3D Gaussian Splatting虽训练/渲染快但表面重建细节和稳定性不足的问题,将场景区分为纹理丰富与弱纹理区域:前者在高斯优化中迭代引入patch-match MVS的深度/法线几何监督,后者使用预训练法线先验补足约束。DTU与Tanks and Temples实验显示,其在重建质量和耗时上优于现有GS表面重建方法,DTU单物体可在约10分钟内完成。

SuperGaussians: Enhancing Gaussian Splatting Using Primitives with Spatially Varying Colors Figure 1
arXiv preprint2024-11-28

SuperGaussians: Enhancing Gaussian Splatting Using Primitives with Spatially Varying Colors

Rui Xu, Wenyue Chen, Jiepeng Wang, Yuan Liu, Peng Wang, Lin Gao, Shiqing Xin, Taku Komura, Xin Li, Wenping Wang

渲染

这篇论文针对传统 Gaussian Splatting 中每个高斯只有单一颜色与不透明度、在复杂纹理场景下需堆叠大量小高斯的问题,提出 SVGS:让单个 2D Gaussian surfel 内部具备随空间位置变化的颜色和透明度,并比较双线性插值、可移动核与微型网络三种实现。实验显示三者均优于 2DGS 基线,其中可移动核在多个数据集的新视角合成上表现最好,同时基本保持几何重建质量。

SADG: Segment Any Dynamic Gaussians Without Object Trackers Figure 1
arXiv preprint2024-11-28

SADG: Segment Any Dynamic Gaussians Without Object Trackers

Yun-Jin Li, Mariia Gladkova, Yan Xia, Daniel Cremers

Munich Center for Machine Learning

动态场景分割

面向动态3D场景中跨时间、跨视角一致的对象分割,TRASE/SADG避免依赖易发生ID切换的视频跟踪器,而是在动态3D Gaussian上用SAM掩码引导的软挖掘对比学习训练32维语义特征,最后直接聚类得到对象。论文还构建动态新视角分割基准,在五类数据集上优于SA4D、DGD等方法,并支持点击/文本分割、移除、合成和风格迁移等编辑。

Gaussians-to-Life: Text-Driven Animation of 3D Gaussian Splatting Scenes Figure 1
3DV 20252024-11-28

Gaussians-to-Life: Text-Driven Animation of 3D Gaussian Splatting Scenes

Thomas Wimmer, Michael Oechsle, Michael Niemeyer, Federico Tombari

Technical University of Munich, Google, Google (United States)

扩散生成动态场景

这篇工作针对静态 3DGS 重建缺少动态表现的问题,提出用文本提示和目标框驱动局部场景动画。核心思路不是重新训练视频扩散模型,而是从多视角生成近似一致的视频引导,并结合深度估计与点跟踪把 2D 运动提升为 3D 高斯锚点轨迹,在尽量保持原场景外观的同时变形。实验在真实场景上优于改造的 DreamGaussian4D,且无需推理时优化,速度更快;但不能新增/删除高斯或显式处理碰撞。

AGS-Mesh: Adaptive Gaussian Splatting and Meshing with Geometric Priors for Indoor Room Reconstruction Using Smartphones Figure 1
arXiv preprint2024-11-28

AGS-Mesh: Adaptive Gaussian Splatting and Meshing with Geometric Priors for Indoor Room Reconstruction Using Smartphones

Xuqian Ren, Matias Turkulainen, Jiepeng Wang, Otto Seiskari, Iaroslav Melekhov, Juho Kannala, Esa Rahtu

网格重建

面向手机采集的室内房间重建,论文针对低分辨率 LiDAR 深度细节差、单目法线多视角不一致导致高斯重建几何不稳的问题,提出在优化中用深度-法线一致性过滤噪声深度,并自适应降低不可靠法线监督,同时加入尺度感知 TSDF/IsoOctree 网格提取。实验显示该策略可作为 2D/3D Gaussian Splatting 插件,提升网格质量和新视角合成效果。

Textured Gaussians for Enhanced 3D Scene Appearance Modeling Figure 1
CVPR 20252024-11-27

Textured Gaussians for Enhanced 3D Scene Appearance Modeling

Brian Chao, Hung-Yu Tseng, Lorenzo Porzi, Chen Gao, Tuotuo Li, Qinbo Li, Ayush Saraf, Jia-Bin Huang, Johannes Kopf, Gordon Wetzstein, Changil Kim

Stanford University, Meta

野外场景渲染纹理建模

针对 3DGS 中单个高斯只能表达近似单色椭球、难以刻画细纹理和复杂轮廓的问题,论文将传统纹理/Alpha 映射引入每个高斯,提出带 A、RGB 或 RGBA 贴图的 Textured Gaussians,并通过光线-高斯相交与 CUDA 纹理查询实现空间变化的颜色和透明度。实验显示其在物体与场景级数据上以相近或更少高斯提升渲染质量,低高斯数量时优势更明显。

CAT4D: Create Anything in 4D with Multi-View Video Diffusion Models Figure 1
CVPR 20252024-11-27

CAT4D: Create Anything in 4D with Multi-View Video Diffusion Models

Rundi Wu, Ruiqi Gao, Ben Poole, Alex Trevithick, Changxi Zheng, Jonathan T. Barron

Google DeepMind, Google (United States), Google DeepMind (United Kingdom), Columbia University

扩散生成动态场景

面向单目视频难以获取同步多视角、动态区域新视角伪影严重的问题,CAT4D用混合真实/合成数据训练多视角视频扩散先验,将单目视频扩展为可控相机位姿与时间的多视角视频,再优化可变形3D Gaussian实现4D重建。实验显示其在新视角合成和动态重建上接近依赖额外深度、轨迹等先验的SOTA,并能从真实或生成视频产生场景级4D内容;但运动物理准确性和时间外推仍有限,增益可能主要来自data/scaling。

SelfSplat: Pose-Free and 3D Prior-Free Generalizable 3D Gaussian Splatting Figure 1
CVPR 20252024-11-26

SelfSplat: Pose-Free and 3D Prior-Free Generalizable 3D Gaussian Splatting

Gyeongjin Kang, Jisang Yoo, Jihyeon Park, Seungtae Nam, Hyeonsoo Im, Sangheon Shin, Sangpil Kim, Eunbyung Park

Sungkyunkwan University, Department of Electrical and Computer Engineering, Sungkyunkwan University, Department of Electrical and Computer Engineering, Sungkyunkwan University, Department of Intelligent Robotics, Department of Intelligent Robotics, Yonsei University, Department of Artificial Intelligence, Yonsei University, Department of Artificial Intelligence, Korea University

前馈重建位姿估计

SelfSplat针对传统NeRF/3DGS依赖精确相机位姿、逐场景优化或预训练3D先验的问题,尝试从无位姿多视图/单目视频中前馈重建3D高斯。其核心是把自监督深度与位姿估计同显式3DGS联合训练,并加入匹配感知位姿网络和深度细化模块来缓解高斯错位与跨视图不一致。论文在RealEstate10K、ACID和DL3DV上报告外观、几何质量及跨数据集泛化优于既有方法。

SWinGS: Sliding Windows for Dynamic 3D Gaussian Splatting Figure 1
arXiv preprint2024-11-26

SWinGS: Sliding Windows for Dynamic 3D Gaussian Splatting

Richard Shaw, Michal Nazarczuk, Jifei Song, Arthur Moreau, Sibi Catley-Chandar, Helisa Dhamo, Eduardo Pérez-Pellitero

Huawei Noah’s Ark Lab, London, UK, Huawei Noah’s Ark Lab, Huawei Technologies (United Kingdom), Queen Mary University of London, London, UK, Queen Mary University of London

动态场景

SWinGS针对单一动态3DGS在长序列、大运动或拓扑变化中易模糊且时序不稳的问题,将视频按运动量自适应划分为重叠滑动窗口,每窗训练局部 canonical 3D Gaussian,并用可调MLP区分静态/动态运动模式,再通过跨窗自监督一致性微调抑制闪烁。实验显示其在通用动态场景上保持较高重建质量并支持实时交互查看。

Compact 3D Scene Representation via Self-Organizing Gaussian Grids Figure 1
arXiv preprint2024-11-26

Compact 3D Scene Representation via Self-Organizing Gaussian Grids

Wieland Morgenstern, Florian Barthel, Anna Hilsmann, Peter Eisert

Fraunhofer Heinrich Hertz Institute, HHI, Berlin, Germany, Fraunhofer Heinrich Hertz Institute, Fraunhofer Institute for Telecommunications, Heinrich Hertz Institute, Fraunhofer Institute for Telecommunications, Heinrich Hertz Institute, Humboldt University of Berlin, Berlin, Germany, Humboldt University of Berlin

压缩

针对 3D Gaussian Splatting 渲染快但模型动辄数百 MB、难以在端侧或 Web 部署的问题,论文利用高斯参数排列不唯一这一冗余,将无序高维参数通过 GPU 并行 PLAS 排成局部平滑的 2D 网格,并在训练中加入平滑约束,再用图像压缩编码。实验显示在基本保持视觉质量和训练时间的同时,复杂场景存储缩小约 17–42 倍,解压后仍兼容原 3DGS 渲染器。

Quark: Real-time, High-resolution, and General Neural View Synthesis Figure 1
ACM TOG 20242024-11-25

Quark: Real-time, High-resolution, and General Neural View Synthesis

John Flynn, Michael Broxton, Lukas Murmann, Lucy Chai, Matthew DuVall, Clément Godard, Kathryn Heal, Srinivas Kaza, Stephen Lombardi, Xuan Luo, Kira Prabhu, Tiancheng Sun, Lynn Tsai, Ryan Overbeck

Google, Mountain View, United States of America, Google, Google (United States), Google, Mountain View, CA, United States of America

前馈重建渲染

Quark针对现有新视角合成虽可实时渲染但重建仍慢的问题,提出纯前馈的重建-渲染一体化框架。其核心是为每个目标视角动态生成并丢弃分层深度图,在多尺度UNet式render-and-refine中迭代优化,并用One-to-many Transformer融合多输入视角以降低计算。实验显示其可在A100上以1080p、30fps从稀疏相机生成高质量结果,实时方法中达到SOTA,并在部分场景接近或超过离线优化方法。

Quadratic Gaussian Splatting for Efficient and Detailed Surface Reconstruction Figure 1
arXiv preprint2024-11-25

Quadratic Gaussian Splatting for Efficient and Detailed Surface Reconstruction

Ziyu Zhang, Binbin Huang, Hanqing Jiang, Liyang Zhou, Xiaojun Xiang, Shunhan Shen

Second-order Geometric Primitives, The University of Hong Kong, SenseTime Research

网格重建

针对3DGS/2DGS在表面重建中几何不准、平面 surfel 难以表达高曲率且易过平滑的问题,QGS将基元扩展为可变形二次曲面,并用测地距离定义表面感知密度,结合射线-二次曲面求交保持高效渲染。实验在DTU、Tanks and Temples和Mip-NeRF360上取得领先重建精度,DTU Chamfer误差较2DGS降33%、较GOF降27%,同时保持有竞争力的外观质量。

MAGiC-SLAM: Multi-Agent Gaussian Globally Consistent SLAM Figure 1
CVPR 20252024-11-25

MAGiC-SLAM: Multi-Agent Gaussian Globally Consistent SLAM

Vladimir Yugay, Theo Gevers, Martin R. Oswald

University of Amsterdam, Netherlands, University of Amsterdam, Netherlands

同步定位与建图

MAGiC-SLAM面向多机器人/多相机同时建图中现有NVS-SLAM速度慢、难以全局一致融合且多限于双智能体的问题,采用支持刚体变换的3D Gaussian子地图表示,结合基于基础视觉模型的回环检测、位姿图优化、地图校正与融合以及更稳健的高斯跟踪。实验在合成与真实数据上显示,其相较现有多智能体神经表示方法具有更高定位/重建精度和更快运行速度。

HO-Gaussian: Hybrid Optimization of 3D Gaussian Splatting for Urban Scenes Figure 1
arXiv preprint2024-11-25

HO-Gaussian: Hybrid Optimization of 3D Gaussian Splatting for Urban Scenes

Zhuopeng Li, Yilin Zhang, Chenming Wu, Jianke Zhu, Liangjun Zhang

Zhejiang University, Hangzhou, China, Zhejiang University, Baidu Research, Beijing, China, Baidu Research, Baidu (China)

密度控制其他

面向自动驾驶城市场景,传统 3DGS 依赖 SfM/SLAM 初始点云,难以覆盖远处、天空和低纹理区域且存储开销大。HO-Gaussian 用网格体与高斯联合优化学习高斯位置,并通过点增密、方向编码替代球谐、神经 warping 提升多相机一致性。实验表明其在常用多相机驾驶数据集上可实时生成较逼真的新视角图像。

GS2Mesh: Surface Reconstruction from Gaussian Splatting via Novel Stereo Views Figure 1
arXiv preprint2024-11-24

GS2Mesh: Surface Reconstruction from Gaussian Splatting via Novel Stereo Views

Yaniv Wolf, Ron Kimmel

Technion - Israel Institute of Technology, Haifa, Israel, Technion - Israel Institute of Technology, Technion – Israel Institute of Technology

二维高斯网格重建

GS2Mesh针对3DGS虽擅长新视角合成、但高斯位置受光度损失驱动而难以直接形成一致表面的矛盾,提出不从高斯属性取几何,而是渲染与训练位姿对应的虚拟双目图像,用预训练立体匹配模型估深并经TSDF融合成网格。该思路把真实世界几何先验注入3DGS后处理,在手机野外场景、Tanks and Temples和DTU上得到更平滑、更细致且更快的重建,并报告达到SOTA。

SplatSDF: Boosting Neural Implicit SDF via Gaussian Splatting Fusion Figure 1
arXiv preprint2024-11-23

SplatSDF: Boosting Neural Implicit SDF via Gaussian Splatting Fusion

Runfa Blark Li, Keito Suzuki, Bang Du, Ki Myung Brian Lee, Nikolay Atanasov, Truong Nguyen

网格重建

SDF-NeRF虽能同时支持逼真渲染和连续距离查询,但训练慢、易出现表面/空域歧义,限制机器人应用。SplatSDF的关键不是再加3DGS一致性损失,而是在训练架构中稀疏注入3D Gaussian属性嵌入来引导表面附近的SDF表示,推理时仍可去掉3DGS。实验显示其在相同几何精度下约快3倍收敛,并在Chamfer距离和PSNR上优于现有SDF-NeRF基线。

FATE: Full-head Gaussian Avatar with Textural Editing from Monocular Video Figure 1
CVPR 20252024-11-23

FATE: Full-head Gaussian Avatar with Textural Editing from Monocular Video

Jiawei Zhang, Zijian Wu, Zhiyang Liang, Yicheng Gong, Dongfang Hu, Yao Yao, Xun Cao, Hao Zhu

Nanjing University

数字人动态场景编辑单目重建纹理建模

FATE面向单目视频重建可驱动3D头部数字人时常见的后脑缺失、3DGS点冗余和纹理难编辑问题,提出基于采样的增密以控制高斯分布与规模,用神经烘焙把离散高斯映射为连续UV属性图,并借助生成式先验补全非正面外观。实验显示其在定量和视觉效果上优于既有方法,可生成可编辑、360°可渲染的全头头像。

HeadStudio: Text to Animatable Head Avatars with 3D Gaussian Splatting Figure 1
arXiv preprint2024-11-22

HeadStudio: Text to Animatable Head Avatars with 3D Gaussian Splatting

Zhenglin Zhou, Fan Ma, Hehe Fan, Zongxin Yang, Yi Yang

ReLER, CCAI, Zhejiang University, Hangzhou, China, Zhejiang University, State Key Laboratory of Brain-machine Intelligence, Zhejiang University, Hangzhou, China, State Key Laboratory of Brain-machine Intelligence

数字人

HeadStudio针对文本生成头部数字人中“静态质量高但难动画、可动画模型细节不足”的矛盾,将3D Gaussian Splatting绑定到FLAME等可动画头部先验上,并通过超密初始化、去噪SDS、面部关键点条件和自适应几何正则联合优化形状、纹理与表情驱动。实验显示其可在单张A6000约2小时训练后生成可由语音或视频驱动的高保真头像,并以1024分辨率实现约40fps实时渲染。

Per-Gaussian Embedding-Based Deformation for Deformable 3D Gaussian Splatting Figure 1
arXiv preprint2024-11-21

Per-Gaussian Embedding-Based Deformation for Deformable 3D Gaussian Splatting

Jeongmin Bae, Seoha Kim, Youngsik Yun, Hahyun Lee, Gun Bang, Youngjung Uh

Yonsei University, Seoul, 03722, Korea, Yonsei University, Electronics and Telecommunications Research Institute, Daejeon, 34129, Korea, Electronics and Telecommunications Research Institute

动态场景

本文针对动态 3DGS 中常用坐标式形变场难以刻画复杂运动的问题,指出高斯本身是多中心场的混合,不宜用单一坐标函数统一预测形变。方法为每个 Gaussian 分配可学习嵌入,并与时间嵌入共同驱动形变,同时将运动分解为粗、细两级并加入邻域平滑正则。实验显示其在动态区域细节、复杂相机设置下优于已有 deformable 3DGS,且保持较快渲染和较低模型容量。

On the Error Analysis of 3D Gaussian Splatting and an Optimal Projection Strategy Figure 1
arXiv preprint2024-11-21

On the Error Analysis of 3D Gaussian Splatting and an Optimal Projection Strategy

Letian Huang, Jiayang Bai, Jie Guo, Yuanqi Li, Yanwen Guo

Nanjing University, Nanjing, China, Nanjing University

渲染

针对3D Gaussian Splatting中常被忽视的投影近似误差,论文从投影函数一阶泰勒余项出发,分析误差与高斯均值位置及视场/焦距的关系,并据此提出沿相机中心到各高斯均值方向、在相应切平面上投影的Optimal Gaussian Splatting。实验显示该策略仅需小幅代码改动、不损害实时性,可减少宽视场和短焦场景中的拉伸、云雾状伪影与模糊,提升渲染真实感。

GRM: Large Gaussian Reconstruction Model for Efficient 3D Reconstruction and Generation Figure 1
arXiv preprint2024-11-21

GRM: Large Gaussian Reconstruction Model for Efficient 3D Reconstruction and Generation

Yinghao Xu, Zifan Shi, Wang Yifan, Hansheng Chen, Ceyuan Yang, Sida Peng, Yujun Shen, Gordon Wetzstein

Stanford University, Stanford, USA, Stanford University, Stanford, The Hong Kong University of Science and Technology, Sai Kung, Hong Kong, The Hong Kong University of Science and Technology, Hong Kong University of Science and Technology, University of Hong Kong, Shanghai AI Laboratory, Shanghai, China, Shanghai AI Laboratory, Shanghai Artificial Intelligence Laboratory, Zhejiang University, Hangzhou, China, Zhejiang University

稀疏表示

面向机器人、游戏等场景中高质量 3D 资产构建耗时、现有前馈方法受 triplane/体渲染效率限制的问题,GRM 用纯 Transformer 将稀疏多视图像素直接映射为像素对齐 3D Gaussians,并通过窗口注意力上采样保留细节。实验显示其在四视图重建中约 0.1 秒完成推理,质量和速度优于多类基线;结合多视图扩散模型后,也能提升文本/单图到 3D 生成效率与效果。

Analytic-Splatting: Anti-Aliased 3D Gaussian Splatting via Analytic Integration Figure 1
arXiv preprint2024-11-21

Analytic-Splatting: Anti-Aliased 3D Gaussian Splatting via Analytic Integration

Zhihao Liang, Qi Zhang, Wenbo Hu, Lei Zhu, Ying Feng, Kui Jia

South China University of Technology, Guangzhou, China, South China University of Technology, Tencent AI Lab, Shenzhen, China, Tencent AI Lab, Tencent (China), City University of Hong Kong, Hong Kong, China, City University of Hong Kong, School of Data Science, The Chinese University of Hong Kong, Shenzhen, China, School of Data Science, The Chinese University of Hong Kong, Chinese University of Hong Kong, Shenzhen, Chinese University of Hong Kong

抗锯齿渲染

这篇论文针对 3DGS 在缩放或分辨率变化时把像素当作单点采样、忽略像素面积而产生模糊和锯齿的问题,提出 Analytic-Splatting:用条件 logistic 近似高斯 CDF,通过解析积分估计像素窗口内的 2D 高斯响应,并将其用于体渲染透射率计算。实验显示其相比离散采样和低通预滤波方法能更好抗锯齿,同时保留更多细节和保真度。

latentSplat: Autoencoding Variational Gaussians for Fast Generalizable 3D Reconstruction Figure 1
arXiv preprint2024-11-20

latentSplat: Autoencoding Variational Gaussians for Fast Generalizable 3D Reconstruction

Christopher Wewer, Kevin Raj, Eddy Ilg, Bernt Schiele, Jan Eric Lenssen

Max Planck Institute for Informatics, Saarland Informatics Campus, Saarbrücken, Germany, Max Planck Institute for Informatics, Saarland University, Saarland Informatics Campus, Saarbrücken, Germany, Saarland University

前馈重建稀疏表示

面向少量视图的通用3D重建,现有回归方法高效但易在未观测区域模糊,生成方法能补全但扩展性差。latentSplat将VAE式不确定性建模引入3D高斯,在潜空间预测可变分语义高斯,经splatting和轻量2D生成解码器渲染。论文报告其在两视图重建、外推和高分辨率场景上优于近期方法,并支持较快三维一致视图与后续网格重建。

Revising Densification in Gaussian Splatting Figure 1
arXiv preprint2024-11-20

Revising Densification in Gaussian Splatting

Samuel Rota Bulò, Lorenzo Porzi, Peter Kontschieder

Meta Reality Labs, Zurich, Switzerland, Meta Reality Labs

密度控制

本文针对 3D Gaussian Splatting 中自适应密度控制依赖位置梯度、阈值不直观且易漏掉高频欠拟合区域的问题,改用逐像素重建误差回传到高斯基元来指导增密,并修正克隆时不透明度带来的合成偏置,同时加入基元数量上限控制。实验在 Mip-NeRF 360、Tanks and Temples、Deep Blending 等基准上相较 3DGS 和 Mip-Splatting 获得稳定质量提升,且基本不牺牲效率。

Generating 3D-Consistent Videos from Unposed Internet Photos Figure 1
CVPR 20252024-11-20

Generating 3D-Consistent Videos from Unposed Internet Photos

Gene Chou, Kai Zhang, Sai Bi, Hao Tan, Zexiang Xu, Fujun Luan, Bharath Hariharan, Noah Snavely

Cornell University, Adobe Research, Adobe Systems (United States)

前馈重建野外场景位姿估计

论文关注少量无位姿互联网照片能否生成真实相机运动视频,动机是检验视频模型对场景几何与身份的一致理解。核心做法是在视频扩散模型上联合多视角补全与视角插值,自监督学习3D感知而不依赖相机参数。实验显示其在几何和外观一致性上优于帧插值与商业视频模型,并可辅助3DGS等可控相机应用。

Gaussian Frosting: Editable Complex Radiance Fields with Real-Time Rendering Figure 1
arXiv preprint2024-11-20

Gaussian Frosting: Editable Complex Radiance Fields with Real-Time Rendering

Antoine Guédon, Vincent Lepetit

Centre National de la Recherche Scientifique, Laboratoire d'Informatique Gaspard-Monge

动态场景编辑网格重建

针对原始3DGS难以结构化编辑、SuGaR等网格化方法又会削弱毛发/草地等体积细节的问题,Gaussian Frosting在可编辑基网格外构建自适应厚度的高斯“糖霜”层,并约束高斯随网格变形自动更新。实验显示其能从RGB图像重建可动画化场景,实时渲染质量优于现有表面方法,并更好保留复杂模糊材质。

EAGLES: Efficient Accelerated 3D Gaussians with Lightweight EncodingS Figure 1
arXiv preprint2024-11-20

EAGLES: Efficient Accelerated 3D Gaussians with Lightweight EncodingS

Sharath Girish, Kamal Gupta, Abhinav Shrivastava

University of Maryland, College Park, USA, University of Maryland, College Park, University of Maryland, College Park

加速训练密度控制

EAGLES针对3D Gaussian Splatting在高质量新视角合成中需数百万高斯、存储和显存开销过高的问题,提出对颜色、旋转和不透明度等点属性进行量化嵌入压缩,并结合由粗到细训练与剪枝减少冗余高斯。实验显示在保持接近3D-GS重建质量的同时,场景存储降低约10–20倍,训练与渲染速度也有所提升。

DreamScene: 3D Gaussian-based Text-to-3D Scene Generation via Formation Pattern Sampling Figure 1
arXiv preprint2024-11-20

DreamScene: 3D Gaussian-based Text-to-3D Scene Generation via Formation Pattern Sampling

Haoran Li, Haolin Shi, Wenli Zhang, Wenjun Wu, Yong Liao, Lin Wang, Lik-Hang Lee, Peng Yuan Zhou

CCCD Key Lab of Ministry of Culture and Tourism, Hefei, China, CCCD Key Lab of Ministry of Culture and Tourism, University of Science and Technology of China, Hefei, China, University of Science and Technology of China, University of Hong Kong, The Hong Kong Polytechnic University, Hung Hom, Hong Kong, The Hong Kong Polytechnic University, Hong Kong Polytechnic University, Aarhus University, Aarhus, Denmark, Aarhus University

扩散生成

DreamScene面向文本到3D场景生成中质量低、跨视角不一致且难以编辑的问题,采用3D Gaussian表示,并提出Formation Pattern Sampling:用多时间步采样平衡语义与形状、过滤冗余高斯稳定优化,再结合重建生成纹理;同时设计室内/室外三阶段相机采样和物体-环境分离融合。实验显示其在物体与场景生成质量、一致性和可编辑性上优于现有方法。

DGD: Dynamic 3D Gaussians Distillation Figure 1
arXiv preprint2024-11-20

DGD: Dynamic 3D Gaussians Distillation

Noam Issachar, Itai Lang, Sagie Benaim

The Hebrew University of Jerusalem, Jerusalem, Israel, The Hebrew University of Jerusalem, Hebrew University of Jerusalem, University of Chicago, Chicago, USA, University of Chicago

动态场景

DGD面向单目视频中的动态3D场景理解:现有动态3DGS能重建和跟踪点,却难以按文本或点击持续追踪语义实体。论文将颜色、几何、形变与由2D基础模型蒸馏的语义特征统一到动态高斯中,并联合优化语义与外观以反向约束几何。实验显示其可在真实和合成场景中实现快速渲染、密集3D语义分割与时空跟踪,但低帧率、透明物体和细粒度语义仍是失败点。

V^3: Viewing Volumetric Videos on Mobiles via Streamable 2D Dynamic Gaussians Figure 1
ACM TOG 20242024-11-19

V^3: Viewing Volumetric Videos on Mobiles via Streamable 2D Dynamic Gaussians

Penghao Wang, Zhirui Zhang, Liao Wang, Kaixin Yao, Siyuan Xie, Jingyi Yu, Minye Wu, Lan Xu

NeuDim Inc., Shanghai, China, ShanghaiTech University, Shanghai, China, ShanghaiTech University

数字人

为解决动态 3DGS 体视频在手机端受算力、带宽和逐帧存储限制而难以流式播放的问题,V³ 将高斯属性按时序烘焙成可由硬件视频编解码器处理的 2D Gaussian Video,并用运动/外观解耦的两阶段训练、剪枝、残差熵损失和时序损失提升压缩友好性与连续性。实验显示其可在常见移动设备上实现高质量实时流式渲染与交互播放,但大场景和实时重建仍受限。

StyleGaussian: Instant 3D Style Transfer with Gaussian Splatting Figure 1
arXiv preprint2024-11-19

StyleGaussian: Instant 3D Style Transfer with Gaussian Splatting

Kunhao Liu, Fangneng Zhan, Muyu Xu, Christian Theobalt, Ling Shao, Shijian Lu

Nanyang Technological University, Singapore, Singapore, Nanyang Technological University, Max Planck Institute for Informatics, Saarbrücken, Germany, Max Planck Institute for Informatics, UCAS-Terminus AI Lab, UCAS, Hefei, China, UCAS-Terminus AI Lab

风格迁移

针对现有辐射场风格迁移需要逐风格优化、交互慢且易破坏多视角一致性的问题,StyleGaussian把VGG内容特征嵌入3D高斯后用类似AdaIN的统计对齐完成零样式优化迁移,并通过先渲染低维特征再映射到高维来降显存,配合基于KNN邻域的3D CNN直接在高斯空间解码,避免2D CNN带来的视角不一致。实验显示其可约10 fps即时风格化,保持实时渲染和严格多视角一致性,但仅改颜色不改几何。

Representing Long Volumetric Video with Temporal Gaussian Hierarchy Figure 1
ACM TOG 20242024-11-19

Representing Long Volumetric Video with Temporal Gaussian Hierarchy

Zhen Xu, Yinghao Xu, Zhiyuan Yu, Sida Peng, Jiaming Sun, Hujun Bao, Xiaowei Zhou

State Key Lab of CAD and CG, Zhejiang University, Hangzhou, China, State Key Lab of CAD and CG, Zhejiang University, Zhejiang University, Hangzhou, China, Stanford University, California, United States of America, Stanford University, Department of Mathematics, Hong Kong University of Science and Technology, Hangzhou, China, Department of Mathematics, Hong Kong University of Science and Technology, University of Hong Kong, State Key Laboratory of CAD & CG, Zhejiang University, Hangzhou, China, State Key Laboratory of CAD & CG

加速训练动态场景

针对现有动态视角合成方法难以扩展到分钟级体视频、显存和存储随时长膨胀的问题,本文提出 Temporal Gaussian Hierarchy,利用动态场景中不同区域时间冗余不同的洞察,以多层 4D Gaussian 在不同时间尺度共享不变内容,并结合紧凑外观模型与硬件加速光栅化。实验显示其在长序列上保持近似恒定 GPU 内存,并在训练成本、渲染速度和存储占用上优于基线,1080p 18k 帧可达约 450 FPS。

Mini-Splatting2: Building 360 Scenes within Minutes via Aggressive Gaussian Densification Figure 1
arXiv preprint2024-11-19

Mini-Splatting2: Building 360 Scenes within Minutes via Aggressive Gaussian Densification

Guangchi Fang, Bing Wang

加速训练密度控制

针对 3DGS 训练中高斯分布不均、低对比结构欠重建以及对不可见/低贡献高斯仍统一更新导致的低效,Mini-Splatting2 将几何先验引入密度控制与优化:通过模糊区域补密、深度重初始化、冗余简化,以及早期激进克隆关键高斯和按视角可见性裁剪,加速结构成形并减少计算。实验显示在保持相近渲染质量下,高斯数量最多减少约 4×、优化加速约 3×,示例场景相对 3DGS-accel 达到 4.2× 训练加速。

Gradient-Weighted Feature Back-Projection: A Fast Alternative to Feature Distillation in 3D Gaussian Splatting Figure 1
arXiv preprint2024-11-19

Gradient-Weighted Feature Back-Projection: A Fast Alternative to Feature Distillation in 3D Gaussian Splatting

Joji Joseph, Bharadwaj Amrutur

Indian Institute of Science, Bangalore, India, Indian Institute of Science, Indian Institute of Science Bangalore

编辑语言嵌入分割

针对3DGS中特征蒸馏训练开销大、渲染特征与单个高斯特征不匹配而导致3D分割不稳的问题,本文提出无需训练的梯度加权特征反投影:按每个高斯对最终像素渲染的影响,将2D基础模型特征聚合回预训练高斯。实验显示其在2D/3D分割上接近或优于训练式方法,速度和可扩展性更好,并可用于语言查询、场景编辑与机器人可供性迁移。

3DGSR: Implicit Surface Reconstruction with 3D Gaussian Splatting Figure 1
ACM TOG 20242024-11-19

3DGSR: Implicit Surface Reconstruction with 3D Gaussian Splatting

Xiaoyang Lyu, Yang-Tian Sun, Yi-Hua Huang, Xiuzhe Wu, Ziyi Yang, Yilun Chen, Jiangmiao Pang, Xiaojuan Qi

The University of Hong Kong (HKU), Hong Kong, Hong Kong, The University of Hong Kong (HKU), University of Hong Kong, Shanghai AI Lab, Shang Hai, China, Shanghai AI Lab, Shanghai Artificial Intelligence Laboratory

网格重建渲染

3DGSR针对3D Gaussian Splatting虽渲染高效但几何点云噪声大、难以直接恢复连续表面的缺陷,将轻量SDF嵌入高斯表示,并通过松耦合约束高斯分布与朝向、再用体渲染深度/法线一致性补足稀疏监督,使SDF与3DGS联合优化。实验显示其在保持3DGS渲染质量和效率的同时,获得更细致的网格重建,并优于多种GS重建方法。

GPS-Gaussian+: Generalizable Pixel-wise 3D Gaussian Splatting for Real-Time Human-Scene Rendering from Sparse Views Figure 1
IEEE Transactions on Pattern Analysis and Machine Intelligence2024-11-18

GPS-Gaussian+: Generalizable Pixel-wise 3D Gaussian Splatting for Real-Time Human-Scene Rendering from Sparse Views

Boyao Zhou, Shunyuan Zheng, Hanzhang Tu, Ruizhi Shao, Boning Liu, Shengping Zhang, Liqiang Nie, Yebin Liu

Department of Automation, Tsinghua University, Beijing, P.R.China, Department of Automation, Tsinghua University, School of Computer Science and Technology, Harbin Institute of Technology, Weihai, P.R.China, School of Computer Science and Technology, Harbin Institute of Technology, School of Computer Science and Technology, Harbin Institute of Technology, Shenzhen, P.R.China

加速训练动态场景渲染

针对 NeRF/3DGS 类自由视角人体渲染通常需逐主体或逐场景优化、难以交互实时的问题,GPS-Gaussian+用双目立体匹配提供确定性几何,并在源视图上回归像素级高斯参数图,配合极线注意力、深度残差与几何一致性正则,使模型可仅用渲染监督训练并泛化到人体-场景。多数据集结果显示其画质优于对比方法,同时单卡约 25 FPS 实时渲染。

DGS-SLAM: Gaussian Splatting SLAM in Dynamic Environment Figure 1
arXiv preprint2024-11-16

DGS-SLAM: Gaussian Splatting SLAM in Dynamic Environment

Mangyu Kong, Jaewon Lee, Seongwon Lee, Euntai Kim

同步定位与建图

DGS-SLAM针对现有高斯泼溅SLAM默认静态场景、在动态物体下易产生位姿和建图不一致的问题,将动态过滤贯穿高斯插入、关键帧选择与联合优化;其关键在于跨关键帧光度一致性的鲁棒掩码,以及利用高斯关联关键帧ID的回环感知窗口选择。在TUM和Bonn动态RGB-D基准上,方法在相机跟踪和新视角合成上优于多种GS/辐射场SLAM,并显示消融组件有效。

SPARS3R: Semantic Prior Alignment and Regularization for Sparse 3D Reconstruction Figure 1
CVPR 20252024-11-15

SPARS3R: Semantic Prior Alignment and Regularization for Sparse 3D Reconstruction

Yutao Tang, Yuxiang Guo, Deming Li, Cheng Peng

Johns Hopkins University

三维重建位姿估计稀疏表示

SPARS3R针对稀疏视角3DGS重建中初始点云过稀导致模糊、深度先验虽稠密但位姿不准易产生漂浮物的问题,将DUSt3R/MASt3R稠密点云与COLMAP SfM稀疏点云结合:先用RANSAC全局Procrustes对齐,再对全局外点借助语义分割做局部对齐。实验显示其在多个稀疏NVS基准上较现有方法获得更清晰、定量更优的渲染结果。

BillBoard Splatting (BBSplat): Learnable Textured Primitives for Novel View Synthesis Figure 1
ICCV 20252024-11-13

BillBoard Splatting (BBSplat): Learnable Textured Primitives for Novel View Synthesis

David Svitov, Pietro Morerio, Lourdes Agapito, Alessio Del Bue

University of Genoa, Italian Institute of Technology, University College, Department of Computer Science, London, University College, Department of Computer Science, University College London

优化方法纹理建模

针对2D Gaussian便于表面重建但渲染质量弱于3DGS的问题,BBSplat将场景表示为可优化的带RGB纹理与alpha形状图的平面billboard原语,可替换GS管线中的高斯,并用稀疏纹理正则、量化和字典压缩降低存储。实验在Tanks&Temples、DTU、Mip-NeRF-360上验证,DTU全高清PSNR达29.72,网格提取优于多数3D高斯方法,模型存储最高较3DGS减少约17倍。

GaussianSpa: An "Optimizing-Sparsifying" Simplification Framework for Compact and High-Quality 3D Gaussian Splatting Figure 1
CVPR 20252024-11-09

GaussianSpa: An "Optimizing-Sparsifying" Simplification Framework for Compact and High-Quality 3D Gaussian Splatting

Yangming Zhang, Wenqi Jia, Wei Niu, Miao Yin

University of Texas at Arlington, Dept. of Computer Science, University of Texas at Arlington, The University of Texas at Arlington, School of Computing University of Georgia, University of Georgia

压缩密度控制

3DGS虽渲染高效,但大量高斯带来显著显存与存储开销;GaussianSpa将删点式压缩改写为带目标稀疏约束的训练内优化问题,通过“优化—稀疏化”交替步骤逐步压低高斯数量,避免一次性剪枝造成信息丢失。在多个数据集上,相比现有简化方法保持更高画质;在高斯数约减少10倍时,Deep Blending平均PSNR较原始3DGS提升0.9 dB。

End-to-End Rate-Distortion Optimized 3D Gaussian Representation Figure 1
arXiv preprint2024-11-04

End-to-End Rate-Distortion Optimized 3D Gaussian Representation

Henan Wang, Hanxin Zhu, Tianyu He, Runsen Feng, Jiajun Deng, Jiang Bian, Zhibo Chen

University of Science and Technology of China, Hefei, China, University of Science and Technology of China, Microsoft Research Asia, Beijing, China, Microsoft Research Asia, Microsoft Research Asia (China), The University of Adelaide, Adelaide, Australia, The University of Adelaide, University of Adelaide

压缩

针对3D Gaussian Splatting存储开销过大、现有压缩多在固定失真下分阶段降码率且颜色参数分配粗糙的问题,RDO-Gaussian将紧凑表示学习写成端到端率失真优化,通过动态高斯剪枝、自适应球谐阶数剪枝和熵约束向量量化联合控制码率与画质。实验证明其在真实与合成场景中可相对原始3DGS压缩超过40倍,并优于已有方法的率失真表现。

Deblurring 3D Gaussian Splatting Figure 1
arXiv preprint2024-11-04

Deblurring 3D Gaussian Splatting

Byeonghyeon Lee, Howoong Lee, Xiangyu Sun, Usman Ali, Eunbyung Park

Department of Electrical and Computer Engineering, Sungkyunkwan University, Seoul, South Korea, Department of Electrical and Computer Engineering, Sungkyunkwan University, Department of Artificial Intelligence, Sungkyunkwan University, Seoul, South Korea, Department of Artificial Intelligence

去模糊

针对3D Gaussian Splatting在训练图像含散焦、运动或相机抖动模糊时重建质量显著下降的问题,本文提出首个面向3D-GS的实时去模糊框架:训练阶段用小型MLP按高斯调整均值/协方差来模拟空间变化模糊,并通过补点与剪枝缓解模糊图像SfM点云稀疏;推理阶段不启用MLP,保持原3D-GS渲染流程。实验显示其在多项基准上达到或接近SOTA画质,同时保持超过800 FPS。

Gaussian Grouping: Segment and Edit Anything in 3D Scenes Figure 1
arXiv preprint2024-11-02

Gaussian Grouping: Segment and Edit Anything in 3D Scenes

Mingqiao Ye, Martin Danelljan, Fisher Yu, Lei Ke

Computer Vision Lab, ETH Zurich, Zurich, Switzerland, Computer Vision Lab, ETH Zurich

编辑分割

这篇论文针对3D Gaussian Splatting只重建外观/几何、缺少对象级理解的问题,提出为每个高斯加入低维Identity Encoding,并用SAM生成的多视角2D掩码和3D空间一致性正则来学习分组,从而无需3D标注获得开放场景实例/物体分割。实验显示其在保持高质量、实时渲染和快速训练的同时,分割质量优于多类NeRF式方案,并可直接支持物体移除、补全、重组、上色和风格迁移等局部3D编辑。

DynMF: Neural Motion Factorization for Real-time Dynamic View Synthesis with 3D Gaussian Splatting Figure 1
arXiv preprint2024-11-01

DynMF: Neural Motion Factorization for Real-time Dynamic View Synthesis with 3D Gaussian Splatting

Agelos Kratimenos, Jiahui Lei, Kostas Daniilidis

University of Pennsylvania, Philadelphia, PA, USA, University of Pennsylvania

动态场景

面向动态场景新视角合成中运动复杂、单目约束弱且实时性难兼顾的问题,DynMF 将每个高斯点的运动分解为少量共享神经轨迹基,并用稀疏系数约束实现可解释的运动解耦与控制;时间查询的小 MLP 使训练约 5 分钟达较好质量、半小时优于或接近 SOTA,1K 渲染超过 120 FPS,存储约为静态场景两倍。

CityGaussianV2: Efficient and Geometrically Accurate Reconstruction for Large-Scale Scenes Figure 1
arXiv preprint2024-11-01

CityGaussianV2: Efficient and Geometrically Accurate Reconstruction for Large-Scale Scenes

Yang Liu, Chuanchen Luo, Zhongkai Mao, Junran Peng, Zhaoxiang Zhang

NLPR, MAIS, Institute of Automation, Chinese Academy of Sciences, Institute of Automation, Chinese Academy of Sciences, University of Chinese Academy of Sciences, Shandong University, University of Science and Technology Beijing

大规模场景网格重建

该文针对3DGS/2DGS在大规模城市级场景中表面几何不准、收敛慢且高斯数量膨胀的问题,提出CityGaussianV2:用分解梯度致密化和Depth-Anything V2深度回归消除模糊surfels,以伸长过滤抑制退化,并重构并行训练、剪枝与量化流程。实验显示其在大场景网格重建上几何质量优于2DGS/CityGaussian,同时训练时间至少降25%、显存降约50%、存储最高压缩10倍。

Gaussian Splatting on the Move: Blur and Rolling Shutter Compensation for Natural Camera Motion Figure 1
arXiv preprint2024-10-31

Gaussian Splatting on the Move: Blur and Rolling Shutter Compensation for Natural Camera Motion

Otto Seiskari, Jerry Ylilammi, Valtteri Kaatrasalo, Pekka Rantalankila, Matias Turkulainen, Juho Kannala, Esa Rahtu, Arno Solin

Aalto University, Espoo, Finland, Aalto University, ETH Zurich, Zurich, Switzerland, ETH Zurich, University of Oulu, Oulu, Finland, University of Oulu, Tampere University, Tampere, Finland, Tampere University

去模糊渲染

这篇工作针对手持手机/相机采集时常见的运动模糊与卷帘快门畸变,弥补标准 3DGS 依赖清晰静态照片的限制。核心做法不是先做2D去模糊,而是利用 VIO 估计的线/角速度,在曝光期间建模非静止相机轨迹,并用屏幕空间近似构建可微渲染以同时补偿模糊和RS、优化位姿。合成与真实数据上相较 Splatfacto、BAD-NeRF及学习式预处理基线在 PSNR/SSIM/LPIPS 和视觉锐度上均更好。

Dual-Camera Smooth Zoom on Mobile Phones Figure 1
arXiv preprint2024-10-31

Dual-Camera Smooth Zoom on Mobile Phones

Renlong Wu, Zhilu Zhang, Yu Yang, Wangmeng Zuo

Harbin Institute of Technology, Harbin, China, Harbin Institute of Technology

其他渲染

手机在超广角与广角双摄间变焦时会因视角、内参和 ISP 差异产生几何与颜色跳变。论文将其定义为 DCSZ,并用 ZoomGS 构建“数据工厂”:通过相机特定编码为连续虚拟相机生成对应 3DGS 模型与合成序列,再微调帧插值模型。实验显示,多种 FI 模型在合成与真实双摄评测上较原模型明显提升,说明主要收益来自更贴近任务的数据构造。

SWAG: Splatting in the Wild images with Appearance-conditioned Gaussians Figure 1
arXiv preprint2024-10-30

SWAG: Splatting in the Wild images with Appearance-conditioned Gaussians

Hiba Dahmani, Moussab Bennehar, Nathan Piasco, Luis Roldão

Noah’s Ark, Huawei Paris Research Center, Boulogne-Billancourt, France, Huawei Paris Research Center, Noah's Ark, Huawei Paris Research Center, Boulogne-Billancourt, France, Huawei Technologies (France)

野外场景渲染

SWAG针对3D Gaussian Splatting在野外无约束照片集中易受光照、曝光变化和行人等瞬态遮挡影响的问题,将每张图像的外观嵌入用于调制高斯颜色,并学习图像相关的不透明度变化来无监督吸收遮挡、恢复静态场景。在Phototourism和NeRF-OSR等户外场景上,它相较NeRF-W、Ha-NeRF等方法取得更好的新视角渲染质量,同时保持3DGS的训练与实时渲染效率。

Gaussian in the Wild: 3D Gaussian Splatting for Unconstrained Image Collections Figure 1
arXiv preprint2024-10-30

Gaussian in the Wild: 3D Gaussian Splatting for Unconstrained Image Collections

Dongbin Zhang, Chuming Wang, Weitao Wang, Peihao Li, Minghan Qin, Haoqian Wang

Tsinghua Shenzhen International Graduate School, Tsinghua University, Beijing, China, Tsinghua Shenzhen International Graduate School, Tsinghua University

野外场景渲染

本文针对野外无约束照片集中光照、天气差异和行人车辆等瞬态遮挡导致的重建退化问题,提出 GS-W:以 3D Gaussian 表示场景,并为每个高斯点分离建模固有外观与动态外观,再通过自适应采样捕获局部环境变化、用 2D 可见性图削弱遮挡影响。实验显示其在重建细节和画质上优于多种 NeRF 式 in-the-wild 方法,且借助高斯光栅化实现超过千倍的渲染速度提升。

CoherentGS: Sparse Novel View Synthesis with Coherent 3D Gaussians Figure 1
arXiv preprint2024-10-29

CoherentGS: Sparse Novel View Synthesis with Coherent 3D Gaussians

Avinash Paliwal, Wei Ye, Jinhui Xiong, Dmytro Kotovenko, Rakesh Ranjan, Vikas Chandra, Nima Khademi Kalantari

Meta Reality Labs, Fremont, USA, Meta Reality Labs, Texas A&M University, College Station, USA, Texas A&M University, College Station, META Health

稀疏表示

CoherentGS瞄准极稀疏输入下3DGS易过拟合、在新视角呈“针状/漂浮”伪影的问题;核心是把每个输入像素绑定一个高斯,在2D图像空间用隐式卷积解码器、深度TV与光流一致性约束高斯协同移动,并用单目深度初始化位置。实验显示其在多数据集上较稀疏视角NeRF和同期3DGS方法获得更清晰纹理与更平滑几何,还便于遮挡区域补全;但透明物体和错误深度仍是限制。

GVGEN: Text-to-3D Generation with Volumetric Representation Figure 1
arXiv preprint2024-10-28

GVGEN: Text-to-3D Generation with Volumetric Representation

Xianglong He, Junyi Chen, Sida Peng, Di Huang, Yangguang Li, Xiaoshui Huang, Chun Yuan, Wanli Ouyang, Tong He

Shanghai AI Laboratory, Shanghai, China, Shanghai AI Laboratory, Tsinghua Shenzhen International Graduate School, Shenzhen, China, Tsinghua Shenzhen International Graduate School, Shanghai Artificial Intelligence Laboratory, Tsinghua–Berkeley Shenzhen Institute, Shanghai Jiao Tong University, Shanghai, China, Shanghai Jiao Tong University, Zhejiang University, Hangzhou, China, Zhejiang University, The Chinese University of Hong Kong, Sha Tin, Hong Kong, The Chinese University of Hong Kong

扩散生成

GVGEN针对文本到3D中优化式方法耗时长、前馈方法难以直接生成高质量3D高斯的问题,将无序高斯整理为固定分辨率的GaussianVolume,并用候选池剪枝/增密提升细节;生成端采用先预测Gaussian Distance Field粗几何、再由3D U-Net补全高斯属性的粗到细流程。实验显示其在定性和定量上优于多种基线,同时单个资产约7秒生成,较好平衡质量与效率。

CityGaussian: Real-time High-quality Large-Scale Scene Rendering with Gaussians Figure 1
arXiv preprint2024-10-28

CityGaussian: Real-time High-quality Large-Scale Scene Rendering with Gaussians

Yang Liu, Chuanchen Luo, Lue Fan, Naiyan Wang, Junran Peng, Zhaoxiang Zhang

NLPR, MAIS, Institute of Automation, Chinese Academy of Sciences, Beijing, China, Institute of Automation, Chinese Academy of Sciences, University of Chinese Academy of Sciences, Beijing, China, University of Chinese Academy of Sciences, Shandong University, Jinan, China, Shandong University, University of Science and Technology Beijing, Beijing, China, University of Science and Technology Beijing, Centre for Artificial Intelligence and Robotic, New Territories, Hong Kong, Centre for Artificial Intelligence and Robotic, Centre for Artificial Intelligence and Robotics

大规模场景

CityGaussian面向城市级3DGS训练显存爆炸和百万级高斯深度排序导致的实时渲染瓶颈,采用空间分块并行训练、全局粗高斯先验对齐与自适应视角分配来减少冗余并保证块间融合,再以块级LoD压缩和按视锥/距离选择细节层。实验显示其在MatrixCity等大规模场景上取得领先或接近领先的画质,并在跨尺度视角下显著提升实时渲染速度。

Neural Fields in Robotics: A Survey Figure 1
arXiv preprint2024-10-26

Neural Fields in Robotics: A Survey

Muhammad Zubair Irshad, Mauro Comi, Yen-Chen Lin, Nick Heppert, Abhinav Valada, Zsolt Kira, Rares Ambrus, Johnathan Trembley

综述机器人

机器人需要比点云、体素等离散地图更精细且可微的环境表示,以支撑感知到决策的闭环。本文的核心洞察是将神经场作为统一视角,梳理 Occupancy、SDF、NeRF 与 3D Gaussian Splatting 四类表示,并分析其在位姿估计、操作、导航、物理推理和自动驾驶中的作用。基于 200 余篇工作,综述总结了神经场在高质量重建、多传感器融合、语义/基础模型结合上的潜力,同时指出实时性、动态场景和可部署性仍是主要瓶颈。

LoopGaussian: Creating 3D Cinemagraph with Multi-view Images via Eulerian Motion Field Figure 1
arXiv preprint2024-10-26

LoopGaussian: Creating 3D Cinemagraph with Multi-view Images via Eulerian Motion Field

Jiyang Li, Lechao Cheng, Zhangye Wang, Tingting Mu, Jingxuan He

Zhejiang University, Hangzhou, China, Zhejiang University, Hefei University of Technology, Hefei, China, Hefei University of Technology, University of Manchester, Manchester, United Kingdom, University of Manchester

物理建模渲染

针对现有 Cinemagraph 多停留在 2D 图像空间、缺少真实几何导致视角受限和不一致的问题,LoopGaussian 从多视图静态图像重建 3D Gaussian 场景,并利用场景自相似性、SuperGaussian 聚类与两阶段估计得到欧拉运动场,再通过双向动画生成可无缝循环的 3D 动态效果。实验表明其能在树枝、旗帜、衣物等软性非刚体场景中产生较自然且支持新视角渲染的结果。

Texture-GS: Disentangling the Geometry and Texture for 3D Gaussian Splatting Editing Figure 1
arXiv preprint2024-10-25

Texture-GS: Disentangling the Geometry and Texture for 3D Gaussian Splatting Editing

Tian-Xing Xu, Wenbo Hu, Yu-Kun Lai, Ying Shan, Song-Hai Zhang

Tsinghua University, Beijing, China, Tsinghua University, Tencent AI Lab, Shenzhen, China, Tencent AI Lab, Tencent (China), Cardiff University, Cardiff, UK, Cardiff University

编辑纹理建模

Texture-GS针对3D Gaussian Splatting将颜色与几何绑定、导致纹理替换和局部外观编辑不便的问题,把视角无关外观显式表示为可学习2D纹理。其关键是在高斯中心学习UV映射,并用局部Taylor展开近似射线-高斯交点的UV,从而兼顾连续纹理查询与实时渲染。DTU实验显示可恢复较平滑纹理,支持全局换纹理和细粒度编辑,并在RTX 2080 Ti上约58 FPS。

SGS-SLAM: Semantic Gaussian Splatting For Neural Dense SLAM Figure 1
arXiv preprint2024-10-25

SGS-SLAM: Semantic Gaussian Splatting For Neural Dense SLAM

Mingrui Li, Shuhong Liu, Heng Zhou, Guohao Zhu, Na Cheng, Tianchen Deng, Hongyu Wang

Dalian University of Technology, Dalian, China, Dalian University of Technology, The University of Tokyo, Bunkyo City, Japan, The University of Tokyo, Bunkyo University, Columbia University, New York, USA, Columbia University, Shanghai Jiao Tong University, Shanghai, China, Shanghai Jiao Tong University

同步定位与建图

针对 NeRF/隐式 SLAM 在物体边界过平滑、语义解耦困难和渲染效率低的问题,SGS-SLAM 将语义特征显式融入 3D Gaussian Splatting,联合优化外观、几何与语义,并用语义损失和语义引导关键帧选择稳定建图。实验显示其在位姿估计、稠密重建、语义分割和物体级几何上优于多类基线,同时保持实时渲染能力。

SC4D: Sparse-Controlled Video-to-4D Generation and Motion Transfer Figure 1
arXiv preprint2024-10-25

SC4D: Sparse-Controlled Video-to-4D Generation and Motion Transfer

Zijie Wu, Chaohui Yu, Yanqin Jiang, Chenjie Cao, Fan Wang, Xiang Bai

DAMO Academy, Alibaba Group, Hangzhou, China, DAMO Academy, Alibaba Group, Huazhong University of Science and Technology, Wuhan, China, Huazhong University of Science and Technology, Alibaba Group (China)

扩散生成

SC4D针对单目视频生成4D物体时参考视角对齐、时空一致性与运动保真难以兼顾的问题,将外观建模为密集3D Gaussian、运动建模为约512个稀疏控制点,并用两阶段优化与LBS驱动细化结果;AG初始化和GA损失用于缓解形状膨胀、位移和纹理模糊。实验显示其在质量和效率上优于既有video-to-4D方法,并可将学到的控制点运动迁移到文本指定的新实体上。

MVSplat: Efficient 3D Gaussian Splatting from Sparse Multi-View Images Figure 1
arXiv preprint2024-10-25

MVSplat: Efficient 3D Gaussian Splatting from Sparse Multi-View Images

Yuedong Chen, Haofei Xu, Chuanxia Zheng, Bohan Zhuang, Marc Pollefeys, Andreas Geiger, Tat-Jen Cham, Jianfei Cai

Monash University, Melbourne, Australia, Monash University, ETH Zurich, Zürich, Switzerland, ETH Zurich, University of Tübingen, Tübingen AI Center, Tübingen, Germany, University of Tübingen, Tübingen AI Center, VGG, University of Oxford, Oxford, UK, University of Oxford, Microsoft, Zurich, Switzerland, Microsoft, Microsoft (Switzerland)

前馈重建稀疏表示

MVSplat面向稀疏多视图下快速场景重建与新视角合成,动机是避免NeRF/逐场景优化的慢速以及直接回归高斯深度带来的几何噪声。其核心是用平面扫描构建代价体,把跨视图特征匹配作为定位3D Gaussian中心的几何线索,并端到端仅用光度监督预测其他高斯参数。在RealEstate10K和ACID上达到SOTA,22 fps,相比pixelSplat参数少10倍、推理快2倍以上且几何与泛化更好。

DiffGS: Functional Gaussian Splatting Diffusion Figure 1
arXiv preprint2024-10-25

DiffGS: Functional Gaussian Splatting Diffusion

Yu-Shen Liu, Weiqi Zhang, Junsheng Zhou

School of Software, Tsinghua University, Beijing, China, School of Software, Tsinghua University

扩散生成

DiffGS针对3D Gaussian Splatting离散、无结构而难以直接生成的问题,将3DGS解耦为概率、颜色和变换三个连续函数,并在其VAE潜空间训练扩散模型;再用八叉树引导采样与优化离散化出任意数量高斯。实验覆盖无条件、文本/图像/部分3DGS条件生成及Point-to-Gaussian,在ShapeNet和DeepFashion3D上优于同期方法。

CG-SLAM: Efficient Dense RGB-D SLAM in a Consistent Uncertainty-aware 3D Gaussian Field Figure 1
arXiv preprint2024-10-25

CG-SLAM: Efficient Dense RGB-D SLAM in a Consistent Uncertainty-aware 3D Gaussian Field

Jiarui Hu, Xianhao Chen, Boyin Feng, Guanglin Li, Liangjing Yang, Hujun Bao, Guofeng Zhang, Zhaopeng Cui

State Key Lab of CAD&CG, Zhejiang University, Hangzhou, China, State Key Lab of CAD&CG, Zhejiang University, ZJU-UIUC Institute, International Campus, Zhejiang University, Hangzhou, China, ZJU-UIUC Institute

同步定位与建图

CG-SLAM针对NeRF式稠密RGB-D SLAM体渲染开销大、3D Gaussian直接用于SLAM易过拟合且几何不稳定的问题,构建不确定性感知的高一致性Gaussian场;通过姿态导数分析与CUDA框架解耦跟踪/建图,引入尺度正则、深度对齐和深度不确定性筛选有效高斯。多数据集结果显示其在跟踪、重建和效率上优于或接近基线,跟踪速度最高约15Hz。

BAGS: Blur Agnostic Gaussian Splatting through Multi-Scale Kernel Modeling Figure 1
arXiv preprint2024-10-25

BAGS: Blur Agnostic Gaussian Splatting through Multi-Scale Kernel Modeling

Cheng Peng, Yutao Tang, Yifan Zhou, Nengyu Wang, Xijun Liu, Deming Li, Rama Chellappa

Johns Hopkins University, Baltimore, MD, 21218, USA, Johns Hopkins University

去模糊

BAGS关注真实采集图像常见的运动、离焦和降采样模糊会使3D Gaussian Splatting过拟合噪声、依赖稀疏点云初始化的问题。其核心是在3DGS训练中加入逐像素模糊核建模的BPN,结合空间、颜色、深度特征预测卷积核与模糊区域掩码,并用由粗到细的核与分辨率优化缓解2D退化和3D重建的歧义。实验显示其在多类模糊场景下较NeRF和现有去模糊方法取得更高PSNR/SSIM与更清晰的新视角渲染。

BAD-Gaussians: Bundle Adjusted Deblur Gaussian Splatting Figure 1
arXiv preprint2024-10-25

BAD-Gaussians: Bundle Adjusted Deblur Gaussian Splatting

Lingzhe Zhao, Peng Wang, Peidong Liu

Westlake University, Hangzhou, China, Westlake University, Zhejiang University, Hangzhou, China, Zhejiang University

去模糊渲染

BAD-Gaussians针对3D-GS依赖清晰图像和准确位姿、在运动模糊输入下重建退化的问题,将曝光期间的相机运动轨迹显式纳入可微高斯渲染,用起止位姿插值生成虚拟清晰帧并平均成模糊图像,从而联合优化高斯场景与轨迹。实验显示其在合成和真实数据上优于既有去模糊NeRF方法,并保留实时渲染能力。

A Compact Dynamic 3D Gaussian Representation for Real-Time Dynamic View Synthesis Figure 1
arXiv preprint2024-10-25

A Compact Dynamic 3D Gaussian Representation for Real-Time Dynamic View Synthesis

Kai Katsumata, Duc Minh Vo, Hideki Nakayama

The University of Tokyo, Tokyo, Japan, The University of Tokyo

压缩动态场景

针对动态场景中 3DGS 需逐时刻存储高斯参数、内存随序列长度增长且依赖密集多视角的问题,本文将高斯位置和旋转建模为时间函数,位置用 Fourier 近似、旋转用线性近似,而尺度、颜色、不透明度保持不变,并加入光流监督提升时序一致性。实验显示该表示可用于单目和多视角动态重建,画质接近现有 NeRF/动态方法,同时在单 GPU 上以 1352×1014 分辨率达到 118 FPS。

Sort-free Gaussian Splatting via Weighted Sum Rendering Figure 1
arXiv preprint2024-10-24

Sort-free Gaussian Splatting via Weighted Sum Rendering

Qiqi Hou, Randall Rauwendaal, Zifeng Li, Hoang Le, Farzad Farhadzadeh, Fatih Porikli, Alexei Bourd, Amir Said

Qualcomm AI Research, Graphics Research Team

加速训练渲染

该文针对 3D Gaussian Splatting 在透明 alpha blending 中必须按视角深度排序、导致移动端开销和 popping 伪影的问题,提出用可学习的 Weighted Sum Rendering 近似体渲染,将混合改为可交换求和,并引入深度权重与视角相关 opacity 以弥补无排序带来的质量损失。实验显示其画质接近原 3DGS,在移动 GPU 上平均渲染加速约 1.23×。

STAG4D: Spatial-Temporal Anchored Generative 4D Gaussians Figure 1
arXiv preprint2024-10-24

STAG4D: Spatial-Temporal Anchored Generative 4D Gaussians

Yifei Zeng, Yanqin Jiang, Siyu Zhu, Yuanxun Lu, Youtian Lin, Hao Zhu, Weiming Hu, Xun Cao, Yao Yao

Nanjing University, Nanjing, China, Nanjing University, Institute of Automation, Fudan University, Shanghai, China, Fudan University

扩散生成

STAG4D针对现有4D生成易模糊、时空不一致且优化慢的问题,将预训练扩散模型与动态3D Gaussian结合:先以输入视频帧为锚生成多视角序列,并在自注意力中融合首帧作时间锚,再用条件SDS优化4D高斯,同时通过自适应增密稳定梯度。实验显示其在文本/图像/视频到4D任务中提升渲染质量、一致性和鲁棒性,并较Consistent4D约快2倍且可实时渲染。

RGBD GS-ICP SLAM Figure 1
arXiv preprint2024-10-24

RGBD GS-ICP SLAM

Seongbo Ha, Jiung Yeon, Hyeonwoo Yu

Sungkyunkwan University, Suwon, South Korea, Sungkyunkwan University

同步定位与建图

面向RGB-D稠密SLAM中3DGS跟踪依赖2D渲染、解耦视觉里程计又带来额外计算的问题,本文将G-ICP与3D Gaussian Splatting耦合到同一高斯地图中,让跟踪与建图共享点/协方差,并用尺度对齐和关键帧策略减少冗余、加速收敛。实验报告整套系统最高107 FPS,同时保持较好的轨迹跟踪与重建质量。

Large Spatial Model: End-to-end Unposed Images to Semantic 3D Figure 1
arXiv preprint2024-10-24

Large Spatial Model: End-to-end Unposed Images to Semantic 3D

Wenyan Cong, Zhiwen Fan, Boris Ivanovic, Achuta Kadambi, Renjie Li, Marco Pavone, Peihao Wang, Yue Wang, Zhangyang Wang, Kairun Wen, Danfei Xu, Jian Zhang, Shijie Zhou

前馈重建位姿估计分割

针对SfM/MVS/NeRF/3DGS流水线依赖位姿、耗时且误差传递的问题,LSM将未标定位姿RGB图像直接映射为带语义的3D高斯辐射场;其关键在于用跨视角Transformer预测像素对齐点图,再经多尺度局部聚合与2D语言分割特征提升,统一几何、外观和开放词汇语义。实验显示其可单次前馈完成重建、分割和新视角渲染,并达到实时语义3D重建。

2D-Guided 3D Gaussian Segmentation Figure 1
arXiv preprint2024-10-24

2D-Guided 3D Gaussian Segmentation

Kun Lan, Haoran Li, Haolin Shi, Wenjun Wu, Lin Wang, Yong Liao

University of Science and Technology of China

分割

针对现有 3D Gaussian 分割流程复杂、训练较慢且难以快速多目标分割的问题,本文用预训练 2D 分割图监督每个高斯的类别概率向量学习,并通过渲染分割图对齐、KNN 聚类和统计滤波修正语义歧义与误分割。实验在 LLFF、NeRF-360、Mip-NeRF 360 等场景验证,可在少于 2 分钟学习语义、单视角 1–2 秒完成多目标分割,mIOU 约 86%,性能接近以往单目标方法。

GaussianImage: 1000 FPS Image Representation and Compression by 2D Gaussian Splatting Figure 1
arXiv preprint2024-10-21

GaussianImage: 1000 FPS Image Representation and Compression by 2D Gaussian Splatting

Xinjie Zhang, Xingtong Ge, Tongda Xu, Dailan He, Yan Wang, Hongwei Qin, Guo Lu, Jing Geng, Jun Zhang

SenseTime Research, Hong Kong, China, SenseTime Research, The Hong Kong University of Science and Technology, Hong Kong, China, The Hong Kong University of Science and Technology, Hong Kong University of Science and Technology, Beijing Institute of Technology, Beijing, China, Beijing Institute of Technology, Institute for AI Industry Research (AIR), Tsinghua University, Beijing, China, Institute for AI Industry Research (AIR), Tsinghua University, The Chinese University of Hong Kong, Hong Kong, China, The Chinese University of Hong Kong

二维高斯压缩

针对 INR 图像表示在低端设备上训练慢、显存占用高、解码受限的问题,GaussianImage 将单图像表示改为 2D Gaussian Splatting,并用无深度排序的累加求和渲染替代 3D GS 的 α-blending,同时结合量化/RVQ 构建编码器。实验显示其在接近 INR 表示与 COIN/COIN++率失真表现的同时,显存至少降 3 倍、拟合快 5 倍,渲染/解码约 1500–2000 FPS。

SplatFields: Neural Gaussian Splats for Sparse 3D and 4D Reconstruction Figure 1
arXiv preprint2024-10-19

SplatFields: Neural Gaussian Splats for Sparse 3D and 4D Reconstruction

Marko Mihajlovic, Sergey Prokudin, Siyu Tang, Robert Maier, Federica Bogo, Tony Tung, Edmond Boyer

ETH Zürich, Zurich, Switzerland, ETH Zürich, ETH Zurich, Balgrist University Hospital, Zurich, Switzerland, Balgrist University Hospital, Meta Reality Labs, Zurich, Switzerland, Meta Reality Labs

稀疏表示

本文针对 3D Gaussian Splatting 在稀疏视角下易过拟合、动态场景采集成本高的问题,指出独立 splat 特征缺少空间自相关是关键原因。SplatFields 在优化阶段用隐式神经场、三平面特征和前向流网络预测/正则化 splat 属性与运动,训练后仍可保留高效 splatting 渲染。实验显示其在静态 3D 与动态 4D 稀疏重建中较多种 3DGS/4DGS 基线稳定提升 PSNR、SSIM 和视觉质量。

Reconstruction and Simulation of Elastic Objects with Spring-Mass 3D Gaussians Figure 1
arXiv preprint2024-10-19

Reconstruction and Simulation of Elastic Objects with Spring-Mass 3D Gaussians

Licheng Zhong, Hong-Xing Yu, Jiajun Wu, Yunzhu Li

Stanford University, Stanford, USA, Stanford University, Stanford, Columbia University, New York, USA, Columbia University, University of Illinois Urbana-Champaign, Champaign, USA, University of Illinois Urbana-Champaign

动态场景物理建模

面向机器人操作中弹性物体难以仅凭视觉重建并预测动力学的问题,本文提出 Spring-Gaus,将可学习拓扑和局部刚度的 3D 弹簧-质点模型嵌入 3D Gaussian,并将外观/几何重建与物理参数辨识解耦,以降低优化干扰。合成与真实实验显示其可重建形状、外观和弹性动力学,并支持短期未来预测及不同初始/环境条件下仿真。

HeadGaS: Real-Time Animatable Head Avatars via 3D Gaussian Splatting Figure 1
arXiv preprint2024-10-19

HeadGaS: Real-Time Animatable Head Avatars via 3D Gaussian Splatting

Helisa Dhamo, Yinyu Nie, Arthur Moreau, Jifei Song, Richard Shaw, Yiren Zhou, Eduardo Pérez-Pellitero

Huawei Noah’s Ark Lab, Montreal, Canada, Huawei Noah’s Ark Lab, Huawei Technologies (Canada)

数字人

面向 AR/VR、会议和游戏中的可控高保真数字人,HeadGaS 试图弥合 NeRF 头像质量与实时性的矛盾。其关键做法不是显式移动 3D 高斯,而是为每个高斯引入可学习潜特征基,并用 3DMM 表情参数线性混合,动态预测颜色与透明度,以“过表示”处理表情变化。实验显示其在单目视频头像重建与驱动上较基线最高提升约 2dB,并实现超过 10 倍渲染加速、512² 分辨率约 250fps。

Touch-GS: Visual-Tactile Supervised 3D Gaussian Splatting Figure 1
IROS 20242024-10-14

Touch-GS: Visual-Tactile Supervised 3D Gaussian Splatting

Aiden Swann, Matthew Strong, Won Kyung Do, Gadiel Sznaier Camps, Mac Schwager, Monroe Kennedy

Stanford University, Department of Mechanical Engineering, Stanford, CA, USA, Stanford University, Department of Mechanical Engineering, Stanford, Stanford University, Department of Computer Science, Stanford, CA, USA, Department of Computer Science, Stanford University, Department of Aeronautics and Astronautics, Stanford, CA, USA, Department of Aeronautics and Astronautics

渲染机器人稀疏表示

面向机器人少视角建模中纯视觉难处理遮挡、暗光及反光/透明物体的问题,Touch-GS将光学触觉转为带不确定性的GPIS隐式表面,并与单目深度经两阶段对齐和贝叶斯融合生成深度/方差监督,提出方差加权深度损失训练3DGS。实验使用DenseTact与RealSense,显示相较纯视觉或纯触觉基线在少视角新视角合成中有定量和定性提升。

Reinforcement Learning with Generalizable Gaussian Splatting Figure 1
IROS 20242024-10-14

Reinforcement Learning with Generalizable Gaussian Splatting

Jiaxu Wang, Qiang Zhang, Jingkai Sun, Jiahang Cao, Gang Han, Wen Zhao, Weining Zhang, Yecheng Shao, Yijie Guo, Renjing Xu

The Hong Kong University of Science and Technology (Guangzhou), China, The Hong Kong University of Science and Technology (Guangzhou), Beijing Innovation Center of Humanoid Robotics Co., Ltd, Beijing Innovation Center of Humanoid Robotics Co, Beijing Advanced Sciences and Innovation Center, Zhejiang University, Center for X-Mechanics, China, Zhejiang University, Center for X-Mechanics

其他机器人

视觉强化学习的性能很大程度受环境表征限制,2D图像、点云/体素难以刻画细局部几何,NeRF类方法又依赖掩码且泛化弱。本文提出GSRL,用可泛化3D Gaussian Splatting从视觉观测直接预测并细化3D高斯,作为机器人操作RL表征。RoboMimic四任务、三类RL算法实验显示其整体优于显式视觉基线,在最难任务上成功率分别提升10%、44%和15%。

Radiance Fields for Robotic Teleoperation Figure 1
IROS 20242024-10-14

Radiance Fields for Robotic Teleoperation

Vaishakh Patil, Marco Hutter

ETH Zurich, Robotic Systems Lab, ETH Zurich, Robotic Systems Lab, Robotic Research (United States)

其他机器人

面向遥操作中“直连相机清晰但视角受限、网格重建可操控但保真不足”的矛盾,论文将在线 Radiance Fields 接入 ROS 流水线,用多相机实时数据训练 NeRF/3DGS,并提供 RViz 与 VR 可视化。实验覆盖机械臂、移动底盘和移动机械臂,对比 Voxblox 网格与用户研究显示其在照片级视图和沉浸感上更优,但实时性与不同方法取舍仍受算力约束。

PEGASUS: Physically Enhanced Gaussian Splatting Simulation System for 6DOF Object Pose Dataset Generation Figure 1
IROS 20242024-10-14

PEGASUS: Physically Enhanced Gaussian Splatting Simulation System for 6DOF Object Pose Dataset Generation

Lukas Meyer, Floris Erich, Yusuke Yoshiyasu, Marc Stamminger, Noriaki Ando, Yukiyasu Domae

National Institute of Advanced Industrial Science and Technology, Tokyo, Japan, National Institute of Advanced Industrial Science and Technology

其他

面向机器人操作中6DoF姿态估计依赖大量真实标注、纯合成又存在现实差距的问题,PEGASUS用3D Gaussian Splatting扫描环境与物体并模块化合成场景,再结合物理引擎生成自然摆放,输出RGB、深度、掩码和姿态等BOP格式数据。实验显示,用其数据训练DOPE可迁移到真实UR5抓取任务,并发布Ramen与PEGASET数据集。

MM3DGS SLAM: Multi-modal 3D Gaussian Splatting for SLAM Using Vision, Depth, and Inertial Measurements Figure 1
IROS 20242024-10-14

MM3DGS SLAM: Multi-modal 3D Gaussian Splatting for SLAM Using Vision, Depth, and Inertial Measurements

Lisong C. Sun, Neel P. Bhatt, Jonathan C. Liu, Zhiwen Fan, Zhangyang Wang, Todd E. Humphreys, Ufuk Topcu

University of Texas at Austin, Austin, TX, USA, University of Texas at Austin, The University of Texas at Austin

同步定位与建图

针对 NeRF/现有 3DGS SLAM 难以兼顾实时渲染、尺度感知与稳健跟踪的问题,MM3DGS 将未标定位姿的单目或 RGB-D 图像、深度估计和 IMU 预积分约束融合到关键帧式 3D Gaussian 地图优化中,并发布 UT-MM 多模态数据集。实验显示其相对 SplaTAM 轨迹跟踪约提升 3 倍、渲染质量提升约 5%,同时支持高分辨率稠密地图实时渲染。

MM-Gaussian: 3D Gaussian-based Multi-modal Fusion for Localization and Reconstruction in Unbounded Scenes Figure 1
IROS 20242024-10-14

MM-Gaussian: 3D Gaussian-based Multi-modal Fusion for Localization and Reconstruction in Unbounded Scenes

Chenyang Wu, Yifan Duan, Xinran Zhang, Yu Sheng, Jianmin Ji, Yanyong Zhang

University of Science and Technology of China, School of Computer Science and Technology, Hefei, China, 230026, University of Science and Technology of China, School of Computer Science and Technology

位姿估计

面向室外无边界场景中单目深度不准、RGB-D量程不足导致的3D Gaussian SLAM失效问题,MM-Gaussian将Livox固态激光雷达的稀疏几何与相机颜色融合,在线估计位姿并增量构建可实时渲染的高斯地图;同时利用高斯渲染设计重定位以应对退化场景。校园等实验显示其定位与建图优于既有3D Gaussian SLAM,但速度和精度仍有提升空间。

Language-Embedded Gaussian Splats (LEGS): Incrementally Building Room-Scale Representations with a Mobile Robot Figure 1
IROS 20242024-10-14

Language-Embedded Gaussian Splats (LEGS): Incrementally Building Room-Scale Representations with a Mobile Robot

Justin Yu, Kush Hari, Kishore Srinivas, Karim El-Refai, Adam Rashid, Chung Min Kim, Justin Kerr, Richard Cheng, Muhammad Zubair Irshad, Ashwin Balakrishna, Thomas Kollar, Ken Goldberg

The AUTOLab at UC Berkeley, University of California, Berkeley, University of California, Berkeley, Berkeley Systems (United States), Toyota Research Institute, Los Altos, CA, Toyota Research Institute

语言嵌入机器人分割

面向机器人在办公室、仓库等真实空间中按自然语言寻找长尾物体的需求,LEGS 将 3D Gaussian Splatting 的显式几何/外观表示与 CLIP 语言语义结合,并通过多相机 RGBD 流、增量配准与 bundle adjustment 在线构建房间级语义地图。实验在 4 个真实场景中显示,其开放词汇查询成功率与 LERF 相近,但训练速度快 3.5 倍以上,物体定位最高约 66% 准确率。

High-Fidelity SLAM Using Gaussian Splatting with Rendering-Guided Densification and Regularized Optimization Figure 1
IROS 20242024-10-14

High-Fidelity SLAM Using Gaussian Splatting with Rendering-Guided Densification and Regularized Optimization

Shuo Sun, Malcolm Mielle, Achim J. Lilienthal, Martin Magnusson

AASS Research Center, Örebro University, Sweden, AASS Research Center, Örebro University, Independent Researcher

同步定位与建图

针对传统/NeRF式RGBD SLAM难以兼顾实时全分辨率渲染、位姿跟踪与高保真外观重建的问题,本文将3D Gaussian Splatting扩展到在线SLAM:用重渲染损失同时优化位姿和高斯地图,并以渲染误差引导增密来补全未观测区域、细化重观测区域,同时加入正则化缓解连续建图对新帧过拟合导致的遗忘。实验在Replica上达到领先的重建质量和跟踪精度,在TUM-RGBD上结果具竞争力,但真实场景受运动模糊和曝光变化限制。

DarkGS: Learning Neural Illumination and 3D Gaussians Relighting for Robotic Exploration in the Dark Figure 1
IROS 20242024-10-14

DarkGS: Learning Neural Illumination and 3D Gaussians Relighting for Robotic Exploration in the Dark

Tianyi Zhang, Kaining Huang, Weiming Zhi, Matthew Johnson-Roberson

Carnegie Mellon University, Robotics Institute, School of Computer Science, Pittsburgh, PA, USA, 15213, Carnegie Mellon University, Robotics Institute, School of Computer Science

其他机器人

面向地下、搜救等黑暗环境中机器人携带移动光源导致同一区域亮度不一致、3DGS/NeRF光度优化失效的问题,论文将照明建模作为可学习校准任务,提出NeLiS估计相机-光源关系、辐射分布与衰减,并嵌入DarkGS构建可重光照3D高斯。真实场景实验显示其能在现有方法失败时生成实时、较逼真的新视角和重光照结果,但阴影、强高光与白平衡仍未充分解决。

3DGS-Calib: 3D Gaussian Splatting for Multimodal SpatioTemporal Calibration Figure 1
IROS 20242024-10-14

3DGS-Calib: 3D Gaussian Splatting for Multimodal SpatioTemporal Calibration

Quentin Herau, Moussab Bennehar, Arthur Moreau, Nathan Piasco, Luis Roldão, Cyrille Migniot, Pascal Vasseur, Cédric Demonceaux

Huawei Paris Research Center, Noah’s Ark, France, Huawei Paris Research Center, Huawei Technologies (France), Huawei London Research Center, Noah’s Ark, United Kingdom, Huawei London Research Center, CAOR - Centre de Robotique (60, boulevard Saint-Michel, CAOR - Centre de Robotique (60, Huawei Technologies (United Kingdom), Centre de Robotique, Centre National de la Recherche Scientifique

其他

该文针对 NeRF 式无标靶多传感器时空标定训练过慢的问题,提出用 3D Gaussian Splatting 替代隐式体渲染:以 LiDAR 点云初始化/约束高斯位置,并通过几何与光度一致性联合优化相机—激光外参和时间偏移。在 KITTI-360 驾驶序列上,方法相较 NeRF 标定取得更高或更稳的精度,并显著缩短训练时间。

Generalizable and Animatable Gaussian Head Avatar Figure 1
arXiv preprint2024-10-10

Generalizable and Animatable Gaussian Head Avatar

Xuangeng Chu, Tatsuya Harada

The University of Tokyo, generalization and controllability with real-time reenactment speeds

数字人

针对一张图重建可驱动头部数字人时,NeRF方案渲染慢且常需身份级优化的问题,GAGAvatar用单次前向预测3D Gaussian参数;其关键是双向lifting从图像平面生成近闭合高斯分布,并结合3DMM先验与全局特征构造可控表情高斯。实验显示其在未见身份上无需测试时优化,可实时重演,并在重建质量和表情准确性上优于已有方法。

Efficient Perspective-Correct 3D Gaussian Splatting Using Hybrid Transparency Figure 1
Computer Graphics Forum 20252024-10-10

Efficient Perspective-Correct 3D Gaussian Splatting Using Hybrid Transparency

Florian Hahlbohm, Fabian Friederichs, Tim Weyrich, Linus Franke, Moritz Kappel, Susana Castillo, Marc Stamminger, Martin Eisemann, Marcus Magnor

Computer Graphics Lab, TU Braunschweig Germany, Computer Graphics Lab, University College London United Kingdom, University College London, University of New Mexico USA, University of New Mexico

三维高斯泼溅

该文针对3DGS中仿射投影和按primitive深度排序导致的多视角不一致、近距离形变与运动popping问题,提出无需矩阵求逆的逐像素射线-3D高斯评估,并用混合透明度仅精排前K个片元、对尾部做顺序无关累积。两部分可独立接入现有系统,组合后在常见基准上较传统3DGS最高实现2倍帧率和2倍优化加速,同时保持或提升图像质量并减少伪影。

SplaTraj: Camera Trajectory Generation with Semantic Gaussian Splatting Figure 1
arXiv preprint2024-10-08

SplaTraj: Camera Trajectory Generation with Semantic Gaussian Splatting

Xinyi Liu, Tianyi Zhang, Matthew Johnson-Roberson, Weiming Zhi

语言嵌入渲染

SplaTraj 面向高保真 Gaussian Splatting 场景难以按语言意图自动生成可观看图像序列的问题,将相机运动建模为连续时间轨迹优化;它利用语言嵌入定位用户指定物体,并把其渲染到视野中的位置、距离和遮挡等转化为可微代价,直接对渲染与轨迹求梯度。实验在多个 3DGS 场景和指令上显示,生成轨迹能平滑依次拍摄目标,画面更居中、距离更合适且减少遮挡。

HiSplat: Hierarchical 3D Gaussian Splatting for Generalizable Sparse-View Reconstruction Figure 1
arXiv preprint2024-10-08

HiSplat: Hierarchical 3D Gaussian Splatting for Generalizable Sparse-View Reconstruction

Shengji Tang, Weicai Ye, Peng Ye, Weihao Lin, Yang Zhou, Tao Chen, Wanli Ouyang

Fudan University, Shanghai AI Laboratory, State Key Lab of CAD&CG, Zhejiang University, State Key Lab of CAD&CG, Zhejiang University

前馈重建稀疏表示

HiSplat针对可泛化3D Gaussian Splatting在双/稀疏视角下单尺度高斯难以兼顾大结构与细纹理、易错位和伪影的问题,引入由粗到细的层级高斯表示,并用误差感知补偿与调制融合修复实现跨尺度联合优化。实验显示其在RealEstate10K等数据集提升重建质量,并在Replica零样本测试中表现出更强跨数据集泛化。

MonST3R: A Simple Approach for Estimating Geometry in the Presence of Motion Figure 1
arXiv preprint2024-10-04

MonST3R: A Simple Approach for Estimating Geometry in the Presence of Motion

Junyi Zhang, Charles Herrmann, Junhwa Hur, Varun Jampani, Trevor Darrell, Forrester Cole, Deqing Sun, Ming-Hsuan Yang

Google DeepMind

三维重建动态场景稀疏表示

MonST3R针对动态视频中运动物体破坏传统SfM/SLAM和多阶段深度、光流管线鲁棒性的问题,提出“几何优先”的简化路线:将DUSt3R的pointmap扩展到每个时间步,并用少量带位姿与深度的动态数据微调,而不显式建模运动。实验显示其在视频深度、相机位姿估计上更稳且更高效,并具备前馈式4D重建潜力。

SuperGS: Super-Resolution 3D Gaussian Splatting via Latent Feature Field and Gradient-guided Splitting Figure 1
arXiv preprint2024-10-03

SuperGS: Super-Resolution 3D Gaussian Splatting via Latent Feature Field and Gradient-guided Splitting

Shiyun Xie, Zhiru Wang, Yinghao Zhu, Chengwei Pan

Beihang University

密度控制前馈重建渲染

SuperGS针对低分辨率输入训练出的3DGS高斯基元过粗、难以直接用于高分辨率新视角合成的问题,采用由粗到细两阶段训练:先用潜在特征场保留低分辨率场景表征,再引入SISR伪标签、可变分残差特征和不确定性引导的密度控制/损失,并通过多视角联合学习抑制伪标签不一致。实验显示其在真实与合成数据上优于现有HRNVS方法,但具体增益在多模块间的归因仍需看消融细节。

GI-GS: Global Illumination Decomposition on Gaussian Splatting for Inverse Rendering Figure 1
arXiv preprint2024-10-03

GI-GS: Global Illumination Decomposition on Gaussian Splatting for Inverse Rendering

Hongze Chen, Zehong Lin, Jun Zhang

The Hong Kong University of Science and Technology

光线追踪重光照

GI-GS针对3DGS逆渲染中材质与光照纠缠、间接光在重光照时被静态烘焙而失真的问题,将3DGS先栅格化为含几何与材质的G-buffer,仅用PBR求直接光,再通过延迟着色上的轻量路径追踪动态计算遮挡和间接光,并扩展到世界空间场景。实验显示其在新视角合成、重光照质量和效率上优于已有3DGS逆渲染基线。

FSGS: Real-Time Few-shot View Synthesis using Gaussian Splatting Figure 1
arXiv preprint2024-10-02

FSGS: Real-Time Few-shot View Synthesis using Gaussian Splatting

Zehao Zhu, Zhiwen Fan, Yifan Jiang, Zhangyang Wang

The University of Texas at Austin, Austin, USA, The University of Texas at Austin

稀疏表示

针对少量观测下新视角合成在画质与效率之间难以兼顾的问题,FSGS将3D Gaussian Splatting用于few-shot场景,提出邻近度引导的Gaussian Unpooling在稀疏点云间补充更有代表性的高斯,并通过虚拟视角与单目深度几何正则缓解过拟合和纹理过平滑。实验覆盖NeRF-Synthetic、LLFF、Shiny和Mip-NeRF360,最少3张图即可生成较高质量结果,推理超过200 FPS,显著快于稀疏视角NeRF方法。

EVER: Exact Volumetric Ellipsoid Rendering for Real-time View Synthesis Figure 1
ICCV 20252024-10-02

EVER: Exact Volumetric Ellipsoid Rendering for Real-time View Synthesis

Alexander Mai, Peter Hedman, George Kopanas, Dor Verbin, David Futschik, Qiangeng Xu, Falko Kuester, Jonathan T. Barron, Yinda Zhang

University of California, San Diego, University of California, University of California San Diego, Google, Google (United States)

光线追踪渲染

针对 3D Gaussian Splatting 因按中心排序且忽略重叠而产生的视角切换 popping,EVER 将场景表示为常密度体椭球,并在光线追踪中记录每个椭球入/出交点,对任意重叠基元精确积分体渲染方程。该设计同时支持散焦、鱼眼等相机效应,在 RTX4090 上 720p 约 30 FPS,并在 Zip-NeRF 大场景取得领先 SSIM,优于 3DGS 基线且接近/超过离线方法。

DreamScene360: Unconstrained Text-to-3D Scene Generation with Panoramic Gaussian Splatting Figure 1
arXiv preprint2024-10-02

DreamScene360: Unconstrained Text-to-3D Scene Generation with Panoramic Gaussian Splatting

Shijie Zhou, Zhiwen Fan, Dejia Xu, Haoran Chang, Pradyumna Chari, Tejas Bharadwaj, Suya You, Zhangyang Wang, Achuta Kadambi

University of California, Los Angeles, USA, University of California, University of California, Los Angeles, University of Texas at Austin, Austin, USA, University of Texas at Austin, The University of Texas at Austin, DEVCOM Army Research Laboratory, Adelphi, USA, DEVCOM Army Research Laboratory

扩散生成

面向 VR/MR 等场景中低成本生成完整可漫游 3D 环境的需求,DreamScene360 将文本先生成全景图,并用 GPT-4V 做提示自精炼,再通过单目深度对齐、全局点云初始化和语义/几何正则提升为 3D Gaussian Splatting。实验显示其在 360°覆盖、全局一致性和视觉质量上优于 Text2Room、LucidDreamer 等基线,但分辨率仍受全景扩散模型限制。

3DGS-DET: Empower 3D Gaussian Splatting with Boundary Guidance and Box-Focused Sampling for 3D Object Detection Figure 1
arXiv preprint2024-10-02

3DGS-DET: Empower 3D Gaussian Splatting with Boundary Guidance and Box-Focused Sampling for 3D Object Detection

Yang Cao, Yuanliang Jv, Dan Xu

Hong Kong University of Science and Technology

三维高斯泼溅

面向机器人/AR等室内三维感知,论文指出 NeRF 式隐式表示限制3D检测,而直接用3DGS又有高斯分布模糊、背景 blob 过多的问题。3DGS-DET用2D边界监督重塑物体/背景空间分布,并以2D框投影成3D概率空间进行面向框的采样,保留前景、抑制噪声;在ScanNet较NeRF-Det++提升+6.0 mAP@0.25、+7.8 mAP@0.5,在ARKITScenes提升+14.9 mAP@0.25。

View-Consistent 3D Editing with Gaussian Splatting Figure 1
arXiv preprint2024-09-29

View-Consistent 3D Editing with Gaussian Splatting

Yuxuan Wang, Xuanyu Yi, Zike Wu, Na Zhao, Long Chen, Hanwang Zhang

Nanyang Technological University, Singapore, Singapore, Nanyang Technological University, Institute for Infocomm Research, A*STAR, Singapore, Singapore, Institute for Infocomm Research, Agency for Science, Technology and Research, Technology and Research, Singapore University of Technology and Design, Singapore, Singapore, Singapore University of Technology and Design, Hong Kong University of Science and Technology, Clear Water Bay, Hong Kong, Hong Kong University of Science and Technology

编辑

本文针对基于2D扩散编辑多视角图像来优化3DGS时,各视角指导不一致会引发模式坍塌和闪烁伪影的问题,提出VcEdit:把3DGS显式几何和快速渲染嵌入编辑过程,通过跨注意力一致性模块与编辑结果一致性模块在迭代“编辑—更新3DGS”中校准多视角指导。实验显示其在多种真实场景中提升视角一致性和编辑质量,优于现有3DGS编辑方法。

Radiative Gaussian Splatting for Efficient X-ray Novel View Synthesis Figure 1
arXiv preprint2024-09-29

Radiative Gaussian Splatting for Efficient X-ray Novel View Synthesis

Yuanhao Cai, Yixun Liang, Jiahao Wang, Angtian Wang, Yulun Zhang, Xiaokang Yang, Zongwei Zhou, Alan Yuille

Johns Hopkins University, Baltimore, USA, Johns Hopkins University, Shanghai Jiao Tong University, Shanghai, China, Shanghai Jiao Tong University

医学影像

本文针对 X-ray 新视角合成中 NeRF 训练和渲染过慢的问题,将 3D Gaussian Splatting 改造为适合透射成像的 X-Gaussian:用与视角无关的辐射强度响应替代 RGB 3DGS 的视角相关表示,并设计 CUDA 可微辐射栅格化与基于扫描仪参数的 ACUI 初始化,避免 SfM。实验显示其较 SOTA 提升约 6.5 dB,训练时间低于 15%,推理快 73 倍以上,并可辅助稀疏视角 CT 重建。

ManiGaussian: Dynamic Gaussian Splatting for Multi-task Robotic Manipulation Figure 1
arXiv preprint2024-09-29

ManiGaussian: Dynamic Gaussian Splatting for Multi-task Robotic Manipulation

Guanxing Lu, Shiyi Zhang, Ziwei Wang, Changliu Liu, Jiwen Lu, Yansong Tang

Shenzhen Key Laboratory of Ubiquitous Data Enabling, Shenzhen International Graduate School, Tsinghua University, Beijing, China, Shenzhen Key Laboratory of Ubiquitous Data Enabling, Shenzhen International Graduate School, Tsinghua University, University Town of Shenzhen, Department of Automation, Tsinghua University, Beijing, China, Department of Automation, Nanyang Technological University, Singapore, Singapore, Nanyang Technological University, Carnegie Mellon University, Pittsburgh, USA, Carnegie Mellon University

动态场景机器人

面向语言条件的多任务操作,现有感知或重建式方法多只学静态语义,难以处理遮挡和物体交互动态。ManiGaussian 将动态 Gaussian Splatting 用作场景表示,并通过 Gaussian world model 预测/重建未来场景,在嵌入空间学习语义传播以辅助动作预测。在 RLBench 10 个任务、166 个变体上,平均成功率较现有方法提升 13.1%,且声称计算量更低。

Large Multi-View Gaussian Model for High-Resolution 3D Content Creation Figure 1
arXiv preprint2024-09-29

Large Multi-View Gaussian Model for High-Resolution 3D Content Creation

Jiaxiang Tang, Zhaoxi Chen, Xiaokang Chen, Tengfei Wang, Gang Zeng, Ziwei Liu

National Key Lab of General AI, Peking University, Beijing, China, National Key Lab of General AI, Peking University, S-Lab, Nanyang Technological University, Singapore, Singapore, S-Lab, Nanyang Technological University, Shanghai AI Lab, Shanghai, China, Shanghai AI Lab, Shanghai Artificial Intelligence Laboratory

扩散生成

针对前馈式 3D 生成虽快但受低分辨率训练和三平面 NeRF 表达限制、细节不足的问题,LGM 将多视图扩散产生的图像融合为大量 3D Gaussian,并用非对称 U-Net 替代重型 Transformer 以支持 512 分辨率端到端监督,还提供 Gaussian 到网格转换。实验显示其在文本/单图到 3D 中约 5 秒生成最多 65k Gaussian,兼顾速度与更高细节,但质量仍依赖多视图扩散输入一致性。

GeoGaussian: Geometry-aware Gaussian Splatting for Scene Rendering Figure 1
arXiv preprint2024-09-29

GeoGaussian: Geometry-aware Gaussian Splatting for Scene Rendering

Yanyan Li, Chenyu Lyu, Yan Di, Guangyao Zhai, Gim Hee Lee, Federico Tombari

National University of Singapore, Singapore, Singapore, National University of Singapore, Technical University of Munich, Munich, Germany, Technical University of Munich, Tianjin University, Tianjin, China, Tianjin University, Google, Zurich, Switzerland, Google, Google (Switzerland)

密度控制渲染

GeoGaussian针对3D Gaussian Splatting在低纹理墙面、天花板等区域优化时几何退化、导致大视角新视图渲染不稳的问题,引入几何感知表示:从点云法向识别平滑区域,将高斯初始化为贴合表面的薄椭球,并在分裂/克隆密度控制中保持共面关系,同时加入邻域共面约束。实验显示其在公开数据集上提升了新视图渲染质量和几何重建精度,稀疏训练视角下更稳健。

EgoLifter: Open-world 3D Segmentation for Egocentric Perception Figure 1
arXiv preprint2024-09-29

EgoLifter: Open-world 3D Segmentation for Egocentric Perception

Qiao Gu, Zhaoyang Lv, Duncan Frost, Simon Green, Julian Straub, Chris Sweeney

Meta Reality Labs, Redmond, WA, 98052, USA, Meta Reality Labs, University of Toronto, Toronto, ON, M5S 1A1, Canada, University of Toronto, META Health

分割

面向可穿戴/AR 场景,自我中心视频常由自然运动采集,视角覆盖稀疏且充满手物交互动态,传统 3D 重建与分割难以稳定处理。EgoLifter 将 SAM 的 2D 开放世界分割通过对比学习提升到 3D Gaussian 表示中,并用无额外监督的 transient prediction 过滤动态物体,从而学习静态场景的可查询实例分解。论文在 Aria Digital Twin 上建立基准并取得领先结果,也在多种自我中心数据集展示 3D 物体提取和场景编辑潜力。

HAC: Hash-grid Assisted Context for 3D Gaussian Splatting Compression Figure 1
arXiv preprint2024-09-28

HAC: Hash-grid Assisted Context for 3D Gaussian Splatting Compression

Yihang Chen, Qianyi Wu, Weiyao Lin, Mehrtash Harandi, Jianfei Cai

Monash University, Clayton, Australia, Monash University, Shanghai Jiao Tong University, Shanghai, China, Shanghai Jiao Tong University

压缩

3DGS 虽渲染快且质量高,但大量高斯/锚点属性带来巨大存储,且点云稀疏无序使结构冗余难以利用。HAC 的核心洞察是无序锚点属性与结构化哈希网格存在互信息,因此学习二值哈希网格作为上下文,预测量化属性分布以做熵编码,并结合自适应量化与掩码剔除无效锚点/高斯。实验显示其平均相对原始 3DGS 压缩超过 75×,相对 Scaffold-GS 超过 11×,同时保持或提升重建质量。

MASt3R-SfM: a Fully-Integrated Solution for Unconstrained Structure-from-Motion Figure 1
3DV 20252024-09-27

MASt3R-SfM: a Fully-Integrated Solution for Unconstrained Structure-from-Motion

Bardienus Pieter Duisterhof, Lojze Zust, Philippe Weinzaepfel, Vincent Leroy, Yohann Cabon, Jerome Revaud

Carnegie Mellon University, University of Ljubljana, Naver Labs Europe

三维高斯泼溅

传统 SfM 依赖手工流水线、RANSAC 与逐步初始化,在低重叠、小位移或无序图像下易失败。MASt3R-SfM 利用冻结的 MASt3R 基础模型产生局部三维与匹配,并复用编码器做近线性图像检索,再以三维匹配损失和二维重投影损失对齐到全局坐标。多基准实验显示其在多种场景更稳定,尤其在小中规模数据上优于现有方法。

EdgeGaussians -- 3D Edge Mapping via Gaussian Splatting Figure 1
WACV 20252024-09-19

EdgeGaussians -- 3D Edge Mapping via Gaussian Splatting

Kunal Chelani, Assia Benbihi, Torsten Sattler, Fredrik Kahl

Chalmers University of Technology, Czech Institute of Informatics, Robotics and Cybernetics, Czech Technical University in Prague, Czech Institute of Informatics, Czech Technical University in Prague, Institute of Informatics of the Slovak Academy of Sciences

渲染

本文面向多视角三维边缘重建中线段匹配不稳、隐式场训练慢且需体素级采样后处理的问题,提出用 3D Gaussian Splatting 直接学习位于边缘上的有向高斯点,并约束最大方差方向沿边缘,便于聚类成独立边。实验显示其精度达到或略优于现有学习式方法,训练速度约快一个数量级。

Gradient-Driven 3D Segmentation and Affordance Transfer in Gaussian Splatting Using 2D Masks Figure 1
arXiv preprint2024-09-18

Gradient-Driven 3D Segmentation and Affordance Transfer in Gaussian Splatting Using 2D Masks

Joji Joseph, Bharadwaj Amrutur

分割

针对3DGS特征场方法在三维分割中易产生漂浮碎片、训练成本高的问题,论文利用2D掩码过滤推理时反传梯度,并将每个高斯的梯度影响作为多视角投票来定位目标高斯;同一梯度信号还用于高斯剪枝,并结合DINO做少样本可供性从2D到3D迁移。实验显示分割比Feature-3DGS更干净,剪枝最高约21%且不损精度,但效果依赖2D掩码和视角,速度慢于特征场方法。

PSHuman: Photorealistic Single-view Human Reconstruction using Cross-Scale Diffusion Figure 1
arXiv preprint2024-09-16

PSHuman: Photorealistic Single-view Human Reconstruction using Cross-Scale Diffusion

Peng Li, Wangguandong Zheng, Yuan Liu, Tao Yu, Yangguang Li, Xingqun Qi, Mengfei Li, Xiaowei Chi, Siyu Xia, Wei Xue, Wenhan Luo, Qifeng Liu, Yike Guo

method facilitates detailed geometry and realistic

数字人扩散生成网格重建

PSHuman面向单张野外全身图像重建中遮挡严重、背面歧义和人脸失真的问题,引入SMPL-X约束的多视角扩散生成彩色/法线图,并用身体—人脸跨尺度扩散保持身份与局部细节,再以SMPL-X初始化的显式雕刻快速生成带纹理网格。论文在CAPE和THuman2.1上显示其几何细节、纹理保真度和泛化优于对比方法,重建约一分钟。

BEINGS: Bayesian Embodied Image-goal Navigation with Gaussian Splatting Figure 1
arXiv preprint2024-09-16

BEINGS: Bayesian Embodied Image-goal Navigation with Gaussian Splatting

Wugang Meng, Tianfu Wu, Huan Yin, Fumin Zhang

自动驾驶机器人

针对图像目标导航中学习方法依赖海量训练、纯探索方法在复杂场景效率不足的问题,BEINGS将任务重写为最优控制并用蒙特卡洛MPC求解,以3D Gaussian Splatting作为可渲染场景先验预测未来视角,再通过贝叶斯更新根据实时观测修正目标位置分布。论文在仿真和实体实验中展示了绕障、长距离搜索与到达目标的能力,消融表明贝叶斯估计和采样规划共同贡献了误差下降。

HGS-Mapping: Online Dense Mapping Using Hybrid Gaussian Representation in Urban Scenes Figure 1
IEEE RA-L 20242024-09-13

HGS-Mapping: Online Dense Mapping Using Hybrid Gaussian Representation in Urban Scenes

Ke Wu, Kaizhao Zhang, Zhiwei Zhang, Muer Tie, Shanshuai Yuan, Jieru Zhao, Zhongxue Gan, Wenchao Ding

Academy for Engineering and Technology, Fudan University, Shanghai, China, Academy for Engineering and Technology, Fudan University, Department of Computer Science and Engineering, Shanghai Jiao Tong University, Shanghai, China, Department of Computer Science and Engineering, Shanghai Jiao Tong University

大规模场景

面向自动驾驶在线稠密建图,NeRF 渲染慢而直接用 3DGS 又受 LiDAR 覆盖不足和城市级高斯数量过大限制。HGS-Mapping 用球面高斯、3D 高斯与 2D 高斯平面的混合表示,配合混合初始化和按重建损失自适应增删高斯,补全远处/高处缺少几何先验的区域。实验显示其达到 SOTA 重建质量,仅用约 66% 高斯并带来约 20% 重建加速。

DualGS: Robust Dual Gaussian Splatting for Immersive Human-centric Volumetric Videos Figure 1
arXiv preprint2024-09-12

DualGS: Robust Dual Gaussian Splatting for Immersive Human-centric Volumetric Videos

Yuheng Jiang, Zhehao Shen, Yu Hong, Chengcheng Guo, Yize Wu, Yingliang Zhang, Jingyi Yu, Lan Xu

YUHENG JIANG, ShanghaiTech University, China and NeuDim Digital Technology (Shanghai) Co., Ltd., China, ShanghaiTech University, ZHEHAO SHEN, ShanghaiTech University, China, YU HONG, ShanghaiTech University, China, CHENGCHENG GUO, ShanghaiTech University, China, YIZE WU, ShanghaiTech University, China, YINGLIANG ZHANG, DGene Digital Technology Co., Ltd., China

数字人

面向人体中心体视频,传统网格流程需大量人工清理且资产过大,难以进入 VR。DualGS 用少量关节高斯表示运动、大量皮肤高斯表示外观,并通过锚定关系与粗到细优化提升跟踪和时序一致性;再分别压缩运动与外观属性。文中报告最高约 120× 压缩、约 350KB/帧,并可在 VR 中实时高保真播放复杂表演。

Self-Evolving Depth-Supervised 3D Gaussian Splatting from Rendered Stereo Pairs Figure 1
arXiv preprint2024-09-11

Self-Evolving Depth-Supervised 3D Gaussian Splatting from Rendered Stereo Pairs

Sadra Safadoust, Fabio Tosi, Fatma Güney, Matteo Poggi

Department of Computer Engineering, and KUIS AI Center, Koç University, and KUIS AI Center, Koç University, Department of Computer Science and, University of Bologna, Italy, University of Bologna

三维高斯泼溅

该文针对3D Gaussian Splatting虽渲染逼真但几何深度差、易产生漂浮伪影的问题,系统比较SfM、单目深度、深度补全、MVS等深度先验的监督效果,并提出训练中由GS自身渲染虚拟校正双目图像、交给预训练立体匹配网络生成深度监督的自进化框架。ETH3D、ScanNet++和BlendedMVS实验显示,该方法在稀疏视角下较传统深度先验同时提升新视角图像质量与深度准确性。

Fisheye-GS: Lightweight and Extensible Gaussian Splatting Module for Fisheye Cameras Figure 1
arXiv preprint2024-09-07

Fisheye-GS: Lightweight and Extensible Gaussian Splatting Module for Fisheye Cameras

Zimu Liao, Siyan Chen, Rong Fu, Yi Wang, Zhongling Su, Hao Luo, Li Ma, Linning Xu, Bo Dai, Hengjie Li, Zhilin Pei, Xingcheng Zhang

Shanghai Artificial Intelligence Laboratory, The Chinese University of Hong Kong, Hong Kong University of Science and Technology

渲染

该文针对原生 3DGS 依赖针孔投影、难以直接处理鱼眼相机大视场和强畸变的问题,提出仅替换几何投影预处理的 Fisheye-GS:推导等距鱼眼投影下高斯均值、协方差及训练梯度,并实现 CUDA 模块。实验显示其相比先去畸变再训练能获得更好视觉质量,且可接入 FlashGS 等高效渲染管线并扩展到全景相机。

PRoGS: Progressive Rendering of Gaussian Splats Figure 1
WACV 20252024-09-03

PRoGS: Progressive Rendering of Gaussian Splats

Brent Zoomers, Maarten Wijnants, Ivan Molenaers, Joni Vanherck, Jeroen Put, Lode Jorissen, Nick Michiels

Hasselt University - Flanders Make - Digital Future Lab, Diepenbeek, Belgium, Hasselt University - Flanders Make - Digital Future Lab, Hasselt University

压缩渲染

3DGS 场景虽可高质量实时渲染,但显式高斯数量大,现有压缩仍常需整场景下载后才能显示。PRoGS 的核心是在训练后估计每个 Gaussian 对最终画面的贡献,按重要性组织传输/加载顺序,并可结合视锥裁剪、八叉树优先级和已有压缩方法,使客户端先用少量 splats 得到近似画面再逐步细化。实验显示其在相同 splat 比例下优于现有 Web 渐进查看方案,各指标质量更高,并具备进一步节省带宽的潜力。

DynOMo: Online Point Tracking by Dynamic Online Monocular Gaussian Reconstruction Figure 1
3DV 20252024-09-03

DynOMo: Online Point Tracking by Dynamic Online Monocular Gaussian Reconstruction

Jenny Seidenschwarz, Qunjie Zhou, Bardienus P. Duisterhof, Deva Ramanan, Laura Leal-Taixé

Technical University of Munich, NVIDIA, Carnegie Mellon University

动态场景单目重建

DynOMo面向机器人导航、混合现实中更现实的无位姿单目在线视频,解决动态场景下2D/3D任意点跟踪需离线处理或多相机的问题。其核心是用动态3D Gaussian Splatting同时重建、定位相机并扩展新内容,引入深度、语义和特征描述子重建及相似性加权正则,使轨迹无需显式对应监督而涌现。实验显示其在同等在线单目设置下明显优于基线,并接近部分离线2D跟踪器,但尚未达到实时。

OmniRe: Omni Urban Scene Reconstruction Figure 1
arXiv preprint2024-08-29

OmniRe: Omni Urban Scene Reconstruction

Janick Martinez

Shanghai Jiao Tong University, University of Toronto, Stanford University, NVIDIA Research, University of Southern California

自动驾驶

OmniRe针对自动驾驶数字孪生中动态重建多聚焦车辆、难以覆盖行人/骑行者等非刚体并支持交互仿真的问题,基于3D Gaussian Splatting构建动态神经场景图,在局部规范空间中分别建模背景、刚体车辆、SMPL驱动的人体及无模板动态体。Waymo及另5个数据集实验显示其在重建和新视角合成上优于既有方法,全文PSNR分别提升约1.88和2.38,并能约60Hz仿真人车交互场景。

3D Reconstruction with Spatial Memory Figure 1
3DV 20252024-08-28

3D Reconstruction with Spatial Memory

Hengyi Wang, Lourdes Agapito

University College London, Department of Computer Science, University College London, Department of Computer Science

三维重建同步定位与建图

传统 SfM/MVS/SLAM 管线依赖匹配、位姿估计和全局优化,难以在线扩展;DUSt3R 虽能直接回归点图,但多图仍需逐场景对齐。Spann3R 的核心是引入外部空间记忆,用 Transformer 将历史三维预测编码并供下一帧查询,从而直接在统一坐标系中增量生成点图。实验显示其在未见数据集上保持有竞争力的稠密重建质量,并可在有序图像序列上超过 50fps、无需测试时优化。

Splatt3R: Zero-shot Gaussian Splatting from Uncalibrated Image Pairs Figure 1
arXiv preprint2024-08-25

Splatt3R: Zero-shot Gaussian Splatting from Uncalibrated Image Pairs

Victor Adrian

Active Vision Lab, University of Oxford, Active Vision Lab, University of Oxford, Visual Geometry Group, University of Oxford

三维重建渲染

Splatt3R瞄准双目自然图像缺少相机标定时,传统NeRF/3DGS需多视角和逐场景优化、难以随手重建的问题。它在MASt3R点云基础上直接预测每点的高斯属性,并先用几何监督再做新视角渲染训练,配合基于视锥与共视性的损失掩码处理外推视角。实验显示其可从未标定图像对零样本生成3D Gaussian,512²下约4FPS重建并实时渲染,视觉质量优于构造的MASt3R和前馈splatting基线。

Subsurface Scattering for 3D Gaussian Splatting Figure 1
arXiv preprint2024-08-22

Subsurface Scattering for 3D Gaussian Splatting

Hendrik P. A

University of Tübingen

重光照渲染

针对蜡、玉石、皮肤等半透明材料中次表面散射难以被传统 3D Gaussian Splatting 表达的问题,本文将显式高斯表面、空间变化 BRDF 与隐式散射辐射场结合,并用学习的入射光场和延迟着色处理阴影与高光。在多视角 OLAT 数据上联合优化后,可交互地进行新视角合成、重光照和材质编辑;合成与真实光场数据实验显示,其效果接近或优于 NeRF 类 SSS 方法,同时训练和渲染开销显著更低。

GSLoc: Efficient Camera Pose Refinement via 3D Gaussian Splatting Figure 1
arXiv preprint2024-08-20

GSLoc: Efficient Camera Pose Refinement via 3D Gaussian Splatting

Changkun Liu, Shuai Chen, Yash Bhalgat, Siyan Hu, Ming Cheng, Zirui Wang, Victor Adrian Prisacariu, Tristan Braud

University of Oxford, Dartmouth College, manner, achieving greater accuracy compared to the iterative neural refinement method, such as, achieving greater accuracy compared to the iterative neural refinement method

位姿估计

针对APR精度不足、SCR计算较重以及NeRF式位姿细化收敛慢且需训练专用特征的问题,GS-CPR用3D Gaussian Splatting快速渲染合成RGB与深度,并借助MASt3R在RGB间匹配以建立2D-3D对应,加入曝光自适应增强户外鲁棒性。该方法可在单张查询图和粗初始位姿下做一次性细化,在7Scenes、12Scenes和Cambridge Landmarks上提升APR/SCR精度,并在两个室内数据集达到新SOTA且快于NeRF方案。

FlashGS: Efficient 3D Gaussian Splatting for Large-scale and High-resolution Rendering Figure 1
CVPR 20252024-08-15

FlashGS: Efficient 3D Gaussian Splatting for Large-scale and High-resolution Rendering

Guofeng Feng, Siyan Chen, Rong Fu, Zimu Liao, Yi Wang, Tao Liu, Boni Hu, Lining Xu, Zhilin Pei, Hengjie Li, Xiuhong Li, Ninghui Sun, Xingcheng Zhang, Bo Dai

Institute of Computing Technology, CAS, Institute of Computing Technology, Independent Researcher, Shanghai Artificial Intelligence Laboratory, Beijing Academy of Artificial Intelligence, The Chinese University of Hong Kong, Chinese University of Hong Kong, Peking University, The University of Hong Kong, University of Hong Kong

加速训练

FlashGS面向大规模、高分辨率3D Gaussian Splatting在单张消费级GPU上受计算与显存限制的问题,先剖析原始光栅化流程中的无效Gaussian-tile配对、体渲染冗余计算和访存瓶颈,再通过精确冗余消除、重设计渲染流水线、线程调度与预取、显存层级和汇编级优化实现加速。实验显示其在多类合成与真实场景中保持PSNR基本不变,平均约4倍提速,并最高减少49%显存。

HeadGAP: Few-shot 3D Head Avatar via Generalizable Gaussian Priors Figure 1
3DV 20252024-08-12

HeadGAP: Few-shot 3D Head Avatar via Generalizable Gaussian Priors

Xiaozheng Zheng, Chao Wen, Zhaohu Li, Weiyi Zhang, Zhuo Su, Xu Chang, Yang Zhao, Zheng Lv, Xiaoyuan Zhang, Yongjie Zhang, Guidong Wang, Lan Xu

ShanghaiTech University

数字人动态场景

HeadGAP针对少量野外图像难以重建高保真、可稳定驱动3D头部数字人的问题,先在大规模多视角动态数据上学习可泛化的3D Gaussian头部先验,再通过基于3DGS的auto-decoder、分部动态建模、身份潜码反演与微调完成个性化。实验表明其在少样本设置下提升照片级渲染、多视角一致性和动画稳定性,并可用于消费级设备拍摄数据。

FMGS: Foundation Model Embedded 3D Gaussian Splatting for Holistic 3D Scene Understanding Figure 1
International Journal of Computer Vision2024-08-12

FMGS: Foundation Model Embedded 3D Gaussian Splatting for Holistic 3D Scene Understanding

Xingxing Zuo, Pouya Samangouei, Yunwen Zhou, Yan Di, Mingyang Li

Google, Mountain View, USA, Google, Google (United States)

语言嵌入分割

FMGS针对机器人/AR中仅有几何重建或闭集语义不足的问题,将CLIP/DINO等基础模型特征蒸馏进3D Gaussian Splatting,并用多分辨率哈希编码避免为海量高斯逐个存特征;多视角训练与像素对齐损失提升开放词汇查询的空间一致性。实验显示其在语言驱动3D目标检测上较已有方法提升约10.2%,推理速度快851倍。

Query3D: LLM-Powered Open-Vocabulary Scene Segmentation with Language Embedded 3D Gaussian Figure 1
arXiv preprint2024-08-07

Query3D: LLM-Powered Open-Vocabulary Scene Segmentation with Language Embedded 3D Gaussian

Amirhosein Chahe, Lifeng Zhou

Drexel University

语言嵌入分割

面向自动驾驶中的开放词汇3D场景查询,Query3D针对LE3DGS/LERF依赖固定短语、难处理上下文问题,引入LLM在推理时生成查询词、情境化canonical phrases和辅助正词,并用GPT-3.5构造数据微调小模型以便车端部署。在WayveScenes101上,LLM引导分割优于固定短语方法;小模型接近专家模型且更快,但辅助正词收益与模型规模相关,可能部分来自scaling/data。

LumiGauss: Relightable Gaussian Splatting in the Wild Figure 1
WACV 20252024-08-06

LumiGauss: Relightable Gaussian Splatting in the Wild

Joanna Kaleta, Kacper Kania, Marek Kowalski

Warsaw University of Technology, Microsoft, Microsoft Research (United Kingdom)

野外场景重光照

LumiGauss面向野外无约束照片中几何、反照率与光照难以解耦且NeRF式重光照难以实时集成的问题,将2D Gaussian Splatting改造成逆图形管线,并用球谐表示环境光和每个splat的辐射传输以建模阴影。作者在NeRF-OSR上报告优于基线的重建与重光照质量,并可用未见环境贴图合成较真实结果,适合接入游戏引擎。

Point'n Move: Interactive Scene Object Manipulation on Gaussian Splatting Radiance Fields Figure 1
IET Image Processing 20242024-07-26

Point'n Move: Interactive Scene Object Manipulation on Gaussian Splatting Radiance Fields

Jiajun Huang, Hongchuan Yu, Jianjun Zhang

National Centre for Computer Animation, Fern Barrow Poole Dorset UK, National Centre for Computer Animation, National Centre for Computer Animation, Fern Barrow, Poole, Dorset, UK

编辑

面向NeRF/辐射场场景中物体难以直观选择、移动后孔洞需重训修补的问题,Point’n Move利用3D Gaussian Splatting的显式点云式表示,从2D点击自提示传播到3D掩码,并结合掩码细化、显露区域剪枝与重投影初始化进行补全;在前向和360°场景中实现无需逐次训练的实时平移、旋转、移除,移除质量优于或接近现有方法且补全更快。

3D Gaussian Splatting: Survey, Technologies, Challenges, and Opportunities Figure 1
IEEE Transactions on Circuits and Systems for Video Technology2024-07-24

3D Gaussian Splatting: Survey, Technologies, Challenges, and Opportunities

Yanqi Bao, Tianyu Ding, Jing Huo, Yaoli Liu, Yuxin Li, Wenbin Li, Yang Gao, Jiebo Luo

State Key Laboratory for Novel Software Technology, Nanjing University, Nanjing, China, State Key Laboratory for Novel Software Technology, Nanjing University, State Key Laboratory for Novel Software Technology, Nanjing University, China, Applied Sciences Group, Microsoft Corporation, Redmond, WA, USA, Microsoft Corporation, Applied Sciences Group, Microsoft Corporation, Redmond, USA, Microsoft (United States), Department of Computer Science, University of Rochester, Rochester, NY, USA, Department of Computer Science, University of Rochester, Department of Computer Science, University of Rochester, America

综述

针对3DGS在实时新视角合成中迅速扩张、但任务分类与技术脉络分散的问题,本文从任务与技术两层重组已有工作:按优化、应用、扩展梳理3DGS变体,并归纳初始化、属性配置、正则化等九类技术模块的改进。主要结果是建立下游任务与共性技术之间的对应关系,总结四类核心挑战并给出未来机会,同时提供持续更新的3DGS论文仓库。

GaussianGrasper: 3D Language Gaussian Splatting for Open-vocabulary Robotic Grasping Figure 1
IEEE RA-L 20242024-07-23

GaussianGrasper: 3D Language Gaussian Splatting for Open-vocabulary Robotic Grasping

Yuhang Zheng, Xiangyu Chen, Yupeng Zheng, Songen Gu, Runyi Yang, Bu Jin, Pengfei Li, Chengliang Zhong, Zengmao Wang, Lina Liu, Chao Yang, Dawei Wang, Zhen Chen, Xiaoxiao Long, Meiqing Wang

Institute of Automation, Chinese Academy of Sciences (CASIA), Haidian, China, Institute of Automation, Chinese Academy of Sciences (CASIA), SMEA, Beihang University and EncoSmart, Haidian, China, Beihang University and EncoSmart, Chinese Academy of Sciences, Beihang University, AIR, Tsinghua University, Haidian, China, Tsinghua University, Imperial College London, London, U.K, Imperial College London, Wuhan University, Wuhan, China

语言嵌入机器人分割

面向语言指令抓取,论文针对 NeRF 式语言特征场视角需求多、推理慢且定位边界模糊的问题,提出用 3D Gaussian Splatting 显式表示场景;通过 EFD 将 SAM/CLIP 特征高效蒸馏到高斯基元,并利用渲染法线筛选力闭合抓取。真实桌面实验表明,该系统可用少量 RGB-D 视角完成开放词汇目标查询、定位与抓取,并支持操作后的场景更新。

TIP-Editor: An Accurate 3D Editor Following Both Text-Prompts And Image-Prompts Figure 1
ACM TOG 20242024-07-19

TIP-Editor: An Accurate 3D Editor Following Both Text-Prompts And Image-Prompts

Jingyu Zhuang, Di Kang, Yan-Pei Cao, Guanbin Li, Liang Lin, Ying Shan

Sun Yat-sen University, Guangzhou, China, Sun Yat-sen University, Tencent AI Lab, Guangzhou, China, Tencent AI Lab, Tencent (China), Tencent AI Lab, Shenzhen, China, Research Institute of Multiple Agents and Embodied Intelligence, Peng Cheng Laboratory, Shenzhen, China, Research Institute of Multiple Agents and Embodied Intelligence, Peng Cheng Laboratory

编辑

TIP-Editor针对纯文本3D编辑难以精确指定外观与位置的问题,引入文本提示、参考图像和3D包围盒联合控制。方法采用分步2D个性化学习场景与目标内容,并用定位损失约束编辑区域,同时基于3D Gaussian Splatting实现局部修改和背景保持。实验覆盖物体、人脸与户外场景,在编辑质量、提示对齐和用户偏好上优于基线。

StopThePop: Sorted Gaussian Splatting for View-Consistent Real-time Rendering Figure 1
ACM TOG 20242024-07-19

StopThePop: Sorted Gaussian Splatting for View-Consistent Real-time Rendering

Lukas Radl, Michael Steiner, Mathias Parger, Alexander Weinrauch, Bernhard Kerbl, Markus Steinberger

Graz University of Technology, Graz, Austria, Graz University of Technology, Huawei Technologies, Graz, Austria, Huawei Technologies

渲染

3D Gaussian Splatting 为实时新视角渲染牺牲了深度精度,用高斯中心的全局视深排序会在相机旋转时产生 popping 和混合不一致。StopThePop 的关键是用分层光栅化在相邻像素/层级间复用相干性,交替进行剔除、深度估计与局部重排序,近似实现逐像素排序而避免全量排序开销。结果显示其视觉上接近完整逐像素排序,平均仅比原 3DGS 慢约 4%;在一致性约束下可减少约一半高斯,质量近似不变,最终约 1.6× 更快且显存减半。

VR-GS: A Physical Dynamics-Aware Interactive Gaussian Splatting System in Virtual Reality Figure 1
arXiv preprint2024-07-12

VR-GS: A Physical Dynamics-Aware Interactive Gaussian Splatting System in Virtual Reality

Ying Jiang, Chang Yu, Tianyi Xie, Xuan Li, Yutao Feng, Huamin Wang, Minchen Li, Henry Lau, Feng Gao, Yin Yang, Chenfanfu Jiang

University of Hong Kong, Hong Kong and UCLA, United States of America, University of Hong Kong, University of Utah, United States of America and Zhejiang University, China, University of Utah, United States of America and Zhejiang University, United States University, Zhejiang University, Style3D Research, China, Style3D Research, Carnegie Mellon University, United States of America, Carnegie Mellon University, University of Hong Kong, Hong Kong

网格重建物理建模虚拟现实

VR-GS面向传统3D内容编辑门槛高、NeRF难以低延迟交互和变形的问题,将显式3D Gaussian Splatting接入VR物理交互。核心是为分割后的GS构建四面体笼,并用XPBD驱动可变形仿真,再通过两级嵌入把平滑形变传回高斯,减少尖刺伪影。系统还结合重建、分割、补全和动态阴影,展示了可实时进行物理合理编辑的沉浸式效果。

RTG-SLAM: Real-time 3D Reconstruction at Scale using Gaussian Splatting Figure 1
arXiv preprint2024-07-12

RTG-SLAM: Real-time 3D Reconstruction at Scale using Gaussian Splatting

Zhexi Peng, Tianjia Shao, Yong Liu, Jingke Zhou, Yin Yang, Jingdong Wang, Kun Zhou

State Key Lab of CAD&CG, Zhejiang University, China, State Key Lab of CAD&CG, Zhejiang University, University of Utah, United States of America, University of Utah, Baidu Research, China, Baidu Research, Baidu (China)

同步定位与建图

面向大规模 RGB-D 实时重建中 NeRF-SLAM 渲染慢、显存高以及传统 SLAM 外观真实感不足的问题,RTG-SLAM 将场景约束为不透明/近透明高斯,并用独立深度渲染让单个不透明高斯覆盖局部表面;同时只增补新观测、高颜色误差和高深度误差像素的高斯,并仅优化不稳定高斯。实验显示其在大场景中达到实时重建,质量接近或优于 NeRF-SLAM,速度约提升 2 倍、内存约减半。

MonoGaussianAvatar: Monocular Gaussian Point-based Head Avatar Figure 1
arXiv preprint2024-07-12

MonoGaussianAvatar: Monocular Gaussian Point-based Head Avatar

Yufan Chen, Lizhen Wang, Qijing Li, Hongjiang Xiao, Shengping Zhang, Hongxun Yao, Yebin Liu

Harbin Institute of Technology, China, Harbin Institute of Technology, Tsinghua University, China, Tsinghua University, Communication University of China, China, Communication University of China

数字人单目重建

面向从单目人像视频构建可驱动、写实头部数字人的需求,MonoGaussianAvatar针对3DMM拓扑固定、NeRF训练渲染慢、PointAvatar点数负担和运动空洞等问题,引入可调形状的3D Gaussian点表示与连续高斯形变场,并设计两阶段初始化及点增删策略以保留头发、眼镜等结构。实验在SSIM、图像相似度和PSNR等指标上优于既有方法,达到当时SOTA。

4D Gaussian Splatting: Towards Efficient Novel View Synthesis for Dynamic Scenes Figure 1
arXiv preprint2024-07-12

4D Gaussian Splatting: Towards Efficient Novel View Synthesis for Dynamic Scenes

Yuanxing Duan, Fangyin Wei, Qiyu Dai, Yuhang He, Wenzheng Chen, Baoquan Chen

Peking University, China, Peking University, Princeton University, United States of America, Princeton University, Peking University, China and State Key Laboratory of General Artificial Intelligence, China, China and State Key Laboratory of General Artificial Intelligence, NVIDIA, Canada and Peking University, China, NVIDIA, Canada and Peking University

动态场景

面向动态场景新视角合成中复杂运动、突现消失与实时渲染难兼得的问题,本文将3DGS提升为显式XYZT各向异性4D高斯,用时间切片得到每帧3D高斯,并以4D rotor表示时空旋转,配合熵损失、4D一致性损失和CUDA实现。实验显示其在多类运动场景中质量优于既有方法,RTX 3090/4090上分别可达277/583 FPS。

2D Gaussian Splatting for Geometrically Accurate Radiance Fields Figure 1
arXiv preprint2024-07-12

2D Gaussian Splatting for Geometrically Accurate Radiance Fields

Binbin Huang, Zehao Yu, Anpei Chen, Andreas Geiger, Shenghua Gao

ShanghaiTech University, China, ShanghaiTech University, University of Tübingen, Germany, University of Tübingen

二维高斯网格重建

针对3DGS虽能实时高质量渲染但体高斯与薄表面不匹配、导致多视角几何不一致的问题,本文将场景表示“压扁”为带朝向的2D椭圆高斯盘,并通过射线-面片相交的透视正确splatting、深度畸变与法线一致性正则来稳定优化。结果显示其可从多视图图像重建更干净细致的网格,在几何精度上优于现有显式表示,同时保持接近3DGS的外观质量、训练速度和实时渲染能力。

3D Gaussian Ray Tracing: Fast Tracing of Particle Scenes Figure 1
ACM TOG 20242024-07-09

3D Gaussian Ray Tracing: Fast Tracing of Particle Scenes

Nicolas Moenne-Loccoz, Ashkan Mirzaei, Or Perel, Riccardo de Lutio, Janick Martinez Esturo, Gavriel State, Sanja Fidler, Nicholas Sharp, Zan Gojcic

NVIDIA, Montreal, Canada, NVIDIA, NVIDIA, Toronto, Canada, University of Toronto, Toronto, Canada, University of Toronto, NVIDIA, Tel Aviv, Israel, NVIDIA, San Francisco, United States of America, Nvidia (United States), NVIDIA, Munich, Germany, NVIDIA, Seattle, United States of America, Seattle University, NVIDIA, Zurich, Switzerland

光线追踪

这篇论文针对 3D Gaussian Splatting 依赖光栅化、难以处理畸变相机、滚动快门及反射/阴影等非相干二次光线的问题,改用 GPU 光线追踪来渲染半透明粒子。核心做法是为高斯粒子构建 BVH 和包围网格代理,结合 k-buffer 按深度批量着色,并支持反向优化;同时引入广义核以减少命中次数。实验显示其画质接近或超过原 3DGS 光栅器,仍可实时渲染,并能自然支持景深、镜面、阴影和随机采样训练等应用。

Recent Advances in 3D Gaussian Splatting Figure 1
Computational Visual Media2024-07-07

Recent Advances in 3D Gaussian Splatting

Tong Wu, Yu-Jie Yuan, Ling-Xiao Zhang, Jie Yang, Yan-Pei Cao, Ling-Qi Yan, Lin Gao

Institute of Computing Technology, Chinese Academy of Sciences, Beijing 100190, China, Institute of Computing Technology, Chinese Academy of Sciences, Institute of Computing Technology, Chinese Academy of Sciences, Beijing, 100190, China, Tencent AI Lab, Beijing 100089, China; VAST, Beijing 100000, China, Tencent AI Lab, Tencent AI Lab, Beijing, 100089, China, Tencent (China), Department of Computer Science, University of California, Santa Barbara, CA 93106, USA, Department of Computer Science, University of California, Department of Computer Science, University of California, Santa Barbara, CA, 93106, USA

综述

面对3DGS论文快速增长、NeRF虽高质但训练与渲染较慢的问题,本文从点基渲染源流出发,梳理3DGS以显式高斯椭球和光栅化替代密集采样的核心优势,并按三维重建、编辑和下游应用归纳近期工作。综述指出3DGS可在约30分钟收敛并实现1080p实时渲染,推动大场景、SLAM、数字人等应用,但在稀疏视角、复杂光照、几何精度和独立编辑上仍有明显挑战。

Segment Any 4D Gaussians Figure 1
arXiv preprint2024-07-05

Segment Any 4D Gaussians

Shengxiang Ji, Guanjun Wu, Jiemin Fang, Jiazhong Cen, Taoran Yi, Wenyu Liu, Qi Tian, Xinggang Wang

School of CS, Huazhong University of Science and Technology, School of CS, Huazhong University of Science and Technology, Huawei Inc, MoE Key Lab of Artificial Intelligence, AI Institute, Shanghai Jiao Tong University, MoE Key Lab of Artificial Intelligence, AI Institute, Shanghai Jiao Tong University, School of EIC, Huazhong University of Science and Technology, School of EIC

分割

面向动态4D场景中缺少可交互对象级分割的问题,SA4D将SAM式分割提升到4D Gaussian表示;其关键是用时序身份特征场从稀疏且噪声的视频跟踪掩码中学习随时间变化的高斯身份,以缓解非刚体运动下的Gaussian drifting,并通过4D分割细化去伪影。论文报告在RTX 3090上约10秒内完成高质量分割,并支持动态对象移除、重着色、组合与高质量mask渲染。

Optimizing Dynamic NeRF and 3DGS with No Video Synchronization Figure 1
OpenReview2024-07-01

Optimizing Dynamic NeRF and 3DGS with No Video Synchronization

Seoha Kim, Jeongmin Bae, Youngsik Yun, HyunSeung Son, Hahyun Lee, Gun Bang, Youngjung Uh

Yonsei University, Seoul 03722, Korea, Yonsei University, Electronics and Telecommunications Research Institute, Daejeon 34129, Korea, Electronics and Telecommunications Research Institute

动态场景

这篇论文针对多视角动态 NeRF/3DGS 默认视频同步而现实数据常有时间错位的问题,指出同一帧共享时间嵌入会把不同场景状态硬对齐,导致训练视角也出现鬼影和运动重建失败。方法为每个相机视频引入可学习时间偏移,并与辐射场联合优化,可接入离散嵌入、网格表示和 3DGS 等基线。在 Plenoptic Video 与新建非同步 Blender 数据集上,实验显示该校准能显著改善非同步视角拟合和动态重建质量。

GSCore: Efficient Radiance Field Rendering via Architectural Support for 3D Gaussian Splatting Figure 1
ASPLOS 20242024-07-01

GSCore: Efficient Radiance Field Rendering via Architectural Support for 3D Gaussian Splatting

Junseo Lee, Seokwon Lee, Jungi Lee, Junyong Park, Jaewoong Sim

Seoul National University, Seoul, Republic of Korea, Seoul National University

加速训练渲染

面向移动/VR等功耗受限场景,3D Gaussian Splatting虽快于 NeRF,但排序与光栅化仍难实时。论文通过分析发现朴素高斯-像素相交、无效排序和 alpha blending 中大量无贡献高斯是瓶颈,提出形状感知相交测试、分层排序与子 tile 跳过,并设计专用加速器 GSCore。28nm 综合与周期级仿真显示,其在多类场景上相对移动 GPU 平均加速 15.86×,面积约 3.95mm²且能耗更低。

GS-Octree: Octree-based 3D Gaussian Splatting for Robust Object-level 3D Reconstruction Under Strong Lighting Figure 1
Computer Graphics Forum 20242024-06-26

GS-Octree: Octree-based 3D Gaussian Splatting for Robust Object-level 3D Reconstruction Under Strong Lighting

J. Li, Z. Wen, L. Zhang, J. Hu, F. Hou, Z. Zhang, Y. He

College of Computing and Data Science Nanyang Technological University Singapore, College of Computing and Data Science, Nanyang Technological University, Singapore, College of Computing and Data Science, Nanyang Technological University, Center for Health and Gender Equity, International School of Information Science and Engineering Dalian University of Technology China, International School of Information Science and Engineering, Dalian University of Technology, China, International School of Information Science and Engineering, Dalian University of Technology, Dalian University, Key Laboratory of System Software (CAS) and State Key Laboratory of Computer Science ISCAS China, University of Chinese Academy of Sciences China

网格重建

GS-Octree针对3D Gaussian Splatting在强光、高光下几何约束不足、表面易出洞的问题,将八叉树SDF隐式表面与高斯点联合优化:先用低分辨率八叉树重建粗SDF,再让SDF引导高斯、由高斯反向细化SDF并剪除低贡献高斯。实验显示其在物体级重建中几何更准确,尤其能缓解强光镜面高光造成的伪影,同时减少超过50%的高斯数量并保持较快渲染。

Dynamic Gaussian Marbles for Novel View Synthesis of Casual Monocular Videos Figure 1
arXiv preprint2024-06-26

Dynamic Gaussian Marbles for Novel View Synthesis of Casual Monocular Videos

Colton Stearns, Adam Harley, Mikaela Uy, Florian Dubost, Federico Tombari, Gordon Wetzstein, Leonidas Guibas

Stanford University, Stanford, United States of America, Stanford University, Stanford, Google Research, Mountain View, United States of America, Google Research, Google (United States), Google Research, Munich, Germany

动态场景单目重建

本文面向随手拍单目动态视频的新视角合成,指出现有4D Gaussian在缺少多视角监督时约束不足、易失败。方法以各向同性“marbles”降低自由度,结合分治式轨迹合并和深度、分割、点跟踪及几何正则,引导全局一致运动优化。在Nvidia Dynamic Scenes、DyCheck等数据上优于Gaussian基线,质量接近NeRF类方法,并保留快速渲染、跟踪和可编辑性。

Reducing the Memory Footprint of 3D Gaussian Splatting Figure 1
Proceedings of the ACM on Computer Graphics and Interactive Techniques2024-06-24

Reducing the Memory Footprint of 3D Gaussian Splatting

Panagiotis Papantonakis, Georgios Kopanas, Bernhard Kerbl, Alexandre Lanvin, George Drettakis

Inria, Université Côte d'Azur, France, Inria, Inria, Université Côte d'Azur, France and TU Wien, Austria

压缩

这篇论文针对 3D Gaussian Splatting 场景文件动辄数百 MB 到 1GB、难以存储和移动端传输的问题,指出冗余主要来自过多高斯基元、统一使用高阶球谐和过高属性精度。方法结合分辨率感知剪枝、自适应 SH 阶数选择与码本/半精度量化,在标准数据集上将磁盘体积压缩约 27 倍,同时渲染速度提升约 1.7 倍,并显著缩短移动端下载时间。

Splatter a Video: Video Gaussian Representation for Versatile Processing Figure 1
arXiv preprint2024-06-19

Splatter a Video: Video Gaussian Representation for Versatile Processing

Yang-Tian Sun, Yi-Hua Huang, Lin Ma, Xiaoyang Lyu, Yan-Pei Cao, Xiaojuan Qi

The University of Hong Kong

三维高斯泼溅

该文针对视频像素/2.5D表示难以处理复杂遮挡和依赖隐式3D不便编辑的问题,提出视频高斯表示:在规范3D空间用显式高斯建模外观,并为每个高斯绑定随时间变化的3D运动,再用光流、单目深度等基础模型先验蒸馏约束学习。实验展示其可用于密集跟踪、跨帧一致深度/特征优化、几何与外观编辑、插帧、一定程度的新视角合成和立体视频生成。

SplatPose & Detect: Pose-Agnostic 3D Anomaly Detection Figure 1
CVPR 20242024-06-17

SplatPose & Detect: Pose-Agnostic 3D Anomaly Detection

Mathis Kruse, Marco Rudolph, Dominik Woiwode, Bodo Rosenhahn

Leibniz University, Institute for Information Processing, Hannover, Leibniz University, Institute for Information Processing, Leibniz University Hannover

其他位姿估计

针对工业异常检测中物体姿态变化导致2D方法失效、NeRF方案计算开销过高的问题,本文提出SplatPose,用3D Gaussian Splatting显式建模正常物体,并通过可微点云变换估计未知视角姿态,再与对齐渲染图做特征匹配定位异常。在MAD基准上取得检测与分割新SOTA,训练最高快55倍、推理快13倍,且用60%训练视角仍优于使用全量数据的对手。

Gaussian Splatting Decoder for 3D‑aware Generative Adversarial Networks Figure 1
CVPR 20242024-06-17

Gaussian Splatting Decoder for 3D‑aware Generative Adversarial Networks

Florian Barthel, Arian Beckmann, Wieland Morgenstern, Anna Hilsmann, Peter Eisert

Fraunhofer Heinrich Hertz Institute, HHI, Fraunhofer Heinrich Hertz Institute, Fraunhofer Institute for Telecommunications, Heinrich Hertz Institute, Fraunhofer Institute for Telecommunications, Heinrich Hertz Institute

数字人

这篇工作针对EG3D等NeRF式3D GAN渲染慢、难导入Unity/Blender等显式3D环境的问题,提出从GAN的tri-plane隐式特征一次性解码出3D Gaussian Splatting的位置、颜色、尺度、旋转和透明度,并配合人头区域采样、顺序解码器和骨干微调,避免逐场景优化与超分模块。结果表明其可生成可导出的高质量数字人头3DGS资产,支持高分辨率GAN反演和实时编辑。

Depth-Regularized Optimization for 3D Gaussian Splatting in Few-Shot Images Figure 1
CVPR 20242024-06-17

Depth-Regularized Optimization for 3D Gaussian Splatting in Few-Shot Images

Jaeyoung Chung, Jeongtaek Oh, Kyoung Mu Lee

ASRI, Department of ECE, Department of ECE, Seoul National University, IPAI, Seoul, Korea, Seoul National University

稀疏表示

本文针对少量视图下 3D Gaussian Splatting 易过拟合训练视角、产生漂浮伪影的问题,引入单目深度先验作为几何正则:用 COLMAP 稀疏点对预训练深度图做尺度与偏移对齐,并结合深度损失、平滑约束和基于深度损失的早停。作者在 NeRF-LLFF few-shot 设置中验证,相比原始 3DGS 可获得更稳定的几何和更合理的新视角结果,但效果仍依赖单目深度与 COLMAP 质量。

A Hierarchical 3D Gaussian Representation for Real-Time Rendering of Very Large Datasets Figure 1
ACM TOG 20242024-06-17

A Hierarchical 3D Gaussian Representation for Real-Time Rendering of Very Large Datasets

Bernhard Kerbl, Andreas Meuleman, Georgios Kopanas, Michael Wimmer, Alexandre Lanvin, George Drettakis

Inria, Université Nice Cote d'Azur, Sophia-Antipolis, France, Inria

大规模场景

针对原始 3D Gaussian Splatting 在公里级街景中训练与渲染显存/算力不可承受的问题,论文提出按场景分块训练、再合并为可优化的层次化 3D Gaussian,并用动态 LOD 做远景选择与平滑过渡。该方法在数千米轨迹、最多约 2.8 万张图像的数据上实现实时新视角浏览,主要贡献在于把 3DGS 扩展到超大规模场景。

pixelSplat: 3D Gaussian Splats from Image Pairs for Scalable Generalizable 3D Reconstruction Figure 1
CVPR 20242024-06-16

pixelSplat: 3D Gaussian Splats from Image Pairs for Scalable Generalizable 3D Reconstruction

David Charatan, Sizhe Lester Li, Andrea Tagliasacchi, Vincent Sitzmann

Massachusetts Institute of Technology, Simon Fraser University, University of Toronto, Simon Fraser University, University of Toronto

前馈重建稀疏表示

pixelSplat面向稀疏图像输入下的可泛化新视角合成,试图摆脱NeRF/光场Transformer训练与渲染开销高、缺少显式3D结构的问题。其关键做法是从两张图像前馈预测3D Gaussian场,并用密集3D概率分布与可微重参数化采样来缓解原语优化的局部极小,同时用极线Transformer估计场景尺度。在RealEstate10k和ACID宽基线设置中,它优于光场Transformer,渲染加速约2.5个数量级,并输出可解释、可编辑的3D表示。

VastGaussian: Vast 3D Gaussians for Large Scene Reconstruction Figure 1
CVPR 20242024-06-16

VastGaussian: Vast 3D Gaussians for Large Scene Reconstruction

Jiaqi Lin, Zhihao Li, Xiao Tang, Jianzhuang Liu, Shiyong Liu, Jiayue Liu, Yangdi Lu, Xiaofei Wu, Songcen Xu, Youliang Yan, Wenming Yang

Tsinghua University, Huawei Noah's Ark Lab, Huawei Technologies (Sweden), Shenzhen Institute of Advanced Technology, Shenzhen Institutes of Advanced Technology

大规模场景

VastGaussian针对3D Gaussian Splatting在大规模场景中显存受限、整体优化慢且光照/曝光变化易产生漂浮伪影的问题,采用渐进式场景分块,并基于空域感知可见性为各cell分配相机和点云,支持并行训练后无缝合并;同时在训练阶段引入解耦外观建模,推理时丢弃以保持实时渲染。实验显示其在多个大场景数据集上优于NeRF类方法,取得更高保真重建和更快渲染。

Triplane Meets Gaussian Splatting: Fast and Generalizable Single-View 3D Reconstruction with Transformers Figure 1
CVPR 20242024-06-16

Triplane Meets Gaussian Splatting: Fast and Generalizable Single-View 3D Reconstruction with Transformers

Zhipeng Yu, Yangguang Li, Ding Liang

BNRist, Tsinghua University, Tsinghua University, Qinghai University

其他

针对单图三维重建中扩散/SDS方法需逐物体优化、隐式体渲染慢而显式高斯又难直接回归的问题,TGS用Transformer点解码器先给出粗点云,再由triplane解码器查询并生成高斯属性,实现位置显式、外观与细节隐式的混合表示。实验显示其在合成与真实图像上重建和新视角渲染质量优于已有方法,并将重建降到秒级。

Text-to-3D using Gaussian Splatting Figure 1
CVPR 20242024-06-16

Text-to-3D using Gaussian Splatting

Zilong Chen, Feng Wang, Yikai Wang, Huaping Liu

Tsinghua University, Beijing National Research Center for Information Science and Technology (BNRist), Department of Computer Science and Technology, Tsinghua University, Beijing National Research Center for Information Science and Technology (BNRist), Department of Computer Science and Technology

扩散生成

针对SDS式文本到3D常见的几何塌陷、Janus多脸问题和细节生成慢,本文提出Gsgen,用显式3D Gaussian Splatting替代隐式表示,并在优化中引入点云扩散的3D先验。方法先用2D/3D扩散共同约束粗几何,再通过紧致性驱动的高斯增密细化外观。实验显示其在几何一致性、纹理保真和毛发等高频细节上优于多种基线,但复杂语义提示仍受Point-E和CLIP理解能力限制。

SuGaR: Surface-Aligned Gaussian Splatting for Efficient 3D Mesh Reconstruction and High-Quality Mesh Rendering Figure 1
CVPR 20242024-06-16

SuGaR: Surface-Aligned Gaussian Splatting for Efficient 3D Mesh Reconstruction and High-Quality Mesh Rendering

Antoine Guédon, Vincent Lepetit

Centre National de la Recherche Scientifique, Laboratoire d'Informatique Gaspard-Monge

网格重建

SuGaR针对3D Gaussian Splatting渲染快但高斯无序、难以转成可编辑网格的问题,提出表面对齐正则,使高斯贴合场景表面,并基于深度图采样等值面点后用Poisson重建网格;再将高斯绑定到网格联合优化。结果显示其可在数分钟内获得较准确网格,较依赖Neural SDF的方案更快,并保持或提升新视角渲染质量。

SplattingAvatar: Realistic Real-Time Human Avatars with Mesh-Embedded Gaussian Splatting Figure 1
CVPR 20242024-06-16

SplattingAvatar: Realistic Real-Time Human Avatars with Mesh-Embedded Gaussian Splatting

Zhijing Shao, Zhaolong Wang, Zhuang Li, Duotun Wang, Xiangru Lin, Yu Zhang, Mingming Fan, Zeyu Wang

The Hong Kong University of Science and Technology, Guangzhou, The Hong Kong University of Science and Technology, Prometheus Vision Technology Co., Ltd, Prometheus Vision Technology Co, Prometheus Research (United States), Prometheus (United States)

数字人

面向高真实感数字人实时渲染中网格难表细节、NeRF/MLP 变形又慢且有映射歧义的问题,SplattingAvatar 将 3D Gaussian 以重心坐标和法向位移嵌入三角网格,用网格显式驱动运动与形变、Gaussian 表达高频外观,并用 lifted optimization 联合优化可在网格上移动的嵌入点。该方法可由单目视频训练,在全身和头部数据集上取得领先画质,现代 GPU 超 300 FPS、移动端约 30 FPS,并兼容骨骼动画、blend shape 与网格编辑。

Splatter Image: Ultra-Fast Single-View 3D Reconstruction Figure 1
CVPR 20242024-06-16

Splatter Image: Ultra-Fast Single-View 3D Reconstruction

Stanislaw Szymanowicz, Christian Rupprecht, Andrea Vedaldi

Visual Geometry Group – University of Oxford, University of Oxford

前馈重建稀疏表示

针对单目3D重建中NeRF/三平面等方法渲染和训练开销大的问题,论文将Gaussian Splatting改造成前馈重建器:用2D U-Net为输入图像每个像素预测一个3D高斯,并以“Splatter Image”组织稀疏表示,可扩展到多视图交叉注意力融合。该设计在标准GPU上约38 FPS推理、588 FPS渲染,单/双GPU即可训练,并在ShapeNet、CO3D、Objaverse/GSO等数据集上达到或超过多种较慢方法的PSNR、LPIPS表现。

SplaTAM: Splat, Track & Map 3D Gaussians for Dense RGB-D SLAM Figure 1
CVPR 20242024-06-16

SplaTAM: Splat, Track & Map 3D Gaussians for Dense RGB-D SLAM

Jay Karhade, Gengshan Yang, Sebastian Scherer, Deva Ramanan, Jonathon Luiten

Moscow Institute of Thermal Technology

同步定位与建图

SplaTAM针对传统显式SLAM依赖几何特征、隐式辐射场优化慢且难以增量扩展的问题,将场景表示为可微渲染的3D高斯,并用轮廓掩码区分已建图区域与新区域,实现在线跟踪、高斯增密和地图更新。实验显示其在ScanNet++、Replica、TUM-RGBD等数据集上提升位姿估计、重建与新视角合成,部分指标较既有方法最高约2倍,同时支持高分辨率地图约400 FPS渲染。

Spacetime Gaussian Feature Splatting for Real-Time Dynamic View Synthesis Figure 1
CVPR 20242024-06-16

Spacetime Gaussian Feature Splatting for Real-Time Dynamic View Synthesis

Zhan Li, Zhang Chen, Zhong Li, Yi Xu

OPPO US Research Center

动态场景

面向动态场景新视角合成中“高画质、实时渲染、低存储”难以兼得的问题,论文将3D Gaussian扩展为带时间透明度和参数化运动/旋转的时空高斯,并用可splat的神经特征替代球谐以建模视角/时间相关外观,再以误差和粗深度引导补采样。实验显示其在多组真实多视角视频上达到SOTA质量与速度,lite版可在RTX 4090上8K 60FPS。

Scaffold-GS: Structured 3D Gaussians for View-Adaptive Rendering Figure 1
CVPR 20242024-06-16

Scaffold-GS: Structured 3D Gaussians for View-Adaptive Rendering

Tao Lu, Mulin Yu, Linning Xu, Yuanbo Xiangli, Limin Wang, Dahua Lin, Bo Dai

Shanghai Artificial Intelligence Laboratory, Beijing Academy of Artificial Intelligence, Nanjing University, Cornell University

密度控制渲染

针对 3D-GS 为拟合训练视角产生大量冗余高斯、对大视角变化和材质/光照效应鲁棒性不足的问题,Scaffold-GS 用稀疏锚点组织局部神经高斯,并按视线方向与距离动态预测颜色、透明度、尺度等属性,同时通过锚点生长/剪枝改进覆盖。实验显示其在多数据集上达到或超过 3D-GS 的渲染质量,显著降低存储量,并保持约 1K 分辨率 100 FPS 级实时速度。

Relightable Gaussian Codec Avatars Figure 1
CVPR 20242024-06-16

Relightable Gaussian Codec Avatars

Shunsuke Saito, Gabriel Schwartz, Tomas Simon, Junxuan Li, Giljoo Nam

Codec Avatars Lab, Meta, Codec Avatars Lab, Meta

数字人重光照

面向可动画数字人,论文针对头部重光照中发丝、皮肤毛孔与眼部高频反射难以实时统一建模的问题,提出以3D Gaussian表示可驱动几何,并用可学习辐射传输结合漫反射球谐与镜面球形高斯,外加显式可控眼模型。实验显示其在皮肤、头发和眼睛的全频环境光重光照质量上优于实时基线,并可在消费级VR头显上实时运行。

Reflective Gaussian Splatting Figure 1
CVPR 20242024-06-16

Reflective Gaussian Splatting

Yingwenqi Jiang, Jiadong Tu, Yuan Liu, Xifeng Gao, Xiaoxiao Long, Wenping Wang, Yuexin Ma

ShanghaiTech University, The University of Hong Kong, University of Hong Kong, Tencent America, Tencent (China), Texas A&M University

网格重建光线追踪重光照

针对 NeRF/3DGS 在高反射物体上难以同时实现实时、高质量渲染和互反射建模的问题,Ref-Gaussian 将物理延迟渲染、split-sum 近似与基于高斯/网格的互反射结合,并用 2D Gaussian、材质感知法线传播和逐高斯着色初始化改善几何。实验显示其在反射与非反射场景的画质、指标和效率上优于多种基线,并支持重光照与编辑。

PhysGaussian: Physics-Integrated 3D Gaussians for Generative Dynamics Figure 1
CVPR 20242024-06-16

PhysGaussian: Physics-Integrated 3D Gaussians for Generative Dynamics

Tianyi Xie, Zeshun Zong, Yuxing Qiu, Xuan Li, Yutao Feng, Yin Yang, Chenfanfu Jiang

Zhejiang University, University of Utah

物理建模

针对 NeRF/3DGS 动态生成常需先网格化或嵌入代理几何、导致仿真与渲染不一致的问题,PhysGaussian 将 3D Gaussian 同时作为可视化与 MPM 物理仿真的离散粒子,为高斯核加入速度、应变、应力和塑性等连续介质属性,实现“所见即所仿真”。实验展示其可生成弹性体、金属、非牛顿材料和颗粒介质的逼真新运动,简单场景可接近实时,但阴影演化和材料参数自动估计仍未充分解决。

Photo-SLAM: Real-time Simultaneous Localization and Photorealistic Mapping for Monocular, Stereo, and RGB-D Cameras Figure 1
CVPR 20242024-06-16

Photo-SLAM: Real-time Simultaneous Localization and Photorealistic Mapping for Monocular, Stereo, and RGB-D Cameras

Huajian Huang, Longwei Li, Hui Cheng, Sai-Kit Yeung

The Hong Kong University of Science and Technology, Hong Kong University of Science and Technology, Sun Yat-sen University

同步定位与建图

针对神经渲染式 SLAM 依赖隐式表示、计算开销大且难以上移动平台的问题,Photo-SLAM 用“超基元”地图同时承载 ORB 几何特征与可学习光度特征,结合 3D Gaussian Splatting、基于几何的增密和高斯金字塔训练,实现定位与照片级在线建图。实验覆盖单目、双目和 RGB-D,Replica 上 PSNR 提升约 30%、渲染快数百倍,并可在 Jetson AGX Orin 实时运行。

Neural Parametric Gaussians for Monocular Non-Rigid Object Reconstruction Figure 1
CVPR 20242024-06-16

Neural Parametric Gaussians for Monocular Non-Rigid Object Reconstruction

Devikalyan Das, Christopher Wewer, Raza Yunus, Eddy Ilg, Jan Eric Lenssen

Saarland University, Saarland Informatics Campus, Germany, Saarland University, Max Planck Institute for Informatics, Saarland Informatics Campus, Germany, Max Planck Institute for Informatics

动态场景单目重建

该文针对单目视频中非刚体物体重建欠约束、跨大视角新视图不稳定的问题,提出 Neural Parametric Gaussians:先学习低秩粗形变点模型以建立时序对应和正则,再在其驱动的局部有向体中优化共享 3D Gaussians 表达细节与外观。实验显示其在缺少多视角线索的真实动态序列上,相比既有动态 NeRF/重建方法能生成更一致、质量更高的新视图。

Multi-Scale 3D Gaussian Splatting for Anti-Aliased Rendering Figure 1
CVPR 20242024-06-16

Multi-Scale 3D Gaussian Splatting for Anti-Aliased Rendering

Zhiwen Yan, Weng Fei Low, Yu Chen, Gim Hee Lee

National University of Singapore, Department of Computer Science, National University of Singapore, Department of Computer Science

抗锯齿渲染

针对 3D Gaussian Splatting 在低分辨率或远距离视角下小高斯过度挤入像素导致走样、且 alpha 混合变慢的问题,论文引入类似 mipmap/LOD 的多尺度高斯表示:训练中聚合小高斯生成粗尺度大高斯,渲染时按像素覆盖选择合适尺度并过滤过大/过小高斯。在 Mip-NeRF360 等数据上,低分辨率渲染 PSNR 提升 13%-66%,速度提升 160%-2400%,同时基本保持原分辨率质量与速度。

Mip-Splatting Alias-free 3D Gaussian Splatting Figure 1
CVPR 20242024-06-16

Mip-Splatting Alias-free 3D Gaussian Splatting

Zehao Yu, Anpei Chen, Binbin Huang, Torsten Sattler, Andreas Geiger

University of Tübingen, TH Bingen University of Applied Sciences, ShanghaiTech University, Czech Technical University in Prague

抗锯齿渲染

针对 3DGS 在改变焦距、相机距离或渲染分辨率时出现膨胀、侵蚀和高频伪影的问题,Mip-Splatting 将根因归结为缺少 3D 频率约束与固定 2D dilation。方法用基于输入视图采样上限的 3D smoothing 约束高斯尺度,并以近似像素盒滤波的 2D Mip filter 替代 dilation。实验显示其在单尺度训练、多尺度测试的缩放场景中明显优于 3DGS/EWA,并保持接近实时的 3DGS 框架改动成本。

MANUS: Markerless Hand-Object Grasp Capture using Articulated 3D Gaussians Figure 1
CVPR 20242024-06-16

MANUS: Markerless Hand-Object Grasp Capture using Articulated 3D Gaussians

Chandradeep Pokhariya, Ishaan Nikhil Shah, Angela Xing, Zekun Li, Kefan Chen, Avinash Sharma, Srinath Sridhar

Indian Institute of Technology Hyderabad, Brown University, John Brown University

其他

面向机器人与混合现实中需要精确手物接触的抓取捕捉问题,MANUS用关节化3D Gaussian替代骨架、网格或MANO等模板表示,同时以静态Gaussian建模物体,使多视角像素对齐后的形状可直接高效计算瞬时与累积接触。作者还构建50+相机、约700万帧的MANUS-Grasps,并用颜料转移作接触评测,结果显示其接触精度优于模板方法;但性能可能较依赖大量视角和数据规模。

LucidDreamer: Towards High-Fidelity Text-to-3D Generation via Interval Score Matching Figure 1
CVPR 20242024-06-16

LucidDreamer: Towards High-Fidelity Text-to-3D Generation via Interval Score Matching

Yixun Liang, Xin Yang, Jiantao Lin, Haodong Li, Xiaogang Xu, Yingcong Chen

University of Hong Kong

扩散生成

该文针对文本到3D生成中SDS蒸馏易产生过平滑、细节不足的问题,指出根因在于随机扩散轨迹和一步伪GT重建带来不一致且低质量的更新方向。作者提出Interval Score Matching,用DDIM反演构造可逆确定轨迹,并在扩散区间内匹配分数;结合3D Gaussian Splatting后,在视觉质量和训练效率上超过Magic3D、Fantasia3D、ProlificDreamer等方法。

Language Embedded 3D Gaussians for Open-Vocabulary Scene Understanding Figure 1
CVPR 20242024-06-16

Language Embedded 3D Gaussians for Open-Vocabulary Scene Understanding

Jin-Chuan Shi, Miao Wang, Hao-Bin Duan, Shao-Hua Guan

State Key Laboratory of Virtual Reality Technology and Systems, SCSE, Beihang University, State Key Laboratory of Virtual Reality Technology and Systems, Beihang University

语言嵌入分割

该文针对开放词汇3D查询中,NeRF式语言场训练/渲染慢、而直接把高维CLIP/DINO语义嵌入3D Gaussian会带来巨大显存和性能下降的问题,提出Language Embedded 3D Gaussians:先量化多视角密集语言特征以压缩冗余语义,再用基于空间位置与不确定性的嵌入机制平滑跨视角不一致。实验显示其在新视角质量、语言查询mAP和速度上优于LERF、3DOVS等,并可在单桌面GPU实时渲染。

LangSplat: 3D Language Gaussian Splatting Figure 1
CVPR 20242024-06-16

LangSplat: 3D Language Gaussian Splatting

Minghan Qin, Wanhua Li, Jiawei Zhou, Haoqian Wang, Hanspeter Pfister

Tsinghua University, Harvard University, Harvard University Press

语言嵌入分割

针对 NeRF 语言场在开放词汇 3D 查询中渲染慢、物体边界模糊的问题,LangSplat 将 CLIP 语言特征嵌入 3D Gaussian Splatting,并用场景级自编码器压缩显式特征;同时借助 SAM 的层级掩码监督,缓解点对应多尺度语义的歧义。实验显示其在 3D 目标定位和语义分割上优于 LERF,并在 1440×1080 下实现约 199× 加速。

HumanGaussian: Text-Driven 3D Human Generation with Gaussian Splatting Figure 1
CVPR 20242024-06-16

HumanGaussian: Text-Driven 3D Human Generation with Gaussian Splatting

Xian Liu, Xiaohang Zhan, Jiaxiang Tang, Ying Shan, Gang Zeng, Dahua Lin, Xihui Liu, Ziwei Liu

University of Hong Kong, Tencent AI Lab, Tencent (China)

数字人扩散生成

面向文本到三维数字人的细节不足与优化耗时问题,HumanGaussian 将 3D Gaussian Splatting 引入人体生成,并用 SMPL-X 结构约束高斯初始化、增密与剪枝;其 Structure-Aware SDS 同时蒸馏 RGB 与深度,退火负提示引导缓解过饱和并减少漂浮伪影。实验显示该方法在效率上优于既有网格/NeRF式方案,生成质量具有竞争力,但手脚等部位仍受文本到图像先验限制。

Human Gaussian Splatting: Real-time Rendering of Animatable Avatars Figure 1
CVPR 20242024-06-16

Human Gaussian Splatting: Real-time Rendering of Animatable Avatars

Arthur Moreau, Jifei Song, Helisa Dhamo, Richard Shaw, Yiren Zhou, Eduardo Pérez-Pellitero

Huawei Noah's Ark Lab, Huawei Technologies (Sweden)

数字人

针对 NeRF/SDF 数字人渲染慢、在未见姿态下质量下降的问题,HuGS 将人体表示为规范空间中的 3D Gaussian,并用前向 LBS 处理骨架运动、姿态条件 MLP 细化衣物等局部非刚性形变,从而避开逆蒙皮并适配宽松服装。实验在多个数据集达到或优于同期方法,THuman4 上 PSNR 提升约 1.5 dB,512×512 可约 80 fps 实时渲染。

HiFi4G: High-Fidelity Human Performance Rendering via Compact Gaussian Splatting Figure 1
CVPR 20242024-06-16

HiFi4G: High-Fidelity Human Performance Rendering via Compact Gaussian Splatting

Yuheng Jiang, Zhehao Shen, Penghao Wang, Zhuo Su, Yu Hong, Yingliang Zhang, Jingyi Yu, Lan Xu

ShanghaiTech University, ByteDance

数字人

面向VR/AR中动态真人表演难以兼顾高保真、实时光栅化和低存储的问题,HiFi4G将3D Gaussian Splatting与非刚性跟踪显式结合,用粗形变图提供关键帧运动先验、细粒度高斯图与自适应时空正则抑制抖动,并配套残差压缩。实验显示其在渲染质量、优化速度和存储开销上优于已有方法,约25倍压缩后每帧低于2MB。

HUGS: Human Gaussian Splats Figure 1
CVPR 20242024-06-16

HUGS: Human Gaussian Splats

Muhammed Kocabas, Jen-Hao Rick Chang, James Gabriel, Oncel Tuzel, Anurag Ranjan

Apple, Apple (United Kingdom)

数字人

针对 NeRF 类数字人从单目视频重建可动画化人体时训练和渲染慢、且难以同时分离场景的问题,HUGS 将人体与静态场景统一表示为 3D Gaussian,并以 SMPL 初始化人体高斯、允许其偏离身体模型以刻画衣物头发,同时联合学习 LBS 权重减少姿态驱动伪影。方法用 50–100 帧野外单目视频约 30 分钟训练,在 NeuMan、ZJU-MoCap 上优于 NeuMan/Vid2Avatar,支持新视角/新姿态渲染并达到 60 FPS,训练约快 100 倍。

HUGS: Holistic Urban 3D Scene Understanding via Gaussian Splatting Figure 1
CVPR 20242024-06-16

HUGS: Holistic Urban 3D Scene Understanding via Gaussian Splatting

Hongyu Zhou, Jiahao Shao, Lu Xu, Dongfeng Bai, Weichao Qiu, Bingbing Liu, Yue Wang, Andreas Geiger, Yiyi Liao

Zhejiang University, Huawei Noah's Ark Lab, Huawei Technologies (Sweden), University of Tübingen, TH Bingen University of Applied Sciences

自动驾驶

面向自动驾驶中仅用位姿 RGB 图像实现城市动态场景的几何、外观、语义与运动统一理解,HUGS 将场景分解为静态与刚体动态 3D Gaussian,并用单车模型物理约束正则化噪声 3D 轨迹,同时联合 RGB、语义和光流监督。其在 KITTI、KITTI-360、Virtual KITTI 2 上提升新视角合成、语义渲染与 3D 语义重建,并支持实时渲染。

Guess The Unseen: Dynamic 3D Scene Reconstruction from Partial 2D Glimpses Figure 1
CVPR 20242024-06-16

Guess The Unseen: Dynamic 3D Scene Reconstruction from Partial 2D Glimpses

Inhee Lee, Byungjun Kim, Hanbyul Joo

Seoul National University

数字人

这篇论文面向野外单目视频中人物常被遮挡、裁切且观测稀疏的问题,尝试同时重建静态场景与多名动态人体。核心做法是用统一的 3D Gaussian Splatting 表示背景和人体,并在规范空间融合稀疏线索,借助预训练 2D 扩散模型补全未见视角且保持外观一致。实验显示其可生成可动画化的人体与可编辑 4D 场景,在遮挡、少样本等设置下优于若干 NeRF/人体重建基线,但仍依赖 SMPL 拟合,合成区域有伪影。

GoMAvatar: Efficient Animatable Human Modeling from Monocular Video Using Gaussians-on-Mesh Figure 1
CVPR 20242024-06-16

GoMAvatar: Efficient Animatable Human Modeling from Monocular Video Using Gaussians-on-Mesh

Jing Wen, Xiaoming Zhao, Zhongzheng Ren, Alexander G. Schwing, Shenlong Wang

University of Illinois Urbana-Champaign

数字人单目重建

面向低成本单目视频生成可驱动数字人的需求,GoMAvatar指出纯NeRF难驱动、纯网格外观不足、自由高斯缺少表面约束的问题,提出将每个3D Gaussian绑定到可变形网格面上的Gaussians-on-Mesh表示,并用伪反照率/伪阴影分解处理视角相关外观。其在ZJU-MoCap、PeopleSnapshot和公开视频上达到或超过现有单目人体建模质量,同时实现43 FPS渲染和每人3.63 MB存储。

GaussianShader: 3D Gaussian Splatting with Shading Functions for Reflective Surfaces Figure 1
CVPR 20242024-06-16

GaussianShader: 3D Gaussian Splatting with Shading Functions for Reflective Surfaces

Yingwenqi Jiang, Jiadong Tu, Yuan Liu, Xifeng Gao, Xiaoxiao Long, Wenping Wang, Yuexin Ma

ShanghaiTech University, The University of Hong Kong, University of Hong Kong, Tencent America, Tencent (China), Texas A&M University

重光照渲染

针对 3D Gaussian Splatting 在镜面/反射表面上难以表达强视角相关高光的问题,GaussianShader 将简化 shading function 显式接入高斯表示,用漫反射、直接反射与残差颜色近似复杂外观,并用高斯最短轴方向加法线残差及法线-几何一致性约束解决离散高斯法线估计。实验中其在高光物体数据集上较 3DGS 提升 1.57dB PSNR,同时相对 Ref-NeRF 将优化时间从约 23 小时降至 0.58 小时并保持实时渲染。

GaussianEditor: Swift and Controllable 3D Editing with Gaussian Splatting Figure 1
CVPR 20242024-06-16

GaussianEditor: Swift and Controllable 3D Editing with Gaussian Splatting

Yiwen Chen, Zilong Chen, Chi Zhang, Feng Wang, Xiaofeng Yang, Yikai Wang, Zhongang Cai, Lei Yang, Huaping Liu, Guosheng Lin

Nanyang Technological University, S-Lab, Nanyang Technological University, S-Lab, Tsinghua University, Department of Computer Science and Technology, Tsinghua University, Department of Computer Science and Technology, School of Computer Science and Engineering, Nanyang Technological University, School of Computer Science and Engineering, Sense Time Research, The Sense Innovation and Research Center

编辑

面向NeRF编辑速度慢、局部控制弱以及传统网格/点云难以真实表达复杂场景的问题,GaussianEditor以3D Gaussian Splatting为基础,通过高斯语义追踪在训练中持续锁定待编辑区域,并用分层高斯约束缓解扩散引导下的随机不稳定;同时支持物体移除与插入。实验显示其在可控性、效果和编辑速度上优于既有3D编辑方法,单次编辑通常约5–10分钟。

GaussianEditor: Editing 3D Gaussians Delicately with Text Instructions Figure 1
CVPR 20242024-06-16

GaussianEditor: Editing 3D Gaussians Delicately with Text Instructions

Junjie Wang, Jiemin Fang, Xiaopeng Zhang, Lingxi Xie, Qi Tian

Huawei Inc

编辑

针对基于2D扩散的3D编辑常把整幅图一起改、难以局部精确控制的问题,GaussianEditor利用3D Gaussian的显式、可独立操作特性,将文本指令解析为RoI,经图像分割对齐到高斯点并约束只更新目标区域。相比Instruct-NeRF2NeRF,它能产生更细致的局部编辑,且单V100训练约20分钟,快于45分钟至2小时的基线。

GaussianAvatars: Photorealistic Head Avatars with Rigged 3D Gaussians Figure 1
CVPR 20242024-06-16

GaussianAvatars: Photorealistic Head Avatars with Rigged 3D Gaussians

Shenhan Qian, Tobias Kirschstein, Liam Schoneveld, Davide Davoli, Simon Giebenhain, Matthias Nießner

Technical University of Munich, Toyota Motor Corporation (United States)

数字人

面向可动画、可换视角且高保真的头部数字人,本文指出 NeRF/动态场景方法虽逼真但缺少表情与姿态控制。GaussianAvatars 将 3D Gaussian 绑定到 FLAME 三角面片,并用局部坐标、位移优化与绑定继承在增删 splat 时保持可控性,同时联合微调模型参数。多视角重建与驱动视频复现中,其新视角合成和 reenactment 指标明显优于 INSTA、PointAvatar 等方法。

GaussianAvatar: Towards Realistic Human Avatar Modeling from a Single Video via Animatable 3D Gaussians Figure 1
CVPR 20242024-06-16

GaussianAvatar: Towards Realistic Human Avatar Modeling from a Single Video via Animatable 3D Gaussians

Liangxiao Hu, Hongwen Zhang, Yuxiang Zhang, Boyao Zhou, Boning Liu, Shengping Zhang, Liqiang Nie

Harbin Institute of Technology, Beijing Normal University, Tsinghua University

数字人

针对单目视频数字人重建中2D观测难以稳定融合、人体姿态估计误差会放大衣褶伪影的问题,GaussianAvatar将人体表面显式表示为可动画3D Gaussians,并用动态外观网络与可优化特征张量学习姿态相关细节,同时联合优化运动与外观。实验在公开和自采数据上显示其外观质量与渲染效率优于对比方法,并能一定程度修正初始动作错位。

Gaussian-Flow: 4D Reconstruction with Dynamic 3D Gaussian Particle Figure 1
CVPR 20242024-06-16

Gaussian-Flow: 4D Reconstruction with Dynamic 3D Gaussian Particle

Youtian Lin, Zuozhuo Dai, Siyu Zhu, Yao Yao

Nanjing University, State Key Laboratory for Novel Software Technology, Nanjing, China, Nanjing University, State Key Laboratory for Novel Software Technology, Alibaba Group, Alibaba Group (United States), Fudan University

动态场景

该工作面向动态场景4D重建中 NeRF 训练/渲染慢、逐帧3DGS存储与训练开销高的问题,提出用动态3D高斯粒子显式建模随时间变化的属性形变,并通过时域多项式与频域傅里叶拟合的 DDDM 捕捉长序列复杂运动,避免为每帧训练独立模型。实验称训练较逐帧3DGS快约5倍,新视角渲染质量优于既有方法;但给定文本片段疑似混入4D-GS论文内容,部分增益来源需谨慎解读。

Gaussian Splatting SLAM Figure 1
CVPR 20242024-06-16

Gaussian Splatting SLAM

Hidenobu Matsuki, Riku Murai, Paul H. J. Kelly, Andrew J. Davison

Imperial College London, Dyson Robotics Laboratory, Imperial College London, Dyson Robotics Laboratory, Software Performance Optimisation Group, Imperial College London

同步定位与建图

面向单目在线SLAM难以同时获得精确位姿、稠密几何与高质量渲染的问题,论文将3D Gaussian Splatting作为唯一地图表示,引入基于高斯的直接位姿优化、李群解析雅可比、各向同性正则与几何验证/裁剪,使3DGS摆脱离线SfM位姿依赖并可扩展到RGB-D。系统约3fps运行,在多数据集上取得有竞争力的轨迹估计、视图合成和细小/透明物体重建效果。

Gaussian Shell Maps for Efficient 3D Human Generation Figure 1
CVPR 20242024-06-16

Gaussian Shell Maps for Efficient 3D Human Generation

Rameen Abdal, Wang Yifan, Zifan Shi, Yinghao Xu, Ryan Po, Zhengfei Kuang, Qifeng Chen, Dit-Yan Yeung, Gordon Wetzstein

Stanford University, USA, Stanford University, Hong Kong University of Science and Technology

数字人

面向数字人生成,现有3D GAN依赖体渲染而训练/渲染慢,网格又难表现头发、宽松衣物并常需带来多视图不一致的2D上采样。论文提出Gaussian Shell Maps:在可关节化的SMPL多层壳上由CNN生成纹理特征,并采样3D Gaussian进行可微splatting渲染。方法可用单视图SHHQ、DeepFashion训练,在512×512原生分辨率下保持多视图一致,报告渲染125 FPS、含生成35 FPS,质量和多样性接近SOTA。

Gaussian Shadow Casting for Neural Characters Figure 1
CVPR 20242024-06-16

Gaussian Shadow Casting for Neural Characters

Luis Bolanos, Shih-Yang Su, Helge Rhodin

The University of British Columbia, University of British Columbia

数字人重光照渲染

论文针对神经数字人在强定向光下易把阴影烘焙进纹理、导致新视角/新姿态和重光照伪影的问题,提出用各向异性 Gaussian 密度代理做解析式阴影积分,并结合 deferred neural rendering 将二次光线降为每像素一次。结果显示该方法能更好分离反照率、明暗和投影阴影,可自动优化光照方向,在户外硬阴影场景的新姿态合成与 HDRI 重光照中较现有方法伪影更少、更真实。

Gaussian Head Avatar: Ultra High-fidelity Head Avatar via Dynamic Gaussians Figure 1
CVPR 20242024-06-16

Gaussian Head Avatar: Ultra High-fidelity Head Avatar via Dynamic Gaussians

Yuelang Xu, Bengwang Chen, Zhe Li, Hongwen Zhang, Lizhen Wang, Zerong Zheng, Yebin Liu

Tsinghua University, Department of Automation, Tsinghua University, Department of Automation, Beijing Normal University

数字人

面向稀疏视角下高保真可驱动头部数字人,本文用可控 3D Gaussian 替代 NeRF/模板网格,并以表达条件 MLP 学习非线性形变,配合 SDF+DMTet 几何初始化提升收敛;扩展版还将头部与头发分开建模,引入时序状态和遮挡感知以处理发丝惯性运动。实验显示其在夸张表情和 2K 渲染下优于现有稀疏视角方法,动态头发表现更合理。

GauHuman: Articulated Gaussian Splatting from Monocular Human Videos Figure 1
CVPR 20242024-06-16

GauHuman: Articulated Gaussian Splatting from Monocular Human Videos

Shoukang Hu, Tao Hu, Ziwei Liu

Nanyang Technological University, S-Lab, Nanyang Technological University, S-Lab

数字人

针对 Human NeRF 训练耗时、渲染慢且难落地的问题,GauHuman 将 3D Gaussian Splatting 放在规范空间,并用 LBS 映射到人体姿态空间,同时引入姿态与蒙皮权重修正;优化上利用 SMPL 人体先验初始化/裁剪高斯,并用 KL 指导 split/clone 与合并减少冗余。在 ZJU_Mocap 和 MonoCap 上达到 SOTA 级新视角质量,约 1–2 分钟训练、最高 189 FPS,约 13k 高斯即可建模。

GS-SLAM: Dense Visual SLAM with 3D Gaussian Splatting Figure 1
CVPR 20242024-06-16

GS-SLAM: Dense Visual SLAM with 3D Gaussian Splatting

Chi Yan, Delin Qu, Dan Xu, Bin Zhao, Zhigang Wang, Dong Wang, Xuelong Li

Shanghai AI Laboratory, ShangHai JiAi Genetics & IVF Institute, Shanghai Artificial Intelligence Laboratory, Hong Kong University of Science and Technology

同步定位与建图

针对 NeRF 式稠密 SLAM 在高分辨率渲染与优化中速度受限、细节易丢失的问题,GS-SLAM 将 3D Gaussian Splatting 引入 RGB-D SLAM,用可微 splatting 加速建图与位姿跟踪,并通过自适应高斯增删和粗到细可靠高斯选择提升重建与定位稳定性。在 Replica、TUM-RGBD 上取得与实时 SOTA 竞争的跟踪和建图表现,系统约 8.43 FPS。

GS-IR: 3D Gaussian Splatting for Inverse Rendering Figure 1
CVPR 20242024-06-16

GS-IR: 3D Gaussian Splatting for Inverse Rendering

Zhihao Liang, Qi Zhang, Ying Feng, Ying Shan, Kui Jia

South China University of Technology, Tencent AI Lab, Tencent (China), School of Data Science, The Chinese University of Hong Kong, Shenzhen, School of Data Science, The Chinese University of Hong Kong, Chinese University of Hong Kong, Shenzhen, Chinese University of Hong Kong

重光照渲染

GS-IR针对NeRF式逆渲染表达能力与计算成本受限、3DGS又缺少可靠法线和遮挡追踪的问题,将3D Gaussian Splatting扩展到未知光照多视图场景的几何、材质与环境光估计。核心做法是用深度导数正则约束高斯法线,并通过烘焙式遮挡体缓存间接光照,从而支持PBR重光照。实验在合成与真实场景上显示其在新视角合成、材质分解和重光照质量上优于基线,同时保持较高效率。

GPS-Gaussian: Generalizable Pixel-wise 3D Gaussian Splatting for Real-time Human Novel View Synthesis Figure 1
CVPR 20242024-06-16

GPS-Gaussian: Generalizable Pixel-wise 3D Gaussian Splatting for Real-time Human Novel View Synthesis

Shunyuan Zheng, Boyao Zhou, Ruizhi Shao, Boning Liu, Shengping Zhang, Liqiang Nie, Yebin Liu

Harbin Institute of Technology, Tsinghua University

数字人

面向稀疏相机下数字人自由视角渲染,传统 NeRF 或 3DGS 往往需逐人优化,难以交互使用。GPS-Gaussian 将每个源视图前景像素映射为高斯参数图,并与迭代双目深度估计联合训练,前馈生成可渲染的 3D Gaussians。实验显示其在多数据集优于现有方法,并可在单张 RTX 3090 上实现 2K 超 25FPS 渲染。

GES: Generalized Exponential Splatting for Efficient Radiance Field Rendering Figure 1
CVPR 20242024-06-16

GES: Generalized Exponential Splatting for Efficient Radiance Field Rendering

Abdullah Hamdi, Luke Melas-Kyriazi, Jinjie Mai, Guocheng Qian, Ruoshi Liu, Carl Vondrick, Bernard Ghanem, Andrea Vedaldi

Visual Geometry Group, University of Oxford, University of Oxford, King Abdullah University of Science and Technology (KAUST), King Abdullah University of Science and Technology, Columbia University

渲染

论文针对 3D Gaussian Splatting 为刻画边缘和高频细节需大量小高斯、导致显存占用高的问题,提出用带形状参数的广义指数函数替代高斯粒子,并配合频率调制损失逐步学习细节。其洞察是高斯的低通特性不适合自然场景中的突变信号,而 GEF 可用更少粒子拟合尖锐结构。实验显示在新视角合成中质量保持竞争力,同时存储降至一半以下、渲染最高提速 39%。

GAvatar: Animatable 3D Gaussian Avatars with Implicit Mesh Learning Figure 1
CVPR 20242024-06-16

GAvatar: Animatable 3D Gaussian Avatars with Implicit Mesh Learning

Ye Yuan, Xueting Li, Yangyi Huang, Shalini De Mello, Koki Nagano, Jan Kautz, Umar Iqbal

NVIDIA

数字人网格重建

GAvatar针对文本生成可动画数字人中网格拓扑受限、NeRF渲染慢以及朴素3D Gaussian难以随姿态一致变形、优化不稳和几何退化的问题,提出将高斯嵌入姿态驱动primitive,并用隐式场预测高斯属性以摊销百万高斯优化;同时把SDF与透明度关联来约束表面并提取纹理网格。实验显示其在外观和几何质量上优于已有方法,并可在1K分辨率达到约100fps渲染。

GART: Gaussian Articulated Template Models Figure 1
CVPR 20242024-06-16

GART: Gaussian Articulated Template Models

Jiahui Lei, Yufu Wang, Georgios Pavlakos, Lingjie Liu, Kostas Daniilidis

University of Pennsylvania, California University of Pennsylvania, UC Berkeley, Berkeley College, University of California, Berkeley, University of California, Berkeley

数字人

针对单目视频中人/动物等非刚性关节主体重建难以兼顾细节、可动画化与实时渲染的问题,GART用规范空间的3D高斯混合显式近似辐射场,并结合SMPL/SMAL模板、可学习前向蒙皮和潜在骨骼建模衣物等额外形变。实验显示其在人像与动物重建渲染上达到SoTA,训练可到秒/分钟级,540×540新姿态渲染超过150 FPS。

FreGS: 3D Gaussian Splatting with Progressive Frequency Regularization Figure 1
CVPR 20242024-06-16

FreGS: 3D Gaussian Splatting with Progressive Frequency Regularization

Jiahui Zhang, Fangneng Zhan, Muyu Xu, Shijian Lu, Eric Xing

Nanyang Technological University, Max Planck Institute for Informatics, Carnegie Mellon University

密度控制

FreGS针对3D Gaussian Splatting在密度增殖中易由少量大高斯覆盖高变化区域、造成过重建、模糊和伪影的问题,提出从频域约束增殖过程:通过傅里叶低/高通分解与频率退火,先低频后高频地匹配渲染图和真值的幅度/相位谱,形成由粗到细的高斯密度控制。其在Mip-NeRF360、Tanks-and-Temples、Deep Blending等基准上提升新视角合成质量,并持续优于3D-GS及相关方法。

FlashAvatar: High-fidelity Head Avatar with Efficient Gaussian Embedding Figure 1
CVPR 20242024-06-16

FlashAvatar: High-fidelity Head Avatar with Efficient Gaussian Embedding

Jun Xiang, Xuan Gao, Yudong Guo, Juyong Zhang

University of Science and Technology of China

数字人

面向低成本、可实时交互的高保真头部数字人,FlashAvatar针对3DMM细节不足、隐式方法训练和渲染慢的问题,将3D Gaussian均匀嵌入参数化人脸表面,并用UV采样与偏移网络补足头发、皱纹等非表面和细微动态。实验显示其可由短单目视频在数分钟内重建,在消费级GPU上以约300FPS渲染,视觉质量和个性化细节优于对比方法。

Feature 3DGS: Supercharging 3D Gaussian Splatting to Enable Distilled Feature Fields Figure 1
CVPR 20242024-06-16

Feature 3DGS: Supercharging 3D Gaussian Splatting to Enable Distilled Feature Fields

Shijie Zhou, Haoran Chang, Sicheng Jiang, Zhiwen Fan, Zehao Zhu, Dejia Xu, Pradyumna Chari, Suya You, Zhangyang Wang, Achuta Kadambi

University of California, Los Angeles, University of California, University of Texas at Austin, The University of Texas at Austin, DEVCOM Army Research Laboratory

编辑语言嵌入分割

针对 NeRF 特征场蒸馏训练/渲染慢、隐式特征易出现连续性伪影的问题,Feature 3DGS 将 2D 基础模型特征蒸馏到 3D Gaussian Splatting,在每个高斯上学习低维语义特征,并用轻量卷积解码器上采样以缓解高维特征带来的开销和分辨率不匹配。实验覆盖新视角语义分割、语言引导编辑和 SAM 点/框提示操作,报告相比 NeRF 类方法最高 2.7× 加速、分割 mIoU 最高提升 23%。

DrivingGaussian: Composite Gaussian Splatting for Surrounding Dynamic Autonomous Driving Scenes Figure 1
CVPR 20242024-06-16

DrivingGaussian: Composite Gaussian Splatting for Surrounding Dynamic Autonomous Driving Scenes

Xiaoyu Zhou, Zhiwei Lin, Xiaojun Shan, Yongtao Wang, Deqing Sun, Ming-Hsuan Yang

Wangxuan Institute of Computer Technology, Peking University, Wangxuan Institute of Computer Technology, Peking University, Google Research, Google (United States)

自动驾驶

该文针对自动驾驶中高速自车、多相机外向视角重叠少、动态目标频繁遮挡导致的大规模360°场景重建困难,提出将场景拆成静态背景与动态实例的 Composite Gaussian Splatting:背景用增量式3D Gaussian顺序扩展,运动物体用动态Gaussian图单独建模再合成,并引入LiDAR几何先验提升细节与多视角一致性。实验显示其在公开驾驶数据集上优于既有NeRF/GS方法,可生成高保真环视新视角并支持动态场景与corner case仿真。

Deformable 3D Gaussians for High-Fidelity Monocular Dynamic Scene Reconstruction Figure 1
CVPR 20242024-06-16

Deformable 3D Gaussians for High-Fidelity Monocular Dynamic Scene Reconstruction

Ziyi Yang, Xinyu Gao, Wen Zhou, Shaohui Jiao, Yuqing Zhang, Xiaogang Jin

Zhejiang University, ByteDance Inc

三维高斯泼溅

针对动态 NeRF 在单目动态场景中细节不足、训练/渲染慢且难以实时的问题,本文将 3D Gaussian Splatting 扩展到可变形建模:在 canonical space 学习高斯,并用时间条件形变场预测位置、旋转和尺度偏移,同时引入退火平滑训练缓解位姿误差导致的时间抖动。实验显示其在新视角合成、时间插值和实时渲染上整体优于 HyperNeRF、NeRF-DS 等基线,但对视角覆盖和位姿精度仍较敏感。

DNGaussian: Optimizing Sparse-View 3D Gaussian Radiance Fields with Global-Local Depth Normalization Figure 1
CVPR 20242024-06-16

DNGaussian: Optimizing Sparse-View 3D Gaussian Radiance Fields with Global-Local Depth Normalization

Jiahe Li, Jiawei Zhang, Xiao Bai, Jin Zheng, Xin Ning, Jun Zhou, Lin Gu

School of Computer Science and Engineering, Jiangxi Research Institute, Beihang University, State Key Laboratory of Complex & Critical Software Environment, School of Computer Science and Engineering, Jiangxi Research Institute, Beihang University, State Key Laboratory of Complex & Critical Software Environment, Institute of Semiconductors, Chinese Academy of Sciences, Institute of Semiconductors, Chinese Academy of Sciences, School of Information and Communication Technology, Griffith University, School of Information and Communication Technology, Griffith University

稀疏表示

DNGaussian针对稀疏视角下3D Gaussian Splatting几何易退化、而稀疏NeRF训练和渲染成本高的问题,提出用单目深度约束高斯位置而非整体形状:通过硬/软深度正则分别调节中心与不透明度,并用全局-局部深度归一化放大小尺度深度变化。实验在LLFF、DTU、Blender上取得优于或相当于SOTA的质量,同时训练约快25倍、渲染快3000倍以上。

Control4D: Efficient 4D Portrait Editing with Text Figure 1
CVPR 20242024-06-16

Control4D: Efficient 4D Portrait Editing with Text

Ruizhi Shao, Jingxiang Sun, Cheng Peng, Zerong Zheng, Boyao Zhou, Hongwen Zhang, Yebin Liu

Tsinghua University, Department of Automation, Tsinghua University, Department of Automation

动态场景编辑

Control4D面向文本驱动的动态4D人像编辑,针对动态NeRF类表示训练渲染慢、2D扩散编辑在视角和时间上不一致的问题,提出GaussianPlanes用空间三平面与时间流平面结构化高斯点,并用4D生成器从不一致编辑图像中学习连续潜空间而非直接监督。实验显示其训练时间更短,渲染质量和时空一致性优于4D版InstructNeRF2NeRF。

Compressed 3D Gaussian Splatting for Accelerated Novel View Synthesis Figure 1
CVPR 20242024-06-16

Compressed 3D Gaussian Splatting for Accelerated Novel View Synthesis

Simon Niedermayr, Josef Stumpfegger, Rüdiger Westermann

Technical University of Munich

压缩

3D Gaussian Splatting 虽能高质量实时新视角合成,但百万级高斯带来 GB 级存储和带宽压力,限制网络传输与低功耗设备部署。本文抓住 SH 颜色与高斯参数冗余,采用敏感度感知向量聚类、量化感知微调及熵/游程编码压缩,并用 GPU 光栅化直接渲染压缩表示。实验显示真实场景最高约 31× 压缩且画质损失很小,在轻量 GPU 上帧率可较优化 compute 管线提升最高约 4×。

Compact 3D Gaussian Representation for Radiance Field Figure 1
CVPR 20242024-06-16

Compact 3D Gaussian Representation for Radiance Field

Joo Chan Lee, Daniel Rho, Xiangyu Sun, Jong Hwan Ko, Eunbyung Park

Sungkyunkwan University

压缩

针对 3D Gaussian Splatting 为保持画质需存储大量高斯点和属性、单场景常达 GB 级的问题,本文提出紧凑 3DGS:用可学习体积 mask 删除贡献小的冗余高斯,以哈希网格神经场替代逐点球谐颜色,并用向量量化码本压缩尺度/旋转等几何属性。实验在多类真实与合成场景中基本保持重建质量,同时约 15× 降低存储并提升渲染速度;结合量化和熵编码后压缩率超过 25×,Deep Blending 上还报告了更高 PSNR 与近 40% 速度增益。

CoGS: Controllable Gaussian Splatting Figure 1
CVPR 20242024-06-16

CoGS: Controllable Gaussian Splatting

Heng Yu, Joel Julin, Zoltán Á. Milacski, Koichiro Niinuma, László A. Jeni

Robotics Institute, Carnegie Mellon University, Robotics Institute, Carnegie Mellon University, Fujitsu Research of America

动态场景

CoGS针对动态/关节物体重建与再动画中多相机标定成本高、单目NeRF训练渲染慢且难以直接操控的问题,将3D Gaussian Splatting扩展到单目动态场景,并利用显式高斯表示结合形变场、2D到3D掩码投影与属性控制,实现无需预计算控制信号的实时场景元素编辑。在合成和真实动态数据上,其视觉保真度优于已有动态和可控神经表示。

COLMAP-Free 3D Gaussian Splatting Figure 1
CVPR 20242024-06-16

COLMAP-Free 3D Gaussian Splatting

Yang Fu, Xiaolong Wang, Sifei Liu, Amey Kulkarni, Jan Kautz, Alexei A. Efros

NVIDIA, UC Berkeley, Berkeley College, University of California, Berkeley, University of California, Berkeley

位姿估计

本文针对 3DGS/NeRF 依赖 COLMAP 等预先相机位姿的问题,利用 3D Gaussian 的显式点云表示和视频连续性,先由单目深度初始化局部高斯并通过光度误差估计相邻位姿,再顺序扩展全局 3DGS。实验在 Tanks and Temples 与 CO3D-V2 大运动序列上显示,其新视角合成质量和 ATE/RPE 位姿精度优于 Nope-NeRF,尤其在大旋转场景更稳。

Animatable Gaussians: Learning Pose-dependent Gaussian Maps for High-fidelity Human Avatar Modeling Figure 1
CVPR 20242024-06-16

Animatable Gaussians: Learning Pose-dependent Gaussian Maps for High-fidelity Human Avatar Modeling

Zhe Li, Zerong Zheng, Lizhen Wang, Yebin Liu

Tsinghua University, Department of Automation, Tsinghua University, Department of Automation

数字人

针对 NeRF/MLP 难以刻画随姿态变化的服装高频细节、显式网格又依赖稠密重建的问题,论文将人体模板驱动的 3D Gaussian 参数化为前后两张 canonical Gaussian maps,用 StyleGAN 式 2D CNN 预测姿态相关高斯属性,并用 PCA 姿态投影提升新姿态泛化;扩展版还引入 PBR 分解材质与光照。实验显示其在动画质量、服装动态细节和重光照效果上优于多类已有 avatar 方法,但头发等非骨骼驱动运动仍建模不足。

Align Your Gaussians: Text-to-4D with Dynamic 3D Gaussians and Composed Diffusion Models Figure 1
CVPR 20242024-06-16

Align Your Gaussians: Text-to-4D with Dynamic 3D Gaussians and Composed Diffusion Models

Huan Ling, Seung Wook Kim, Antonio Torralba, Sanja Fidler, Karsten Kreis

NVIDIA, Moscow Institute of Thermal Technology

扩散生成

题名指向Text-to-4D动态高斯,但抽取正文实际为Video LDM,判断受限于PDF文本抽取或元数据错配。就正文看,动机是降低高分辨率长视频扩散训练成本;核心是为预训练图像LDM加入并微调时间层,同时对上采样器做时间对齐;在驾驶视频和文生视频中改善FVD与主观偏好,可生成512×1024及更高分辨率视频。

ASH: Animatable Gaussian Splats for Efficient and Photoreal Human Rendering Figure 1
CVPR 20242024-06-16

ASH: Animatable Gaussian Splats for Efficient and Photoreal Human Rendering

Haokai Pang, Heming Zhu, Adam Kortylewski, Christian Theobalt, Marc Habermann

Max Planck Institute for Informatics, Saarland Informatics Campus, Max Planck Institute for Informatics

数字人

ASH面向可控数字人在实时渲染中难兼顾NeRF级真实感与速度的问题,将动态人体表示为附着在可变形模板网格上的3D Gaussian splats,并借助UV参数化把姿态到高斯参数的学习转化为2D纹理空间的图像到图像预测,从而避免昂贵3D建模。多视视频监督实验显示,其在可动画人体渲染上显著优于现有实时方法,并达到或超过部分离线方法。

4D Gaussian Splatting for Real-Time Dynamic Scene Rendering Figure 1
CVPR 20242024-06-16

4D Gaussian Splatting for Real-Time Dynamic Scene Rendering

Guanjun Wu, Taoran Yi, Jiemin Fang, Lingxi Xie, Xiaopeng Zhang, Wei Wei, Wenyu Liu, Qi Tian, Xinggang Wang

School of CS, Huazhong University of Science and Technology, School of CS, Huazhong University of Science and Technology, School of EIC, Huazhong University of Science and Technology, School of EIC, Huawei Inc

动态场景

针对动态场景新视角合成中运动建模准确性与实时渲染难以兼顾的问题,4D-GS不为每帧单独维护3D高斯,而是在规范3D高斯上结合4D神经体素、分解式时空编码和轻量MLP预测位置、旋转与尺度形变,从而压缩存储并连接邻近高斯的运动信息。实验显示其在RTX 3090上可达800×800分辨率82 FPS、真实场景约30 FPS,质量与或优于同期SOTA。

3DGStream: On-the-Fly Training of 3D Gaussians for Efficient Streaming of Photo-Realistic Free-Viewpoint Videos Figure 1
CVPR 20242024-06-16

3DGStream: On-the-Fly Training of 3D Gaussians for Efficient Streaming of Photo-Realistic Free-Viewpoint Videos

Jiakai Sun, Han Jiao, Guangyuan Li, Zhanjie Zhang, Lei Zhao, Wei Xing

Zhejiang University

动态场景

3DGStream面向动态场景自由视点视频的在线流式重建,针对既有NeRF/动态重建方法依赖整段离线训练且渲染难实时的问题,用3D Gaussian表示场景,并以紧凑的Neural Transformation Cache建模逐帧位姿变化,再通过自适应新增Gaussian处理新出现物体。实验显示其可在每帧约12秒内训练、约200 FPS实时渲染,同时在画质和存储上保持与SOTA竞争。

3DGS-Avatar: Animatable Avatars via Deformable 3D Gaussian Splatting Figure 1
CVPR 20242024-06-16

3DGS-Avatar: Animatable Avatars via Deformable 3D Gaussian Splatting

Zhiyin Qian, Shaofei Wang, Marko Mihajlovic, Andreas Geiger, Siyu Tang

ETH Zurich, University of Tubingen, University of Tübingen

数字人

该工作针对单目视频数字人重建中 NeRF 方案训练慢、推理难实时的问题,将可变形 3D Gaussian Splatting 引入可动画 clothed avatar:结合人体骨骼刚性关节、姿态相关非刚性形变网络,并对高斯均值和协方差施加近等距正则以提升未见大姿态泛化。实验显示其质量可比或优于现有方法,训练小于 30 分钟、渲染 50+ FPS,约实现 400×训练和 250×推理加速。

3D Geometry-aware Deformable Gaussian Splatting for Dynamic View Synthesis Figure 1
CVPR 20242024-06-16

3D Geometry-aware Deformable Gaussian Splatting for Dynamic View Synthesis

Zhicheng Lu, Xiang Guo, Le Hui, Tianrui Chen, Min Yang, Xiao Tang, Feng Zhu, Yuchao Dai

Northwestern Polytechnical University, Samsung R&D Institute

动态场景

针对单目动态视图合成中 NeRF 式隐式形变缺乏几何约束、易产生不一致运动的问题,本文将动态场景表示为可随时间平移和旋转的 3D Gaussian,并用稀疏 3D 卷积从高斯点云中提取局部几何特征来指导形变学习,同时采用连续 6D 旋转和改进密度控制。合成与真实数据实验显示其在动态新视角合成和重建上优于已有方法,达到新的 SOTA。

PUP 3D-GS: Principled Uncertainty Pruning for 3D Gaussian Splatting Figure 1
CVPR 20252024-06-14

PUP 3D-GS: Principled Uncertainty Pruning for 3D Gaussian Splatting

Alex Hanson, Allen Tu, Vasu Singla, Mayuka Jayawardhana, Matthias Zwicker, Tom Goldstein

University of Maryland, College Park, University of Maryland, College Park

密度控制

3D-GS 在复杂场景中常含数百万高斯,限制存储、显存和端侧部署;现有剪枝多依赖启发式,高压缩下易丢前景细节。PUP 3D-GS 用训练视图重建误差对高斯空间参数的二阶/Fisher 近似构造敏感度分数,并采用多轮剪枝-微调的后处理流程,无需改训练。实验在多数据集剪掉 90% 高斯后平均提速 3.56×,且图像指标和前景保真优于 LightGaussian 等方法。

Grounding Image Matching in 3D with MASt3R Figure 1
arXiv preprint2024-06-14

Grounding Image Matching in 3D with MASt3R

Vincent Leroy, Yohann Cabon, Jerome Revaud

Naver Labs Europe, Meylan, France, Naver Labs Europe

三维高斯泼溅

本文针对图像匹配长期被当作2D局部对应、在大视角变化和弱纹理场景中易失效的问题,主张把匹配显式落到3D几何中。MASt3R在DUSt3R上增加密集局部特征头,并用匹配损失训练,再结合粗到细与快速互惠匹配降低密集搜索开销。实验显示其在相机位姿、视觉定位和MVS等任务上超过DUSt3R及多种匹配方法,Map-free定位VCRE AUC相对最佳已发表方法有30个百分点绝对提升。

GGHead: Fast and Generalizable 3D Gaussian Heads Figure 1
arXiv preprint2024-06-13

GGHead: Fast and Generalizable 3D Gaussian Heads

Tobias Kirschstein, Simon Giebenhain, Jiapeng Tang, Markos Georgopoulos, Matthias Nießner

Technical University of Munich, Munich, Germany, Technical University of Munich

三维高斯泼溅

GGHead针对现有3D头部GAN在高分辨率训练与渲染中速度慢、依赖2D超分而破坏三维一致性的问题,将3D Gaussian Splatting引入GAN框架,并在模板头网格UV空间用2D CNN预测高斯属性,配合UV坐标总变分正则提升几何稳定性。其仅用单视角2D图像训练,在FFHQ上达到既有3D头部GAN质量,同时显著更快,并实现1024²分辨率实时、三维一致的头部生成与渲染。

ICE-G: Image Conditional Editing of 3D Gaussian Splats Figure 1
arXiv preprint2024-06-12

ICE-G: Image Conditional Editing of 3D Gaussian Splats

Hannah Hanyun, Muhammad Zubair

Georgia Institute of Technology, 2Toyota Research Institute, 3Stability AI, 4Google Research, Georgia Institute of Technology, Toyota Research Institute, Google Research

编辑

ICE-G针对现有3D场景编辑速度慢、质量或可控性不足的问题,提出从单张参考图驱动Gaussian Splats/NeRF重风格化:用SAM分割编辑图与多视角数据,再以DINO特征匹配语义区域,将颜色或纹理传播到对应视图并微调3D表示。实验在多个NeRF/真实场景数据集上显示,其在局部颜色、纹理和多风格组合编辑中比文本或框选式基线更精确、伪影更少,但几何编辑与反射材质保持受限。

Trim 3D Gaussian Splatting for Accurate Geometry Representation Figure 1
arXiv preprint2024-06-11

Trim 3D Gaussian Splatting for Accurate Geometry Representation

Lue Fan, Yuxue Yang, Minxing Li, Hongsheng Li, Zhaoxiang Zhang

MMLab, CUHK, MMLab, Shanghai AI Lab

二维高斯密度控制

针对3DGS常出现“渲染好但几何乱”的问题,TrimGS不再只依赖强几何正则,而是按alpha blending贡献度逐步裁剪冗余或不准的高斯,并通过密度控制维持较小尺度以保留细节、改善优化梯度;该策略可结合3DGS/2DGS及法线一致性正则,在实验中持续提升几何重建精度和感知质量,但仍存在几何正则导致部分渲染指标下降的权衡。

4Real: Towards Photorealistic 4D Scene Generation via Video Diffusion Models Figure 1
arXiv preprint2024-06-11

4Real: Towards Photorealistic 4D Scene Generation via Video Diffusion Models

Laszlo A

Carnegie Mellon University

扩散生成

4Real针对现有4D生成依赖合成物体多视图先验、结果偏物体中心且不够真实的问题,改用真实视频训练的文本到视频扩散模型生成参考视频与“冻结时间”视频,并以可变形3D Gaussian Splats重建规范空间、显式吸收伪多视图不一致再学习时序形变。结果显示其可从文本生成可换视角、可随时间播放的近真实动态场景,计算约1.5小时/A100,低于部分10小时级方法。

Lighting Every Darkness with 3DGS: Fast Training and Real-Time Rendering for HDR View Synthesis Figure 1
arXiv preprint2024-06-10

Lighting Every Darkness with 3DGS: Fast Training and Real-Time Rendering for HDR View Synthesis

Xin Jin, Pengyi Jiao, Zheng-Peng Duan, Xingchao Yang, Chun-Le Guo, Bo Ren, Chongyi Li

VCIP, CS, Nankai University, Nankai University

渲染

该文面向夜间/高动态范围场景中 RawNeRF 训练慢、无法实时渲染的问题,将 3D Gaussian Splatting 改造到 RAW 线性空间。核心做法包括用锥形散射初始化补足低信噪比下 SfM 远景点云,以 Color MLP 替代球谐表示 HDR 颜色,并加入深度畸变与近远正则改善结构。实验显示其在接近体渲染方法质量的同时,将训练时间降至约 1%,2K 渲染帧率最高提升约 4000 倍,并支持实时 HDR、重对焦和色调映射调整。

Adversarial Generation of Hierarchical Gaussians for 3d Generative Model Figure 1
arXiv preprint2024-06-05

Adversarial Generation of Hierarchical Gaussians for 3d Generative Model

Sangeek Hyun, Jae-Pil Heo

Sungkyunkwan University

三维高斯泼溅

本文针对现有 3D GAN 依赖射线体渲染、训练和高分辨率渲染成本高的问题,将 3D Gaussian Splatting 引入对抗式三维生成;关键洞察是直接生成高斯会因缺少初始化与 densification 约束而不稳定,因此设计粗到细的层级高斯,使细层位置依附粗层、尺度逐层减小以正则化几何。实验显示其在保持相近三维生成质量和多视角一致性的同时,渲染速度约提升 100 倍。

MoDGS: Dynamic Gaussian Splatting from Causually-captured Monocular Videos Figure 1
arXiv preprint2024-06-01

MoDGS: Dynamic Gaussian Splatting from Causually-captured Monocular Videos

Qingming Liu, Yuan Liu, Jiepeng Wang, Xianqiang Lv, Peng Wang, Wenping Wang, Junhui Hou

City University of HongKong

动态场景单目重建

MoDGS面向随手拍单目动态视频中新视角合成:相机静止或缓慢移动时,传统动态 NeRF/3DGS 难以依赖多视角一致性恢复几何。方法引入单目深度先验,并用3D感知初始化学习形变场,进一步以保序深度损失缓解跨帧深度尺度不一致。实验在 Nvidia、DyNeRF、Davis及野外视频上显示,其在严格单目设置下较现有方法明显提升渲染质量。

ContextGS: Compact 3D Gaussian Splatting with Anchor Level Context Model Figure 1
arXiv preprint2024-05-31

ContextGS: Compact 3D Gaussian Splatting with Anchor Level Context Model

Lanqing Guo, Alex Kot, Zhihao Li, Yufei Wang, Bihan Wen, Wenhan Yang

Nanyang Technological University, Singapore, Nanyang Technological University, PengCheng Laboratory, China, PengCheng Laboratory

压缩

ContextGS针对3DGS场景动辄数GB、现有方法多独立编码高斯/锚点而忽略空间相关性的问题,在Scaffold-GS锚点表示上引入分层自回归上下文模型:先编码粗层锚点,再用已解码邻域预测细层分布,并用低维量化hyperprior辅助熵编码。实验称在画质相当或略优下,相比原始3DGS压缩超100倍、相比Scaffold-GS约15倍。

RT-GS2: Real-Time Generalizable Semantic Segmentation for 3D Gaussian Representations of Radiance Fields Figure 1
arXiv preprint2024-05-28

RT-GS2: Real-Time Generalizable Semantic Segmentation for 3D Gaussian Representations of Radiance Fields

Mihnea-Bogdan Jurca, Remco Royen, Ion Giosan, Adrian Munteanu

Department ETRO, Computer Science Department, Technical University of Cluj-Napoca

点云分割虚拟现实

针对 NeRF/3DGS 语义分割难以同时兼顾未见场景泛化与实时性的痛点,RT-GS2 在 3D Gaussian 表示上自监督学习视角无关特征,并通过 VDVI 融合结合视角相关与无关信息以提升跨视角一致性。实验覆盖三个数据集,在 Replica 上 mIoU 提升 8.01%,推理达 27.03 FPS,相比既有可泛化方法加速约 901 倍。

Vidu4D: Single Generated Video to High-Fidelity 4D Reconstruction with Dynamic Gaussian Surfels Figure 1
arXiv preprint2024-05-27

Vidu4D: Single Generated Video to High-Fidelity 4D Reconstruction with Dynamic Gaussian Surfels

Zilong Chen, Fuchun Sun, Xinzhou Wang, Yikai Wang, Zhengyi Wang, Jun Zhu

Department of Computer Science and Technology, BNRist Center, Tsinghua University, Department of Computer Science and Technology, BNRist Center, Tsinghua University, College of Electronic and Information Engineering, Tongji University, College of Electronic and Information Engineering, Tongji University

二维高斯动态场景

Vidu4D针对生成视频虽具3D一致性、但单视频4D重建易受非刚性运动和帧畸变破坏的问题,提出动态Gaussian Surfels:先初始化非刚性变形场,再用时变warp将静态表面高斯映射到动态状态,并加入法线几何正则及旋转/尺度细化以抑制闪烁。实验显示其在生成视频上较现有4DGS类方法获得更稳定的外观和几何,可与视频生成模型组合实现文本到4D内容生成。

Sp 2 360: Sparse-view 360 Scene Reconstruction using Cascaded 2D Diffusion Priors Figure 1
arXiv preprint2024-05-26

Sp 2 360: Sparse-view 360 Scene Reconstruction using Cascaded 2D Diffusion Priors

Soumava Paul, Christopher Wewer, Bernt Schiele, Jan Eric Lenssen

Max Planck Institute for Informatics, Saarland Informatics Campus, Germany, Max Planck Institute for Informatics, Saarland University, Saarland Informatics Campus, Germany, Saarland University

稀疏表示

该文针对360°场景在仅有少量输入视角时重建欠约束、3DGS易过拟合并产生背景坍塌的问题,提出Sp²360:先用稀疏视图拟合3D高斯,再级联微调的2D扩散补全与伪影去除模型生成伪新视角,并迭代蒸馏回3D表示。实验显示其在Mip-NeRF360上优于稀疏视角重建基线,9张图即可生成较完整的前景与背景细节,但仍受SfM稀疏几何先验限制。

Don't Splat your Gaussians: Volumetric Ray-Traced Primitives for Modeling and Rendering Scattering and Emissive Media Figure 1
ACM TOG 20252024-05-24

Don't Splat your Gaussians: Volumetric Ray-Traced Primitives for Modeling and Rendering Scattering and Emissive Media

Jorge Condor, Sebastien Speierer, Lukas Bode, Aljaz Bozic, Simon Green, Piotr Didyk, Adrian Jarabo

Meta Reality Labs, Zurich, Switzerland and Universita della Svizzera Italiana, Lugano, Switzerland, Meta Reality Labs, Meta Reality Labs, Zurich, Switzerland, Meta Reality Labs, Lausanne, Switzerland, Meta Reality Labs, London, United Kingdom of Great Britain and Northern Ireland, Meta (United Kingdom), Meta Reality Labs, Zaragoza, Spain

物理建模光线追踪重光照渲染全景重建

针对体素网格内存开销大、3D Gaussian Splatting 难以纳入物理路径追踪的问题,论文将高斯/Epanechnikov 等核混合建模为真正的三维体积基元,推导透射率、发光与自由程采样的闭式或高效数值解,并用硬件光追遍历基元边界。结果显示云等散射介质可用数千基元压缩到数百 KB,渲染速度优于高分辨率体素参考,同时支持逆渲染、重光照和辐射场渲染,但辐射场画质仍略低于 3DGS。

DOGS: Distributed-Oriented Gaussian Splatting for Large-Scale 3D Reconstruction Via Gaussian Consensus Figure 1
arXiv preprint2024-05-22

DOGS: Distributed-Oriented Gaussian Splatting for Large-Scale 3D Reconstruction Via Gaussian Consensus

Yu Chen, Gim Hee Lee

National University of Singapore

大规模场景

DOGS针对大规模场景中3DGS训练显存高、耗时长且难以像NeRF那样分布式切分的问题,采用递归场景分块并将ADMM式高斯共识引入训练:从节点优化局部3DGS,主节点维护全局模型并同步共享高斯,推理时仅保留全局模型。实验显示其在大规模场景上相较原始3DGS训练加速6倍以上,同时保持或达到较好的渲染质量。

Gaussian Head & Shoulders: High Fidelity Neural Upper Body Avatars with Anchor Gaussian Guided Texture Warping Figure 1
arXiv preprint2024-05-20

Gaussian Head & Shoulders: High Fidelity Neural Upper Body Avatars with Anchor Gaussian Guided Texture Warping

Tianhao Wu, Jing Yang, Zhilin Guo, Jingyi Wan, Fangcheng Zhong, Cengiz Oztireli

TIANHAO WU, University of Cambridge, United Kingdom, University of Cambridge, JING YANG, University of Cambridge, United Kingdom, ZHILIN GUO, University of Cambridge, United Kingdom, JINGYI WAN, University of Cambridge, United Kingdom, FANGCHENG ZHONG, University of Cambridge, United Kingdom, CENGIZ OZTIRELI, Google Research, University of Cambridge, United Kingdom, Google Research

数字人动态场景

该工作针对单目视频数字人只重建头部、而衣物胸肩区域用普通 3D Gaussian 易模糊和产生漂浮伪影的问题,提出用神经纹理表示上半身细节,并以稀疏锚点 Gaussian 约束从图像平面到纹理空间的神经变形场,绕开精确几何和 UV 需求。实验显示其在手机和互联网视频的自重演、交叉重演中提升衣物高频细节与鲁棒性,并通过去除推理期 MLP 达到约 130 FPS。

GarmentDreamer: 3DGS Guided Garment Synthesis with Diverse Geometry and Texture Details Figure 1
3DV 20252024-05-20

GarmentDreamer: 3DGS Guided Garment Synthesis with Diverse Geometry and Texture Details

Boqian Li, Xuan Li, Ying Jiang, Tianyi Xie, Feng Gao, Huamin Wang, Yin Yang, Chenfanfu Jiang

Amazon, Amazon (Germany), Style3D Research, Krylov State Research Center, University of Utah

数字人动态场景渲染纹理建模

面向传统服装建模耗时、文本生成服装常有多视角不一致或需从人体中分离的问题,GarmentDreamer用3D Gaussian Splatting作为一致的几何与纹理引导,结合基于法线/RGBA的粗到细网格变形和NeTF+VSD纹理优化,直接生成可穿戴、可仿真的非封闭服装网格。实验显示其在形状细节、纹理一致性和整体质量上优于多种现有方法。

LIV-GaussMap: LiDAR-Inertial-Visual Fusion for Real-time 3D Radiance Field Map Rendering Figure 1
IEEE RA-L 20242024-05-13

LIV-GaussMap: LiDAR-Inertial-Visual Fusion for Real-time 3D Radiance Field Map Rendering

Sheng Hong, Junjie He, Xinhu Zheng, Chunran Zheng

Department of Electronic Computer Engineering, The Hong Kong University of Science and Technology, Hong Kong, SAR, China, Department of Electronic Computer Engineering, The Hong Kong University of Science and Technology, Hong Kong University of Science and Technology, University of Hong Kong, System Hub, The Hong Kong University of Science and Technology, Guangzhou, China, System hub, The Hong Kong University of Science and Technology (Guangzhou), Guangzhou, China, The Hong Kong University of Science and Technology (Guangzhou), Department of Mechanical Engineering, The University of Hong Kong, Hong Kong, SAR, China, Department of Mechanical Engineering, The University of Hong Kong

大规模场景

针对传统视觉/激光 SLAM 难以同时获得精确几何与逼真外观、且在低纹理或非朗伯表面易退化的问题,LIV-GaussMap 将 LiDAR-惯性初始化的自适应体素表面结构与可微 3D Gaussian 表达结合,再用视觉光度梯度和球谐外观系数细化密度与颜色。实验覆盖多类激光雷达和室内外场景,显示其在保持实时渲染的同时,相比视觉式方法获得更好的几何结构和新视角渲染质量。

3D Gaussian Splatting with Deferred Reflection Figure 1
arXiv preprint2024-04-29

3D Gaussian Splatting with Deferred Reflection

Keyang Ye, Qiming Hou, Kun Zhou

Zhejiang University, China, Zhejiang University, Zhejiang University Hangzhou, China, Zhejiang University Hangzhou, State Key Lab of CAD&CG

三维高斯泼溅

针对 3D Gaussian Splatting 难以稳定拟合高频镜面反射、且会用“幻觉”高斯破坏几何的问题,本文将反射改为延迟着色:先泼溅得到颜色、法线和反射强度,再用环境图按像素查询反射,并通过法线传播让可靠法线梯度扩散到相邻高斯。实验显示其在合成与真实反射场景上较现有方法稳定提升 PSNR,同时帧率几乎保持原版 3DGS 水平。

PhyRecon: Physically Plausible Neural Scene Reconstruction Figure 1
arXiv preprint2024-04-25

PhyRecon: Physically Plausible Neural Scene Reconstruction

Yixin Chen, Bo Dai, Siyuan Huang, Nan Jiang, Bohan Jing, Puhao Li, Junfeng Ni, Bin Wang, Song-Chun Zhu, Yixin Zhu

Tsinghua University, State Key Laboratory of General Artificial Intelligence, BIGAI, State Key Laboratory of General Artificial Intelligence, Peking University

动态场景网格重建物理建模

PHYRECON针对多视角神经重建只依赖渲染监督、易生成物理上不稳定形状和遗漏细长支撑结构的问题,将可微渲染与可微粒子物理仿真联合用于SDF隐式表面学习,并通过SP-MC在隐式场与显式表面点间可微转换,同时用渲染/物理不确定性过滤错误几何先验并引导采样。ScanNet、ScanNet++和Replica实验显示其提升重建质量,在Isaac Gym评估中各数据集物理稳定性至少提高40%。

InFusion: Inpainting 3D Gaussians via Learning Depth Completion from Diffusion Prior Figure 1
arXiv preprint2024-04-17

InFusion: Inpainting 3D Gaussians via Learning Depth Completion from Diffusion Prior

Ka Leong

University of Science and Technology of China, The Hong Kong University of Science and Technology, Alibaba Group

编辑

InFusion面向3D Gaussian编辑中的补全问题:仅靠多视图2D修复再生长高斯点,易因几何初始化不准导致纹理模糊且优化慢。论文的核心洞察是先用带扩散先验、图像条件的深度补全模型恢复与原场景尺度对齐的缺失深度,再反投影为初始3D点并优化高斯。实验显示其在前向和360度场景中提升视觉一致性与保真度,并相较既有方法约快20倍,还支持指定纹理补全和新物体插入。

DeblurGS: Gaussian Splatting for Camera Motion Blur Figure 1
arXiv preprint2024-04-17

DeblurGS: Gaussian Splatting for Camera Motion Blur

Kyoung Mu

IPAI, Seoul National University, Korea, Seoul National University, Dept. of ECE & ASRI, Seoul National University, Korea

去模糊渲染

针对视频式采集常伴随相机运动模糊、且模糊会使 SfM 初始位姿显著出错的问题,DeblurGS 将 3D Gaussian Splatting 与每帧 6DoF 潜在相机运动联合优化,通过沿运动轨迹累积子帧渲染来匹配模糊观测,并用 Gaussian Densification Annealing 避免早期位姿不准时生成错误高斯。实验显示其在合成、真实多视图及手机视频上超过既有 NeRF 去模糊方法,尤其能在噪声位姿初始化下恢复清晰三维场景。

LetsGo: Large-Scale Garage Modeling and Rendering via LiDAR-Assisted Gaussian Primitives Figure 1
ACM TOG 20242024-04-15

LetsGo: Large-Scale Garage Modeling and Rendering via LiDAR-Assisted Gaussian Primitives

Jiadi Cui, Junming Cao, Fuqiang Zhao, Zhipeng He, Yifan Chen, Yuhui Zhong, Lan Xu, Yujiao Shi, Yingliang Zhang, Jingyi Yu

ShanghaiTech University, Shanghai, China, ShanghaiTech University, Stereye Inc., Shanghai, China, NeuDim Inc., Shanghai, China, Shanghai Advanced Research Institute, Chinese Academy of Sciences, Shanghai, China, Shanghai Advanced Research Institute, Chinese Academy of Sciences, University of Chinese Academy of Sciences, Beijing, China, University of Chinese Academy of Sciences, DGene Inc., Shanghai, China

大规模场景

大规模地下车库因弱纹理、重复结构、反光与透明车窗导致 SfM 和纯视觉 3DGS 易失效。LetsGo 的核心是用自研 Polar 采集器提供配准 LiDAR/IMU/鱼眼数据,将 LiDAR 点云引入高斯初始化与深度正则,并设计多分辨率 LOD 高斯表示以降低渲染负担。在 GarageWorld、ScanNet++、KITTI-360 上,方法提升了渲染质量、减少漂浮伪影,并支持消费级设备上的实时网页渲染。

3D Gaussian Splatting as Markov Chain Monte Carlo Figure 1
arXiv preprint2024-04-15

3D Gaussian Splatting as Markov Chain Monte Carlo

Kwang Moo

University of British Columbia, 2Google Research, University of British Columbia, Google Research, Google DeepMind, 4Simon Fraser University, 5University of Toronto, Google DeepMind, Simon Fraser University, University of Toronto

密度控制

针对 3D Gaussian Splatting 依赖手工克隆/分裂/剪枝、对 SfM 初始化敏感且难以预设高斯数量的问题,论文将高斯集合解释为场景分布的 MCMC 样本,在更新中加入噪声形成 SGLD,并用近似保持渲染概率的重定位替代启发式密度控制,辅以稀疏正则移除无效高斯。实验显示其在多类标准场景上提升渲染质量,可控高斯预算,并对随机或点云初始化更稳健。

Gaga: Group Any Gaussians via 3D-aware Memory Bank Figure 1
arXiv preprint2024-04-11

Gaga: Group Any Gaussians via 3D-aware Memory Bank

Weijie Lyu, Xueting Li, Abhijit Kundu, Yi-Hsuan Tsai, Ming-Hsuan Yang

University of California, Merced, University of California, NVIDIA Research, Google DeepMind

编辑分割

Gaga针对开放世界3D分割中多视角2D无类别掩码ID不一致的问题,认为同一物体在不同视角应对应相同3D Gaussians集合,并用3D-aware memory bank按空间重叠关联掩码,而非依赖连续视角跟踪或对比聚类。该方法可接入SAM、EntitySeg等不同2D分割源,在稀疏视角和姿态变化下仍保持标签一致,实验显示优于现有方法;但对2D掩码本身的过/欠分割仍敏感。

Zero-shot Point Cloud Completion Via 2D Priors Figure 1
arXiv preprint2024-04-10

Zero-shot Point Cloud Completion Via 2D Priors

Tianxin Huang, Zhiwen Yan, Yuyang Zhao, Gim Hee Lee

School of Computing, National University of Singapore University, School of Computing, National University of Singapore University

扩散生成点云

针对传统点云补全依赖类别内训练、零样本文本引导方法又需手写提示的问题,论文提出在测试时用部分点云估计参考视角,经3D Gaussian Splatting渲染参考图,再借助Zero-1-to-3等2D扩散先验优化高斯并抽取均匀点云。合成与真实扫描实验显示其在未见类别上优于网络式和既有测试时方法,但单例优化仍需较长时间。

SpikeNVS: Enhancing Novel View Synthesis from Blurry Images via Spike Camera Figure 1
arXiv preprint2024-04-10

SpikeNVS: Enhancing Novel View Synthesis from Blurry Images via Spike Camera

Gaole Dai, Zhenyu Wang, Qinwen Xu, Ming Lu, Wen Chen, Boxin Shi, Shanghang Zhang, Tiejun Huang

National Key Laboratory for Multimedia Information Processing, School of Computer Science, Peking University, School of Computer Science, Peking University

去模糊其他

针对RGB训练图像运动模糊会削弱NeRF/3DGS新视角合成清晰度的问题,SpikeNVS引入脉冲相机的时间积分信息,而非事件相机的时间差分,提出Texture from Spike损失,将TFI/TFP重建纹理作为监督以兼顾前景与静态背景并降低训练开销。作者还搭建同步spike-RGB采集系统和真实数据集,在合成与真实场景中验证其可提升NeRF和3DGS的去模糊式新视角合成质量。

Gaussian-LIC: Photo-realistic LiDAR-Inertial-Camera SLAM with 3D Gaussian Splatting Figure 1
arXiv preprint2024-04-10

Gaussian-LIC: Photo-realistic LiDAR-Inertial-Camera SLAM with 3D Gaussian Splatting

Xiaolei Lang, Laijian Li, Hang Zhang, Feng Xiong, Mu Xu, Yong Liu, Xingxing Zuo, Jiajun Lv

同步定位与建图

Gaussian-LIC针对现有辐射场/3DGS SLAM多局限于室内RGB-D、在室外无界场景和剧烈运动、光照变化下易失效的问题,将LiDAR-IMU-相机紧耦合位姿估计与3D Gaussian在线建图结合,并用激光点和三角化视觉点共同初始化高斯,加入天空与曝光建模及C++/CUDA加速。实验显示其在真实室内外数据上保持实时性,照片级建图优于同类方法,甚至超过部分使用真值位姿的映射基线。

GaSpCT: Gaussian Splatting for Novel CT Projection View Synthesis Figure 1
arXiv preprint2024-04-04

GaSpCT: Gaussian Splatting for Novel CT Projection View Synthesis

Emmanouil Nikolakakis, Utkarsh Gupta, Jonathan Vengosh, Justin Bui, Razvan Marinescu

University of California, Santa Cruz, Electrical and Computer Engineering Department, University of California, Electrical and Computer Engineering Department, University of California, Santa Cruz, Computer Science and Engineering Department, Computer Science and Engineering Department

医学影像

GaSpCT面向稀疏视角CT中投影不足导致的伪影与剂量问题,将3D Gaussian Splatting改造为无需SfM的脑CT新视角投影合成方法,并用DICOM近似相机、椭球脑区初始化及TV/Beta稀疏正则强化前景背景分离。在PPMI脑CT模拟DRR上,合成视图更接近真实投影,优于其他隐式表示,训练约5–10分钟,表示存储较体素网格低17%。

TCLC-GS: Tightly Coupled LiDAR-Camera Gaussian Splatting for Surrounding Autonomous Driving Scenes Figure 1
arXiv preprint2024-04-03

TCLC-GS: Tightly Coupled LiDAR-Camera Gaussian Splatting for Surrounding Autonomous Driving Scenes

Cheng Zhao, Su Sun, Ruoyu Wang, Yuliang Guo, Jun-Jun Wan, Zhou Huang, Xinyu Huang, Yingjie Victor Chen, Liu Ren

Bosch Research North America & Bosch Center for Artificial Intelligence (BCAI), Purdue University

自动驾驶

面向自动驾驶周围场景中视角稀疏、仅用 LiDAR 点初始化 3D-GS 难以充分利用几何与图像信息的问题,TCLC-GS 将 LiDAR-相机数据构造成彩色显式网格与层次八叉树隐式特征,用网格对齐初始化高斯并提供稠密深度监督,用隐式特征增强外观建模。在 Waymo 与 nuScenes 上取得 SOTA,并在单张 RTX 3090 Ti 上实现约 90/120 FPS 的 RGB 与深度实时渲染。

Hash3D: Training-free Acceleration for 3D Generation Figure 1
arXiv preprint2024-04-02

Hash3D: Training-free Acceleration for 3D Generation

Xingyi Yang, Xinchao Wang

National University of Singapore

加速训练扩散生成

面向SDS类3D生成每个物体需大量视角与时间步扩散推理、优化耗时过长的问题,Hash3D观察到相邻相机位姿和去噪时间步的中间特征高度冗余,提出免训练的自适应网格哈希来缓存并复用特征,减少重复计算。其可插入多种文生/图生3D流程,在5个文生3D和3个图生3D模型上实现约1.3–4倍加速,并改善平滑性与多视角一致性;结合3D Gaussian Splatting时文生3D约10分钟、图生3D约30秒。

Feature Splatting: Language-Driven Physics-Based Scene Synthesis and Editing Figure 1
arXiv preprint2024-04-01

Feature Splatting: Language-Driven Physics-Based Scene Synthesis and Editing

Ri-Zhao Qiu, Ge Yang, Weijia Zeng, Xiaolong Wang

University of California San Diego, Massachusetts Institute of Technology, Institute for Artificial Intelligence and Fundamental Interactions

编辑语言嵌入物理建模分割

这篇论文面向静态 3D Gaussian 场景难以按对象同时编辑外观与物理属性的问题,提出 Feature Splatting:将视觉语言模型的对象级语义特征蒸馏到高斯基元中,用文本查询完成场景分解和材质赋值,并接入基于 MPM 的粒子仿真生成动态效果。实验和消融展示其可进行语言驱动的分割、外观/几何编辑与物理运动合成,但移除物体后的背景修补仍会产生伪影。

InstantSplat: Unbounded Sparse-view Pose-free Gaussian Splatting in 40 Seconds Figure 1
arXiv preprint2024-03-29

InstantSplat: Unbounded Sparse-view Pose-free Gaussian Splatting in 40 Seconds

Zhiwen Fan, Wenyan Cong, Kairun Wen, Kevin Wang, Jian Zhang, Xinghao Ding, Danfei Xu, Boris Ivanovic, Marco Pavone, Georgios Pavlakos, Zhangyang Wang, Yue Wang

University of Texas at Austin, Nvidia Research, Xiamen University, Georgia Institute of Technology, Stanford University, University of Southern California

稀疏表示

InstantSplat针对稀疏视角下COLMAP/SfM易失效、传统3D-GS依赖密集图像和长时间优化的问题,利用MASt3R稠密立体先验与共可视几何初始化,并提出Gaussian Bundle Adjustment以光度误差联合优化相机位姿和高斯表示。实验在2–3张无位姿图像上即可重建,大幅减少视图需求,3视角SSIM由0.3755提升至0.7624,优化约7.5秒、整体加速超过20倍,并可接入3D-GS、2D-GS和Mip-Splatting。

SA-GS: Scale-Adaptive Gaussian Splatting for Training-Free Anti-Aliasing Figure 1
arXiv preprint2024-03-28

SA-GS: Scale-Adaptive Gaussian Splatting for Training-Free Anti-Aliasing

Xiaowei Song, Jv Zheng, Shiran Yuan, Huan-ang Gao, Jingwei Zhao, Xiang He, Weihao Gu, Hao Zhao

Institute for AI Industry Research (AIR), Tsinghua University, Institute for AI Industry Research (AIR), Tsinghua University, Tongji University, Ocean University of China, Duke Kunshan University

抗锯齿渲染

SA-GS针对3D Gaussian Splatting在测试分辨率或视距变化时因训练/测试高斯投影尺度不一致而产生的侵蚀、膨胀和锯齿问题,提出无需重训的测试时插件:用感知测试频率的2D尺度自适应滤波保持投影分布一致,再结合像素内积分/超采样抗混叠。实验在Mip-NeRF 360和Blender等场景中显示,其效果与需改训练流程的Mip-Splatting相当或更优,尤其缩小时PSNR可提升约1.1dB。

GaussianCube: A Structured and Explicit Radiance Representation for 3D Generative Modeling Figure 1
arXiv preprint2024-03-28

GaussianCube: A Structured and Explicit Radiance Representation for 3D Generative Modeling

Bowen Zhang, Yiji Cheng, Jiaolong Yang, Chunyu Wang, Feng Zhao, Yansong Tang, Dong Chen, Baining Guo

University of Science and Technology of China, Tsinghua University, Microsoft Research Asia

扩散生成

本文针对 NeRF 类表示依赖共享隐式解码器、3DGS 又缺乏空间结构而难以接入主流 3D 扩散模型的问题,提出 GaussianCube:先用受限增密拟合固定数量高斯,再通过最优传输排列到体素网格,使显式高斯具备卷积友好的结构。该表示参数量较同质量结构化表示少一到两个数量级,可直接用 3D U-Net 训练扩散模型;在无条件/类别条件物体生成、头像生成和文本到 3D 上均报告优于既有方法的定量与视觉结果。

GSDF: 3DGS Meets SDF for Improved Rendering and Reconstruction Figure 1
arXiv preprint2024-03-25

GSDF: 3DGS Meets SDF for Improved Rendering and Reconstruction

Mulin Yu, Tao Lu, Linning Xu, Lihan Jiang, Yuanbo Xiangli, Bo Dai

Shanghai Artificial Intelligence Laboratory, 2The Chinese University of Hong Kong, Shanghai Artificial Intelligence Laboratory, The Chinese University of Hong Kong, Cornell University, University of Science and Technology of China

网格重建渲染

GSDF针对多视图场景中高质量渲染与精确网格重建相互牵制的问题,提出并行优化的3DGS渲染分支与SDF重建分支:用GS深度指导SDF采样,用SDF约束高斯增删,并对齐深度/法线。实验显示其加快SDF收敛、重建更细几何,同时让高斯更贴近表面,减少漂浮物和边缘模糊,提升真实与合成场景的渲染稳定性。

DreamPolisher: Towards High-Quality Text-to-3D Generation via Geometric Diffusion Figure 1
arXiv preprint2024-03-25

DreamPolisher: Towards High-Quality Text-to-3D Generation via Geometric Diffusion

Yuanze Lin, Ronald Clark, Philip Torr

University of Oxford

扩散生成

DreamPolisher针对纯文本生成3D时常见的跨视角不一致、纹理细节不足和“Janus”问题,采用3D Gaussian Splatting两阶段流程:先由文本到点扩散初始化并做几何优化,再用ControlNet外观细化器结合几何一致性损失提升纹理与视角一致性。实验显示其在多类提示上较DreamGaussian、GaussianDreamer、LucidDreamer生成更清晰且语义更贴合的3D资产,但单物体约需30分钟,且不能用图像引导。

Comp4D: LLM-Guided Compositional 4D Scene Generation Figure 1
arXiv preprint2024-03-25

Comp4D: LLM-Guided Compositional 4D Scene Generation

Neel P, Konstantinos N

扩散生成

针对现有 text-to-4D 方法受场景级 3D/4D 数据稀缺限制、常停留在单物体生成的问题,Comp4D 将复杂场景拆成独立物体与交互运动:用 GPT-4 从文本解析实体、尺度和运动轨迹,再以可变形 3D Gaussian 表示各物体,并用轨迹约束的组合式 score distillation 联合图像、视频和 3D 扩散先验优化。实验显示其在组合 4D 场景的视觉质量、运动一致性和物体交互上优于既有 text-to-4D 基线。

GaussianFlow: Splatting Gaussian Dynamics for 4D Content Creation Figure 1
arXiv preprint2024-03-19

GaussianFlow: Splatting Gaussian Dynamics for 4D Content Creation

Quankai Gao, Qiangeng Xu, Zhe Cao, Ben Mildenhall, Wenchao Ma, Le Chen, Danhang Tang, Ulrich Neumann

University of Southern California, Pennsylvania State University, Max Planck Institute for Intelligent Systems, where other existing methods suffer from undesirable artifacts

动态场景

该文针对4D Gaussian Splatting仅靠光度或SDS监督时运动欠约束、快动作易失真和颜色漂移的问题,提出Gaussian flow:将3D高斯的平移、旋转、尺度变化可微地splat到图像平面,用像素级光流直接约束高斯动态。该监督几乎不破坏GS效率,并在4D生成与4D新视角合成中提升动态一致性,尤其改善大幅运动场景,实验报告达到同类最佳效果。

Untitled 3DGS Paper 317 Figure 1
arXiv preprint2024-03-18

Untitled 3DGS Paper 317

Zhiyang Guo, Wengang Zhou, Li Li, Min Wang, Houqiang Li

动态场景

全文短总结尚未生成。

NEDS-SLAM: A Novel Neural Explicit Dense Semantic SLAM Framework using 3D Gaussian Splatting Figure 1
arXiv preprint2024-03-18

NEDS-SLAM: A Novel Neural Explicit Dense Semantic SLAM Framework using 3D Gaussian Splatting

Yiming Ji, Yang Liu, Guanghu Xie, Boyu Ma, Zongwu Xie

同步定位与建图

面向机器人在导航、交互等任务中对稠密且语义一致地图的需求,NEDS-SLAM针对3DGS语义SLAM中分割特征时序不一致、语义嵌入占内存以及离群高斯破坏跟踪的问题,提出空间一致特征融合、语义特征压缩编码和虚拟视角剪枝。其在Replica与ScanNet上取得有竞争力的建图和位姿精度,并展示较好的稠密语义重建效果。

Fed3DGS: Scalable 3D Gaussian Splatting with Federated Learning Figure 1
arXiv preprint2024-03-18

Fed3DGS: Scalable 3D Gaussian Splatting with Federated Learning

Teppei Suzuki

Denso IT Laboratory, Inc, Denso IT Laboratory

大规模场景

Fed3DGS针对城市级乃至更大规模3D重建中集中式训练带来的服务器负载、存储和维护成本问题,将3D Gaussian Splatting引入联邦学习,并用面向3DGS的蒸馏式全局更新与外观建模处理客户端非IID和季节变化。实验在多个大规模数据集上取得接近集中式方法的渲染质量,同时较FedNeRF减小模型规模、缩短客户端训练时间;但文中未充分处理弱算力客户端等真实联邦部署问题。

Bridging 3D Gaussian and Mesh for Freeview Video Rendering Figure 1
arXiv preprint2024-03-18

Bridging 3D Gaussian and Mesh for Freeview Video Rendering

Yuting Xiao, Xuan Wang, Jiafei Li, Hongrui Cai, Yanbo Fan, Nan Xue, Minghui Yang, Yujun Shen, Shenghua Gao

University of Science and Technology of China, ShanghaiTech University, Xi’an Jiaotong University, Chinese Academy of Sciences, University of Hong Kong

数字人动态场景网格重建

针对单一网格难以刻画头发、睫毛等复杂结构,而纯 3DGS 在光滑面高频纹理上易模糊并浪费 splats 的问题,文中提出 HERA/GauMesh 式混合显式表示:用 UV 纹理网格负责皮肤等平滑表面细节,用绑定到网格面的 3D Gaussian 表达细小几何,并通过混合光栅化与稳定深度排序做 α 融合。实验显示其在新视角与新表情合成上优于多种头像方法,且可实时渲染;但题名与正文方法名存在不一致。

Beyond Uncertainty: Risk-Aware Active View Acquisition for Safe Robot Navigation and 3D Scene Understanding with FisherRF Figure 1
arXiv preprint2024-03-18

Beyond Uncertainty: Risk-Aware Active View Acquisition for Safe Robot Navigation and 3D Scene Understanding with FisherRF

Guangyi Liu, Wen Jiang, Boshu Lei, Vivek Pandey, Kostas Daniilidis, Nader Motee

自动驾驶机器人

面向灾害搜救等未知环境中的安全导航,论文指出仅按重建不确定性选视角会忽略碰撞风险。其核心是将3D Gaussian地图的不确定距离转为AVaR等一致风险度量,用RaEM屏蔽低风险区域,并结合FisherRF在安全关键区域最大化信息增益选择NBV。高保真实验显示风险估计更接近真值并提升任务相关重建效率,但真实机器人验证仍有限。

BAGS: Building Animatable Gaussian Splatting from a Monocular Video with Diffusion Priors Figure 1
arXiv preprint2024-03-18

BAGS: Building Animatable Gaussian Splatting from a Monocular Video with Diffusion Priors

Tingyang Zhang, Qingzhe Gao, Weiyu Li, Libin Liu, Baoquan Chen

National Key Lab of General AI, Peking University, China, National Key Lab of General AI, Peking University, Shandong University, China, Shandong University, The Hong Kong University of Science and Technology

扩散生成动态场景单目重建

BAGS面向单目随手拍视频中视角覆盖不足、NeRF式可动画重建训练和渲染慢的问题,将可动画3D Gaussian Splatting与神经骨骼/LBS结合,并用扩散先验补全未观测视角外观,再以刚性正则抑制先验带来的时序不一致和伪影。作者在多段真实视频上与BANMo等方法比较,报告几何、外观和动画质量更好,同时训练更快、可实时渲染。

Creating Seamless 3D Maps Using Radiance Fields Figure 1
arXiv preprint2024-03-17

Creating Seamless 3D Maps Using Radiance Fields

Sai Tarun, Thomas B

Department of Computer Science, Golisano College of Computing and Information Sciences, Rochester Institute of Technology

其他

面向导航、虚拟旅游和校园/城市建图中从少量2D图像生成真实3D模型的需求,论文比较NeRF与3D Gaussian Splatting,并尝试建立自动化工作流以缓解传统摄影测量对大量图像、里程计及反光表面敏感的问题。其“改进重建结果”的具体方法文中未充分说明;实验显示在RIT场景上Gaussian Splatting的重建与渲染效果优于NeRF。

BrightDreamer: Generic 3D Gaussian Generative Framework for Fast Text-to-3D Synthesis Figure 1
arXiv preprint2024-03-17

BrightDreamer: Generic 3D Gaussian Generative Framework for Fast Text-to-3D Synthesis

Lutao Jiang, Lin Wang

扩散生成

针对现有文本到3D方法需为每个提示反复优化、难以快速泛化到新文本的问题,BrightDreamer将直接生成海量3D Gaussian转化为从锚点形状预测文本引导的三维形变,并用改进的文本引导triplane生成其尺度、旋转、不透明度和SH属性。实验显示其可在约77ms前向生成、705 FPS渲染,并较好处理复杂语义提示,但细粒度几何细节仍受限。

3DGS-ReLoc: 3D Gaussian Splatting for Map Representation and Visual ReLocalization Figure 1
arXiv preprint2024-03-17

3DGS-ReLoc: 3D Gaussian Splatting for Map Representation and Visual ReLocalization

Peng Jiang, Gaurav Pandey, Srikanth Saripalli

位姿估计

面向自动驾驶/机器人在大规模户外场景中的视觉重定位,3DGS-ReLoc将LiDAR初始化的3D Gaussian Splatting作为几何准确且可渲染的地图表示,并用2D体素与KD-tree降低显存和加速查询;重定位时通过NCC检索渲染视图,再以SuperPoint/LightGlue匹配和PnP迭代细化位姿。在KITTI360复访序列上验证了该表示用于重定位的可行性与精度,但相对传统LiDAR/特征地图的定量增益幅度文中未充分说明。

GS-Pose: Cascaded Framework for Generalizable Segmentation-based 6D Object Pose Estimation Figure 1
arXiv preprint2024-03-15

GS-Pose: Cascaded Framework for Generalizable Segmentation-based 6D Object Pose Estimation

Dingding Cai, Janne Heikkilä, Esa Rahtu

Tampere University, Finland, Tampere University, University of Oulu, Finland, University of Oulu

位姿估计

面向机器人操作中未知物体的 RGB-only 6D 位姿估计,GS-Pose 试图摆脱实例重训练、外部检测器和高精 CAD 模型依赖。其核心洞察是不要用单一表示贯穿全流程,而是为检测、粗位姿检索和精修分别构建语义表示、旋转感知模板嵌入与 3D Gaussian 对象,并用可微渲染做 render-and-compare 优化。在 LINEMOD 与 OnePose-LowTexture 上报告达到新的 SOTA,但细长/薄结构仍受分割质量限制。

GGRt: Towards Generalizable 3D Gaussians without Pose Priors in Real-Time Figure 1
arXiv preprint2024-03-15

GGRt: Towards Generalizable 3D Gaussians without Pose Priors in Real-Time

Hao Li, Yuanyuan Gao, Dingwen Zhang, Chenming Wu, Yalun Dai, Chen Zhao, Haocheng Feng, Errui Ding, Jingdong Wang, Junwei Han

Brain and Artificial Intelligence Lab, Northwestern Polytechnical University, Brain and Artificial Intelligence Lab, Northwestern Polytechnical University, Department of Computer Vision, Baidu Inc. {wuchenming, zhaochen03, Department of Computer Vision, Baidu Inc. {wuchenming

位姿估计

GGRt针对现有可泛化新视角合成依赖真实相机位姿、难处理高分辨率且推理/渲染慢的问题,将迭代位姿优化网络与可泛化3D Gaussian联合训练,从图像中估计相对位姿,并用延迟反传和渐进Gaussian缓存提升分辨率与速度。实验在LLFF、KITTI、Waymo上优于无位姿NeRF方法,接近有位姿3D-GS,达到≥5 FPS重建和≥100 FPS渲染。

FDGaussian: Fast Gaussian Splatting from Single Image via Geometric-aware Diffusion Model Figure 1
arXiv preprint2024-03-15

FDGaussian: Fast Gaussian Splatting from Single Image via Geometric-aware Diffusion Model

Qijun Feng, Zhen Xing, Zuxuan Wu, Yu-Gang Jiang

Department of Computer Science, Fudan University

扩散生成

面向单张图像重建3D物体时信息不足、多视角扩散结果不一致且Gaussian Splatting优化冗余的问题,论文提出两阶段框架:用正交平面分解为扩散模型注入几何条件生成一致多视图,再通过极线注意力融合视图,并以GDS剪枝split/clone操作。Objaverse和GSO实验显示其在视角一致性、几何细节和重建质量上优于对比方法,同时加快优化;具体增益分解仍主要依赖消融支撑。

Den-SOFT: Dense Space-Oriented Light Field DataseT for 6-DOF Immersive Experience Figure 1
arXiv preprint2024-03-15

Den-SOFT: Dense Space-Oriented Light Field DataseT for 6-DOF Immersive Experience

Xiaohang Yu, Zhengxian Yang, Shi Pan, Yuqi Han, Haoxiang Wang, Jun Zhang, Shi Yan, Borong Lin, Lei Yang, Tao Yu, Lu Fang

are of interest to researchers in the field of 3D reconstruction during the, Both authors contributed equally to this research

虚拟现实

针对现有3D重建数据集分辨率低、采样稀疏且多偏物体中心,难以支撑大空间6DoF VR的问题,Den-SOFT构建移动多相机密集光场采集系统,以5K、多视角高密度覆盖室内外场景并保留天空、反射、光影等难点元素。作者用IBRNet、NeRF、3DGS等验证数据集,并将3DGS结果接入Unity展示可交互VR空间潜力;效果提升可能主要来自更高密度与更大规模数据。

Controllable Text-to-3D Generation via Surface-Aligned Gaussian Splatting Figure 1
3DV 20252024-03-15

Controllable Text-to-3D Generation via Surface-Aligned Gaussian Splatting

Zhiqi Li, Yiming Chen, Lingzhe Zhao, Peidong Liu

Zhejiang University, Westlake University

扩散生成

本文瞄准文本到3D与图像到3D之间尚少研究的可控文本到3D生成,试图用边缘、深度、法线或涂鸦等条件精细约束多视图一致内容。核心是提出MVControl,将局部/全局条件与相机位姿注入冻结的MVDream,并结合LGM、SDS混合引导和表面对齐的SuGaR高斯-网格表示,以提升效率和几何质量。实验显示其在多种控制信号下能生成更稳定、可控的高质量多视图图像和3D资产。

RAIN-GS: Relaxing Accurate Initialization Constraint for 3D Gaussian Splatting Figure 1
arXiv preprint2024-03-14

RAIN-GS: Relaxing Accurate Initialization Constraint for 3D Gaussian Splatting

Jaewoo Jung, Jisang Han, Honggyu An, Jiwon Kang, Seonghoon Park, Seungryong Kim

Korea University

密度控制渲染

RAIN-GS针对3DGS强依赖SfM精确点云初始化的问题,分析指出SfM主要提供低频结构,而原优化难以把高斯从初始位置远距离搬运。方法以稀疏大方差初始化、渐进低通渲染和ABE-Split密度控制促成由粗到细学习。多数据集结果显示,随机点云训练也能达到甚至超过SfM初始化3DGS,但细节不足问题仍存在。

Hyper-3DG:Text-to-3D Gaussian Generation via Hypergraph Figure 1
arXiv preprint2024-03-14

Hyper-3DG:Text-to-3D Gaussian Generation via Hypergraph

Donglin Di, Jiahui Yang, Chaofan Luo, Zhou Xue, Wei Chen, Xun Yang, Yue Gao

School of Software, Tsinghua University, 100084, Beijing, China, School of Software, Tsinghua University, School of Information Science and Technology, University of Science, School of Information Science and Technology, University of Science, Harbin Institute of Technology, Harbin, 150001, Heilongjiang, China, Harbin Institute of Technology

扩散生成

针对文本到3D生成中几何与纹理高阶关联被忽视、导致过平滑、过饱和和 Janus 问题,Hyper-3DG 在3D Gaussian主流程中加入 HGRefiner,将高斯划分为 patch,并在显式属性与2D特征空间上做超图学习以协同细化局部结构和外观。实验显示其可提升生成质量并加速/不增加主干计算开销,但复杂场景与强文本先验下仍可能退化。

A New Split Algorithm for 3D Gaussian Splatting Figure 1
arXiv preprint2024-03-14

A New Split Algorithm for 3D Gaussian Splatting

Ralph R

QIYUAN FENG, BNRist, Department of Computer Science and Technology, Tsinghua University, China, Department of Computer Science and Technology, Tsinghua University, GENGCHEN CAO, BNRist, Department of Computer Science and Technology, Tsinghua University, China, HAOXIANG CHEN, BNRist, Department of Computer Science and Technology, Tsinghua University, China, TAI-JIANG MU, BNRist, Department of Computer Science and Technology, Tsinghua University, China, RALPH R. MARTIN, Cardiff University, UK, Cardiff University, SHI-MIN HU, BNRist, Department of Computer Science and Technology, Tsinghua University, China

密度控制

该文针对 3D Gaussian Splatting 中高斯尺度/结构不均导致的表面模糊、针状伪影和无纹理区域点云稀疏问题,提出按零/一/二阶矩守恒将任意 N 维高斯闭式拆成两个高斯的密度控制方法。该拆分在保持外观与参数统计一致性的同时使高斯更均匀、更贴合表面;实验显示可改善显式编辑边界、点云抽取密度,并提升新视角渲染质量。

GSEdit: Efficient Text-Guided Editing of 3D Objects via Gaussian Splatting Figure 1
arXiv preprint2024-03-08

GSEdit: Efficient Text-Guided Editing of 3D Objects via Gaussian Splatting

Francesco Palandra, Andrea Sanchietti, Daniele Baieri, Emanuele Rodolà

FRANCESCO PALANDRA∗and ANDREA SANCHIETTI∗, Sapienza University of Rome, Italy, Sapienza University of Rome, DANIELE BAIERI, Sapienza University of Rome, Italy, EMANUELE RODOLÀ, Sapienza University of Rome, Italy, previously proposed methods relying on NeRF-like MLP models, GS-Edit, previously proposed methods relying on NeRF-like MLP models

编辑

面向3D资产编辑中NeRF式方法耗时、网格拓扑难处理的问题,GSEdit用Gaussian Splatting表示对象,并结合Instruct-Pix2Pix生成的多视角监督与改写后的SDS损失直接优化高斯参数。它可从网格或DreamGaussian结果输入,在消费级硬件数分钟内完成文本驱动的外观/形状修改,实验显示较多类NeRF或体素编辑更快,同时基本保持多视角一致性和原物体细节。

GVA: Reconstructing Vivid 3D Gaussian Avatars from Monocular Videos Figure 1
arXiv preprint2024-02-26

GVA: Reconstructing Vivid 3D Gaussian Avatars from Monocular Videos

Xinqi Liu, Chenming Wu, Jialun Liu, Xing Liu, Jinbo Wu, Chen Zhao, Haocheng Feng, Errui Ding, Jingdong Wang

数字人

GVA面向低成本单目视频数字人重建,针对现有3D Gaussian头像在手脚姿态不准、点分布受纹理聚集和初始化偏置影响而导致驱动伪影的问题,提出基于法线与轮廓对齐的姿态细化,以及表面引导的Gaussian重初始化,使点更贴合人体表面。实验显示其在照片级新视角合成上达到SOTA,并支持更细粒度的身体与手部姿态控制。

Spec-Gaussian: Anisotropic View-Dependent Appearance for 3D Gaussian Splatting Figure 1
arXiv preprint2024-02-24

Spec-Gaussian: Anisotropic View-Dependent Appearance for 3D Gaussian Splatting

Ziyi Yang, Xinyu Gao, Yangtian Sun, Yihua Huang, Xiaoyang Lyu, Wen Zhou, Shaohui Jiao, Xiaojuan Qi, Xiaogang Jin

Zhejiang University, The University of Hong Kong, ByteDance Inc

渲染

针对 3D Gaussian Splatting 以低阶球谐表示视角相关外观时难以捕捉高频高光、各向异性材质的问题,Spec-Gaussian 将每个高斯的外观场替换为各向异性球面高斯,并配合由粗到细训练抑制真实场景中的 floaters、提升效率。实验显示其在含镜面高光和各向异性表面的场景中渲染质量优于既有方法,且无需增加高斯数量,但对环境反射的建模仍受限。

GaussianPro: 3D Gaussian Splatting with Progressive Propagation Figure 1
arXiv preprint2024-02-22

GaussianPro: 3D Gaussian Splatting with Progressive Propagation

Kai Cheng, Xiaoxiao Long, Kaizhi Yang, Yao Yao, Wei Yin, Yuexin Ma, Wenping Wang, Xuejin Chen

密度控制

GaussianPro针对3DGS过度依赖SfM初始化、在大场景低纹理区域点云稀疏导致几何噪声和覆盖不足的问题,引入受MVS启发的渐进传播密度控制:从已重建高斯渲染深度/法线,在图像邻域传播候选并用多视图光度一致性筛选,再反投影生成位置和方向更可靠的新高斯,并用法线/平面约束优化。实验在Waymo和MipNeRF360上优于3DGS,Waymo PSNR提升约1.15dB。

How NeRFs and 3D Gaussian Splatting are Reshaping SLAM: a Survey Figure 1
arXiv preprint2024-02-20

How NeRFs and 3D Gaussian Splatting are Reshaping SLAM: a Survey

Martin R

University of Bologna, Italy, University of Bologna, ETH Zurich, Switzerland, ETH Zurich, University of Amsterdam, Netherlands, University of Amsterdam, Netherlands

综述

针对 NeRF 与 3D Gaussian Splatting 快速进入 SLAM、但缺少系统梳理的问题,本文回顾近三年约 80 个辐射场启发的 SLAM 系统,按表示、跟踪、建图、渲染与性能等维度建立分类并比较。核心洞察是连续/可学习表示可缓解离散地图的稀疏、分辨率和补全问题,但实时性、大场景扩展、鲁棒性和统一评测仍是主要瓶颈。

GaussianHair: Hair Modeling and Rendering with Light-aware Gaussians Figure 1
arXiv preprint2024-02-16

GaussianHair: Hair Modeling and Rendering with Light-aware Gaussians

Haimin Luo, Min Ouyang, Zijun Zhao, Suyi Jiang, Longwen Zhang, Qixuan Zhang, Wei Yang, Lan Xu, Jingyi Yu

ShanghaiTech University 2Huazhong University of Science and Technology

数字人

面向数字人中真实发型难以从普通图像重建、又需可重光照和动画的问题,GaussianHair将每根发丝表示为连接的细长圆柱3D高斯,并引入面向发丝的散射模型与可微渲染优化,从而兼顾显式几何、外观和CG流程兼容性。实验显示其在发丝几何与外观保真度上优于既有重建方法,并支持编辑、重光照和动态渲染。

GaussianObject: Just Taking Four Images to Get A High-Quality 3D Object with Gaussian Splatting Figure 1
arXiv preprint2024-02-15

GaussianObject: Just Taking Four Images to Get A High-Quality 3D Object with Gaussian Splatting

Chen Yang, Sikuang Li, Jiemin Fang, Ruofan Liang, Lingxi Xie, Xiaopeng Zhang, Wei Shen, Qi Tian

CHEN YANG∗, MoE Key Lab of Artificial Intelligence, AI Institute, SJTU, China, MoE Key Lab of Artificial Intelligence, AI Institute, SIKUANG LI∗, MoE Key Lab of Artificial Intelligence, AI Institute, SJTU, China, JIEMIN FANG†, Huawei Inc., China, Huawei Inc, RUOFAN LIANG, University of Toronto, Canada, University of Toronto, LINGXI XIE, Huawei Inc., China, XIAOPENG ZHANG, Huawei Inc., China

扩散生成

GaussianObject面向仅有4张环绕照片时物体3D重建易过拟合、缺少多视角一致性且遮挡区域信息缺失的问题,以3D Gaussian Splatting为显式表示,先用视觉壳初始化和漂浮点剔除注入结构先验,再用自生成配对数据训练的扩散修复模型补全受损视图并反向细化高斯。实验在MipNeRF360、OmniObject3D和OpenIllumination上较3DGS、NeRF类少视图方法及FSGS取得更高感知质量,并提供无需精确相机位姿的COLMAP-free变体。

IM-3D: Iterative Multiview Diffusion and Reconstruction for High-Quality 3D Generation Figure 1
arXiv preprint2024-02-13

IM-3D: Iterative Multiview Diffusion and Reconstruction for High-Quality 3D Generation

Luke Melas-Kyriazi, Iro Laina, Christian Rupprecht, Natalia Neverova, Andrea Vedaldi, Oran Gafni, Filippos Kokkinos

扩散生成

针对现有文本/图像到3D方法依赖SDS导致耗时、视角不一致和伪影的问题,IM-3D将多视角生成从图像扩散转向微调的视频扩散模型,一次生成360°一致视图,再用Gaussian Splatting和感知损失直接重建,并通过少量迭代把重建结果反馈给扩散模型修正。实验显示其将2D生成器调用减少10–100倍,约数分钟产出资产,同时提升几何一致性、可用率和视觉质量。

GALA3D: Towards Text-to-3D Complex Scene Generation via Layout-guided Generative Gaussian Splatting Figure 1
arXiv preprint2024-02-11

GALA3D: Towards Text-to-3D Complex Scene Generation via Layout-guided Generative Gaussian Splatting

Xiaoyu Zhou, Xingjian Ran, Yajiao Xiong, Jinlin He, Zhiwei Lin, Yongtao Wang, Deqing Sun, Ming-Hsuan Yang

扩散生成

针对现有文本到3D方法难以生成多物体复杂场景、布局需手工指定且易出现几何失真和视角漂移的问题,GALA3D用LLM从文本解析粗布局,并以布局引导的3D Gaussian Splatting、布局细化和实例—场景组合优化约束几何、纹理、尺度与交互。实验显示其在场景级生成和可控编辑上优于已有方法,但具体量化增益的来源仍需结合消融进一步判断。

3D Gaussian as a New Vision Era: A Survey Figure 1
arXiv preprint2024-02-11

3D Gaussian as a New Vision Era: A Survey

Ben Fei, Jingyi Xu, Rui Zhang, Qingyuan Zhou, Weidong Yang, Ying He

综述

判断受限于 PDF 文本抽取质量。该综述面向 3D Gaussian Splatting 快速扩张但脉络分散的问题,梳理其作为显式场景表示替代 NeRF 的基础、分类与应用;核心洞察是 3D-GS 在机器人 SLAM、导航、目标重建与自动驾驶动态场景建模中可提供更稠密连续的地图和视图合成能力。主要结果是汇总近一年工作并指出与传感器融合、实时规划结合仍需验证。

ImplicitDeepfake: Plausible Face-Swapping through Implicit Deepfake Generation using NeRF and Gaussian Splatting Figure 1
arXiv preprint2024-02-09

ImplicitDeepfake: Plausible Face-Swapping through Implicit Deepfake Generation using NeRF and Gaussian Splatting

Georgii Stanishevskii, Jakub Steczkiewicz, Tomasz Szczepanik, Jacek Tabor

Jagiellonian University, Faculty of Mathematics and Computer Science, Cracow, Poland, Jagiellonian University, Department of Engineering, University of Cambridge, Cambridge, UK, Department of Engineering, University of Cambridge

数字人

面向虚拟环境中高真实感数字头像的低成本生成,论文将单图2D换脸先应用到多视角训练图像,再用NeRF或Gaussian Splatting重建为3D头像,并可叠加扩散模型文本编辑。实验显示该简单流水线能在未见视角保持较一致的换脸效果,其中GS渲染更逼真且指标优于NeRF(平均PSNR 41.73 vs 37.64,LPIPS 0.03 vs 0.06),但安全边界与泛化评估仍较有限。

Mesh-based Gaussian Splatting for Real-time Large-scale Deformation Figure 1
arXiv preprint2024-02-07

Mesh-based Gaussian Splatting for Real-time Large-scale Deformation

Lin Gao, Jie Yang, Bo-Tao Zhang, Jia-Mu Sun, Yu-Jie Yuan, Hongbo Fu, Yu-Kun Lai

动态场景网格重建

针对 3D Gaussian Splatting 虽能实时高质量渲染、但离散高斯缺少拓扑导致大尺度编辑易错位和产生伪影的问题,论文将高斯绑定到显式网格上,用网格面分裂指导高斯分裂,并以网格约束正则化异常高斯;变形时再由网格形变梯度更新高斯参数。实验显示其在新视角合成上 PSNR/SSIM 略优于 3D-GS、SuGaR 等基线,并可实现平均约 65 FPS 的交互式大尺度变形。

GaMeS: Mesh-Based Adapting and Modification of Gaussian Splatting Figure 1
arXiv preprint2024-02-02

GaMeS: Mesh-Based Adapting and Modification of Gaussian Splatting

Piotr Borycki, Jacek Tabor

Jagiellonian University, Doctoral School of Exact and Natural Sciences, Department of Engineering, University of Cambridge

动态场景网格重建

GaMeS针对3D Gaussian Splatting虽能实时渲染但难以对数十万高斯进行可控编辑的问题,将平面高斯绑定到网格三角面顶点,并在无网格时构造可编辑的Triangle Soup伪网格,使位置、尺度和旋转随网格变形自动更新。实验显示其在静态场景质量接近原GS,同时支持实时物体编辑与动画;但大面片大幅变形会产生伪影,面片拆分后的高斯更新文中未充分说明。

Segment Anything in 3D Gaussians Figure 1
arXiv preprint2024-01-31

Segment Anything in 3D Gaussians

Xu Hu, Yuxi Wang, Lue Fan, Junsong Fan, Junran Peng, Zhen Lei, Qing Li, Zhaoxiang Zhang

分割

3D-GS 虽渲染高效,但高斯缺少几何约束,单个高斯可能跨越多个物体,导致交互式物体分割边界粗糙或不完整。该工作将 SAM 以免训练方式提升到 3D-GS,通过单视图点击生成多视图掩码、视角标签分配与跨视角投票获得一致分割,并用 Gaussian Decomposition 拆分边界高斯。实验显示其能改善边界质量,在多场景分割、编辑和碰撞检测中取得较稳定效果,但稀疏高斯区域仍有限制。

Endo-4DGS: Endoscopic Monocular Scene Reconstruction with 4D Gaussian Splatting Figure 1
arXiv preprint2024-01-29

Endo-4DGS: Endoscopic Monocular Scene Reconstruction with 4D Gaussian Splatting

Yiming Huang, Beilei Cui, Long Bai, Ziqi Guo, Mengya Xu, Mobarakol Islam, Hongliang Ren

Department of Electronic Engineering, The Chinese University of Hong Kong (CUHK), Sha Tin, Hong Kong SAR, China, Department of Electronic Engineering, The Chinese University of Hong Kong (CUHK), Shenzhen Research Institute, CUHK, Shenzhen, China, Shenzhen Research Institute, Chinese University of Hong Kong, City University of Hong Kong, Shenzhen Research Institute, City University of Hong Kong, University of Hong Kong, Wellcome/EPSRC Centre for Interventional and Surgical Sciences (WEISS), University College London, London, UK, Wellcome/EPSRC Centre for Interventional and Surgical Sciences (WEISS), University College London

动态场景医学影像

面向机器人辅助手术中内窥镜视野窄、组织持续形变且真实深度难获取的问题,Endo-4DGS将4D Gaussian Splatting用于单目动态重建,用轻量MLP建模高斯形变,并借助Depth-Anything伪深度初始化;再通过置信度引导、法向约束和深度正则降低单目深度噪声影响。在两个真实手术数据集上,方法相比NeRF类方案实现更快训练与实时渲染,同时保持较高重建精度和较低显存开销。

Gaussian Splashing: Dynamic Fluid Synthesis with Gaussian Splatting Figure 1
arXiv preprint2024-01-27

Gaussian Splashing: Dynamic Fluid Synthesis with Gaussian Splatting

Yutao Feng, Xiang Feng, Yintong Shang, Ying Jiang, Chang Yu, Zeshun Zong, Tianjia Shao, Hongzhi Wu, Kun Zhou, Chenfanfu Jiang, Yin Yang

University of Utah, USA, University of Utah, Zhejiang University, China, Zhejiang University

动态场景物理建模渲染

这篇论文针对静态 3DGS 难以支持真实流固交互和流体高光的问题,提出 Gaussian Splashing,将高斯核同时作为渲染与 PBD 物理粒子,并为其加入法线、各向异性约束和面向流体表面的高光估计。结果展示了在重建场景中实现水浪、漂浮物、可变形体和刚体的双向耦合动态,并能从新视角保持较可信的漫反射与镜面渲染。

GauU-Scene: A Scene Reconstruction Benchmark on Large Scale 3D Reconstruction Dataset Using Gaussian Splatting Figure 1
arXiv preprint2024-01-25

GauU-Scene: A Scene Reconstruction Benchmark on Large Scale 3D Reconstruction Dataset Using Gaussian Splatting

Butian Xiong, Zhuo Li, Zhen Li

The Chinese University of Hong Kong, Shenzhen, The Chinese University of Hong Kong

大规模场景

该文针对城市级三维重建缺少同步、多模态真值和屋顶视角的问题,构建了覆盖超1.5平方公里的U-Scene无人机RGB+LiDAR数据集,并以Gaussian Splatting建立基准。核心洞察是仅依赖图像的3DGS在大场景近距离观察会模糊、与LiDAR点云存在结构差异;引入LiDAR先验的融合方法在定量和定性重建上有所提升,但具体增益来源仍可能主要来自数据与尺度。

PSAvatar: A Point-based Morphable Shape Model for Real-Time Head Avatar Creation with 3D Gaussian Splatting Figure 1
arXiv preprint2024-01-23

PSAvatar: A Point-based Morphable Shape Model for Real-Time Head Avatar Creation with 3D Gaussian Splatting

Zhongyuan Zhao, Zhenyu Bao, Qing Li, Guoping Qiu, Kanglin Liu

数字人

PSAvatar面向头部数字人在实时性与高保真之间的矛盾:3DMM难表示头发、眼镜等非面部结构,隐式方法又渲染慢且形变控制复杂。其核心是将FLAME表面及表外采样转为点式形状模型,通过分析-合成对齐头部形变,再结合3D Gaussian建模细节与外观。实验显示可重建多主体高质量头像,并在RTX 3090上以512×512分辨率达到约25fps实时动画。

EndoGaussian: Gaussian Splatting for Deformable Surgical Scene Reconstruction Figure 1
arXiv preprint2024-01-23

EndoGaussian: Gaussian Splatting for Deformable Surgical Scene Reconstruction

Yifan Liu, Chenxin Li, Chen Yang, Yixuan Yuan

The Chinese University of Hong Kong, City University of Hong Kong

三维高斯泼溅

面向内窥镜下可变形组织重建,既有 NeRF 类方法渲染慢,难以用于术中实时场景。EndoGaussian 将动态手术场景表示为规范 3D 高斯加时变形变场,并用深度估计驱动的整体初始化替代稀疏 SfM,再以编码体素和轻量 MLP 跟踪组织运动。公开数据集上其训练约 2 分钟/场景,渲染约 195 FPS、较前法约 100×,同时保持约 37.8 PSNR 的重建质量。

GaussianBody: Clothed Human Reconstruction via 3d Gaussian Splatting Figure 1
arXiv preprint2024-01-18

GaussianBody: Clothed Human Reconstruction via 3d Gaussian Splatting

Mengtian Li, Shengxiang Yao, Zhifeng Xie, Keyu Chen, Yu-Gang Jiang

数字人

GaussianBody面向动态穿衣人体重建中NeRF训练慢、渲染难实时且网格难表达宽松衣物细节的问题,将3D Gaussian Splatting引入人体序列,用SMPL/LBS在规范空间与观测空间间显式驱动高斯变形,并加入物理启发的局部刚性正则、姿态细化和按尺度分裂以减少变形歧义与点云稀疏。实验显示其在单目动态人体上获得更清晰的新视角渲染和显式几何,训练约一小时并接近实时渲染。

CoSSegGaussians: Compact and Swift Scene Segmenting 3D Gaussians Figure 1
arXiv preprint2024-01-11

CoSSegGaussians: Compact and Swift Scene Segmenting 3D Gaussians

Bin Dou, Tianyu Zhang, Yongjia Ma, Zhaohui Wang, Zejian Yuan

Institute of Artificial Intelligence and Robotics, Xi’an Jiaotong University, Institute of Artificial Intelligence and Robotics, Xi’an Jiaotong University

分割

针对零标注神经场景分割中 NeRF 推理慢、3DGS 为每个高斯学习冗余标签而易过拟合跨视角伪标签不一致的问题,CoSSegGaussians 用特征反投影把 DINO 等2D语义特征汇入高斯点,并融合 RandLA-Net 空间特征,经共享浅层解码器和 CoSeg 噪声鲁棒损失输出更紧凑的分割。实验显示其在零样本语义分割上较最佳基线约提升 10% mIoU,分割耗时约为 NeRF 方法的十分之一。

AGG: Amortized Generative 3D Gaussians for Single Image to 3D Figure 1
arXiv preprint2024-01-08

AGG: Amortized Generative 3D Gaussians for Single Image to 3D

Dejia Xu, Ye Yuan, Morteza Mardani, Sifei Liu, Jiaming Song, Zhangyang Wang, Arash Vahdat

University of Texas at Austin

扩散生成

针对单图生成 3D Gaussian 依赖逐实例 SDS/重建优化、推理耗时高的问题,AGG 将其改为摊销式前馈生成:先用几何与纹理解耦的混合表示预测粗 Gaussian,再通过点-体素 UNet 超分辨率细化,并用伪标签预热和固定尺度处理训练不稳定。实验显示其质量接近优化式 Gaussian 与其他采样式 3D 表示方法,同时推理速度提升数个数量级。

Progress and Prospects in 3D Generative AI: A Technical Overview including 3D human Figure 1
arXiv preprint2024-01-05

Progress and Prospects in 3D Generative AI: A Technical Overview including 3D human

Song Bai, Jie Li

University of California, Riverside, University of California

数字人综述

面对2023年以来3D生成、数字人与动作生成论文快速增长,本文梳理NeRF/3DGS表示、多视角扩散一致性、SMPL-X人体先验与LLM文本到动作等技术脉络。核心洞察是当前进展主要来自2D扩散模型迁移、多视角控制和更大3D数据集,而非单一新算法;文中总结了从高质量8K生成到Direct2.5约10秒出模的代表性结果,但缺少统一基准,部分增益可能主要来自scaling / data。

Street Gaussians for Modeling Dynamic Urban Scenes Figure 1
arXiv preprint2024-01-02

Street Gaussians for Modeling Dynamic Urban Scenes

Yunzhi Yan, Haotong Lin, Chenxu Zhou, Weijie Wang, Haiyang Sun, Kun Zhan, Xianpeng Lang, Xiaowei Zhou, Sida Peng

Zhejiang University1

自动驾驶

面向自动驾驶仿真中动态街景重建训练慢、渲染慢的问题,Street Gaussians 将城市场景显式分解为背景与前景车辆的 3D Gaussian 点云,并结合可优化车辆位姿与 4D 球谐外观建模时间变化。该表示便于组合与编辑,在 KITTI、Waymo 上取得优于既有方法的视图合成质量,约半小时训练后可在 1066×1600 分辨率达到 135 FPS。

SyncTweedies: A General Generative Framework Based on Synchronized Diffusions Figure 1
arXiv preprint2024-01-01

SyncTweedies: A General Generative Framework Based on Synchronized Diffusions

Jaihoon Kim, Juil Koo, Minhyuk Sung, Kyeongmin Yeo

扩散生成

论文针对全景、3D 网格/高斯泼溅纹理等非标准视觉内容缺少大规模训练数据的问题,尝试直接复用预训练图像扩散模型。核心洞察是把目标空间与多个图像实例空间用投影/反投影连接,并比较同步扩散的多种聚合时机;作者发现同步 Tweedie 公式估计的干净样本、而非噪声或最终结果,适用性和质量最好。实验显示该零样本方法在全景生成、网格与 3DGS 纹理等任务上优于微调、SDS 优化和迭代更新基线。

MotionGS: Exploring Explicit Motion Guidance for Deformable 3D Gaussian Splatting Figure 1
arXiv preprint2024-01-01

MotionGS: Exploring Explicit Motion Guidance for Deformable 3D Gaussian Splatting

Hanzhi Chang, Jiacheng Deng, Yanzhe Liang, Jiahao Lu, Wenfei Yang, Tianzhu Zhang, Yongdong Zhang, Ruijie Zhu

University of Science and Technology of China

动态场景

MotionGS针对动态3D Gaussian Splatting仅靠外观监督、缺少物体运动约束而易陷入优化困难的问题,引入显式运动先验:将光流解耦为相机流与物体运动流,用后者直接监督高斯形变,并交替细化相机位姿以缓解动态场景位姿误差。在单目动态场景实验中,文中报告其在NeRF-DS和HyperNeRF上优于现有方法,兼顾重建质量与实时渲染。

Mini-Splatting: Representing Scenes with a Constrained Number of Gaussians Figure 1
arXiv preprint2024-01-01

Mini-Splatting: Representing Scenes with a Constrained Number of Gaussians

Guangchi Fang, Bing Wang

Spatial Intelligence Group, The Hong Kong Polytechnic University, Hung Hom, Hong Kong, The Hong Kong Polytechnic University, Hong Kong Polytechnic University

密度控制渲染

Mini-Splatting关注3D Gaussian Splatting在高斯数量受限时的低效表示问题,指出原始3DGS存在中心重叠与表面欠重建,直接按重要性剪枝会破坏局部几何。方法通过模糊区域分裂、深度重初始化,以及保交集和重要性采样来重排高斯空间分布,而非简单删点;在多个数据集上提升渲染质量,同时降低资源消耗并改善存储压缩。

LightGaussian: Unbounded 3D Gaussian Compression with 15x Reduction and 200+ FPS Figure 1
arXiv preprint2024-01-01

LightGaussian: Unbounded 3D Gaussian Compression with 15x Reduction and 200+ FPS

Zhiwen Fan, Kevin Wang, Zhangyang Wang, Kairun Wen, Dejia Xu, Zehao Zhu

The University of Texas at Austin

压缩

LightGaussian针对3D Gaussian Splatting在无界大场景中高斯数量膨胀、单场景存储达GB级且拖慢渲染的问题,认为冗余同时存在于高斯点数量和特征维度。方法用全局重要性评估进行剪枝与恢复,并通过SH蒸馏、伪视角增强和按重要性自适应的向量量化压缩属性。实验在Mip-NeRF 360和Tank & Temple上实现平均约15倍压缩,FPS由144提升到237,SSIM仅小幅下降,并可迁移到Scaffold-GS。

HumanSplat: Generalizable Single-Image Human Gaussian Splatting with Structure Priors Figure 1
arXiv preprint2024-01-01

HumanSplat: Generalizable Single-Image Human Gaussian Splatting with Structure Priors

Zhen Fan, Zeming Li, Chenguo Lin, Yebin Liu, Yadong Mu, Panwang Pan, Tingting Shen, Zhuo Su, Yongjie Zhang

ByteDance, 2Peking University, 3Xiamen University, 4Tsinghua University, ByteDance, Peking University, Xiamen University, Tsinghua University

数字人

HumanSplat针对单图人体重建依赖多视图采集或逐实例优化、难以落地的问题,将多视图扩散模型的外观补全能力与SMPL结构先验结合,在潜空间中用Transformer直接预测3D Gaussian属性,并用语义分层损失强化脸、手等细节。实验显示其在标准基准和野外图像上取得优于现有方法的照片级新视角合成效果。

Gaussian Splatting: 3D Reconstruction and Novel View Synthesis, a Review Figure 1
IEEE Access 20242024-01-01

Gaussian Splatting: 3D Reconstruction and Novel View Synthesis, a Review

Anurag Dalal, Daniel Hagen, Kjell G. Robbersmyr, Kristian Muri Knausgård

Department of Engineering Sciences, Top Research Centre Mechatronics (TRCM), University of Agder, Grimstad, Norway, Department of Engineering Sciences, Top Research Centre Mechatronics (TRCM), University of Agder

综述

面向图像式三维重建与新视角合成中难以同时兼顾高质量、快速训练和实时渲染的问题,本文系统梳理了2023年以来Gaussian Splatting的发展脉络。核心洞察是将方法按输入、模型结构、输出表示和训练策略组织,并对动态场景、可编辑头像等扩展进行归纳。结论显示,GS相较NeRF通常具备更快训练、实时渲染和较少伪影的优势,但薄结构、剧烈非刚体运动、相机位姿误差及统一评测仍是未充分解决的挑战。

Gaussian Pancakes: Geometrically-Regularized 3D Gaussian Splatting for Realistic Endoscopic Reconstruction Figure 1
arXiv preprint2024-01-01

Gaussian Pancakes: Geometrically-Regularized 3D Gaussian Splatting for Realistic Endoscopic Reconstruction

Sierra Bonilla, Shuai Zhang, Danail Stoyanov, Francisco Vasconcelos, Sophia Bano

Department of Computer Science, UCL Hawkes Institute, London, UK, Department of Computer Science, UCL Hawkes Institute, Department of Computer Science, University College London, London, UK, University College London, Hawkeye Community College, Department of Medical Physics and Biomedical Engineering, University College London, London, UK, Department of Medical Physics and Biomedical Engineering

医学影像

针对结肠镜视野有限、缺乏深度导致息肉漏检和肠腔表面难以完整三维重建的问题,本文将RNNSLAM提供的位姿、深度和初始表面与3D Gaussian Splatting结合,并通过几何与深度正则把高斯“压扁”贴合肠壁,减少漂浮伪影、提升纹理真实感。三组数据上相较领先方法PSNR提升18%、SSIM提升16%,渲染快100倍以上、训练时间缩短10倍以上,显示出近实时内镜重建潜力。

Fully Explicit Dynamic Gaussian Splatting Figure 1
arXiv preprint2024-01-01

Fully Explicit Dynamic Gaussian Splatting

Inhwan Bae, Hae-Gon Jeon, Hyunjun Jung, Junoh Lee, Changyeon Won

School of Electrical Engineering and Computer Science 2AI Graduate School, Gwangju Institute of Science and Technology

动态场景

针对动态新视角合成中 NeRF 渲染慢、现有 4D Gaussian 依赖稠密点云或隐式模块的问题,Ex4DGS 将静态与动态高斯分离,只在稀疏关键帧显式存储动态高斯的位置、旋转和不透明度,并用插值恢复连续运动;渐进训练与点回溯进一步改善稀疏点云下收敛和错误点剔除。实验在真实视频数据集上取得领先画质,并在单张 2080Ti 上达到约 62 FPS。

EndoGSLAM: Real-Time Dense Reconstruction and Tracking in Endoscopic Surgeries using Gaussian Splatting Figure 1
arXiv preprint2024-01-01

EndoGSLAM: Real-Time Dense Reconstruction and Tracking in Endoscopic Surgeries using Gaussian Splatting

Kailing Wang, Chen Yang, Yuehao Wang, Sikuang Li, Yan Wang, Qi Dou, Xiaokang Yang, Wei Shen

MoE Key Lab of Artificial Intelligence, AI Institute, Shanghai Jiao Tong University, Shanghai, China, MoE Key Lab of Artificial Intelligence, AI Institute, Shanghai Jiao Tong University, Department of Computer Science and Engineering, The Chinese University of Hong Kong, Ma Liu Shui, Hong Kong, Department of Computer Science and Engineering, The Chinese University of Hong Kong, Shanghai Key Laboratory of Multidimensional Information Processing, East China Normal University, Shanghai, China, Shanghai Key Laboratory of Multidimensional Information Processing, East China Normal University, Chinese University of Hong Kong, The Chinese University of Hong Kong, Hong Kong, China

医学影像同步定位与建图

内窥镜手术需要同时获得精确跟踪、致密组织重建和在线可视化,但传统 SLAM 重建稀疏、NeRF 类方法计算开销大。EndoGSLAM 将 RGB-D 内窥镜 SLAM 表示为简化的各向同性 3D Gaussian,用颜色替代 SH 并结合可微栅格化、未观测区域增量扩展和局部细化,以降低优化成本。实验显示其在跟踪、重建质量与系统效率间优于传统和神经 SLAM,并实现超过 100 fps 渲染。

Effective Rank Analysis and Regularization for Enhanced 3D Gaussian Splatting Figure 1
arXiv preprint2024-01-01

Effective Rank Analysis and Regularization for Enhanced 3D Gaussian Splatting

Jaegul Choo, Susung Hong, Sungwon Hwang, Junha Hyung, Jin-Hwa Kim, Jaeseong Lee

NAVER AI Lab, Korea University

密度控制网格重建

针对 3D Gaussian Splatting 在几何重建中易产生针状高斯、法线不准和新视角伪影的问题,论文用协方差矩阵的有效秩分析训练动态,指出许多高斯会退化到近似秩 1。其核心做法是将可微的有效秩加入正则项,约束高斯从针状转向更适合表面的盘状结构;实验显示该模块可叠加到 3DGS、SuGaR、2DGS 等变体上,改善法线与网格几何并减少针状伪影,同时基本不牺牲视觉保真度。

DreamScene4D: Dynamic Multi-Object Scene Generation from Monocular Videos Figure 1
arXiv preprint2024-01-01

DreamScene4D: Dynamic Multi-Object Scene Generation from Monocular Videos

Wen-Hsuan Chu, Katerina Fragkiadaki, Lei Ke

Carnegie Mellon University

动态场景单目重建

DreamScene4D面向单目视频中多物体快速运动、遮挡和未观测视角难以4D重建的问题,提出“分解—重组”框架:将场景拆为背景与对象轨迹,并把运动分解为相机运动、对象中心形变和对象到世界变换,使对象级生成先验可用于整场景。实验在DAVIS、Kubric和自采视频上优于现有video-to-4D方法,并能产生较稳定的2D持久点轨迹。

DreamGaussian4D: Generative 4D Gaussian Splatting Figure 1
arXiv preprint2023-12-28

DreamGaussian4D: Generative 4D Gaussian Splatting

Jiawei Ren, Liang Pan, Jiaxiang Tang, Chi Zhang, Ang Cao, Gang Zeng, Ziwei Liu

S-Lab, Nanyang Technological University, S-Lab, Nanyang Technological University, Shanghai AI Laboratory, Peking University, University of Michigan

扩散生成

针对现有 4D 生成依赖动态 NeRF、优化耗时数小时且运动难控的问题,DreamGaussian4D 将静态 3D Gaussian Splatting 与 HexPlane 形变场结合,用驱动视频学习时序运动,并通过视频扩散模型细化 UV 纹理以提升时序一致性。论文报告 14 帧动画约 6.5 分钟生成,加入纹理细化约 10 分钟,可导出可在 3D 引擎渲染的动画网格,质量和速度均优于既有方法。

4DGen: Grounded 4D Content Generation with Spatial-temporal Consistency Figure 1
arXiv preprint2023-12-28

4DGen: Grounded 4D Content Generation with Spatial-temporal Consistency

Yuyang Yin, Dejia Xu, Zhangyang Wang, Yao Zhao, Yunchao Wei

扩散生成

4DGen针对文本/图像直接生成4D资产时运动可控性弱、依赖提示工程且高分辨率时空一致性不足的问题,将单目视频作为“落地”条件来约束外观与运动,并用可变形3D Gaussian表示、锚帧时空伪标签、3D感知SDS与平滑正则联合优化。实验显示其比现有video-to-4D基线更能保真输入信号,并在新视角和任意时刻生成更连贯、真实的动态3D渲染。

Human101: Training 100+FPS Human Gaussians in 100s from 1 View Figure 1
arXiv preprint2023-12-23

Human101: Training 100+FPS Human Gaussians in 100s from 1 View

Mingwei Li, Jiachen Tao, Zongxin Yang, Yi Yang

ReLER, CCAI, Zhejiang University, Zhejiang University

数字人

面向单目视频快速生成可交互数字人的需求,Human101用3D Gaussian替代NeRF式隐式表示,并提出以人体/SMPL为中心的前向高斯动画,将高斯位置、旋转与视角相关外观随姿态变形,再用轻量MLP细化非刚性误差。实验在ZJU-MoCap、MonoCap等数据上显示,其训练约100秒,512分辨率渲染超过100FPS、1024分辨率超过60FPS,速度较既有方法最高提升约10倍且画质相当或更好。

Deformable 3D Gaussian Splatting for Animatable Human Avatars Figure 1
arXiv preprint2023-12-22

Deformable 3D Gaussian Splatting for Animatable Human Avatars

HyunJun Jung, Nikolas Brasch, Jifei Song, Eduardo Perez-Pellitero, Yiren Zhou, Zhihao Li, Nassir Navab, Benjamin Busam

数字人

针对现有 NeRF 式可动画数字人依赖多视角和人体掩码、UV/深度等标注且渲染较慢的问题,ParDy-Human 将人体姿态参数引入显式 3D Gaussian Splatting:先用 SMPL 顶点驱动规范 T-pose 高斯变形,再用关节几何编码预测每个高斯的残余形变以补偿衣物等非刚性动态,并通过人/背景高斯初始化实现免掩码训练。实验显示其在 ZJU-MoCap 与 THUman4.0 上较已有方法有更好的定量和视觉效果,可用少量视角甚至单目序列训练,并在消费级硬件上高效全分辨率渲染。

DUSt3R: Geometric 3D Vision Made Easy Figure 1
CVPR 20242023-12-21

DUSt3R: Geometric 3D Vision Made Easy

Shuzhe Wang, Vincent Leroy, Yohann Cabon, Boris Chidlovskii, Jerome Revaud

Aalto University, Naver Labs Europe

三维重建

DUSt3R针对传统SfM/MVS依赖相机内外参、流程长且误差级联的问题,将未标定双目/多视图重建改写为Transformer直接回归稠密3D pointmaps,并用3D空间全局对齐融合多图,从同一表示中恢复深度、匹配和相机。实验显示其在单目/多视深度与相对位姿估计上达到新SOTA,说明端到端点图表示可显著简化几何3D视觉流程。

Repaint123: Fast and High-quality One Image to 3D Generation with Progressive Controllable 2D Repainting Figure 1
arXiv preprint2023-12-20

Repaint123: Fast and High-quality One Image to 3D Generation with Progressive Controllable 2D Repainting

Junwu Zhang, Zhenyu Tang, Yatian Pang, Xinhua Cheng, Peng Jin, Yida Wei, Munan Ning, Li Yuan

Progressive Controllable 2D Repainting, Peking University, Pengcheng Laboratory, National University of Singapore, Wuhan University

扩散生成

Repaint123针对单图到3D中SDS优化常见的多视图不一致、纹理过饱和/过平滑和生成缓慢问题,改为先用2D扩散模型进行渐进可控重绘生成一致的稀疏多视图,并用可见性感知的自适应重绘强度与互自注意力维持局部和长期一致性,再以简单MSE快速优化3D高斯表示。实验显示其约2分钟即可生成细节更好、视图一致性更强的3D物体。

iComMa: Inverting 3D Gaussians Splatting for Camera Pose Estimation via Comparing and Matching Figure 1
arXiv preprint2023-12-14

iComMa: Inverting 3D Gaussians Splatting for Camera Pose Estimation via Comparing and Matching

Yuan Sun, Xuan Wang, Yunfan Zhang, Jie Zhang, Caigui Jiang, Yu Guo, Fei Wang

Xi’an Jiaotong University

位姿估计

iComMa针对基于NeRF反演的位姿估计对初值敏感、遇到大旋转/平移易失效的问题,改用3D Gaussian Splatting构建可微渲染优化,并将像素级render-and-compare损失与端到端关键点匹配损失联合起来,无需针对类别训练或CAD模型。实验显示其在合成与真实复杂场景、较差初始化下比已有方法更稳、更准,并因3DGS渲染效率相较iNeRF约快一个数量级。

Text2Immersion: Generative Immersive Scene with 3D Gaussian Figure 1
arXiv preprint2023-12-14

Text2Immersion: Generative Immersive Scene with 3D Gaussian

Hao Ouyang, Kathryn Heal, Stephen Lombardi, Tiancheng Sun

扩散生成世界生成

针对文本到 3D 生成常局限于单物体、室内场景或渲染慢的问题,Text2Immersion 用预训练 2D 扩散与单目深度逐步生成粗 3D Gaussian 云,再利用 Gaussian 的自适应 split/clone 机制做补全和超分细化。实验显示其在场景多样性、视角一致性和画质上优于 Text2Room/Text2NeRF 等方法,并可在 3070 笔记本 GPU 上约 180 FPS 实时渲染。

Learn to Optimize Denoising Scores for 3D Generation - A Unified and Improved Diffusion Prior on NeRF and 3D Gaussian Splatting Figure 1
arXiv preprint2023-12-08

Learn to Optimize Denoising Scores for 3D Generation - A Unified and Improved Diffusion Prior on NeRF and 3D Gaussian Splatting

Xiaofeng Yang, Yiwen Chen, Cheng Chen, Chi Zhang, Yi Xu, Xulei Yang, Fayao Liu, Guosheng Lin

Nanyang Technological University, OPPO US Research Center

扩散生成

本文针对 SDS 等扩散先验在 3D 生成中易模糊、缺乏多样性的问题,指出关键原因是优化时使用 CFG 与扩散模型训练目标存在偏差。作者提出 LODS,在优化 NeRF 或 3D Gaussian Splatting 的同时学习调整去噪分数,可通过可学习 null embedding 或 LoRA 参数实现不同复杂度配置。实验显示其在文本/图像到 3D 任务中提升细节与颜色一致性,并在 T3Bench 上超过 VSD 等方法。

Gaussian-SLAM: Photo-realistic Dense SLAM with Gaussian Splatting Figure 1
arXiv preprint2023-12-06

Gaussian-SLAM: Photo-realistic Dense SLAM with Gaussian Splatting

Martin R

University of Amsterdam, Netherlands, University of Amsterdam, Netherlands

同步定位与建图

针对现有神经稠密 SLAM 在真实场景中重建慢、渲染不够逼真且难扩展的问题,Gaussian-SLAM 将 3D Gaussian Splatting 引入单目 RGB-D 在线 SLAM,通过新区域高斯播种、子地图独立优化以及光度/几何误差的 frame-to-model 跟踪,实现与场景规模相对解耦的建图。实验显示其在合成与真实数据上的建图、位姿跟踪和实时渲染质量达到或优于多种神经稠密 SLAM 方法。

SC-GS: Sparse-Controlled Gaussian Splatting for Editable Dynamic Scenes Figure 1
CVPR 20242023-12-04

SC-GS: Sparse-Controlled Gaussian Splatting for Editable Dynamic Scenes

Yi-Hua Huang, Yang-Tian Sun, Ziyi Yang, Xiaoyang Lyu, Yan-Pei Cao, Xiaojuan Qi

The University of Hong Kong, University of Hong Kong, Zhejiang University

动态场景

针对动态场景中逐高斯建模运动开销大、轨迹易噪且不利于编辑的问题,SC-GS将外观表示为稠密3D Gaussian、将运动压缩到少量可学习控制点上,由MLP预测控制点随时间变化的6DoF变换,并通过插值驱动高斯形变;同时自适应调整控制点和ARAP约束保持局部刚性。实验显示其在动态新视角合成质量和渲染速度上优于已有方法,并支持保持外观的用户运动编辑。

Mathematical Supplement for the gsplat Library Figure 1
arXiv preprint2023-12-04

Mathematical Supplement for the gsplat Library

Vickie Ye, Angjoo Kanazawa

其他

为降低实现和改造可微 3D Gaussian Splatting 的门槛,本文系统补全 gsplat 库中前向投影、tile 深度排序合成及反向梯度传播的数学推导。核心价值不是提出新模型,而是把均值、协方差、颜色、透明度、尺度和四元数旋转的链式梯度写成可实现公式,并对应到模块化 Python API。主要结果是形成一份可复现的实现参考;文中未充分说明相对原始 3DGS 的性能增益。

Segment Any 3D Gaussians Figure 1
Proceedings of the AAAI Conference on Artificial Intelligence2023-12-01

Segment Any 3D Gaussians

Jiazhong Cen, Jiemin Fang, Chen Yang, Lingxi Xie, Xiaopeng Zhang, Wei Shen, Qi Tian

Shanghai Jiao Tong University, Huawei Technologies Co., Ltd, Huawei Technologies Co, Huawei Technologies (United Kingdom)

编辑分割

面向3D-GS场景中缺少高效可提示分割的问题,SAGA将SAM的2D分割能力蒸馏到每个3D Gaussian的亲和特征中,并用按物理尺度调节通道幅值的soft scale-gate处理部件/物体等多粒度歧义。训练后可由2D提示直接选出对应3D高斯目标,推理约4 ms,分割质量接近当时最优方法。

NeuSG: Neural Implicit Surface Reconstruction with 3D Gaussian Splatting Guidance Figure 1
arXiv preprint2023-12-01

NeuSG: Neural Implicit Surface Reconstruction with 3D Gaussian Splatting Guidance

Gim Hee

Department of Computer Science, National University of Singapore, Department of Computer Science, National University of Singapore

网格重建

NeuSG针对神经隐式重建依赖深度图或稀疏点云时细节被抹平的问题,将3D Gaussian Splatting产生的密集结构作为几何引导,但通过尺度正则把高斯压薄、使中心贴近表面,并用NeuS预测法线反向细化高斯点云,形成联合优化。Tanks and Temples实验显示其能在保持表面完整性的同时恢复更丰富细节。

DISTWAR: Fast Differentiable Rendering on Raster-based Rendering Pipelines Figure 1
arXiv preprint2023-12-01

DISTWAR: Fast Differentiable Rendering on Raster-based Rendering Pipelines

Pawan Kumar

University of Toronto

加速训练渲染

DISTWAR针对3DGS等光栅化可微渲染训练中反向梯度累积被大量原子操作拖慢的问题,指出同一warp内线程常更新相同参数且活跃线程数差异大,因此在SM寄存器中做warp级归约,并按阈值把原子计算分配给SM或L2原子单元。真实GPU实验显示梯度计算平均提速2.44×、最高5.7×,端到端训练也有明显加速。

SparseGS: Real-Time 360° Sparse View Synthesis using Gaussian Splatting Figure 1
arXiv preprint2023-11-30

SparseGS: Real-Time 360° Sparse View Synthesis using Gaussian Splatting

Haolin Xiong, Sairisheek Muttukuru, Rishi Upadhyay, Pradyumna Chari, Achuta Kadambi

全景重建稀疏表示

SparseGS针对3DGS在稀疏视角下易出现漂浮高斯和背景塌陷的问题,核心是将深度先验与softmax-scaling、mode-selection深度渲染结合,并用未见视角正则化和显式剪枝约束几何。实验在Mip-NeRF360、LLFF、DTU上显示,方法可在360°场景约12张、前向场景约3张输入下保持较高重建质量,同时保留快速训练和实时渲染能力。

MD-Splatting: Learning Metric Deformation from 4D Gaussians in Highly Deformable Scenes Figure 1
arXiv preprint2023-11-30

MD-Splatting: Learning Metric Deformation from 4D Gaussians in Highly Deformable Scenes

Bardienus P. Duisterhof, Zhao Mandi, Yunchao Yao, Jia-Wei Liu, Mike Zheng Shou, Shuran Song, Jeffrey Ichnowski

Stanford University, National University of Singapore, Technical University of Munich

动态场景

面向布料等高形变物体操作,论文关注遮挡、阴影和大形变下难以获得稳定3D稠密跟踪的问题。方法以4D高斯/高斯泼溅表示动态场景,学习从规范高斯到度量空间的形变场,并加入动量守恒、等距等物理启发正则及阴影建模,以同时做场景流估计和新视角合成。实验在合成高形变场景中较既有方法显著降低跟踪误差,文中版本报告平均提升约55.8%,真实Robo360上展示了数字孪生和关键点跟踪应用。

Compact3D: Compressing Gaussian Splat Radiance Field Models with Vector Quantization Figure 1
arXiv preprint2023-11-30

Compact3D: Compressing Gaussian Splat Radiance Field Models with Vector Quantization

KL Navaneet, Kossar Pourahmadi Meibodi, Soroush Abbasi Koohpayegani, Hamed Pirsiavash

University of California, Davis, University of California

压缩

该文针对 3D Gaussian Splatting 虽训练和渲染快、但模型存储和显存占用远高于 NeRF 的问题,利用大量高斯参数相似的洞察,在训练中用 K-means 向量量化高斯参数,仅存码本和索引,并结合索引排序/RLE 与透明度正则删除不可见高斯。实验显示在标准和更大规模 3D 数据集上,存储可降约 40–50 倍、渲染加速 2–3 倍,图像质量仅小幅下降。

FisherRF: Active View Selection and Uncertainty Quantification for Radiance Fields using Fisher Information Figure 1
arXiv preprint2023-11-29

FisherRF: Active View Selection and Uncertainty Quantification for Radiance Fields using Fisher Information

Wen Jiang, Boshu Lei, Kostas Daniilidis

University of Pennsylvania

其他

为降低 NeRF/辐射场重建对大量采集视角的依赖,FisherRF 用 Fisher 信息直接度量模型参数的已观测信息,并以期望信息增益选择下一视角;利用体渲染 Hessian 与真实图像无关及稀疏性,将方法扩展到 3D Gaussian Splatting、Plenoxels及批量/路径选择。实验在主动视角选择、主动建图和像素级不确定性估计上优于既有不确定性启发方法。

CG3D: Compositional Generation for Text-to-3D Figure 1
arXiv preprint2023-11-29

CG3D: Compositional Generation for Text-to-3D

Alexander Vilesov, Pradyumna Chari, Achuta Kadambi

扩散生成

针对文本到3D方法难以生成可控多物体场景、物体关系不稳定且物理组合不合理的问题,CG3D将场景表示为可分离的显式3D Gaussian辐射场,并用SDS结合蒙特卡洛搜索估计物体的旋转、平移和尺度,无需人工框约束。实验显示其在复杂物体组合、语义一致性和重力等物理合理性上优于DreamGaussian、Fantasia3D等方法,但对需形变交互和显式对象关系仍有限。

Relightable 3D Gaussian: Real-time Point Cloud Relighting with BRDF Decomposition and Ray Tracing Figure 1
arXiv preprint2023-11-27

Relightable 3D Gaussian: Real-time Point Cloud Relighting with BRDF Decomposition and Ray Tracing

Jian Gao, Chun Gu, Youtian Lin, Hao Zhu, Xun Cao, Li Zhang, Yao Yao

Nanjing University, Fudan University, Huawei Noah’s Ark Lab

光线追踪重光照渲染

针对原始 3D Gaussian Splatting 只能做新视角合成、难以在点表示中处理重光照和阴影的问题,论文为每个高斯点加入法线、BRDF 与入射光,并用物理可微渲染分解材质/光照;同时引入基于 BVH 的点云光线追踪预计算可见性。实验显示其在 BRDF 估计、新视角渲染与重光照效果上优于对比方法,并支持多物体编辑与较真实阴影。

LucidDreamer: Domain-free Generation of 3D Gaussian Splatting Scenes Figure 1
arXiv preprint2023-11-22

LucidDreamer: Domain-free Generation of 3D Gaussian Splatting Scenes

Kyoung Mu

ASRI, Department of ECE, Seoul National University, Seoul, Korea, Department of ECE, Seoul National University

扩散生成世界生成

LucidDreamer针对3D场景生成依赖扫描数据、域泛化差且多视图一致性难保证的问题,利用预训练Stable Diffusion而非重新训练3D生成模型,通过“Dreaming”将已有点云投影为几何约束进行补全并以深度提升到3D,再用“Alignment”融合新旧点云,最终初始化并优化3D Gaussian Splatting。结果显示其可从文本、RGB/RGBD等输入生成更细致的跨域场景,在ScanNet、NYUDepth和扩散生成图像条件下视觉质量优于既有方法。

GaussianDiffusion: 3D Gaussian Splatting for Denoising Diffusion Probabilistic Models with Structured Noise Figure 1
arXiv preprint2023-11-19

GaussianDiffusion: 3D Gaussian Splatting for Denoising Diffusion Probabilistic Models with Structured Noise

Xinhai Li, Huaibin Wang, Kuo-Kun Tseng

Harbin Institute of Technology (Shenzhen)

扩散生成

为缓解NeRF式文本到3D生成渲染慢、多视角几何不一致以及3D Gaussian易陷局部极小产生漂浮物等问题,GaussianDiffusion将3D Gaussian Splatting贯穿扩散蒸馏流程,引入共享噪声源生成的多视角结构化高斯噪声,并用变分Gaussian Splatting稳定优化。文中展示其可加速渲染并获得更真实、较一致的外观,但定量增益和消融细节在给定文本中未充分说明。

SplatArmor: Articulated Gaussian splatting for animatable humans from monocular RGB videos Figure 1
arXiv preprint2023-11-17

SplatArmor: Articulated Gaussian splatting for animatable humans from monocular RGB videos

Ganesh Subramanian

University of Pennsylvania, Amazon.com, Inc, Amazon.com

数字人

面向低成本单目 RGB 视频生成可驱动数字人,SplatArmor 用规范空间中的 3D Gaussian“包覆”SMPL,并将 SMPL 的前向蒙皮扩展到任意高斯位置,避免 NeRF 人体方法常见的逆蒙皮歧义;同时引入 SE(3) 位姿相关场和神经颜色场来建模非刚性变化、正则颜色与高斯定位。实验在 ZJU MoCap 和 People Snapshot 上展示了高保真新视角/新姿态合成,并相较 NeRF 类方法强调更低训练与推理开销。

Real-time Photorealistic Dynamic Scene Representation and Rendering with 4D Gaussian Splatting Figure 1
arXiv preprint2023-10-16

Real-time Photorealistic Dynamic Scene Representation and Rendering with 4D Gaussian Splatting

Zeyu Yang, Hongye Yang, Zijie Pan, Xiatian Zhu, Li Zhang

Fudan University

动态场景

针对动态场景新视角合成中时空结构难以表达、显式形变难以扩展的问题,本文将空间与时间统一为4D体,用可旋转的4D Gaussian及4D Spherindrical Harmonics建模几何和随时间变化的外观,并设计splatting渲染流程。实验覆盖单目与多视角数据,显示其在视觉质量和渲染效率上优于既有动态NeRF/高斯方法,可实现高分辨率实时渲染。

GaussianDreamer: Fast Generation from Text to 3D Gaussian Splatting with Point Cloud Priors Figure 1
arXiv preprint2023-10-12

GaussianDreamer: Fast Generation from Text to 3D Gaussian Splatting with Point Cloud Priors

Taoran Yi, Jiemin Fang, Guanjun Wu, Lingxi Xie, Xiaopeng Zhang

School of EIC, Huazhong University of Science and Technology, School of EIC, Huazhong University of Science and Technology, Huawei Inc, School of CS, Huazhong University of Science and Technology, School of CS

扩散生成

GaussianDreamer针对文本到3D中“3D扩散一致但细节/泛化弱、2D扩散细节强但多视图不稳”的矛盾,将3D扩散生成的点云先验用于初始化3D Gaussian Splatting,再通过噪声点增长、颜色扰动和SDS从2D扩散补充几何与外观。实验显示其在单张RTX 3090上约15分钟生成实例或头像,并可直接实时渲染,质量与多种慢速基线相当或更好。

DreamGaussian: Generative Gaussian Splatting for Efficient 3D Content Creation Figure 1
arXiv preprint2023-09-28

DreamGaussian: Generative Gaussian Splatting for Efficient 3D Content Creation

Jiaxiang Tang, Jiawei Ren, Hang Zhou, Ziwei Liu, Gang Zeng

National Key Laboratory of General AI, School of IST, Peking University, National Key Laboratory of General AI, School of IST, Peking University, S-Lab, Nanyang Technological University, S-Lab, Nanyang Technological University, Baidu Inc

扩散生成

针对SDS式2D提升生成3D资产每个样本优化过慢、NeRF空域剪枝在生成场景中效果有限的问题,DreamGaussian将3D Gaussian Splatting引入生成流程,利用渐进 densification 加速收敛,并设计从高斯到纹理网格的局部密度查询提取与UV空间纹理细化。实验显示其在图生3D和文生3D中保持相近质量的同时显著提速,单图生成高质量纹理网格约2分钟,较既有方法约快10倍。

Dynamic 3D Gaussians: Tracking by Persistent Dynamic View Synthesis Figure 1
3DV 20242023-08-18

Dynamic 3D Gaussians: Tracking by Persistent Dynamic View Synthesis

Jonathon Luiten, Georgios Kopanas, Bastian Leibe, Deva Ramanan

Carnegie Mellon University, USA, Carnegie Mellon University, RWTH Aachen University, Germany, RWTH Aachen University, Inria & Université Côte d’Azur, France, Inria & Université Côte d’Azur

动态场景

面向机器人、AR/自动驾驶等需要持续动态三维世界模型的场景,本文将静态 3D Gaussian Splatting 扩展为随时间移动和旋转、但颜色/透明度/尺度保持持久的动态高斯粒子,并用局部刚性、旋转相似和长时等距正则让跟踪从视图合成优化中自然出现,无需光流或对应监督。在 CMU Panoptic 多视角视频上,方法达到 28.7 PSNR、850 FPS 渲染,2 小时训练,并实现约 2.21cm 的三维长期稠密 6-DOF 跟踪误差。

Gaussian Splatting for Real-Time Radiance Field Rendering Figure 1
ACM TOG 20232023-08-08

Gaussian Splatting for Real-Time Radiance Field Rendering

Bernhard Kerbl, Georgios Kopanas, Thomas Leimkuehler, George Drettakis

Inria, Sophia-Antipolis, France, Inria, Max Planck Institute for Informatics

动态场景渲染

针对 NeRF 类辐射场在高质量新视角合成中训练和渲染开销大、1080p 完整场景难以实时的问题,论文将 SfM 稀疏点初始化为可优化的各向异性 3D Gaussian,并结合密度自适应控制与可见性感知的 tile-based splatting 渲染。结果显示其在多个数据集上达到或接近最佳画质,同时将训练压到分钟级,并实现 1080p 30fps 以上实时渲染。