PeRFception: Perception using Radiance FieldsDownload PDF

Published: 17 Sept 2022, Last Modified: 12 Mar 2024NeurIPS 2022 Datasets and Benchmarks Readers: Everyone
Keywords: dataset, neural radiance field, image classification, 3D shape classification, 3D semantic segmentation
TL;DR: We propose a new dataset, PeRFception dataset, that is a new unified radiance field dataset for the 2D image classification, 3D shape classification, and 3D semantic segmentation.
Abstract: The recent progress in implicit 3D representation, i.e., Neural Radiance Fields (NeRFs), has made accurate and photorealistic 3D reconstruction possible in a differentiable manner. This new representation can effectively convey the information of hundreds of high-resolution images in one compact format and allows photorealistic synthesis of novel views. In this work, using the variant of NeRF called Plenoxels, we create the first large-scale radiance fields datasets for perception tasks, called the PeRFception, which consists of two parts that incorporate both object-centric and scene-centric scans for classification and segmentation. It shows a significant memory compression rate (96.4\%) from the original dataset, while containing both 2D and 3D information in a unified form. We construct the classification and segmentation models that directly take this radiance fields format as input and also propose a novel augmentation technique to avoid overfitting on backgrounds of images. The code and data are publicly available in "".
Author Statement: Yes
Supplementary Material: zip
Dataset Url: PeRFception-CO3D - Data Chunks 1:!As9A9EbDsoWcbnHoOoqWmIB6RLs?e=SYGC03 PeRFception-CO3D - Data Chunks 2:!AgY2evoYo6FgiwomlG1QUiLg7wqy?e=ReG5Yp PeRFception-ScanNet:!AgY2evoYo6FghYVw3MLYwq743fsoUw?e=ylF8KX
Dataset Embargo: We have publicly released the data and benchmarked models in our project page and GitHub. - Project Page: - GitHub:
License: The PeRFception datasets are released under the Attribution-ShareAlike 4.0 International (CC BY-SA 4.0) license\footnote{\url{}}, which allows anybody to use them for remix, transform, or build upon the material.
Contribution Process Agreement: Yes
In Person Attendance: Yes
Community Implementations: [![CatalyzeX](/images/catalyzex_icon.svg) 2 code implementations](
26 Replies