Realsee3D Dataset

High Dynamic Range (HDR) RGB Panorama

Unlike traditional panoramic pipelines that rely on approximated co-centering (e.g., Insta360) or wide-baseline fusion (e.g., Matterport), our system employs nearly perfectly co-centered cameras. This structural advantage significantly simplifies the stitching process and ensures superior physical consistency in the resulting panoramas.

Furthermore, our advanced ISP pipeline guarantees photometric fidelity: stitching is performed directly in the RAW domain for highly consistent color and luminance, and we achieve High Dynamic Range (HDR) quality via multiple-exposure stacking. A sophisticated HDR tone-mapping algorithm preserves this extended range, while the system explicitly handles mixed-illumination conditions, supplying richer, more reliable visual information than most commercial 360° cameras.

LiDAR Depth Data Synthesis

The continuously acquired 3D point cloud data from the LiDAR is post-processed, integrated, and synthesized into a depth image. This depth image is precisely aligned with the panoramic color image via calibrated camera-LiDAR extrinsics. The near-coincidence of the projection centers for the color and depth images is critical for providing geometrically reliable, dense RGB-D data.

The initial pose estimation for each scan position is completed during the data acquisition phase. Whenever a new location is scanned, the system begins a search for potential pairs among all previously completed scan positions and compute feature matches. By combining the calculated feature matches with the corresponding 3D point cloud data, the relative pose is determined. Once the correct pose is confirmed, the new scan position is registered into the existing map.

Since fully automatic registering occasionally results in errors, manual correction is sometimes applied. The initial poses then undergo a global post-processing step for further refinement. This global post-processing considers the co-visibility relationships between all scan positions, using this framework to perform a global optimization incorporating both visual and 3D features. This process ultimately generates the final, corrected pose file.

We developed a hybrid rule-based and learning-based discrete space optimization system. This system integrates two critical components: placement priors derived from Realsee's vast real scene data, and expert knowledge provided by professional interior designers (e.g., circulation flow and storage requirements).

This integration ensures that all generated layouts are structurally valid and functionally plausible. Furthermore, we guarantee aesthetic diversity by employing over 200 distinct style templates and a style matching algorithm, supporting a wide range of specialized room types such as children's rooms, gyms, and tea rooms.

3D Asset Library

Our Realsee "Weilaijia" model library contains >100,000 models across 200+ categories. Unlike low-poly approximations, our models preserve fine geometric details and use PBR materials for realistic surface properties (specular highlights, subsurface scattering).

Ray-Tracing Engine

Built on a modified UE4, our pipeline utilizes Ray Tracing for accurate Global Illumination (GI), Reflections, and Ambient Occlusion. We integrate DLSS and five distinct illumination schemas including Warm Day, Cold Day, Natural Day, Warm Night, and Cold Night to maximize domain variability.