Unreal Synthetic Data Generator

Overview

UE5 plugin for generating synthetic training data for AI/ML:

• 360° Equirectangular Images
• Depth Maps
• Fisheye Images (multiple projection types)
• ROS bag export (.mcap) with camera poses — ⚠️ experimental, not fully tested

Uses six cameras to produce seamless 360° captures with precise depth data.

⚠️ Note: This plugin has been developed and tested exclusively for Windows systems.

• UnrealMVS: FLW-TUDO/UnrealMVS — the Neural network for depth estimation to confirm the usability of the here generated data.
• Datasets (HuggingFace): flw-tu-dortmund/UnrealMVS — the datasets used in this project.

Installation

Prerequisites

Enable OpenEXR support in OpenCV:

1. Open Windows "Environment Variables" (Start → search "environment variables")
2. Add new System Variable:
   • Name: OPENCV_IO_ENABLE_OPENEXR
   • Value: true
3. Restart your IDE/Editor

Option A: Project Plugin (Recommended)

1. Create or open a C++ UE5 project
2. Create a Plugins folder in your project directory
3. Clone this repository with submodules into Plugins/:

   cd YourProject/Plugins
   git clone --recursive https://github.com/FLW-TUDO/UnrealMVS-DataGenerator.git UnrealMVS-DataGenerator
   

   If you already cloned without --recursive, run:

   git submodule update --init --recursive
   

4. Open the project in UE5 - the plugin builds automatically
5. Verify in Edit → Plugins → search "OmniRGBDCamera"

Option B: Build Plugin Standalone

Build the plugin without opening the Editor:

"<UE5_INSTALL_PATH>\Engine\Build\BatchFiles\RunUAT.bat" ^
    BuildPlugin ^
    -Plugin="PATH\TO\OmniRGBDCamera\OmniRGBDCamera.uplugin" ^
    -Package="OUTPUT\PATH" ^
    -TargetPlatforms=Win64

<UE5_INSTALL_PATH> is typically C:\Program Files\Epic Games\UE_5.X

Output Examples

Equirectangular	Depth Map

Fisheye Projections

Sample images use Unreal Engine demo scenes for illustration only.

Citation

This work has been accepted for ETFA 2026. If you use UnrealMVS in your research, please cite it.

Note: The DOI will be added here once it is available.

% Citation will be added here (DOI pending).

---

Acknowledgements

This project was created in cooperation between TU Dortmund University and Westphalian University of Applied Sciences (WHS).