Open3DShow Interface Documentation¶

Introduction¶

Overview: This file defines the Open3DShow class, providing a Python interface to retrieve simulation environment images from Unreal Engine 4 (UE4), enabling visualization and processing of drone simulation visual data using Open3D.

In the RflySim drone simulation platform, visual simulation tasks require acquiring virtual environment image data generated by UE4 to develop and validate algorithms such as object detection, point cloud reconstruction, and visual SLAM. This module belongs to the visual tools component of RflySimSDK, specifically designed for visual development in simulated environments. It simplifies the process of retrieving UE4 simulation images on the Python side, facilitating users to quickly integrate with Open3D for visual data display and algorithm testing. It is suitable for simulation development scenarios including drone visual navigation and environmental perception.

Quick Start¶

A minimal runnable example; copy and modify minimal configuration to run.

from RflySimSDK.vision.Open3DShow import Open3DShow
import time

# Initialize Open3D point cloud visualization object
visualizer = Open3DShow()
# Create visualization window using default ID 0
visualizer.CreatShow(idx=0)

# Simulate loop updating point clouds to demonstrate real-time visualization
try:
    for _ in range(100):
        # Retrieve the latest point cloud data (automatically fetched from the simulation environment)
        visualizer.get_new_point_cloud()
        # Update every 100ms to control refresh rate
        time.sleep(0.1)
except KeyboardInterrupt:
    print("Visualization exited")

Environment and Dependencies¶

Python Environment: >= 3.8.10
Dependencies: numpy, sys, time
Prerequisites: No special prerequisites are required before calling this interface; ensure that RflySimSDK is correctly installed and the relevant Open3DShow dependencies under the vision module are imported.

Core Interface Description¶

The module Open3DShow.py contains configuration variables, helper functions, and the core business class.

Global Constants and Enumerations¶

This section lists all globally accessible constants and enumerations defined in the module.

Standalone Constants¶

None

Global/Standalone Functions¶

None

`Open3DShow` Class¶

An API class for retrieving point cloud data from UE4 and visualizing it using Open3D, suitable for 3D point cloud visualization scenarios within the RflySim simulation environment.

`init()`¶

Function Description: Initializes an instance of the Open3DShow class.
Parameters (Args):
None
Returns:

An instance of Open3DShow

Exceptions (Raises):
None

`get_new_point_cloud()`¶

Function Description: Retrieves new point cloud data.
Parameters (Args):
None
Returns:

None

Exceptions (Raises):
None

`update_callback(vis)`¶

Function Description: Callback function for updating the Open3D visualization window at scheduled intervals to refresh point cloud data.
Parameters (Args):

Parameter Name	Type	Required	Default	Description
`vis`	`open3d.visualization.Visualizer`	Yes	-	Instance of the Open3D visualizer

Returns:

None

Exceptions (Raises):
None

`CreatShow(idx=0)`¶

Function Description: Creates and starts the point cloud visualization window.
Parameters (Args):

Parameter Name	Type	Required	Default	Description
`idx`	`int`	No	`0`	Identifier index for the visualization window

Returns:

None

Exceptions (Raises):
None

`SetFRDView()`¶

Function Description: Sets a top-down view of the FRD coordinate system, simulating a mouse-up swipe gesture to adjust the viewing angle.
Parameters (Args):
None
Returns:

None

Exceptions (Raises):
None

`AppendPcd(Cloud)`¶

Function Description: Appends point cloud data to the current point cloud collection.
Parameters (Args):

Parameter Name	Type	Required	Default	Description
`Cloud`	`open3d.geometry.PointCloud`	Yes	-	Point cloud object to be appended

Returns:

None

Exceptions (Raises):
None

`merge_point_cloud(other_pcd)`¶

Function Description: Merges another point cloud into the current point cloud.
Parameters (Args):

Parameter Name	Type	Required	Default	Description
`other_pcd`	`open3d.geometry.PointCloud`	Yes	-	External point cloud object to be merged

Returns:

None

Exceptions (Raises):

ValueError: Raised when the input point cloud object is invalid.

`show_point_cloud()`¶

Function Description: Displays the currently stored point cloud data.
Parameters (Args):
None
Returns:

None

Exceptions (Raises):
None

`clear_point_cloud()`¶

Function Description: Clears all points from the point cloud and updates the visualization.
Parameters (Args):
None
Returns:

None

Exceptions (Raises):
None

`remove_duplicates()`¶

Function Description: Removes duplicate points (points with identical coordinates).
Parameters (Args):
None
Returns:

None

Exceptions (Raises):
None

`UpdateShow(Cloud)`¶

Function Description: Updates the point cloud data and refreshes the visualization display.
Parameters (Args):

Parameter Name	Type	Required	Default	Description
`Cloud`	`open3d.geometry.PointCloud`	Yes	-	Point cloud object to be updated and displayed

Returns:

None

Exceptions (Raises):
None

`UpdateShowWithColor(Cloud, colors=None)`¶

Function Description: Updates and displays colored point cloud data, applying FRD coordinate transformation.
Parameters (Args):

Parameter Name	Type	Required	Default	Description
`Cloud`	`numpy.ndarray`	Yes	-	Input point cloud coordinate array, shape (N, 3)
`colors`	`numpy.ndarray`	No	`None`	Point cloud color array, shape (N, 3), values in range [0, 1]

Returns:

None

Exceptions (Raises):
None

`UpdatePCD()`¶

Function Description: Updates the point cloud data in the visualization window.
Parameters (Args):
None
Returns:

None

Exceptions (Raises):
None

`CloseShow()`¶

Function Description: Closes the point cloud visualization window.
Parameters (Args):
None
Returns:

None

Exceptions (Raises):
None

Example:

from RflySimSDK.vision import Open3DShow
import numpy as np

# Create a visualization instance and launch the window
pcd_show = Open3DShow()
pcd_show.CreatShow()
# Set FRD top-down view
pcd_show.SetFRDView()

# Generate a sample colored point cloud and update the display
test_points = np.random.randn(1000, 3)
test_colors = np.random.rand(1000, 3)
pcd_show.UpdateShowWithColor(test_points, test_colors)

# Clear the point cloud and close the window
# pcd_show.clear_point_cloud()
# pcd_show.CloseShow()

---

## Advanced Usage Examples

> Demonstrates complex composite scenarios (e.g., multi-agent collaboration, asynchronous control, batch operations)

```python
import RflySimSDK.vision as vis
import numpy as np
from threading import Thread

# Advanced example of multi-agent point cloud stitching: asynchronous acquisition + real-time stitching and visualization
def cooperative_mapping_task():
    # Initialize visualization window, set top-down FRD view
    pcd_show = vis.Open3DShow()
    pcd_show.CreatShow("Multi-Agent Cooperative Mapping")
    pcd_show.SetFRDView(flip_direction=True)

    # Define asynchronous point cloud acquisition callback; each drone collects independently and appends
    collected_pcds = []
    def async_capture(drone_id, capture_interval):
        while True:
            new_pcd = pcd_show.get_new_point_cloud()
            # Simulate point cloud translation transformation under different drone coordinate systems
            transformed_pcd = new_pcd.translate(np.array([drone_id*5, 0, 0]))
            collected_pcds.append(transformed_pcd)
            pcd_show.AppendPcd(transformed_pcd)
            # Automatically deduplicate and update display every 5 frames
            if len(collected_pcds) % 5 == 0:
                pcd_show.remove_duplicates(voxel_size=0.05)
                pcd_show.UpdateShow()
            time.sleep(capture_interval)

    # Start asynchronous point cloud acquisition for 3 drones
    for drone_id in range(3):
        Thread(target=async_capture, args=(drone_id, 1), daemon=True).start()

    # Start the main visualization loop
    pcd_show.show_point_cloud()

if __name__ == "__main__":
    cooperative_mapping_task()

This example combines multi-threading and asynchronous control to achieve collaborative point cloud acquisition across multiple drones. It leverages AppendPcd for incremental addition, remove_duplicates for downsampling and deduplication, and UpdateShow for real-time refresh, thereby realizing edge-side visualization in distributed mapping scenarios. Additionally, the collected point clouds can be merged into a complete map via merge_point_cloud and then saved.

Notes and Pitfall Avoidance Guide¶

Visualization Window Initialization Requirements: The CreatShow method must be called first to create the window before invoking subsequent operations such as SetFRDView or AppendPcd. Failure to initialize the window will cause subsequent operations to throw access exceptions.
Real-Time Update Performance Limitations: Frequent calls to UpdateShow consume excessive OpenGL rendering resources. It is recommended to batch-add point clouds before performing a unified update, and the number of point clouds appended per frame should ideally not exceed 1e5 to avoid screen lag.
Deduplication Parameter Settings: When calling remove_duplicates, the voxel size voxel_size must be set according to the actual scale of the point cloud. An overly small value will fail to achieve effective downsampling, while an overly large value may result in loss of critical structural information in the scene.
Multi-threading Access Safety: The visualization rendering of Open3DShow must run on the main thread. Time-consuming operations such as point cloud acquisition and processing can be offloaded to subthreads; however, calling show_point_cloud to start the main rendering loop from a subthread is strictly prohibited.

Changelog¶

2026-01-31: fix: Optimize point cloud display interface
2026-01-30: fix: Fix point cloud preview orientation issue
2025-12-31: fix: Update Open3D version notice
2025-08-19: fix: Add identification mechanism; use 1400-byte packets in WSL native mode to reduce packet loss probability in WSL
2025-07-26: fix: Fix laser point cloud shared memory bug
2025-07-25: fix: Update Open3D point cloud preview effect
2024-08-05: fix: Add HTML version API documentation
2024-07-17: fix: Update VisionCaptureApi interface
2024-06-03: fix: Update
2024-05-31: fix: Update PDF

Open3DShow Interface Documentation¶

Introduction¶

Quick Start¶

Environment and Dependencies¶

Core Interface Description¶

Global Constants and Enumerations¶

Standalone Constants¶

Global/Standalone Functions¶

Open3DShow Class¶

__init__()¶

get_new_point_cloud()¶

update_callback(vis)¶

CreatShow(idx=0)¶

SetFRDView()¶

AppendPcd(Cloud)¶

merge_point_cloud(other_pcd)¶

show_point_cloud()¶

clear_point_cloud()¶

remove_duplicates()¶

UpdateShow(Cloud)¶

UpdateShowWithColor(Cloud, colors=None)¶

UpdatePCD()¶

CloseShow()¶

Notes and Pitfall Avoidance Guide¶

Changelog¶

`Open3DShow` Class¶

`init()`¶

`get_new_point_cloud()`¶

`update_callback(vis)`¶

`CreatShow(idx=0)`¶

`SetFRDView()`¶

`AppendPcd(Cloud)`¶

`merge_point_cloud(other_pcd)`¶

`show_point_cloud()`¶

`clear_point_cloud()`¶

`remove_duplicates()`¶

`UpdateShow(Cloud)`¶

`UpdateShowWithColor(Cloud, colors=None)`¶

`UpdatePCD()`¶

`CloseShow()`¶