Open3DShow Interface Documentation¶

Introduction¶

Overview: This file defines the Open3DShow class, providing a Python interface to retrieve simulation environment images from Unreal Engine 4 (UE4), enabling visualization and processing of drone simulation visual data using Open3D.

In the RflySim drone simulation platform, visual simulation tasks require acquiring virtual environment image data generated by UE4 to develop and validate algorithms such as object detection, point cloud reconstruction, and visual SLAM. This module belongs to the visual tools component of RflySimSDK, specifically designed for visual development in simulated environments. It simplifies the process of retrieving UE4 simulation images on the Python side, facilitating users to quickly integrate with Open3D for visual data display and algorithm testing. It is suitable for simulation development scenarios including drone visual navigation and environmental perception.

Quick Start¶

The following example reads the RGB555 tag values from a segmented point cloud, decodes them into colors, and updates the Open3D point cloud window.

Reference example: [RflySim installation path]\RflySimAPIs\8.RflySimVision\0.ApiExps\1-UsageAPI\1.ImgSenorAPI\5.SegmentImageDemo

import numpy as np

import Open3DShow
import VisionCaptureApi


show3d = Open3DShow.Open3DShow()
vis = VisionCaptureApi.VisionCaptureApi()

vis.jsonLoad(jsonPath="Config_udp.json")
vis.sendReqToUE4()
vis.startImgCap()
show3d.CreatShow(0)

while True:
    if vis.hasData[2]:
        data = vis.Img[2]

        # Decode the RGB555 tag value in the 4th column into RGB color
        stencil = data[:, 3].astype(np.int32)
        red = ((stencil >> 10) & 0x1F) * 8
        green = ((stencil >> 5) & 0x1F) * 8
        blue = (stencil & 0x1F) * 8
        color = np.stack([red, green, blue], axis=1) / 255.0

        show3d.UpdateShowWithColor(data[:, :3], color)
        vis.hasData[2] = False

Environment and Dependencies¶

Python Environment: >= 3.8.10
Dependencies: numpy, sys, time
Prerequisites: No special prerequisites are required before calling this interface; ensure that RflySimSDK is correctly installed and the relevant Open3DShow dependencies under the vision module are imported.

Core Interface Description¶

The module Open3DShow.py contains configuration variables, helper functions, and the core business class.

Global Constants and Enumerations¶

This section lists all globally accessible constants and enumerations defined in the module.

Standalone Constants¶

None

Global/Standalone Functions¶

None

`Open3DShow` Class¶

An API class for retrieving point cloud data from UE4 and visualizing it using Open3D, suitable for 3D point cloud visualization scenarios within the RflySim simulation environment.

`init()`¶

Function Description: Initializes an instance of the Open3DShow class.
Parameters (Args):
None
Returns:

An instance of Open3DShow

Exceptions (Raises):
None

`get_new_point_cloud()`¶

Function Description: Retrieves new point cloud data.
Parameters (Args):
None
Returns:

None

Exceptions (Raises):
None

`update_callback(vis)`¶

Function Description: Callback function for updating the Open3D visualization window at scheduled intervals to refresh point cloud data.
Parameters (Args):

Parameter Name	Type	Required	Default	Description
`vis`	`open3d.visualization.Visualizer`	Yes	-	Instance of the Open3D visualizer

Returns:

None

Exceptions (Raises):
None

`CreatShow(idx=0)`¶

Function Description: Creates and starts the point cloud visualization window.
Parameters (Args):

Parameter Name	Type	Required	Default	Description
`idx`	`int`	No	`0`	Identifier index for the visualization window

Returns:

None

Exceptions (Raises):
None

`SetFRDView()`¶

Function Description: Sets a top-down view of the FRD coordinate system, simulating a mouse-up swipe gesture to adjust the viewing angle.
Parameters (Args):
None
Returns:

None

Exceptions (Raises):
None

`AppendPcd(Cloud)`¶

Function Description: Appends point cloud data to the current point cloud collection.
Parameters (Args):

Parameter Name	Type	Required	Default	Description
`Cloud`	`open3d.geometry.PointCloud`	Yes	-	Point cloud object to be appended

Returns:

None

Exceptions (Raises):
None

`merge_point_cloud(other_pcd)`¶

Function Description: Merges another point cloud into the current point cloud.
Parameters (Args):

Parameter Name	Type	Required	Default	Description
`other_pcd`	`open3d.geometry.PointCloud`	Yes	-	External point cloud object to be merged

Returns:

None

Exceptions (Raises):

ValueError: Raised when the input point cloud object is invalid.

`show_point_cloud()`¶

Function Description: Displays the currently stored point cloud data.
Parameters (Args):
None
Returns:

None

Exceptions (Raises):
None

`clear_point_cloud()`¶

Function Description: Clears all points from the point cloud and updates the visualization.
Parameters (Args):
None
Returns:

None

Exceptions (Raises):
None

`remove_duplicates()`¶

Function Description: Removes duplicate points (points with identical coordinates).
Parameters (Args):
None
Returns:

None

Exceptions (Raises):
None

`UpdateShow(Cloud)`¶

Function Description: Updates the point cloud data and refreshes the visualization display.
Parameters (Args):

Parameter Name	Type	Required	Default	Description
`Cloud`	`open3d.geometry.PointCloud`	Yes	-	Point cloud object to be updated and displayed

Returns:

None

Exceptions (Raises):
None

`UpdateShowWithColor(Cloud, colors=None)`¶

Function Description: Updates and displays colored point cloud data, applying FRD coordinate transformation.
Parameters (Args):

Parameter Name	Type	Required	Default	Description
`Cloud`	`numpy.ndarray`	Yes	-	Input point cloud coordinate array, shape (N, 3)
`colors`	`numpy.ndarray`	No	`None`	Point cloud color array, shape (N, 3), values in range [0, 1]

Returns:

None

Exceptions (Raises):
None

`UpdatePCD()`¶

Function Description: Updates the point cloud data in the visualization window.
Parameters (Args):
None
Returns:

None

Exceptions (Raises):
None

`CloseShow()`¶

Function Description: Closes the point cloud visualization window.
Parameters (Args):
None
Returns:

None

Exceptions (Raises):
None

Example:

from RflySimSDK.vision import Open3DShow
import numpy as np

# Create a visualization instance and launch the window
pcd_show = Open3DShow()
pcd_show.CreatShow()
# Set FRD top-down view
pcd_show.SetFRDView()

# Generate a sample colored point cloud and update the display
test_points = np.random.randn(1000, 3)
test_colors = np.random.rand(1000, 3)
pcd_show.UpdateShowWithColor(test_points, test_colors)

# Clear the point cloud and close the window
# pcd_show.clear_point_cloud()
# pcd_show.CloseShow()

Advanced Usage Examples¶

The following example uses VisionCaptureApi to acquire LiDAR point clouds and updates the Open3D display window at 10 Hz.

Reference example: [RflySim installation path]\RflySimAPIs\8.RflySimVision\0.ApiExps\1-UsageAPI\3.PointCloudAPI\2.LidarAPIPointCloudDemo

import sys
import time

import Open3DShow
import VisionCaptureApi


show3d = Open3DShow.Open3DShow()
vis = VisionCaptureApi.VisionCaptureApi()

vis.jsonLoad()
if not vis.sendReqToUE4():
    sys.exit(0)
vis.startImgCap()
show3d.CreatShow(0)

lastTime = time.time()
while True:
    lastTime += 1 / 10.0
    sleepTime = lastTime - time.time()
    if sleepTime > 0:
        time.sleep(sleepTime)
    else:
        lastTime = time.time()

    if vis.hasData[0]:
        show3d.UpdateShow(vis.Img[0])
        vis.hasData[0] = False

Notes and Pitfall Guide¶

Visualization Window Initialization Requirement: CreatShow must be called first to create the window before calling SetFRDView, AppendPcd, or other view or point cloud operations. Failing to initialize the window will cause access exceptions in subsequent operations.
Real-time Update Performance Limitations: Frequent calls to UpdateShow will consume excessive OpenGL rendering resources. It is recommended to batch add point clouds before updating uniformly. The number of points appended per frame should not exceed 1e5 to avoid screen lag.
Deduplication Parameter Settings: When calling remove_duplicates, the voxel size voxel_size must be set according to the actual scale of the point cloud. A parameter that is too small will not achieve downsampling, while a parameter that is too large may lose key structural information of the scene.
Multi-threaded Access Safety: The visualization rendering of Open3DShow must run on the main thread. Time-consuming operations such as point cloud acquisition and processing can be placed in child threads. Calling show_point_cloud to start the main rendering loop from a child thread is prohibited.

Update Log¶

2026-01-31: fix: Optimize point cloud display interface
2026-01-30: fix: Fix point cloud preview orientation issue
2025-12-31: fix: Update Open3D version notice
2025-08-19: fix: Add identification mechanism; use 1400-byte packets in WSL native mode to reduce packet loss probability in WSL
2025-07-26: fix: Fix laser point cloud shared memory bug
2025-07-25: fix: Update Open3D point cloud preview effect
2024-08-05: fix: Add HTML version API documentation
2024-07-17: fix: Update VisionCaptureApi interface
2024-06-03: fix: Update
2024-05-31: fix: Update PDF
2026-01-31: fix: Optimize point cloud display interface
2026-01-30: fix: Fix point cloud preview orientation issue
2025-12-31: fix: Update Open3D version notice
2025-08-19: fix: Add identification mechanism; use 1400-byte packets in WSL native mode to reduce packet loss probability in WSL
2025-07-26: fix: Fix laser point cloud shared memory bug
2025-07-25: fix: Update Open3D point cloud preview effect
2024-08-05: fix: Add HTML version API documentation
2024-07-17: fix: Update VisionCaptureApi interface
2024-06-03: fix: Update
2024-05-31: fix: Update PDF

Open3DShow Interface Documentation¶

Introduction¶

Quick Start¶

Environment and Dependencies¶

Core Interface Description¶

Global Constants and Enumerations¶

Standalone Constants¶

Global/Standalone Functions¶

Open3DShow Class¶

__init__()¶

get_new_point_cloud()¶

update_callback(vis)¶

CreatShow(idx=0)¶

SetFRDView()¶

AppendPcd(Cloud)¶

merge_point_cloud(other_pcd)¶

show_point_cloud()¶

clear_point_cloud()¶

remove_duplicates()¶

UpdateShow(Cloud)¶

UpdateShowWithColor(Cloud, colors=None)¶

UpdatePCD()¶

CloseShow()¶

Advanced Usage Examples¶

Notes and Pitfall Guide¶

Update Log¶

`Open3DShow` Class¶

`init()`¶

`get_new_point_cloud()`¶

`update_callback(vis)`¶

`CreatShow(idx=0)`¶

`SetFRDView()`¶

`AppendPcd(Cloud)`¶

`merge_point_cloud(other_pcd)`¶

`show_point_cloud()`¶

`clear_point_cloud()`¶

`remove_duplicates()`¶

`UpdateShow(Cloud)`¶

`UpdateShowWithColor(Cloud, colors=None)`¶

`UpdatePCD()`¶

`CloseShow()`¶