Visual Sensor Configuration Protocol

---

The RflySim toolchain simulates various external sensors independently of the flight controller using RflySim3D (UE engine). Users can flexibly define the type, quantity, mounting method, and data transmission protocol of sensors via the Config.json configuration file, supporting multi-modal perception simulation ranging from monocular cameras to LiDAR.

---

Supported External Sensor Types

External sensors are categorized into three main types based on their output data format: Image, Point Cloud, and Audio.

Image Sensors

Sensor	TypeID	Version Requirement
RGB Image	1	Free Edition
Depth Image	2	Free Edition
Grayscale Image	3	Free Edition
Segmentation Image	4	Full Edition
Distance (Straight-line)	5	Full Edition
Fisheye Camera	8	Full Edition
Gimballed Camera	9	Full Edition
Optical Flow Sensor	10	Full Edition
Simple Visual Sensor	30	Full Edition
Infrared Grayscale Image	40	Full Edition
Thermal Color Image	41	Full Edition

Point Cloud Sensors

Sensor	TypeID	Version Requirement
Depth-to-Pointcloud	7	Full Edition
LiDAR (Mechanical/Special Scanning)	20~23	Full Edition
Simple LiDAR Sensor	31	Under Development

LiDAR Notes

The LiDAR series primarily simulates products from Livox.

Audio Sensors

Sensor	TypeID	Version Requirement
Audio Sensor (Beta)	6	Under Development

---

Config.json Configuration Protocol

The Config.json file is used to configure external sensors, including their quantity, type, and mounting configuration. Within the same sensor, parameter order is not significant.

Basic Parameters

Parameter	Type	Description
SeqID	int	Unique identifier of the sensor within RflySim3D. Set to 0 for automatic sequential numbering
TypeID	int	Sensor type identifier; refer to the table above
TargetCopter	int	Target object ID (vehicle CopterID or sensor SeqID, depending on TargetMountType)
DataWidth	int	Data width (e.g., image pixel width / LiDAR horizontal points)
DataHeight	int	Data height (e.g., image pixel height / LiDAR vertical beam count)
DataCheckFreq	float	Sensor data update frequency (Hz)
CameraFOV	float	Field of View (degrees), representing camera FOV or LiDAR horizontal angular range
otherParams	float[16]	Reserved parameters; default all zeros. Specific meanings depend on sensor type

Mounting Position Parameters

Parameter	Type	Description
SensorPosXYZ	float[3]	Sensor position (meters), relative to the vehicle’s FRD coordinate system origin
EulerOrQuat	int	Angle representation mode: 0 = Euler angles, 1 = quaternion
SensorAngEular	float[3]	Mounting orientation (Euler angles, degrees): [roll, pitch, yaw], relative to the vehicle’s FRD frame
SensorAngQuat	float[4]	Mounting orientation (quaternion): [q0, q1, q2, q3]

TargetMountType — Mounting Method

Value	Mounting Method	Description
0	Fixed to vehicle (geometric center)	Mounted relative to the vehicle’s geometric center
1	Fixed to vehicle (bottom center)	Mounted relative to the vehicle’s bottom center
2	Fixed to ground	Suitable for simulating ground-based surveillance cameras
3	Gimballed camera in ground frame	Mounted on the aircraft, but camera orientation is independent of aircraft attitude
4	Attached to another sensor	TargetCopter refers to the SeqID of the target sensor

Note

When TargetMountType = 4, the target sensor to which the current sensor is attached must be defined first.

SendProtocol — Transmission Protocol Configuration

SendProtocol is a 7-element array defining the data transmission method.

Array Index	Description
[0]	Transmission protocol flag (see table below)
[1-4]	IP address; default 0 (equivalent to 127.0.0.1), auto-returned to localhost
[5]	Port number; default 0 uses 9999 + SeqID for automatic increment
[6-7]	Reserved

SendProtocol[0] Transmission Modes:

Value	Transmission Mode	Applicable Sensors	Characteristics
0	Shared Memory	All	Uncompressed, high speed, low latency; Windows-only, local machine
1	UDP + JPEG Lossy Compression	All	Supports cross-machine/cross-platform/ROS
2	UDP Uncompressed Direct Transmission	Image sensors only	More packets; packet loss possible over networks
3	UDP + PNG Lossless Compression	Image sensors only	Lossless compression transmission

Transmission Mode Selection Guide

• For local Windows environments, Mode 0 (shared memory) is recommended for lowest latency
• For cross-machine/cross-platform/ROS scenarios, Mode 1 (UDP + JPEG) is recommended
• For lossless image data, use Mode 3 (UDP + PNG)

Configuration Example

{
    "SeqID": 1,
    "TypeID": 1,
    "TargetMountType": 0,
    "TargetCopter": 1,
    "DataWidth": 640,
    "DataHeight": 480,
    "DataCheckFreq": 30,
    "SendProtocol": [0, 0, 0, 0, 0, 0, 0, 0],
    "CameraFOV": 90,
    "SensorPosXYZ": [0.1, 0, -0.05],
    "EulerOrQuat": 0,
    "SensorAngEular": [0, 0, 0],
    "otherParams": [0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0]
}

Full Example File

For more configuration examples, see: [Installation Directory]\RflySimAPIs\8.RflySimVision\0.ApiExps\1-UsageAPI\0.ConfigJsonAPI\Config.json

Notes

• SeqID Auto-Increment: When multiple sensors are assigned SeqID = 0, the system automatically assigns sequential IDs in ascending order.
• DataCheckFreq: The actual sensor data refresh rate is jointly determined by the RflySim3D frame rate and DataCheckFreq. The RflySim3D frame rate must be ≥ the maximum detection frequency among all sensors.
• CameraFOV: The field of view (FOV) has a fixed numerical relationship with focal length; FOV can be used to indirectly adjust focal length.
• Port Allocation: Each sensor must be assigned a unique return port to prevent data conflicts.

---

Other Sensor Interfaces

The following interfaces are not configured via Config.json, but are instead invoked directly through the Python SDK:

Interface Function	Functionality
SetUE4RadianceValue	Sets radiance parameters for infrared grayscale images or thermal segmentation maps

---

Appendix: TypeID Sensor Parameter Quick Reference Table

Typical Configuration Reference Values

Sensor Type	TypeID	Resolution (W×H)	Frequency	Protocol	Notes
RGB Image	1	640×480	30 Hz	0–3	—
Depth Image	2	640×480	30 Hz	0–3	otherParams specifies minimum/maximum distance and scale
Grayscale Image	3	640×480	30 Hz	0–3	—
Segmentation Map	4	640×480	30 Hz	0–3	—
Rangefinder	5	N/A	—	0–1	otherParams specifies maximum distance
Mechanical LiDAR	20–22	Points × Lines	30 Hz	0–1	otherParams specifies scan angle range
Spoke-type LiDAR	20–22	Petal Points × Petal Count	Configurable	0–1	otherParams specifies scan distance and petal height
Infrared Grayscale Image	40	640×480	30 Hz	0–3	—

Appendix: Transmission Architecture Diagram