AutoVisConf Interface Documentation

Introduction

Brief: AutoVisConf.py provides visual sensor initialization, image display, pod perspective control, and visual fault injection capabilities for the RflySim PHM automated testing workflow.

This module is centered around the MavVIS class, which automatically locates the Config.json file in the corresponding model directory based on the aircraft configuration, requests visual sensor data from RflySim3D via VisionCaptureApi, and starts image acquisition. During initialization, it also sets the RflySim3D window resolution to 1280x720 and the maximum frame rate to 30 FPS.

In PHM automated testing, MavVIS can display camera images along with overlaid information such as attitude and field of view (FOV), and supports simulating image data loss, pod attitude disturbances, and image Gaussian noise. This class is created by AutoMavCtrl.AutoMavCtrler when visual functionality is enabled in a test case, but can also be used independently when the same directory structure and simulation prerequisites are met.

Quick Start

The following example initializes visual acquisition for a quadcopter SITL scenario and continuously displays the sensor feed. Before running, a RflySim3D scene containing the corresponding visual sensor must be started first.

The parent directory of the script's location must contain a model/Quadcopter directory, and within that directory, there must be at least one visual configuration file whose filename contains Config.json.

from AutoVisConf import MavVIS


conf = ["Quadcopter", "SITL", 1]
visual = MavVIS(conf)

while True:
    visual.visShow()

conf follows the configuration format of the PHM automated testing framework: the first item is the aircraft model name, the second is the simulation mode, and the third is the instance number. AutoVisConf.py currently only directly reads the first item to select the model directory, but it is recommended to pass the complete configuration for consistency with AutoMavCtrl.

Environment and Dependencies

• Python Environment: >= 3.8.10
• Third-party Dependencies: opencv-python (imported as cv2), numpy
• RflySimSDK Dependencies: AutoREG, CameraCtrlApi, UE4CtrlAPI, VisionCaptureApi
• Standard Library Dependencies: math, os, random, re, sys, time
• Prerequisites: RflySim3D is running, the visual sensor configuration is available, and the corresponding aircraft model configuration directory exists relative to the current script path.

Core Interface Description

This module AutoVisConf.py does not define global constants, enumerations, or independent functions; it only provides the MavVIS class.

Global Constants and Enum Definitions

Independent Constants

None

---

Enum Definitions

None

---

Global/Independent Functions

None

---

MavVIS Class

MavVIS encapsulates visual sensor configuration loading, RflySim3D request sending, image acquisition, image display, and pod fault injection. Upon instantiation, it immediately connects to the visual link and starts image acquisition. Therefore, the object should be created after RflySim3D and the visual scene are ready.

Key instance attributes are as follows:

Attribute	Type	Description
conf	list / tuple	PHM scene configuration, typically formatted as [AircraftModel, SimulationMode, InstanceNumber]
vis	VisionCaptureApi	Object for visual sensor configuration, image buffer, and acquisition status
ue	UE4CtrlAPI	Interface object for sending console commands to RflySim3D
vspath	str	Automatically found path to the visual sensor Config.json
LastSensorAngEular	list[float]	Last displayed pod Euler angles [Roll, Pitch, Yaw]
CoordName	str	Name of the attitude component currently displayed on the left side of the image

__init__(conf)

Function Description: Initializes visual acquisition and RflySim3D communication. This method performs the following operations:

1. Determines the aircraft model directory based on conf[0] and AutoREG.RFLYSIM_FRAME.
2. Searches for the first file whose filename contains Config.json under [Current Script Directory]/../model/<AircraftModel>/.
3. Sets the RflySim3D resolution to 1280x720 and the maximum frame rate to 30 FPS.
4. Loads the visual configuration, sends a sensor request to RflySim3D, and starts image acquisition.

Parameters (Args):

Parameter Name	Type	Required	Default Value	Description
conf	list / tuple	Yes	-	PHM configuration. conf[0] must be Quadcopter, Fixedwing, or Vtol; the common complete format is [AircraftModel, SimulationMode, InstanceNumber]

Return Value (Returns):

• MavVIS: The initialized visual control object.

Exceptions and Exit Behavior (Raises / Exit):

• KeyError: conf[0] is not an aircraft model name supported by AutoREG.RFLYSIM_FRAME.
• FileNotFoundError: The model directory for the corresponding aircraft does not exist.
• IndexError: No configuration file with a filename containing Config.json was found in the model directory.
• SystemExit: If sendReqToUE4() returns failure, the source code directly calls sys.exit(0) to terminate the process.

Example:

from RflySimSDK.phm.AutoVisConf import MavVIS


visual = MavVIS(["Quadcopter", "SITL", 1])

---

visShow()

Function Description: Iterates through self.vis.hasData and displays all sensor images that have data using OpenCV. When processing index 0, this method also updates the first visual sensor with the following fixed configuration and sends it to RflySim3D via sendUpdateUEImage():

Configuration Item	Set Value	Description
TargetCopter	1	Binds the perspective to aircraft number 1
TargetMountType	0	Uses the aircraft mounting method
CameraFOV	90	Sets the field of view to 90 degrees
SensorPosXYZ	[0.3, -0.15, 0]	Sets the relative installation position of the sensor
SensorAngEular	[0, 0, 0]	Resets the sensor Euler angles to zero

Parameters (Args): None

Return Value (Returns): None

Exceptions (Raises): Does not explicitly catch OpenCV display errors, visual sensor index errors, or underlying communication exceptions.

Example:

visual.visShow()

Fixed Update of the First Sensor

Each call to visShow() overwrites the target aircraft, mount type, field of view, position, and attitude of sensor 0. If the scene needs to preserve these parameters from Config.json, this method should not be called in a direct loop, or the fixed values in the source code should be modified before calling.

---

gasuss_noise(image, mu, sigma)

Function Description: Adds Gaussian noise to the input image. The source code first converts the image to the floating-point domain by dividing by 255, generates normally distributed noise with the same shape as the image, adds it to the image, and finally clips and converts it to a uint8 image.

The method name is spelled gasuss_noise in the source code; this spelling must be preserved when calling.

Parameters (Args):

Parameter Name	Type	Required	Default Value	Description
image	numpy.ndarray	Yes	-	Input image, typically an OpenCV image with pixel range 0~255
mu	float	Yes	-	Mean of the Gaussian distribution in the normalized image domain
sigma	float	Yes	-	Standard deviation of the Gaussian distribution in the normalized image domain

Return Value (Returns):

• numpy.ndarray: A noisy image with the same shape as the input image and data type uint8.

Exceptions (Raises): Does not explicitly throw exceptions; if the input cannot undergo numerical division, or if parameters cannot be passed to numpy.random.normal(), the underlying exception will propagate directly.

Example:

import cv2


image = cv2.imread("camera.png")
noisy_image = visual.gasuss_noise(image, 0.0, 0.05)
cv2.imshow("Noisy image", noisy_image)
cv2.waitKey(0)

Noise Parameters and Negative Value Handling

mu and sigma operate on the normalized image after division by 255, not on pixel value units. For example, sigma=0.05 corresponds to approximately 12.75 pixel grayscale levels of standard deviation. The source code sets the lower clipping bound to -1 if the result contains negative values, and then directly converts to uint8, which may cause negative values to wrap around. If strict image quality is required, the output should be checked, or the result should be clamped to 0~1 externally.

---

Podfault(PodfalutId)

Function Description: Reads the keyboard pod control state, injects the specified visual fault, draws FOV and attitude overlay information, and displays all visual sensor images. At the end of the method, it writes the CameraFOV and AngEular from the keyboard controller back to visual sensor 0.

This method recreates the CameraCtrlApi.KeyCtrl and CameraCtrlApi.ImageCtrl objects each time it is called and always waits for 2 seconds. The parameter name is spelled PodfalutId in the source code.

Parameters (Args):

Parameter Name	Type	Required	Default Value	Description
PodfalutId	int	Yes	-	Pod visual fault ID; passing other values will not execute the three defined fault branches

Implemented Fault IDs:

Fault ID	Source Code Behavior	Impact
123549	Sets all self.vis.hasData flags to False	Simulates no data from the visual sensor; each image in the current call will output SensorN: No data!
124350	Adds random integers in the range [-10, 10] to Roll, Pitch, and Yaw respectively	Simulates random disturbance of pod attitude
124351	Calls gasuss_noise(image, 0.0, 0.5) on each valid image	Simulates strong Gaussian image noise
Other values	Does not enter the fault injection branch	Still draws image overlay information and updates sensor 0's FOV and attitude

Return Value (Returns): None

Exceptions (Raises): Does not explicitly catch image processing, window display, sensor index, or RflySim3D communication exceptions.

Example:

# Normal display and synchronization of pod control parameters
visual.Podfault(0)

# Inject Gaussian noise into the current valid images
visual.Podfault(124351)

Inconsistent Fault ID Calls

The attitude disturbance and image noise IDs implemented in AutoVisConf.py are 124350 and 124351 respectively, but the fault gating logic in the current AutoMavCtrl.py uses 125340 and 125341. Triggering these two types of faults via AutoMavCtrl will not hit the corresponding branches in Podfault(); the two sets of IDs should be unified before integrated use.

Advanced Usage Example

The following example demonstrates sequentially viewing the normal image, attitude disturbance, and image noise in an already started visual scene. Each call to Podfault() blocks for at least approximately 2 seconds, making it suitable for PHM test steps but not for high-frequency image processing loops.

from AutoVisConf import MavVIS


visual = MavVIS(["Quadcopter", "SITL", 1])

# Normal image, while allowing CameraCtrlApi to handle pod perspective control
visual.Podfault(0)

# Inject a single random pod attitude disturbance
visual.Podfault(124350)

# Inject strong Gaussian noise into all currently valid sensor images
visual.Podfault(124351)

Fault injection directly modifies the sensor state or image buffer in the current memory and does not include parameters for duration, automatic recovery, or thread scheduling. If a test case requires maintaining a fault, the upper-level test loop should repeat the call; if recovery is needed, the upper-level control flow should stop the injection and wait for new sensor frames to overwrite the current buffer.

Notes and Pitfall Avoidance Guide

• Model directory resolved relative to script location: The source code uses sys.path[0]/../model/<aircraft type>, not the current working directory or SDK installation directory as the base. After moving the entry script, ensure the relative directory structure is correct.
• Uncertain configuration file selection: When multiple files containing Config.json in their names exist in the directory, the source code uses the first match returned by os.listdir() without sorting. It is recommended to keep only one target configuration per aircraft type directory to avoid loading results varying with file system enumeration order.
• Process termination on request failure: When a visual request fails, the constructor executes sys.exit(0), with an exit code still of 0. If the upper layer needs to recover or log the failure, adjust this exit strategy before integration.
• visShow() overwrites configuration: This method fixedly controls sensor 0 of aircraft 1 and does not automatically switch the target instance based on conf[2]. Pay special attention during multi-aircraft testing.
• Podfault() is not a real-time loop interface: Each call fixedly executes time.sleep(2) and repeatedly creates keyboard and image control objects. Do not call it directly in threads requiring high refresh rates.
• Fault state is not responsible for recovery: 123549 directly clears the hasData flag, and the module has no corresponding recovery function; recovery depends on subsequent acquisition threads re-writing the state or reinitializing visual acquisition.
• Window resources managed by the caller: The module does not provide methods to stop acquisition or close OpenCV windows. At the end of the program, call cv2.destroyAllWindows() based on the application lifecycle, and manage the visual acquisition thread from the upper layer.
• Unused imports do not represent additional functionality: Although the source code imports PX4MavCtrlV4 and re, the current class implementation does not call them.

Changelog

• 2026-04-08: fix: Corrected cv2 import method
• 2025-04-01: fix