Introduction to Core Components of RflySim Platform

RflySim platform includes many software involved in the development process of unmanned system modeling, simulation and algorithm verification. The core components include CopterSim, QGroundControl, RflySim3D/RflySimUE5, Python38Env, Win10WSL subsystem, SITL/HITLRun one-click running script, MATLAB automatic code generation toolbox, Simulink cluster control interface, PX4 Firmware source code, RflySim supporting data file and supporting hardware system. By learning these core components, users can quickly start the development and testing of unmanned systems.

1、CopterSim

CopterSim is one of the core software of RflySim platform, which is a hardware-in-the-loop simulation software developed for Pixhawk/PX4 autopilot platform. It can configure multi-rotor model in the software and connect with Pixhawk autopilot through USB serial port to realize hardware-in-the-loop simulation. The effect of indoor simulation and outdoor flight test is achieved. It mainly consists of two parts — model and communication. The model means that the simulation can be carried out directly after calculation according to the set model parameters; it also supports running the dynamic model (DLL), and forms the software/hardware in the loop simulation together with other software. CopterSim is the center of all data communication; the flight control and CopterSim are connected through serial port (HITL) or network TCP/UDP (SITL), and MAVLink is used for data transmission to realize closed-loop control and simulate outdoor flight; CopterSim sends aircraft pose and motor data to the 3D engine for visual display; forwards MAVLink messages to Python Vision or QGC ground station to transmit aircraft real-time status for top-level planning control; and so on. At the same time, CopterSim software compresses MAVLink data and sends it to the cluster control software in the form of UDP structure to achieve the purpose of communication simplification (large-scale cluster requirements).

2、RflySim3D/RflySimUE5

Unreal Engine has a powerful graphics engine that supports high-quality 3D graphics and visual effects; the built-in blueprint visualization scripting system allows developers to create complex logic and interactive behavior in a graphical way without writing code; It has a huge community support and resource library, including models, textures, sound effects, plug-ins, etc., which can help developers speed up the development process and improve the quality of models; it supports multiple platforms, including PC, mainframe, mobile devices and virtual reality devices, etc.; Developers can customize and extend the functions and tools of the engine according to their own needs, making Unreal Engine suitable for various types of game and application development.

RflySim3D/RflySimUE5 is a highly realistic simulation software for unmanned system based on Unreal Engine, which inherits the advantages of Unreal Engine and communicates with other software on the platform in the form of UDP to realize highly realistic simulation of unmanned system. Visual image data can be transmitted to QGroundControl, MATLAB, Python and other software by means of screen capture and shared memory to realize the visual algorithm verification and simulation of unmanned system.

At the same time, for users with low computer configuration, RflySim platform provides two other three-dimensional simulation software, namely FlightGear and 3D Display. FlightGear’s team of developers from all over the world, including programmers, pilots, physicists and aircraft manufacturers, provides many different types of aircraft models and scenarios, including various civil and military aircraft models, as well as many different scenarios and environmental simulations. It is a very popular open source flight simulator software, which can receive the flight state sent by Simulink through UDP, and easily observe the flight state of the aircraft during Simulink simulation. 3DDisplay is a virtual flight simulator software developed by the Reliable Flight Control Research Group of Beihang University, which provides three-dimensional models and virtual environments, and supports a variety of aircraft models and scenes. Users can freely switch RflySim3D/RflySimUE5, FlightGear and 3DDisplay simulation software according to the configuration of personal computers.

3. QGroundControl Ground Station

The UAV ground station is a key component of the UAV application control system. The operator can operate the ground station through the mouse, touch screen and remote control handle to achieve the purpose of controlling the UAV. By setting the waypoint information and planning the route on the ground station, the UAV can fly according to the preset path and complete the waypoint task during the flight, including taking photos and aircraft actions. At present, the mainstream open source ground stations are QGroundControl and MissionPlanner, and QGroundControl is an open source ground station designed for the latest architecture of PX 4 software, which uses QT editor C + + language to write its core code, and supports source code modification and secondary development of functions. It is not only suitable for UAV ground station research and experiment, but also suitable for UAV ground station function customization and modification. In comparison, the advantages of QGroundControl are:

1) Open Source: QGroundControl is a completely open source software, which means that users can freely modify and customize it according to their needs.

2) Ease of Use: The user interface is very clear, modern, and easy to use, allowing users to quickly perform mission planning and flight planning.

3) Multi-platform support: QGroundControl runs on multiple operating systems, such as Windows, Linux, and MacOS.

4) Modular architecture: QGroundControl’s modular architecture makes it easy for developers to add and extend new functionality without compromising existing functionality and performance.

Overall, QGroundControl is a modern, easy to use, open source and highly customizable ground station software, which has obvious advantages in multi-platform support, multi-language support, modular architecture and so on.

4、Python38Env

Python is a high-level, object-oriented, interpreted programming language. Originally created by Guido van Rossum in 1989, it has become a popular programming language for developing Web applications, data analysis, artificial intelligence, scientific computing, network programming, and more. Python is a language that is easy to learn, easy to read, and easy to write, so it is also widely used for teaching and entry-level programming.

Python38Env is a Python 3.8 programming language virtual environment, including numpy, pymavlink, OpenCV, pyulog and other libraries, which can quickly develop algorithms related to unmanned systems without requiring users to deploy python runtime environment and various function libraries.

5. MATLAB Automatic Code Generation Toolbox

MATLAB Automatic Code Generation Toolbox is an extended toolkit of MATLAB, which is used to generate C code, executable files, static libraries, dynamic libraries and other forms of executable files from Simulink models. These executables can be run directly on embedded platforms without the need for manual writing and debugging. Supports multiple embedded platforms including ARM Cortex-M and A-series processors, NXP MPC55xx and MPC56xx series, Pixhawk series, and more.

The module library includes GPS data module, battery data module, uORB module and many other modules. Based on RflySim and the Pixhawk Support Package platform, users can design and simulate control algorithms in Simulink, automatically generate C code and PX4 firmware from Simulink models, and burn them directly to the Pixhawk board; Use MATLAB scripts and functions to configure and calibrate the Pixhawk board and its peripheral devices; Read and write data with the Pixhawk board in real time, etc.

6. SITL/HITL batch script

Batch processing technology means that the computer can process several tasks collected in groups, and the whole process is completely automated without human intervention, which can also be called workload automation (WLA) and job scheduling. It has the advantages of speed, cost saving, accuracy and simple operation.

RflySim has developed a number of batch scripts based on batch processing technology, allowing users to quickly deploy multiple, multiple, and multiple unmanned system combined simulations with one click. And improve that development and simulation speed of the unmanned system. The common batch processing scripts of the platform are as follows: (1) SITLRun.bat: a batch processing file for starting multi-machine software-in-the-loop simulation, which essentially starts and configures part of the software and options of the RflySim platform in a script mode; (2) HITLRun.bat: a batch processing file for starting multi-machine hardware-in-the-loop simulation, Double-click the batch file and input the Pixhawk serial port number you want to participate in the simulation according to the prompt to start the hardware-in-the-loop simulation of multiple machines (sort the aircraft IDs in the order of input serial ports). In addition, the RflySim platform provides a number of batch script files. Such as SITLRunPos.bat, SITLRunLowGPU.bat, SITLRunMAVLink.bat, HITLRunPos.bat, HITLPosSysID.bat, HITLPosStr.bat and the like. Users can open these files through the editor, modify the parameters according to their personal needs, realize custom development, and quickly start simulation or algorithm verification.

7. PX4 Firmware source code

PX4 evolved from PIXHAWK, a software and hardware project at the Computer Vision and Geometry Laboratory of the Federal Institute of Technology (ETH) in Zurich, Switzerland. The flight control system is completely open source, providing a low-cost and high-performance high-end autopilot for flight control enthusiasts and research teams around the world. After years of development and improvement by world-class developers from industry and academia, the PX4 flight control system has formed a perfect and reasonable software architecture, and with the Pixhawk series autopilot hardware platform, it constitutes the Pixhawk PX4 autopilot software and hardware platform, which can control multi-rotor, fixed-wing, airship and other vehicles. It is an open source UAV autopilot software and hardware platform widely used in the world.

The RflySim platform supports one-click deployment of the PX4 compilation environment. Different PX4 firmware compilation commands and firmware versions can be selected by customization. The platform will deploy the selected PX4 Firmware source code on the set installation path. If the firmware exists, the old firmware folder will be deleted and a new deployment will be made. Greatly improves the efficiency of PX4 environment deployment.

8. Win10 WSL subsystem

The Win10 WSL subsystem is a subsystem on the Windows operating system where users can run Linux applications, use the Linux command line interface (CLI), and install Linux distributions. The Linux system installed on the RflySim platform with one click is Ubuntu 18.04.5, which is mainly used for compiling the PX4 source code.

This platform also provides two other compiling environments to realize the simulation of Linux compiling environment under Windows platform, which are Msys2Toolchain compiling environment based on Msys2 and CygwinToolchain compiler based on Cygwin. Users can select different compilation environments according to their own PX4 version, and different selections can be made in the one-click deployment and installation interface to complete the switching of different compilation environments.

9. Simulink cluster control interface

The RflySim platform develops a cluster control interface based on Simulink S function, which is realized by C + + mixed programming through Simulink S function, and has the advantages of high efficiency, small operation, low delay, more reliability and strong expansibility with the advantages of the UDP module of Simulink. Users can load the module into their own control system by copying and pasting to help users quickly realize the development of unmanned system cluster control.

10. Supporting documents of RflySim

RflySim platform provides very perfect learning materials and routine files, through PPT courseware materials and RflySim APIs routine files, so that users can gradually and progressively learn the bottom control algorithm of unmanned system, the middle decision-making algorithm and the top learning algorithm. Build and develop the unmanned system you need in a one-stop way.

11. Supporting hardware system

RflySim platform provides a complete set of supporting hardware system, including four-rotor UAV, flight control, remote control and other components. These components are perfectly compatible with the platform, and can realize the software and hardware in the loop simulation experiment in the RflySim platform, and realize the flight of UAV in the real environment based on the generated firmware.

At present, the supported aircraft include Feisi X150, Feisi X200, Feisi X450 and other four-rotor UAVs, among which Feisi X150 is a newly designed micro-four-rotor UAV for indoor cluster control research. Any flight control that supports the PX4 software system can be used on the RflySim platform. Pixhawk 2.4.8 (also known as Pixhawk 1), Pixhawk 6C and Pixhawk 6X are currently supported for a long time. The remote controller used on this platform is recommended to use the “American hand” control mode, that is, the left joystick corresponds to the throttle and yaw control, while the right joystick corresponds to the roll and pitch. In the remote controller, roll, pitch, throttle and yaw respectively correspond to CH1 ~ CH4 channels of the receiver, and the left and right upper levers correspond to CH5/CH6 channels, which are used to trigger flight mode switching. Supported remote control brands and models include Ledi AT9S Pro, Tiandifei ET10, Flowserve i6s, Futuba T14SG, etc.