Skip to content

Quick Start: RflySim Intelligent Unmanned Systems Ecosystem

Welcome to RflySim! This chapter serves as the central hub for exploring the RflySim intelligent unmanned systems toolchain. Before diving into specific environment installation and development/testing, we will help you build a comprehensive understanding of the RflySim ecosystem and guide you through running your first simulation experiment.

🌟 RflySim Platform Positioning and Core Capabilities

The RflySim Intelligent Unmanned Swarm System Automated Design and Development Toolchain (RflySim Toolchain) is a professional, free, and open-source development toolchain jointly developed by the Zhuoyi Intelligent FlyLab (Beijing), Professor Xunhua Dai’s team (Central South University), and Professor Quan Quan’s team (Beihang University). It provides an end-to-end automated solution for unmanned systems, covering the entire workflow from algorithm development, hardware-in-the-loop (HITL) simulation, to real-platform deployment experiments. The toolchain adopts a Model-Based Design (MBD) philosophy and is built upon software systems such as PX4, MATLAB/Simulink, and ROS, along with relevant intelligent hardware. It enables simulation and practical experimentation in areas including (but not limited to) unmanned vehicle control, swarm coordination, and vision-based perception.

  • Cross-Domain Multi-Platform Simulation: Supports simulation of various unmanned platforms, including aerial (multi-rotors, fixed-wing, etc.), ground (unmanned vehicles, robots), surface, and underwater systems.
  • Single-Agent to Swarm Coordination: Beyond single-agent intelligent control validation, it further supports large-scale intelligent unmanned swarm systems (IUSS), enabling cooperative formation, multi-agent confrontation, and reinforcement learning.
  • Embodied Intelligence and Vision Perception: Integrating high-fidelity 3D rendering from Unreal Engine (UE), it builds an end-to-end verification environment supporting physical interaction simulation and advanced vision perception.

alt text

🏗️ Core Technical Architecture and Component Diagram

alt text

The overall architecture of the RflySim toolchain consists of multiple functional modules, primarily including:
1) Software/Hardware Simulation Nodes (comprising the Vehicle Dynamics Simulator and Low-Level Controller),
2) the 3D Simulator, and
3) the Upper-Level Controller for vision and swarm control.

Together, these modules form a complete closed-loop system compatible with both Software-in-the-Loop (SITL) and Hardware-in-the-Loop (HITL) simulations, fully covering the entire workflow of unmanned systems—from algorithm development to verification and testing.

  1. Vehicle Dynamics Simulator: This is one of the core modules of the RflySim toolchain, primarily used to achieve high-accuracy modeling and simulation of vehicle dynamics. Developed within the MATLAB/Simulink environment, it constructs an unmanned system modeling framework with six-degree-of-freedom kinematics and dynamics, suitable for multi-rotor drones, fixed-wing drones, unmanned ground vehicles, and other heterogeneous mobile platforms. After modeling, the system automatically generates high-performance C/C++ source code from the model via Simulink’s code generation functionality, which is further compiled into a dynamic-link library (DLL) for efficient integration with the simulation platform. During runtime, this DLL is loaded into CopterSim, the core scheduler of the RflySim toolchain, and协同 with real-time scheduling mechanisms to perform periodic solving—forming a closed-loop operation with the Low-Level Controller, supporting collaborative simulation in both SITL and HITL modes.

  2. Low-Level Controller: In the RflySim toolchain, the Vehicle Dynamics Simulator simulates physical responses in real-world environments—including kinematics, sensor, and actuator modeling—whereas the Low-Level Controller receives sensor data and generates control commands to drive the simulated vehicle stably. This module includes both control software and hardware systems, offering a complete closed-loop capability for pose control.

  3. 3D Simulator (Virtual Visual World): This module provides drones with “eyes” and a realistic operational space. Based on extensive customization of Unreal Engine (UE4 or UE5), RflySim3D / RflySimUE5 serves as the 3D simulation engine. It receives “simulation ground truth” (i.e., precise position and attitude of the drone in space) from CopterSim via the UDP protocol, then moves the drone model in real time within the virtual 3D world. It supports mounting and simulating various complex visual sensors, such as monocular cameras, stereo cameras, LiDAR, and gimbals, and transmits the rendered 3D visual images to the upper-level controller below.

  4. Upper-Level Controller for Vision and Swarm (Advanced Decision “Brain”): This module acts as the “cerebral cortex” of the drone, handling “intelligent” tasks such as visual navigation and formation flight. It receives real-time images from the 3D simulator (to ensure high frame rate and low latency, shared memory technology is employed). It supports two modes:
    ① Vision Software-in-the-Loop: Run Python/computer vision algorithms directly on a standard desktop (Windows).
    ② Vision Hardware-in-the-Loop: Deploy algorithms onto real onboard computers (e.g., Raspberry Pi or Jetson platforms running Linux/ROS) to test computational performance and real-time capability.

The Swarm Networking and Data Interaction Submodule simulates communication networks among multiple drones. It receives physical state information from drones and distributes visual perception data laterally. The Swarm Decision Controller and Ground Station Decision Layer are high-level algorithm programs written in ROS, Python, or MATLAB. Based on visual recognition results (e.g., “obstacle detected ahead”) or formation requirements, they compute the next action plan. The ground station uses QGroundControl (QGC) for manual monitoring, route planning, and parameter tuning.

🗺️ Learning Path and Practical Navigation

We have designed a complete progression path—from foundational concepts to Hardware-in-the-Loop (HITL)/Software-in-the-Loop (SITL) execution, and finally to advanced development:

graph TD
    A[Build Global Understanding] --> B[Environment Installation & Configuration]
    B --> C{Run Basic Simulation}
    C -->|No Physical Hardware| D[SITL: Software-in-the-Loop Simulation]
    C -->|Connect Real Flight Controller| E[HITL: Hardware-in-the-Loop Simulation]
    D --> F[Advanced Development & API]
    E --> F
    F --> G[Real-Platform Flight Deployment]

    style A fill:#e1f5fe,stroke:#03a9f4,stroke-width:2px
    style B fill:#fff3e0,stroke:#ff9800,stroke-width:2px
    style C fill:#f3e5f5,stroke:#9c27b0,stroke-width:2px

🗂️ Chapter Navigation

Please follow the sequence below to begin your RflySim exploration journey:

Section Description
1. Toolchain Overview Gain deep insight into cutting-edge developments in unmanned systems, the origin of the RflySim name, and core design philosophy
2. Installation and Environment Configuration Covers rapid standardized deployment of Windows host, WSL subsystem, and related development dependencies
3. 5-Minute Quick Start Easily launch and complete your first multi-rotor SITL/HITL simulation workflow