ScreenCapApiV4 Interface Documentation

Introduction

Overview: This module provides a Python interface for multi-window capture and manipulation on the Windows platform. It supports retrieving window handles, extracting window content into OpenCV-format images, and moving specified windows, meeting the requirements for screen content acquisition in multi-window scenarios.

In RflySim drone simulation tasks, simulation visualization windows and various task debugging windows often coexist. Many vision-based autonomous drone tasks require directly capturing real-time frames from simulation windows rather than acquiring them through camera sensor channels. This module adapts to the Windows windowing mechanism, supporting simultaneous enumeration and capture of multiple target windows, and outputs images in a format compatible with OpenCV processing, facilitating developers to directly integrate with various vision detection and recognition algorithms. It is commonly used in RflySim platform computer vision workflows for screen capture in multi-window simulations, simulation demo layout adjustments, and similar tasks.

Quick Start

The following example retrieves the handle and window information of the first RflySim3D window, then continuously captures and displays its frames.

Reference example: [RflySim installation path]\RflySimAPIs\8.RflySimVision\1.BasicExps\1-VisionCtrlDemos\e5_ScreenCapAPI\1-ShootBall

import cv2
import ScreenCapApiV4 as sca


windowHandles = sca.getWndHandls()
if not windowHandles:
    raise RuntimeError("RflySim3D window not found")

windowInfo = sca.getHwndInfo(windowHandles[0])

while True:
    imageBgr = sca.getCVImg(windowInfo)
    cv2.imshow("RflySim3D", imageBgr)
    if cv2.waitKey(1) & 0xFF == ord("q"):
        break

Environment & Dependencies

• Python Environment: >= 3.8.10
• Dependencies: ctypes, cv2, d3dshot, numpy, re, sys, warnings, win32con, win32gui, win32ui
• Prerequisites: Before calling this interface, ensure the system supports screen capture functionality and that the RflySimSDK.vision module has been correctly imported.

Core Interface Description

The module ScreenCapApiV4.py includes configuration variables, helper functions, and core business classes.

Global Constants and Enumerations

This section lists all globally accessible constants and enumeration definitions directly referenceable within the module.

Standalone Constants

Variable Name	Type	Value	Description
_RFLYSIM3D_WINDOW_TITLE	re.Pattern	^RflySim3D(?:\s.*)?-(\d+)$	Matches titles such as RflySim3D-0 and RflySim3D UE4 Full v5.01-0, and extracts the trailing window number.

---

Global/Standalone Functions

window_enumeration_handler(hwnd, window_entries)

Function Description: A callback function for Windows window enumeration. It only collects RflySim3D windows whose class name is UnrealWindow, that are currently visible, and whose title matches the _RFLYSIM3D_WINDOW_TITLE rule. Parameters:

• hwnd: Handle of the window currently being enumerated
• window_entries: Result list; each element is a (window number, window handle, window title) tuple

Return Value:

• None (returning a non-zero value indicates continuing enumeration)

Exceptions: None

---

getWndHandls()

Function Description: Enumerates currently visible RflySim3D windows, sorts them in ascending order by the numeric number at the end of the title, and returns the corresponding handles. If multiple windows share the same number, the function emits a RuntimeWarning; the original EnumWindows order is preserved among windows with the same number, and the caller should verify window identity themselves. Parameters: None Return Value:

• list[int]: A list of RflySim3D window handles sorted by numeric number

Exceptions: None

---

getHwndInfo(hWnd)

Function Description: Retrieves basic information (e.g., position and size) of a specified window based on its handle, for subsequent window screenshot operations. Parameters:

• hWnd: Handle of the target window

Return Value:

• dict: A dictionary containing window handle, position coordinates, and size dimensions

Exceptions: None

---

getCVImg(wInfo)

Function Description: Captures a screenshot of the specified window and converts it into an OpenCV-compatible BGR image format. Parameters:

• wInfo: Window information dictionary containing window handle, position, and size, obtained via getHwndInfo

Return Value:

• numpy.ndarray: BGR screenshot array in OpenCV format

Exceptions: None

---

getCVImgList(wInfoList)

Function Description: Performs batch screenshot capture of multiple windows and converts them into OpenCV format images. Parameters:

• wInfoList: A list of window information dictionaries, each representing a single window

Return Value:

• list[numpy.ndarray]: A list of OpenCV-format screenshots corresponding to each window

Exceptions: None

---

moveWd(hwd, x=0, y=0, topMost=False)

Function Description: Moves the specified window to a given screen coordinate position, optionally setting it to always stay on top. Parameters:

• hwd: Handle of the target window
• x: Target X-coordinate of the window's top-left corner on the screen (default: 0)
• y: Target Y-coordinate of the window's top-left corner on the screen (default: 0)
• topMost: Whether to set the window as always-on-top (default: False, i.e., not always on top)

Return Value:

• None

Exceptions: None

---

clearHWND(wInfo)

Function Description: Releases GDI resources occupied during window screenshot operations to prevent resource leaks. Parameters:

• wInfo: Window information dictionary that has completed screenshot operations and contains GDI resource handles requiring release

Return Value:

• None

Exceptions: None

---

WinInfo Class

Stores resources and dimensional information related to Windows window screenshot operations, providing foundational data structures for screen capture functionality.

__init__(hWnd, width, height, saveDC, saveBitMap, mfcDC, hWndDC)

Function Description: Initializes a window information object, storing various resource handles and dimensional parameters required for window screenshot operations. Parameters (Args):

Parameter Name	Type	Required	Default	Description
hWnd	int	Yes	-	Handle of the target window
width	int	Yes	-	Width of the capture area (in pixels)
height	int	Yes	-	Height of the capture area (in pixels)
saveDC	int	Yes	-	Handle of the compatible device context
saveBitMap	int	Yes	-	Handle of the bitmap object
mfcDC	int	Yes	-	Handle of the MFC device context
hWndDC	int	Yes	-	Handle of the target window's device context

Return Value (Returns):

• WinInfo instance object

Exceptions (Raises):

• None

---

Advanced Usage Example

The following example retrieves two RflySim3D windows, adjusts their screen positions, and uses getCVImgList to batch capture images from both windows.

Reference example: [RflySim installation path]\RflySimAPIs\8.RflySimVision\1.BasicExps\1-VisionCtrlDemos\e5_ScreenCapAPI\2-CrossRing

import cv2
import ScreenCapApiV4 as sca


windowHandles = sca.getWndHandls()
if len(windowHandles) < 2:
    raise RuntimeError("This example requires two RflySim3D windows")

# Place the two windows side by side
nextX, nextY = sca.moveWd(windowHandles[0], 0, 0, True)
sca.moveWd(windowHandles[1], nextX, 0, False)

windowInfoList = [
    sca.getHwndInfo(windowHandles[0]),
    sca.getHwndInfo(windowHandles[1]),
]

while True:
    imageList = sca.getCVImgList(windowInfoList)
    for index, imageBgr in enumerate(imageList):
        cv2.imshow("RflySim3D-" + str(index), imageBgr)
    if cv2.waitKey(1) & 0xFF == ord("q"):
        break

Notes and Pitfall Avoidance Guide

• Window State: getHwndInfo() exits directly when it encounters a minimized window whose client area width and height are both 0; getCVImg() and getCVImgList() also exit when the handle becomes invalid. Do not minimize or close the target window during capture.
• Window Title Rules: getWndHandls() only accepts visible UnrealWindow windows that start with RflySim3D and end with -number; other UE windows will not appear in the returned list.
• Duplicate Numbers: When multiple visible windows share the same number, a RuntimeWarning is triggered. In this case, sorting cannot distinguish the main view from observer windows, and window titles and handles should be used to manually confirm.
• Resource Release: The current default is isNewUE=True, using d3dshot for capture; clearHWND() only releases DC and bitmap resources in the legacy Windows GDI capture path.

Changelog

• 2026-09-11: 🐛 fix: Fixed RflySim3D window recognition and sorting [P2]
• 2024-08-05: fix: Added HTML version API comments
• 2024-07-17: fix: Updated VisionCaptureApi interface API
• 2023-10-23: feat: Add all Python common labs