A complete solution to enable reverse-touch/touch-back and floating-menu/annotation-tool functionality for Interactive Flat Panel Displays (IFPD) with Intel Architecture (IA) platforms.
This project enables touch input from an IFPD to be transferred to a connected computer, creating a seamless reverse touch experience. It consists of three main components:
- Touch Data Capture Service (Windows) - Captures touch input from the IFPD
- MCU Firmware (ESP32-S3) - Acts as a bridge, receiving touch data via UART and presenting it as USB HID
- Floating Menu Application (Windows) - Provides a user-friendly interface for camera management and screen annotation
Note: All three components are designed to work independently. You can use the Touch Data Capture Service standalone or integrate it with the Floating Menu Application based on your requirements.
A Windows service that captures raw HID touch input and forwards it to the ESP32 via serial connection.
Features:
- Raw HID touch event capture
- High-speed serial communication (3 Mbaud)
- Dynamic coordinate scaling
- Process filtering
- Comprehensive logging
ESP32-S3 firmware that bridges UART touch data to USB HID multi-touch digitizer.
Features:
- 10-finger multi-touch support
- USB HID Digitizer protocol
- Low-latency dual-task architecture
- Windows Precision Touchpad compatible
Supported Targets: ESP32-S3
A WPF application providing an edge-docked floating menu for camera management and screen annotation.
Features:
- Edge-docked collapsible UI
- Camera detection and preview (Aforge.NET)
- Signal source management
- Screen annotation integration
- Always-on-top interface
- IFPD Device: Interactive Flat Panel Display with touch support
- ESP32-S3: Development board with USB-OTG support
- Target Computer: Any Windows PC
- Cables: USB data cables, UART connection (TX/RX)
- Windows 11 (for IFPD and Target Computer)
- .NET 10 SDK (for Windows applications)
- ESP-IDF v5.x (for MCU firmware)
- Visual Studio 2022 or higher or Visual Studio Code (optional)
git clone https://github.com/<your-organization>/ifpd-touchback-floatingmenu.git
cd ifpd-touchback-floatingmenucd MCU
# Follow SETUP.md for ESP-IDF configuration
# Follow DEPLOYMENT.md for flashingcd Windows/GetTouchInfo
dotnet build
dotnet publish -c Release
cd ../FloatingMenu
dotnet build
dotnet publish -c Release- Flash the ESP32-S3 with the MCU firmware
- Connect ESP32-S3 UART to IFPD's serial port
- Connect ESP32-S3 USB to Target Computer
Option 1: Touch Back Only (Standalone)
If you only need touch-back functionality without the floating menu:
- Run the Touch Data Capture Service (TouchDataCaptureService.exe) directly on the IFPD
- Touch events will be forwarded to the Target Computer via ESP32-S3
Option 2: With Floating Menu Integration
For full functionality including camera management and screen annotation:
- Configure the COM port in the Floating Menu configuration file
- Launch the Floating Menu Application on the IFPD
- The application will automatically manage the Touch Data Capture Service
- Click on the signal source button to start touch forwarding
- Click the source button again to stop the touch service
Each component has detailed documentation in its respective directory:
- MCU README - ESP32-S3 firmware details
- MCU SETUP - Development environment setup
- MCU DEPLOYMENT - Flashing and deployment guide
- Touch Service README - Touch capture service details
- Floating Menu README - Floating menu application details
- Presentation Mode: Present from IFPD while controlling the connected laptop with touch
- Remote Control: Use IFPD as a large touch interface for a connected computer
- Collaborative Work: Share touch input across multiple devices
- Interactive Displays: Enable touch-back for IA-only IFPD systems
This project is licensed under the Apache License 2.0. See Apache-2.0.txt file for details