Ahsoka.Extensions.VideoPlayer 5.0.1-zzz-develop.57

OpenPV - Ahsoka.Extensions.VideoPlayer

Overview

The Ahsoka.Extensions.VideoPlayer is a service-based C# wrapper around GStreamer functionality, providing a high-level abstraction for video playback operations.

Architecture

System Components

The VideoPlayer system consists of several key components:

1. Video Service Layer - C# service providing high-level video control APIs
2. Platform Implementation Layer - Platform-specific video implementations
3. GStreamer Pipeline Builder - Dynamic GStreamer pipeline generation
4. Native C Video Player - Low-level GStreamer wrapper executable
5. Configuration System - Dynamic player configuration and management

Component Hierarchy

VideoService (C#)
├── VideoPlayerImpl (Base Implementation)
├── Platform Implementations
│   ├── LinuxVideoServiceImplementation (In-process GStreamer)
│   ├── LinuxVideoOutOfProcServiceImplementation (Out-of-process executable)
│   └── DesktopVideoServiceImplementation (Development/Testing)
├── GStreamer Pipeline Builders
│   ├── FilePipeline (File playback)
│   ├── UsbCameraPipeline (USB camera input)
│   ├── CsiCameraPipeline (CSI camera input)
│   └── GmslCameraPipeline (GMSL camera input)
└── Native VideoPlayer (C executable)

Core Components

VideoService (VideoService.cs)

The main service class that handles video playback requests and manages multiple video player instances.

Key Features:

• Multi-player support with unique player IDs
• Platform-specific implementation selection
• Configuration-driven player instantiation
• Message-based communication protocol

Supported Platforms:

• OpenViewLinux - Standard ARM32 embedded Linux
• OpenViewLinuxPro - ARM64 embedded linux
• Windows64 / Ubuntu64 - Development platforms

VideoPlayerImpl (VideoPlayerImpl.cs:8)

Base implementation class providing common video player functionality including state management, playback control, and status reporting.

Core Methods:

• PlayVideo() - Start video playback
• PauseVideo() - Pause current playback
• ResumeVideo() - Resume paused playback
• StopVideo() - Stop playback and cleanup
• PositionVideo() - Seek to specific position
• GetStatus() - Query current playback status

State Management:

• Thread-safe operations using lock synchronization
• Automatic status update timer system
• Callback-based status notifications

Platform Implementations

LinuxVideoServiceImplementation (LinuxVideoServiceImplementation.cs:7)

In-process GStreamer implementation for direct video playback on embedded Linux systems.

Features:

• Direct GStreamer API integration
• Wayland display system support
• Real-time video status monitoring
• Bus message handling for pipeline events

GStreamer Integration:

• Uses GStreamer C# bindings
• Supports fullscreen and windowed playback
• Automatic pipeline state management
• Error handling and recovery

LinuxVideoOutOfProcServiceImplementation (LinuxVideoOutOfProcServiceImplementation.cs)

Out-of-process implementation that launches separate video player executables for enhanced system stability.

Benefits:

• Process isolation for crash protection
• Reduced memory footprint in main application
• Independent video player lifecycle
• Inter-process communication via stdin/stdout

Communication Protocol:

• PLAY - Resume playback
• PAUSE - Pause playback
• QUIT - Exit video player
• POSITION:<ms> - Seek to position
• NOTIFY:<ms> - Set status update interval

GStreamer Pipeline System

GstreamerPipelineBuilder (GstreamerPipelineBuilder.cs)

Factory class for creating appropriate GStreamer pipelines based on input type.

Supported Input Types:

• File - Local file playback
• Usb - USB camera streaming
• Csi - CSI camera interface
• Gmsl - GMSL camera interface

Pipeline Implementations

FilePipeline (FilePipeline.cs:4)

Handles local file playback with platform-specific optimizations.

OpenViewLinux Pipeline:

filesrc location=<file> ! decodebin ! videoconvert ! videoscale ! waylandsink fullscreen=true

OpenViewLinuxPro Pipeline:

playbin uri=file://<file> video-sink=waylandsink

CsiCameraPipeline (CsiCameraPipeline.cs:4)

CSI camera interface pipeline (OpenViewLinuxPro only).

Pipeline Configuration:

v4l2src device=<device> ! video/x-raw, format=YUY2, framerate=15/1 ! waylandsink

Native Video Player (Ahsoka.VideoPlayer.c:1)

C-based GStreamer wrapper providing low-level video control capabilities.

Key Features:

• Command-line GStreamer pipeline execution
• Real-time keyboard/stdin command processing
• Position and duration reporting
• Comprehensive message bus handling

Supported Commands:

• PLAY / PAUSE - Playback control
• POSITION:<ms> - Seek operations
• NOTIFY:<ms> - Status update timing
• INFO - Current status query
• QUIT - Application termination

Status Reporting:

• POSITION: HH:MM:SS.mmm - Current playback position
• DURATION: HH:MM:SS.mmm - Total media duration
• STATE: PLAYING/PAUSED - Current playback state
• END - Playback completion notification

Embedded Linux Integration

Display System Integration

Wayland Support:

• Native Wayland compositor integration
• Hardware-accelerated video overlay
• Multi-surface video rendering
• Surface ID management for multi-display

Environment Variables:

• WAYLANDSINK_SURFACE_ID - Target display surface
• Display selection via environment configuration

Hardware Acceleration

The system leverages hardware video acceleration available on embedded platforms:

• Decode Acceleration - Hardware video decoders
• Scale Acceleration - GPU-based video scaling
• Color Conversion - Hardware colorspace conversion
• Overlay Support - Direct video overlay to framebuffer

Configuration System

VideoPlayer Configuration

Configuration is managed through protobuf-serialized configuration files located in the application directory.

Configuration Structure:

• Player ID management
• Input type specification
• Display surface assignment
• Platform-specific settings

Message Protocol

Service Messages (VideoServiceMessages.cs)

The system uses a structured message protocol for video operations:

Request Types:

• GetVideoPlayers - Query available players
• PlayVideo - Start video playback
• StopVideo - Stop current playback
• PauseVideo - Pause playback
• ResumeVideo - Resume playback
• PositionVideo - Seek to position
• RequestStatus - Query current status

Response Types:

• VideoPlayerInfo - Player information
• VideoStatus - Current playback status

Notifications:

• StatusUpdated - Automatic status updates

Message Flow

Client Request → VideoService → VideoPlayerImpl → Platform Implementation → GStreamer
                                                                            ↓
Client Notification ← VideoService ← Status Updates ← Platform Implementation ← GStreamer Events

Build System Integration

CMake Configuration

The native components integrate with the broader CMake build system:

Build Targets:

• Ahsoka.VideoPlayer - Native video player executable
• Platform-specific output directories
• Automatic executable permissions setup

Cross-Compilation Support:

• ARM cross-compilation toolchains
• Platform-specific SDK integration
• Target-specific optimization flags

Dependencies

Required Libraries:

• GStreamer 1.x (core, base, plugins)
• GLib 2.x (main loop and utilities)
• Wayland client libraries
• Video4Linux2 (camera support)

Optional Dependencies:

• Hardware-specific acceleration libraries
• Platform codec libraries
• Advanced camera interface drivers

Usage Examples

Basic File Playback

// Create video playback request
var request = new VideoPlaybackRequest
{
    VideoPlayerID = 1,
    VideoURL = "/path/to/video.mp4",
    PlayFullScreen = true,
    StatusUpdateInMs = 1000
};

// Send play request
var response = videoService.PlayVideo(request);

Camera Stream Display

// CSI camera configuration
var cameraRequest = new VideoPlaybackRequest  
{
    VideoPlayerID = 2,
    VideoURL = "/dev/video0", // V4L2 device
    InputType = InputType.Csi,
    VideoWidth = 1920,
    VideoHeight = 1080
};

var response = videoService.PlayVideo(cameraRequest);

Multi-Player Management

// Query available players
var playerInfo = videoService.GetVideoPlayers();

// Control specific player
videoService.PauseVideo(new VideoPlayer { VideoPlayerID = 1 });
videoService.ResumeVideo(new VideoPlayer { VideoPlayerID = 1 });

Performance Considerations

Memory Management

• Process Isolation - Out-of-process implementation reduces main application memory usage
• Resource Cleanup - Automatic cleanup of GStreamer resources
• Buffer Management - Efficient video buffer handling

Real-Time Performance

• Hardware Acceleration - Leverage embedded GPU capabilities
• Direct Memory Access - Zero-copy video path where possible
• Priority Scheduling - Appropriate thread priorities for video tasks

System Integration

• Power Management - Coordinate with system power states
• Thermal Management - Monitor system thermal conditions
• Resource Arbitration - Coordinate camera and display resource usage

Troubleshooting

Common Issues

Video Player Executable Not Found:

• Verify Ahsoka.VideoPlayer.<Platform> exists in AhsokaLib directory
• Check executable permissions (automatically set by system)

Wayland Display Issues:

• Verify WAYLANDSINK_SURFACE_ID environment variable
• Ensure Wayland compositor is running
• Check display permissions

GStreamer Pipeline Errors:

• Review GStreamer plugin availability
• Verify codec support for media format
• Check hardware acceleration availability

Debug Information

The system provides comprehensive logging at multiple verbosity levels:

• High - Critical errors and state changes
• Medium - Pipeline setup and configuration
• Low - Detailed message bus events

Logging Categories:

• Pipeline state changes
• Buffer status and timing
• Hardware acceleration status
• Display system integration

For more information about developing in OpenPV Click Here.