XamlMcp.Windows.Input 1.0.0-preview.1

XamlMcp

An MCP-based AI inspection toolkit for XAML UI frameworks — let an AI assistant inspect and drive a running app you build: walk the visual/logical tree, read and write properties, inspect styles and resources, take screenshots, synthesize input, invoke commands and automation patterns, and observe what changed — over an authenticated local transport.

Status: early preview. The current 1.0.0-preview.1 candidate is local and unpublished. Avalonia, modern .NET WPF, self-contained unpackaged WinUI 3, and a logical .NET MAUI agent for Windows and Android are implemented. Protocol version 1 remains stable. MAUI Android has emulator and fresh-package evidence. Its physical-device suite is an optional validation lane and is currently unrun. Native MAUI nodes and Apple targets are not supported.

See the framework feature and tool matrix for per-tool support, capability limits, and milestone evidence. Maintainers can build the unpublished candidate with the local preview release guide.

Packages

Install

Prerequisite: the app you inspect can target .NET 8 or later, but the xamlmcp tool (and building this repo or the sample from source) needs the .NET 10 SDK/runtime.

These packages are not currently published to NuGet. After producing an accepted local preview, add the agent from that run's packages directory:

dotnet add package XamlMcp.Avalonia --version 1.0.0-preview.1 --source <local-feed>
dotnet add package XamlMcp.Wpf --version 1.0.0-preview.1 --source <local-feed>
dotnet add package XamlMcp.WinUI --version 1.0.0-preview.1 --source <local-feed>
dotnet add package XamlMcp.Maui --version 1.0.0-preview.1 --source <local-feed>

The agent's floor is Avalonia 11.3.18. If your project pins older references (the current avalonia.app template pins 11.3.0) restore fails with a NU1605 package-downgrade error — bump your Avalonia.* package references to at least 11.3.18.

Install the MCP server from the same local feed:

dotnet tool install --global XamlMcp.Server --version 1.0.0-preview.1 --add-source <local-feed>

If your NuGet configuration uses package source mapping, map XamlMcp.* to the local feed in a run-specific NuGet.Config instead of relying on source flags.

Attach the Avalonia agent

Two entry points, both dev-gated — nothing listens unless you opt in per build (option B) or per launch (option A):

using XamlMcp.Avalonia;

// Option A - fluent, in BuildAvaloniaApp. Compiled in, but INERT unless the
// XAML_MCP=1 environment variable is set when the app launches.
public static AppBuilder BuildAvaloniaApp()
    => AppBuilder.Configure<App>()
        .UsePlatformDetect()
        .AttachXamlMcp();

// Option B - on the Application instance (e.g. in OnFrameworkInitializationCompleted).
// Marked [Conditional("DEBUG")]: the CALL SITE disappears from YOUR Release builds.
public override void OnFrameworkInitializationCompleted()
{
    this.AttachXamlMcp();
    base.OnFrameworkInitializationCompleted();
}

Why two shapes? [Conditional] requires a void method, so the fluent AppBuilder overload can't use it and gates on the environment variable instead. And why not a plain #if DEBUG inside the package? That would gate on how this package was compiled at pack time and could never see your app's configuration at all. Both overloads instead gate on your build configuration or your launch environment.

Know the difference: option B's call is gone from your Release binaries; option A's code ships in Release but stays inert unless XAML_MCP=1 is present at launch — anyone who controls the launch environment can enable it. If you need hard exclusion with the fluent shape, wrap the .AttachXamlMcp() line in your own #if DEBUG.

Attach the WPF agent

Call the extension on the WPF Application in OnStartup:

using System.Windows;
using XamlMcp.Wpf;

public partial class App : Application
{
    protected override void OnStartup(StartupEventArgs e)
    {
        this.AttachXamlMcp();
        base.OnStartup(e);
    }
}

The WPF method is [Conditional("DEBUG")], so the call site disappears from the consuming app's Release build. There is no environment-enabled WPF overload. M13 supports modern .NET WPF and the Application.Dispatcher that performs attachment. Windows and controls owned by secondary WPF UI threads are deliberately excluded.

Attach the WinUI agent

Attach on the WinUI UI thread, explicitly register each Window, and keep the calls inside the consuming application's development gate:

using Microsoft.UI.Xaml;
using XamlMcp.WinUI;

public partial class App : Application
{
    private WinUiXamlMcpSession? _xamlMcp;
    private IDisposable? _windowRegistration;

    protected override void OnLaunched(LaunchActivatedEventArgs args)
    {
        var window = new MainWindow();
#if DEBUG
        var session = WinUiXamlMcp.Attach();
        _xamlMcp = session;
        _windowRegistration = session.RegisterWindow(window);
        window.Closed += async (_, _) =>
        {
            _windowRegistration?.Dispose();
            await session.DisposeAsync();
        };
#endif
        window.Activate();
    }
}

Attach() is not conditionally compiled: the caller owns the #if DEBUG (or equivalent) development gate. M17 supports one DispatcherQueue per session and explicit windows on that queue. The supported floor is Windows App SDK 2.3.1, target framework net8.0-windows10.0.26100.0, Windows 10 1809 (10.0.17763.0), and win-x64.

WinUI development and publishing are unpackaged. Use WindowsPackageType=None, WindowsAppSDKSelfContained=true, and publish to a fresh directory:

dotnet publish MyWinUiApp.csproj -c Release -r win-x64 --self-contained true -o artifacts/winui

The publish output must retain the generated .xbf and .pri resources. MSIX/package identity, certificates, Store distribution, XAML Islands, and multiple UI threads are outside M17.

Attach the .NET MAUI agent

Call UseXamlMcp() while building the MAUI app, inside the consuming application's debug gate:

using XamlMcp.Maui;

public static MauiApp CreateMauiApp()
{
    var builder = MauiApp.CreateBuilder()
        .UseMauiApp<App>();

#if DEBUG
    builder.UseXamlMcp();
#endif

    return builder.Build();
}

The same package and UseXamlMcp() call work in unpackaged Windows applications and debuggable Android applications on MAUI 10.0.80. The package does not inspect the consumer's configuration, so the caller must own the #if DEBUG or equivalent development gate. Android projects must set SupportedOSPlatformVersion to 26.0 or later; the package's build-transitive target rejects a lower floor.

On Windows, the agent writes a discovery file under %LOCALAPPDATA%/XamlMcp/instances/ (override the directory with XAML_MCP_DIR) and serves JSON-RPC 2.0 on a current-user named pipe. On Android, it listens only on device loopback and stores a per-launch descriptor in app-private storage. The server reads that descriptor through debuggable run-as access and creates an owned adb forward only while connecting. Both transports require the per-launch token before any inspection call.

To discover an Android app, pass its application ID to the MCP server. Select a device explicitly when more than one authorized device is online:

claude mcp add xamlmcp -- xamlmcp --android-package dev.example.app --android-device emulator-5554

The Windows development prerequisites are the .NET 10 SDK and MAUI Windows workload. Android also requires the .NET Android workload, Android SDK platform tools (adb), and an API 26-or-later emulator or device. iOS and Mac Catalyst are not implemented. Building or testing those targets requires macOS and Xcode in addition to a future XamlMcp backend.

Connect an AI client

Register the server with your MCP client. For Claude Code:

claude mcp add xamlmcp -- xamlmcp

or in .mcp.json:

{
  "mcpServers": {
    "xamlmcp": { "command": "xamlmcp" }
  }
}

The flow: launch your app (with XAML_MCP=1 if you used option A), then in the AI client call list-apps → attach(instanceId) → any tool below. Integer PID attach remains a Windows desktop compatibility path.

Two things AI-client authors should know:

• Observation: every mutating tool (set-prop, pseudo-class, input, action) waits a settle window (default 250 ms) and returns a digest of what changed. For rapid sequences pass settleMs: 0 or observe: false.
• Snapshots: node ids are valid for the snapshot that issued them. tree, search, and any call requesting a non-"none" snapshot mint a new one and invalidate all earlier ids — a stale-snapshot error means re-query, not retry.

WPF capability notes

• tree, type/name/text search, ancestors, properties, resources, screenshots, routed input, actions, observation, and packaged assets are enabled and live-verified.
• WPF has no Avalonia-style class or pseudo-class collection. Style-class/pseudo-class search and pseudo-class mutation return unsupported-capability.
• styles is enabled but deliberately degraded: WPF exposes declared styles, setters, triggers, and value sources, not a complete applied selector cascade. Read degraded and degradationReason in the result.
• Routed input raises WPF events and reports mechanism routed; it does not claim physical mouse, focus, capture, or Mouse.DirectlyOver equivalence. Non-empty modifiers return unsupported-capability because routed event construction cannot inject modifier state. input-raw is false because the public InputManager.ProcessInput proof failed those requirements too.
• WPF screenshots render through RenderTargetBitmap. Separate popups must be captured by their own tree ref; native child HWND and GPU/airspace content are outside that render.
• assets enumerates SDK-generated *.g.resources and opens only identifiers it issued. This is a compiler convention—WPF has no public package-resource enumeration API—and ordinary copied Content files are not advertised.

WinUI capability notes

• Visual tree, type/name/text search, ancestors, resources, screenshots, semantic actions, command slots, unpackaged assets, and bounded observation are enabled and live-verified.
• tree.logical=false and props.complete=false. Property enumeration uses the documented known dependency-property catalog; reads and writes outside it return typed errors rather than a completeness claim.
• styles is enabled but degraded because WinUI does not expose a complete applied selector cascade. Screenshots use RenderTargetBitmap; native HWND, airspace/GPU content, and separate top-level surfaces retain the documented rendering limits.
• input is enabled only for mechanism: "raw"; routed injection returns unsupported-capability. Raw click, move, wheel, key, and type use guarded SendInput: the registered owner must be the exact foreground root, pointer coordinates must still resolve to that HWND, and an explicit keyboard target must accept focus. These checks and injection are not atomic, so the residual foreground/z-order race makes this a development-only capability.
• Pseudo-class mutation and style-class/pseudo-class search remain disabled. Visual states are not presented as pseudo-classes.
• Assets are rooted in the unpackaged application directory, use opaque xamlmcp-asset:/// identifiers, enforce containment and byte limits, and reject traversal and reparse-point paths.
• Observation is a bounded sampled diff with Window/Popup journaling. Requesting a scoped snapshot mints a fresh snapshot and invalidates older node refs, as on the other desktop agents.

.NET MAUI capability notes

• tree must use scope: "logical". Node refs identify MAUI IVisualTreeElement objects only; native handler PlatformView objects are never returned.
• Type/name/text/style-class search, ancestors, bindable properties, resources, degraded styles, screenshots, platform actions, packaged assets, and bounded observation are live-verified on Windows and an Android emulator.
• Property enumeration uses public static BindableProperty fields and is incomplete by design. Scalar, enum, color, thickness, rectangle, and point writes are supported where the target type permits them; complex object writes return typed errors.
• Windows input uses OS-level raw injection and reports raw; it does not advertise routed input. Android dispatches touch, wheel, key, and text through the activity/view stack and reports routed; it never advertises raw input.
• Action patterns are node- and platform-dependent. Public ICommand slots remain available on both platforms. Windows maps WinUI automation providers; Android maps accessibility actions. Android advertises only actions with a verifiable node-local postcondition; generic click/invoke and absolute-percentage scroll are deliberately rejected. Read props.patterns for the exact accepted verbs on one node.
• styles is deliberately degraded. MAUI visual states are reported as style frames, not protocol pseudo-classes. Pseudo-class mutation/search and native tree scope return unsupported-capability.
• Screenshots use WinUI rendering on Windows and PixelCopy with an ordinary-view Canvas fallback on Android. Native/airspace content and unattached handlers retain platform limits.
• MauiAsset items are exposed through opaque maui-asset:/// identifiers generated by the package's manifest target. Only identifiers issued by assets can be opened.
• Observation uses bounded logical fingerprints plus platform lifecycle journals. A scoped snapshot mints new refs; optional pixel hashes follow the platform screenshot limits.
• Android has emulator and package-consumer evidence. Its physical-device suite remains optional and unrun; iOS and Mac Catalyst advertise nothing.

Native dialogs & window management (Windows, opt-in)

Native file pickers and message boxes are separate Win32 windows — invisible to tree and unreachable by input. The driver covers them, off by default and Windows-only:

claude mcp add xamlmcp -- xamlmcp --enable-driver

The choreography: click the button that opens the dialog (input/action), then dialog-wait returns a dialog ref plus its structure (button automation ids, whether a file-name edit exists — never control values), then dialog-act drives it semantically. window-list/window-act manage the app's own top-level windows (close requires confirm: true).

Scope and safety:

• Only dialogs owned by the attached app's process are ever matched, and only ones that appeared after your last mutating call — pre-existing windows are never touched.
• set-file-name paths are validated against --driver-file-roots <p1;p2;…> (default: your user profile); anything outside is a typed path-not-allowed error.
• Elevated (UAC) apps and non-interactive sessions are unsupported and report typed errors.
• Dialog refs go stale on detach, dialog close, or app exit — re-run dialog-wait.

Security model

• Dev-gated by default — each framework section shows the caller-owned build or launch gate; nothing listens unless the application opts in.
• Token-first handshake. Every agent transport must authenticate with its per-launch token before any inspection method is dispatched. Unauthenticated calls get a typed error and the connection closes.
• Desktop discovery and pipes. Windows named pipes use PipeOptions.CurrentUserOnly. Discovery files are written owner-only on Unix (0700 directory, 0600 file) and deleted on graceful shutdown; the server prunes entries whose PID is dead or reused.
• Android private discovery. The descriptor stays in app-private storage and is readable by the development server only through debuggable run-as. The agent binds device loopback, and the server creates and removes the exact ADB forward it owns. Tokens, private paths, device ports, host ports, and raw ADB output never enter MCP results or errors.
• Tokens and desktop pipe names never appear in MCP output or error messages.
• One authenticated session at a time; later clients queue at the OS until the current session ends.

Sample

samples/SampleApp is a small Avalonia app wired with .AttachXamlMcp() and something for every tool to touch: a menu bar plus tabbed pages covering selection, tree, toggle, text, picker, and scroll/virtualization controls — the same pages the control-interaction matrix suite drives end-to-end. Run it and drive it:

XAML_MCP=1 dotnet run --project samples/SampleApp

samples/MauiSampleApp is the Windows/Android MAUI fixture. It is debug-gated and runs unpackaged on Windows:

dotnet run --project samples/MauiSampleApp -f net10.0-windows10.0.19041.0

Build its Android target with the installed SDK tooling:

dotnet build samples/MauiSampleApp/MauiSampleApp.csproj -f net10.0-android -c Debug

(PowerShell: $env:XAML_MCP="1"; dotnet run --project samples/SampleApp.)

samples/WpfSampleApp is the WPF playground. Its Debug build attaches automatically:

dotnet run --project samples/WpfSampleApp

samples/WinUiSampleApp is the self-contained, unpackaged WinUI control lab. It exposes stable names for inspection, styles and resources, every supported action pattern, observation digests, screenshots, popup and secondary-window roots, text and binary assets, and a native file dialog for Driver. Agent attachment remains inside #if DEBUG.

dotnet run --project samples/WinUiSampleApp -c Debug -p:Platform=x64

For unattended inspection, pass the sample's off-screen launch switch:

dotnet run --project samples/WinUiSampleApp -c Debug -p:Platform=x64 -- --hidden

Run its real-process MCP coverage with:

dotnet test tests/XamlMcp.WinUI.Tests/XamlMcp.WinUI.Tests.csproj -c Debug --filter FullyQualifiedName~LiveWinUiSampleTests

The interactive Driver fact is opt-in because it opens a native desktop dialog:

$env:XAMLMCP_DESKTOP_TESTS = "1"
dotnet test tests/XamlMcp.WinUI.Tests/XamlMcp.WinUI.Tests.csproj -c Debug --filter FullyQualifiedName~WinUiDriverLiveTests

The guarded raw-input proof is also desktop opt-in because it moves the real pointer and requires the test host to receive foreground ownership:

$env:XAMLMCP_DESKTOP_TESTS = "1"
dotnet test tests/XamlMcp.WinUI.Tests/XamlMcp.WinUI.Tests.csproj -c Debug --filter FullyQualifiedName~LiveWinUiInputTests

To verify the deployable unpackaged output, publish Release to a fresh directory:

$publishDir = "artifacts/winui-sample/$([Guid]::NewGuid().ToString('N'))"
dotnet publish samples/WinUiSampleApp/WinUiSampleApp.csproj -c Release -r win-x64 `
  --self-contained true -p:Platform=x64 -p:WindowsPackageType=None -o $publishDir

The output must contain WinUiSampleApp.exe, WinUiSampleApp.pri, the generated .xbf files, and both Assets/fixture.txt and Assets/fixture.bin.

Building from source

dotnet build                                 # XamlMcp.slnx
dotnet test                                  # suite on Avalonia 11.3.x
dotnet test -p:AvaloniaTestVersion=12.1.0    # same suite on the 12.x line

The solution includes net10.0-android, so a full restore/build requires the .NET Android workload. Device tests remain opt-in: set XAMLMCP_ANDROID_LIVE=1 for the emulator lane and also set XAMLMCP_ANDROID_PHYSICAL=1 for the physical-device lane. Set XAMLMCP_ANDROID_DEVICE when ADB reports more than one authorized device.

License

Apache-2.0 — see LICENSE.