ApiFeatures.FileWatchers.Apis 9.0.3

ApiFeatures.FileWatchers.Apis

I. Overview

ApiFeatures.FileWatchers.Apis is a .NET 9 library for managing file watchers that monitor file system changes and notify consumer modules when files are created. The library provides a complete API for creating, updating, and managing file watchers with MongoDB persistence and support for custom business logic.

Key Features

• Automatic File Monitoring: Watch multiple directories for file creation events
• Event-Driven Architecture: Notify consumer modules when files are created using IModuleEvent handlers
• MongoDB Persistence: Store file watcher configurations in MongoDB
• RESTful API Endpoints: Full CRUD operations for file watcher management
• Custom Business Logic: Optional validation hooks for system path access control
• Real-time Status: Check which file watchers are currently running
• FluentValidation: Built-in validation for all API requests
• Keyed Services: Uses .NET's keyed service pattern for flexible dependency injection

Use Cases

• Monitor upload directories for new measurement files
• Watch configuration folders for changes
• Trigger processing pipelines when new data arrives
• Implement file-based integrations
• Audit file system activity

---

II. Installation

Step 1: Add Package Reference

Add the ApiFeatures.FileWatchers.Apis package to your project:

<ItemGroup>
  <PackageReference Include="ApiFeatures.FileWatchers.Apis" Version="*" />
</ItemGroup>

Step 2: Register Services

In your Program.cs, configure and register the FileWatcher module:

using System.IO.Abstractions;
using ApiFeatures.Commons.Providers;
using ApiFeatures.FileWatchers.Apis.Extensions;
using ApiFeatures.FileWatchers.Apis.Models;
using ApiFeatures.FileWatchers.Apis.Models.Options;

var builder = WebApplication.CreateBuilder(args);

// Configure FileWatcher options
var options = new AddFileWatcherFeatureOptions();

// Required: Configure MongoDB database
options.WithMongoDatabase(new MongoFileWatcherDatabaseOptions
{
    ConnectionString = "mongodb://localhost:27017",
    DatabaseName = "file_watcher_db"
});

// Required: Configure file system
options.WithFileSystem<FileSystem>();
// Or with factory:
// options.WithFileSystem(sp => new FileSystem());

// Required: Configure date/time context
options.WithDateTimeContext<DateTimeContext>();
// Or with factory:
// options.WithDateTimeContext(sp => new DateTimeContext());

// Optional: Register custom business logic
options.WithBusinessLogic(sp => sp.GetRequiredService<MyFileWatcherBusinessLogic>());
// Or with generic:
// options.WithBusinessLogic<MyFileWatcherBusinessLogic>();

// Optional: Register event handlers for file created events
options.WithEventHandler<FileCreatedModuleEvent, MyFileCreatedHandler>();
// Or with factory:
// options.WithEventHandler<FileCreatedModuleEvent, MyFileCreatedHandler>((sp) => new MyFileCreatedHandler(sp));

// Register the module
builder.Services.AddFileWatchersModule(options);

var app = builder.Build();

// Map endpoints (optional authorization)
app.AddFileWatcherEndpoints(new AddFileWatcherEndpointsOptions
{
    AuthorizationPolicyHandler = () => new AuthorizationPolicyBuilder()
        .RequireAuthenticatedUser()
        .Build()
});

app.Run();

Step 3: Implement Custom Event Handler (Optional)

Create a handler to process file creation events:

using ApiFeatures.FileWatchers.Apis.Models.Events;
using ApiFeatures.Modules.Cores.Interfaces;

public class MyFileCreatedHandler : IModuleEventHandler<FileCreatedModuleEvent>
{
    private readonly ILogger<MyFileCreatedHandler> _logger;

    public MyFileCreatedHandler(ILogger<MyFileCreatedHandler> logger)
    {
        _logger = logger;
    }

    public async Task HandleAsync(FileCreatedModuleEvent moduleEvent, CancellationToken cancellationToken = default)
    {
        _logger.LogInformation("File created: {FullPath}", moduleEvent.FullPath);
        
        // Your custom logic here:
        // - Parse the file
        // - Process the data
        // - Store in database
        // - Trigger workflows
        // etc.
        
        await Task.CompletedTask;
    }
}

Step 4: Implement Business Logic (Optional)

Create custom validation for system path access:

using ApiFeatures.FileWatchers.Apis.Services.Interfaces;
using ApiFeatures.Modules.Cores.Models;
using Microsoft.AspNetCore.Http;

public class MyFileWatcherBusinessLogic : IFileWatcherBusinessLogic
{
    public async Task ValidateSystemPathAsync(
        HttpContext httpContext, 
        SystemPath systemPath, 
        CancellationToken cancellationToken = default)
    {
        // Validate user has access to this path
        // Check against allowed directories
        // Verify permissions
        // Check quotas
        
        var allowedPaths = new[] { "/data", "/uploads", "/measurements" };
        
        if (!allowedPaths.Any(allowed => systemPath.Path.StartsWith(allowed)))
        {
            throw new UnauthorizedAccessException($"Access to path '{systemPath.Path}' is not allowed");
        }
        
        await Task.CompletedTask;
    }
}

---

III. Module Events

The FileWatchers module emits events that consumer modules can handle by registering custom event handlers.

FileCreatedModuleEvent

Triggered whenever a file is created in a monitored directory.

Namespace: ApiFeatures.FileWatchers.Apis.Models.Events

Event Properties:

Property	Type	Description
FullPath	string	The full absolute path of the created file

Example Event:

var fileCreatedEvent = new FileCreatedModuleEvent
{
    FullPath = "/data/measurements/sensor_data_2024-01-15.csv"
};

When Triggered:

• A file is created in any directory being monitored by an enabled file watcher
• The file watcher's FileSystemWatcher.Created event fires
• Filters: By default, monitors *.csv and *.txt files with subdirectories included

Handling the Event:

To handle this event, implement IModuleEventHandler<FileCreatedModuleEvent> and register it:

using ApiFeatures.FileWatchers.Apis.Models.Events;
using ApiFeatures.Modules.Cores.Interfaces;

public class FileCreatedHandler : IModuleEventHandler<FileCreatedModuleEvent>
{
    private readonly ILogger<FileCreatedHandler> _logger;
    private readonly IMyDataService _dataService;

    public FileCreatedHandler(
        ILogger<FileCreatedHandler> logger,
        IMyDataService dataService)
    {
        _logger = logger;
        _dataService = dataService;
    }

    public async Task HandleAsync(
        FileCreatedModuleEvent moduleEvent, 
        CancellationToken cancellationToken = default)
    {
        _logger.LogInformation("Processing new file: {FullPath}", moduleEvent.FullPath);

        try
        {
            // Example: Read and process the file
            var fileContent = await File.ReadAllTextAsync(moduleEvent.FullPath, cancellationToken);

            // Example: Parse and store the data
            await _dataService.ProcessFileAsync(moduleEvent.FullPath, fileContent, cancellationToken);

            _logger.LogInformation("Successfully processed file: {FullPath}", moduleEvent.FullPath);
        }
        catch (Exception ex)
        {
            _logger.LogError(ex, "Failed to process file: {FullPath}", moduleEvent.FullPath);
            throw;
        }
    }
}

Registration:

// In Program.cs or startup configuration
options.WithEventHandler<FileCreatedModuleEvent, FileCreatedHandler>();

// Register handler's dependencies
builder.Services.AddScoped<IMyDataService, MyDataService>();

Multiple Handlers:

You can register multiple handlers for the same event. They will all be invoked sequentially:

options
    .WithEventHandler<FileCreatedModuleEvent, LoggingHandler>()       // First: Log the event
    .WithEventHandler<FileCreatedModuleEvent, ValidationHandler>()    // Second: Validate the file
    .WithEventHandler<FileCreatedModuleEvent, ProcessingHandler>()    // Third: Process the data
    .WithEventHandler<FileCreatedModuleEvent, NotificationHandler>(); // Fourth: Send notifications

Important Notes:

• Handlers are invoked in the order they are registered
• Each handler runs in its own service scope
• If a handler throws an exception, it's logged but doesn't stop other handlers
• Handlers receive the FullPath of the created file and can perform any async operation

---

IV. Configuration Options

AddFileWatcherFeatureOptions Methods

WithMongoDatabase(MongoFileWatcherDatabaseOptions options) [REQUIRED]

Configures MongoDB database connection for storing file watcher configurations.

Parameters:

• ConnectionString (string, required): MongoDB connection string
• DatabaseName (string, required): Name of the database
• Collections (MongoFileWatcherDatabaseCollectionOptions, optional): Custom collection names

Example:

options.WithMongoDatabase(new MongoFileWatcherDatabaseOptions
{
    ConnectionString = "mongodb://localhost:27017",
    DatabaseName = "file_watcher_db",
    Collections = new MongoFileWatcherDatabaseCollectionOptions
    {
        FileWatchers = "CustomFileWatchersCollection" // Default: "FileWatchers"
    }
});

---

WithFileSystem<T>() or WithFileSystem(Func<IServiceProvider, IFileSystem>) [REQUIRED]

Registers the file system abstraction for directory/file operations.

Generic version:

options.WithFileSystem<FileSystem>();

Factory version:

options.WithFileSystem(sp => new FileSystem());
// Or for testing:
options.WithFileSystem(sp => new MockFileSystem());

Parameters:

• T: Type that implements IFileSystem (from System.IO.Abstractions package)
• factory: Factory function that returns an IFileSystem instance

Lifetime: Singleton (keyed with ServiceKeys.Default)

---

WithDateTimeContext<T>() or WithDateTimeContext(Func<IServiceProvider, IDateTimeContext>) [REQUIRED]

Registers the date/time context for timestamp operations.

Generic version:

options.WithDateTimeContext<DateTimeContext>();

Factory version:

options.WithDateTimeContext(sp => new DateTimeContext());
// Or custom implementation:
options.WithDateTimeContext(sp => new CustomDateTimeContext());

Parameters:

• T: Type that implements IDateTimeContext
• factory: Factory function that returns an IDateTimeContext instance

Lifetime: Scoped (keyed with ServiceKeys.Default)

---

WithBusinessLogic<T>() or WithBusinessLogic(Func<IServiceProvider, IFileWatcherBusinessLogic>) [OPTIONAL]

Registers custom business logic for validating file watcher operations.

Generic version:

options.WithBusinessLogic<MyFileWatcherBusinessLogic>();

Factory version:

options.WithBusinessLogic(sp => new MyFileWatcherBusinessLogic(
    sp.GetRequiredService<IAuthorizationService>()
));

Business Logic Interface:

public interface IFileWatcherBusinessLogic
{
    Task ValidateSystemPathAsync(
        HttpContext httpContext, 
        SystemPath systemPath, 
        CancellationToken cancellationToken = default);
}

When Used:

• Called in CreateAsync endpoint for each target path
• Called in UpdateAsync endpoint when targets are updated

Parameters:

• T: Type that implements IFileWatcherBusinessLogic
• factory: Factory function that returns an IFileWatcherBusinessLogic instance

Lifetime: Singleton (keyed with ServiceKeys.Default)

---

WithEventHandler<TEvent, TEventHandler>() or WithEventHandler<TEvent, TEventHandler>(Func<IServiceProvider, TEventHandler>) [OPTIONAL]

Registers custom event handlers for file system events.

Generic version:

options.WithEventHandler<FileCreatedModuleEvent, MyFileCreatedHandler>();

Factory version:

options.WithEventHandler<FileCreatedModuleEvent, MyFileCreatedHandler>(
    sp => new MyFileCreatedHandler(
        sp.GetRequiredService<ILogger<MyFileCreatedHandler>>(),
        sp.GetRequiredService<IMyService>()
    )
);

Event Type:

public class FileCreatedModuleEvent : IModuleEvent
{
    public string FullPath { get; set; }
}

Handler Interface:

public interface IModuleEventHandler<TEvent> where TEvent : IModuleEvent
{
    Task HandleAsync(TEvent moduleEvent, CancellationToken cancellationToken = default);
}

Parameters:

• TEvent: Event type that implements IModuleEvent
• TEventHandler: Handler type that implements IModuleEventHandler<TEvent>
• factory: Optional factory function for handler creation

Lifetime: Singleton (keyed with ServiceKeys.FileWatcherModule)

Note: Multiple event handlers can be registered by calling this method multiple times.

---

IV. API Endpoints

File Watchers Management

GET /api/file-watchers

Get file watchers with pagination.

Query Parameters:

• pageIndex (int, optional): Page index (default: 0)
• pageSize (int, optional): Page size (default: 10)

Response:

{
  "items": [
    {
      "id": "018d1234-5678-7abc-def0-123456789abc",
      "name": "Measurements Watcher",
      "description": "Monitors measurement uploads",
      "targets": [
        { "path": "/data/measurements", "type": "Absolute" }
      ],
      "enabled": true,
      "createdTime": "2024-01-15T10:30:00Z"
    }
  ],
  "totalRecords": 1
}

---

GET /api/file-watchers/status

Get the runtime status of file watchers.

Query Parameters:

• ids (Guid[], required): Array of file watcher IDs

Example: GET /api/file-watchers/status?ids=018d1234-5678-7abc-def0-123456789abc&ids=018d9876-5432-1fed-cba0-987654321fed

Response:

[
  {
    "id": "018d1234-5678-7abc-def0-123456789abc",
    "status": "Running"
  },
  {
    "id": "018d9876-5432-1fed-cba0-987654321fed",
    "status": "Unknown"
  }
]

---

POST /api/file-watcher

Create a new file watcher.

Request Body:

{
  "name": "Measurements Watcher",
  "description": "Monitors measurement uploads",
  "targets": [
    {
      "path": "/data/measurements",
      "type": "Absolute"
    },
    {
      "path": "../relative/path",
      "type": "Relative"
    }
  ],
  "enabled": true
}

Response:

{
  "id": "018d1234-5678-7abc-def0-123456789abc"
}

---

PUT /api/file-watcher/{id}

Update an existing file watcher.

Path Parameters:

• id (Guid, required): File watcher ID

Request Body:

{
  "name": {
    "value": "Updated Name",
    "hasModified": true
  },
  "description": {
    "value": "Updated description",
    "hasModified": true
  },
  "targets": {
    "value": [
      { "path": "/new/path", "type": "Absolute" }
    ],
    "hasModified": true
  },
  "enabled": {
    "value": false,
    "hasModified": true
  }
}

Note: Only send properties you want to update with hasModified: true.

---

POST /api/file-watcher/{id}/watch

Start watching a file watcher (enable monitoring).

Path Parameters:

• id (Guid, required): File watcher ID

Response: 200 OK

---

DELETE /api/file-watcher/{id}/watch

Stop watching a file watcher (disable monitoring).

Path Parameters:

• id (Guid, required): File watcher ID

Response: 200 OK

---

POST /api/file-watcher/file-created-event

Manually trigger a file created event (for testing/debugging).

Request Body:

{
  "measurementFile": "/full/path/to/file.csv"
}

Response: 200 OK

---

V. Advanced Configuration

Custom Business Logic

Implement IFileWatcherBusinessLogic to add custom validation:

public class MyFileWatcherBusinessLogic : IFileWatcherBusinessLogic
{
    private readonly IAuthorizationService _authService;
    private readonly ILogger _logger;

    public MyFileWatcherBusinessLogic(
        IAuthorizationService authService,
        ILogger<MyFileWatcherBusinessLogic> logger)
    {
        _authService = authService;
        _logger = logger;
    }

    public async Task ValidateSystemPathAsync(
        HttpContext httpContext,
        SystemPath systemPath,
        CancellationToken cancellationToken = default)
    {
        // Example: Check user permissions
        var user = httpContext.User;
        var result = await _authService.AuthorizeAsync(user, systemPath.Path, "FileWatcherAccess");
        
        if (!result.Succeeded)
        {
            throw new UnauthorizedAccessException($"User does not have access to path: {systemPath.Path}");
        }

        // Example: Validate path is within allowed directories
        var allowedBasePaths = new[] { "/data", "/uploads", "/measurements" };
        
        if (!allowedBasePaths.Any(basePath => systemPath.Path.StartsWith(basePath)))
        {
            throw new ArgumentException($"Path '{systemPath.Path}' is outside allowed directories");
        }

        _logger.LogInformation("System path validated: {Path}", systemPath.Path);
    }
}

// Register in Program.cs:
builder.Services.AddScoped<MyFileWatcherBusinessLogic>();
options.WithBusinessLogic(sp => sp.GetRequiredService<MyFileWatcherBusinessLogic>());

Multiple Event Handlers

You can register multiple event handlers for the same event:

options
    .WithEventHandler<FileCreatedModuleEvent, LoggingHandler>()
    .WithEventHandler<FileCreatedModuleEvent, ProcessingHandler>()
    .WithEventHandler<FileCreatedModuleEvent, NotificationHandler>();

All handlers will be invoked when a file is created.

---

VI. Configuration File Example

appsettings.json

{
  "FileWatcher": {
    "MongoDatabase": {
      "ConnectionString": "mongodb://localhost:27017",
      "DatabaseName": "file_watcher_db"
    }
  }
}

Program.cs with Configuration Binding

var mongoOptions = builder.Configuration
    .GetSection("FileWatcher:MongoDatabase")
    .Get<MongoFileWatcherDatabaseOptions>()!;

options.WithMongoDatabase(mongoOptions);

---

VII. Dependencies

Required Packages

• MongoDB.Driver (3.5.1+)
• FluentValidation (12.1.1+)
• System.IO.Abstractions (22.1.0+)
• IotVn.CoreFeatures (9.1.0+)
• ApiFeatures.Modules.Cores (9.0.1+)
• BusinessFeatures.Cores (latest)
• DataMagic.Abstractions (1.0.1+)

Framework

• .NET 9.0

---

VIII. Features

Automatic Watcher Lifecycle

• Auto-start on creation: When a file watcher is created with enabled: true, it automatically starts watching
• Auto-restart on update: When targets are updated on a running watcher, it stops, updates, and restarts
• Graceful shutdown: Watchers are disposed properly when stopped or updated

Thread-Safe Operations

• All directory watcher operations are thread-safe using lock statements
• Concurrent file creation events are handled safely

GUID Serialization

• Uses GuidRepresentation.CSharpLegacy for MongoDB compatibility
• Automatically configures GUID serialization on database context initialization

Validation

All API endpoints use FluentValidation:

• System path validation
• Unique file watcher names
• Required fields validation
• Path existence checks

---

IX. Error Handling

Common Exceptions

Exception	When Thrown
FileWatcherByIdNotFoundException	File watcher with specified ID not found
FileWatcherDuplicateException	File watcher with same name already exists
DirectoryDoesNotExistException	Target directory path does not exist
PathIsNotDirectoryException	Target path is a file, not a directory
ValidationException	Request validation failed

---

X. Service Keys

The module uses keyed services for dependency injection:

Service	Key	Lifetime
IValidator<*>	ServiceKeys.Default	Scoped
IFileSystem	ServiceKeys.Default	Singleton
IDateTimeContext	ServiceKeys.Default	Scoped
IFileWatcherBusinessLogic	ServiceKeys.Default	Singleton
FileWatcherDatabaseContext	ServiceKeys.Default	Singleton
IMongoCollection<FileWatcherEntity>	ServiceKeys.Default	Scoped
IModuleEventHandler<TEvent>	ServiceKeys.FileWatcherModule	Singleton

ServiceKeys Values:

• ServiceKeys.Default = "ApiFeatures.FileWatchers.Apis"
• ServiceKeys.FileWatcherModule = "ApiFeatures.FileWatchers.Module"

---

XI. License

[Your License Here]

---

XII. Contributing

[Your Contributing Guidelines Here]