Architecture¶

This page describes the architecture of Blink, including its components and how they interact.

Overview¶

Blink is designed with a modular architecture that separates concerns and allows for flexibility in how it's used. The main components are:

1. Watcher: Monitors directories for changes with improved performance
2. Event Batcher: Batches events to reduce processing overhead
3. Event Filter: Filters events based on patterns and types
4. EventStreamer Interface: Defines the contract for event streaming implementations
5. SSE Streamer: Implements event streaming via Server-Sent Events (SSE)
6. WebSocket Streamer: Implements event streaming via WebSockets
7. Multi Streamer: Combines multiple streaming methods
8. Event Server: Coordinates event delivery through streamers
9. Webhook Manager: Sends webhooks when events occur
10. CLI: Provides a command-line interface

Component Diagram¶

+------------------+     +------------------+
|   File System    | --> |     Watcher      |
+------------------+     +------------------+
                              |
                              v
+------------------+     +------------------+
|  Event Batcher   | --> |   Event Filter   |
+------------------+     +------------------+
                              |
                              v
                        +------------------+
                        |   Event Server   |
                        +------------------+
                              |
                              v
          +------------------+------------------+------------------+
          |                  |                  |                  |
          v                  v                  v                  v
+------------------+ +------------------+ +------------------+ +------------------+
|  EventStreamer   | |   SSEStreamer    | | WebSocketStreamer| |  MultiStreamer  |
|    Interface     | |  Implementation  | |  Implementation  | |  Implementation |
+------------------+ +------------------+ +------------------+ +------------------+
          |                  |                  |                  |
          v                  v                  v                  v
+------------------+ +------------------+ +------------------+ +------------------+
|    Clients       | |    SSE Clients   | | WebSocket Clients| |  Mixed Clients  |
+------------------+ +------------------+ +------------------+ +------------------+

+------------------+     +------------------+
| Webhook Manager  |     |       CLI        |
+------------------+     +------------------+

Components¶

Watcher¶

The Watcher is responsible for watching directories for changes with improved performance and reliability. It uses the fsnotify package to receive file system events and provides them to other components.

Key features:

• Event batching for improved performance
• Separate file and directory event handling
• Configurable batching delay
• Periodic polling for new files
• Recursive directory watching
• Symbolic link support

Event Batcher¶

The EventBatcher is responsible for batching file system events to reduce redundant processing. It groups related events together and sends them as a batch after a configurable delay.

Key features:

• Configurable batching delay
• Thread-safe event collection
• Efficient event delivery
• Reduced processing overhead

Event Filter¶

The EventFilter is responsible for filtering events based on file patterns and event types. It allows users to focus on specific files or event types.

Key features:

• File pattern filtering (include/exclude)
• Event type filtering (include/ignore)
• Efficient pattern matching
• Combined path and event type filtering

EventStreamer Interface¶

The EventStreamer interface defines the contract that all streaming implementations must follow. This abstraction allows for different streaming protocols to be used interchangeably.

Key methods:

• Start(ctx context.Context) error: Initializes and starts the streamer
• Stop() error: Gracefully shuts down the streamer
• Send(event fsnotify.Event) error: Delivers an event to all connected clients

SSE Streamer¶

The SSEStreamer implements the EventStreamer interface using Server-Sent Events (SSE). It provides a unidirectional stream of events to clients.

Key features:

• Standard SSE protocol implementation
• Automatic reconnection handling
• Event ID support
• Compatible with older browsers and systems
• Lightweight connection management

WebSocket Streamer¶

The WebSocketStreamer implements the EventStreamer interface using WebSockets. It provides a bidirectional communication channel between the server and clients.

Key features:

• Full-duplex communication
• Lower latency for high-frequency events
• Binary data support
• Connection pooling and management
• JSON-formatted event delivery

Multi Streamer¶

The MultiStreamer implements the EventStreamer interface by combining multiple streamers. It allows events to be broadcast through multiple protocols simultaneously.

Key features:

• Protocol-agnostic event broadcasting
• Parallel delivery to multiple streamers
• Graceful error handling
• Unified start/stop management

Event Server¶

The EventServer is responsible for coordinating event delivery through streamers. It integrates with the watcher and manages the lifecycle of streamers.

Key features:

• Configurable streaming method (SSE, WebSocket, or both)
• CORS support
• HTTP server management
• Integration with the watcher and filter components

Webhook Manager¶

The WebhookManager is responsible for sending webhooks when events occur. It allows integration with other systems.

Key features:

• HTTP webhook support
• Custom headers
• Retries and timeouts
• Event debouncing
• JSON payload formatting

CLI¶

The CLI provides a command-line interface for using Blink. It allows users to configure and run Blink from the command line.

Key features:

• Command-line flags
• Configuration management
• Subcommands
• Stream method selection

Data Flow¶

1. The Watcher receives file system events from the operating system.
2. Events are batched by the EventBatcher to reduce processing overhead.
3. Batched events are passed to the EventFilter to determine if they should be processed.
4. If events pass the filter, they're sent to the EventServer.
5. The EventServer determines which streaming method(s) to use based on configuration:
6. For SSE: Events are formatted and sent via the SSEStreamer to connected clients
7. For WebSocket: Events are converted to JSON and sent via the WebSocketStreamer to connected clients
8. For both: Events are sent through the MultiStreamer to both SSE and WebSocket clients
9. Simultaneously, events may be sent to the WebhookManager for delivery to configured webhook endpoints.

Stream Method Selection¶

Blink supports three streaming methods, controlled by the StreamMethod type:

1. StreamMethodSSE: Uses only Server-Sent Events (default)
2. StreamMethodWebSocket: Uses only WebSockets
3. StreamMethodBoth: Uses both SSE and WebSockets simultaneously

When using StreamMethodBoth, Blink serves:

• SSE events at the path specified by the configuration (default: /events)
• WebSocket events at the same path with /ws appended (default: /events/ws)

Configuration¶

Blink's configuration is managed by the viper package, which provides a flexible configuration system with support for:

• Command-line flags
• Environment variables
• Configuration files
• Default values

The configuration includes options for:

• Directories to watch
• Event filtering
• Streaming method selection
• Server settings
• Webhook configuration
• Performance tuning

Error Handling¶

Blink uses a combination of error logging and error channels to handle errors:

• The Watcher provides an error channel for receiving file system errors.
• Each EventStreamer implementation handles protocol-specific errors.
• The LogError function is used for logging errors.
• The FatalExit function is used for fatal errors that should terminate the program.

Performance Considerations¶

Blink is designed for high performance, with several optimizations:

• Event batching to reduce processing overhead
• Separate file and directory event handling
• Configurable batching delay for different workflows
• Periodic polling for new files
• Non-blocking channel operations
• Efficient memory usage with periodic cleanup
• Protocol-specific optimizations:
• SSE: Lightweight connections for many clients
• WebSocket: Efficient binary communication for high-frequency events