Performance¶

Blink is designed for high performance, with several optimizations to ensure efficient file system monitoring even for large directory structures.

Performance Features¶

Watcher¶

Blink includes a high-performance watcher implementation that significantly improves performance and reliability:

• Event Batching: Groups related events together to reduce processing overhead
• Configurable Delay: Adjustable batching delay (default: 100ms) to optimize for different editors and workflows
• Separate File/Directory Handling: Specialized processing for file and directory events
• Periodic Polling: Automatically detects new files that might have been created outside the watch system

Parallel Directory Scanning¶

Blink uses worker pools to scan directories in parallel, which significantly improves performance when watching large directory structures:

• Uses a number of workers based on available CPU cores
• Distributes directory scanning work across multiple goroutines
• Caps the number of workers to avoid excessive resource usage

Event Debouncing¶

To reduce the number of events and avoid overwhelming consumers, Blink implements event debouncing:

• Groups multiple events for the same file within a short time window
• Only sends one event per file within the debounce duration
• Configurable debounce duration for webhooks
• Smart batching for editors like Vim that generate multiple events when saving files

Non-blocking Channel Operations¶

Blink uses non-blocking channel operations to prevent goroutines from getting stuck:

• Uses buffered channels to handle bursts of events
• Implements select statements with default cases to avoid blocking
• Logs dropped events when channels are full

Efficient Memory Usage¶

Blink is designed to be memory-efficient, with several features to minimize memory usage:

• Periodically cleans up old events to prevent memory leaks
• Uses a map to track recent events for debouncing
• Implements efficient data structures for event tracking

Benchmarks¶

Blink includes benchmarks to measure performance of critical functions:

# Run benchmarks
make benchmark

Benchmark results show excellent performance:

• ShouldIgnoreFile: ~30.57 ns/op, 0 B/op, 0 allocs/op
• RemoveOldEvents: ~121779 ns/op for 1000 events
• Subfolders: Fast directory scanning with optimized memory usage

Performance Tips¶

Optimizing Event Batching¶

You can optimize event batching for your specific use case:

# Set a custom batching delay (in milliseconds)
blink --batch-delay 200

CPU Usage¶

You can control CPU usage with the --max-procs flag:

# Limit to 4 CPUs
blink --max-procs 4

Memory Usage¶

To reduce memory usage, consider:

• Using event filtering to reduce the number of events
• Setting a shorter refresh duration to clean up old events more frequently
• Using webhooks with debouncing to reduce the number of HTTP requests

Large Directory Structures¶

When watching large directory structures:

• Use the --exclude flag to ignore directories you don't need to watch
• Consider watching specific subdirectories instead of the entire structure
• Use a longer refresh duration to reduce the frequency of event processing

High Event Rates¶

For high event rates:

• Use event filtering to focus on specific files or event types
• Implement debouncing for webhooks
• Consider using a custom consumer that can handle high event rates
• Adjust the event batching delay to balance responsiveness and efficiency