HNSW¶

HNSW is a high-performance Go implementation of the Hierarchical Navigable Small World (HNSW) algorithm for approximate nearest neighbor search in Go. It provides efficient similarity search capabilities for high-dimensional vector data with excellent performance characteristics.

What is HNSW?¶

Hierarchical Navigable Small World (HNSW) is a graph-based algorithm for approximate nearest neighbor search. It creates a multi-layered graph structure that allows for fast and accurate similarity searches in high-dimensional spaces. The algorithm offers logarithmic complexity for search operations, making it suitable for large-scale applications.

Learn more about the Graph Structure and Search Algorithms that make HNSW so efficient.

Key Features¶

• High Performance: Optimized implementation with excellent search and insertion speeds
• Type Safety: Fully leverages Go's generics for type-safe operations
• Customizable: Configurable parameters to balance between search speed and accuracy
• Extensible: Modular design that allows for easy extensions and customizations
• Production Ready: Thoroughly tested and benchmarked for production use

Use Cases¶

HNSW is ideal for a wide range of applications that require similarity search:

• Recommendation Systems: Find similar products, articles, or content
• Image Search: Locate visually similar images
• Natural Language Processing: Semantic search and document similarity
• Anomaly Detection: Identify outliers in high-dimensional data
• Clustering: Group similar data points together

Check out our Examples for practical implementations of these use cases and Advanced Techniques for production deployments.

Getting Started¶

package main

import (
    "fmt"

    "github.com/TFMV/hnsw"
)

func main() {
    // Create a new HNSW graph
    graph := hnsw.NewGraph[int]()

    // Add some vectors
    graph.Add(hnsw.Node[int]{
        Key:   1,
        Value: []float32{0.1, 0.2, 0.3},
    })

    graph.Add(hnsw.Node[int]{
        Key:   2,
        Value: []float32{0.2, 0.3, 0.4},
    })

    // Search for similar vectors
    results, _ := graph.Search([]float32{0.15, 0.25, 0.35}, 2)

    for _, result := range results {
        fmt.Printf("Key: %d, Distance: %f\n", result.Key, result.Dist)
    }
}

For more detailed instructions, see our Quick Start Guide and Basic Usage documentation.

Core Library¶

The core library provides the fundamental functionality of HNSW:

• Graph Structure: Learn about the hierarchical structure of HNSW
• Distance Functions: Understand the different distance metrics available
• Search Algorithms: Explore the efficient search algorithms used
• Performance Tuning: Optimize HNSW for your specific use case
• Graph Analyzer: Evaluate and diagnose the quality of your HNSW graphs

Extensions¶

HNSW provides several extensions to enhance its functionality:

• Metadata Extension: Store and retrieve JSON metadata alongside vectors
• Faceted Search: Filter search results based on facets or attributes
• Parquet Extension: Persistent storage of HNSW graphs using Apache Parquet
• Hybrid Extension: Multi-strategy approach combining HNSW with complementary techniques
• Creating Extensions: Build your own custom extensions

See the Extensions Overview for more information.

Performance¶

HNSW is designed for high performance, with careful attention to memory usage and computational efficiency. Benchmarks show that it can handle millions of high-dimensional vectors with sub-millisecond query times.

Learn more about Performance Tuning to get the most out of HNSW.

License¶

HNSW is released under the MIT LICENSE.