Skip to content

Basic Usage

This guide covers the fundamental operations and concepts for using Quiver effectively.

Core Concepts

Quiver is a vector database that allows you to:

  • Store and index high-dimensional vectors
  • Perform similarity searches
  • Attach and query metadata
  • Scale efficiently with your data

Basic Operations

1. Creating an Index

config := quiver.Config{
    Dimension: 384,  // Vector dimension
    StoragePath: "path/to/store",
    Distance: quiver.Cosine,  // or quiver.L2
}

idx, err := quiver.New(config, logger)
if err != nil {
    // Handle error
}
defer idx.Close()

2. Adding Vectors

vector := []float32{0.1, 0.2, 0.3, ...}  // Your vector data
metadata := map[string]interface{}{
    "category": "product",
    "name": "example"
}

err = idx.Add(1, vector, metadata)
if err != nil {
    // Handle error
}

3. Searching Vectors

queryVector := []float32{0.1, 0.2, 0.3, ...}
results, err := idx.Search(queryVector, 10, 1, 10)  // k=10, page=1, pageSize=10
if err != nil {
    // Handle error
}

Working with Metadata

Adding Metadata

metadata := map[string]interface{}{
    "category": "electronics",
    "price": 299.99,
    "tags": []string{"laptop", "computer"},
}

Querying with Metadata

results, err := idx.SearchWithFilter(
    queryVector,
    10,
    "SELECT * FROM metadata WHERE json->>'category' = 'electronics'"
)

Best Practices

  1. Vector Normalization
  2. Normalize vectors before adding them to the index

  3. Use consistent normalization methods

  4. Batch Operations

  5. Use batch operations for better performance when adding multiple vectors

  6. Configure appropriate batch sizes

  7. Resource Management

  8. Always close the index when done

  9. Monitor memory usage for large datasets

  10. Error Handling

  11. Implement proper error handling
  12. Use logging for debugging

Common Configurations

Performance Tuning

config := quiver.Config{
    HNSWM: 32,            // Number of connections per layer
    HNSWEfConstruct: 200, // Search width during construction
    HNSWEfSearch: 100,    // Search width during querying
    BatchSize: 1000,      // Size of batches for insertion
}

Persistence Settings

config := quiver.Config{
    PersistInterval: 5 * time.Minute,
    BackupInterval: 1 * time.Hour,
    BackupCompression: true,
}

Next Steps

After mastering these basics, you can explore: