Show HN: BloomSearch – Keyword search with hierarchical bloom filters

Summary

BloomSearch Keyword search engine with hierarchical bloom filters for massive datasets BloomSearch provides extremely low memory usage and low cold-start searches through pluggable storage interfaces. Memory efficient : Bloom filters have constant size regardless of data volume : Bloom filters have constant size regardless of data volume Pluggable storage : DataStore and MetaStore interfaces for any backend (can be same or separate) : DataStore and MetaStore interfaces for any backend (can be same or separate) Fast filtering : Hierarchical pruning via partitions, minmax indexes, and bloom filters : Hierarchical pruning via partitions, minmax indexes, and bloom filters Flexible queries : Search by field , token , or field:token with AND/OR combinators : Search by , , or with AND/OR combinators Disaggregated storage and compute: Unbound ingest and query throughput Perfect for logs, JSON documents, and high-cardinality keyword search. Quick start go get github.com/danthegoodman1/bloomsearch // Initialize stores dataStore := NewFileSystemDataStore ( "./data" ) metaStore := dataStore // FileSystemDataStore also implements MetaStore // Create engine with default config engine := NewBloomSearchEngine ( DefaultBloomSearchEngineConfig (), metaStore , dataStore ) engine . Start () // Insert data asynchronously (no wait for flush) engine . IngestRows ( ctx , [] map [ string ] any {{ "level" : "error" , "message" : "database connection failed" , "service" : "auth" , }}, nil ) // Provide a `chan error` to wait for flush doneChan := make ( chan error ) engine . IngestRows ( ctx , [] map [ string ] any {{ "level" : "info" , "message" : "login successful" , "service" : "auth" , }}, doneChan ) if err := <- doneChan ; err != nil { log . Fatal ( err ) } // Collect the resulting rows that match resultChan := make ( chan map [ string ] any , 100 ) // If any of the workers error, they report it here errorChan := make ( chan error , 10 ) err := engine . Query ( ctx , // Query for rows whe...