Lakshmi Sahithi | DIPr Lab at PSU

Fall 2025 Week 9

Wed, 26 Nov 2025 00:00:00 +0000

Title	SIEVE: Effective Filtered Vector Search with Collection of Indexes
Authors	Zhaoheng Li, et al.
Abstract	Real-world tasks such as recommending videos tagged kids can be reduced to finding similar vectors associated with hard predicates. This task, filtered vector search, is challenging as prior state-of-the-art graph-based (unfiltered) similarity search techniques degenerate when hard constraints are considered: effective graph-based filtered similarity search relies on sufficient connectivity for reaching similar items within a few hops. To consider predicates, recent works propose modifying graph traversal to visit only items that satisfy predicates. However, they fail to offer the just-a-few-hops property for a wide range of predicates: they must restrict predicates significantly or lose efficiency if only few items satisfy predicates. We propose an opposite approach: instead of constraining traversal, we build many indexes each serving different predicate forms. For effective construction, we devise a three-dimensional analytical model capturing relationships among index size, search time, and recall, with which we follow a workload-aware approach to pack as many useful indexes as possible into a collection. At query time, the analytical model is employed yet again to discern the one that offers the fastest search at a given recall. We show superior performance and support on datasets with varying selectivities and forms: our approach achieves up to 8.06 x speedup while having as low as 1% build time versus other indexes, with less than 2.15 x memory of a standard HNSW graph and modest knowledge of past workloads.

Summer 2025 Week 3

Wed, 06 Aug 2025 00:00:00 +0000

Title	HoneyBee: Efficient Role-based Access Control for Vector Databases via Dynamic Partitioning
Authors	Hongbin Zhong, Matthew Lentz, Nina Narodytska, Adriana Szekeres, Kexin Rong
Abstract	As vector databases gain traction in enterprise applications, robust access control has become critical to safeguard sensitive data. Access control in these systems is often implemented through hybrid vector queries, which combine nearest neighbor search on vector data with relational predicates based on user permissions. However, existing approaches face significant trade-offs: creating dedicated indexes for each user minimizes query latency but introduces excessive storage redundancy, while building a single index and applying access control after vector search reduces storage overhead but suffers from poor recall and increased query latency. This paper introduces HoneyBee, a dynamic partitioning framework that bridges the gap between these approaches by leveraging the structure of Role-Based Access Control (RBAC) policies. RBAC, widely adopted in enterprise settings, groups users into roles and assigns permissions to those roles, creating a natural "thin waist" in the permission structure that is ideal for partitioning decisions. Specifically, HoneyBee produces overlapping partitions where vectors can be strategically replicated across different partitions to reduce query latency while controlling storage overhead. By introducing analytical models for the performance and recall of the vector search, HoneyBee formulates the partitioning strategy as a constrained optimization problem to dynamically balance storage, query efficiency, and recall. Evaluations on RBAC workloads demonstrate that HoneyBee reduces storage redundancy compared to role partitioning and achieves up to 6x faster query speeds than row-level security (RLS) with only 1.4x storage increase, offering a practical middle ground for secure and efficient vector search.

Spring 2025 Week 3

Fri, 18 Apr 2025 00:00:00 +0000

Title	PDX: A Data Layout for Vector Similarity Search
Authors	Leonardo Kuffo, Elena Krippner, and Peter Boncz from CWI Amsterdam, The Netherlands
Abstract	We propose Partition Dimensions Across (PDX), a data layout for vectors (e.g., embeddings) that, similar to PAX, stores multiple vectors in one block, using a vertical layout for the dimensions (Figure 1). PDX accelerates exact and approximate similarity search thanks to its dimension-by-dimension search strategy that operates on multiple-vectors-at-a-time in tight loops. It beats SIMD-optimized distance kernels on standard horizontal vector storage (avg 40% faster), only relying on scalar code that gets auto-vectorized. We combined the PDX layout with recent dimension-pruning algorithms ADSampling and BSA that accelerate approximate vector search. We found that these algorithms on the horizontal vector layout can lose to SIMD-optimized linear scans, even if they are SIMD-optimized. However, when used on PDX, their benefit is restored to 2-7x. We find that search on PDX is especially fast if a limited number of dimensions has to be scanned fully, which is what the dimension-pruning approaches do. We finally introduce PDX-BOND, an even more flexible dimension-pruning strategy, with good performance on exact search and reasonable performance on approximate search. Unlike previous pruning algorithms, it can work on vector data "as-is" without preprocessing; making it attractive for vector databases with frequent updates.

Fine Grained Access Control in Vector Databases

Thu, 20 Feb 2025 00:00:00 +0000

Vector databases are particularly well-suited for similarity search using search algorithms like approximate nearest neighbor (ANN) search and they are used in development of Retrieval-Augmented Generation (RAG) systems, to reduce hallucinations in responses of AI systems. One significant challenge in using vector databases, especially in applications like RAG, is ensuring data privacy and security. For example, a clothing company that builds an AI chatbot that interacts with a vector database containing customer orders and product data could expose sensitive customer information without proper access restrictions. Incorporating Fine-Grained Access Control in vector databases is important for enforcing user preferences on data sharing and complying with privacy regulations. This project explores how to embed fine-grained access control within vector databases to ensure secure and privacy-compliant query answering.