Multi-tenancy

[bladehliu]

When building an LLM-based AI application, we need the right way to store user/tenant-specific vector data. And it is the common case that there are millions or billions of users while each user's data is not too much - say several document files, i.e., there are lots of spaces and most of them is small-sized.

Vearch needs a more efficient way to support millions of small spaces in one cluster.

[bladehliu]

design consideration

we can divide spaces by two kinds: public vs private. Public spaces are accessed by lots of users and usually partitioned for performance or capacity. A private space is owned and accessed exclusively by a single user so does not needed to be partitioned.

Note it is a bad idea that a private space corresponds to one raft replication state machine - too much overhead (heartbeats, goroutines, et al). And there are two solution candidates:

1, multiple private spaces are assigned to one replication group in the current shared-nothing architecture
2, separation of storage/compute, no replication at the compute layer, multiple spaces hosted by a server, #749

[bladehliu]

Goal:

1. separated data space per tenant
2. scale to millions of tenants in a cluster of hundreds of servers, with tens of thousands of active tenants per server
3. inactive tenants should be offloaded to release the RAM resource and will be loaded on demand

[CharlesJQuarra]

Goal:

1. separated data space per tenant

This is a common use case for RocksDB column families. They can be created and removed on demand and they avoid adding global indexing overheads to separate column families.

2. scale to millions of tenants in a cluster of hundreds of servers, with tens of thousands of active tenants per server

3. inactive tenants should be offloaded to release the RAM resource and will be loaded on demand

RocksDB should be able to manage this transparently during regular compaction