[HELP] How to merge multiple indexes into one ?

[byronz3d]

Hi,

I want to create several collections in my Zep server, and then when a prompt/query arrives, I want to combine several collections into one index and be able to query the combined index.
Is this even possible?

What I tried to do, unsuccessfully, is something like:

def load_zep(collection_name):
        vector_store = ZepVectorStore(
            api_url=zep_api_url, collection_name=collection_name
        )

        return StorageContext.from_defaults(vector_store=vector_store,persist_dir=f'./storage/{collection_name}')

                from llama_index import load_index_from_storage, StorageContext

                #load different collections 
                storage_context1 = load_zep('collection1')
                storage_context2 = load_zep('collection2')
                storage_context3 = load_zep('collection3')

                #some how combine the above 3 collections into one

                #tried various ways like combined = list(), but nothing worked

                #load index from above combined 
                index = load_index_from_storage(combined) 

The logic behind this is I have various user levels and based on the user I want to give them a different index. Higher level users gets more indexes, while low level user gets a single index to query. I don't want to create the same index over and over if I can combine them somehow.

Thanks in advance!

[danielchalef]

Search scores are cosine distance normalized to (0,1). As such, you may be able to accomplish this by combining results from separate searches over the different Collections, ordering by score, and selecting your top n results.

I'm not super familiar with LlamaIndex, but you should be able to extract the texts and scores from the separate result sets in order to achieve this.