standalone milvus with batch search query

[dumbPy]

I am trying to make a batch query on a standalone installation.
when calling with a batch of embeddings, it returns

'message': 'fail to search on QueryNode 1: worker(1) query failed: Assert "slice_nqs_prefix_sum_[num_slices_] == total_nq_"

I went though #25550 but couldn't figure out what needs to be done exactly to make this work. I also saw that milvus.yaml already has maxNQ=1000 set so couldn't figureout what needs to be changes to make batch query work.

in short, my embeddings is only 15k and I would like to make batch search on these embeddings on a standalone installation. since my dataset is so small, I dont want to use a cluster for this and so it has to be a single docker instance that should support batch queries.

Why?

it works fine with faiss for me (given small dataset and lack of scale) but I need it over an api hence trying out milvus (also why I am using aiohttp.request instead of python client).

Alternative Solution (slower)

a working workaround is to make individual requests and await them all with asyncio.gather but that takes about 35ms while faiss takes about 5ms, so I am hoping a batch request would be faster than 35ms.

[yhmo]

Which version of your milvus?

Do you have the full message of "slice_nqs_prefix_sum_[num_slices_] == total_nq_"?
A full message should be like this: "message":"fail to search on QueryNode 1: worker(1) query failed: Assert "slice_nqs_prefix_sum_[num_slices_] == total_nq_" at /go/src/github.com/milvus-io/milvus/internal/core/src/segcore/Reduce.cpp:42\n =\u003e illegal req sizes, slice_nqs_prefix_sum_[last] = 1, total_nq = 2"

Show me your client code to call the search() interface.

[dumbPy]

I am using docker milvusdb/milvus:v2.4.0

here's what I am calling from python

requests.post('http://localhost:19530/v2/vectordb/entities/search', json={
    "collectionName": "icons_db",
    "annsField":"embedding",
    "data":query_embeddings.tolist(), # [[0.19..,0.8..], [...], [...]] batch of n=8
    "outputFields":["id"],
    "limit" : 1
    }).json()

where query embeddings is a batch of 8 embeddings of shape 8x1536 as list[list[float]]

and here's the actual response

{'code': 65535,
 'message': 'fail to search on QueryNode 1: worker(1) query failed: Assert "slice_nqs_prefix_sum_[num_slices_] == total_nq_" at [/go/src/github.com/milvus-io/milvus/internal/core/src/segcore/Reduce.cpp:42](https://file+.vscode-resource.vscode-cdn.net/go/src/github.com/milvus-io/milvus/internal/core/src/segcore/Reduce.cpp:42)\n => illegal req sizes, slice_nqs_prefix_sum_[last] = 1, total_nq = 8'}

[yhmo]

I believe this is a bug of v2.4.0.
I use this script to test:

import requests

from pymilvus import (
    MilvusClient,
    connections,
    FieldSchema, CollectionSchema, DataType,
    Collection,
    utility,
)

# milvus_client = MilvusClient("http://localhost:19530", user="root", password="Milvus")
# print(milvus_client.list_collections())

connections.connect(host='localhost', port='19530')
print(utility.get_server_version())


collection_name = "AAA"
dim = 1536
metric_type = "L2"

fields=[
    FieldSchema(name="id", dtype=DataType.INT64, is_primary=True, auto_id=True),
    FieldSchema(name="embedding", dtype = DataType.FLOAT_VECTOR, dim=dim),
]

schema = CollectionSchema(fields=fields)

if utility.has_collection(collection_name):
    utility.drop_collection(collection_name)

collection = Collection(name=collection_name, schema=schema)
print(f"Collection '{collection_name}' created")


collection = Collection(collection_name)

batch_count = 10000
data = [
    [[10 * (k + d) / batch_count for d in range(dim)] for k in range(batch_count)],  # vector
]
ret = collection.insert(data)
print("insert done")

collection.flush()
print("flush done")

collection = Collection(collection_name)
index_params = {
    'metric_type': metric_type,
    'index_type': "IVF_FLAT",
    'params': {"nlist": 128},
}
collection.create_index(field_name="embedding", index_params=index_params)
print("index done")

collection = Collection(collection_name)

collection.load()


res = requests.post('http://localhost:19530/v2/vectordb/entities/search', json={
    "collectionName": collection_name,
    "annsField":"embedding",
    "data":[[0.5 for d in range(dim)] for k in range(8)],
    "outputFields":["id"],
    "limit" : 1
    }).json()
print(res)

Milvus v2.4.0 returns the error "Assert "slice_nqs_prefix_sum_[num_slices_] == total_nq_"
Milvus v2.4.1 works well, the result is correct.

So, upgrade your Milvus to v2.4.1. The v2.4.1 has been released yesterday.
docker pull milvusdb/milvus:v2.4.1

[PowderLi]

we can found such logs from milvus-standalone container

Version:   v2.4.0
Built:     Tue Apr 16 09:33:01 UTC 2024
GitCommit: ffb6edd4

according to the git commit,
so i believe the image is built from tag: v2.4.0,
branch: 2.4-hotfix, which branch missed the fix of issue #32356

maybe you need to upgrade milvus, v2.4.1 is released.

[dumbPy]

Thanks a lot guys :)
upgrading to v2.4.1 fixed it.

[dumbPy]

there's one issue here though @yhmo
sending a request of shape list[list[float]], I would expect it to return a batch of docs so of shape list[list[dict]]
but instead it returns just a list of doc. so it's not really a batch request.

here's your script from above but modified to print both rest api response and client response for a batch request. rest api returns list[dict] while client returns the expected list[list[hit]]

import requests

from pymilvus import (
    MilvusClient,
    connections,
    FieldSchema, CollectionSchema, DataType,
    Collection,
    utility,
)

# milvus_client = MilvusClient("http://localhost:19530", user="root", password="Milvus")
# print(milvus_client.list_collections())

connections.connect(host='localhost', port='19530')
print(utility.get_server_version())


collection_name = "AAA"
dim = 1536
metric_type = "L2"

fields=[
    FieldSchema(name="id", dtype=DataType.INT64, is_primary=True, auto_id=True),
    FieldSchema(name="embedding", dtype = DataType.FLOAT_VECTOR, dim=dim),
]

schema = CollectionSchema(fields=fields)

if utility.has_collection(collection_name):
    utility.drop_collection(collection_name)

collection = Collection(name=collection_name, schema=schema)
print(f"Collection '{collection_name}' created")


collection = Collection(collection_name)

batch_count = 10000
data = [
    [[10 * (k + d) / batch_count for d in range(dim)] for k in range(batch_count)],  # vector
]
ret = collection.insert(data)
print("insert done")

collection.flush()
print("flush done")

collection = Collection(collection_name)
index_params = {
    'metric_type': metric_type,
    'index_type': "IVF_FLAT",
    'params': {"nlist": 128},
}
collection.create_index(field_name="embedding", index_params=index_params)
print("index done")

collection = Collection(collection_name)

collection.load()


res = requests.post('http://localhost:19530/v2/vectordb/entities/search', json={
    "collectionName": collection_name,
    "annsField":"embedding",
    "data":[[0.5*d for d in range(dim)] for k in range(8)],
    "outputFields":["id"],
    "limit" : 1
    }).json()
print(f"Rest api response:\n{res}")

res = collection.search(data=[[0.5*d for d in range(dim)] for k in range(8)], limit=1, anns_field="embedding", param={"metric_type":metric_type})
batch_response = [[hit.to_dict() for hit in batch] for batch in res]
print(f"Client response:\n{batch_response}")

[yhmo]

I think it is a bug of restful API. I have raised an issue: #32837
The restful search api always returns result of the first vector.