New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
universal-query: Sketch request types pipeline #4198
Conversation
//! Pipeline of type conversion: | ||
//! | ||
//! 1. `rest::QueryRequest`, `grpc::QueryPoints`: rest or grpc request. Used in API | ||
//! 2. `CollectionQueryRequest`: Direct representation of the API request, but to be used as a single type. Created at API to enter ToC. | ||
//! 3. `ShardQueryRequest`: same as the common request, but all point ids have been substituted with vectors. Created at Collection | ||
//! 4. `QueryShardPoints`: to be used in the internal service. Created for RemoteShard, converts to and from ShardQueryRequest | ||
//! 5. `PlannedQuery`: an easier-to-execute representation. Created in LocalShard |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is the proposed type pipeline description
8509242
to
48df0f8
Compare
uint64 limit = 5; | ||
} | ||
|
||
Prefetch prefetch = 1; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
repeated?
} | ||
|
||
message Prefetch { | ||
Prefetch prefetch = 1; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
repeated?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think it makes to merge this PR sooner than later.
This is a sketch of the types required to get us started, perfection is not expected.
This PR exists because we want to avoid a huge changing set at once.
With it we can start a more bottom approach.
Main idea: Each user request will be converted into a new type at each step of the search: API -> ToC -> Collection -> Shards.
This PR adds the types used at Shards level only, as well as a PoC of how the request will be handled at
LocalShard
.Note that a significant amount of changed lines come from moving the
LocalShard
implementations ofdo_search
scroll
andquery
to their own filesAll Submissions:
dev
branch. Did you create your branch fromdev
?New Feature Submissions:
cargo +nightly fmt --all
command prior to submission?cargo clippy --all --all-features
command?