-
Notifications
You must be signed in to change notification settings - Fork 81
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Graph function framework #3486
Graph function framework #3486
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I have a bunch of comments. We can discuss later today. Here is one other comment I have:
I saw the following in function.h:
// Currently we only one variable-length function which is list creation. The expectation is
// that all parameters must have the same type as parameterTypes[0].
- First there is a typo here. we only one -> we only have one
- Second, what is a variable-length function?
- Third, is the comment still accurate? That is do we still have 1 variable-length function.
|
||
private: | ||
AlgorithmCallInfo info; | ||
std::shared_ptr<AlgorithmCallSharedState> sharedState; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't think we need a shared state because AlgorithmCall operator should not be sharing its state with any other operators, at least in the current design, right? This is a sink operator that should write data to FactorizedTable, or at least for now. See my other comments.
We can keep a separate "graph" and FactorizedTable" fields, which are currently wrapped in AlgorithmSharedState, as part of the Algorithm interface (assuming we want to provide those as part of the Algorithm interface). But we don't need to have a notion of a SharedState yet.
|
||
struct AlgorithmCallSharedState { | ||
std::mutex mtx; | ||
std::shared_ptr<FactorizedTable> fTable; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Factorized Table: Given that we will force users to indicate the schema of their output for binding purposes, we should also do the work of constructing the factorized table's schema on behalf of algorithm designer, so users can assume that by the time their Algorithm code is running, they have a factorized table that is consistent with the schema they defined. We could do this inside AlgorithmCall:initLocalStateInternal function, which could call super:initLocalStateInternal (or C++ equivalent of this) to let the implementing algorithm, e.g., PRAlgorithm to continue initializing local state internal.
In fact, we can wrap FactorizedTable around a new class, ImmutableSchemaFactorizedTable class, which has AlgorithmCall as a friend class, so that AlgorithmCall implementers, e.g., PRAlgorithm cannot further modify the schema of FactorizedTable.
3ccebd5
to
8b488ff
Compare
bdd0d30
to
1a59e95
Compare
8235300
to
9308219
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Would be good if I took another quick look.
ab456d8
to
4e41e03
Compare
…test_path, weakly_connected_component, page_rank
b025596
to
67bf4cf
Compare
No description provided.