Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Graph function framework #3486

Merged
merged 2 commits into from
May 23, 2024
Merged

Graph function framework #3486

merged 2 commits into from
May 23, 2024

Conversation

andyfengHKU
Copy link
Contributor

No description provided.

Copy link
Contributor

@semihsalihoglu-uw semihsalihoglu-uw left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have a bunch of comments. We can discuss later today. Here is one other comment I have:

I saw the following in function.h:

    // Currently we only one variable-length function which is list creation. The expectation is
    // that all parameters must have the same type as parameterTypes[0].
  • First there is a typo here. we only one -> we only have one
  • Second, what is a variable-length function?
  • Third, is the comment still accurate? That is do we still have 1 variable-length function.

src/binder/bind/read/bind_in_query_call.cpp Outdated Show resolved Hide resolved

private:
AlgorithmCallInfo info;
std::shared_ptr<AlgorithmCallSharedState> sharedState;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think we need a shared state because AlgorithmCall operator should not be sharing its state with any other operators, at least in the current design, right? This is a sink operator that should write data to FactorizedTable, or at least for now. See my other comments.

We can keep a separate "graph" and FactorizedTable" fields, which are currently wrapped in AlgorithmSharedState, as part of the Algorithm interface (assuming we want to provide those as part of the Algorithm interface). But we don't need to have a notion of a SharedState yet.

src/include/function/algorithm_function.h Outdated Show resolved Hide resolved

struct AlgorithmCallSharedState {
std::mutex mtx;
std::shared_ptr<FactorizedTable> fTable;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Factorized Table: Given that we will force users to indicate the schema of their output for binding purposes, we should also do the work of constructing the factorized table's schema on behalf of algorithm designer, so users can assume that by the time their Algorithm code is running, they have a factorized table that is consistent with the schema they defined. We could do this inside AlgorithmCall:initLocalStateInternal function, which could call super:initLocalStateInternal (or C++ equivalent of this) to let the implementing algorithm, e.g., PRAlgorithm to continue initializing local state internal.

In fact, we can wrap FactorizedTable around a new class, ImmutableSchemaFactorizedTable class, which has AlgorithmCall as a friend class, so that AlgorithmCall implementers, e.g., PRAlgorithm cannot further modify the schema of FactorizedTable.

src/include/processor/operator/algorithm_call.h Outdated Show resolved Hide resolved
src/function/algorithm/demo_algorithm.cpp Outdated Show resolved Hide resolved
@andyfengHKU andyfengHKU force-pushed the draft-graph-function branch 2 times, most recently from 3ccebd5 to 8b488ff Compare May 22, 2024 05:16
@andyfengHKU andyfengHKU marked this pull request as ready for review May 22, 2024 05:16
@andyfengHKU andyfengHKU changed the title Draft graph function Graph function framework May 22, 2024
Copy link
Contributor

@semihsalihoglu-uw semihsalihoglu-uw left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would be good if I took another quick look.

src/function/algorithm/page_rank.cpp Outdated Show resolved Hide resolved
src/function/CMakeLists.txt Outdated Show resolved Hide resolved
src/function/algorithm/CMakeLists.txt Outdated Show resolved Hide resolved
src/function/algorithm/CMakeLists.txt Outdated Show resolved Hide resolved
src/function/algorithm/CMakeLists.txt Outdated Show resolved Hide resolved
src/function/algorithm/variable_length_path.cpp Outdated Show resolved Hide resolved
src/function/algorithm/variable_length_path.cpp Outdated Show resolved Hide resolved
src/function/algorithm/shortest_path.cpp Outdated Show resolved Hide resolved
src/function/algorithm/shortest_path.cpp Outdated Show resolved Hide resolved
src/function/algorithm/page_rank.cpp Outdated Show resolved Hide resolved
@andyfengHKU andyfengHKU force-pushed the draft-graph-function branch 3 times, most recently from ab456d8 to 4e41e03 Compare May 23, 2024 05:49
…test_path, weakly_connected_component, page_rank
@andyfengHKU andyfengHKU merged commit 3908d7f into master May 23, 2024
@andyfengHKU andyfengHKU deleted the draft-graph-function branch May 23, 2024 13:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants