Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

benchmark: benchmarking cylon #626

Open
laszewsk opened this issue Oct 6, 2022 · 2 comments
Open

benchmark: benchmarking cylon #626

laszewsk opened this issue Oct 6, 2022 · 2 comments
Assignees
Labels
bug Something isn't working documentation Improvements or additions to documentation priority:high Tasks with high priority

Comments

@laszewsk
Copy link
Collaborator

laszewsk commented Oct 6, 2022

How do we run a repeatable benchmark on different machines using cylon?

We should address this:

Cylon has many function, we need to test them
Cylon can be run on multiple machines, can number of nodes be variable in benchmark
Is cylon also using threads, should they be benchmarked to?
The input data can be of various size, can that be part of the benchmark?

I think it would be appropriate to have a standrad benchmark so we can run all over the place by various people on various platforms.

Does this exist?

@laszewsk laszewsk added question Further information is requested enhancement New feature or request labels Oct 6, 2022
@nirandaperera
Copy link
Collaborator

This is already there @laszewsk. We can reuse the experiments in cylon_experiments https://github.com/cylondata/cylon_experiments

One small change we need to do is, add a data set creation logic for weak scaling

@laszewsk
Copy link
Collaborator Author

laszewsk commented Oct 6, 2022

Unfortunately the README is empty, so if you can develop one, I can run more experiments on summit

@laszewsk laszewsk added bug Something isn't working priority:high Tasks with high priority documentation Improvements or additions to documentation labels Aug 2, 2023
@laszewsk laszewsk removed question Further information is requested enhancement New feature or request labels Aug 2, 2023
@laszewsk laszewsk changed the title benchmarking cylon benchmark: benchmarking cylon Aug 2, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working documentation Improvements or additions to documentation priority:high Tasks with high priority
Projects
None yet
Development

No branches or pull requests

4 participants