Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Auto Parallel]: Speed up intra-op plan generation by 44% #5446

Open
wants to merge 3 commits into
base: main
Choose a base branch
from

Conversation

stephankoe
Copy link
Contributor

📌 Checklist before creating the PR

  • I have created an issue for this PR for traceability
  • The title follows the standard format: [doc/gemini/tensor/...]: A concise description
  • I have added relevant tags if possible for us to better distinguish different PRs

🚨 Issue number

resolves #5436

📝 What does this PR do?

As stated in issue #5436, the generation of a DimSpec object is comparatively costly because a dictionary is created every time and a deep copy of two strings is being made. The large volume of DimSpec objects created leads to several seconds spent in total by creating DimSpec objects when generating intra-op plans with ColossalAuto.

This pull requests addresses this inefficiency by converting this dict to a class attribute of the DimSpec class, so it's shared among all its instances. This dict is initialized lazily the first time the property difference_dict is used. This is possible because the contents of difference_dict are not modified by other portions of the code.

Additionally, the methods build_difference_2d_dict and convert_str_to_shard_list are made class/static methods and private because they don't need access to the instance properties.

Effect

These changes reduce the end-to-end wall-clock time to build the strategy constructor by 44% while running the script examples/tutorial/auto_parallel/auto_parallel_with_resnet.py. The time required to compute a solution on a laptop with Intel Core i7 7700HQ was on average 56.79 s before the change. After applying above-mentioned changes, the wall-clock time reduced to about 31.93 s.

💥 Checklist before requesting a review

  • I have linked my PR to an issue (instruction)
  • My issue clearly describes the problem/feature/proposal, with diagrams/charts/table/code if possible
  • I have performed a self-review of my code
  • I have added thorough tests.
  • I have added docstrings for all the functions/methods I implemented

⭐️ Do you enjoy contributing to Colossal-AI?

  • 🌝 Yes, I do.
  • 🌚 No, I don't.

This change considerably speeds up construction speed of DimSpec objects. The difference_dict is the same for each DimSpec object, so a single copy of it is enough.
@stephankoe stephankoe requested a review from a team as a code owner March 12, 2024 07:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[PROPOSAL]: Speed improvement of Intra-Op plan generation in ColossalAuto
1 participant