Eager-SGD is a decentralized asynchronous SGD. It utilizes novel partial collectives operations to accumulate the gradients across all the processes.
-
Updated
Nov 18, 2021 - Python
#
partial-allreduce
Here are 2 public repositories matching this topic...
WAGMA-SGD is a decentralized asynchronous SGD based on wait-avoiding group model averaging. The synchronization is relaxed by making the collectives externally-triggerable, namely, a collective can be initiated without requiring that all the processes enter it. It partially reduces the data within non-overlapping groups of process, improving the…
-
Updated
Jun 30, 2021 - Python
Improve this page
Add a description, image, and links to the partial-allreduce topic page so that developers can more easily learn about it.
Add this topic to your repo
To associate your repository with the partial-allreduce topic, visit your repo's landing page and select "manage topics."