Is it possible for HVD to support other Collective Communication Librariese as plugins?

[hanwei131]

Is your feature request related to a problem? Please describe.
No.

Describe the solution you'd like
Adding new Collective Communication Library without changing HVD's code.

Describe alternatives you've considered
HVD involve a plugin frameword for CCL extention.

Additional context
NA.

[maxhgerlach]

As an example, what library have you thought of?

IMO it is extremely unlikely that this would be possible without any code changes in Horovod.

[hanwei131]

Something like HCCL (Huawei Collective Communication Library)

[maxhgerlach]

I only briefly looked at some documentation for HCCL, but there's a good chance that it would be relatively straightforward to add backend implemantations to Horovod that use that library. We currently have those for MPI, NCCL, Gloo, oneCCL, DDL. The code is separated out pretty well and compiled only conditionally (lots of #if directives). See CreateOperationManager for an entrypoint:

horovod/horovod/common/operations.cc

Line 144 in 8d34c85

OperationManager* CreateOperationManager(HorovodGlobalState& state) {

But there's no plugin mechanism at the moment. An advantage is that refactorings and feature additions currently typically don't break any backend implementations.