horovod 0.28.1


Distributed training framework for TensorFlow, Keras, PyTorch, and Apache MXNet.

Distributed training framework for TensorFlow, Keras, PyTorch, and Apache MXNet.

Stars: 13474, Watchers: 13474, Forks: 2204, Open Issues: 359

The horovod/horovod repo was created 5 years ago and the last code push was 6 days ago.
The project is extremely popular with a mindblowing 13474 github stars!

How to Install horovod

You can install horovod using pip

pip install horovod

or add it to a project with poetry

poetry add horovod

Package Details

The Horovod Authors
Apache 2.0
GitHub Repo:


  • Scientific/Engineering/Artificial Intelligence
No  horovod  pypi packages just yet.


A list of common horovod errors.

Code Examples

Here are some horovod code examples and snippets.

GitHub Issues

The horovod package has 359 open issues on GitHub

  • Cannot install horovod with tensorflow framework
  • mpirun check failed with version
  • Problem during installation
  • Elastic Horovod incompatible with tf.compat.v1.train.MonitoredTrainingSession
  • Pre-load dataset to memory
  • failed call to cuInit after calling hvd.init()
  • Build Horovod with temporarily installed CMake if necessary
  • learning rate can't update in resuming training
  • sometimes fails on Buildkite
  • can horovod work only with Gloo,without MPI?
  • Horovd + AMP Contrastive Learning Imprecision
  • GPU Head tests for elastic ray are flaky in CI
  • integrate with Lightning ecosystem CI
  • Error on installing: pip install horovod[pytorch] in Virtualbox ubuntu 18.04
  • Multi worker inference in Databricks

See more issues on GitHub

Related Packages & Articles

thinc 8.1.10

A refreshing functional take on deep learning, compatible with your favorite libraries

datasets 2.14.2

HuggingFace community-driven open-source library of datasets

clearml 1.12.1

ClearML - Auto-Magical Experiment Manager, Version Control, and MLOps for AI

nlp 0.4.0

HuggingFace/NLP is an open library of NLP datasets.


H2O, Fast Scalable Machine Learning, for python