gpustat 1.1.1

Python Script to download hundreds of images from 'Google Images'. It is a ready-to-run code!

A cross-platform curses-based monitoring tool

Celery Flower

fastai simplifies training fast and accurate neural nets using modern best practices

Epsagon Instrumentation for Python

django-watchman exposes a status endpoint for your backing services

Django middlewares to monitor your application with Prometheus.io.

Run checks on services like databases, queue servers, celery processes, etc.

DeepSpeed is a Python package developed by Microsoft that provides a deep learning optimization library designed to scale across multiple GPUs and servers. It is capable of training models with billions or even trillions of parameters, achieving excellent system throughput and efficiently scaling to thousands of GPUs.

DeepSpeed is particularly useful for training and inference of large language models, and it falls under the category of Machine Learning Frameworks and Libraries. It is designed to work with PyTorch and offers system innovations such as Zero Redundancy Optimizer (ZeRO), 3D parallelism, and model-parallelism to enable efficient training of large models.