Contents

auto-gptq 0.7.1

0

An easy-to-use LLMs quantization package with user-friendly apis, based on GPTQ algorithm.

An easy-to-use LLMs quantization package with user-friendly apis, based on GPTQ algorithm.

Stars: 5026, Watchers: 5026, Forks: 531, Open Issues: 274

The AutoGPTQ/AutoGPTQ repo was created 2 years ago and the last code push was 10 months ago.
The project is extremely popular with a mindblowing 5026 github stars!

How to Install auto-gptq

You can install auto-gptq using pip

pip install auto-gptq

or add it to a project with poetry

poetry add auto-gptq

Package Details

Author
PanQiWei
License
Homepage
https://github.com/PanQiWei/AutoGPTQ
PyPi:
https://pypi.org/project/auto-gptq/
GitHub Repo:
https://github.com/PanQiWei/AutoGPTQ

Classifiers

No  auto-gptq  pypi packages just yet.

Errors

A list of common auto-gptq errors.

Code Examples

Here are some auto-gptq code examples and snippets.

Related Packages & Articles

optimum 2.1.0

Optimum Library is an extension of the Hugging Face Transformers library, providing a framework to integrate third-party libraries from Hardware Partners and interface with their specific functionality.

farm-haystack 1.26.4.post0

LLM framework to build customizable, production-ready LLM applications. Connect components (models, vector DBs, file converters) to pipelines or agents that can interact with your data.

transformers 5.2.0

Transformers: the model-definition framework for state-of-the-art machine learning models in text, vision, audio, and multimodal models, for both inference and training.

deeplake 4.5.2

Deep Lake is a Database for AI powered by a unique storage format optimized for deep-learning and Large Language Model (LLM) based applications. It simplifies the deployment of enterprise-grade LLM-based products by offering storage for all data types (embeddings, audio, text, videos, images, pdfs, annotations, etc.), querying and vector search, data streaming while training models at scale, data versioning and lineage for all workloads, and integrations with popular tools such as LangChain, LlamaIndex, Weights & Biases, and many more.

beir 2.2.0

A Heterogeneous Benchmark for Information Retrieval