deepsparse 1.8.0
0
An inference runtime offering GPU-class performance on CPUs and APIs to integrate ML into your appli
Contents
An inference runtime offering GPU-class performance on CPUs and APIs to integrate ML into your application
Stars: 2986, Watchers: 2986, Forks: 173, Open Issues: 31The neuralmagic/deepsparse
repo was created 3 years ago and the last code push was 2 months ago.
The project is very popular with an impressive 2986 github stars!
How to Install deepsparse
You can install deepsparse using pip
pip install deepsparse
or add it to a project with poetry
poetry add deepsparse
Package Details
- Author
- Neuralmagic, Inc.
- License
- Neural Magic DeepSparse Community License
- Homepage
- https://github.com/neuralmagic/deepsparse
- PyPi:
- https://pypi.org/project/deepsparse/
- GitHub Repo:
- https://github.com/neuralmagic/deepsparse
Classifiers
- Scientific/Engineering
- Scientific/Engineering/Artificial Intelligence
- Scientific/Engineering/Mathematics
- Software Development
- Software Development/Libraries/Python Modules
Related Packages
Errors
A list of common deepsparse errors.
Code Examples
Here are some deepsparse
code examples and snippets.
GitHub Issues
The deepsparse package has 31 open issues on GitHub
- Bug Fix: Make Numpy Array Outputs JSON Serializable for Server
- [Text Generation] [Fix] Raise error when we use deepsparse engine and
prompt_processing_length == sequence_lenght
- [Text Generation] Optimize the slow
update
method in the KVCacheDecoder - [BugFix] Delay torch import until needed for
deepsparse.transformers.eval_downstream
- Update text_generation.py
- DS eval for OPT on WikiText
- [CLIP] Validation Script
- Changes to support pass@k evaluation on the HumanEval dataset
- [Text Generation] Turn off the (currently) inefficient external KV cache logic when internal KV cache management enabled
- Implement OpenAI-compatible server
- Generalize disabling batch size across engine interfaces
- [Text Generation] KVCacheStorage Implementation
- [Text Generation][Doc] Point to KV Cache Injection
- Can't use DeepSparse with VITS ONNX file from Coqui TTS.
- Encountered an issue when trying to optimize Donut model.