hf-xet 1.2.0

Fast transfer of large files with the Hugging Face Hub.

Stars: 407, Watchers: 407, Forks: 56, Open Issues: 30

The huggingface/xet-core repo was created 1 years ago and the last code push was 4 hours ago.
The project is popular with 407 github stars!

How to Install hf-xet

You can install hf-xet using pip

pip install hf-xet

or add it to a project with poetry

poetry add hf-xet

Package Details

Author

None

License

None

Homepage

None

PyPi:

https://pypi.org/project/hf-xet/

Documentation:

https://huggingface.co/docs/hub/en/storage-backends#using-xet-storage

GitHub Repo:

https://github.com/huggingface/xet-core

Classifiers

• Scientific/Engineering/Artificial Intelligence

Related Packages

Errors

A list of common hf-xet errors.

Code Examples

Here are some hf-xet code examples and snippets.

GitHub Issues

The hf-xet package has 30 open issues on GitHub

• Upgrade GitHub Actions to latest versions
• Add hash_files() function to hf_xet Python package
• Parallel HashMap/HashSet for large-scale MerkleHash keyed collections.
• Consolidate configuration handling in data/configuration.rs into xet_config
• Bump rsa from 0.9.9 to 0.9.10
• Race condition in FileProvider causes corrupted blob content during parallel downloads
• Add Unix domain socket support for CAS HTTP connections
• CAS Bridge not working for large files
• Transparently mount super-lage directory using FUSE
• Increased failure rate with xet over conventional hub downloads
• Bump tracing-subscriber from 0.3.19 to 0.3.20 in /hf_xet

See more issues on GitHub