@adithya_s_k: HF storage buckets are so underrated and makes life so much simpler if you're doing anything with data at scale. Before…
Summary
Hugging Face storage buckets are praised as a cost-effective and simple solution for large-scale data management, avoiding high egress costs of other providers.
View Cached Full Text
Cached at: 05/16/26, 03:22 PM
HF storage buckets are so underrated and makes life so much simpler if you’re doing anything with data at scale.
Before this, it was such a pain when switching between compute providers like Azure, Modal, or AWS.
Azure storage is expensive, and the egress costs are insane. I once spent thousands of dollars just to migrate TBs of data off Azure.
Now I can simply mount a storage bucket on a Modal compute instance or an Azure/AWS VM and not worry to much
clem 🤗 (@ClementDelangue): AI teams shouldn’t have to choose between expensive object storage and painful git workflows.
@huggingface Storage is built for model weights, datasets, checkpoints and artifacts:
- simple per-TB pricing
- built-in CDN
- Xet deduplication
- private by default when needed
Store
Similar Articles
Introducing Storage Buckets on the Hugging Face Hub
Hugging Face introduces Storage Buckets, a new mutable, S3-like object storage feature on the Hub optimized for production ML workflows using its Xet backend for efficient deduplication.
Hugging Face Storage Buckets (Website)
Hugging Face introduces Storage Buckets, a scalable object storage service for AI teams with per-TB pricing, Xet deduplication, built-in CDN, and no git overhead, designed for datasets, model checkpoints, and ML artifacts.
@ClementDelangue: AI teams shouldn’t have to choose between expensive object storage and painful git workflows. @huggingface Storage is b…
Hugging Face launches Storage Buckets, a purpose-built storage solution for AI teams offering per-TB pricing, built-in CDN, and Xet deduplication for model weights, datasets, and checkpoints.
@ClementDelangue: Great to see @CommonCrawl using and recommending @huggingface Buckets for large constantly evolving training datasets! …
Hugging Face announces Storage Buckets, a storage solution for large, evolving training datasets with built-in CDN and deduplication, recommended by CommonCrawl.
@julien_c: “Your data lives on Hugging Face. Your compute runs on Vast GPUs.” I love this!!
vast.ai now supports Hugging Face Storage Buckets as a cloud connection, allowing rented GPU instances to pull datasets and checkpoints directly from HF buckets and push results back without manual transfers.