DumpsterCluster: From Dumpster Diving to Serving LLaMA-70B on $60 GPUs

Hugging Face Daily Papers Papers
gpu-repurposing llm-inference cost-optimization environmental-impact pipeline-parallelism

Summary

This paper explores using retired GPUs to build low-cost clusters for serving LLaMA-70B, finding economic viability in regions with cheap electricity but highlighting potential high carbon emissions without clean energy sources.

As AI datacenters retire functional GPUs, vast quantities of still capable accelerators enter secondary markets. This paper investigates whether these retired GPUs can find a productive afterlife to form a DumpsterCluster that can serve modern LLM inference, and under what conditions such repurposing is economically viable and environmentally sustainable. We physically built a 128-GPU DumpsterCluster from scratch using only second-hand components and ran it for one year. At current market prices (\22K for the DumpsterCluster vs. 600K for an 8-GPU B200 system), the economic advantages are substantial. Through pipeline-parallel optimizations, our V100 based DumpsterCluster achieves competitive LLaMA-70B throughput, validating production viability. However, our deployment reveals critical context dependencies. Older GPUs consume significantly more energy per token, making total cost of ownership favorable only in regions with inexpensive electricity. Under grid-average carbon intensity, second-hand systems can produce approximately 4x higher total carbon emissions per token for 8B models, and over 40x for 70B models, compared to current-generation hardware. These findings show that GPU afterlife is not universally sustainable - hardware repurposing must be strategically coupled with low carbon energy sources. When deployed in regions with favourable energy economics and clean electricity, second-hand GPUs offer a viable pathway for expanding AI capacity while advancing affordability, energy security, and environmental responsibility.
Original Article
View Cached Full Text

Cached at: 08/18/26, 07:51 AM

Paper page - DumpsterCluster: From Dumpster Diving to Serving LLaMA-70B on $60 GPUs

Source: https://huggingface.co/papers/2608.14614 Published on Jul 10

·

Submitted byhttps://huggingface.co/iliashum

ion Aug 18

Abstract

Retired GPUs can form low-cost clusters for LLM inference, but their economic and environmental viability depends heavily on local electricity prices and carbon intensity.

As AI datacenters retire functional GPUs, vast quantities of still capable accelerators enter secondary markets. This paper investigates whether these retired GPUs can find a productive afterlife to form aDumpsterClusterthat can serve modern LLM inference, and under what conditions such repurposing is economically viable and environmentally sustainable. We physically built a 128-GPUDumpsterClusterfrom scratch using only second-hand components and ran it for one year. At current market prices (\22K for theDumpsterClustervs. 600K for an 8-GPU B200 system), the economic advantages are substantial. Throughpipeline-parallel optimizations, our V100 basedDumpsterClusterachieves competitiveLLaMA-70Bthroughput, validating production viability. However, our deployment reveals critical context dependencies. Older GPUs consume significantly more energy per token, makingtotal cost of ownershipfavorable only in regions with inexpensive electricity. Under grid-averagecarbon intensity, second-hand systems can produce approximately 4x higher total carbon emissions per token for 8B models, and over 40x for 70B models, compared to current-generation hardware. These findings show that GPU afterlife is not universally sustainable - hardware repurposing must be strategically coupled with low carbon energy sources. When deployed in regions with favourable energy economics and clean electricity,second-hand GPUsoffer a viable pathway for expanding AI capacity while advancing affordability, energy security, and environmental responsibility.

View arXiv pageView PDFAdd to collection

Get this paper in your agent:

hf papers read 2608\.14614

Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash

Models citing this paper0

No model linking this paper

Cite arxiv.org/abs/2608.14614 in a model README.md to link it from this page.

Datasets citing this paper0

No dataset linking this paper

Cite arxiv.org/abs/2608.14614 in a dataset README.md to link it from this page.

Spaces citing this paper0

No Space linking this paper

Cite arxiv.org/abs/2608.14614 in a Space README.md to link it from this page.

Collections including this paper0

No Collection including this paper

Add this paper to acollectionto link it from this page.

Similar Articles

Nobody knows what a used GPU cluster is worth

Hacker News Top

An analysis of the challenges in valuing used GPU clusters as collateral for debt financing, highlighting operational dependencies and hardware failure rates that make traditional asset appraisal impossible.