$60k in Macs for Local LLM vs $10 Subscription

Reddit r/AI_Agents News

Summary

The article discusses a YouTuber's experiment demonstrating that expensive local hardware for running LLMs is not cost-effective compared to affordable cloud subscriptions, emphasizing the current practical limitations of local AI for everyday use.

Alex Zisking, one of my favorite YouTubers - does a lot of videos on local LLMs. He's no neophyte. In this video he says: Lee has been telling you guys the truth, Local LLMs are not ready on normal people hardware. Ok, so he said nothing about me, but he made the point I've been making for some time now. All those "Stop paying Anthropic $200/mo, use free local llms" is click bait, not truth. He runs the most powerful to date open weight model, Kimi K3. People rave that is near Fable 5 power. Yes, but not on YOUR hardware. In a data center. Alex networks 4 512gb Mac studios for 2tb of ram to run the model with enough space for a large context window too. It took 4 hours, 17 tok/s output, to develop a simple yet rather nice Web dashboard - using mock data. It worked. The output was nice. But even $60k in hardware gave it nowhere remotely near the performance of a $10 subscription. Right now I have two simultaneous development efforts running. I've been running them both since about 6 hours. They work on a sprint for an hour or so, I view results, add input and direction if necessary, then move onto the next sprint. I'm paying more than $10/mo for my cloud subscriptions. But that 4 hours the Mac cluster took, is only doing the work of about a 15minute job. I'm doing "all day work, multiple projects" -- local AI can't meet the need. Yet. Probably not for you either. Link to the video in the comments.
Original Article

Similar Articles

Local LLMs vs AI APIs: Worth It? Yes.

Reddit r/ArtificialInteligence

An opinion piece weighing the pros and cons of running local LLMs versus using cloud AI APIs, concluding that local models are worthwhile.

Cost Analysis of my $6.4k Local LLM Server

Reddit r/LocalLLaMA

A detailed cost analysis of building and running a local LLM server for $6,400, comparing total ownership costs (electricity, depreciation) to equivalent API usage and coding plan subscriptions, concluding that local hosting can be more economical over time.