Python or Rust? Coding with local LLMs
Summary
A discussion post asking whether Rust or Python is better for coding with local LLMs, exploring whether Rust's compiler feedback helps compensate for weaker small models running on consumer hardware.
Similar Articles
Fast and Hard Code
The article discusses how LLMs are reducing friction in programming language choice, leading developers to adopt performance-oriented languages like Rust and Zig for fast, small software and enabling work with complex technologies previously considered difficult.
Building a Rust Inference Engine That Matches Llama.cpp
Ferrox is a pure-Rust inference engine that loads GGUF models and runs local LLMs on CPU, Metal, or CUDA, with a CLI and an OpenAI-compatible server. It aims to match llama.cpp's performance while being written from scratch with no bindings.
Testing Local LLMs in Practice: Code Generation, Quality vs. Speed
The author built a benchmark harness to evaluate local LLMs for autonomous Go code generation, focusing on log parser generation for SIEM pipelines, and published results comparing quality vs. speed.
Writing Rust code that's faster than state-of-the-art libraries by asking agents to make the code faster
The article describes using agentic LLMs to iteratively optimize Rust code, achieving significant speedups (2x-20x) over state-of-the-art libraries through benchmarks and experiments.
@martinfowler: NEW POST Birgitta Böckeler recently spent some time trying out running local LLMs for some programming tasks. In this m…
Birgitta Böckeler shares her experience running local LLMs for coding tasks, outlining factors like RAM, response speed, tool calling, and quality of outcomes that influence their viability.