I turned an Android phone into a Vulkan-accelerated local LLM node (GGUF + LiteLLM + Tailscale)
Summary
An Android phone is repurposed as a portable GGUF inference server with Vulkan acceleration, exposing an OpenAI-compatible endpoint via LiteLLM and Tailscale mesh for integration into a self-hosted AI cluster.
Similar Articles
I made this android app which runs ai models locally
A developer created an Android app that runs AI models locally, supporting GGUF and LiteRT formats with multiple ways to add models.
I clustered two old Android phones to run a 27B LLM locally. Bypassed RAM limits via C++ RPC split and Hadamard Ternary math (No overheating)
A high school student built a custom C++ engine (fork of llama.rn) that clusters two Android phones over USB via RPC to run a 27B ternary-quantized LLM locally, using runtime Hadamard transforms for addition-only math that keeps the phone cool at ~30°C.
@TheTechDiggest: [OpenSource - Distributed AI & Mesh LLM Inference] Buying an expensive enterprise GPU isn't the only way to run massive…
A tweet thread introduces mesh-llm, an open-source tool that pools local network devices into a unified, OpenAI-compatible API for running large LLMs without expensive enterprise GPUs.
Infermeld: a Linux kit for running one GGUF across AMD + NVIDIA GPUs with llama.cpp
Infermeld is an experimental open-source Linux companion kit (v0.1.0) for running one GGUF model across mixed AMD and NVIDIA GPUs using llama.cpp, packaging device selection, build instructions, preflight checks, and a thermal guard. Tested on an RX 6000 XT + RTX 3080 with Qwen3.6-35B-A3B, with the maintainer seeking other hardware combos for reproduction.
Unexpected use of local llm
A smartphone battery reviewer uses two local LLMs (Qwen3.6 models) running on high-end NVIDIA GPUs to power a vision-language agent that autonomously controls a robotic arm for realistic battery drain tests, achieving sub-two-second latency.