RWKV is a novel language model architecture that combines the efficiency of RNNs with the parallelizability of Transformers, achieving strong LLM performance with linear time complexity and constant memory footprint.
# RWKV Language Model
Source: [https://www.rwkv.com/](https://www.rwkv.com/)
- [Github](https://github.com/BlinkDL/RWKV-LM)
- [Twitter](https://twitter.com/BlinkDL_AI)
- [Discord](https://discord.gg/bDSBUMeFpc)
RWKV \(pronounced RwaKuv\) is an RNN with great LLM performance and parallelizable like a Transformer\. Check[RWKV\-7 "Goose" reasoning models](https://huggingface.co/BlinkDL/rwkv7-g1)\.
It's combining the best of RNN and transformer \- great performance, linear time, constant space \(no kv\-cache\), fast training, infinite ctxlen, and free text embedding\. And it's 100% attention\-free, and a[Linux Foundation AI project](https://lfaidata.foundation/projects/rwkv/)\.
[7B Demo](https://huggingface.co/spaces/BlinkDL/RWKV-Gradio-1)[7B Demo](https://huggingface.co/spaces/BlinkDL/RWKV-Gradio-2)[✨7B Demo](https://huggingface.co/spaces/BlinkDL/RWKV-Gradio-3)[✨13B Demo](https://huggingface.co/spaces/BlinkDL/RWKV-Gradio-4)
[](https://arxiv.org/abs/2503.14456)## RWKV\-Projects
[RWKV\-LM Training RWKV \(and latest developments\)](https://github.com/BlinkDL/RWKV-LM)[RWKV App RWKV App for Android / iOS / PC / Mac / Linux](https://github.com/RWKV-APP/RWKV_APP)[Albatross Very efficient inference \(7B fp16 bsz960 = 10250\+ tps on 5090\)](https://github.com/BlinkDL/Albatross)[RWKV\-Runner RWKV GUI with API \(desktop\)](https://github.com/josStorer/RWKV-Runner)[RWKV pip package RWKV pip reference \(slower\) package](https://pypi.org/project/rwkv/)[RWKV\-PEFT Finetuning RWKV \(9GB VRAM can finetune 7B\)](https://github.com/JL-er/RWKV-PEFT)[RWKV\-server WebGPU inference \(NVIDIA/AMD/Intel\), nf4/int8/fp16](https://github.com/cgisky1980/ai00_rwkv_server)[More\.\.\. \(800\+ RWKV projects\)](https://github.com/search?o=desc&q=rwkv&s=updated&type=Repositories)
## Use RWKV
[RWKV\-7 weights All latest RWKV weights](https://huggingface.co/BlinkDL)[RWKV\-7 GGUF weights GGUF RWKV weights](https://huggingface.co/collections/shoumenchougou/rwkv7-gxx-gguf)[RWKV\-7 Ollama weights Ollama GGUF RWKV weights](https://ollama.com/mollysama)[RWKV\-related papers](https://scholar.google.com/scholar?scisbd=2&q=rwkv&as_sdt=0,5)[RWKV wiki with history of RWKV from v1 to v7 \(note: AI\-written\)](https://wiki.rwkv.com/)
## RWKV\-Papers
## RWKV\-8 explained
## RWKV\-7 explained
## RWKV\-7 illustrated
## RWKV\-6 illustrated
BlinkDL announces RWKV-7 G1g, a pure RNN LLM that claims to be the best in its class and competitive with general LLMs, with high-speed inference on a single RTX 5090.
River-LLM proposes a training-free early-exit framework for decoder-only LLMs that uses KV-sharing to eliminate KV-cache gaps, achieving 1.71–2.16× speedup without quality loss.
Introduces Looped Latent Attention (LLA), a post-training codec that compresses KV cache in looped transformers by exploiting low-rank structure across recurrence steps, achieving significant compression ratios while maintaining performance.
Key-Value Means (KVM) is a novel attention mechanism that combines the strengths of transformers and RNNs with controllable computational complexity and memory usage. It supports fixed-size or growing state, offers subquadratic prefill time and sublinear state growth, and can be implemented without custom kernels.
Explores the growing memory bottleneck of KV-cache in transformer inference, explaining why alternative architectures with fixed-size memory like Mamba and RWKV are gaining renewed attention.