ruvnet/RuView Source: https://github.com/ruvnet/RuView # π RuView > Beta Software — Under active development. APIs and firmware may change. Known limitations: > - ESP32-C3 and original ESP32 are not supported (single-core, insufficient for CSI DSP) > - Single ESP32 deployments have limited spatial resolution — use 2+ nodes or add a Cognitum Seed (https://cognitum.one) for best results > - Camera-free pose accuracy is limited — use camera ground-truth training for 92.9% PCK@20 > > Contributions and bug reports welcome at Issues (https://github.com/ruvnet/RuView/issues). ## See through walls with WiFi ## Turn ordinary WiFi into a sensing system. Detect people, measure breathing and heart rate, track movement, and monitor rooms — through walls, in the dark, with no cameras or wearables. Just physics. ### π RuView is a WiFi sensing platform that transforms radio signals into spatial intelligence. Every WiFi router already floods your space with radio waves. When people move, breathe, or even sit still, they disturb those waves in measurable ways. RuView captures these disturbances using Channel State Information (CSI) from low-cost ESP32 sensors and converts them into actionable data: who’s present, what they’re doing, and whether they’re okay. What it senses: - Presence and occupancy — detect people through walls, count them, track entries and exits - Vital signs — breathing rate and heart rate, contactless, while sleeping or sitting - Activity recognition — walking, sitting, gestures, falls — from temporal CSI patterns - Environment mapping — RF fingerprinting identifies rooms, detects moved furniture, spots new objects - Sleep quality — overnight monitoring with sleep stage classification and apnea screening Built on RuVector (https://github.com/ruvnet/ruvector/) and Cognitum Seed (https://cognitum.one), RuView runs entirely on edge hardware — an ESP32 mesh (as low as $9 per node) paired with a Cognitum Seed for persistent memory, cryptographic attestation, and AI integration. No cloud, no cameras, no internet required. The system learns each environment locally using spiking neural networks that adapt in under 30 seconds, with multi-frequency mesh scanning across 6 WiFi channels that uses your neighbors’ routers as free radar illuminators. Every measurement is cryptographically attested via an Ed25519 witness chain. RuView also supports pose estimation (17 COCO keypoints via the WiFlow architecture), trained entirely without cameras using 10 sensor signals — a technique pioneered from the original DensePose From WiFi research at Carnegie Mellon University. ### Built for low-power edge applications Edge modules are small programs that run directly on the ESP32 sensor — no internet needed, no cloud fees, instant response. Rust 1.85+ (https://www.rust-lang.org/) License: MIT (https://opensource.org/licenses/MIT) Tests: 1463 (https://github.com/ruvnet/RuView) Docker: multi-arch (https://hub.docker.com/r/ruvnet/wifi-densepose) Vital Signs ESP32 Ready crates.io (https://crates.io/crates/wifi-densepose-ruvector) > | What | How | Speed | > |——|—–|—––| > | Pose estimation | CSI subcarrier amplitude/phase → 17 COCO keypoints | 171K emb/s (M4 Pro) | > | Breathing detection | Bandpass 0.1-0.5 Hz → zero-crossing BPM | 6-30 BPM | > | Heart rate | Bandpass 0.8-2.0 Hz → zero-crossing BPM | 40-120 BPM | > | Presence sensing | Trained model + PIR fusion — 100% accuracy | 0.012 ms latency | > | Through-wall | Fresnel zone geometry + multipath modeling | Up to 5m depth | > | Edge intelligence | 8-dim feature vectors + RVF store on Cognitum Seed | 140 total BOM | > | **Camera-free training** | 10 sensor signals, no labels needed | 84s on M4 Pro | > | **Camera-supervised training** | MediaPipe + ESP32 CSI → 92.9% PCK@20 | 19 min on laptop | > | **Multi-frequency mesh** | Channel hopping across 6 bands, neighbor APs as illuminators | 3x sensing bandwidth | ```bash # Option 1: Docker (simulated data, no hardware needed) docker pull ruvnet/wifi-densepose:latest docker run -p 3000:3000 ruvnet/wifi-densepose:latest # Open http://localhost:3000 # Option 2: Live sensing with ESP32-S3 hardware (9) # Flash firmware, provision WiFi, and start sensing: python -m esptool –chip esp32s3 –port COM9 –baud 460800 \ write_flash 0x0 bootloader.bin 0x8000 partition-table.bin \ 0xf000 ota_data_initial.bin 0x20000 esp32-csi-node.bin python firmware/esp32-csi-node/provision.py –port COM9 \ –ssid “YourWiFi” –password “secret” –target-ip 192.168.1.20 # Option 3: Full system with Cognitum Seed (140) # ESP32 streams CSI → bridge forwards to Seed for persistent storage + kNN + witness chain node scripts/rf-scan.js --port 5006 # Live RF room scan node scripts/snn-csi-processor.js --port 5006 # SNN real-time learning node scripts/mincut-person-counter.js --port 5006 # Correct person counting ``` > [!NOTE] > **CSI-capable hardware recommended.** Presence, vital signs, through-wall sensing, and all advanced capabilities require Channel State Information (CSI) from an ESP32-S3 (9) or research NIC. The Docker image runs with simulated data for evaluation. Consumer WiFi laptops provide RSSI-only presence detection. > Hardware options for live CSI capture: > > | Option | Hardware | Cost | Full CSI | Capabilities | > |––––|–––––|——|–––––|———––| > | ESP32 + Cognitum Seed (recommended) | ESP32-S3 + Cognitum Seed (https://cognitum.one) | ~140 | Yes | Pose, breathing, heartbeat, motion, presence + persistent vector store, kNN search, witness chain, MCP proxy | > | **ESP32 Mesh** | 3-6x ESP32-S3 + WiFi router | ~54 | Yes | Pose, breathing, heartbeat, motion, presence | > | Research NIC | Intel 5300 / Atheros AR9580 | ~$50-100 | Yes | Full CSI with 3x3 MIMO | > | Any WiFi | Windows, macOS, or Linux laptop | 0 | No | RSSI-only: coarse presence and motion | > > No hardware? Verify the signal processing pipeline with the deterministic reference signal: `python v1/data/proof/verify.py` > --- ### What's New in v0.7.0 Camera Ground-Truth Training — 92.9% PCK@20 **v0.7.0 adds camera-supervised pose training** using MediaPipe + real ESP32 CSI data: | Capability | What it does | ADR | |-----------|-------------|-----| | **Camera ground-truth collection** | MediaPipe PoseLandmarker captures 17 COCO keypoints at 30fps, synced with ESP32 CSI | [ADR-079](docs/adr/ADR-079-camera-ground-truth-training.md) | | **ruvector subcarrier selection** | Variance-based top-K reduces input by 50% (70→35 subcarriers) | ADR-079 O6 | | **Stoer-Wagner min-cut** | Person-specific subcarrier cluster separation for multi-person training | ADR-079 O8 | | **Scalable WiFlow model** | 4 presets: lite (189K) → small (474K) → medium (800K) → full (7.7M params) | ADR-079 | ```bash # Collect ground truth (camera + ESP32 simultaneously) python scripts/collect-ground-truth.py --duration 300 --preview python scripts/record-csi-udp.py --duration 300 # Align CSI windows with camera keypoints node scripts/align-ground-truth.js --gt data/ground-truth/*.jsonl --csi data/recordings/*.csi.jsonl # Train WiFlow model (start lite, scale up as data grows) node scripts/train-wiflow-supervised.js --data data/paired/*.jsonl --scale lite # Evaluate node scripts/eval-wiflow.js --model models/wiflow-real/wiflow-v1.json --data data/paired/*.jsonl ``` **Result: 92.9% PCK@20** from a 5-minute data collection session with one ESP32-S3 and one webcam. | Metric | Before (proxy) | After (camera-supervised) | |--------|----------------|--------------------------| | PCK@20 | 0% | **92.9%** | | Eval loss | 0.700 | **0.082** | | Bone constraint | N/A | **0.008** | | Training time | N/A | **19 minutes** | | Model size | N/A | **974 KB** | Pre-trained model: HuggingFace ruv/ruview/wiflow-v1 (https://huggingface.co/ruv/ruview) ### Pre-Trained Models (v0.6.0) — No Training Required Download from HuggingFace and start sensing immediately Pre-trained models are available on HuggingFace: > **https://huggingface.co/ruv/ruview** (primary) | mirror (https://huggingface.co/ruvnet/wifi-densepose-pretrained) Trained on 60,630 real-world samples from an 8-hour overnight collection. Just download and run — no datasets, no GPU, no training needed. | Model | Size | What it does | |-------|------|-------------| | `model.safetensors` | 48 KB | Contrastive encoder — 128-dim embeddings for presence, activity, environment | | `model-q4.bin` | 8 KB | 4-bit quantized — fits in ESP32-S3 SRAM for edge inference | | `model-q2.bin` | 4 KB | 2-bit ultra-compact for memory-constrained devices | | `presence-head.json` | 2.6 KB | 100% accurate presence detection head | | `node-1.json` / `node-2.json` | 21 KB | Per-room LoRA adapters (swap for new rooms) | ```bash # Download and use (Python) pip install huggingface_hub huggingface-cli download ruv/ruview --local-dir models/ # Or use directly with the sensing pipeline node scripts/train-ruvllm.js --data data/recordings/*.csi.jsonl # retrain on your own data node scripts/benchmark-ruvllm.js --model models/csi-ruvllm # benchmark ``` **Benchmarks (Apple M4 Pro, retrained on overnight data):** | What we measured | Result | Why it matters | |-----------------|--------|---------------| | **Presence detection** | **100% accuracy** | Never misses a person, never false alarms | | **Inference speed** | **0.008 ms** per embedding | 125,000x faster than real-time | | **Throughput** | **164,183 embeddings/sec** | One Mac Mini handles 1,600+ ESP32 nodes | | **Contrastive learning** | **51.6% improvement** | Strong pattern learning from real overnight data | | **Model size** | **8 KB** (4-bit quantized) | Fits in ESP32 SRAM — no server needed | | **Total hardware cost** | **140** | ESP32 (9) + Cognitum Seed (https://cognitum.one) (131) | ### 17 Sensing Applications (v0.6.0) Health, environment, security, and multi-frequency mesh sensing All applications run from a single ESP32 + optional Cognitum Seed. No camera, no cloud, no internet. Health & Wellness: | Application | Script | What it detects | |————|––––|––––––––| | Sleep Monitor | node scripts/sleep-monitor.js | Sleep stages (deep/light/REM/awake), efficiency, hypnogram | | Apnea Detector | node scripts/apnea-detector.js | Breathing pauses >10s, AHI severity scoring | | Stress Monitor | node scripts/stress-monitor.js | Heart rate variability, LF/HF stress ratio | | Gait Analyzer | node scripts/gait-analyzer.js | Walking cadence, stride asymmetry, tremor detection | Environment & Security: | Application | Script | What it detects | |————|––––|––––––––| | Person Counter | node scripts/mincut-person-counter.js | Correct occupancy count (fixes #348) | | Room Fingerprint | node scripts/room-fingerprint.js | Activity state clustering, daily patterns, anomalies | | Material Detector | node scripts/material-detector.js | New/moved objects via subcarrier null changes | | Device Fingerprint | node scripts/device-fingerprint.js | Electronic device activity (printer, router, etc.) | Multi-Frequency Mesh (requires --hop-channels provisioning): | Application | Script | What it detects | |————|––––|––––––––| | RF Tomography | node scripts/rf-tomography.js | 2D room imaging via RF backprojection | | Passive Radar | node scripts/passive-radar.js | Neighbor WiFi APs as bistatic radar illuminators | | Material Classifier | node scripts/material-classifier.js | Metal/water/wood/glass from frequency response | | Through-Wall | node scripts/through-wall-detector.js | Motion behind walls using lower-frequency penetration | All scripts support --replay data/recordings/*.csi.jsonl for offline analysis and --json for programmatic output. ### What’s New in v0.5.5 Advanced Sensing: SNN + MinCut + WiFlow + Multi-Frequency Mesh v0.5.5 adds four new sensing capabilities built on the ruvector (https://github.com/ruvnet/ruvector) ecosystem: | Capability | What it does | ADR | |———–|———––|—–| | Spiking Neural Network | Adapts to your room in <30s with STDP online learning — no labels, no batches, 16-160x less compute | ADR-074 | | MinCut Person Counting | Stoer-Wagner min-cut on subcarrier correlation graph — fixes #348 (was always 4, now correct) | ADR-075 | | CNN Spectrogram Embeddings | Treat CSI as a 64×20 image → 128-dim embedding for environment fingerprinting (0.95+ similarity) | ADR-076 | | WiFlow SOTA Architecture | TCN + axial attention + pose decoder → 17 COCO keypoints, 1.8M params (881 KB at 4-bit) | ADR-072 | | Multi-Frequency Mesh | Channel hopping across 6 bands, neighbor WiFi as passive radar illuminators | ADR-073 | bash # Live RF room scan (spectrum visualization) node scripts/rf-scan.js --port 5006 --duration 30 # Correct person counting (fixes #348) node scripts/mincut-person-counter.js --port 5006 # SNN real-time adaptation node scripts/snn-csi-processor.js --port 5006 # CNN spectrogram embeddings node scripts/csi-spectrogram.js --replay data/recordings/*.csi.jsonl # WiFlow 17-keypoint pose training node scripts/train-wiflow.js --data data/recordings/*.csi.jsonl # Enable channel hopping on ESP32 python firmware/esp32-csi-node/provision.py --port COM9 --hop-channels "1,6,11" Validated benchmarks: | Metric | v0.5.4 | v0.5.5 | |––––|––––|––––| | Person counting | Broken (always 4) | Correct (MinCut, 24/24) | | WiFi channels | 1 | 6 (multi-freq hopping) | | Null subcarriers | 19% blocked | 16% (frequency diversity) | | Pose model | 16K params (FC only) | 1.8M params (WiFlow) | | Online adaptation | None | <30s (SNN STDP) | | Fingerprint dims | 8 | 128 (CNN spectrogram) | | Multi-node fusion | Average | GATv2 attention | | New scripts | 0 | 15+ | | New ADRs | 3 | 8 (069-076) | ### What’s New in v0.5.4 Cognitum Seed Integration + Camera-Free Pose Training v0.5.4 transforms RuView from a real-time sensing tool into a persistent edge AI system. Your ESP32 now remembers what it senses, learns without cameras, and proves its data cryptographically. | Capability | Details | Hardware | |———–|———|–––––| | Persistent vector store | Every sensing event stored as searchable 8-dim vector in RVF format | ESP32 + Cognitum Seed (https://cognitum.one) (140) | | **kNN similarity search** | "Find the 10 most similar states to right now" — anomaly detection, fingerprinting | Cognitum Seed | | **Witness chain** | SHA-256 tamper-evident audit trail for every measurement (1,747 entries validated) | Cognitum Seed | | **Camera-free pose training** | 17 COCO keypoints from 10 sensor signals — PIR, RSSI triangulation, subcarrier asymmetry, vibration, BME280 | 2x ESP32 + Seed | | **Pre-trained model** | 82.8 KB (8 KB at 4-bit quantization), 100% presence accuracy, 0 skeleton violations | Download from release | | **Sub-ms inference** | 0.012 ms latency, 171,472 embeddings/sec on M4 Pro | Any machine with Node.js | | **SONA adaptation** | Adapts to new rooms in <1ms without retraining | ruvllm runtime | | **LoRA room adapters** | Per-node fine-tuning with 2,048 parameters per adapter | Automatic | | **114-tool MCP proxy** | AI assistants (Claude, GPT) query sensors directly via JSON-RPC | Cognitum Seed | | **Multi-frequency mesh** | Channel hopping across ch 1/3/5/6/9/11 — neighbor WiFi as passive radar | 2x ESP32 (18) | | RF room scanner | Real-time spectrum visualization: nulls, reflectors, movement, multipath | node scripts/rf-scan.js | | Security hardened | Bearer tokens, TLS, source IP validation | Cognitum Seed |