Prevent CUDA OOM in PyTorch with dynamic lane switching

I built MEM v3 to solve a frustrating problem in PyTorch: CUDA Out-of-Memory crashes during long training and fine-tuning runs. Instead of restarting when memory spikes or keeping batch sizes overly small just to be safe, MEM acts as a memory governor. It watches VRAM and throughput in real-time, then dynamically adjusts batch size and gradient accumulation on the fly without stopping the process. What it does: - Dynamic lane switching: Scales batch size up or down in milliseconds based on actual GPU memory pressure. - Chaos resistance: Tested against sudden +10 GB VRAM allocation shocks without crashing. - Crash-proof checkpoints: Uses atomic file replacement with SHA-256 checks across rotating slots, so power outages won't corrupt saved weights. - Live telemetry: Built-in local web dashboard to track loss, throughput, and lane switches. You can test it directly on a free Colab GPU without setting anything up locally: https://colab.research.google.com/github/nobazzy/mem-llm-orchestrator/blob/main/notebooks/mem_orchestrator_interactive_demo.ipynb Repo: https://github.com/nobazzy/mem-llm-orchestrator Would love to hear your thoughts and feedback!