Tag
The Bit-Mass Theory proposes that the total number of weight bits determines model accuracy, not the computation format, with experiments on MNIST showing equivalent performance between binary and floating-point networks at the same bit-mass.
A side project presents a Hebbian architecture AI model that avoids backpropagation and gradients, achieving 50 epochs on CIFAR-10 with emergent behaviors like accuracy dips followed by jumps and recovery after targeted damage.
HeLa-Mem is a bio-inspired memory architecture for LLM agents that models memory as a dynamic graph using Hebbian learning dynamics, featuring episodic and semantic memory stores to improve long-term coherence. Experiments on LoCoMo show superior performance across question categories while using fewer context tokens.