Tag
This paper introduces a biologically motivated neural architecture that respects Dale's constraint—neurons and synapses have fixed sign—while still supporting backpropagation-like learning using non-negative error signals and local Hebbian updates, showing gains on Tiny ImageNet.
The Bit-Mass Theory proposes that the total number of weight bits determines model accuracy, not the computation format, with experiments on MNIST showing equivalent performance between binary and floating-point networks at the same bit-mass.
A side project presents a Hebbian architecture AI model that avoids backpropagation and gradients, achieving 50 epochs on CIFAR-10 with emergent behaviors like accuracy dips followed by jumps and recovery after targeted damage.
HeLa-Mem is a bio-inspired memory architecture for LLM agents that models memory as a dynamic graph using Hebbian learning dynamics, featuring episodic and semantic memory stores to improve long-term coherence. Experiments on LoCoMo show superior performance across question categories while using fewer context tokens.