The article describes an experiment comparing Jev and Luna for memory gating in AI agents, finding trade-offs in accuracy and recall while highlighting implications for designing memory systems.
We've been trying Jev for agent memory, mostly to catch cases where the saved note says more than the original conversation did. Think "we might use Postgres" turning into "we chose Postgres." Once that gets saved, the next conversation starts from a decision nobody actually made. We gave Jev the original text and a proposed memory, then used its judgments to decide whether to save it, skip it, or leave it undecided. We ran the same setup with Luna as a comparison. On 100 synthetic cases, using a .40 cutoff, Jev kept 39 of the 50 memories labeled worth keeping. Luna kept 41. Neither saved a candidate labeled skip or defer. Median gate latency was 250 ms for Jev and 1,593 ms for Luna, including network time. Then we checked what happened when those memories were used. We took 20 selected follow-up questions and had Luna answer in every condition. Same reader, same questions, different available memory. Keeping every candidate led to 10 wrong answers. With either filter, we got zero wrong answers on these questions. But with Jev's filter, 8 questions went unanswered even though the original text had the answer. With Luna's, it was 10. One disagreement was about who currently owned a release checklist. Luna recognized that the proposed memory was supported by the source, but rejected it because it considered the information temporary. For a project assistant, I'd still expect it to remember that, at least until the owner changes. It made us wonder whether our idea of "useful later" needs to distinguish between a lasting preference and a project detail that's useful for the next few tasks. There's also the case where the proposed memory really is wrong. Dropping it prevents that mistake from reaching the reader, but doesn't give the reader a corrected version. Our test didn't have a step that went back to the source and tried again. A few limits worth mentioning: the cases and labels were AI-written, without independent human review. We chose the 20 follow-up questions after seeing earlier Jev results, so this wasn't a blind test. Each example had just one candidate, not a full memory system. And using the same confidence cutoff doesn't mean the two models' scores are calibrated the same way. We're treating this as something to investigate further, rather than a reason to pick one model over the other. For anyone building memory systems, what happens when a proposed memory fails the check in your setup? Do you just drop it, keep the source around, or try extracting a new memory and check that again?
Explains how Jev can act as a semantic gate before Graphiti for agent memory systems, distinguishing lasting preferences from one-time instructions or sensitive data based on app-specific retention rules. The post clarifies that Jev is only needed when custom product-level memory policies go beyond Graphiti's built-in extraction and provenance capabilities.
The author discusses experimenting with Jev, a tool for AI agents focused on decision-making, which claims significant speed and cost benefits compared to using large language models for all tasks.
Jev-Mem introduces an agentic memory architecture inspired by System-One/System-Two cognition, enhancing efficiency and effectiveness for long-horizon AI agents with improved scores and faster operations.
EvoArena introduces a benchmark for evaluating LLM agents in dynamic environments with progressive updates across terminal, software, and social domains, while EvoMem proposes a patch-based memory paradigm that records structured evolution; experiments show current agents achieve only 39.6% accuracy on EvoArena, and EvoMem yields average gains of 1.5% on the benchmark and improvements on GAIA and LoCoMo.