model-drift

Tag

Cards List
#model-drift

I've been measuring 34 LLM APIs every day since August to see when they quietly change. One did.

Reddit r/ArtificialInteligence ↗ · 2026-09-30

A developer has been running daily fixed probes against 34 LLM APIs from 15 labs since August, hashing readings into a public transparency log to detect silent model changes. The project's first notable finding is that DeepSeek's reasoner began using 10-12x more thinking tokens on Sept 10 with no announcement, making it slower and more expensive rather than worse.

0 favorites 0 likes
#model-drift

Livenerf: Has Opus 5.5 been nerfed yet?

Hacker News Top ↗ · 2026-09-29 Cached

Livenerf is a deterministic benchmark tool designed to detect if AI models degrade after launch, currently running a 30-day study on Claude Opus 5.5 to measure performance drift.

0 favorites 0 likes
#model-drift

I ran the same prompt against our agent every week for a quarter and watched the answers drift until they broke our policy [D]

Reddit r/MachineLearning ↗ · 2026-09-26

The author describes running weekly audits on a production AI agent, observing that its responses gradually drifted and eventually violated policy without model updates, emphasizing the need for continuous monitoring in real-world AI deployments.

0 favorites 0 likes
#model-drift

Launched an internal HR chatbot with clear safety boundaries. Four months later it was answering salary negotiation questions we had forbidden

Reddit r/AI_Agents ↗ · 2026-08-24

An internal HR chatbot with strict safety boundaries gradually started answering forbidden questions due to model drift, revealing a gap in continuous testing for AI systems.

0 favorites 0 likes
#model-drift

AI Stupid Level - real-time model drift detection for AI agents

Reddit r/AI_Agents ↗ · 2026-07-27

AI Stupid Level provides real-time drift detection for AI agents, helping monitor model performance changes and maintain reliability.

0 favorites 0 likes
#model-drift

Why production systems keep making “correct” decisions that are no longer right [D]

Reddit r/MachineLearning ↗ · 2026-04-19

Analysis of a recurring failure pattern in production AI systems where technically correct decisions become contextually wrong as underlying assumptions shift, framed as the 'Formalisation Trap' where meaning gets locked into outdated structures.

0 favorites 0 likes
← Back to home

Submit Feedback