@rohanpaul_ai: Netflix has explained the system behind those short "because you watched" lines: an AI writes them, another AI grades t…

Netflix has explained the system behind those short "because you watched" lines: an AI writes them, another AI grades them, and humans audit weekly. An AI judge is usually validated once and then trusted forever. New Netflix paper argues a judge running in production has a lifecycle and needs maintaining like any other model. At Netflix, one model writes the short lines telling members why a title was recommended, and another scores every one before it is shown. Netflix splits that work into four stages: building labelled examples, tuning the judge, running it as a gate, and watching it for drift. Checking whether the judge agrees with human labels is not enough. It also has to reject a bad explanation for the same reason a person would, so tuning runs on written rationales rather than pass-fail marks. A weekly human review then sets the bar by how far the raters disagree among themselves. In a 5-week test against no explanation at all, members shifted slightly toward titles they had not watched and more often ended a browse by playing something. – arxiv. org/abs/2608.18300 Title: "The Lifecycle of LLM-as-a-Judge for Large-Scale Recommendation Explanations"