A symbolic engine that refuses instead of guessing

Reddit r/artificial Tools

Summary

Chiron is an exact-or-refuse evidence gate for structured outputs that verifies claims as VERIFIED, REFUTED, or REFUSED, with a public evaluation history and source-available code.

Most AI evaluation ends with a score. That is useful, but it does not answer a narrower question: is this particular output strong enough to release without asking a person again? I built Chiron for the cases where the answer can be established exactly. It is an exact-or-refuse evidence gate for supported structured outputs. For sequence-style data, Chiron recovers a constrained candidate rule and then tests it on held-out terms it did not see. Exact prediction earns verified: true; a formula that merely fits the visible data stays a candidate. For supported claims in text, it reports each claim as VERIFIED, REFUTED, or REFUSED, then shows the coverage boundary. The surrounding free-form text is not silently approved just because one check passed. Try a non-sensitive example here: https://jiannotti5040.github.io/chiron/ The public evidence is deliberately bounded and runnable. The current frozen external evaluation has: 22 stamped outputs 22 externally correct 0 false stamps 12 refusals The failure history is public too: an earlier 109-sequence sweep found 3 false stamps. Those failures were published, fixed at the root, and re-run. I think that is more useful than presenting a flawless-looking system with no stated way to be falsified. eval/grade.py grades those frozen outputs against OEIS ground truth, and challenge.py lets you choose sequences. The protocol is here: https://github.com/jiannotti5040/chiron/tree/main/eval This is not a claim to certify arbitrary prose, replace experts, or solve generic "AI safety." It is meant to sit beside tracing, LLM judges, and broader evals at the point where a supported result needs an exact release condition—or an explicit stop. The repo is source-available for noncommercial use; the full engine is commercially licensed for organizational deployment. I would especially value counterexamples, scope objections, and examples of workflows where an exact-or-refuse gate should not be used.
Original Article

Similar Articles

ScientistOne: Towards Human-Level Autonomous Research via Chain-of-Evidence

arXiv cs.AI

ScientistOne introduces Chain-of-Evidence, a verifiability framework for autonomous research agents that ensures every claim is traceable to evidence, achieving zero hallucinated references, perfect score verification, and the highest method-code alignment across 75 papers while matching or exceeding human expert performance on frontier research tasks.

Beyond a Single Direction: Chain-of-Thought Disrupts Simple Steering of Refusal

arXiv cs.AI

This paper investigates how chain-of-thought reasoning in large reasoning models complicates activation-based steering of refusal behavior. Experiments on DeepSeek-R1-Distill-LLaMA-8B show that refusal is jointly encoded in residual stream activations and the CoT trace, making models more robust to activation-level interventions but exposing the CoT as an alternative attack surface.