The gap isn’t that AI security tools are bad, it’s that two good ones can’t agree on what they found

Reddit r/ArtificialInteligence Tools

Summary

The post highlights how independent AI security scanners name the same behavioral vulnerabilities differently, creating tracking and audit overhead. It introduces AVE, an open-source taxonomy of stable IDs for agentic AI vulnerability classes, noting that an independent developer's scanner findings converged on the same IDs.

Different angle than my last post here, this one's less about the philosophical shift and more about a concrete, boring-sounding problem that turns out to matter a lot in practice. Two separate security scanners, checking the same MCP server, can both correctly flag the same underlying issue and give it two completely different names. Neither tool is wrong. There's just nothing forcing independent teams to agree on what to call a behavioral pattern once they've found it. Once you're running more than one tool in a pipeline, and most serious setups do, this stops being a curiosity and becomes real overhead: you can't track a finding consistently, can't tell an auditor two alerts are the same issue, can't build a risk register that doesn't double-count. Conventional software solved this decades ago and nobody thinks about it anymore. A SQL injection gets a CVE ID, maps to a CWE category, every tool that finds it afterward references the same thing. Agentic AI components never had that, not because nobody thought of it, but because CVE anchors to a package and version and CWE describes a weakness in code, and neither has a slot for a behavioral pattern that isn't tied to either. We built AVE as an attempt at that missing layer, stable IDs for distinct behavioral vulnerability classes, 70 records now, severity scored against OWASP's own AIVSS framework rather than something invented for this. The part that actually made me trust it holds up beyond my own head, an independent developer built a completely unrelated static config auditor, crosswalked his own findings against this taxonomy on his own initiative, then tested it directly against the reference scanner on the same files. No shared code between the two tools at all. Most of the overlapping findings converged on the identical ID, unprompted. That's a stronger signal than anything either of us could have written about the project ourselves. github.com/aveproject/ave if useful. Curious whether the naming-fragmentation problem is something people here have actually run into with multiple tools, in this space or elsewhere.
Original Article

Similar Articles

AI and hackers - bad?

Reddit r/ArtificialInteligence

A discussion questioning whether AI's ability to find software bugs is a problem or an opportunity for companies like Google and Microsoft to proactively fix vulnerabilities.

AI research tools are still too eager to turn public signals into certainty

Reddit r/artificial

The author critiques AI research tools for overconfidence in weak signals, praising Komo AI's rapid discovery and source-attached summaries but highlighting the need for better uncertainty and contradiction handling. They describe a workflow that splits discovery, verification, and structured checking across multiple AI tools.

AI is breaking two vulnerability cultures

Hacker News Top

AI is disrupting traditional vulnerability disclosure cultures (coordinated disclosure vs. bugs-are-bugs) by accelerating the detection and exploitation of security flaws, making long embargoes less effective and forcing a need for faster, AI-assisted responses.