black-box-audit

Ventor-QTest proposes a black-box audit method for vendor-hosted LLM APIs, using repeated and long-sequence probes to measure fidelity loss and detect degradation in long-horizon agentic tasks.

This paper introduces a framework using generative AI agents to automate black-box audits of personalization algorithms, demonstrating with 1,120 agents on X after the 2024 U.S. election that the algorithmic feed amplifies toxic, polarizing, and right-leaning content compared to the chronological feed.