AI is now being used to catch forged and deepfake documents in visa processing. Where does this go wrong?

Visa processing is starting to use AI to screen documents. Earlier this month, BLS International, which runs visa and consular services for governments in over 100 countries, announced an integration with Signzy, an identity-verification company. The stated capabilities are: Document intelligence: scanning submitted documents for anomalies that could point to manipulation Deepfake detection: flagging material that has been AI-generated or digitally altered The operating model they describe is: technology flags, people verify, governments decide. Final visa decisions stay with governments. What the announcement doesn’t include is any numbers: no accuracy rates, false positive rates or effect on processing times. That’s normal for an integration announcement, but it means the interesting questions are still open. Where I think this could go wrong: 1. False positives land on real people. A bank statement re-saved as a PDF, a photo scanned on a cheap printer, a stamp that’s faded. To a model, these can all look like tampering. Even a 1% false flag rate at visa scale means thousands of genuine applicants getting extra scrutiny. 2. “Human review” can become a rubber stamp. If reviewers see a red flag next to a file and have 90 seconds per case, how often do they overrule the machine? The safeguard only works if reviewers have the time and authority to disagree. 3. Uneven document quality across countries. Documents from some countries or issuing authorities are less standardised. A model trained mostly on one kind of document may flag others more often. 4. The arms race. The tools for generating fake documents are improving as fast as the tools for detecting them. A detection system is only as good as its last update. 5. Privacy. Document and biometric data is about as sensitive as data gets. Where it is processed, how long it’s kept and who audits the model all matter as much as detection accuracy. Where it clearly helps: Manual checks are slow and inconsistent, and reviewers get tired. A system that catches obvious manipulation early lets people spend their time on the hard cases. In theory, that’s faster for honest applicants too. Question for the sub: How should organisations balance faster automated checks with human verification? And what would you want disclosed (error rates, audits, appeal routes) before trusting a system like this? Source in the first comment.