Building a proxy that serves clean HTML to AI agents and crawlers: looking for feedback (I won’t promote)

Reddit r/AI_Agents Products

Summary

The author is seeking feedback on a self-serve proxy service designed to enhance SEO and AI agent readability by serving clean HTML, dynamically adding meta tags, and offering analytics.

Hi all! During the last two years I’m running a B2B SaaS that helps websites with SEO, increase organic traffic, and get discovered by AI chats (SEO/AEO/GEO). Disclosure: this is feedback request only. No links, no brand names, no promo. B2B works well, but it pains to scale, has long sales cycles, onboarding, manual configuration and integration. It funded a team and brought us some savings. Now I want to make it self-serve SaaS for a broader public. Our main shift from expensive manual integration to integration in a few clicks. That changed service a lot and it has to sit as a proxy in front of a website, which means issuing TLS certs, managing DNS, and operating as a CDN. The SEO tool became edge infrastructure at that moment. What it does: - Fast discovery by crawlers and AI agents: Serve fast, clean HTML to AI agents and crawlers so new sites are readable by LLMs from day one - Add meta tags on the fly: Fill missing metadata in responses to bots, crawlers, and agents - Speed up discovery in search engines: Automatic IndexNow submission of new and changed URLs - AI agents analytics and insights: Cookie-free, JS-free visitor analytics for crawlers, bots, and AI-agents. Questions: 1. Is “readable by AI + indexed fast” a real pain for new projects, or a nice-to-have? 2. Would you put a proxy/CDN in front of your site for this? 3. What would you expect from a free tier? View Poll
Original Article

Similar Articles

Free AI SEO Auditor

Product Hunt

A 100% open-source utility designed to audit websites for optimization in the emerging AI search landscape, currently featured on Product Hunt.