Launch HN: Respan (YC W24) – Open-source Rust AI gateway built around plugins
The developers of Respan have launched an open-source Rust-based AI gateway designed to simplify the management of multiple AI providers through a plugin-based architecture. The project aims to solve the maintenance challenges associated with large, monolithic AI gateways like LiteLLM.
Why it matters
As organizations scale their AI infrastructure, modular and maintainable gateway solutions are becoming critical for managing diverse LLM integrations.
Hey HN, this is Andy from Respan ( ). We are building an open-source Rust AI gateway. It is a decoupled gateway, so everything is a plugin, including different providers, protocols (MCP, voice), modalities (images, videos), and gateway features (fallbacks, PII redaction, smart routing). Instead of packaging tons of providers and features into one gateway, Respan provides a small core and public interfaces that teams can use to assemble the AI infrastructure their application needs. That way, users can build their own features as additional plugins without having to change the gateway's core. Here’s a video demo: There are countless LLM gateways and most of them solve a common problem: calling different providers from a single endpoint. However, adding new features can become prohibitively difficult. So developers can end up building another gateway to solve a specific problem.
Get smarter about the news
Sign up free for a feed built around what you actually care about, Dive Deeper research on any story, and the full text of every article.
Create free accountAlready have an account? Sign in