I was more skeptical of openrouter a year ago. Not hard to implement multiple models, but after getting burned multiple times (eg Google in particular) I see openrouter as serving a few key functions: (1) aggregating demand across neoclouds for performance etc (2) providing a great developer experience (3) easily switching models and normalizing the idiosyncratic nature of each. There are so many (good) model providers today this is more compelling now with 10+ solid model companies than a year ago when I thought it might be just 3.
So I have shifted load away from calling some models to openrouter.
This is really interesting, because I'm actively building an open source SaaS router product. We're starting in the FAL area rather than the OpenRouter area (much higher margin), but they eventually intersect.
My thinking is businesses will want the ability to fire their provider and bring things on-prem if they want (but that they'd rather stick with a good provider as long as they're happy). Not so much that they'll do it, more that they'll want the option.
Flexibility isn't super important now as most routers are fungible, but as we move into an era dominated by open source models and fine tunes, that becomes a major impediment to switching. So building this now seems especially important. Especially if we guarantee data and model exports to customers.
A customer with fine tunes of open source models won't find it simple and easy to migrate. But we can build this escape hatch now before that story has a chance to evolve.
And of course there's a lot of add-ons that can be sprinkled on top later: handling customer data/outputs (videos, thumbnails, etc.), spinning up domains for products (eg. Three.js video game LLM outputs), etc. Vercel/Itch.io as a service, Vimeo as a service, media catalogue search as a service, etc. It'd be nice if this was all open source and portable.
Is anyone here interested in a closed beta of a Rust-based "Open" OpenRouter, Fal, (and eventually RunPod and Featherless)? We host it for you, but the entire stack is OSS and you can bring it on-prem if you want. I'd be happy to send invites. My email is "echelon at gmail".
We also do BYOK forwarding, so it's super simple to switch without even paying us. We even support router keys for double routing (OpenRouter and FAL keys in addition to OpenAI, Google, etc.)
Might need to rethink our Stripe payment processing. They have our customer book.
I would not be suprised to see Cloudflare launch a competitor in the next 6 mo. They've done similar before, and it fits their "middle man/service provider" vibe well.
OpenRouter is good because of the market ecosystem play. I see this fragmenting similar to game distribution -- a handful of 3rd-party disti, 1 first-party disti, and some exclusivity/lockup/limited support outside of first party.
I am very interested to see how Stripe develops this. I would be happy to see a less-clunky top-up solution, for example, perhaps with micropayments. Not sure what else they are planning.
Part of the issue -- cloudlfare is an inference provider for models (eg glm-4.7-flash) on openrouter. Very very hard to get partners to signup if you're going to directly compete with them.
It's pretty bad though compared to openrouter tbh. I expected it to have some inbuilt cloudflare access access control, but it doesn't have that for improving security either within the team Easier to just use openrouter tbh.
Basically if you look at OpenRouter in a isolation this could be confusing, but when bundled with Stripe's integrated product solutions for the lower end, developer centric market, this makes infitite sense.
https://s-1.vercel.app/posts/what-stripe-can-become-broader-...
Why do you need 10? One or two fallbacks is sufficent. Maybe my workloads are vastly different than yours but I’m not a huge fan of having to switch between models. Even within a single model determinism sucks, debugging why something worked yesterday and doesn’t work today because a model switch is not fun. I can’t imagine the pain of having our services flip randomly between 10 different models.
It is changing so often that to keep the costs and quality at a pareto level you need to experiment with a lot of different providers. And the pain you are describing should be a normal process in AI. I use model pools where the tasks are optimized for multiple AI providers. For example I was experimenting with GPT Luna and it turned out that the model is good but tool shy so I had to improve the instructions. Now this model is my main model for the chat in my app. Next week it can be a different model.
It might just not be for you. I use OpenRouter because I do like being able to quickly check whether a new model works better, but mine is a “human in the loop” dev process so it won’t ruin a day of batch processing or anything.
I do like being able to try eg GLM without having to set up a new account. It’s also nice that I don’t have to top up per-model accounts. I think I have a couple accounts with $7 in API credits sitting around.
Probably not something I would do for actual business processing, where the stability is dramatically more important than tinkering with new models.
Not exactly related, but I've seen people suggesting a new path for opens source funding that would be around tokens instead of cash. One might be able to pledge tokens towards a feature or fix instead of cash.
Even the suggestion of this absent an implementation hints at a future for tokens as a commodity. But it is interesting to think of it as some kind of new asset class, between commodity and currency.
I mean, we already think of tokens in the form of currency in crypto (e.g. NFT). There may be more than just similarity in the name choice when it comes to AI tokens. One could even think of things like "exchange rates", etc.
I both think OpenRouter is not worth 7B$ and Stripe might have made a brilliant move in making this purchase, honestly just troubling thoughts around model inference and model token as the new traded currency go through my head.
Imagine if Stripe's end game is to control which producers of tokens become visible for a price aka middleman tax for tokens. If AI is as big a change as people claim honestly AWS should have bought openrouter instead.
But either way though 7B$ is just too rich no matter what, because the middleman tax is only worth it if you have a moat which OpenRouter doesn't.
All it takes is one of the bigger players to get serious and they will have a similar platform up and working in months if not weeks. (like Vercel & Cloudflare are already doing but even they are small compared to the true behemoths)
If hyperscalers start their own open routing service and cut off openrouter completely and become price competitive, I feel like they could wipe them out.
7B$ is just too much. Without enough of a big defensible moat. Especially since you can use a router in-front of openrouter and move users piece meal.
Can anyone explain what makes OpenRouter specifically worth 7B$ over everyone else? I say that as someone who has used a lot of similar services with similar results. Openrouter is better but only marginally.
I specifically think many of the hyperscalers are a bad choice to platform on because they have their own models. You cannot build a viable platform business while directly competing with your potential customers and partners, the incentives are too out-of-whack.
Stripe/CF/Vercel are presumably not going to do foundational model development, so safe there. They are already platform plays to begin with, this strengthens that position.
OpenRouter is a two-sided inference market with an established network and spinning flywheel. If Stripe can manage not killing it, it is going to print them money for a long time.
If LLM providers continue grow and become such a standard part of the tech stack of any deployment, this acquisition makes a lot of sense. It's just another payment management system for stripe.
It's fun to see someone being able to exit so well without owning any model.
The acquisition makes no sense, as most LLM providers only reason not to forbid them right now, is because they were funded by exactly the same VCs as the providers. Go check.
Of all the no moats they are the example of no moat. Its seconds switch to another router if even that is tolerated. And lets not even talk, the lack of compliance in an enterprise environment, and as they defer to providers on that...
But the acquisition makes a LOT of SENSE, if you realize that as the same VCs sit on Stripe, Anthropic, OpenAI and most important OpenRouter. Said VCs have a lot of interest in recovering their money...by pushing one of their companies to buy another one of their own, before LLMs providers financial pressures cut OpenRouter legs.
Is this stuff even legal? I will wait for Patrick Boyle video...
> And lets not even talk, the lack of compliance in an enterprise environment, and as they defer to providers on that...
A middleman that adds compliance & regulation while skimming a slice off the top for themselves: You just described credit cards - stripe's core business...
You over-estimate the amount of sway investors have on Stripe. If VCs had the influence you're describing, Stripe would have gone public many years ago.
Stripe isn’t exactly a dumb company. When someone smart does something seemingly super dumb, more likely than not you’re the one missing something, not them. It’s an opportunity to ask, “what am I missing?”, and learn from it.
> I didn't say it was super dumb, it was more of a reflection on that's how much a wrapper is worth today.
Are you sure that the value is in the wrapper and not rather in the fact that investors who invested into OpenRouter also invested into companies that can buy OpenRouter? ;-)
It is not just the software. Let's not forget the existing customer base, an opportunity cost to build something as mature as openrouter (even with LLMs). Perhaps it is worth it for them.
I guess hardly anybody claims that. I think the suspicion is rather about whether there exist some other "puppetmasters" (e.g. venture capitalists) who pressured Stripe into buying OpenRouter.
That's just one take on the situation. What makes up that bundle is what I'm referring to. They aren't just blindly buying companies. Why exactly is OR interesting to Stripe?
So I have shifted load away from calling some models to openrouter.
My thinking is businesses will want the ability to fire their provider and bring things on-prem if they want (but that they'd rather stick with a good provider as long as they're happy). Not so much that they'll do it, more that they'll want the option.
Flexibility isn't super important now as most routers are fungible, but as we move into an era dominated by open source models and fine tunes, that becomes a major impediment to switching. So building this now seems especially important. Especially if we guarantee data and model exports to customers.
A customer with fine tunes of open source models won't find it simple and easy to migrate. But we can build this escape hatch now before that story has a chance to evolve.
And of course there's a lot of add-ons that can be sprinkled on top later: handling customer data/outputs (videos, thumbnails, etc.), spinning up domains for products (eg. Three.js video game LLM outputs), etc. Vercel/Itch.io as a service, Vimeo as a service, media catalogue search as a service, etc. It'd be nice if this was all open source and portable.
Is anyone here interested in a closed beta of a Rust-based "Open" OpenRouter, Fal, (and eventually RunPod and Featherless)? We host it for you, but the entire stack is OSS and you can bring it on-prem if you want. I'd be happy to send invites. My email is "echelon at gmail".
We also do BYOK forwarding, so it's super simple to switch without even paying us. We even support router keys for double routing (OpenRouter and FAL keys in addition to OpenAI, Google, etc.)
Might need to rethink our Stripe payment processing. They have our customer book.
OpenRouter is good because of the market ecosystem play. I see this fragmenting similar to game distribution -- a handful of 3rd-party disti, 1 first-party disti, and some exclusivity/lockup/limited support outside of first party.
I am very interested to see how Stripe develops this. I would be happy to see a less-clunky top-up solution, for example, perhaps with micropayments. Not sure what else they are planning.
Recent overview: https://blog.cloudflare.com/ai-platform/
Payment processing for tokens - sounds like the thing Stripe does.
I do like being able to try eg GLM without having to set up a new account. It’s also nice that I don’t have to top up per-model accounts. I think I have a couple accounts with $7 in API credits sitting around.
Probably not something I would do for actual business processing, where the stability is dramatically more important than tinkering with new models.
Even the suggestion of this absent an implementation hints at a future for tokens as a commodity. But it is interesting to think of it as some kind of new asset class, between commodity and currency.
I mean, we already think of tokens in the form of currency in crypto (e.g. NFT). There may be more than just similarity in the name choice when it comes to AI tokens. One could even think of things like "exchange rates", etc.
Imagine if Stripe's end game is to control which producers of tokens become visible for a price aka middleman tax for tokens. If AI is as big a change as people claim honestly AWS should have bought openrouter instead.
But either way though 7B$ is just too rich no matter what, because the middleman tax is only worth it if you have a moat which OpenRouter doesn't.
All it takes is one of the bigger players to get serious and they will have a similar platform up and working in months if not weeks. (like Vercel & Cloudflare are already doing but even they are small compared to the true behemoths)
If hyperscalers start their own open routing service and cut off openrouter completely and become price competitive, I feel like they could wipe them out.
7B$ is just too much. Without enough of a big defensible moat. Especially since you can use a router in-front of openrouter and move users piece meal.
Can anyone explain what makes OpenRouter specifically worth 7B$ over everyone else? I say that as someone who has used a lot of similar services with similar results. Openrouter is better but only marginally.
Stripe/CF/Vercel are presumably not going to do foundational model development, so safe there. They are already platform plays to begin with, this strengthens that position.
edit: Fixed the poor AM English.
(I switched to a major bank who offers much better pricing and integrates the Clover API.)
It's fun to see someone being able to exit so well without owning any model.
Of all the no moats they are the example of no moat. Its seconds switch to another router if even that is tolerated. And lets not even talk, the lack of compliance in an enterprise environment, and as they defer to providers on that...
But the acquisition makes a LOT of SENSE, if you realize that as the same VCs sit on Stripe, Anthropic, OpenAI and most important OpenRouter. Said VCs have a lot of interest in recovering their money...by pushing one of their companies to buy another one of their own, before LLMs providers financial pressures cut OpenRouter legs.
Is this stuff even legal? I will wait for Patrick Boyle video...
A middleman that adds compliance & regulation while skimming a slice off the top for themselves: You just described credit cards - stripe's core business...
Are you sure that the value is in the wrapper and not rather in the fact that investors who invested into OpenRouter also invested into companies that can buy OpenRouter? ;-)
(see also https://news.ycombinator.com/item?id=49330668 ).
I guess hardly anybody claims that. I think the suspicion is rather about whether there exist some other "puppetmasters" (e.g. venture capitalists) who pressured Stripe into buying OpenRouter.