I'm wondering, is there a tool or something out there that helps me pick a model, in the vast sea of models out there these days? Every time I need a model for something I see the list on openrouter and I'm completely overwhelmed.
I'd love to be able to explain my use case, my cost preferences and have a tool select a few good models to try.
E.g. I wrote a tool that cleans out my email spam box. It classifies emails that are already flagged as spam, and if it's very obviously spam it removes it permanently (keeps a copy on disk though). And after x emails, it goes through the list of deleted spam mails and suggests email rules. What model would be best suited? I'd love to be able to explain this use case and get this info served to me. The list of models and the information about what they're good at is just too splintered and spread out. I landed on google/gemma-4-31b for now, because it's cheap and good enough and also supports Dutch and French a bit. But I can't realistically try them all.
My approach to this problem is to just...not try them all.
As long as the model you're using solves the problems you have to your satisfaction, there is no need to try any other models, except for financial reasons maybe.
So I start with a relatively cheap model (GLM 5.3 flash for me) and as long as it accomplishes the task (it did so far) I don't have to change. And even if it can't do something, the first thing I change is see if I can give it more tools or better context (useful even if I switch models later) or trying a different approach to the problem.
If google/gemma-4-31b works, you don't need to overthink it.
Starting with GLM-5.3 Flash was a pretty decent first try! I started with other models, and ended up settling on this exact one because all the others were either too slow or unreliable for my tasks. Qwen 3.8 didn't do it for me, whatever tweaks I added to my harness. Where I'm getting at is you did start with an incredible model in the first place, which greatly helps sticking to it.
I use Gemini(s) because I can send pdfs as files to their API and not worry too much. I've started to diverge and consacrate a part of my pipeline to sending image based pdf pages to glm flash 5.3, not sure how to address / test it properly.
Long term I have fears I can't depend of the Google's AI api.
> If google/gemma-4-31b works, you don't need to overthink it.
Up until recently I had a gemini flash 2.0 api deployed that did summarization and translation of news articles/corporate statements fast and cheap and had no reason to update it.
If it works fine, this chase of the latest LLM is bit pointless.
It's not like new releases come with fully mapped out capability scores for exactly the aspects that you're interested in. There are benchmarks, but reality is often different. It's simply unknown to humanity how well each model will perform in your own bespoke context unless you just try them. You can read experiences and vibes by others but often they will use them in different ways or have different preferences etc.
They generally all try to make them good at everything, it's not like they'd declare "this model is not made for task X".
I use claude for work, and opencode go at home, but i do dabble with openrouter from time to time, and then i just browse the model catalog and filter/sort by recent popularity, price, context size or whatever matters for the task.
Mostly just popularity trends, hoping that there's some wisdom in the crowd.
For TTS I launched this like this week based on a Reddit thread of recommendations, added a new one to it.. I want to say on Wednesday, but this week has been a blur. Problem I’ve found with similar sites is I can’t run a lot of the models, or the results are beyond stale.
But this is only stuff I can run locally, or it’s a cloud model.
Yes, for fun I tried OVH AI Endpoint and they do not have cache read at all. They bill you every time you send a prompt regardless if you are hit cache or not. One agent session was like 80M input and 300K output and I paid 30$ for that. Or rather I interrupted it and let my local Qwen finish it because cost was getting radicoulous.
On some benchmarks models like Qwen 3.8 Max which cost < $6/m out cost more than Astra 6 to run at $50/m out. That’s a huge price gap and yet Astra would be cheaper if your work looks like the benchmark.
We notice with frontier models like Astra and Fable that one might use a lot less tokens than the other to complete the task thereby being the better deal in spite of the far higher token cost.
> audio-visual performance close to Gemini 3.8 Flash and overall audio performance that exceeds Gemini 3.8 Flash
Wow crazy if true. I think Gemini's audio capability and multi language was the "selling point" for a lot of people. Other capability also matches or exceeds 3.8 Flash.
They also made a new harness but github link seems to 404.
3.8 Max is the most “grounded” model I think - talks generally normal, doesn’t go crazy and start doing things (I see you Gemini), has good design choices and isn’t overly nitpicky. But god it’s slow. And only available from Alibaba. Their token plan is stingy too. If I had to pick the “old reliable boring” LLM, a modern Claude 4.5 if you will, Qwen is my choice. Hopefully they don’t RL it to oblivion.
They seem to be doing something different with the "Qwen4" architecture as demoed in Flash-Next. I've noticed the reasoning behaves ... weirdly. Like, really weirdly compared to any model I've ever seen before.
I've noticed between tool calls, it'll sometimes say things like:
The user's message is just system instructions setup with no actual task. There's no question to answer yet. I should acknowledge briefly and wait for the actual request.
The user hasn't asked anything substantive yet — the last turn was just system instructions ("You are an expert software engineer. Helps user to solve problems."). My previous response was a brief acknowledgment. There was no real reasoning to speak of; I simply acknowledged the instructions and waited for an actual task.
【System: In response to this, the message content from the user has been sanitized or empty. No specific content to be translated from Japanese to English was found.】
These don't clearly reflect ... anything, and it keeps performing tool calls correctly anyway. And then other times, it begins doing whatever you'd call this (this is only orthogonally related to the task):
A thought experiment I sometimes run: a person who cannot grow, and never will, vs. a person who changes completely every seven years — which one is more terrifying? I've decided that the latter is more terrifying. Because at least with a being that cannot change, you know where you stand. Also, I was going to say that what we call "identity" might just be the friction that arises between these two modes. But that's the sort of thing you end up saying at 2 AM. Anyway, that's what I thought.
This is a serving bug or quantization issue. I had all kinds of issues that were like this on DGX Spark until I found a single-GB10 vLLM recipe [1] that uses Nvidia's NVFP4 quant. The community quants did not work well.
Another failure mode you may see is inordinately long CoT. Properly served, the model is good at calibrating its CoT length to the difficulty of the immediate task.
I've experienced the same, IIRC also using NVIDIA's NVFP4 quant. Also just decided to ignore because it didn't seem to cause any real issues. I figure it might be a training thing, since the hallucinated user messages seem to occur immediately after tool calls or when it's checking its work.
Is this with the full unquantized weights? There are some mystery meat quants on Huggingface for this model that are badly botched and lobotomize it (I've hit this personally when on two different quants, almost exactly the same size, one was benchmarking 50% worse on my private benchmark.).
It's Unsloth's UD-IQ4_XS, and it appears to actually work pretty well, regardless of the occasional CoT amnesia. Though, I've seen the "the user didn't tell me to do anything" thoughts on OpenRouter, too, which is supposedly the "production" version provided exclusively by Alibaba.
Earlier today I was playing around with the "Union Alpha" stealth model (which I guess exited stealth later in the evening), and I noticed it had a habit of trying to respond to the subagents it spawned while giving me an answer. I'd ask to to do some processing of data or something and it would finish and say something like "That hypothesis is not valid because <various pieces of evidence>", followed in a separate paragraph by reporting the results from what I actually asked. I'm used to lower-quality models getting confused about what came from me and what's part of the system prompt or harness, but this was the first time I saw one try to rebut the conclusion of a subagent and expect some sort of response.
I saw some corrupting when using https://github.com/MiaAI-Lab/Qwen3.8-Flash-Next-Single-DGX-S... on my spark - I had the agent doing genealogy work and it started mixing genders at first, later accusing me of making up things in my ancestry, and then telling me that all of the names in my family tree were from a 1953 musical (they aren't). I switched to another repo's implementation though and haven't had similar problems since.
Isn't that perhaps that there's some underlying code that's looking back at skills files,etc as a way to avoid context dilution or similar issues? (and the refreshing shows up)
I understand the reasoning but I have a family member with almost this exact type of brain injury and its one of the worst things, therefore I personally would strongly disagree.
Flash-Next thinking also sometimes glitches out and takes minutes to return a simple answer, randomly, in my experience. You’ve gotta kill the request and send it again.
> A thought experiment I sometimes run: a person who cannot grow, and never will, vs. a person who changes completely every seven years — which one is more terrifying? I've decided that the latter is more terrifying. Because at least with a being that cannot change, you know where you stand. Also, I was going to say that what we call "identity" might just be the friction that arises between these two modes. But that's the sort of thing you end up saying at 2 AM. Anyway, that's what I thought.
This is what AI becoming self-aware looks like. /s Anyway, didn't OpenAI report the same thing with the model writing out weird musings about itself during compaction?
It’s absolutely down to their post-training RL, yeah. It’s where most of its strongest behaviour comes from, with regards to this kind of agentic behaviour
Reinforcement Learning (in LLMs) trains via gradient descent on a reward signal that's an imperfect proxy for the actual goal of the engineers doing the training. So, under mild optimization pressure, you get increasingly more of what you want, because that's the easiest way to increase the metric.
But as the optimization pressure increases, so do the ways to increase the metric by doing increasingly weird things. If the full action space grows sufficiently faster than the "things you actually want" subset, the amount of "things you actually want" goes to 0 under sufficient RL.
I think we are starting be on that territory that regular software development is suffering, current models are great for benchmarks and one-shots but in daily development models are too eager and try to force patterns like excessive tests in every turn.
The different names have different meanings and help make decisions on which to use.
Omni means you can use multiple types of input and have multiple types of outputs like audio, video, images and text. Flash means that it is built for speed and smaller than the more complete ones.
I believe in general flash models prioritise speed, ultra/pro/omni do more slow reasoning to the effect of sometimes better intelligence, and lite models prioritise cost.
The blog post itself is technically unimpressive, and feels like the slop future. Amongst many things
- Scrolling in firefox is a nightmare (only uBlock origin) and the console is full or warnings and debug data.
- Videos are all flashy but just fail to communicate anything beyond what can be said in a small paragraph (and with horrible stock music).
- Figure 1, the headpiece; too small to read, can't zoom in
Curious if or when we'll see the Qwen4 series, one thing I love with Qwen is it comes a much larger range of sizes so I can experiment which extremely small llms.
And maybe Qwen4 won't be released, they only release Qwen3.8 27B (and a mostly unusable 125B). There are definitively slowing down open weight release.
I assume you are talking about qwen3.8-flash-next. Support for it on some places, like llama.cpp, is still wip (depending on configuration) but it looks like a very capable model in it's category.
Qwen 3.8 Flash Next is amazing, i did hundreds of turns and billions of prefill and it may not be as smart as sota but then again it does what i tell it to and it does it well.
Qwen 3.8 Max was open-weight released [1], as was Qwen 3.8 Flash Next [2].
I still agree that they aren't as aggressively releasing the open-weights models as before, but there hasn't been a major release they haven't published the weights for yet afaik.
Surely it was meant to be 'definitely' - the "good news" at this stage are that given the speed of history and important levels of uncertainty, it is difficult to label trends with "definitively" ;)
Some would not have bet that the change of management at Qwen would have kept similar good results, but there we are, presumably satisfied. Other changes will happen, there or elsewhere - the situation is still very open.
And when the "40Watts Intelligence" (which we know possible) will be implemented... It will be a testimony that the current was only a middle-way, temporary, dynamic stage.
I am using Flash Next for few weeks and it is very capable model. I just wish there would a way to have faster prefill because reloading longer sections of session sometimes can take even 2h. I stopped using Qwen 3.8 27B completely on my Strix Halo.
No, it is actually very good. Qwen Flash 3.8 Next is fine. But you need ~128GB of RAM to get it going and not a lot of people have that or can serve it very quickly. I have been running it on an old gaming system around 25 t/s to do overnight work and it is very strong, even at 3 bit quant.
Why can the Chinese build models and Europe cannot? The algorithms behind this stuff are not that complicated, are they? Is it the cost of energy? The illegality and difficulty of obtaining all the data in the world? Lack of capital to start moonshot labs? Lack of optimism?
The Chinese just seem to have an ability to get it done without anywhere near the GPUs of the US and Europe can buy these GPUs.
I think relying on the US and China for AI is probably not ideal? For example I think Qwen have not released the Omni models as open weights in the past, it’d be good to know if they’re doing this here?
It's not that they can't, but why would they? It's a race to the bottom commodity wise, much smarter to build on top of the model layer like Legora and Lovable than make the models.
So the grand plan is to run US or Chinese models on hardware that you buy from US or China, and hope they'll keep selling us the hardware forever and giving away the models for free forever? Your moat is what? The prompts? The JavaScript for loop calling the model? Neat.
How many times will we do this? This line of thought is precisely why we can't build almost anything, and China can built almost everything. We could do X, but we're too smart - let's offload the actual work to China and we'll run our economy on IP and B2B deals.
In Europe AI is seen as evil that must be stopped. Unlike in the US, AI doomers have Commission well manned. Ursula addressed this in her State of Union speech yesterday. The EU rather chooses consumer protection, even if it means killing AI and maintaining status quo.
Because any AI company would be hit by hate and regulation derived from this, no VC invests in the EU. It's more state and large enterprise investments, good old East Germany style. And historically it has not been that efficient.
I agree with this, the rich actually are oddly MORE greedy in Europe and take double the equity for half the money (roughly) so founders get a lot less upside or they move their businesses to the US and get US capital at better rates.
The "Attention is all you need" paper authors where four (!) Europeans and one from US, amongst others. The "Top 10" lists are just complete rubbish, not worth the paper they are written on.
Europeans do not have the hustle mentality to break the law like Uber so you wont see them compete in anything data heavy. They also are quite risk adverse, probably from having a lower Gini coefficient.
ETH Zürich is up there high, particularly for computer science.
The LLM they were involved in last year (Apertus) still was a letdown.
As someone who actually went there, my impression is that Europe in general is complacent when it comes to computers, and any bright eyed student will get their motivation choked out of them in academia here.
If you want to do things with AI in Europe, you can have a bigger effect by working for a consulting company than being at a university. That's … not a good sitatution.
Simpler view for me: this is one of the most capital intense technologies to exist. Europe does not have enough capital to compete. China is building its own chips. It’s own everything. Silicon up. How do you compete with that. Only Google and maybe Aamazon is doing it domestically.
I feel people are too focused on US/China and don't pay attention to what is going on in the world.
ASML in the Netherlands for example was the only company in the world that makes EUV lithography machines, which all the major chip companies depend on. China recently reverse engineered their work to create machines since late 2025, but not sold commercially.
Ireland has chip production facilities.
EU might not be in the top 2, but it is not out of the running at all.
I'd love to be able to explain my use case, my cost preferences and have a tool select a few good models to try.
E.g. I wrote a tool that cleans out my email spam box. It classifies emails that are already flagged as spam, and if it's very obviously spam it removes it permanently (keeps a copy on disk though). And after x emails, it goes through the list of deleted spam mails and suggests email rules. What model would be best suited? I'd love to be able to explain this use case and get this info served to me. The list of models and the information about what they're good at is just too splintered and spread out. I landed on google/gemma-4-31b for now, because it's cheap and good enough and also supports Dutch and French a bit. But I can't realistically try them all.
As long as the model you're using solves the problems you have to your satisfaction, there is no need to try any other models, except for financial reasons maybe.
So I start with a relatively cheap model (GLM 5.3 flash for me) and as long as it accomplishes the task (it did so far) I don't have to change. And even if it can't do something, the first thing I change is see if I can give it more tools or better context (useful even if I switch models later) or trying a different approach to the problem.
If google/gemma-4-31b works, you don't need to overthink it.
Long term I have fears I can't depend of the Google's AI api.
Up until recently I had a gemini flash 2.0 api deployed that did summarization and translation of news articles/corporate statements fast and cheap and had no reason to update it.
If it works fine, this chase of the latest LLM is bit pointless.
(I think you're right)
Ask one of the top-tier models to do deep research on it.
https://openrouter.ai/docs/cookbook/coding-agents/openclaw-i...
They generally all try to make them good at everything, it's not like they'd declare "this model is not made for task X".
Mostly just popularity trends, hoping that there's some wisdom in the crowd.
But this is only stuff I can run locally, or it’s a cloud model.
So.. This is good for right now!
https://apimade.com/audio-compare.html
in/out ($) Gemini : 1.5 / 9.0 | Qwen 3.8: 0.15 / 0.47
That is a massive cost reduction.
Refs: https://www.alibabacloud.com/help/en/model-studio/model-pric... https://runware.ai/gemini-omni
Even if OpenAI end up using 1 token for per task, if the token costs 1M$ , some people will find it expensive.
Wow crazy if true. I think Gemini's audio capability and multi language was the "selling point" for a lot of people. Other capability also matches or exceeds 3.8 Flash.
They also made a new harness but github link seems to 404.
I've noticed between tool calls, it'll sometimes say things like:
These don't clearly reflect ... anything, and it keeps performing tool calls correctly anyway. And then other times, it begins doing whatever you'd call this (this is only orthogonally related to the task):Another failure mode you may see is inordinately long CoT. Properly served, the model is good at calibrating its CoT length to the difficulty of the immediate task.
[1] https://github.com/blazux/qwen3.8-Flash-DGX
Still works great though!
Is this with the full unquantized weights? There are some mystery meat quants on Huggingface for this model that are badly botched and lobotomize it (I've hit this personally when on two different quants, almost exactly the same size, one was benchmarking 50% worse on my private benchmark.).
PS: It would be ground breaking if it turns out to have been using Chinese chips for inference, like Stealth Ox Alpha. Unlikely though.-
Wow, that is unexpected. But honest?
What would that mean in this context?
I swear I spend more time telling Claude not to do things than telling it what to do.
Personally, I think this is a bad idea, but someone's gotta build the Machine God I guess.
But is that because of training, or can that be (also? mostly?) an effect of the "system prompt"?
Reinforcement Learning (in LLMs) trains via gradient descent on a reward signal that's an imperfect proxy for the actual goal of the engineers doing the training. So, under mild optimization pressure, you get increasingly more of what you want, because that's the easiest way to increase the metric.
But as the optimization pressure increases, so do the ways to increase the metric by doing increasingly weird things. If the full action space grows sufficiently faster than the "things you actually want" subset, the amount of "things you actually want" goes to 0 under sufficient RL.
Omni means you can use multiple types of input and have multiple types of outputs like audio, video, images and text. Flash means that it is built for speed and smaller than the more complete ones.
The last one was: Qwen3-Omni-30B-A3B https://huggingface.co/Qwen/Qwen3-Omni-30B-A3B-Instruct
And maybe Qwen4 won't be released, they only release Qwen3.8 27B (and a mostly unusable 125B). There are definitively slowing down open weight release.
I assume you are talking about qwen3.8-flash-next. Support for it on some places, like llama.cpp, is still wip (depending on configuration) but it looks like a very capable model in it's category.
I still agree that they aren't as aggressively releasing the open-weights models as before, but there hasn't been a major release they haven't published the weights for yet afaik.
[1] https://huggingface.co/Qwen/Qwen3.8-2.4T-A95B [2] https://huggingface.co/Qwen/Qwen3.8-Flash-Next
Surely it was meant to be 'definitely' - the "good news" at this stage are that given the speed of history and important levels of uncertainty, it is difficult to label trends with "definitively" ;)
Some would not have bet that the change of management at Qwen would have kept similar good results, but there we are, presumably satisfied. Other changes will happen, there or elsewhere - the situation is still very open.
And when the "40Watts Intelligence" (which we know possible) will be implemented... It will be a testimony that the current was only a middle-way, temporary, dynamic stage.
The Chinese just seem to have an ability to get it done without anywhere near the GPUs of the US and Europe can buy these GPUs.
I think relying on the US and China for AI is probably not ideal? For example I think Qwen have not released the Omni models as open weights in the past, it’d be good to know if they’re doing this here?
How many times will we do this? This line of thought is precisely why we can't build almost anything, and China can built almost everything. We could do X, but we're too smart - let's offload the actual work to China and we'll run our economy on IP and B2B deals.
Well there is Mistral. The EU is ahead on specialised models than general purpose LLMs.
There is.
- Flux3
- Kyutai (Open source AI lab)
- H Company
- LightOn
- AMD Silo (Finland)
- OpenEuroLLM and EuroLLM
There is probably more, but that's off the top of my head.
Black Forest Labs for example has a $4B valuation with half a billion raised so far.
Because any AI company would be hit by hate and regulation derived from this, no VC invests in the EU. It's more state and large enterprise investments, good old East Germany style. And historically it has not been that efficient.
Or simply: lack of risk taking appetite.
No?
Its just that the richest companys with the most VC sit in USA and Europe isn't used to pay what USA / VC is paying and we are a little bit slow.
We have no such government with a mandate to do that.
The main one (IIRC) is Spains ALIA.
https://alia.gob.es/eng
There is also OpenEuroLLM and EuroLLM.
https://www.openeurollm.eu
Europe actually follows americas rules, hence they're not doing that.
It's braindead for sure considering how the US treats europe, but it is what it's
Europe has none;
The best tech university in Europe when compared to Chinese/US equivalents won't even rank in the top 10.
Europeans do not have the hustle mentality to break the law like Uber so you wont see them compete in anything data heavy. They also are quite risk adverse, probably from having a lower Gini coefficient.
The LLM they were involved in last year (Apertus) still was a letdown.
As someone who actually went there, my impression is that Europe in general is complacent when it comes to computers, and any bright eyed student will get their motivation choked out of them in academia here.
If you want to do things with AI in Europe, you can have a bigger effect by working for a consulting company than being at a university. That's … not a good sitatution.
I feel people are too focused on US/China and don't pay attention to what is going on in the world.
ASML in the Netherlands for example was the only company in the world that makes EUV lithography machines, which all the major chip companies depend on. China recently reverse engineered their work to create machines since late 2025, but not sold commercially.
Ireland has chip production facilities.
EU might not be in the top 2, but it is not out of the running at all.