I wish they wouldn't call them "open source models". They aren't open source. They didn't publish the training data. They didn't publish the tools they used to train the model.
They published the weights. It's an "open weight model", a term that it seems nearly everyone has agreed is appropriate. Why is this company not using it?
Compiling source code also takes computing resources, only the scale is different. It's a very close analogy: source code with training data, and compiled binaries with model weights.
You say that but for like 2 years there was a guy on huggingface releasing quants as TheBloke. No identity no nothing except he likely had a grant or a university job that let him do it. Quants aren't the same as training but it was beyond 99% of people at the time (as you say this is)
Cant stand when you guys try and force something as impossible on the rest of us simply because you could never accomplish it.
Unfortunately, electricity prices did not fall by the same factor, so I fear that training a frontier model will still cause a an unsustainable dent in my monthly budget.
I can understand not wanting to call it open source if they don't give you the algorithm and software used for training, but wanting the training data too? That's completely different
The weights are the preferred form for modifying or integrating with other models. There is no obligation in open source to transitively open source all of the documentation / tools used to create the open source project.
>a term that it seems nearly everyone has agreed is appropriate
Models being considered open source even if the original training code / data is not released also is something almost everyone has agreed to be appropriate.
> The weights are the preferred form for modifying or integrating with other models
It's the 2nd time I hear this argument and I'm already fed up with it
Is it the preferred way only because training is expensive? It's like saying binaries are the preferred way of modifying program because you can't afford to have a fast enough machine to compile it yourself.
Most people don't have resources to compile their own browser, but what makes some browsers open is access to the source.
Maybe it's the preferred way because the people sharing models are themselves working with weights and no data?
> Is it the preferred way only because training is expensive?
No, it's the preferred way because that's literally how you train it. Contrary to popular misconceptions, you don't "compile" data into weights. You initialize a model (based on architecture, config, etc) and then you modify it via training. But crucially they (i.e. model creators) modify it the same way (technically speaking) as you would. That's what the license grants you. Nothing less, nothing more. The "how" as in knowhow has never been something covered by a license.
Open-weight is something dreamt up by people misunderstanding the basics of model creation and training, and having ideological biases against AI and/or LLMs. It is what it is, but you should know that you are technically wrong. A model released under an open source license (Apache, MIT, etc) is an open source model. Because the weights are the source of the models. Training is not "compilation". Training is the "how" as in knowhow to modify the model. Training deals with values. Source deals with everything, including values.
In the past, if someone would have released a piece of software (say a PID controller algo) w/ hardcoded values, no-one would bat an eye. LLMs are just that, with billions of hardcoded values. Nothing less nothing more.
Per the OSD definition of source code, "the source code must be the preferred form in which a programmer would modify the program." which means an argument could be made (as charcircuit is making) that the weights, being the preferred form to modify, are the source.
I do prefer open weights as being more precise (like, is it even really software that has source code in the first place?) but I feel like at this point the ship has sailed somewhat (though if this is something you're willing to spend your time arguing then... moral support I guess?)
Incidentally, I do. I fucking hate the open source maximalists like Stallman who suggest that not having source code is equivalent to not having control over what runs on your computer. But you always[1] have control over what runs on your computer. Reverse engineer it if you care so much! Every form of DRM is inevitably defeated, because you do have control over program execution, regardless of access to the source code. Stallman's argument seems more lazy and entitled than principled.
[1] ...setting aside hardware attestation bullshit, which does need to die in a fire.
I think you are failing to see how the weights are the preferred form of working with a model. It's like if I shared a png with others to use as a meme template. Even if I don't share the source code to photoshop other people can use that "binary file dump" to make derivative images of their own with it.
The models are open source, it’s never been a requirement for someone to document the entire process of creating something to be “open source”. Just sharing the source, in this case, the weights, meets the criteria. Software engineering’s obsession with precise terms is actually one of the things I’m glad is dying with ai automation.
They’re not really even being precise. The relevant software freedom, from the FSF is []
The freedom to study how the program works, and change it so it does your computing as you wish (freedom 1). Access to the source code is a precondition for this.
Ported to the model world, this is fulfilled by sharing the weights and implementation. There’s almost nothing that having the training data gets you (other than actually training it). The weights plus a reference implementation let you see all the states to study the behavior, and let you fine tune it to do your bidding (the abliteration etc). The freedom is satisfied.
Some might argue that without the training data you couldn’t do some classes of experiments to see how it works, say leave-one-out retraining. I’d argue things like that are not really about the model but about ML research or the class of models, which while interesting is not a free software pre-requisite.
The source is actually all training data, plus the software used for training, including some scripts or instructions to run the entire thing end to end on your own computer. That's what open source has always been about.
The weights are the output of a program, it's a binary. It's not source.
> Software engineering’s obsession with precise terms is actually one of the things I’m glad is dying with ai automation.
Yes. The next time you go to your doctor you should hope he's not being overly precise; or the engineer that builds the bridge; or the software engineer that implemented the embedded software in your insulin pump.
There is no precise thinking without precise terms.
How is the capex on 8 * MI354X even remotely justified at less than $10/h?
Even without the base system, power and every other expenses:
365d * 24h * $2.95 = $25842/a invoicable.
That doesn't add up within one year, that doesn't add up in three years and it is questionable that it brings in the money during the lifetime of the device?
Where do you see less than $10/h for 8 * MI354X? I can only find $2.50 for 1 * MI355X (lowest I can find for rent on other websites is $2.65, but maybe they got a better deal).
That can’t be real. Modal rents out an RTX 6000 Pro for more than that. Nvidia will rent your GPUs at a fixed cheap rate if you buy from them. Perhaps AMD has a different subsidy style program. Because those MI355X are going cheap here.
This part sounds like AI assisted setting this up and benchmarking it:
>The fix was trivially simple: zero-pad the head count 12→16, run the fast kernel, and extract the real 12 heads from the output.
I've recently used a frontier AI (ChatGPT 5.6 Sol on ultra) to set up a much smaller local model, and the performance optimizations it introduced left the model totally incoherent. (The model just repeats a single character, etc.)
When I see a line like the one I just quoted, it leaves me wondering if the setup is still coherent like a stock install of Kimi K3 on supported hardware.
Did they run any benchmarks on it to see if it is still correct?
I have never used sol, but I regularly use Claude to set up and benchmark local models per task. It is very thorough and has always returned good setups. The only thing you have to do is make sure you point at the model card. It’s always incredulous that models exist after its training cutoff.
K3 does an ok job of setting up, but its config searching isn’t nearly as thorough and its will confidently tell you it’s found the best setup when it’s only turned a few knobs. It’s also not a good evaluator of its own output. It rates its work too highly and seems kind of defensive when benchmarking. Still a good check because it does find stuff, but open a fresh session and don’t tell it where the results came from.
What K3 does do more than any other model I’ve found is investigate folder structures. If I want to benchmark it and other models I have to move the testing methodology and any reference to other results out of the project folder bc Kimi is like an ls bloodhound. It will find them.
Wafer is making themselves synonymous with slop in the inference space. Exaggerated unfair comparisons in all their results, twitter hype posts with alarm emojis etc.
> $2.50/GPU-hr for the MI355X, $6.00 for the B300, and $4.25 for the B200.
This is not an accurate price comparison for real terms.
This is a company that launched a token subscription (WaferPass), before weeks later, rugpulling the plan for being unsustainable while simultaneously claiming they achieved incredible inference efficiency gains worthy of paying them mind.
I went the gpus.io website & it’s $2.95/hr right now, this is like comparing MSRP to actual market price. B300s are in demand & hence cost more, but these lazy editors at wafer.ai can't be bothered to do TCO on actual ownership nor share code to replicate their setups. Instead just relying on current market prices to win one row, which isnt even about per/$ on actual MSRPs.
If you do good work you should at least take the time to review the slop that details that work for slopiness. Otherwise it's hard to take it seriously. Especially the prefill section.
" The only problem with AMD is software support — slower kernels and less day-0 support on inference frameworks make serving frontier models on AMD a real engineering effort."
The B200 is being forcefully compared against something which is not gonna fit within it's memory in a single node & not much details about multi-node interconnectivity, disagg or not. As expected of a shoddy slop.
The only point it won is of cost per hour is one aggregation website for rentals, the premium a B300 commands against the $3/hr AMD chip which no provider has in abundance. Never bothered to do TCO of owning the hardware either.
To the B200’s defence, its numbers are somewhat deflated by the fact that it pays a cross-node all-reduce on the decode critical path (RoCE v2 at ~195 Gb/s) — it’s the only config here that spans two nodes
They published the weights. It's an "open weight model", a term that it seems nearly everyone has agreed is appropriate. Why is this company not using it?
The weights are literally a binary blob.
Open source has benefits even if you can run yourself. You can read the code for understanding/insights. Other labs could replicate/build on it.
I sure wish I had a few 100M of disposable income to train a frontier model.
> or in the future it could be useful when training is cheaper.
I do not think that physics will allow hardware getting that much faster. But maybe we will have different, cheaper architectures by then.
Cant stand when you guys try and force something as impossible on the rest of us simply because you could never accomplish it.
Unfortunately, electricity prices did not fall by the same factor, so I fear that training a frontier model will still cause a an unsustainable dent in my monthly budget.
I can understand not wanting to call it open source if they don't give you the algorithm and software used for training, but wanting the training data too? That's completely different
You can‘t reproduce the LLM without the same data
>a term that it seems nearly everyone has agreed is appropriate
Models being considered open source even if the original training code / data is not released also is something almost everyone has agreed to be appropriate.
It's the 2nd time I hear this argument and I'm already fed up with it
Is it the preferred way only because training is expensive? It's like saying binaries are the preferred way of modifying program because you can't afford to have a fast enough machine to compile it yourself.
Most people don't have resources to compile their own browser, but what makes some browsers open is access to the source.
Maybe it's the preferred way because the people sharing models are themselves working with weights and no data?
No, it's the preferred way because that's literally how you train it. Contrary to popular misconceptions, you don't "compile" data into weights. You initialize a model (based on architecture, config, etc) and then you modify it via training. But crucially they (i.e. model creators) modify it the same way (technically speaking) as you would. That's what the license grants you. Nothing less, nothing more. The "how" as in knowhow has never been something covered by a license.
Open-weight is something dreamt up by people misunderstanding the basics of model creation and training, and having ideological biases against AI and/or LLMs. It is what it is, but you should know that you are technically wrong. A model released under an open source license (Apache, MIT, etc) is an open source model. Because the weights are the source of the models. Training is not "compilation". Training is the "how" as in knowhow to modify the model. Training deals with values. Source deals with everything, including values.
In the past, if someone would have released a piece of software (say a PID controller algo) w/ hardcoded values, no-one would bat an eye. LLMs are just that, with billions of hardcoded values. Nothing less nothing more.
Open source means open source code. Open weight means a binary file dump, not unlike an exe file. There is nothing open source about it.
Its like having a closed source text editor that censors certain words, and an open source text editor that censors certain words.
The latter can easily be recompiled, the former requires reverse engineering. Both may give you a license to use them freely.
I do prefer open weights as being more precise (like, is it even really software that has source code in the first place?) but I feel like at this point the ship has sailed somewhat (though if this is something you're willing to spend your time arguing then... moral support I guess?)
[1] ...setting aside hardware attestation bullshit, which does need to die in a fire.
They’re not really even being precise. The relevant software freedom, from the FSF is []
Ported to the model world, this is fulfilled by sharing the weights and implementation. There’s almost nothing that having the training data gets you (other than actually training it). The weights plus a reference implementation let you see all the states to study the behavior, and let you fine tune it to do your bidding (the abliteration etc). The freedom is satisfied.Some might argue that without the training data you couldn’t do some classes of experiments to see how it works, say leave-one-out retraining. I’d argue things like that are not really about the model but about ML research or the class of models, which while interesting is not a free software pre-requisite.
[] https://www.gnu.org/philosophy/free-sw.html#four-freedoms
The weights are the output of a program, it's a binary. It's not source.
Yes. The next time you go to your doctor you should hope he's not being overly precise; or the engineer that builds the bridge; or the software engineer that implemented the embedded software in your insulin pump.
There is no precise thinking without precise terms.
Even without the base system, power and every other expenses: 365d * 24h * $2.95 = $25842/a invoicable.
That doesn't add up within one year, that doesn't add up in three years and it is questionable that it brings in the money during the lifetime of the device?
Is anyone actually renting them out that cheap? The very cheapest on-demand price I see online is $14, and most providers are a lot higher
>The fix was trivially simple: zero-pad the head count 12→16, run the fast kernel, and extract the real 12 heads from the output.
I've recently used a frontier AI (ChatGPT 5.6 Sol on ultra) to set up a much smaller local model, and the performance optimizations it introduced left the model totally incoherent. (The model just repeats a single character, etc.)
When I see a line like the one I just quoted, it leaves me wondering if the setup is still coherent like a stock install of Kimi K3 on supported hardware.
Did they run any benchmarks on it to see if it is still correct?
K3 does an ok job of setting up, but its config searching isn’t nearly as thorough and its will confidently tell you it’s found the best setup when it’s only turned a few knobs. It’s also not a good evaluator of its own output. It rates its work too highly and seems kind of defensive when benchmarking. Still a good check because it does find stuff, but open a fresh session and don’t tell it where the results came from.
What K3 does do more than any other model I’ve found is investigate folder structures. If I want to benchmark it and other models I have to move the testing methodology and any reference to other results out of the project folder bc Kimi is like an ls bloodhound. It will find them.
> $2.50/GPU-hr for the MI355X, $6.00 for the B300, and $4.25 for the B200.
This is not an accurate price comparison for real terms.
" The only problem with AMD is software support — slower kernels and less day-0 support on inference frameworks make serving frontier models on AMD a real engineering effort."
GPUs. 8× MI355X (TP8) B300 (TP8+DCP8)
Decode tok/s per stream 118 tok/s 172 tok/s
Peak aggregate. 952 tok/s 1,568 tok/s
Peak aggregate per GPU 119 tok/s 196 tok/s
On every row the B300 beat the MI355X
The B200 is being forcefully compared against something which is not gonna fit within it's memory in a single node & not much details about multi-node interconnectivity, disagg or not. As expected of a shoddy slop.
The only point it won is of cost per hour is one aggregation website for rentals, the premium a B300 commands against the $3/hr AMD chip which no provider has in abundance. Never bothered to do TCO of owning the hardware either.
When I read your original comment, I was thinking you had just asked it to evaluate the article. (Like just "evaluate this article" or something.)
I don't think anything anyone (or any AI) has ever written or published (including Sol itself) wouldn't be torn apart by the prompt you gave though.