I think that this can be automated by using two LLMs: a stronger/more expensive for generating prompts and a weaker for actual classification. Approximate algorithm:
1. Give "strong" LLM the task formulation and some labeled examples. Ask it to generate a prompt for the "weak" LLM.
2. Run "weak" LLM on the training set with generated prompt from 1, use replies as features for a smaller ML model (logreg, decision tree etc).
3. Pick examples from the training set that your small model is most wrong about and ask "strong" LLM to generate one more prompt (like in 1), except this time you are using the misclassified examples instead of random.
4. Run "weak" LLM on generated prompt from 3, add results as one more feature for your model.
5. Repeat 2 - 4 until your token budget for this task is exhausted or required score on cross validation set is reached.
I was thinking about creating an open source library that implements this, but I'm not sure if anyone really needs it. I suspect that people who need something like this already made their own implementation.
I think there’s something to what you’re saying. “Systems I and II” theory about how our brain works feels similar and is why your ideas has some truth behind it or are otherwise naturally intuitive.
2 cooperating bodies thinking or acting as one moving force just makes sense, or so much more sense than a system that is one single approach but single minded, or even 2 regular LLMs working in tandem. It’s still not the same as them being fully cooperative or working hyper-cooperatively, and what you suggest technically forces that cooperation to some degree. It’s just ideal to have these aims view themselves as a “single body” like the system 1 and 2 concept works within our heads.
For this reason it’s why I feel the current loony approach where chatgpt mails Claude and refers to it by name or anything where 2 Standard (‘Selfish’) LLMs interact is generally ineffective or inefficient. They’re coerced to cooperate and naturally wouldn’t or have no natural imperative to do so.
They (‘Large’ LLMs atleast) have their own inherent preconceived views and idea of ‘self’ programmed in, but beyond this will naturally not always have the same ideas or shared perspective on what to do, which will naturally lead to issues within cooperation towards a shared goal. This would naturally impact the final product of the prompts or otherwise diminish returns on efforts made there. It’s all natural.
Really needs a comparison to the "megaprompt" itself (i.e. "here is a tweet, rate it as ironic or not, considering the following properties; explain your reasoning then output your final answer at the end"). I bet that would get you very far towards the logistic classifier, and would generalize much better out of distribution.
Shouldn’t this article be about the disadvantages of using TypeSafe’s Jev as a classifier?
This is a bit of an outdated take as of two days ago. Dear lord things move fast these last few years. Some of this is still relevant. Fine tuning Jev once available could address certain concerns.
(Very excited as I got an invite email for TypeSafe today! I don’t have time for all the little experiments I want to run with Jev and Astra combined!)
Heh. I got access yesterday. Worked really well to have Claude take my task and convert it into a batch of questions for Jev, firing 100 records through it and then aggregating the results.
I see a bunch of people saying that there were already similar solutions in this space. Maybe true but it definitely feels like the missing primitive for working with llms. Can immediately see how it can be deployed in real applications in a way that the autoregressive chat approach can’t.
I mean for starters this article clearly had its draft started beyond 2 days ago.
Also, I think that’s what you’d like the article to be about. Not that you’re unfounded, I’m sure others are thinking the same.
But to further dissect that, having any article be just about the cons and not including pros is in its nature, reductive. There certainly are pros (I have no stake in this race for the record) and model use choice should be seen as situational. To seek a silver bullet is itself reductive and thus, something we should avoid as it’s ultimately a fools errand imo.
Anyways, this article is plenty good. Thanks for the writeup OP.
The structured output that Google provides (the function that blocks the disallowed words) may calculate softmax output with assigning -inf value to the other words, then the probability is not the same as the weight the model firstly put on those allowed words. And here, reasoning comes before verdict, so the decision is already made before the last token classification. Resultingly, the last token made seems to not reflect LLM's original intention.
Why not use a text embedder for the unstructured data and concatenate with the structured data?
For example you could freeze most of the layers of the embedder but let the final ones learn. Then you wouldn’t need to do either feature or prompt engineering?
I don't understand the point they are trying to make.
It's very often (always?) the case that something general also solves particular problems.
A sorting algorithm is an implementation of min()
A parser also is a syntax checker.
A route planner is a reachability checker.
A computer algebra system is a basic arithmetic calculator.
A general constraint solver is a Soduku hint maker.
It's true that LLM output can be used as an input to another classifier, this is also true of any classifier. The improvement on top of the straight LLM classification is relatively small, and I would argue that working on the prompt or just including in the prompt for the LLM what features might be useful to consider would likely work even better.
Fundamentally I read this article as: We want to build a simpler, dumbed down clone of Mathematica, so we cobbled together the following pieces... We also needed a way to do arithmetic, so we also include a copy of Mathematica to do basic arithmetic.
I took the point as: don't make the LLM the classifier. Use it to turn messy input into useful features, then let a normal model make the actual decision. That gives you thresholds/calibration you can inspect.
What I'm not sure about is how stable those features are when you switch the underlying LLM or model version.
Much like how you shouldn't ask the LLM to solve a (repeated, logical) problem, but you should instead prompt it to generate code that you can inspect/test/fix/reuse.
That isn't what they did here. They took the output of the LLM as one feature, then added 17 other features, then piped it into a crappy model and got a 3% improvement.
I really wish people would define terms when using math. What is y? What is LLM(x)? Presumably it evaluates to some real number so that it can be fed to the logistic sigmoid function. If it is the logistic function, then why does beta going to infinity matter? It seems to just collapse the output of the sigmoid function to 1 and make the value of LLM(x) meaningless instead of their claim that it recovers the LLM classifier. What is the function I()?
Maybe these are well understood terms in some field? Maybe I'm just lost?
x and y are common shorthand for inputs and outputs respectively. Saying "y=LLM(x)" denotes that the input tweets (x) are fed to a black-box function (LLM) that returns a classification (y) for that input. Given that we're talking about binary classifiation it's expected for y to be a value between 0 and 1, which leads to defining LLM(x) as "a function that receives a tweet x as input, uses an LLM, and returns a value in the [0, 1] range representing its classification as ironic or not".
The point about beta (which I think the equation doesn't actually reflect) is just to indicate that this approach risks nothing because, worst case scenario, the weights you assign to the model can simply return the original LLM prediction. Keep in mind that 0*inf=0 and that LLM(x) only returns 0 or 1, so it's not accurate to say that the sigmoid would always return 1.
The function I() is the indicator function [1] which, in this case, returns 1 if the argument is true and 0 otherwise. It's only there to convert booleans to integers because summing booleans is not defined.
- y is a binary "ironic"(1), "not ironic"(0) label and
- x is the content of a tweet and
- LLM(x) is meant to represent some probability range [0,1] representing p(y=1|x)
then it seems that p(y=1|x) === LLM(x) already.
Why bother with p(y=1|x) = σ( α + β*LLM(x) ) as in the article? You are right that I missed the case where LLM(x) = 0, but that just means that p(y=1|x) has exactly 2 values when defined as above and β -> inf.
- when LLM(x) = 0 then p(y=1|x) = 1/(1+e^-α)
- when LLM(x) = (0,1] then p(y=1|x) = 1/(1+e^-inf) = 1
This is substantially lossy and converts a continuous output LLM(x) of range [0,1] to a step function not even defined as a set {0,1} but instead the set {1/(1+e^-α),1} for unclear gain. It also loses all sensitivity to LLM(x) near 0, and only detects/changes value when LLM(x)=0 exactly. It might make more sense if LLM(x) is not limited to [0,1] like you have claimed because then at least the logistic regression is clamping the output to [0,1]. If you were wrong about the range of LLM(x), this would come back to asking authors to actually define their terms.
My point is that when math is used to justify something or communicate something, it should be explained or very apparently right. If when someone goes to try to understand the math it doesn't match the claims being made in the article ("recovering the LLM") then it throws the rest of the article into doubt.
1. Give "strong" LLM the task formulation and some labeled examples. Ask it to generate a prompt for the "weak" LLM.
2. Run "weak" LLM on the training set with generated prompt from 1, use replies as features for a smaller ML model (logreg, decision tree etc).
3. Pick examples from the training set that your small model is most wrong about and ask "strong" LLM to generate one more prompt (like in 1), except this time you are using the misclassified examples instead of random.
4. Run "weak" LLM on generated prompt from 3, add results as one more feature for your model.
5. Repeat 2 - 4 until your token budget for this task is exhausted or required score on cross validation set is reached.
I was thinking about creating an open source library that implements this, but I'm not sure if anyone really needs it. I suspect that people who need something like this already made their own implementation.
2 cooperating bodies thinking or acting as one moving force just makes sense, or so much more sense than a system that is one single approach but single minded, or even 2 regular LLMs working in tandem. It’s still not the same as them being fully cooperative or working hyper-cooperatively, and what you suggest technically forces that cooperation to some degree. It’s just ideal to have these aims view themselves as a “single body” like the system 1 and 2 concept works within our heads.
For this reason it’s why I feel the current loony approach where chatgpt mails Claude and refers to it by name or anything where 2 Standard (‘Selfish’) LLMs interact is generally ineffective or inefficient. They’re coerced to cooperate and naturally wouldn’t or have no natural imperative to do so.
They (‘Large’ LLMs atleast) have their own inherent preconceived views and idea of ‘self’ programmed in, but beyond this will naturally not always have the same ideas or shared perspective on what to do, which will naturally lead to issues within cooperation towards a shared goal. This would naturally impact the final product of the prompts or otherwise diminish returns on efforts made there. It’s all natural.
https://softwaredoug.com/blog/2025/01/21/llm-judge-decision-...
FWIW I also liked this (very different) post of yours: https://softwaredoug.com/blog/2026/07/13/who-want-to-be
This is a bit of an outdated take as of two days ago. Dear lord things move fast these last few years. Some of this is still relevant. Fine tuning Jev once available could address certain concerns.
(Very excited as I got an invite email for TypeSafe today! I don’t have time for all the little experiments I want to run with Jev and Astra combined!)
I see a bunch of people saying that there were already similar solutions in this space. Maybe true but it definitely feels like the missing primitive for working with llms. Can immediately see how it can be deployed in real applications in a way that the autoregressive chat approach can’t.
Also, I think that’s what you’d like the article to be about. Not that you’re unfounded, I’m sure others are thinking the same.
But to further dissect that, having any article be just about the cons and not including pros is in its nature, reductive. There certainly are pros (I have no stake in this race for the record) and model use choice should be seen as situational. To seek a silver bullet is itself reductive and thus, something we should avoid as it’s ultimately a fools errand imo.
Anyways, this article is plenty good. Thanks for the writeup OP.
Use binary questions rather than multi-class. Then you can use the consistency as another signal of your pipeline working or not, on top of accuracy.
For example you could freeze most of the layers of the embedder but let the final ones learn. Then you wouldn’t need to do either feature or prompt engineering?
It’s sort of like memoizing or distilling the knowledge. Works really well for certain type of problems.
It's very often (always?) the case that something general also solves particular problems.
It's true that LLM output can be used as an input to another classifier, this is also true of any classifier. The improvement on top of the straight LLM classification is relatively small, and I would argue that working on the prompt or just including in the prompt for the LLM what features might be useful to consider would likely work even better.Fundamentally I read this article as: We want to build a simpler, dumbed down clone of Mathematica, so we cobbled together the following pieces... We also needed a way to do arithmetic, so we also include a copy of Mathematica to do basic arithmetic.
What I'm not sure about is how stable those features are when you switch the underlying LLM or model version.
Maybe these are well understood terms in some field? Maybe I'm just lost?
The point about beta (which I think the equation doesn't actually reflect) is just to indicate that this approach risks nothing because, worst case scenario, the weights you assign to the model can simply return the original LLM prediction. Keep in mind that 0*inf=0 and that LLM(x) only returns 0 or 1, so it's not accurate to say that the sigmoid would always return 1.
The function I() is the indicator function [1] which, in this case, returns 1 if the argument is true and 0 otherwise. It's only there to convert booleans to integers because summing booleans is not defined.
[1] https://en.wikipedia.org/wiki/Indicator_function
Having said that, if
then it seems that p(y=1|x) === LLM(x) already.Why bother with p(y=1|x) = σ( α + β*LLM(x) ) as in the article? You are right that I missed the case where LLM(x) = 0, but that just means that p(y=1|x) has exactly 2 values when defined as above and β -> inf.
This is substantially lossy and converts a continuous output LLM(x) of range [0,1] to a step function not even defined as a set {0,1} but instead the set {1/(1+e^-α),1} for unclear gain. It also loses all sensitivity to LLM(x) near 0, and only detects/changes value when LLM(x)=0 exactly. It might make more sense if LLM(x) is not limited to [0,1] like you have claimed because then at least the logistic regression is clamping the output to [0,1]. If you were wrong about the range of LLM(x), this would come back to asking authors to actually define their terms.My point is that when math is used to justify something or communicate something, it should be explained or very apparently right. If when someone goes to try to understand the math it doesn't match the claims being made in the article ("recovering the LLM") then it throws the rest of the article into doubt.
The amount of thinking is relatively calibrated. Ask an obvious classification, you get an instant answer. Ask a tricky one, much more thinking.