We're already seeing this at the enterprise level. Companies have dictates from leadership that "if you're writing code manually, you're doing it wrong."
Okay, that kind of works for a while. We are indeed producing a shit-ton of code, but the reality is that engineers are pumping out code faster than the humans can understand and (honestly) review it. That sounds great until you realize that "hey Claude, read this Jira ticket and implement the feature in this code base" isn't really worth $200K/year.
This is all complicated by the fact that we're also losing our grasp on reality from the other direction because we have leadership air dropping AI generated manifestos on the product owners and product owners having to use AI to transmute all that shit into 1,500 word Jira tickets that are 10% necessary feature work and 90% LLM boilerplate.
So now you have software engineers whose job has changed radically to the point that the hardest part about being a software engineer is just filtering through AI generated artifacts from all directions just to try to get a feature out the door.
We have a product guy on the team who was in a deeply not technical role before AI who is trying to do the “hey Claude, read this Jira ticket, implement” thing.
It doesn’t work for the vast majority of tickets he attempts because he doesn’t have the necessary understanding to even start thinking about if the solution that the autocomplete generates is even remotely workable. And that’s with fancy dev loops and whatnot.
The spacer between the keyboard and the chair still matters in my experience.
Not everyone has this, but I always felt that a significant percentage of the value I bring is in immediately recognizing what you shouldn’t do.
I have a hard enough time explaining why “yet another bespoke application on some unmanaged azure resources” is a bad idea when they have more reasonable alternatives at hand.
Now these goofs can (very nearly) press a button and do it anyway, with no comprehension of the consequences. It’s high fives and pats on the back, until I’m cleaning up the mess.
This lands. Llms are bad precisely at following what not to do. They work best off of positive constraints.
I have a design principles + tech preferences doc I force llms "lint" their approach against. It's not perfect but it helps. I call it a bias field, pushes them toward hopefully the happy and harmonious (with the rest of the system) paths. Obviously this is only partial and imperfect enforcement, but if it's applied to everything consistently it naturally encodes some self-consistency and harmony.
> I have a design principles + tech preferences doc I force llms "lint" their approach against. It's not perfect but it helps.
I’ve done the same but it’s a moving target as models advance and I find half of my points are ignored until I’m prompting “No wtf why are you still trying to symlink the global Python executable just use the virtual environment that’s already activated”.
Anyway, companies are pouring billions into improving AI tooling user experience so most of what I do manually I just anticipate to be a waste of time. There’s no way my hobby fiddling will outpace whatever gets released in the next couple months.
In the meantime, real linting does work pretty well, if you can write a detector for whatever antipattern you find LLMs fall into (like multiline comments).
> Anyway, companies are pouring billions into improving AI tooling user experience so most of what I do manually I just anticipate to be a waste of time.
I get patches into the biggest AI projects all the time to make inference faster on my 3090 and all you have to do is read the contribution docs and open a small PR.
It’s not a cabal of super geniuses. We are literally in the Wright Brothers era of AI.
You can trivially outpace what big companies are doing it’s insane, why do you think so many innovations are coming out of scrappy Chinese labs? They are willing to engage instead of being defeatist about it.
I've had some success with: here is this android bug ticket in a product that I don't know much about, fix it! And it works sometimes. I'm still technical, so it isn't the same, but it was crazy that AI could just solve some problems automagically without me know much about the system being modified.
For the completion to work, the source text needs to be ‘good’. That’s a basic kernel of how the thing works. Even with a perfect oracle autocomplete if the source text is ‘bullshit’, the output is too.
Or, slightly changing this. The source text needs to speak the correct vocabulary and language to produce a good completion. See the chat where Terry Tao is doing maths with an LLM. There’s _no way in hell_ I could get to his output because I just have no idea, and can’t speak the language.
fucking is it? AI labs certainly want you to think so but they're running out of time and money. they've been telling me that the advancements of the last three months have made this a reality for the last five years. it's starting to feel a lot like Elon promising fully automated taxi services by next year... 11 years ago
So far, the LLMs I've used need problems to be fairly specifically scoped, or they don't produce the correct solution. Scoping problems correctly is a different skill-set than implementation, but it's still a technical skill that is expected from mid-level and higher engineers.
Yep and I just ran a simple enterprisey "ambiguity" bench on the big three (US) model providers: same ambiguous initial-prompt with same clarifications and pushback prompt sequence afterwards.
The edge of correct/better when facing ambiguity is very fuzzy, all models from the past 6 month or so have similar random ways of spinning between too-literal avenues and oddly misplaced misled fixations. Taking the right initiatives in face of uncertainty is definitely AGI, and its not there, and perceptrons + attention layers just ain't got what it takes no matter how hard you push.
Probably worth not getting too comfortable. It's only a matter of time before a new generation of product people can do this. Also, I have said this before, instead of 1 product person and 10 engineers on a team, you may have 2-3 product+eng people and 3 engineers, etc.
> That sounds great until you realize that "hey Claude, read this Jira ticket and implement the feature in this code base" isn't really worth $200K/year.
Nor even €50k/year.
Two things are true:
1) The coding part of my career is over. LLMs are capable of doing everything I've ever been paid to *write*.
2) My actual job also included non-coding work: Does this attempted solution even work at all? Is this solving the right problem? Even if it's a valid solution to the right problem, is it the best solution given the time constraints?
That last one, "given the time constraints", is a place where my experience is still useful. The AI is as lazy (or as optimised for fast wins) as the humans whose examples it was trained upon; but an LLM costs so little that the answer is always "do it right" rather than "do it fast". A lot of people don't know what "do it right" even looks like, having only ever known e.g. websites that take 5 seconds to load because of all the adverts and analytics, and never the world where machines with 1% of the CPU and no GPU at all could fit a fully playable first person shooter in the same memory footprint as that page.
At least, I hope this "is a place where my experience is still useful"; I may just be telling myself a nice story, same as all the other people through history who have found themselves obsoleted when the automation came for them.
> The coding part of my career is over. LLMs are capable of doing everything I've ever been paid to write.
Remember we used to spend enormous amount of time in school and in our spare time studying computer science? Algorithms, operating systems, compilers, and etc. All kinds of insights. All kinds of fun. All kinds of hard engineering. Yet, how much time do we really need to spend in our day-to-day work implementing or using the algorithms and etc that we have learned?
Engineers have done amazing work of abstracting away the hard algorithms and data structures. In the meantime, there has been little progress or few new fields in the past 10 years or so in business that ask for implementation of new algorithms. In contrast, getting LLM to work is a new field, so it requires tons of new implementations: KV caches, speculative decoding, all kinds of variants of attention like FlashAttention, all kinds of parallel processing techniques, RL pipelines, post-training pipelines, and etc. It's just that the field is so concentrated that only luck few get to work on them.
So, maybe it's not LLM per se that removes the need of writing code. It is the maturity of the software engineering that has done so. It's just that LLM fills the last gap: making knowledge transfer so much faster and cheaper. All that's left for most of us is slicing and dicing of what has been already been implemented.
Before, if I would start working / thinking on a problem and discover it was harder than anticipated, that often was a signal that the feature may not be worth it. Because implementing it doesn't fit the current model of how things work or similar. Then it was back to the drawing table and find a better way of solving the underlying issue.
But an LLM just happily chugs along and does it, no one feels the friction, which means you never stop up to think if you're solving the right problem, or in the right way. I'm already seeing this bite us in the ass, as you get hacks upon hacks.
So exactly true! With ideas people unbridled, the painting-yourself-in-a-corner tendency that was always there has gone exponential, add to it the not-completely-aligned incentives, and the oh-so-clever impressive writer of long prose LLMs just end up churning. The real value add may end up being the lesson in honesty and humility to us all...
I first brought this up late last year – that there used to exist a kind of selection pressure against both developers and against features which would prevent certain functionality from ever being shipped, specifically:
- Functionality which exceed the technical ability or knowledge of the developer which built it; and
- Functionality which would require an excessive number of changes that time/cost became a constraint (likely because it wasn't an incremental change but a significant rework, or just a bad fit with the existing product).
Sometimes I had the displeasure of joining a company where you could see developers hit these blockers and tried to fight through them (likely under pressure from management) rather than avoid them. And in the process they created a mess of buggy, half-functional spaghetti code which someone else later had to resolve.
Today however, people can use agents to smash through these blockers and ship an incomprehensible amount of crap. And what's worse is they're celebrated by management who don't know any better and see this all as a productivity win with no downside.
I suppose there are two ways to look at this – some would argue that engineers don't need to understand what they're building in detail anymore so non of this matters. Instead they can always use agents to explain what's going on and prompt them to fix any issues that come up.
Then there's another camp which might argue that agents will fundamentally have all the same limitations as humans, and that at some point a codebase will grow too complex that it exceeds even the limits of an agent's knowledge retention or intelligence. Or a codebase may just grow so large that it costs 1 million tokens to make even a simple change.
Unless agents start saying, "there's no way I'm shipping that" like humans used to I don't really see how we avoid the latter scenario... Complexity simply must have limits even if agents allow the bounds of those limits to grow.
At some point models will have to take control or we risk hitting these limits in irrecoverable ways since complexity far exceed that of what a human can reason about well.
If this wasn't true then "build me all the software" would be a reasonable prompt. Because why wouldn't we just get an agent to build everything we could ever possibly need? It's obvious that in the limit there will be limits in knowledge, intelligence and cost.
Whether it's humans or agents, someone needs to manage complexity. That is the most important thing a good SWE used to do. It's why technology selection mattered, it's why good architecture mattered, it's why clean code mattered.
In my personal project I've found having a hard lines of code limit has been a good limit on complexity. Makes me really ask if I need things.
One theory I have floating around in my head is that if a whole code base was microservices and micro front ends that were all less than 5,000 loc then you could fit the whole thing into a 100k token AI context window when working on it. And being few lines of code would force them to be somewhat simple
>My actual job also included non-coding work: Does this attempted solution even work at all? Is this solving the right problem? Even if it's a valid solution to the right problem, is it the best solution given the time constraints?
I feel like there is also the "Nerd Factor" to consider which is this unrelenting passion to type away at a screen all day, whether that is yesterday's code or tomorrow's prompt. Few non-technical people have the attention span to suffer such monotony. (Even most devs don't have that to varying extents)
I'm trying to write a novel, have been for a while now; one thing I hear from professional writers is that one should only do this if you are prepared to re-read whole thing six times before you even send it to the editor, and when you do send it to the editor, you and they will likely spend several months repeatedly re-reading the whole thing as you respond to a long list of changes they give you.
It's only monotone from someone looking, not from someone doing. If you look at digital painting tutorial, you'll see a lot of timelapse, because it's a very slow process where you spend the majority of time correcting stuff. But if you're the one doing it, each individual brush stroke is the result of a conscious decision.
Working on a passion project or working through the hairy details of a complex algorithm or whatever can be fun. But that's not really what the job is like most of the time.
All the boring parts I've encountered have soon been automated out of my workflow. There's the waiting part of some process, but that's why I have HN for.
> The coding part of my career is over. LLMs are capable of doing everything I've ever been paid to write.
That is certainly not true. LLMs can't write code worth a damn still, and you have to babysit them to make sure they aren't doing stupid stuff. A human is still by far the best choice for doing programming, and when trend-chasing companies realize the damage they've done to their businesses they will stop pushing LLMs so hard.
I think it depends on the - ahem - context. In my experience they usually can, sometimes they can't.
But generally their sense of software architecture remains abysmal, so even if their writing is ok, you need to have architectural authority the very least to guide them.
I guess that somewhat describes me - and guess what, that also transfers to "should we even throw AI at that specific problem, or are there other solutions?". Early curiousity in figuring out the limits of LLMs now is turning into revenue by customers with problems that didn't exist a few years ago at all - which currently is quite an exciting field to work in as a lot of things just don't have pre-established solutions yet.
It's a similar level of excitement as when we started doing devops stuff before devops even was a thing (which has been one of our main things for a very long time now).
I have also tried developing games with them. I find them very weak at these things.
Often they can't figure out how to test the thing, so "does this attempted solution even work at all?" is its guess from reading the source code, and sometimes I've even caught them writing "tests" which are a regex on the source code, not functionality.
They know almost nothing about "is this solving the right problem?", they're trained to assume the user is right, not to frame-challenge.
My general experience both in game and non-game projects is that it will be lazy by default rather than solving things correctly. Sometimes I spot this from their responses, other times I only notice with manual testing.
Works for a demo, but quickly stagnates. Future models may of course push the limits further, but right now it quickly goes sideways without someone knowledgeable at the helm.
You don't design things with never ending details though. If your developing a feature there's a limited scope to it. If your designing the direction of a large project you're working with higher level concepts.
The point is that AI is aware of constraints and can manage a round it. You've got to keep in mind the size of a typical software project and plan. Most people aren't writing a kernel. They're writing some backend tool and need a project that fits into a couple of quarters with a handful of people at most involved.
> The point is that AI is aware of constraints and can manage a round it.
Only if you instruct it with constraints. Otherwise, it's happy to implement whatever workaround it needs. But it still takes a dev to know what those constraints are and why they're needed.
I have found it intriguingly difficult in my own tests to make LLMs deliberately fuck up grammar in any way similar to how humans do. I suppose this is actually an architectural limitation.
write a comment about how to use ChatGPT to write a game like you're a barely literate reddit user, short answer, maximum incoherency
and the result was a pretty good simulation. I tried less insulting prompts first, this did not produce a good simulation.
Not perfect, gptzero.me still knew it was AI generated (tayo42 is human by the same measure), but if for some reason someone was using an LLM prompted with that pattern, I suspect it would fool me in a Turing test unless I found the keyword to force the agent to change the role it was playing.
> the reality is that engineers are pumping out code faster than the humans can understand and (honestly) review it
The word "engineering" means doing things using a repeatable process to get predictable results. If you can get predictable results (e.g. guaranteeing the absence of data integrity issues, security issues, anything that will cause downtime, etc.) without looking at the code, you're still doing the work of engineering.
Your job as an engineer is to choose high-value problems to work on, solve them in the correct way, and guarantee that the functional and non-functional requirements are met. If you no longer need to read the code to guarantee that it has the right set of ilities, then I'm not sure that's obviously a bad thing, as long as the ilities you're enforcing result in a codebase that is sustainably secure and maintainable over the long term.
Boris: "I don't prompt Claude anymore. I have loops prompting Claude and figuring what to do".
Boris: "I haven’t written a line of code by hand in, I think, eight months now… Claude Code, 100% written by Claude Code".
Boris: "There’s no manually written code anywhere at the company… All of the SQL is written by models. Everything is just built by the models... Claude instances communicate with each other (e.g., over Slack) in autonomous loops"
This does not sound like they review the code either. So, either the frontier labs like Anthropic have figured out something that very few companies could replicate, or they are being incredibly deceptive. I don't know which is true.
They work on research problems you can define a clear solution criteria for.
The kinds of business software I work on don't have those characteristics. If I needed something like a utils library, I think I could easily have Claude write the whole thing and not read the code.
People are fundamentally short-term thinkers. Just imagine spraying neurotoxins on your lawn. You're "saved" from the "terror" of noticing more than one kind of grass, but of course you're incrementally increased the cancer risk for yourself, your children, and who knows who else. This is just one example; people do this constantly. Plastic fleece puts an untold amount of poison into the environment, but "I need to be warm RIGHT NOW." etc. Or lead paint. "It's only toxic when it breaks down, and that won't be for decades!"
All the issues you mention here have a death horizon: the people engaged in the behaviour will be dead when its effects are most acute. That they engage in it then isnt very mysterious.
> That sounds great until you realize that "hey Claude, read this Jira ticket and implement the feature in this code base" isn't really worth $200K/year.
When you factor in overhead and benefits, many companies were regularly paying that much for someone (many someones) to "read this Jira ticket and implement the feature".
We are currently in the "centaur" phase where a human-AI combination produces the best output, but I think some people are betting on the fact that the AI only product will eventually outperform the centaur. And with the cost of tokens falling thanks to fierce competition from the Chinese open weight models, it's definitely possible that those that bet on AI early will reap payoffs in lower expenditures for more output.
Honestly not sure which side I land on that bet but I definitely can't rule it out.
We've gone through this with chess, and the days when a human can do anything to help AI there are long past; the human can only drag it down. It seems to me to be a form of the bitter lesson. I expect this will happen in every field where we don't add guardrails to require humans in the network. I have a hard time seeing regulation happening around this in the US but vanity may also serve, will CEOs be satisfied ruling armies of virtual assistants, or do they need big buildings full of human thralls?
This is not a silly question, because it's kind of silly:
Has an AI ever actually gone through all of the steps necessary to become a chess master? Go to competitions, raise in the ranks, take a plane to the masters, organize support and all the real life details necessary? Playing chess is just the small formal part here, and all of the obvious things humans obviously do, well...
> the days when a human can do anything to help AI there are long past; the human can only drag it down.
I disagree, currently a human guiding an AI agent is far better (more productive for less money) than an AI agent with very loose non-technical guidance and The Ralph Wiggum Loop.
I don't know how much longer humans have, but I don't think they can be cut out of the loop entirely just yet.
1. Execution of engineering task - Companies happen to focus on this mostly and this is the metric to measure easily. Using LLM tools gives a impression of improvement on this area which is what everyone is chasing
2. Growth of the Engineer - This is the one which was always a side product of company culture, individuals interest, work being done, time being spent to understand, learn from failures. A job being executed perfectly for the first time itself does not gives the opportunity for learning, no memories/experiences are built up mind of the person executing the job after a while.
The second part is the one which is under appreciated in current scheme of things since it was a by product. There is a concept of muscle memory which pretty much applies to everything.
My job improved significantly since I managed to get into their (managers) thick skulls to tell Claude to not be verbose. I'm tired of 1200-word tickets for "put the totals in the automated emails".
It's like a tool that has caused a fan out of both good and bad outcomes.
Good: you can refactor your codebase at will, throw out legacy cruft by the megabyte, improve build/CI time, and simplify ruthlessly. Not to mention kick out new features in simple and consistent ways that align with what a user actually wants.
Or you could add megabytes of vibe coded crap, solutions that add a ton of mass but don't actually solve the problem at hand (seriously!), implement abstractions that are logically inconsistent with the rest of the system, etc.
Both of these are happening right now, and I think the latter is happening at a rate far higher than the former. But our fundamental dynamics are still at play - the ball of mud is still a ball of mud, even if AI lets you make it 100x bigger. The problem just gets more entrenched.
Eventually AI will learn how to simplify code, understand coupling, etc - and hopefully it will just iron out problems as it goes. I think we're a long way away from that. But this is uncharted territory, and I don't think anyone really knows. I certainly don't hear anyone focusing on that as a target of optimization however.
My hope is that we'll see a number of companies collapse as they scale - with basically no hope of rescue, and perhaps we can re-learn these lessons yet again. It kind of feels like GitHub might be the first example of this.
> My hope is that we'll see a number of companies collapse as they scale - with basically no hope of rescue
If you worked in SV like I did from 2010-2020 you know that the exact opposite of that will happen. I saw hundreds of companies successfully scale out of their garbage stack such as Facebook and PHP.
The only thing that matters is the problem you’re solving and the quality of the code is irrelevant.
> A bad or below average dev with AI will run your product and company into the ground in a matter of weeks.
I had a recent experience with this. Although it was months and not weeks and there were other factors into play such as the market etc but the last company i was in had a poor engineering culture with inexperienced engineers equipped with AI output some of the lowest quality features for both customer-facing products and internal dev tools. A lot of customers churned, and im sure quality us part of the reason.
> We are indeed producing a shit-ton of code, but the reality is that engineers are pumping out code faster than the humans can understand and (honestly) review it. That sounds great until you realize that "hey Claude, read this Jira ticket and implement the feature in this code base" isn't really worth $200K/year.
My argument here is that what is worth $200k+ is the ability to distinguish the changes that must get thorough, critical review from those that need only a couple of specific things verified and those that require no manual review at all.
Our jobs have never been to write code. We’ve been saying for decades that LoC isn’t a rational way to measure engineering output, and now we have our chance to structurally change that system before processes re-solidify. In fact, I think the flexibility to adopt new systems and the experience and foresight to choose a path that is better than the status quo without throwing out everything we’ve learned is going to be what sets companies apart and makes individual careers over the next few years.
> we have leadership air dropping AI generated manifestos on the product owners and product owners having to use AI to transmute all that shit into 1,500 word Jira tickets that are 10% necessary feature work and 90% LLM boilerplate.
There’s an important criticism here IMO - the relationship between “business” and “engineering” is changing drastically. It’s going to be a challenge to set the expectation that just because Marketing was able to vibe-code a prototype in a day, actual implementation may well take weeks or months. Engineering should be considering things like security, scalability, and systems integration that aren’t a concern for Marketing - that’s why we’re being paid!
> So now you have software engineers whose job has changed radically to the point that the hardest part about being a software engineer is just filtering through AI generated artifacts from all directions just to try to get a feature out the door.
I think it’s an extremely heterogenous landscape right now. Where I work I’m struggling mostly with organization - keeping the (literal) dozens of inbound features that come in every day straight long enough to hook up an agent harness, review, validate, and deploy. At other companies the issues seem to revolve around their Agile-based processes. Or maybe it’s non-technical vibe-coders and their expectations. Or maybe it’s executive leadership flirting with AI psychosis.
Everything is in flux. It’s stressful and exciting, and I’m thankful to be around for it, even if I am in my 40s at this point and looking at the core skills I’ve built rapidly become worth exponentially less. It’s a huge opportunity for growth.
That led to a reduction in knowledge of assembly in the average programmer but the people who specialize in it haven't gotten any worse at it.
The result was a generation of programmers who make useful software while very few of them understand the machine they program. You could easily make either a positive or negative value judgement about that result.
Engineers, free from writing thousands of lines of boilerplate, now have more time to review code and be more efficient in that armed with LLMs. E.g. instead of asking a question about the code and waiting for the author to respond, I can ask an LLM and get the answer almost immediately. Or, if I see some strange code being changed, I am always curious why it was there in the first place and, instead of tracing it through git blame over formatting, renaming, and moving files around dirs I can delegate this to an LLM. If I have doubts about it working in some environment I can, again, make an LLM test it out instead of asking the author etc. etc.
And no, telling Claude to implement a Jira ticket is not worth $200K/year. Checking if it has not done something stupid and correcting it when it's trying to - is.
The snake eating it's own tail for llm software development has really been met with a shoulder shrug whenever it gets brought up. At best you might have a small cohort of developers that don't cook their brains with AI and their reward for that appears to be having to review terrible AI code written by people who have cooked their brains.
// The need for ongoing friction in long-term skill formation.
The subtitle of the story tells it all.
There are some people who seek out friction. Think about an athlete or a hardcore nerd.
The best engineers are ones who were fascinated with computers and learning as kids and persued it at every opportunity. Found their own friction in other words.
For those kinds of people, friction-seeking is the constant and what LLMs did is moved the point of where the friction occurs.
For example - the best engineers I've worked with didn't necessarily have lots of experience coding in assembly because that kind of friction was no longer necessary. But they could solve hard problems (and if a problem really required assembly they could go learn it)
What I think will be hit much harder by AI is the low tier engineer. Someone who was never truly curious and committed to it, for whom it was just a job. For example a typical offshore ticket pusher kind of person. That kind of person never went out to find friction and that's the kind of thing that's never going to fly again - if I want mediocre or average, the LLMs are sufficient
I think this expresses what I've been trying to form in my mind.
I've been trying to map the LLM advancements and the current state of software development onto prior technological improvements. History is littered with similar cases where the abstraction layer ends up getting lifted, and people struggle with getting accustomed to working at that higher abstraction level.
For the people that fall in love with a single abstraction layer or don't have an interest in learning new paradigms, when their known pattern is abstracted away, they're condemned to being left behind, either unwilling or unable to adapt.
I don't think any industry is free from this, any person in any industry/profession over a period of 20 years or more has likely had to undergo massive adjustments as technology changed their field.
We're not unique, but that doesn't stop it from feeling so jarring when it happens to us
> History is littered with similar cases where the abstraction layer ends up getting lifted, and people struggle with getting accustomed to working at that higher abstraction level.
But there's no abstraction layer that ends up getting lifted. When I use a library like SDL or a standard like POSIX, I don't tend to look at the underlying implementation. Instead I work with the high level concepts that they come up with. There's no such things with AI tooling. The most similar is when fully vibing software and everyone knows the quality of the result.
I've learned something at every abstraction layer in computing from electronics (hardware), theory of computation (software) to high level programming languages with their paradigms. Same with several domains embodied by libraries. LLM tooling is more like shamanic ritual than engineering.
Yeah, a bespoke program that does exactly what I need it to do, at a speed that I had forgotten was possible on computers, with customization that is an exact fit to me, at a cost that is smaller than a rounding error.
I get that LLMs struggle with the old paradigm of a single piece of software meant to serve every conceivable use case of every conceivable user, but I kinda hope that paradigm dies.
I think coding experts who are only experts at writing code will also be hit pretty hard. Every problem given to them is solved by writing code, and only code, their way and only their way. I think people like that are having a very hard time relinquishing control regardless of the quality and correctness of the LLM output. If it didn't come from them, or conform to their conventions and style, then it's wrong. These are the people that seem to be terrific developers but then when asked to be a team lead and bring a group of developers to their level completely fail.
Curiosity is really key, and I find that LLMs diminish curiosity in people that might otherwise have it because the allure of the answer is often a lot stronger than the allure of the friction. I guess that's just the distilling of the industry over time, but that could certainly lead to a shortage of expertise.
I mean if you think about it, that sounds like a miserable existence to begin with.
I would not rule out that some people will use this.. uh.. disruption, to start pursuing something that actually brings them joy.
Of course, bills need to be paid and it's not all that simple. But at least there might be such a silver lining.
As a tech educator I 100% agree. LLMs are not going to become a "new compiler" where we don't have to worry about the code any more. There's a reason we trust deterministic systems.
I've been worried about this a lot, I even created an agent skill called do-i-understand that's designed for novice devs (and experienced too, because atrophy) where the LLM asks you questions about the PR you're about to submit. I've found it helps a lot: https://github.com/AnthonyPAlicea/skills/blob/main/skills/do...
One way or another, there will be a skill reckoning.
they probably mean deterministic in the sense of traditional computer programs consisting of if-else decision points and ordinary cpu logic, as opposed to numerical models (which can of course be deterministic too, as you pointed out)
I strongly agree with the concept that cognitive friction is the engine of learning.
First and foremost, it's an issue of "dependency": if you stop training the "muscle" of logic and reasoning, it gradually atrophies, just like unused physical muscles. You become dependent on external tools that replace a capability you once had yourself.
A historical example that brought about a similar shift is this: when the production process moved from the craftsman's mind and hands to the Fordist factory (and the assembly line), the skill of building things shifted from human craftsmanship to anonymous, structured processes.
Bit by bit, traditional artisans lost their knowledge and "know-how." Today, having a piece of furniture in our home depends on a massive production and supply chain; the "average" person no longer has the ability to build it themselves.
> "...if you stop training the "muscle" of logic and reasoning, it gradually atrophies, just like unused physical muscles. You become dependent on external tools that replace a capability you once had yourself."
For a bit of extra unease, consider that these people will retain their right to vote, both in making business decisions as employees, in shareholder meetings as shareholders, and in government elections as voters, despite their atrophied reasoning abilities. Who will wind up whispering what to vote for in the ears of these reverse centaurs?
> cognitive friction is the engine of learning.
I agree 100% too.
This is why in order to understand new codebase or to ramp up to new projects, I use AI to generate a textbook style reading material for me along with "verify yourself" types of exercises along the way. Then I print them on a paper and read it using a pencil/pen and take notes.
Because of my math training, I am in the habit of slowing down to read textbook style texts which helps.
Exactly the type of usage I would hope to see advocated for in the future. As I wrote, when leveraged with Socratic methods and a sort of "interactive documentation", they can really boost one's understanding. Of course, this is predicated on the individual having enough information to know the right questions to ask.
I also do this. It would be better I think to generate exact references to primary and canonical sources to avoid learning LLMisms, or to feed it these and have it ensure its material is anchored to those good references.
Another example is autocorrect absolutely destroying the world's ability to spell even the most basic words. Ever since mobile phones came along and removed the friction of walking to the dictionary, everywhere I look people write as if they have extremely serious brain trauma - grown adults struggling to write as well as we used to as children!
> Today, having a piece of furniture in our home depends on a massive production and supply chain; the "average" person no longer has the ability to build it themselves.
Tangential, but this is one of many reasons that I (and I suspect many others here) have taken up wood working.
> Bainbridge argues that new, severe problems are caused by automating most of the work, while the human operator is responsible for tasks that can not be automated. Thus, operators will not practice skills as part of their ongoing work. Their work now also includes exhausting monitoring tasks. Thus, rather than needing less training, operators need to be trained more to be ready for the rare but crucial interventions
TBH it was already pretty bad. There is a stark difference between the best and the average in my experience. The top, say, ten percent of coders are vastly better than anyone else when it comes to anything but boilerplate glue code(which is still needed and is better done by average coders anyway).
This is speaking from my experience as a systems/c/c++ guy. If you are a js web frontend guy, python, or whatever I have no idea if this applies to you.
My personal antidote is to write unassisted Zig in my spare time (though C would be fine as well). While I can't back with data I certainly feel sharper.
As always, the innovators will thrive and become the new experts in the field. Everyone else will die out.
It's impossible to buy the FUD that expertise will somehow vanish because people use AI. People come in a variety of capabilities, challenges, and personal interests. Expertise varies from person to person. Those with personality traits that are more likely to take risks and persist against adversity will be the innovators and experts. We've seen this played out countless times.
You have to ignore everything you know intrinsically but can't articulate about innovation, technology, and evolution to drink this FUD Kool-Aid.
Most computer science programs don't teach "coding" though. There is pretty heavy emphasis on datastructures, algorithms, and system design. I don't think AI atrophies those skills / knowledge as much as the coding language and I think in many cases actually improves them, especially if you are reading the code that is produced and at least understanding the flow of it.
I agree that AI reliance is hurting engineer's language understanding, but I don't know how much we should care.
I don't think that's quite correct; every CS program I've seen involves writing code, quite a lot of it. What they don't teach is /software engineering/: working large codebases, maintaining code, dealing with legacy code, long-term collaboration, working with product managed and designers, etc. That's stuff that you mostly end up having to pickup in the real world outside school.
> especially if you are reading the code that is produced and at least understanding the flow of it.
This is the key point. Should we stay vigilant and review all the code, ensuring we have a decept grasp of what it does. Or just green light everything and treat the code itself as a black box. I would hope we do the former, but many are pushing for the latter.
The issue with this in practice is that the rate at which code is produced is often substantially higher than what any human reviewer is capable of processing mentally.
Example from my workplace: pull requests that were already too long before LLMs by a few hundred or a couple of thousand lines are now several thousand lines longer and they get opened much more frequently. The primary source of these PRs is a well-respected senior team member with a ton more experience than everyone else by a measure of decades so nobody dares tell him to stop. Other team members aren’t far behind in the PR opening rate and admittedly I am one of them, but I try to mitigate the issue by doing multiple LLM reviews and anticipating when it’s ok to just approve it myself (have never had an issue in making this judgement thankfully). Part of this IMO is almost certainly because there’s now peer-pressure within the team to churn out a ton more code and that’s adding to the already clear message from leadership that we need to leverage AI to deliver things at unprecedented levels.
My point is that it’s not just that some people are pushing for greenlighting everything without due diligence, it’s also that they’re pushing others to toward that outcome with their extreme AI enthusiasm which they don’t see the ramifications of or just don’t care.
> Most computer science programs don't teach "coding" though. There is pretty heavy emphasis on datastructures, algorithms, and system design
Those skills are often taught in an idealized situation like very small projects. In the real world, there are a lot more practices and techniques to develop software in a pragmatic way. Those "engineering practices" are much more useful than the above, unless you're in some specific domains.
The video seemed to indicate it was a study they ran (she refers to the examples as "our subjects were two of twenty four participants"). It seems they are just citing the study. While it doesn't change the study or the takeaways, I'll update the post for clarification!
I'm looking forward to having the same experience as COBOL engineers did after retirement: companies backing up the brinks truck to anyone with the skills to keep the lights on.
now that's a phrase I have only heard in one other context: as an alleged quote from former Boston Celtics player Isaiah Thomas (who was really about 5'7 despite his official listed 5'9; yes, professional basketball player in NBA). This was around 2017 when he had just come off a career year and was soon due for a contract extension in the low 9 figures. He got injured and bounced around for a few years before eventually leaving the league, and never got that payday.
It’s commonly used in college sports where one school’s fan base argue for poaching a coach at another school. “Back up the Brinks truck” == throw money at him.
I often think back to the early GPT days before agents as maybe the last time I will deeply learn a technical subject. I think that was when I actually learned the most as I had to stay completely in the loop. I had recently started a new job and had to work with k8s for the first time. Using GPT to help me implement new services and help me diagnose and fix issues with kubectl taught me so much.
Now with agents I don't have any insights into what its doing with kubectl and would have no reason to learn how it works (other than my own curiosity).
I agree that the human no longer gains expertise, but I am not sure that will matter in the long run.
The end result is a total and complete reliance on AI providers. It's not really a secret; that's why I started the article with the direct quote from Altman. They would love to see a world where nobody is able to do any type of development work without an LLM subscription enabled...even though they also admit that these models perform best when they are steered by someone who is highly capable and knowledgeable in the first place. They're powerful, but they are also VERY fallible.
Local AI would thwart this vision (which is why they're desperately also trying to ruin the hardware market at the same time), but the effort it would take to get local models to run as well as these frontier models is an odd thing to pursue just to avoid learning the fundamentals and gaining direct experience.
While I agree Local AI would be a good goal, even the huge open weight models that can't be run locally are already thwarting the US AI lab's duopoly. Other companies can and do serve up these open weight models, and without having to amortize R&D they are able to serve the tokens at a pretty cheap rate.
I hope that this will expose the true value of education. It's a new scale perhaps, but not a new problem. We sit in classrooms even though books and YouTube videos make available the teachings of far more illustrious teachers than the one in front of us. We watch things written out agonizingly slowly in chalk. We learned to do algebra and calculus even though we had our TI-86s and Mathematica. We learn in Haskell and Scheme even though those are rarely used later. We write toy compilers and write essays and repeat experiments with known outcomes in labs. We do it to learn how to think, how to build. We just have to be even more deliberate about it now.
So far I think I have learned far more from LLMs than I've lost to them. I forget some syntax, definitely. But I now reach for a much wider range of tools that I have become familiar with because of LLMs.
Yeah, I have to say some of this deskilling argument sounds like Socrates complaining about the invention of writing ruining people's ability to truly own the text. I really don't mourn my lost assembly skills or my lost C skills. And I'm not missing the details of whatever web framework du jour my Claude is vibing for me on my extremely useful dataviz internal tools; and I have learned how to rein the agents in to not ruin my high-performance Go code and yet debug concurrency issues. Maybe coding qua coding is what some people find their summum bonum, but for me programming is instrumentive - to build real things in the world that do things to make something better. When I want pure virtuosity of intellectual construction, I read maths papers in arxiv.
(Not to say that the current state of the art with LLMs merits borrowing money for a multi-trillion spend on hardware that will be obsolete in 5 years, but for me that is a different issue).
The master decrying the invention that resulted in more wisdom for more people than anything prior:
"For this invention will produce forgetfulness in the minds of those who learn to use it, because they will not practice their memory. Their trust in writing, produced by external characters which are no part of themselves, will discourage the use of their own memory within them. You have invented an elixir not of memory, but of reminding; and you offer your pupils the appearance of wisdom, not true wisdom, for they will read many things without instruction and will therefore seem [275b] to know many things, when they are for the most part ignorant and hard to get along with, since they are not wise, but only appear wise." (quoting from https://www.historyofinformation.com/detail.php?id=3439)
I posted this elsewhere, but that story has a lot more nuance.
Plato (who actually wrote these quotes, because Socrates only spoke) wasn't against just "writing" in a general sense, or that he felt it was going to hold humanity back. He was against treatises and felt that someone could "memorize facts" without having to actually think about them on a deep level. Which I think, ironically, is not all that off the mark especially in the age of the internet and now LLMs, right? He also felt that memory could decay if we relied on written facts instead of having dialogues, which again, not all that off the mark, either. He underestimated the compounding capability of technology and our ability to record data and information, but people remember a lot less these days than they used to because we'd largely given up that ability in exchange for the instant information machines.
Of course, he's just one man (or two, if you consider he was conveying Socrates' thoughts, as well) who lived in ancient times and couldn't possibly foresee how technology would evolve...but when I went back and read his concerns, it was ironic to admit that much of what he warned about still applies to this day. People are far less informed despite having access to more information than we've ever had because we've exchanged the ability to remember for the ability to just look things up. And, we also have less meaningful dialog than ever before; people just sit on social media, copying and pasting "facts" to each other, instead of having actual productive discourse. When I look around, the world doesn't seem brimming with mindful critical thinkers and there's reasons for that (many reasons, of course).
AI tooling can be a boon to learning, but it requires us to stop using them for code generation as a primary purpose (at least for juniors) and instead advocate for Socratic workflows that still require manual coding practices. When someone decides to code something, it's not just learning syntax; it engages a variety of mental disciplines from critical thinking to planning to creative problem solving to logic and math, even.
There is an argument to be made though that writing stuff down yourself results in better retention than simply reading the same material. People still write by hand and take notes. And they still process ideas and thoughts verbally. Doing so is often more effective than going in circles with a chatbot.
Yeah. Thought in Socrates' defense I think the jury is still out on writing, in the scale of human existence it's still very new. Give it a hundred thousand years or so.
It's basically laid out in the article: the reason that you are able to gain so much from LLMs is because of your decades of experience that predated their existence. Ability begets ability, but if that ability doesn't take root, these more advanced toolsets don't yield the same outcome (and are likely to do harm).
To be clear, I feel the same way as you. I have been doing this for about twenty-five years, and I have developed a keen sense of taste and judgment to know how to properly scrutinize and utilize these systems. I definitely feel like I am able to learn things faster, but that is directly correlated to having experienced a lot of friction over the years that cultivated said taste and judgment in the first place.
I imagine it's sort of similar to someone who's just getting started in mathematics, but is introduced to WolframAlpha, versus someone with decades of experience in the field and what they learn with it.
I fall into the category of senior engineers who benefit from LLMs for all the reasons mentioned in this post. I find it's possible to agree completely with sentiments like this and still feel as if this is all written in the sand below the high tide line, and ten years from now nobody will care about this.
Horsemanship and sailing were both specialized skills of high value to society, and now they're not. But in each case there was probably a liminal period, when being an accomplished horseman or sailor was still valuable, even as motors were taking over. Eventually that period ended, as the new generations without those skills found ways to get by with cars and motorboats.
This analogy breaks down because computers aren't going away unlike horse drawn carriages or sailing vessels. Computers aren't being replaced by something. It's that fewer people will understand how they work while the efficiency and reach of computers extends into areas of life previously unreachable due to the scale of having humans build it or the refusal of humans.
I could wax philosophical about what that leads to but enough people already have.
I think we now entering this state where code is cheap but good well made programs will be rarer then ever, a lot of crap can be made and a lot of programs made without ever thinking the problem through properly will occur.
It might be like how an archeologist since the invention of plastic can date the period of the soil as post plastic, if one could cut open the software stack of the coming systems that will be built in the near future one would could data that code as post agentic LLM as programs are going to be mostly bloated ad-hoc, poorly thought-out and patched in a way that does not concern it's self with the correctness of the algorithms or data structures chosen.
I am not Anti-AI but I think it's going to be interesting and I am surprised at how bad a program ends up when one tries to "vibe code", though often ends up working, though in isolation and when used with discipline (that the tools themselves psychologically make it harder to do) can produce some very good code and being able to use loops to solve difficult problems or problems that simply would have require banging ones head against the problem many times is very profitable.
Yes every time I tried to understand ffmpeg I got a little of it for a little bit and then lost it by 2 days or so. Just getting LLM's to do their thing for ffmpeg is great. the other one is socat an utility I used before but LLM's can do magic with it.
> This applied friction is directly what builds "developer intuition" (or "taste"). The Germans have a great word for this: Fingerspitzengefühl (fingertip feeling). It’s the muscle memory that triggers when a developer looks at something and thinks, “yeah...this is probably going to cause problems.”
No, we do not have that word for that.
Yes, we have that word. No, it does not mean that.
Any LLM proofreading could've told the author that, for that matter.
__
What Fingerspitzengefühl actually means could be described as "tact", a precise approach to something and general attention to detail that leads to success.
Or.. the lack of all of that leading to blunt failure.
__
I would of course comment with more Fingerspitzengefühl for the emotional needs of the author if the writing wasn't just a sales funnel for their courses and whatever else (+ posted by them themselves.)
This happened before with “real” engineering. Engineers a hundred years ago used to be proficient with slide rules and mental math and 3D assembly by hand. Products designed in that era often lasted a hundred years, like Singer sewing machines and Lada cars and 500 year old churches. Now things aren’t built like they used to be, and few people can rebuild civilization by hand like they did. But the world moves on fine!
I don’t think there’s a big societal problem here, except that the agentic programming developers may be out of a job like the rest of us. If AI continues improving it will just keep taking over whatever skills are involved in the current AI coding meta are. If it stops improving then the meta will stabilize and after 10 years everyone will have longevity in it.
I also see no evidence that AI programming is a difficult skill that cannot be learned by any intelligent person in much shorter amounts of time than previous professional skills required.
Depends on what you are programming. I have been using it to build things that have never existed before, that I have wanted for over a decade. Experience gives one more imagination.
> I also see no evidence that AI programming is a difficult skill that cannot be learned by any intelligent person in much shorter amounts of time than previous professional skills required.
I do not think anyone could do the job I do with AI without my decades of experience, is my point. For now.
Substantiveness of this article notwithstanding, I do want to point out that the author also runs (or is planning on running) a programming course which to me may be a slight conflict of interest. I think the author may altogether be well-intentioned, but it IS something that I'm going to keep in the back of my mind as I consume this content.
I do understand the sentiment of applying "constraints" over AI usage to certain scopes of our development work.
This has an issue in practice.
As an example, Function Programming (FP) has a high constraint on programming, where some don't even have loops (gotta use recursions all the way), and almost no state mutation (Exlixir I believe have a mutable state? forgot).
FP sounds great with all those constraints. But what about the adoption?
It's hard to switch your mindset, and not as natural, thus long leraning curve, and not as wide adopted <- productivity goes down as all others need to know and use it well. That's why still imperative and OOP languages are ruling the world.
Same for these constraints. The author promoting using AIs only to subsets (no coding? wtf) is something the majority won't follow. Yes, the constraints sound great, but at what cost? By the time one learns everything, and everyone moves along with AI building stuff fast and cheap, they will be left behind.
---
I do not like the author trying to make his points authoritative by using Halo effect using quotes from the industry and could have had been taken out of context (That's where AIs mess up the most, they only get chunks of text and lost context of before and after exerpts).
Your inclusion of FP is interesting in an unintended way: LLMs actually thrive in a highly constrained system like you describe. "It's hard to switch your mindset" - isn't nearly as true for an LLM, it can usually adopt those languages quite readily.
LLMs don't particularly care how hard the type system or borrow checker is (as long as it has good error messages that can guide it to a solution), they can grind away at these things, and what you describe as a negative actually becomes a strong positive. I think we'll see strongly typed systems plus some level of formal verification become popular as an AI-preferred language.
When the burden of review is lowered because you have purely functional building blocks, with lots of formally verified chunks - people will choose that simply out of (productive, beneficial) laziness. Not to mention we're in a ram crunch, and being able to run your stuff far cheaper has an immediate financial incentive.
> LLMs can detect patterns at a scale that no human ever could, but patterns only get you so far.
This is asserted without evidence, and from a scientific standpoint, unjustified.
First, "detect patterns" makes it sound like a classification task, "is this a picture of a cat". But LLMs transform the input.
For example, an LLM can translate text between two languages while properly handling the names of the people described, no matter what those names are. That shows they are representing the text in a somewhat abstract way, that they can perform operations on that representation, and also convert it to useful output.
And, what I just described is the most general form of information processing algorithm. Science is not aware of any limitations in principle on such systems.
I am not saying LLMs have no limits, but "they only recognize patterns, and that is a true limit" is not a good argument.
Yes, but it's also building other skills. There's no friction to learning something new anymore, and that's really quite amazing.
E.g. I've gotten pretty good at git thanks to AI because it forced me into more complex workflows. I still can't remember the commands but I really grok the concepts and terminology on a far deeper level than before, where I made a conscious effort to stay on the happy path lest I get into some odd state I don't understand. Pretty sure those "odd states" would seem quite clear to me now.
And so what? How do you think people are going to be working on software in 10 years. They're going to use coding agents and harnesses that work much better than the current ones (although I don't think we're getting anything but incremental improvement from here on out). They will run on commodity hardware. I first got paid for writing code in 1999, I've worked at publicly traded software companies and failed startups, in every language out there. And if LLMs "atrophy" the expertise I built up over those years, then so what? What is lost, exactly? I stopped programming in C++ about 10 years ago when Rust seemed to get good. I no longer remember a lot of the specialized knowledge I had in C++. My life goes on. I am now becoming an expert at building software with agents. The problem is still the same, you still have to apply engineering principles. You just don't have to write the code.
I've, more than once, gone long spans of time doing purely management and architecture level work, then returned to programming. A little rusty? Sure. But I never forgot how to program or lost the skills that actually matter. Seems mostly like a non-issue for people already in the field.
The real issue will be with people that never learn it in the first place.
And even that isn't so bad, learning programming isn't some insurmountable hurdle. It took me less than 6 months from no knowledge to my first job.
> It took me less than 6 months from no knowledge to my first job.
I agreed with you up until this sentence. Entry-level devs with 6 months of require an enormous amount of time and mentoring until they're able to really perform well. I think the greater concern being expressed is not will people be able to type syntactically correct code?, it's will people be able to write software well?.
I think this is more “gray” than this. I feel like I can rip through ideas more quickly then ever before and as a result get a lot better at designing systems and (for my work) get a lot better at designing data/model experiments.
But I empathize with the author. I think the frustration comes from the fact that other folks in your organization use these tools to “get by” more effectively then before. In the past, a disinterested engineer might ship less under the umbrage of a difficult engineering problem. With the pressure to “ship more” now that there’s AI, the same engineer is using AI to spam PRs to prove they’re working, even the quality of work is low (because quality of thought is low).
Then you make it write code in the tickets for the tricky stuff.
Have it reference relevant documentation/APIs.
Then there is no lost knowledge.
(Still, sometimes it sneaks in some "helpful" belt and suspenders, but for the most part, I know everything that is going on in the code base with this method.)
Writing code was never the job. It was a small part of the job that can now be delegated to AI.
Eliciting the right requirements and user needs. Creating a sound architecture and design. Validating final product meets the requirements. Ensuring there is good test coverage. You know `Engineering`. Mechanical and Electrical engineers didn't go away because of CAD.
> Eliciting the right requirements and user needs. Creating a sound architecture and design. Validating final product meets the requirements. Ensuring there is good test coverage.
Besides your first point, the rest can be delegated to AI as well. And you don't need to be a "software engineer" to spec out requirements and user needs.
Literally all those skills are the result of decades of friction, challenge, and deep domain knowledge...all of which are circumvented when LLMs are utilized early in the process.
CAD is also useless to Mechanical and Electrical engineers who didn't learn fundamental the skills that it streamlines.
Does anyone else feel like the web has become so "readable" it is illegible?
I have such a hard time quickly reading pages like TFA. Headings, bullets, line-spacing, width, link hover animations, even the font. Even reader mode defaults aren't great for me.
In my experience human design -> agent code -> agent review -> release -> agent observer, does not (at all!) work. One absolutely has to be understand what is going on, how the system works, how the H/W works, etc, to catch agent errors.
The proposition of us losing those skills is terrifying.
Maybe AI will get better and we truly won't need a human in the loop. Then please also have all PRs signed by whatever model wrote them, and do not come to me for trouble-shooting.
But what we're gonna manage the codebase in, then? What kind of language?
Will it be a natural language, or a formal one?
I feel that "coding" cannot collapse. Coding is just translation from natural to formal language. Somebody needs to write the specs. And writing/maintaining them in natural language brings a lot of fun - shifting interpretation, inconsistency, missing specification, etc.
I don't think it's progress for the field of SW engineering. But at least more "shareholder value" will be created.
This just feels like someone whining about how ORMs result in $#1tty data access code because lazy programmers don't take the time to learn how to use a database.
When that happens, (developer thinks using an ORM means they don't need to know how to use a database,) the project eventually fails. It's the same thing with AI; just like companies learned that they need to make sure developers actually know how to use a database, companies will learn that their engineers actually need to know how to program.
Coders should be treated as as well as as other software users. Tools like Claude code, Antigravity, and others are applying AI to solving that problem.
I know how to create a multi year P&L projection from scratch. It gives me some insight into how much capital a startup will need to get to breakeven. A good AI tool that creates that same spreadsheet in minutes should also be able to explain why different businesses have different shaped cash flow troughs. I don't think people using tools like that is bad even if they don't gain deep insight.
At some point it won’t be realistic to ask a human to solve a technical problem any more. I’m not suggesting it will happen soon but I can at least imagine that day coming. Whereas before modern “AI”, that possibility wasn’t even on my radar.
Something I noticed trying to learn rust through models, they're inherently tuned to just finish everything with even the slightest provocation so it's very hard to use them to actually learn anything, they kinda just want to do everything themselves.
One interesting feature of vibe coding is that it allows you to write code in specialized areas where you have no expert knowledge. That makes the code effectively impossible to review in any useful way.
For example, I have a vibe coded Python monstrosity which I use for stitching together scans of large format negatives. The enlarging lens that I use for scanning has strong chromatic aberration in the red channel but is otherwise very sharp†. I got Codex to figure out some way of using the green and blue channels to sharpen the luminance while preserving the color information from the red channel. I sort of vaguely understand how this works, but not nearly well enough to review all the horrid numpy code that implements it.
For my use case, there is really no reason to look at the code, as I am only interested in the output. As disconcerting as it is, I feel that this is going to become more and more normalized. Looking at the code that your LLM generates will become like looking at the assembly output of your C compiler.
† This makes a lot of sense, if you think about the color of a darkroom safelight...
Coding expertise was on a downward trend way before AI arrived. Does nobody remember how most "engineers" were literally copying and pasting snippets from StackOverflow they didn't understand, and the snippets usually had tons of bugs? How many of you know Assembly well? How many of you can decompile a program, or write your own compiler or VM? How many of you could pass a computer science course (that you didn't take at Uni)?
We've been hearing about the demise of skills since before the invention of the loom. The only skills we've truly lost are for things nobody uses or makes anymore. What actually happens is, we develop a new tool that's better than the old tool, and we get good at using the new tool. We then have a class of people whose job it is to maintain the new tools. We don't need that many of them because the whole point is to eliminate that labor.
Your jobs as software engineers are going away, it's that simple. Your new job is to use the AI tools to make products. Somebody else's job will be to make the AI a better software engineer.
Speaking on a handful of these that I'm familiar enough with to opine on:
- Photography hasn't erased other visual arts. Both require a separate skillset; both have their uses and their enthusiasts. Photography benefits from having a lower skill floor, in my experience doing it, and this in turn has led to commercial photography eclipsing commercial painting (barring things like painting buildings) as a viable business model, largely.
I'm not sure if we're better or worse off as a society for having gone through that shift.
- Television is arguably not a net positive in society (I won't say net negative yet, but there are some reasonable arguments for that position). It enables a form of passive consumption that allows bad arguments to come through as persuasive through the gradual (on the scale of months and years) numbing of critical appraisal of the media being consumed.
This has more- and less-pronounced areas. In the US (which is geographically close to me and which has an outsize proportion on my own country's culture), Fox News is an excellent example of how TV can erode critical thinking.
- Smartphones arguably reproduce and amplify this problem, and tack on others: they make it increasingly easy to dissociate from daily life, and make it easier to continue consuming content from actors increasingly interested in steering conversations in one direction or another (on the most benign end, it's influencers trying to get you to buy things, which obviously has negative economic impacts on an individual scale; on the more overtly harmful end, they serve as a hardware channel for less-regulated or unregulated extreme and hateful content).
Just like with TV, smartphones have clear benefits and utility in daily life. They aren't abjectly bad inventions, but they have a serious societal toll, and it's difficult to ignore that.
- In aviation, the first generation of (relatively modern) automation did lead to a high number of accidents and loss of life. American Airlines flight 965, in 1995, is a well-studied case of an over-reliance on flight automation before it was widely understood among pilots what the limits of these early-modern incarnations of autoflight were.
In the wake of that accident, many aviation regulators instituted or began requiring that manufacturers and airlines formalize training and operational procedures that protect thoroughly against this sort of cognitive gap.
I think that last example (and many aviation accidents) are an interesting example to study in the context of software. Aviation's extremely and conspicuously responsive to accidents, often requiring procedural and hardware changes as a result of these. They can be as simple as a software patch, or as complex as the redesign of a component because a human factors analysis found that it could be confusing under stress.
In software, we like to think that we're a serious industry, sometimes. We spent much of the 2010s going in what I found to be a good direction: the development of IaC, the popularization and standardization of tools like Git, registries for vulnerabilities like the NVD, and even the development of languages like Rust which address historical technical shortfalls of high-performance, low-level languages like C; but we always fell short, industry-wide, on matching these toolchain upgrades with technically-rigorous procedures.
It kinda seems like wishful thinking on our part as software engineers that one day we'll be able to go "Hah, miss me now?" but that just isn't the writing that I see on the wall.
Over the past year, I've AI-generated two large pieces of software over 2000 commits without having much depth in either domain, and my ability to deliver value as a veteran software engineer has only shrunk over the months or it can be trivially extracted into a reusable markdown file like "design bar: ensure things 'by construction' where possible", a lesson I learned viscerally over decades.
Because I can still reason and dig into things and ask questions and have the domain explained to me out of curiosity, the same way I could build anything that was originally out of my depth.
But what I've noticed is that the need for any corrective power has gone or is going to nil and I'm mainly doing directional work. And even then, I can constantly have sota models "rank the top options and recommend one", and it's almost always the right way forward.
I used to read every line of every plan that LLMs generated, but over time I've realized I have less and less to correct or, more impressively, the LLM had foresights I never considered. The same thing happens with implementation.
Periodically I can spawn a bunch of agents to evaluate the system adversarially to find improvements, and the findings have been so good that it's evident that soon I can just automate that too.
In other words, the skills I need to excel here are more curiosity and patience than tech expertise -- the things that got me into software engineering in the first place since that was the only way to build things.
You got it. You may well be right, but from my standpoint it's hard to see how you'd know if you weren't. Yeah, the models fill in vagueness with detail sourced from "what's likely and helpful". (In one of my particular domains, that's "use SHAP values for everything everywhere".) Yeah, that detail's increasingly unlikely to be "wrong" in a way that you could explicitly argue. Is it what an expert would come up with? Is it the best it could be? Is there something else that we would have done had we been forced to think it through? Will it ever be _surprising_?
I know, all irrelevant little philosophical roadbumps. The artifact satisfies the buyer. I think I'm going to have to step away from this industry pretty soon, at least for a while.
look to past examples. when the greeks developed writing, people complained that the youth were losing the ability to memorize much content. and that is true, but with writing they were able to do better still and that old skill -- though useful -- was lost.
Plato wasn't against just "writing" in a general sense, or that he felt it was going to hold humanity back. He was against treatises and felt that someone could "memorize facts" without having to actually think about them on a deep level. Which I think, ironically, is not all that off the mark especially in the age of the internet and now LLMs, right? He also felt that memory could decay if we relied on written facts instead of having dialogues, which again, not all that off the mark, either. He underestimated the compounding capability of technology and our ability to record data and information, but people remember a lot less these days than they used to because we'd largely given up that ability in exchange for the instant information machines.
Of course, he's just one man (or two, if you consider he was conveying Socrates' thoughts, as well) who lived in ancient times and couldn't possibly foresee how technology would evolve...but when I went back and read his concerns, it was ironic to admit that much of what he warned about still applies to this day. People are far less informed despite having access to more information than we've ever had because we've exchanged the ability to remember for the ability to just look things up. And, we also have less meaningful dialog than ever before; people just sit on social media, copying and pasting "facts" to each other, instead of having actual productive discourse.
AI tooling can be a boon to learning, but it requires us to stop using them for code generation as a primary purpose (at least for juniors) and instead advocate for Socratic workflows that still require manual coding practices. When someone decides to code something, it's not just learning syntax; it engages a variety of mental disciplines from critical thinking to planning to creative problem solving to logic and math, even.
This seems right to me, but I think that it's likely that we haven't fully figured out how to integrate AI into coding imo. I think there is user side thing that we haven’t fully worked out yet.
That's the whole scam. It's a ransom with more steps.
First they fuck up your codebase with so much slop that no human does (or could) understand it.
Now if you want to keep making bug fixes, you're on the hook for whatever Anthropic wants to charge you.
The solution is to make AI developers the scapegoat. Fire the people at the top of the AI leaderboards, with malice. Otherwise, lazy AI-addict developers are going to destroy what remains of your company's IP.
I used to think this was a problem. But now? I don’t think it matters.
Why? Because people are going to get so much more ambitious about the stuff they build with AI, that code expertise for the stuff they are building is already scarce anyway. People will choose difficult and less popular languages, and will replace open source packages with stuff built entirely in-house by AI.
Going forward, there is no choice except for AI to fully replace the need for code expertise. Depending on humans won’t be scalable.
Do we even need to get LLMs to create code in the end then? Can we imagine a future where the thing simply does it all, you deploy the LLM like a docker image and have it process all requests directly. Sounds far fetched today, coming to your cluster in five years.
In theory yes, but you would need LLMs that basically have frontier level capabilities but with zero latency, meaning you enter a prompt and as you type the prompt is regenerating instantly like some kind of autocomplete.
I think that would feel like crack for some people who are addicted to using or building stuff with LLMs.
Very few people know how to hand set typography anymore, but many of the conventions still exist and people don't have to understand the origin of a lot of them to use them properly.
Actually, they have for a large majority of the population. And fun fact: that debate is still going on, and the Mathematical Association of America (MAA) enacted a rule prohibiting calculator use in AMC.
Math fosters critical thinking, and we've been in a critical thinking deficit for decades and decades. I don't think we can afford to sink even deeper...
My question would be if "math" is the right tool for that. It can achieve that as a side-effect, but how we express it and talk about it is full of legacy cruft, cargo-culting and general bullshit that makes it not exactly snap into place for all kinds of brain wirings.
Math I believe is not an end in itself. Maybe with that perspective, solutions that satisfy our needs there can be found
Because the 1890s and 1930s were famous for their population wide critical thinking, of course. Good heavens, what do they teach in the schools these days.
I mean you may be trying to make a point but the counterpoint is Einstein, Bohrs, Planck, Shannon, Turing, Von Nuemann, Feynman, Bohr and most of the other big names all came out during that time.
In contrast since then we've had almost not noteable physicists or Computer Scientists that I'm aware, we've got a couple, Hawking and Knuth. But apart from that I don't know if there is a single scientist I could point out today that has revolutionized our understanding or even done any real breakthrough work in the field of Physics since Einsteins era.
We have gotten better at working as a group. The standard model is a tour de force. It was a collaborative effort over decades but is a lot more complex than anything Einstein did. To some extent we haven’t gotten new physics since then because of reality - we have built higher energy experiments but haven’t got new results.
But my reply was I guess implicitly assuming the post I was replying to was talking about the median level of critical thinking. Your reply is talking about the peaks of creative accomplishment which is not what I was addressing.
A more apt analogy is the diminishing understanding of assembly language and machine instruction sets. I love programming and something got lost in both transitions, first to high level languages and now to coding agents. I am quite sure some contexts will require programming by hand for a long while as there are still people who are writing amd64/avx512/sass by hand. I hope in time we will find new ways to enjoy solving problems with this new programming modality. By now, I am pretty sure it is here to stay and evolve from its current crude form.
> the diminishing understanding of assembly language and machine instruction sets
This is true and nobody complains that GCC or LLVM is making programmers dumber. They are code-generation tools from higher-level source code. Much like LLMs are code-generation tools for higher-level language/prompting.
Man, Usenet had the best online discourse - hardly any angry personal flaming because people dared to disagree. And Twitter, wow that was just pure rationality. slashdot - practically a masters program.
Now, they all did have gems of interaction that would could find by sifting thru the dumb stuff, but I find that property of discourse to be pretty steady. I've even found some interesting comments on YouTube videos and newpaper websites.
Not really the same, since your average calculator doesn't take over executive function. The user thought and decided what needs to happen, down to one extremely well-specified task, which is so unambiguous that it can be completed by rote and without context.
This is more like paying someone else to read your math homework, generate answers, and maybe even submit them on your behalf. If that had been the (average) student experience, math skills really would have collapsed.
I think we'll find that the people doing that, like the people spending all their time and energy reading slasher pulp fiction in the 1950s, won't be doing as well as the people learning to enhance the power of their education and their expertise via LLMs and strategic alternating between GenAI and thinking about problems. And eventually, if society ever steadies up a bit, people will learn the useful ways of using technology in school and youth; and we'll be glad to be alive to see such marvels.
Assuming we can re-establish the will for democracy and taking care of the whole group.
I don't think it's quite as bad, but I'm imagining someone in the 1950s going: "People just have to use methamphetamines responsibly in their work, and then we'll see the massive benefits as widespread adoption unlocks the potential of the human mind."
In other words, a positive outcome with thing X is possible but there are also big behavioral and cybernetic pitfalls, ones which are suspiciously persistent.
I mean I use coffee quite a lot to prop up my software career. I don’t really find purity to be useful for anything except to answer questions like “should I steal this money now that I am alone” or “should I backstab this friend who pissed me off today.”
Those are some big ifs and assumptions unfortunately, and the use of AI unfortunately has an isolating effect that is only likely to move the needle in the wrong direction.
I also can't make butter by churning or soap from lye and animal fat or what ever.
However, I can use mathematica to demonstrate the criteria for various forms topological stability of solutions to many types of differential equations. I'd rather live now.
That's a very good point. People don't pull out their calculator in the grocery store to do analysis of unit costs or whatever, it's too much of a hassle for that small kind of analysis. But calculators have robbed us of the need to be good at doing that math in our heads. So the math doesn't get done.
Many grocery stores will already give you a per oz or per unit price so you can compare pricing for different brands that have different sizes. It's a very handy feature.
My most-common frustration with it is when products I want to compare don't have the same quantity-denominator. (Either a different unit, or sometimes a different dimension like weight versus volume.)
I've many times seen it wrong but more often the same item will have some tags expressed on cost per ounce, others per lb, etc. To discourage comparison. But I dont think people are paying much attention to those numbers anyway. Math has become a scary topic for them altogether.
This is exactly the right analogy. Granted LLMs are a more sophisticated calculator and engineering is still adapting to this wonky/evolving technology. But to pretend it's a coming catastrophe is to ignore every past parallel development - where this has not happened.
Math skills are at their lowest point ever in modern society, and many are still advocating for the reduction in reliance on calculators for this reason.
"Mental math skills" literally did collapse tho??? "Math skills" to write proofs (which a calculator cannot do) is a fundamentally different skill set than that "math skills" of mechanically doing arithmetic (which a calculator does do).
Seems like critical reading skills have already collapsed.
With a calculator you still need to understand the fundamentals of mathematics. You also need to understand how to operate a calculator and know its functions.
Even when removing numbers and manual calculation, being a function or formula oriented mathematician still requires you to know the theory and functionality.
With an LLM, you don't even need to know anything other than asking the robot to perform. You don't understand the output, nor can an unskilled person explain it.
This is the same kind of analogy people make when they "poo-pooh" any sort of technological advance, especially with AI.
"Oh, that old simple device didn't displace anything, so this one which is now quickly taking over every job will be just like that simple one". So I guess people today know how to sew and knit and bake bread?
"Oh, humans were always needed, so it'll be the same this time." Is that true of horses and oxen after cars and tractors were invented?
1. function trampolines, such as in libraries like ATL (Microsoft's COM template library);
2. hand-optimized encoders/decoders for things like ffmpeg;
3. niche applications such as Ben Eater's 6502 breadboard computer (using a modified WozMon and Microsoft BASIC);
4. reading ASM output from C/C++ code in things like the GodBolt site for performance comparisons/analysis;
5. reverse engineering old DOS games;
6. people working on things like compilers (generating optimized assembly), JIT engines, or other code generation -- though this will typically be machine code, possibly with assembly instruction comments;
and other performance critical places where assembly can speed up that part of the code.
In an abstract sense, yes. Compilers let programmers work at a higher level of abstraction. LLMs bring it to an even higher level, though the abstraction is much leakier.
For the time being, a good understanding of programming and domain knowledge gets you much farther than without it. There might not be room for as many (professional) programmers as before, though.
That's because the institutional knowledge was passed along through the act of coding. Skills that I learned in Assembly and FORTRAN are applicable to even modern programming languages...but not to "natural language."
I don't agree. Certain optimization skills are learned on the job if people work in specific areas of compilers or performance.
Also the reality is that modern CPU microarchitectures are so different from what old-school assembly programmers used that their supposed optimizations are likely to be counterproductive.
I'm also concerned about people asking an opaque machine for an answer and stopping there instead of going to different sources, evaluating their credibility, exercising their own ability to construct their own worldview from disparate facts and opinions.
Flagged for the expressing the most obvious and legitimate concern about LLM-backed "research".
"I use AI to generate a summary which I print out and read slowly every time I use a new codebase" is the new propaganda line pushed by Anthropic and OpenAI. They cannot tolerate this rebuttal.
Are engineers still making 100s of thousands a year still just for vibe coding while a nurse and other skilled / highly trained medical professionals working their butt off make half of that?
Do we think there is going to be a correction for the vibe coders?
Pardon I know such question is controversial but thinking ahead.
There's something weird about nurse pay where it doesn't seem to raise even during large shortages of qualified nurses. I think it's artificially lowered by hospital admin, with temporary traveling nurses acting as "scabs" to avoid the normal wage increasing
They do, it’s called the AMA, American Medical Association. Very tight control of who can be a doctor, how many new doctors per year, etc. which ensures salaries stay high. They also fight against other roles like nurses doing too much doctor-like work, helping doctor pay while harming nursing pay.
At least where I live there’s a shortage of doctors so you can’t negotiate with them . There’s a shortage of nurses too, but if you lose 1 nurse because they’re striking it’s bad but things don’t collapse.
With 1 fewer doctor who also knows and is allowed to do things nurse don’t / can’t things get really bad. They also have access to power, are often elected, and are respected.
Nurses need to unionize I think and strike to get something .
Is it my job to make sure that nurses are paid more than I am?
I'll readily admit that they deserve it, I'm a lazy dickhead and they help people, but I think your blame is misplaced if you're insinuating that software engineers are to blame for nurses being paid less than ourselves.
While it depends on the market, job, and hours, but the bedside nurses I know make in the low $100k range. Typical shift structure at the hospital is three 12 hour shifts in a row and then four days off. Most of the time, they are union jobs, so they receive many benefits from collective bargaining such as increased holiday and overtime pay in the range of 1.5-2x normal rate. A nurse practitioner tends to make something in the 180-220k realm.
Again, to be clear, this is highly dependent on the market, job, and hours and one needs to understand the difference between a CNA, RN, NP, etc. However, it's a regulated typically union job that's very secure and in high demand. It probably pays more than what you expect.
Yes, they are still making that much. I am. I don't fully understand why yet but I think it is because of a few reasons:
1. We still need to keep the lights on. If things go wrong, which they still do, humans have to debug it.
2. Jevon's paradox: now the expectation is that we ship stuff faster and there is still a LOT of stuff to do, not less.
3. We are still in the very early days of this and it is too soon for corporations to make a dramatic call like laying off 80% of engineering. We just don't know how that would look like.
I'm seeing it happen in real time. I've seen several folks claim they "vibe-coded" something and upon review it's some seriously low quality garbage. Unfortunately it's just "functional enough" where it's encouraging the one who talked to the robot to continue thinking they can code for some reason.
It is important to ensure that any LLM code is named and shamed.
The industry has always had niches where quality is really important, such as NASA where you install your product in a multi-billion-dollar, one-of-a-kind robot and launch it into space. However, for the most part all that matters is that it roughly does what it advertises to do and ships as quickly as possible. LLMs are ideally suited to the latter form of software and the need for humans to ever see or think about the code will drop to zero soon, if it hasn't already.
Keep in mind also that LLMs are currenctly heavily subsidized. Once VCs money are gonna run out, you will see the real price, and your 200K per year dev is probably nothing.
I learned BASIC, Pascal and x86 Assembly language by the age of 14-15. None of which I remember (nor care to). The strict instructions (aka language) we gave the compiler was not the point. Those experiences taught me how to think logically, and over time develop a taste/structure for how software should be built, as well as foot guns to avoid. I don't know how you teach that without coding, but the "coding expertise" was always a means to an end. LLMs are just another abstraction layer above programming languages, machine language, etc.
Even Robert "Uncle Bob" Martin, who's as AI pilled as they come and doesn't read code any longer, says that junior developers shouldn't even look at AI tooling for the first three years:
In contrast with Pascal, I could see in my mind the PDP-11 instructions behind my C code. By the time I started running code on x86, there were debuggers that could step through source code, and I didn't really want to think like an i386.
> LLMs are just another abstraction layer above programming languages, machine language, etc.
Programming languages are (largely) deterministic. LLMs are not. The boundary between an abstract concept and something executable sits comfortably in the mind of the programmer in non-LLM contexts. This boundary shifts when using an LLM and becomes shared between the programmer and the matrix multiplication machine.
The "layer of abstraction" analogy is therefore technically true but essentially false. "Just another" papers over how seismic of a paradigm shift all of this is.
Sure, you're right - at the end of the day, LLMs are non-deterministic code generators. I certainly wouldn't trust one for a medical device manufacturer's software. For line of business apps it's totally adequate. And to be clear, I don't love it. I'm just coming to terms with the fact that these LLMs have gotten good enough, that with enough instruction and guidance, they can generate code/features at speeds we can only dream of if we were to code them by hand.
We're already seeing this at the enterprise level. Companies have dictates from leadership that "if you're writing code manually, you're doing it wrong."
Okay, that kind of works for a while. We are indeed producing a shit-ton of code, but the reality is that engineers are pumping out code faster than the humans can understand and (honestly) review it. That sounds great until you realize that "hey Claude, read this Jira ticket and implement the feature in this code base" isn't really worth $200K/year.
This is all complicated by the fact that we're also losing our grasp on reality from the other direction because we have leadership air dropping AI generated manifestos on the product owners and product owners having to use AI to transmute all that shit into 1,500 word Jira tickets that are 10% necessary feature work and 90% LLM boilerplate.
So now you have software engineers whose job has changed radically to the point that the hardest part about being a software engineer is just filtering through AI generated artifacts from all directions just to try to get a feature out the door.
It doesn’t work for the vast majority of tickets he attempts because he doesn’t have the necessary understanding to even start thinking about if the solution that the autocomplete generates is even remotely workable. And that’s with fancy dev loops and whatnot.
The spacer between the keyboard and the chair still matters in my experience.
I have a hard enough time explaining why “yet another bespoke application on some unmanaged azure resources” is a bad idea when they have more reasonable alternatives at hand.
Now these goofs can (very nearly) press a button and do it anyway, with no comprehension of the consequences. It’s high fives and pats on the back, until I’m cleaning up the mess.
Knowing what not to do has always been important, but it is perhaps more important than ever.
I have a design principles + tech preferences doc I force llms "lint" their approach against. It's not perfect but it helps. I call it a bias field, pushes them toward hopefully the happy and harmonious (with the rest of the system) paths. Obviously this is only partial and imperfect enforcement, but if it's applied to everything consistently it naturally encodes some self-consistency and harmony.
I’ve done the same but it’s a moving target as models advance and I find half of my points are ignored until I’m prompting “No wtf why are you still trying to symlink the global Python executable just use the virtual environment that’s already activated”.
Anyway, companies are pouring billions into improving AI tooling user experience so most of what I do manually I just anticipate to be a waste of time. There’s no way my hobby fiddling will outpace whatever gets released in the next couple months.
In the meantime, real linting does work pretty well, if you can write a detector for whatever antipattern you find LLMs fall into (like multiline comments).
I get patches into the biggest AI projects all the time to make inference faster on my 3090 and all you have to do is read the contribution docs and open a small PR.
It’s not a cabal of super geniuses. We are literally in the Wright Brothers era of AI.
You can trivially outpace what big companies are doing it’s insane, why do you think so many innovations are coming out of scrappy Chinese labs? They are willing to engage instead of being defeatist about it.
Or, slightly changing this. The source text needs to speak the correct vocabulary and language to produce a good completion. See the chat where Terry Tao is doing maths with an LLM. There’s _no way in hell_ I could get to his output because I just have no idea, and can’t speak the language.
Same with any field.
The edge of correct/better when facing ambiguity is very fuzzy, all models from the past 6 month or so have similar random ways of spinning between too-literal avenues and oddly misplaced misled fixations. Taking the right initiatives in face of uncertainty is definitely AGI, and its not there, and perceptrons + attention layers just ain't got what it takes no matter how hard you push.
Nor even €50k/year.
Two things are true:
1) The coding part of my career is over. LLMs are capable of doing everything I've ever been paid to *write*.
2) My actual job also included non-coding work: Does this attempted solution even work at all? Is this solving the right problem? Even if it's a valid solution to the right problem, is it the best solution given the time constraints?
That last one, "given the time constraints", is a place where my experience is still useful. The AI is as lazy (or as optimised for fast wins) as the humans whose examples it was trained upon; but an LLM costs so little that the answer is always "do it right" rather than "do it fast". A lot of people don't know what "do it right" even looks like, having only ever known e.g. websites that take 5 seconds to load because of all the adverts and analytics, and never the world where machines with 1% of the CPU and no GPU at all could fit a fully playable first person shooter in the same memory footprint as that page.
At least, I hope this "is a place where my experience is still useful"; I may just be telling myself a nice story, same as all the other people through history who have found themselves obsoleted when the automation came for them.
Remember we used to spend enormous amount of time in school and in our spare time studying computer science? Algorithms, operating systems, compilers, and etc. All kinds of insights. All kinds of fun. All kinds of hard engineering. Yet, how much time do we really need to spend in our day-to-day work implementing or using the algorithms and etc that we have learned?
Engineers have done amazing work of abstracting away the hard algorithms and data structures. In the meantime, there has been little progress or few new fields in the past 10 years or so in business that ask for implementation of new algorithms. In contrast, getting LLM to work is a new field, so it requires tons of new implementations: KV caches, speculative decoding, all kinds of variants of attention like FlashAttention, all kinds of parallel processing techniques, RL pipelines, post-training pipelines, and etc. It's just that the field is so concentrated that only luck few get to work on them.
So, maybe it's not LLM per se that removes the need of writing code. It is the maturity of the software engineering that has done so. It's just that LLM fills the last gap: making knowledge transfer so much faster and cheaper. All that's left for most of us is slicing and dicing of what has been already been implemented.
But an LLM just happily chugs along and does it, no one feels the friction, which means you never stop up to think if you're solving the right problem, or in the right way. I'm already seeing this bite us in the ass, as you get hacks upon hacks.
- Functionality which exceed the technical ability or knowledge of the developer which built it; and
- Functionality which would require an excessive number of changes that time/cost became a constraint (likely because it wasn't an incremental change but a significant rework, or just a bad fit with the existing product).
Sometimes I had the displeasure of joining a company where you could see developers hit these blockers and tried to fight through them (likely under pressure from management) rather than avoid them. And in the process they created a mess of buggy, half-functional spaghetti code which someone else later had to resolve.
Today however, people can use agents to smash through these blockers and ship an incomprehensible amount of crap. And what's worse is they're celebrated by management who don't know any better and see this all as a productivity win with no downside.
I suppose there are two ways to look at this – some would argue that engineers don't need to understand what they're building in detail anymore so non of this matters. Instead they can always use agents to explain what's going on and prompt them to fix any issues that come up.
Then there's another camp which might argue that agents will fundamentally have all the same limitations as humans, and that at some point a codebase will grow too complex that it exceeds even the limits of an agent's knowledge retention or intelligence. Or a codebase may just grow so large that it costs 1 million tokens to make even a simple change.
Unless agents start saying, "there's no way I'm shipping that" like humans used to I don't really see how we avoid the latter scenario... Complexity simply must have limits even if agents allow the bounds of those limits to grow.
At some point models will have to take control or we risk hitting these limits in irrecoverable ways since complexity far exceed that of what a human can reason about well.
If this wasn't true then "build me all the software" would be a reasonable prompt. Because why wouldn't we just get an agent to build everything we could ever possibly need? It's obvious that in the limit there will be limits in knowledge, intelligence and cost.
Whether it's humans or agents, someone needs to manage complexity. That is the most important thing a good SWE used to do. It's why technology selection mattered, it's why good architecture mattered, it's why clean code mattered.
One theory I have floating around in my head is that if a whole code base was microservices and micro front ends that were all less than 5,000 loc then you could fit the whole thing into a 100k token AI context window when working on it. And being few lines of code would force them to be somewhat simple
This is not non-coding work. This is coding.
I'm trying to write a novel, have been for a while now; one thing I hear from professional writers is that one should only do this if you are prepared to re-read whole thing six times before you even send it to the editor, and when you do send it to the editor, you and they will likely spend several months repeatedly re-reading the whole thing as you respond to a long list of changes they give you.
That is certainly not true. LLMs can't write code worth a damn still, and you have to babysit them to make sure they aren't doing stupid stuff. A human is still by far the best choice for doing programming, and when trend-chasing companies realize the damage they've done to their businesses they will stop pushing LLMs so hard.
I think it depends on the - ahem - context. In my experience they usually can, sometimes they can't.
But generally their sense of software architecture remains abysmal, so even if their writing is ok, you need to have architectural authority the very least to guide them.
It's a similar level of excitement as when we started doing devops stuff before devops even was a thing (which has been one of our main things for a very long time now).
I was just doing that yesterday designing a game demo lol
Often they can't figure out how to test the thing, so "does this attempted solution even work at all?" is its guess from reading the source code, and sometimes I've even caught them writing "tests" which are a regex on the source code, not functionality.
They know almost nothing about "is this solving the right problem?", they're trained to assume the user is right, not to frame-challenge.
My general experience both in game and non-game projects is that it will be lazy by default rather than solving things correctly. Sometimes I spot this from their responses, other times I only notice with manual testing.
The point is that AI is aware of constraints and can manage a round it. You've got to keep in mind the size of a typical software project and plan. Most people aren't writing a kernel. They're writing some backend tool and need a project that fits into a couple of quarters with a handful of people at most involved.
Only if you instruct it with constraints. Otherwise, it's happy to implement whatever workaround it needs. But it still takes a dev to know what those constraints are and why they're needed.
Human test passed. AIs make giant flashy mistakes. They don't fuck up basic grammar.
Not perfect, gptzero.me still knew it was AI generated (tayo42 is human by the same measure), but if for some reason someone was using an LLM prompted with that pattern, I suspect it would fool me in a Turing test unless I found the keyword to force the agent to change the role it was playing.
Conveying the errors typical in a Raise The Colours group on Facebook, that's the hard stuff.
The word "engineering" means doing things using a repeatable process to get predictable results. If you can get predictable results (e.g. guaranteeing the absence of data integrity issues, security issues, anything that will cause downtime, etc.) without looking at the code, you're still doing the work of engineering.
Your job as an engineer is to choose high-value problems to work on, solve them in the correct way, and guarantee that the functional and non-functional requirements are met. If you no longer need to read the code to guarantee that it has the right set of ilities, then I'm not sure that's obviously a bad thing, as long as the ilities you're enforcing result in a codebase that is sustainably secure and maintainable over the long term.
Boris: "I haven’t written a line of code by hand in, I think, eight months now… Claude Code, 100% written by Claude Code".
Boris: "There’s no manually written code anywhere at the company… All of the SQL is written by models. Everything is just built by the models... Claude instances communicate with each other (e.g., over Slack) in autonomous loops"
This does not sound like they review the code either. So, either the frontier labs like Anthropic have figured out something that very few companies could replicate, or they are being incredibly deceptive. I don't know which is true.
1. Yes, we know, and can tell.
2. No, they haven't figured out anything. Just vibe coding it with their bleeding dege models.
The kinds of business software I work on don't have those characteristics. If I needed something like a utils library, I think I could easily have Claude write the whole thing and not read the code.
Why would programming be any different?
That doesnt clearly apply here
When you factor in overhead and benefits, many companies were regularly paying that much for someone (many someones) to "read this Jira ticket and implement the feature".
We are currently in the "centaur" phase where a human-AI combination produces the best output, but I think some people are betting on the fact that the AI only product will eventually outperform the centaur. And with the cost of tokens falling thanks to fierce competition from the Chinese open weight models, it's definitely possible that those that bet on AI early will reap payoffs in lower expenditures for more output.
Honestly not sure which side I land on that bet but I definitely can't rule it out.
This is not a silly question, because it's kind of silly:
Has an AI ever actually gone through all of the steps necessary to become a chess master? Go to competitions, raise in the ranks, take a plane to the masters, organize support and all the real life details necessary? Playing chess is just the small formal part here, and all of the obvious things humans obviously do, well...
I disagree, currently a human guiding an AI agent is far better (more productive for less money) than an AI agent with very loose non-technical guidance and The Ralph Wiggum Loop.
I don't know how much longer humans have, but I don't think they can be cut out of the loop entirely just yet.
The nincompoops at the top think that if mountains of code can appear by snapping your fingers, they will be less of a problem.
If the mountain doesn't work, no problem; since it was so cheap to produce, just scrap it and snap you fingers again. The next one will be better.
Comparatively evaluating 2, 3, ... mountains of code is harder than trying to understand one.
1. Execution of engineering task - Companies happen to focus on this mostly and this is the metric to measure easily. Using LLM tools gives a impression of improvement on this area which is what everyone is chasing
2. Growth of the Engineer - This is the one which was always a side product of company culture, individuals interest, work being done, time being spent to understand, learn from failures. A job being executed perfectly for the first time itself does not gives the opportunity for learning, no memories/experiences are built up mind of the person executing the job after a while.
The second part is the one which is under appreciated in current scheme of things since it was a by product. There is a concept of muscle memory which pretty much applies to everything.
A bad or below average dev with AI will run your product and company into the ground in a matter of weeks.
Good: you can refactor your codebase at will, throw out legacy cruft by the megabyte, improve build/CI time, and simplify ruthlessly. Not to mention kick out new features in simple and consistent ways that align with what a user actually wants.
Or you could add megabytes of vibe coded crap, solutions that add a ton of mass but don't actually solve the problem at hand (seriously!), implement abstractions that are logically inconsistent with the rest of the system, etc.
Both of these are happening right now, and I think the latter is happening at a rate far higher than the former. But our fundamental dynamics are still at play - the ball of mud is still a ball of mud, even if AI lets you make it 100x bigger. The problem just gets more entrenched.
Eventually AI will learn how to simplify code, understand coupling, etc - and hopefully it will just iron out problems as it goes. I think we're a long way away from that. But this is uncharted territory, and I don't think anyone really knows. I certainly don't hear anyone focusing on that as a target of optimization however.
My hope is that we'll see a number of companies collapse as they scale - with basically no hope of rescue, and perhaps we can re-learn these lessons yet again. It kind of feels like GitHub might be the first example of this.
If you worked in SV like I did from 2010-2020 you know that the exact opposite of that will happen. I saw hundreds of companies successfully scale out of their garbage stack such as Facebook and PHP.
The only thing that matters is the problem you’re solving and the quality of the code is irrelevant.
I had a recent experience with this. Although it was months and not weeks and there were other factors into play such as the market etc but the last company i was in had a poor engineering culture with inexperienced engineers equipped with AI output some of the lowest quality features for both customer-facing products and internal dev tools. A lot of customers churned, and im sure quality us part of the reason.
My argument here is that what is worth $200k+ is the ability to distinguish the changes that must get thorough, critical review from those that need only a couple of specific things verified and those that require no manual review at all.
Our jobs have never been to write code. We’ve been saying for decades that LoC isn’t a rational way to measure engineering output, and now we have our chance to structurally change that system before processes re-solidify. In fact, I think the flexibility to adopt new systems and the experience and foresight to choose a path that is better than the status quo without throwing out everything we’ve learned is going to be what sets companies apart and makes individual careers over the next few years.
> we have leadership air dropping AI generated manifestos on the product owners and product owners having to use AI to transmute all that shit into 1,500 word Jira tickets that are 10% necessary feature work and 90% LLM boilerplate.
There’s an important criticism here IMO - the relationship between “business” and “engineering” is changing drastically. It’s going to be a challenge to set the expectation that just because Marketing was able to vibe-code a prototype in a day, actual implementation may well take weeks or months. Engineering should be considering things like security, scalability, and systems integration that aren’t a concern for Marketing - that’s why we’re being paid!
> So now you have software engineers whose job has changed radically to the point that the hardest part about being a software engineer is just filtering through AI generated artifacts from all directions just to try to get a feature out the door.
I think it’s an extremely heterogenous landscape right now. Where I work I’m struggling mostly with organization - keeping the (literal) dozens of inbound features that come in every day straight long enough to hook up an agent harness, review, validate, and deploy. At other companies the issues seem to revolve around their Agile-based processes. Or maybe it’s non-technical vibe-coders and their expectations. Or maybe it’s executive leadership flirting with AI psychosis.
Everything is in flux. It’s stressful and exciting, and I’m thankful to be around for it, even if I am in my 40s at this point and looking at the core skills I’ve built rapidly become worth exponentially less. It’s a huge opportunity for growth.
That led to a reduction in knowledge of assembly in the average programmer but the people who specialize in it haven't gotten any worse at it.
The result was a generation of programmers who make useful software while very few of them understand the machine they program. You could easily make either a positive or negative value judgement about that result.
And no, telling Claude to implement a Jira ticket is not worth $200K/year. Checking if it has not done something stupid and correcting it when it's trying to - is.
Completely unsustainable.
The subtitle of the story tells it all.
There are some people who seek out friction. Think about an athlete or a hardcore nerd.
The best engineers are ones who were fascinated with computers and learning as kids and persued it at every opportunity. Found their own friction in other words.
For those kinds of people, friction-seeking is the constant and what LLMs did is moved the point of where the friction occurs.
For example - the best engineers I've worked with didn't necessarily have lots of experience coding in assembly because that kind of friction was no longer necessary. But they could solve hard problems (and if a problem really required assembly they could go learn it)
What I think will be hit much harder by AI is the low tier engineer. Someone who was never truly curious and committed to it, for whom it was just a job. For example a typical offshore ticket pusher kind of person. That kind of person never went out to find friction and that's the kind of thing that's never going to fly again - if I want mediocre or average, the LLMs are sufficient
I've been trying to map the LLM advancements and the current state of software development onto prior technological improvements. History is littered with similar cases where the abstraction layer ends up getting lifted, and people struggle with getting accustomed to working at that higher abstraction level.
For the people that fall in love with a single abstraction layer or don't have an interest in learning new paradigms, when their known pattern is abstracted away, they're condemned to being left behind, either unwilling or unable to adapt.
I don't think any industry is free from this, any person in any industry/profession over a period of 20 years or more has likely had to undergo massive adjustments as technology changed their field.
We're not unique, but that doesn't stop it from feeling so jarring when it happens to us
But there's no abstraction layer that ends up getting lifted. When I use a library like SDL or a standard like POSIX, I don't tend to look at the underlying implementation. Instead I work with the high level concepts that they come up with. There's no such things with AI tooling. The most similar is when fully vibing software and everyone knows the quality of the result.
I've learned something at every abstraction layer in computing from electronics (hardware), theory of computation (software) to high level programming languages with their paradigms. Same with several domains embodied by libraries. LLM tooling is more like shamanic ritual than engineering.
Yeah, a bespoke program that does exactly what I need it to do, at a speed that I had forgotten was possible on computers, with customization that is an exact fit to me, at a cost that is smaller than a rounding error.
I get that LLMs struggle with the old paradigm of a single piece of software meant to serve every conceivable use case of every conceivable user, but I kinda hope that paradigm dies.
Of course, bills need to be paid and it's not all that simple. But at least there might be such a silver lining.
I've been worried about this a lot, I even created an agent skill called do-i-understand that's designed for novice devs (and experienced too, because atrophy) where the LLM asks you questions about the PR you're about to submit. I've found it helps a lot: https://github.com/AnthonyPAlicea/skills/blob/main/skills/do...
One way or another, there will be a skill reckoning.
First and foremost, it's an issue of "dependency": if you stop training the "muscle" of logic and reasoning, it gradually atrophies, just like unused physical muscles. You become dependent on external tools that replace a capability you once had yourself.
A historical example that brought about a similar shift is this: when the production process moved from the craftsman's mind and hands to the Fordist factory (and the assembly line), the skill of building things shifted from human craftsmanship to anonymous, structured processes.
Bit by bit, traditional artisans lost their knowledge and "know-how." Today, having a piece of furniture in our home depends on a massive production and supply chain; the "average" person no longer has the ability to build it themselves.
The exact same thing is happening to software.
We are the (now "former") software craftsmen.
For a bit of extra unease, consider that these people will retain their right to vote, both in making business decisions as employees, in shareholder meetings as shareholders, and in government elections as voters, despite their atrophied reasoning abilities. Who will wind up whispering what to vote for in the ears of these reverse centaurs?
This is why in order to understand new codebase or to ramp up to new projects, I use AI to generate a textbook style reading material for me along with "verify yourself" types of exercises along the way. Then I print them on a paper and read it using a pencil/pen and take notes.
Because of my math training, I am in the habit of slowing down to read textbook style texts which helps.
Tangential, but this is one of many reasons that I (and I suspect many others here) have taken up wood working.
> Bainbridge argues that new, severe problems are caused by automating most of the work, while the human operator is responsible for tasks that can not be automated. Thus, operators will not practice skills as part of their ongoing work. Their work now also includes exhausting monitoring tasks. Thus, rather than needing less training, operators need to be trained more to be ready for the rare but crucial interventions
This is speaking from my experience as a systems/c/c++ guy. If you are a js web frontend guy, python, or whatever I have no idea if this applies to you.
It's impossible to buy the FUD that expertise will somehow vanish because people use AI. People come in a variety of capabilities, challenges, and personal interests. Expertise varies from person to person. Those with personality traits that are more likely to take risks and persist against adversity will be the innovators and experts. We've seen this played out countless times.
You have to ignore everything you know intrinsically but can't articulate about innovation, technology, and evolution to drink this FUD Kool-Aid.
I agree that AI reliance is hurting engineer's language understanding, but I don't know how much we should care.
This is the key point. Should we stay vigilant and review all the code, ensuring we have a decept grasp of what it does. Or just green light everything and treat the code itself as a black box. I would hope we do the former, but many are pushing for the latter.
Example from my workplace: pull requests that were already too long before LLMs by a few hundred or a couple of thousand lines are now several thousand lines longer and they get opened much more frequently. The primary source of these PRs is a well-respected senior team member with a ton more experience than everyone else by a measure of decades so nobody dares tell him to stop. Other team members aren’t far behind in the PR opening rate and admittedly I am one of them, but I try to mitigate the issue by doing multiple LLM reviews and anticipating when it’s ok to just approve it myself (have never had an issue in making this judgement thankfully). Part of this IMO is almost certainly because there’s now peer-pressure within the team to churn out a ton more code and that’s adding to the already clear message from leadership that we need to leverage AI to deliver things at unprecedented levels.
My point is that it’s not just that some people are pushing for greenlighting everything without due diligence, it’s also that they’re pushing others to toward that outcome with their extreme AI enthusiasm which they don’t see the ramifications of or just don’t care.
Would recommend your senior team look into stacked PRs, which was made to handle this scenario exactly.
Those skills are often taught in an idealized situation like very small projects. In the real world, there are a lot more practices and techniques to develop software in a pragmatic way. Those "engineering practices" are much more useful than the above, unless you're in some specific domains.
The article links to this as the study: https://dl.acm.org/doi/epdf/10.1145/3632620.3671116. It doesn't appear to be related to JetBrains in any way?
https://www.youtube.com/watch?v=HTUh0OO6Kmo
The video seemed to indicate it was a study they ran (she refers to the examples as "our subjects were two of twenty four participants"). It seems they are just citing the study. While it doesn't change the study or the takeaways, I'll update the post for clarification!
now that's a phrase I have only heard in one other context: as an alleged quote from former Boston Celtics player Isaiah Thomas (who was really about 5'7 despite his official listed 5'9; yes, professional basketball player in NBA). This was around 2017 when he had just come off a career year and was soon due for a contract extension in the low 9 figures. He got injured and bounced around for a few years before eventually leaving the league, and never got that payday.
Now with agents I don't have any insights into what its doing with kubectl and would have no reason to learn how it works (other than my own curiosity).
I agree that the human no longer gains expertise, but I am not sure that will matter in the long run.
Local AI would thwart this vision (which is why they're desperately also trying to ruin the hardware market at the same time), but the effort it would take to get local models to run as well as these frontier models is an odd thing to pursue just to avoid learning the fundamentals and gaining direct experience.
While I agree Local AI would be a good goal, even the huge open weight models that can't be run locally are already thwarting the US AI lab's duopoly. Other companies can and do serve up these open weight models, and without having to amortize R&D they are able to serve the tokens at a pretty cheap rate.
So, partly agree, partly disagree.
(Not to say that the current state of the art with LLMs merits borrowing money for a multi-trillion spend on hardware that will be obsolete in 5 years, but for me that is a different issue).
The master decrying the invention that resulted in more wisdom for more people than anything prior:
"For this invention will produce forgetfulness in the minds of those who learn to use it, because they will not practice their memory. Their trust in writing, produced by external characters which are no part of themselves, will discourage the use of their own memory within them. You have invented an elixir not of memory, but of reminding; and you offer your pupils the appearance of wisdom, not true wisdom, for they will read many things without instruction and will therefore seem [275b] to know many things, when they are for the most part ignorant and hard to get along with, since they are not wise, but only appear wise." (quoting from https://www.historyofinformation.com/detail.php?id=3439)
Plato (who actually wrote these quotes, because Socrates only spoke) wasn't against just "writing" in a general sense, or that he felt it was going to hold humanity back. He was against treatises and felt that someone could "memorize facts" without having to actually think about them on a deep level. Which I think, ironically, is not all that off the mark especially in the age of the internet and now LLMs, right? He also felt that memory could decay if we relied on written facts instead of having dialogues, which again, not all that off the mark, either. He underestimated the compounding capability of technology and our ability to record data and information, but people remember a lot less these days than they used to because we'd largely given up that ability in exchange for the instant information machines.
Of course, he's just one man (or two, if you consider he was conveying Socrates' thoughts, as well) who lived in ancient times and couldn't possibly foresee how technology would evolve...but when I went back and read his concerns, it was ironic to admit that much of what he warned about still applies to this day. People are far less informed despite having access to more information than we've ever had because we've exchanged the ability to remember for the ability to just look things up. And, we also have less meaningful dialog than ever before; people just sit on social media, copying and pasting "facts" to each other, instead of having actual productive discourse. When I look around, the world doesn't seem brimming with mindful critical thinkers and there's reasons for that (many reasons, of course).
AI tooling can be a boon to learning, but it requires us to stop using them for code generation as a primary purpose (at least for juniors) and instead advocate for Socratic workflows that still require manual coding practices. When someone decides to code something, it's not just learning syntax; it engages a variety of mental disciplines from critical thinking to planning to creative problem solving to logic and math, even.
To be clear, I feel the same way as you. I have been doing this for about twenty-five years, and I have developed a keen sense of taste and judgment to know how to properly scrutinize and utilize these systems. I definitely feel like I am able to learn things faster, but that is directly correlated to having experienced a lot of friction over the years that cultivated said taste and judgment in the first place.
I imagine it's sort of similar to someone who's just getting started in mathematics, but is introduced to WolframAlpha, versus someone with decades of experience in the field and what they learn with it.
Horsemanship and sailing were both specialized skills of high value to society, and now they're not. But in each case there was probably a liminal period, when being an accomplished horseman or sailor was still valuable, even as motors were taking over. Eventually that period ended, as the new generations without those skills found ways to get by with cars and motorboats.
I could wax philosophical about what that leads to but enough people already have.
It might be like how an archeologist since the invention of plastic can date the period of the soil as post plastic, if one could cut open the software stack of the coming systems that will be built in the near future one would could data that code as post agentic LLM as programs are going to be mostly bloated ad-hoc, poorly thought-out and patched in a way that does not concern it's self with the correctness of the algorithms or data structures chosen.
I am not Anti-AI but I think it's going to be interesting and I am surprised at how bad a program ends up when one tries to "vibe code", though often ends up working, though in isolation and when used with discipline (that the tools themselves psychologically make it harder to do) can produce some very good code and being able to use loops to solve difficult problems or problems that simply would have require banging ones head against the problem many times is very profitable.
By way of example I use Kdenlive 90% less and just use Codex to issue the ffmpeg CLI commands directly when I want to crop/format video on my PC.
Genuine curiosity in learning how stuff works, not necessarily for financial gain, e.g., from a salary or using web to deliver ads
No real "pressure" for hobbyist to use "AI"
IMO as end user, some of the best software available was originally written by a single unpaid author, not salaried teams
Some of the worst software ever written was produced by overpaid teams
Hobbyists may use relatively old, handwritten code to learn from
That's not necessarily bad. Software quality has declined over time
No, we do not have that word for that.
Yes, we have that word. No, it does not mean that.
Any LLM proofreading could've told the author that, for that matter.
__
What Fingerspitzengefühl actually means could be described as "tact", a precise approach to something and general attention to detail that leads to success. Or.. the lack of all of that leading to blunt failure.
__
I would of course comment with more Fingerspitzengefühl for the emotional needs of the author if the writing wasn't just a sales funnel for their courses and whatever else (+ posted by them themselves.)
They've become a meat proxy[0] prompting LLM to root cause for them and the ability to fix is capped by LLMs ability, not the engineer's.
I've tried to convince management before this happened and yet here we are.
[0] https://gruhn.me/blog/2026-08-03/
I also see no evidence that AI programming is a difficult skill that cannot be learned by any intelligent person in much shorter amounts of time than previous professional skills required.
I do not think anyone could do the job I do with AI without my decades of experience, is my point. For now.
People are distrusting whole swaths of concepts because they do not understand how things actually work.
Like going away from zerotrust architecture to holding open a long polling connection.
At least I seem to remember that long polling wasn’t that great, but i start to even doubt myself.
Anyone wondering how the Roman Empire was wiped out? I guess we will find out soon
This has an issue in practice.
As an example, Function Programming (FP) has a high constraint on programming, where some don't even have loops (gotta use recursions all the way), and almost no state mutation (Exlixir I believe have a mutable state? forgot).
FP sounds great with all those constraints. But what about the adoption?
It's hard to switch your mindset, and not as natural, thus long leraning curve, and not as wide adopted <- productivity goes down as all others need to know and use it well. That's why still imperative and OOP languages are ruling the world.
Same for these constraints. The author promoting using AIs only to subsets (no coding? wtf) is something the majority won't follow. Yes, the constraints sound great, but at what cost? By the time one learns everything, and everyone moves along with AI building stuff fast and cheap, they will be left behind.
---
I do not like the author trying to make his points authoritative by using Halo effect using quotes from the industry and could have had been taken out of context (That's where AIs mess up the most, they only get chunks of text and lost context of before and after exerpts).
LLMs don't particularly care how hard the type system or borrow checker is (as long as it has good error messages that can guide it to a solution), they can grind away at these things, and what you describe as a negative actually becomes a strong positive. I think we'll see strongly typed systems plus some level of formal verification become popular as an AI-preferred language.
When the burden of review is lowered because you have purely functional building blocks, with lots of formally verified chunks - people will choose that simply out of (productive, beneficial) laziness. Not to mention we're in a ram crunch, and being able to run your stuff far cheaper has an immediate financial incentive.
This is asserted without evidence, and from a scientific standpoint, unjustified.
First, "detect patterns" makes it sound like a classification task, "is this a picture of a cat". But LLMs transform the input.
For example, an LLM can translate text between two languages while properly handling the names of the people described, no matter what those names are. That shows they are representing the text in a somewhat abstract way, that they can perform operations on that representation, and also convert it to useful output.
And, what I just described is the most general form of information processing algorithm. Science is not aware of any limitations in principle on such systems.
I am not saying LLMs have no limits, but "they only recognize patterns, and that is a true limit" is not a good argument.
E.g. I've gotten pretty good at git thanks to AI because it forced me into more complex workflows. I still can't remember the commands but I really grok the concepts and terminology on a far deeper level than before, where I made a conscious effort to stay on the happy path lest I get into some odd state I don't understand. Pretty sure those "odd states" would seem quite clear to me now.
LLM chatbots are good at making you feel good. Facts don't care about your feelings.
The real issue will be with people that never learn it in the first place.
And even that isn't so bad, learning programming isn't some insurmountable hurdle. It took me less than 6 months from no knowledge to my first job.
I agreed with you up until this sentence. Entry-level devs with 6 months of require an enormous amount of time and mentoring until they're able to really perform well. I think the greater concern being expressed is not will people be able to type syntactically correct code?, it's will people be able to write software well?.
But I empathize with the author. I think the frustration comes from the fact that other folks in your organization use these tools to “get by” more effectively then before. In the past, a disinterested engineer might ship less under the umbrage of a difficult engineering problem. With the pressure to “ship more” now that there’s AI, the same engineer is using AI to spam PRs to prove they’re working, even the quality of work is low (because quality of thought is low).
You treat the AI like a junior.
You make it write pseudo code in the tickets.
Then you make it write code in the tickets for the tricky stuff.
Have it reference relevant documentation/APIs.
Then there is no lost knowledge.
(Still, sometimes it sneaks in some "helpful" belt and suspenders, but for the most part, I know everything that is going on in the code base with this method.)
Eliciting the right requirements and user needs. Creating a sound architecture and design. Validating final product meets the requirements. Ensuring there is good test coverage. You know `Engineering`. Mechanical and Electrical engineers didn't go away because of CAD.
Besides your first point, the rest can be delegated to AI as well. And you don't need to be a "software engineer" to spec out requirements and user needs.
CAD is also useless to Mechanical and Electrical engineers who didn't learn fundamental the skills that it streamlines.
that is a very good point! I'm going to use it in the future.
I have such a hard time quickly reading pages like TFA. Headings, bullets, line-spacing, width, link hover animations, even the font. Even reader mode defaults aren't great for me.
Compared to, for example, https://www.paulgraham.com/best.html
In my experience human design -> agent code -> agent review -> release -> agent observer, does not (at all!) work. One absolutely has to be understand what is going on, how the system works, how the H/W works, etc, to catch agent errors.
The proposition of us losing those skills is terrifying.
Maybe AI will get better and we truly won't need a human in the loop. Then please also have all PRs signed by whatever model wrote them, and do not come to me for trouble-shooting.
But what we're gonna manage the codebase in, then? What kind of language?
Will it be a natural language, or a formal one?
I feel that "coding" cannot collapse. Coding is just translation from natural to formal language. Somebody needs to write the specs. And writing/maintaining them in natural language brings a lot of fun - shifting interpretation, inconsistency, missing specification, etc.
I don't think it's progress for the field of SW engineering. But at least more "shareholder value" will be created.
When that happens, (developer thinks using an ORM means they don't need to know how to use a database,) the project eventually fails. It's the same thing with AI; just like companies learned that they need to make sure developers actually know how to use a database, companies will learn that their engineers actually need to know how to program.
I know how to create a multi year P&L projection from scratch. It gives me some insight into how much capital a startup will need to get to breakeven. A good AI tool that creates that same spreadsheet in minutes should also be able to explain why different businesses have different shaped cash flow troughs. I don't think people using tools like that is bad even if they don't gain deep insight.
For example, I have a vibe coded Python monstrosity which I use for stitching together scans of large format negatives. The enlarging lens that I use for scanning has strong chromatic aberration in the red channel but is otherwise very sharp†. I got Codex to figure out some way of using the green and blue channels to sharpen the luminance while preserving the color information from the red channel. I sort of vaguely understand how this works, but not nearly well enough to review all the horrid numpy code that implements it.
For my use case, there is really no reason to look at the code, as I am only interested in the output. As disconcerting as it is, I feel that this is going to become more and more normalized. Looking at the code that your LLM generates will become like looking at the assembly output of your C compiler.
† This makes a lot of sense, if you think about the color of a darkroom safelight...
We've been hearing about the demise of skills since before the invention of the loom. The only skills we've truly lost are for things nobody uses or makes anymore. What actually happens is, we develop a new tool that's better than the old tool, and we get good at using the new tool. We then have a class of people whose job it is to maintain the new tools. We don't need that many of them because the whole point is to eliminate that labor.
Your jobs as software engineers are going away, it's that simple. Your new job is to use the AI tools to make products. Somebody else's job will be to make the AI a better software engineer.
- Writing (starting from Platos Phaedrus)
- Printed books
- Photography
- Typewriters, keyboards
- Telephones
- Recorded music
- Television
- Wikipedia
- GPS navigation
- Smartphones
- Automation in aviation, medicine, industries
- Photography hasn't erased other visual arts. Both require a separate skillset; both have their uses and their enthusiasts. Photography benefits from having a lower skill floor, in my experience doing it, and this in turn has led to commercial photography eclipsing commercial painting (barring things like painting buildings) as a viable business model, largely.
- Television is arguably not a net positive in society (I won't say net negative yet, but there are some reasonable arguments for that position). It enables a form of passive consumption that allows bad arguments to come through as persuasive through the gradual (on the scale of months and years) numbing of critical appraisal of the media being consumed. - Smartphones arguably reproduce and amplify this problem, and tack on others: they make it increasingly easy to dissociate from daily life, and make it easier to continue consuming content from actors increasingly interested in steering conversations in one direction or another (on the most benign end, it's influencers trying to get you to buy things, which obviously has negative economic impacts on an individual scale; on the more overtly harmful end, they serve as a hardware channel for less-regulated or unregulated extreme and hateful content). - In aviation, the first generation of (relatively modern) automation did lead to a high number of accidents and loss of life. American Airlines flight 965, in 1995, is a well-studied case of an over-reliance on flight automation before it was widely understood among pilots what the limits of these early-modern incarnations of autoflight were. I think that last example (and many aviation accidents) are an interesting example to study in the context of software. Aviation's extremely and conspicuously responsive to accidents, often requiring procedural and hardware changes as a result of these. They can be as simple as a software patch, or as complex as the redesign of a component because a human factors analysis found that it could be confusing under stress.In software, we like to think that we're a serious industry, sometimes. We spent much of the 2010s going in what I found to be a good direction: the development of IaC, the popularization and standardization of tools like Git, registries for vulnerabilities like the NVD, and even the development of languages like Rust which address historical technical shortfalls of high-performance, low-level languages like C; but we always fell short, industry-wide, on matching these toolchain upgrades with technically-rigorous procedures.
For every borrow checker, there's vibe coding.
It kinda seems like wishful thinking on our part as software engineers that one day we'll be able to go "Hah, miss me now?" but that just isn't the writing that I see on the wall.
Over the past year, I've AI-generated two large pieces of software over 2000 commits without having much depth in either domain, and my ability to deliver value as a veteran software engineer has only shrunk over the months or it can be trivially extracted into a reusable markdown file like "design bar: ensure things 'by construction' where possible", a lesson I learned viscerally over decades.
I don't mean to pick on you, but to present this in the service of this kind of fatalism... you know the inevitable follow-up question, right?
Because I can still reason and dig into things and ask questions and have the domain explained to me out of curiosity, the same way I could build anything that was originally out of my depth.
But what I've noticed is that the need for any corrective power has gone or is going to nil and I'm mainly doing directional work. And even then, I can constantly have sota models "rank the top options and recommend one", and it's almost always the right way forward.
I used to read every line of every plan that LLMs generated, but over time I've realized I have less and less to correct or, more impressively, the LLM had foresights I never considered. The same thing happens with implementation.
Periodically I can spawn a bunch of agents to evaluate the system adversarially to find improvements, and the findings have been so good that it's evident that soon I can just automate that too.
In other words, the skills I need to excel here are more curiosity and patience than tech expertise -- the things that got me into software engineering in the first place since that was the only way to build things.
I know, all irrelevant little philosophical roadbumps. The artifact satisfies the buyer. I think I'm going to have to step away from this industry pretty soon, at least for a while.
Of course, he's just one man (or two, if you consider he was conveying Socrates' thoughts, as well) who lived in ancient times and couldn't possibly foresee how technology would evolve...but when I went back and read his concerns, it was ironic to admit that much of what he warned about still applies to this day. People are far less informed despite having access to more information than we've ever had because we've exchanged the ability to remember for the ability to just look things up. And, we also have less meaningful dialog than ever before; people just sit on social media, copying and pasting "facts" to each other, instead of having actual productive discourse.
AI tooling can be a boon to learning, but it requires us to stop using them for code generation as a primary purpose (at least for juniors) and instead advocate for Socratic workflows that still require manual coding practices. When someone decides to code something, it's not just learning syntax; it engages a variety of mental disciplines from critical thinking to planning to creative problem solving to logic and math, even.
Sounds like a good compromise, if not even positive from many points of view.
What is valuable now is productivity. Which in today's day and age is the ability to quickly and efficiently drive AI.
Hence the never-ending growth and spewing out of tooling: harnesses, TUIs, multiplexers, agent frameworks, software "factories", etc, etc.
Software engineers are chasing opportunity and expertise in the only thing that's left.
Very interesting is how divisive is this statement. In that almost every response is either hard agree or hard disagree.
Does agreement come from observer judgement, or lack thereof?
First they fuck up your codebase with so much slop that no human does (or could) understand it.
Now if you want to keep making bug fixes, you're on the hook for whatever Anthropic wants to charge you.
The solution is to make AI developers the scapegoat. Fire the people at the top of the AI leaderboards, with malice. Otherwise, lazy AI-addict developers are going to destroy what remains of your company's IP.
Why? Because people are going to get so much more ambitious about the stuff they build with AI, that code expertise for the stuff they are building is already scarce anyway. People will choose difficult and less popular languages, and will replace open source packages with stuff built entirely in-house by AI.
Going forward, there is no choice except for AI to fully replace the need for code expertise. Depending on humans won’t be scalable.
I think that would feel like crack for some people who are addicted to using or building stuff with LLMs.
https://ivyleaguecenter.org/2024/03/12/over-reliance-on-calc...
Math fosters critical thinking, and we've been in a critical thinking deficit for decades and decades. I don't think we can afford to sink even deeper...
I kinda agree, but I also kinda disagree.
My question would be if "math" is the right tool for that. It can achieve that as a side-effect, but how we express it and talk about it is full of legacy cruft, cargo-culting and general bullshit that makes it not exactly snap into place for all kinds of brain wirings.
Math I believe is not an end in itself. Maybe with that perspective, solutions that satisfy our needs there can be found
Compared to when?
In contrast since then we've had almost not noteable physicists or Computer Scientists that I'm aware, we've got a couple, Hawking and Knuth. But apart from that I don't know if there is a single scientist I could point out today that has revolutionized our understanding or even done any real breakthrough work in the field of Physics since Einsteins era.
But my reply was I guess implicitly assuming the post I was replying to was talking about the median level of critical thinking. Your reply is talking about the peaks of creative accomplishment which is not what I was addressing.
This is true and nobody complains that GCC or LLVM is making programmers dumber. They are code-generation tools from higher-level source code. Much like LLMs are code-generation tools for higher-level language/prompting.
Online discourse though is collapsing under dumb AI talking points.
Now, they all did have gems of interaction that would could find by sifting thru the dumb stuff, but I find that property of discourse to be pretty steady. I've even found some interesting comments on YouTube videos and newpaper websites.
This is more like paying someone else to read your math homework, generate answers, and maybe even submit them on your behalf. If that had been the (average) student experience, math skills really would have collapsed.
Assuming we can re-establish the will for democracy and taking care of the whole group.
In other words, a positive outcome with thing X is possible but there are also big behavioral and cybernetic pitfalls, ones which are suspiciously persistent.
People can no longer drive a stick-shift car, whereas it used to be that a valet would always be able to drive one to park it.
However, I can use mathematica to demonstrate the criteria for various forms topological stability of solutions to many types of differential equations. I'd rather live now.
My most-common frustration with it is when products I want to compare don't have the same quantity-denominator. (Either a different unit, or sometimes a different dimension like weight versus volume.)
https://ivyleaguecenter.org/2024/03/12/over-reliance-on-calc...
Seems like critical reading skills have already collapsed.
With a calculator you still need to understand the fundamentals of mathematics. You also need to understand how to operate a calculator and know its functions.
Even when removing numbers and manual calculation, being a function or formula oriented mathematician still requires you to know the theory and functionality.
With an LLM, you don't even need to know anything other than asking the robot to perform. You don't understand the output, nor can an unskilled person explain it.
what point is there to be a human calculator if nobody rewards you for it?
and I'm not talking about wealthy Bay Area tech workers doing it so that they can write their substack blogs but rather the average person
If we're going to be the last generation that understands code at this level, we need leave it in a much better state before we die.
"Oh, that old simple device didn't displace anything, so this one which is now quickly taking over every job will be just like that simple one". So I guess people today know how to sew and knit and bake bread?
"Oh, humans were always needed, so it'll be the same this time." Is that true of horses and oxen after cars and tractors were invented?
Also, yes I am dependent on a calculator.
1. function trampolines, such as in libraries like ATL (Microsoft's COM template library);
2. hand-optimized encoders/decoders for things like ffmpeg;
3. niche applications such as Ben Eater's 6502 breadboard computer (using a modified WozMon and Microsoft BASIC);
4. reading ASM output from C/C++ code in things like the GodBolt site for performance comparisons/analysis;
5. reverse engineering old DOS games;
6. people working on things like compilers (generating optimized assembly), JIT engines, or other code generation -- though this will typically be machine code, possibly with assembly instruction comments;
and other performance critical places where assembly can speed up that part of the code.
For the time being, a good understanding of programming and domain knowledge gets you much farther than without it. There might not be room for as many (professional) programmers as before, though.
Also the reality is that modern CPU microarchitectures are so different from what old-school assembly programmers used that their supposed optimizations are likely to be counterproductive.
"I use AI to generate a summary which I print out and read slowly every time I use a new codebase" is the new propaganda line pushed by Anthropic and OpenAI. They cannot tolerate this rebuttal.
Do we think there is going to be a correction for the vibe coders?
Pardon I know such question is controversial but thinking ahead.
Nurses need to unionize I think and strike to get something .
I'll readily admit that they deserve it, I'm a lazy dickhead and they help people, but I think your blame is misplaced if you're insinuating that software engineers are to blame for nurses being paid less than ourselves.
Again, to be clear, this is highly dependent on the market, job, and hours and one needs to understand the difference between a CNA, RN, NP, etc. However, it's a regulated typically union job that's very secure and in high demand. It probably pays more than what you expect.
1. We still need to keep the lights on. If things go wrong, which they still do, humans have to debug it.
2. Jevon's paradox: now the expectation is that we ship stuff faster and there is still a LOT of stuff to do, not less.
3. We are still in the very early days of this and it is too soon for corporations to make a dramatic call like laying off 80% of engineering. We just don't know how that would look like.
It is important to ensure that any LLM code is named and shamed.
You need to take it further.
We need to shame and mock the people (read: delusional imbeciles) who think this is acceptable if we want it to stop.
AI users need to fear for their careers and reputations if we want to get software development back on the right track.
That's the neat part: you don't!
Even Robert "Uncle Bob" Martin, who's as AI pilled as they come and doesn't read code any longer, says that junior developers shouldn't even look at AI tooling for the first three years:
https://www.youtube.com/watch?v=RxxxGkFIUJ0&t=1356s
Programming languages are (largely) deterministic. LLMs are not. The boundary between an abstract concept and something executable sits comfortably in the mind of the programmer in non-LLM contexts. This boundary shifts when using an LLM and becomes shared between the programmer and the matrix multiplication machine.
The "layer of abstraction" analogy is therefore technically true but essentially false. "Just another" papers over how seismic of a paradigm shift all of this is.