The best term for this is Cargo Cult Development. Cargo Cults arose in the Pacific during World War II, where native islanders would see miraculous planes bringing food, alcohol and goods to the islands and then vanishing into the blue. The islanders copied what they saw the soldiers doing, praying that their bamboo planes and coconut gadgets would impress the gods and restart the flow of cargo to the area.
The issue of course is the islanders did not understand the science behind planes, Wallis talkies, guns, etc.
Likewise, cargo cult devs see what is possible, but do not understand first principles, so they mimic what they see their high priests of technology doing, hoping they can copy their success.
Hence the practice of copying, pasting, trying, fiddling, googling, tugging, pulling and tweaking hoping that this time it will be just right enough to kind of work. Badly, and only with certain data on a Tuesday evening.
I see this often on our codebase. It was mostly written by ex-C# developers who were new to writing Go, and there’s many ham-handed C#-isms in there. At some point, someone took a guess at how something should be, then subsequent changes were done by copy-paste. Years down the road, another copy-paste job happens, and when I point out that the patterns within are not good (like, can actually be buggy), I get a confused response, because that is what was there.
There is an implicit assumption that the code written espouses best-practices, but that is far from the truth.
I have an alternate theory: about 10% of developers can actually start something from scratch because they truly understand how things work (not that they always do it, but they could if needed). Another 40% can get the daily job done by copying and pasting code from local sources, Stack Overflow, GitHub, or an LLM—while kinda knowing what’s going on. That leaves 50% who don’t really know much beyond a few LeetCode puzzles and have no real grasp of what they’re copying and pasting.
Given that distribution, I’d guess that well over 50% of Makefiles are just random chunks of copied and pasted code that kinda work. If they’re lifted from something that already works, job done—next ticket.
I’m not blaming the tools themselves. Makefiles are well-known and not too verbose for smaller projects. They can be a bad choice for a 10,000-file monster—though I’ve seen some cleanly written Makefiles even for huge projects. Personally, it wouldn’t be my first choice. That said, I like Makefiles and have been using them on and off for at least 30 years.
> That leaves 50% who don’t really know much beyond a few LeetCode puzzles and have no real grasp of what they’re copying and pasting.
Small nuance: I think people often don’t know because they don’t have the time to figure it out. There are only so many battles you can fight during a day. For example if I’m a C++ programmer working on a ticket, how many layers of the stack should I know? For example, should I know how the CPU registers are called? And what should an AI researcher working always in Jupyter know? I completely encourage anyone to learn as much about the tools and stack as possible, but there is only so much time.
If you spend 80% of your time (and mental energy) applying the knowledge you already have and 20% learning new things, you will very quickly be able to win more battles per day than someone who spends 1% of their time learning new things.
Specifically for the examples at hand:
- at 20%, you will be able to write a Makefile from scratch within the first day of picking up the manual, rather than two or three weeks if you only invest 1%.
- if you don't know what the CPU registers are, the debugger won't be able to tell you why your C++ program dumped core, which will typically enable you to resolve the ticket in a few minutes (because most segfaults are stupid problems that are easy to fix when you see what the problem is, though the memorable ones are much hairier.) Without knowing how to use the disassembly in the debugger, you're often stuck debugging by printf or even binary search, incrementally tweaking the program until it stops crashing, incurring a dog-slow C++ build after every tweak. As often as not, a fix thus empirically derived will merely conceal the symptom of the bug, so you end up fixing it two or three times, taking several hours each time.
Sometimes the source-level debugger works well enough that you can just print out C++-level variable values, but often it doesn't, especially in release builds. And for performance regression tickets, reading disassembly is even more valuable.
(In C#, managed C++, or Python, the story is of course different. Until the Python interpreter is segfaulting.)
How long does it take to learn enough assembly to use the debugger effectively on C and C++ programs? Tens of hours, I think, not hundreds. At 20% you get there after a few dozen day-long debugging sessions, maybe a month or two. At 1% you may take years.
What's disturbing is how many programmers never get there. What's wrong with them? I don't understand it.
This is the 40% that OP mentioned. But there's a proportion on people/engineers that are just clueless and are incapable of understanding code. I don't know the proportion so can't comment on the 50% number, but hey definitely exist.
If you never worked with them, you should count yourself lucky.
We can’t really call the field engineering if this is the standard. A fundamental understanding of what one’s code actually makes the machine do is necessary to write quality code regardless of how high up the abstraction stack it is
Yes and they’re far less efficient and require far more maintenance than an equivalent electric or even diesel engine, where equivalent power is even possible
Sure if you are doing embedded programming in C. How does one do this in web development though where there are hundreds of dependencies that get updated monthly and still add functionality and keep their job?
The current state of web development is unfortunately a perfect example of this quality crisis. The tangle of dependencies either directly causes or quickly multiplies the inefficiency and fragility we’ve all come to expect from the web. The solution is unrealistic because it involves design choices which are either not trendy enough or precluded by the platform
Yes, and I should overrule half the business decisions of the company while I am at it. Oh, and I'll push back on "we need the next feature next week" and I'll calmly respond "we need to do excellent engineering practices in this company".
And everybody will clap and will listen to me, and I will get promoted.
...Get real, dude. Your comments come across a bit tone-deaf. I am glad you are in a privileged position but you seem to have fell for the filter bubble effect and are unaware to how most programmers out there have to work if they want to pay the bills.
> I completely encourage anyone to learn as much about the tools and stack as possible, but there is only so much time.
That seems like a weird way to think about this. I mean, sure, there's no time today to learn make to complete your C++ ticket or whatever. But yesterday? Last month? Last job?
Basically, I think this matches the upthread contention perfectly. If you're a working C++ programmer who's failed to learn the Normal Stable of Related Tools (make, bash, python, yada yada) across a ~decade of education and experience, you probably never will. You're in that 50% of developers who can't start stuff from scratch. It's not a problem of time, but of curiosity.
Actually it is trivial to write a very simple Makefile for a 10,000 file project, despite the fact that almost all Makefiles that I have ever seen in open-source projects are ridiculously complicated, far more complicated than a good Makefile would be.
In my opinion, it is a mistake almost always when you see in a Makefile an individual rule for making a single file.
Normally, there should be only generic building rules that should be used for building any file of a given type.
A Makefile should almost never contain lists of source files or of their dependencies. It should contain only a list with the directories where the source files are located.
Make should search the source directories, find the source files, classify them by type, create their dependency lists and invoke appropriate building rules. At least with GNU make, this is very simple and described in its user manual.
If you write a Makefile like this, it does not matter whether a project has 1 file or 10,000 files, the effort in creating or modifying the Makefile is equally negligible. Moreover, there is no need to update the Makefile whenever source files are created, renamed, moved or deleted.
If everything in your tree is similar, yes. I agree that's going to be a very small Makefile.
While this is true, for much larger projects, that have lived for a long time, you will have many parts, all with slight differences. For example, over time the language flavour of the day comes and goes. Structure changes in new code. Often different subtrees are there for different platforms or environments.
The Linux kernel is a good, maybe extreme, but clear example. There are hundreds of Makefiles.
I like Makefiles, but just for me. Each time I create a new personal project, I add a Makefile at the root, even if the only target is the most basic of the corresponding language. This is because I can't remember all the variations of all the languages and frameworks build "sequences". But "$ make" is easy.
I’d be curious to hear your ratio. It really varies. In some small teams with talented people, there are hardly any “fake” developers. But in larger companies, they can make up a huge chunk.
Where I am now, it’s easily over 50%, and most of the real developers have already left.
PS: The fakes aren’t always juniors. Sometimes you have junior folks who are actually really good—they just haven’t had time yet to discover what they don’t know. It’s often absolutely clear that certain juniors will be very good just from a small contribution.
My personal experience:
- 5% geniuses. This are people who are passionate about what they do, they are always up to date. Typically humble, not loud people.
- 15% good, can do it properly. Not passionate, but at least have a strong sense of responsibility. Want to do “the right thing” or do it right. Sometimes average intelligence, but really committed.
- 80% I would not hire. People who talk a lot, and know very little. Probably do the work just because they need the money.
That applies for doctors, contractors, developers, taxi drivers, just about anything and everything. Those felt percentages had been consistent across 5 countries, 3 continents and 1/2 a century of life
PS: results are corrected for seniority. Even in the apprentice level I could tell who was in each category.
At my work I've noticed another contributing factor: tools/systems that devs need to interact with at some point, but otherwise provide little perceived value to learn day-to-day.
Example is build system and CI configuration. We absolutely need these but devs don't think they should be expected to deal with them day to day. CI is perceived as a system that should be "set and forget", like yeah we need it but really I have to learn all this just to build the app? Devs expect it to "just work" and if there are complexities then another team (AKA my role) deals with that. As a result, any time devs interact with the system, there's a high motivation to copy from the last working setup and move on with their day to the "real" work.
The best solution I see is meet the devs halfway. Provide them with tooling that is appropriate simple/complex for the task, provide documentation, minimise belief in "magic". Tools like Make kinda fail here because they are too complex and black-box-like.
The local part is my big problem too. I used azure Dev ops in work. I find clicking through the UI to be a miserable experience, Id love to have it running locally so I could view inputs and outputs on the file system. Also yaml is an awful choice, no one I know enjoys working with it. The white space issues just get worse and worse longer your files get.
The office coffee machine is not „set and forget”, but you wouldn’t expect the entire responsibility for it’s maintenance to be evenly distributed between all people that use it. Similarly, CI needs ownership and having it fall on the last developer that attempted to use it is not an efficient way of working.
Yeah, I think this is the real issue. Too many different tool types that need to interact, so you don't get a chance to get deep knowledge in any of them. If only every piece of software/CI/build/webapp/phone-app/OS was fully implemented in GNU make ;-) There's a tension between using the best tool for the job vs adding yet another tool/dependency.
I think Makefile is maybe the wrong analogy - the problem with most people and makefiles is they write so few of them, the general idea of what make does is at hand, but the muscle memory of how to do it from scratch is not.
But, point taken - I've seen so much code copy-pasta'd from the web, there will be like a bunch of dead stuff in it that's actually not used. A good practice here is to keep deleting stuff until you break it, then put whatever that was back... And delete as much as possible - certainly everything you're not using at the moment.
This is exactly the problem I face with many tools, Makefiles, KVM setups, docker configurations, CI/CD pipelines. My solution so far has been to create a separate repository with all my notes, shell script example programs etc, for these tool, libraries or frameworks. Every time I have to use these tools, I refer to my notes to refresh my memory, and if I learn something new in the process, I update the notes. I can even point an LLM at it now and ask it questions.
The repository is personal, and contains info on tools that are publicly available.
I keep organisation specific knowledge in a similar but separate repo, which I discard when my tenure with a client or employer ends.
I'm usually contractually obligated to destroy all client IP that I may posses at the end of an engagement. My contracts usually specify that I will retain engagement specific information for a period of six months beyond the end of the contract. If they come back within that time, then I'll have prior context. Otherwise it's gone. Occasionally, a client does come back after a year or two, but most of the knowledge would have been obsolete and outdated anyway.
As for LLMs. I have a couple of python scripts that concatenate files in the repo into a context that I pass to Google's Gemini API or Google AI studio, mostly the latter. It can get expensive in some situations. I don't usually load the whole repository. And I keep the chat context around so I can keep asking question around the same topic.
I wouldn't say this is necessarily a bad thing. I wrote my first version of a Makefile with automatic dependencies and out-of-tree builds 10+ years ago and I have been copying and improving it since. I do try to remove unneeded stuff when possible.
The advantage is that one can go in and modify any aspect of build process easily, provided one takes care to remove cruft so that the Makefile does not become huge. This is very important for embedded projects. For me, the advantages have surpassed the drawbacks (which I admit are quite a few).
You could, in theory, abstract much of this common functionality away in a library (whether for Make or any other software), however properly encapsulating the functionality is additional work, and Make does not have great built-in support for modularization.
In this sense I would not say Make is overly complex but rather the opposite, too simple. Imagine how it would be if in C global variables were visible across translation units. So, in a way, the "Makefile effect" is in part due to the nature of the problem being solved and part due to limitations in Make.
Is it not a problem which is basically COMPLETELY SOLVED by LLMs ?
The reason this happens is because Makefiles (or CI/CD pipelines / linters config, bash scripts) are more or less "complete language" on their own, that are not worth learning when you can do ... exactly what the author says (copy/pasting/modifying until it works) 99% of the time.
But LLMs in general know the language so if you ask "write a minimal Makefile that does this" or even "please simplify the Makefile that i copy/pasted/modified", my experience is that they do that very well actually.
This is “Copy-Pasta Driven Development” [0] and it’s not even related to makefiles. It’s related to the entire industry copying code from here to there without even knowing what they are copying.
TBH I think copilot has made this even worse, as we are blindly accepting chucks of code into our code bases.
Blame the business people. I tried becoming an expert in `make` probably at least 7 times in a row, was never given time to work with it daily until I fully memorized it.
At one point I simply gave up; you can never build the muscle memory and it becomes a cryptic arcane knowledge you have to relearn from scratch every time you need it. So I moved to simpler tools.
The loss of deep work is not the good programmers' fault. It's the fault of the business people.
If I had a nickel for every time I have seen a Makefile straight up copied from other projects and modified to "work" while leaving completely unrelated unnecessary build steps and targets in place.
You find the first part in your stack that is documented (e.g., make is documented, even if your makefile is not) and use that documentation to understand the undocumented part.
You then write down your findings for the next person.
If you don’t have enough time, write down whatever pieces you understood, and write down what parts “seem to work, but you don’t understand“ to help make progress towards better documentation.
If you put the documentation as comments into the file, this can make copy&pasting working examples into a reasonably solid process.
I have made conscious effort in the past to never copy/paste the initial fleshing-out of a Makefile or a PHP class, or HTML boilerplate, or whatever. Like, for years I stuck to that. Then I stopped making that effort because there is no upside. Or rather, there is no downside to copy+paste+modify. It's faster and you save your brain power for things that actually matter.
There's a subtle difference between a nice template and a fully-working implementation that you then modify though.
(e.g. in that they were designed with different goals in mind, so the former is likely to have stopped at the point where it was general enough, to save you time, but not too specific to create footguns).
Bonus points if your template explicitly has fail patterns that prevent your code from silently failing.
Okey but to me, copying - pasting working code (even with sone extra unused bits) really looks no more different than inheriting a library - provided base class, and then extending it to one's needs.
That's literally the basis of all software. There is no need to invent "a Makefile effect/syndrome"
Yes that's an indication that a code sharing mechanism is needed but not implemented. Copying pasting solves that. You don't expect people to rewrite http client for every project which interacts with APIs, so you?
I think this is a good point. As somewhat of a tangent I have vaguely been thinking of the difference between copy pasting and explicitly extending for a bit.
It seems that in many cases, adapting copy pasted code has some benefits over importing and adjusting some library code. https://ui.shadcn.com/ is an example of going the copy paste direction. It seems to me this is preferable when tweaking the exact behaviour is more important than keeping up to date with upstream or adhering to an exact standard. If you customize the behaviour a lot the extra abstraction layer only gets in the way.
This insight might be a bit mundane. But I remember myself bending over backwards a bit too much trying to reuse when copy pasting is fine.
Well, I expect people to understand http clients and if things don't work to be sufficiently knowledgeable to recognize when they have a performance problem and figure out why they have it. For that one needs language, library and networking skills which to a degree most developers have because they do it every day.
At issue however are niche skills. We are dealing with the long tail of a distribution and heuristics which work most of the time might not - the author mentions e.g. security. The way I look at this is risk i.e. security, bus factor, disruptions due to software moving from state "works and is not understood" to "broken and is not understood" and last but not least ability to predict behavior of this niche technology when it is going to be pushed into an larger project.
I end up doing the copy paste thing quite a lot with build tools, it was very common in Ant, Maven and then in Scala build tool. When your projects all have the same fundamental top level layout and you are doing the same actions over and over you solve the problem once then you copy and paste it and remove the bits that don't apply.
These types of tools there isn't much you do differently they don't give you much in the way of abstractions its just a list of actions which are very similar between projects. Since you typically with them are working in declarations rather than the usual programming primitives it often fundamentally falls down to "does my project need this build feature or not?".
I guess this is an effect of declarative programming and layered abstractions. The declarative syntax and abstraction are an answer to code being repetitive and long and hard to follow, but this then creates its own issues by making it harder to reason (especially for beginners or occasional users) about what is actually going on. The price for learning how to get it right just becomes much higher with every layer of abstraction inbetween, because you always have to learn what's going on underneath the "cushions" anyway.
For me typical examples are Terraform configurations with their abstracted configuration syntax, which just mimicks some other configuration (e.g. AWS) and executes it in an environment where I don't necessarily have access to. Of course I'm not going to run endless experiments by reading documentation, assembling my own config and running it in painful slow CI pipelines until it works. I'll rather copy it from another project where it works and then go back to work on things that are actually relevant and specific for the business.
I did that, then I needed to tweak things so I added options, then I needed to use the package somewhere that needed to be self-contained, so I started copy-pasting ;). I've done similar things with makefiles, tox configs, linter settings (all of which started from an initial version I wrote from scratch).
I suspect the real reason this effect exists is because there's copy-pasting is the best way to solve the problem, due to a varying mix of: there being no way of managing the dependencies, needing to avoid (unmanaged) dependencies (i.e. vendoring is the same, only we have a tool managing it), the file (or its contents) needing to exist there specifically (e.g. the various CI locations) and no real agreement on what template/templating tool to use (and a template is just as likely to include useless junk). Copy-pasting is viewed as a one-time cost, and the thing copy-pasted isn't expected to change all that much.
I guess that there's a very important difference between copying something that you understand (or at least the details of which, like syntax, you can easily remember - here comments become important),
and copying something that not only you do not understand, but you were not the one that made it in the first place, and you never understood it !
> Does it need syntax of its own? As a corollary: can it reuse familiar syntax or idioms from other tools/CLIs?
I’m with the author here 100%. Stop inventing new syntaxes and formats for things that don’t need it. It’s not clever, it’s a PITA when it doesn’t work as expected at 3:30 on a Friday.
Is not this a very generic phenomenon? I would argue it applies broadly. For example budgeting, you usually start from last year's budget and tweak that, rather than start from scratch. Or when you write an application letter, or a ServiceNow ticket, or whatever. Now I regret that I have brought in ServiceNow in the discussion, it kills the good mood....
But as I understand it and I am not an accountant (IANAA?), for non-ZBB budgets last years budget is usually used as a starting point and increases are justified.
"Here's why I need more money to do the same things as last year, plus more money if you want me to do anything extra".
I'd be curious what our man Le Cost Cutter Elon Musk does for budgeting?
I have observed the Makefile effect many times for LaTeX documents. Most researchers I worked with had a LaTeX file full of macros that they have been carrying from project to project for years. These were often inherited from more senior researchers, and were hammered into heavily-modified forks of article templates used in their field or thesis templates used at their institution.
This is a great example of an instance of this "Makefile effect" with a possible solution: use Markdown and Pandoc where possible. This won't work in every situation, but sometimes one can compose a basic Beamer presentation or LaTeX paper quickly using largely simple TeX and the same Markdown syntax you already know from GitHub and Reddit.
That won’t solve any problem that LaTeX macros solve. Boilerplate in LaTeX has 2 purposes.
The first is to factor frequently-used complex notations. To do this in Markdown you’d need to bolt on a macro preprocessor on top of Markdown.
The second one is to fine-tune typography and layout details (tables are a big offender). This is something that simply cannot be done in Markdown. A table is a table and if you don’t like the style (which is most of the time inadequate) then there is no solution.
> the tool (or system) is too complicated (or annoying) to use from scratch.
Or boring: some systems require boilerplate with no added value. It's normal to copy & paste from previous works.
Makefiles are a good example. Every makefile author must write their own functionally identical "clean" target. Shouldn't there be an implicit default?
C is not immune, either. How many bits of interesting information do you spot in the following excerpt?
The printf alone is the real payload, the rest conveys no information. (Suggestion for compiler authors: since the programs that include stdio.h outnumber those that don't, wouldn't it be saner for a compiler to automatically do it for us, and accept a flag to not do it in those rare cases where we want to deviate?)
> since the programs that include stdio.h outnumber those that don't
I don't think that is true. There is a lot of embedded systems C out there, plus there are a lot of files in most projects, and include is per file not per project. The project might use stdio in a few files, and not use it in many others.
> Makefiles are a good example. Every makefile author must write their own functionally identical "clean" target. Shouldn't there be an implicit default?
At some point you have to give the system something to go on, and the part where it starts deleting files seems like a good one where not to guess.
It's plenty implicit in other places. You can for example, without a Makefile even, just do `make foo` and it will do its best to figure out how to do that. If there's a foo.c you'll get a `foo` executable from that with the default settings.
> The printf alone is the real payload, the rest conveys no information.
What are you talking about? Every line is important.
#include <stdio.h>
This means you need IO in your program. C is a general purpose language , it shouldn't include that unless asked for. You could claim it should include stuff by default, but that would go completely against what C stands for. Code shouldn't have to depend on knowing which flags you need to use to compile successfully (at least not in general like this).
int main(int argc, char** argv)
Every program requires a main function. Scripting languages pretend they don't, but they just wrap all top-level code in one. Having that be explicit, again, is important for a low level language like C. By the way, the C standard lets you declare it in a simplified manner:
int main(void)
Let's ignore the braces as you could just place them on the same line.
printf("Hello\n");
You could just use `puts` here, but apart from that, yeah that's the main payload, cool.
return 0;
The C standard actually makes this line optional. Funny but I guess it addresses your complaint that "common stuff" perhaps should not be spelled out all the time?
So, here is the actual minimalist Hello world:
#include <stdio.h>
int main(void) {
puts("Hello world\n");
}
Yeah, I've always been mystified by the idea that writing a new Makefile is some kind of wizardly mystery. Make has its design flaws, for sure, but how hard is it really to write this?
I haven't tested what I just typed above, but I'm reasonably sure that if I biffed it in a way that makes it nonfunctional, it will be obvious how to correct the problem.
I mean, not that you can't do better than that (I'm pretty sure anyone experienced can see some problems!), or that there aren't tricky and annoying tradeoffs, but it just doesn't seem like a big activation barrier the way people sometimes make it out to be?
Maybe those people just need to spend an afternoon once in their life working through a basic make tutorial? Maybe not the first time they work on a project using make, but, maybe, after the fifth or sixth project when they realize that this somewhat primitive inference engine is going to be something they interact with daily for years? At some point you're getting into "lead a horse to water" or "teach a man to fish" territory. There's a limit to how much you can empower someone who's sabotaging themself.
There's a slightly less minimal example in https://www.gnu.org/software/make/manual/html_node/Simple-Ma... with a full explanation. You can read it in a few minutes, but of course you have to experiment to actually learn it. The whole GNU Make 4.4.1 manual in PDF form is only 229 pages, so you can read it after dinner one night, or on your commute on the train over the course of a few days. And then you'll know the complete rules of the game.
Same with programming: You just copy some old code and modify it, if you have something lying around.
Same with frameworks (Angular, Spring Boot, ...). The tools even come with templates to generate new boilerplate for people who don't have existing ones somewhere.
I see this effect in Java Maven pom.xml files. It's hard to get a straightforward answer on why each build step is needed, what each attribute means, what parts are optional or mandatory, etc. There seems to be a culture of copying these XML files and tweaking a few things without truly understanding what the whole file means. I briefly looked at Ant and Gradle, and their ecosystems don't look any better. The build configuration files seem to have too much unexplainable magic in them.
Imo, the only solution is to avoid boilerplate generators and the parent poms projects like spring boot use for things like pom files: you can look at the boilerplate to get ideas for what might be necessary, but, if you’re starting a project, write the pom yourself. It’s a pain the first couple times, but it gets easier to know what you need.
> I briefly looked at …Gradle… The build configuration files seem to have too much unexplainable magic in them.
This is largely due to the use of groovy. When the Kotlin DSL is used instead, it can usually be introspected by (eg) IntelliJ. Otherwise, it’s pretty opaque.
Unless you know this, there's zero way you will come up with this by typing `configure` and using just auto-completion. Might as well use Groovy and a String for the name of the thing you're configuring. Good tooling would be able to auto-complete from there whether it's Groovy or Kotlin (or Java etc).
That wasn’t my experience a few years ago with a large groovy-dsl project. Since groovy will take a look in several different namespaces to automatically resolve things in a script, editors I tried had no hope of telling me what anything was.
Also, groovy allows modification of private instance variables which leads to … un-fun situations. I converted tens of thousands of lines of groovy to Kotlin. A lot of those lines were automated. Too many were not automatable for myriad reasons.
As far as the magic in Kotlin, I can easily click through all keywords and jump to the implementation in IJ. Groovy (at the time and in the project I was in) was utterly hopeless in this regard.
Groovy closure delegates' type can be declared, giving as much information as with Kotlin. The reason you couldn't follow the code was that the people who wrote those things either didn't declare types, or IntelliJ wasn't using the type declarations (I believe Groovy support in Gradle files is less good than in general Groovy files, where the IDE does support this). You're correct that some plugins will resolve things dynamically and those cannot be resolved by the IDE. But that's not the fault of the language, if you're going to rewrite in Kotlin with types, you could just as well add types to your Groovy declarations for the same result.
Honestly for Java I really like Bazel. You should give it a shot. I have a project with a self contained jvm and jars from maven central. Its more explicit than the other options but way less magical IMO.
This is pretty thought provoking. I think the issue is "80% of the use of this complicated tool is for very simple ends". From there you get a lot of "I can't be bothered to learn git/make/sed/helm/jenkins, all I'm doing is X 15 minutes a year". My guess is SWEs hate ceilings, so we don't want to use tools that have them, even though they'd be far more fit for purpose. We also don't want to build tools with ceilings: why limit your potential userbase/market?
Copy+tweak happens IRL all the time. There's no reason everyone who bakes should have to reinvent biscuits from scratch. There's no reason chip manufacturers should have to reinvent N-type P-type sandwiches from scratch. The existence of adaptations of previous success does not suggest that baking, or physics, or Make, is overly complicated.
This only happens because people treat build code at a lower standard than app code. IMO you should treat all code with the same rigour. From build scripts to app code to test code.
Why write hacks in build tools when you wouldn’t do in your app code.
We build tool code with the same quality as the app code. That’s why most tooling we use are written in typescript: type safety, code reuse…
I would argue the main reason is that Make is just bad. There are easier to use alternatives such as scons or rake that don't have this effect applied to them.
To me it seems fine that a tool that is both complexity and versatile needs a config file that is beyond memorization. So I think this line of reasoning has limitations.
I could see it with say CLI tools though. Like if I need to reference my notes for a CLI command then that may well indicate a failure in tool design.
>repeatedly copy a known-good solution and accrete changes over time.
Alternative phrasing would be that it evolves. Arguably there is a positive trajectory there
Makefiles have an even more interesting issue: They lost their main purpose. In many, many projects that I've seen, they only consist of phony targets. No dependency tracking is used whatsoever.
How many Makefiles are there that just Wrap npm, pip, or some other tool like that? A Makefile is supposed to be the build system, not trigger it.
Okay but make is a shitty build system. What it does have going for it is you can nearly universally expect it to be already installed or easy to install. That makes it a good way to name commands shorter in a portable way, with some dependencies maybe thrown in.
It’s used for the same reason we write shell scripts
> However, at the point of design, this suggests a tool design (or tool application) that is flawed: the tool (or system) is too complicated (or annoying) to use from scratch.
As someone who teaches and sees college-level students ask chatgpt what's 1 + 1, I disagree that it has anything to do with complexity or annoyance.
I think this is completely normal for tools that you program seldomly. I write makefiles a couple of times a year, I've been using make for more than 40 years now, I use it every day, but I seldomly program it, and when I want something more than simple dependancies I often clone something that already works.
On the other hand, there are cases where (beneficial/desired) verbosity prompts copy-paste and tweaking - not due to complexity but from some form of scale or size of the input.
In many cases this is a sign of something that should be dynamic data (put it in a db instead of conf) but that's not always the case and worth the tradeoff in the moment.
Old IBM mainframe scripting in JCL https://en.wikipedia.org/wiki/Job_Control_Language (so "OS JCL" now, I suppose) used to have a terrible reputation for this, but I've never actually touched the stuff myself.
Honestly, my .zshrc file started out as a .kshrc file that was passed down to me by an older developer about 20 years ago, when I was still in university. I've added and removed a lot of things over the years, but there are still a few parts of it that I don't totally understand, simply because they work and I've never had a reason to think about them. The guy I got it from, in turn, got his from someone else.
In the old days, I had a .fvwm2rc config file that I got from my boss in the university computing center. I had no idea how it worked! And neither did he -- he got it from a professor when he was in university.
amazon's internal build tool experiences this same phenomena. engineers are hired based on their leetcode ability; which means the average engineer has gaps in their infrastructure and config tool knowledge/skillset. until the industrys hiring practices shift, this trend will continue.
As an undergrad, I did group projects with people who quite literally could not compile and run any actual project on their system outside of a pre-packaged classwork assignment, who essentially could not code at all outside of data structure and algorithm problem sets, who got Google internships the next semester.
But they were definitely brighter than I when it came to such problem sets. I suppose we need both sorts of engineer to make great things
A better name for this might be the JCL effect, as even experienced mainframe sysprogs copypasta the JCL it takes to build their COBOL programs from a known-good example and then mutatis the mutandis, rather than attempt to build a mental model of how JCL works from the impenetrable documentation and write JCL de novo.
It's no big deal to me to write a small Makefile from scratch. My editor (Emacs) even knows to always use tabs when I hit TAB in a Makefile, removing the confusion of whether I inserted tabs (correct) or spaces (horribly incorrect) on the lines with the commands to build a particular target.
We recycle known good stuff to avoid getting bogged down and introducing fresh flaws.
The admonition to know what we're doing and act deliberately applies to so much in life, but flies in the face of Milton Friedman's point in "I, Pencil" => https://youtu.be/67tHtpac5ws?si=yhheE1Y5ELfjWXs-
The issue of course is the islanders did not understand the science behind planes, Wallis talkies, guns, etc.
Likewise, cargo cult devs see what is possible, but do not understand first principles, so they mimic what they see their high priests of technology doing, hoping they can copy their success.
Hence the practice of copying, pasting, trying, fiddling, googling, tugging, pulling and tweaking hoping that this time it will be just right enough to kind of work. Badly, and only with certain data on a Tuesday evening.
There is an implicit assumption that the code written espouses best-practices, but that is far from the truth.
Given that distribution, I’d guess that well over 50% of Makefiles are just random chunks of copied and pasted code that kinda work. If they’re lifted from something that already works, job done—next ticket.
I’m not blaming the tools themselves. Makefiles are well-known and not too verbose for smaller projects. They can be a bad choice for a 10,000-file monster—though I’ve seen some cleanly written Makefiles even for huge projects. Personally, it wouldn’t be my first choice. That said, I like Makefiles and have been using them on and off for at least 30 years.
Small nuance: I think people often don’t know because they don’t have the time to figure it out. There are only so many battles you can fight during a day. For example if I’m a C++ programmer working on a ticket, how many layers of the stack should I know? For example, should I know how the CPU registers are called? And what should an AI researcher working always in Jupyter know? I completely encourage anyone to learn as much about the tools and stack as possible, but there is only so much time.
Specifically for the examples at hand:
- at 20%, you will be able to write a Makefile from scratch within the first day of picking up the manual, rather than two or three weeks if you only invest 1%.
- if you don't know what the CPU registers are, the debugger won't be able to tell you why your C++ program dumped core, which will typically enable you to resolve the ticket in a few minutes (because most segfaults are stupid problems that are easy to fix when you see what the problem is, though the memorable ones are much hairier.) Without knowing how to use the disassembly in the debugger, you're often stuck debugging by printf or even binary search, incrementally tweaking the program until it stops crashing, incurring a dog-slow C++ build after every tweak. As often as not, a fix thus empirically derived will merely conceal the symptom of the bug, so you end up fixing it two or three times, taking several hours each time.
Sometimes the source-level debugger works well enough that you can just print out C++-level variable values, but often it doesn't, especially in release builds. And for performance regression tickets, reading disassembly is even more valuable.
(In C#, managed C++, or Python, the story is of course different. Until the Python interpreter is segfaulting.)
How long does it take to learn enough assembly to use the debugger effectively on C and C++ programs? Tens of hours, I think, not hundreds. At 20% you get there after a few dozen day-long debugging sessions, maybe a month or two. At 1% you may take years.
What's disturbing is how many programmers never get there. What's wrong with them? I don't understand it.
If you never worked with them, you should count yourself lucky.
And everybody will clap and will listen to me, and I will get promoted.
...Get real, dude. Your comments come across a bit tone-deaf. I am glad you are in a privileged position but you seem to have fell for the filter bubble effect and are unaware to how most programmers out there have to work if they want to pay the bills.
For everything else, there's MasterCard.
That seems like a weird way to think about this. I mean, sure, there's no time today to learn make to complete your C++ ticket or whatever. But yesterday? Last month? Last job?
Basically, I think this matches the upthread contention perfectly. If you're a working C++ programmer who's failed to learn the Normal Stable of Related Tools (make, bash, python, yada yada) across a ~decade of education and experience, you probably never will. You're in that 50% of developers who can't start stuff from scratch. It's not a problem of time, but of curiosity.
In my opinion, it is a mistake almost always when you see in a Makefile an individual rule for making a single file.
Normally, there should be only generic building rules that should be used for building any file of a given type.
A Makefile should almost never contain lists of source files or of their dependencies. It should contain only a list with the directories where the source files are located.
Make should search the source directories, find the source files, classify them by type, create their dependency lists and invoke appropriate building rules. At least with GNU make, this is very simple and described in its user manual.
If you write a Makefile like this, it does not matter whether a project has 1 file or 10,000 files, the effort in creating or modifying the Makefile is equally negligible. Moreover, there is no need to update the Makefile whenever source files are created, renamed, moved or deleted.
While this is true, for much larger projects, that have lived for a long time, you will have many parts, all with slight differences. For example, over time the language flavour of the day comes and goes. Structure changes in new code. Often different subtrees are there for different platforms or environments.
The Linux kernel is a good, maybe extreme, but clear example. There are hundreds of Makefiles.
Where I am now, it’s easily over 50%, and most of the real developers have already left.
PS: The fakes aren’t always juniors. Sometimes you have junior folks who are actually really good—they just haven’t had time yet to discover what they don’t know. It’s often absolutely clear that certain juniors will be very good just from a small contribution.
That applies for doctors, contractors, developers, taxi drivers, just about anything and everything. Those felt percentages had been consistent across 5 countries, 3 continents and 1/2 a century of life
PS: results are corrected for seniority. Even in the apprentice level I could tell who was in each category.
Who likely wouldn't have a job if it weren't for LLMs.
Example is build system and CI configuration. We absolutely need these but devs don't think they should be expected to deal with them day to day. CI is perceived as a system that should be "set and forget", like yeah we need it but really I have to learn all this just to build the app? Devs expect it to "just work" and if there are complexities then another team (AKA my role) deals with that. As a result, any time devs interact with the system, there's a high motivation to copy from the last working setup and move on with their day to the "real" work.
The best solution I see is meet the devs halfway. Provide them with tooling that is appropriate simple/complex for the task, provide documentation, minimise belief in "magic". Tools like Make kinda fail here because they are too complex and black-box-like.
- They're often slow
- They're often proprietary
- They're often dealing with secrets which limits who can work on them
- You generally can't run them locally
So the feedback cycle for working on them is incredibly long. And working on them is therefore a massive pain.
I recognize that this is such a disincentive for me taking the initiative to fiddle with and learn about anything like this
GitLab CI gives you local runners. You can completely self-host CI.
But, point taken - I've seen so much code copy-pasta'd from the web, there will be like a bunch of dead stuff in it that's actually not used. A good practice here is to keep deleting stuff until you break it, then put whatever that was back... And delete as much as possible - certainly everything you're not using at the moment.
The repository is personal, and contains info on tools that are publicly available.
I keep organisation specific knowledge in a similar but separate repo, which I discard when my tenure with a client or employer ends.
On a more practical note, what structure, formats and tools do you use that enable you to feed it to an LLM?
As for LLMs. I have a couple of python scripts that concatenate files in the repo into a context that I pass to Google's Gemini API or Google AI studio, mostly the latter. It can get expensive in some situations. I don't usually load the whole repository. And I keep the chat context around so I can keep asking question around the same topic.
The advantage is that one can go in and modify any aspect of build process easily, provided one takes care to remove cruft so that the Makefile does not become huge. This is very important for embedded projects. For me, the advantages have surpassed the drawbacks (which I admit are quite a few).
You could, in theory, abstract much of this common functionality away in a library (whether for Make or any other software), however properly encapsulating the functionality is additional work, and Make does not have great built-in support for modularization.
In this sense I would not say Make is overly complex but rather the opposite, too simple. Imagine how it would be if in C global variables were visible across translation units. So, in a way, the "Makefile effect" is in part due to the nature of the problem being solved and part due to limitations in Make.
The reason this happens is because Makefiles (or CI/CD pipelines / linters config, bash scripts) are more or less "complete language" on their own, that are not worth learning when you can do ... exactly what the author says (copy/pasting/modifying until it works) 99% of the time.
But LLMs in general know the language so if you ask "write a minimal Makefile that does this" or even "please simplify the Makefile that i copy/pasted/modified", my experience is that they do that very well actually.
TBH I think copilot has made this even worse, as we are blindly accepting chucks of code into our code bases.
[0] https://andrew.grahamyooll.com/blog/copy-pasta-driven-develo...
At one point I simply gave up; you can never build the muscle memory and it becomes a cryptic arcane knowledge you have to relearn from scratch every time you need it. So I moved to simpler tools.
The loss of deep work is not the good programmers' fault. It's the fault of the business people.
It's a major pet peeve of mine.
Trial and error?
Well have fun with that :p
If you don’t have enough time, write down whatever pieces you understood, and write down what parts “seem to work, but you don’t understand“ to help make progress towards better documentation.
If you put the documentation as comments into the file, this can make copy&pasting working examples into a reasonably solid process.
I interpret it in a bit of different way.
Makefile is relatively simple and unopinionated like a brick. Also makefile defines/reflects project’s structure.
From simple blocks one can build any shape one want. Total freedom.
Problem is, make doesn’t impose best practice and doesn’t steer you clear of common pitfalls of project structuring and building and publishing.
One example for illustration: Out of source builds is rather good idea, but not imposed by make.
So makefile is not enough, one needs all the life-lessons of using make, so inherited makefiles are better than written from scratch.
(e.g. in that they were designed with different goals in mind, so the former is likely to have stopped at the point where it was general enough, to save you time, but not too specific to create footguns).
Bonus points if your template explicitly has fail patterns that prevent your code from silently failing.
That's literally the basis of all software. There is no need to invent "a Makefile effect/syndrome"
Yes that's an indication that a code sharing mechanism is needed but not implemented. Copying pasting solves that. You don't expect people to rewrite http client for every project which interacts with APIs, so you?
It seems that in many cases, adapting copy pasted code has some benefits over importing and adjusting some library code. https://ui.shadcn.com/ is an example of going the copy paste direction. It seems to me this is preferable when tweaking the exact behaviour is more important than keeping up to date with upstream or adhering to an exact standard. If you customize the behaviour a lot the extra abstraction layer only gets in the way.
This insight might be a bit mundane. But I remember myself bending over backwards a bit too much trying to reuse when copy pasting is fine.
At issue however are niche skills. We are dealing with the long tail of a distribution and heuristics which work most of the time might not - the author mentions e.g. security. The way I look at this is risk i.e. security, bus factor, disruptions due to software moving from state "works and is not understood" to "broken and is not understood" and last but not least ability to predict behavior of this niche technology when it is going to be pushed into an larger project.
These types of tools there isn't much you do differently they don't give you much in the way of abstractions its just a list of actions which are very similar between projects. Since you typically with them are working in declarations rather than the usual programming primitives it often fundamentally falls down to "does my project need this build feature or not?".
For me typical examples are Terraform configurations with their abstracted configuration syntax, which just mimicks some other configuration (e.g. AWS) and executes it in an environment where I don't necessarily have access to. Of course I'm not going to run endless experiments by reading documentation, assembling my own config and running it in painful slow CI pipelines until it works. I'll rather copy it from another project where it works and then go back to work on things that are actually relevant and specific for the business.
I suspect the real reason this effect exists is because there's copy-pasting is the best way to solve the problem, due to a varying mix of: there being no way of managing the dependencies, needing to avoid (unmanaged) dependencies (i.e. vendoring is the same, only we have a tool managing it), the file (or its contents) needing to exist there specifically (e.g. the various CI locations) and no real agreement on what template/templating tool to use (and a template is just as likely to include useless junk). Copy-pasting is viewed as a one-time cost, and the thing copy-pasted isn't expected to change all that much.
and copying something that not only you do not understand, but you were not the one that made it in the first place, and you never understood it !
I’m with the author here 100%. Stop inventing new syntaxes and formats for things that don’t need it. It’s not clever, it’s a PITA when it doesn’t work as expected at 3:30 on a Friday.
But as I understand it and I am not an accountant (IANAA?), for non-ZBB budgets last years budget is usually used as a starting point and increases are justified.
"Here's why I need more money to do the same things as last year, plus more money if you want me to do anything extra".
I'd be curious what our man Le Cost Cutter Elon Musk does for budgeting?
That won’t solve any problem that LaTeX macros solve. Boilerplate in LaTeX has 2 purposes.
The first is to factor frequently-used complex notations. To do this in Markdown you’d need to bolt on a macro preprocessor on top of Markdown.
The second one is to fine-tune typography and layout details (tables are a big offender). This is something that simply cannot be done in Markdown. A table is a table and if you don’t like the style (which is most of the time inadequate) then there is no solution.
Or boring: some systems require boilerplate with no added value. It's normal to copy & paste from previous works.
Makefiles are a good example. Every makefile author must write their own functionally identical "clean" target. Shouldn't there be an implicit default?
C is not immune, either. How many bits of interesting information do you spot in the following excerpt?
The printf alone is the real payload, the rest conveys no information. (Suggestion for compiler authors: since the programs that include stdio.h outnumber those that don't, wouldn't it be saner for a compiler to automatically do it for us, and accept a flag to not do it in those rare cases where we want to deviate?)I don't think that is true. There is a lot of embedded systems C out there, plus there are a lot of files in most projects, and include is per file not per project. The project might use stdio in a few files, and not use it in many others.
At some point you have to give the system something to go on, and the part where it starts deleting files seems like a good one where not to guess.
It's plenty implicit in other places. You can for example, without a Makefile even, just do `make foo` and it will do its best to figure out how to do that. If there's a foo.c you'll get a `foo` executable from that with the default settings.
What are you talking about? Every line is important.
This means you need IO in your program. C is a general purpose language , it shouldn't include that unless asked for. You could claim it should include stuff by default, but that would go completely against what C stands for. Code shouldn't have to depend on knowing which flags you need to use to compile successfully (at least not in general like this). Every program requires a main function. Scripting languages pretend they don't, but they just wrap all top-level code in one. Having that be explicit, again, is important for a low level language like C. By the way, the C standard lets you declare it in a simplified manner: Let's ignore the braces as you could just place them on the same line. You could just use `puts` here, but apart from that, yeah that's the main payload, cool. The C standard actually makes this line optional. Funny but I guess it addresses your complaint that "common stuff" perhaps should not be spelled out all the time?So, here is the actual minimalist Hello world:
no
I haven't tested what I just typed above, but I'm reasonably sure that if I biffed it in a way that makes it nonfunctional, it will be obvious how to correct the problem.
I mean, not that you can't do better than that (I'm pretty sure anyone experienced can see some problems!), or that there aren't tricky and annoying tradeoffs, but it just doesn't seem like a big activation barrier the way people sometimes make it out to be?
Maybe those people just need to spend an afternoon once in their life working through a basic make tutorial? Maybe not the first time they work on a project using make, but, maybe, after the fifth or sixth project when they realize that this somewhat primitive inference engine is going to be something they interact with daily for years? At some point you're getting into "lead a horse to water" or "teach a man to fish" territory. There's a limit to how much you can empower someone who's sabotaging themself.
There's a slightly less minimal example in https://www.gnu.org/software/make/manual/html_node/Simple-Ma... with a full explanation. You can read it in a few minutes, but of course you have to experiment to actually learn it. The whole GNU Make 4.4.1 manual in PDF form is only 229 pages, so you can read it after dinner one night, or on your commute on the train over the course of a few days. And then you'll know the complete rules of the game.
Same with frameworks (Angular, Spring Boot, ...). The tools even come with templates to generate new boilerplate for people who don't have existing ones somewhere.
This is largely due to the use of groovy. When the Kotlin DSL is used instead, it can usually be introspected by (eg) IntelliJ. Otherwise, it’s pretty opaque.
Also, groovy allows modification of private instance variables which leads to … un-fun situations. I converted tens of thousands of lines of groovy to Kotlin. A lot of those lines were automated. Too many were not automatable for myriad reasons.
As far as the magic in Kotlin, I can easily click through all keywords and jump to the implementation in IJ. Groovy (at the time and in the project I was in) was utterly hopeless in this regard.
Reminds me of the early internet. Auras of class, cred, erudition, intelligence, mystery, imagination. Thank you.
Why write hacks in build tools when you wouldn’t do in your app code.
We build tool code with the same quality as the app code. That’s why most tooling we use are written in typescript: type safety, code reuse…
I could see it with say CLI tools though. Like if I need to reference my notes for a CLI command then that may well indicate a failure in tool design.
>repeatedly copy a known-good solution and accrete changes over time.
Alternative phrasing would be that it evolves. Arguably there is a positive trajectory there
How many Makefiles are there that just Wrap npm, pip, or some other tool like that? A Makefile is supposed to be the build system, not trigger it.
It’s used for the same reason we write shell scripts
As someone who teaches and sees college-level students ask chatgpt what's 1 + 1, I disagree that it has anything to do with complexity or annoyance.
Humans be humans; that's mostly it.
Sometimes it's better to duplicate code rather than make a special routine to do it.
Sometimes it's not only easier to copy/paste, but it is better than adding a level of abstraction
On the other hand, there are cases where (beneficial/desired) verbosity prompts copy-paste and tweaking - not due to complexity but from some form of scale or size of the input.
In many cases this is a sign of something that should be dynamic data (put it in a db instead of conf) but that's not always the case and worth the tradeoff in the moment.
— Broccoli Man <https://www.youtube.com/watch?v=3t6L-FlfeaI>
In the old days, I had a .fvwm2rc config file that I got from my boss in the university computing center. I had no idea how it worked! And neither did he -- he got it from a professor when he was in university.
I cannot think of a single tool which is complex enough but does not show the makefile effect
But they were definitely brighter than I when it came to such problem sets. I suppose we need both sorts of engineer to make great things
It's a very microsoft feeling pile of crap
It's no big deal to me to write a small Makefile from scratch. My editor (Emacs) even knows to always use tabs when I hit TAB in a Makefile, removing the confusion of whether I inserted tabs (correct) or spaces (horribly incorrect) on the lines with the commands to build a particular target.
We recycle known good stuff to avoid getting bogged down and introducing fresh flaws.
The admonition to know what we're doing and act deliberately applies to so much in life, but flies in the face of Milton Friedman's point in "I, Pencil" => https://youtu.be/67tHtpac5ws?si=yhheE1Y5ELfjWXs-
Now AI does a great job of getting you 90-100% of the way there.