A long time ago, I wrote an essay called “Why Computers Suck” (it was given the title “Computers” and “What’s Wrong With Computers” in two later revisions, and the original title never saw the light of day). The article was fairly long, but it basically came down to the idea that computers suck because programmers create crazy complicated stuff that nobody else can understand, and complexity builds on complexity until every aspect of a program becomes unmanageable.
What I didn’t know at the time was why programmers did this. It was obvious that they did do it, but why would the software development industry produce so many crazy, complex masses of unreadable code? Why did it keep happening, even when developers should have learned their lesson after their first bad experience? What was it that made programmers not just make bad code, but keep on making bad code?
Well, this was a mystery, but I didn’t worry too much about it at first. Just the revelation that “bad programs are caused entirely by bad programmers”, as simple and obvious as it may seem, was enough to fuel an entire investigation and study into the field of programming, one which had some pretty good results (that’s mostly what I’ve written about on this blog, and it’s also the subject of a book that’s in development). The problem had been defined (bad programmers who create complexity), it seemingly had a solution (describe laws of software design that would prevent this), and that was enough for me.
But it still baffled me that the world’s universities, technical schools, and training programs could turn out such terrible programmers, even with all of the decades of advancement in software development techniques. Sure, a lot of the principles of software design hadn’t been codified, but a lot of good advice was floating around, a lot of it very common. Even if people hadn’t gone to school, didn’t they read any of this advice?
Well, the truth was beyond my imagination, and it took almost five years of working on the Bugzilla Project with a vast number of separate contributors until one day I suddenly realized an appalling fact:
The vast majority (90% or more) of programmers have absolutely no idea what they are doing.
It’s not that they haven’t read about software design (though they likely haven’t). It’s not that the programming languages are too complex (though they are). It’s that the vast majority of programmers didn’t have the first clue what they were really doing. They were just mimicking the mistakes of other programmers–copying code and typing more-or-less meaningless incantations at the machine in the hope that it would behave like they wanted, without any real understanding of the mechanics of the computer, the principles of software design, or the meanings of each individual word and symbol they were typing into the computer.
That is a bold, shocking, and offensive statement, but it has held up in my experience. I have personally reviewed and given feedback on the code of scores of programmers. I have read the code of many others. I have talked to many, many programmers about software development, and I’ve read the writings of hundreds of developers. The number of programmers who really understand what they are doing comprise only about 10% of all the programmers I’ve ever talked to, worked with, or heard about.
In open source, we get the cream of the crop–people who want to program in their spare time. And even then, I’d say only about 20% of open source programmers have a really good handle on what they are doing.
So why is this? What’s the problem? How could there be so many people working in this field who have absolutely no clue what they’re doing?
Well, that sounds a bit like they’re somehow “stupid.” But what is stupidity? People are not stupid simply for not knowing something. There’s a lot of stuff that everybody doesn’t know. That doesn’t make them stupid. That may make them ignorant about certain things, but it doesn’t make them stupid. No, stupidity, real stupidity, is not knowing that you don’t know. Stupid people think they know something when they don’t, or they have no idea that there is something more to know.
This sort of stupidity is something that can be found in nearly every field, and software development is no exception. Many programmers simply don’t know that there could be laws or general guidelines for software development, and so they don’t even go looking for them. At many software companies, there’s no attempt to improve developers’ understanding of the programming language they’re using–perhaps simply because they think that the programmers must “already know it if they were hired to do it”.
Unfortunately, it’s particularly harmful to have this sort of mindset in software development, because there is so much to know if you really want to be good. Anybody who thinks they already know everything (or who has a “blind spot” where they can’t see that there’s more to learn) is having their ability to produce excellent code crippled by a lack of knowledge–knowledge they don’t even know exists and that they don’t even know they lack.
No matter how much you know, there is almost always more to know about any field, and computer programming is no exception. So it’s always wrong to think you know everything.
Sometimes it’s hard to figure out what one should be learning about, though. There’s so much data, where does one start? Well, to help you out, I’ve come up with a few questions you can ask yourself or others to help figure out what areas might need more study:
- Do you know as much as possible about every single word and symbol on every page of code you’re writing?
- Did you read and completely understand the documentation of every single function you’re using?
- Do you have an excellent grasp of the fundamental principles of software development–such a good grasp that you could explain them flawlessly to novice programmers at your organization?
- Do you understand how each component of the computer functions, and how they all work together?
- Do you understand the history of computers, and where they’re going in the future, so that you can understand how your code will function on the computers that will be built in the future?
- Do you know the history of programming languages, so that you can understand how the language you’re using evolved and why it works like it does?
- Do you understand other programming languages, other methods of programming, and other types of computers than the one you’re using, so that you know what the actual best tool for each job is?
From top to bottom, those are the most important things for any programmer to know about the code they’re writing. If you can truthfully answer “yes” to all those questions, then you are an excellent programmer.
It may seem like an overwhelming list. “Wow, the documentation for every single function? Reading that is going to take too long!” Well, you know what else takes a long time? Becoming a good programmer if you don’t read the documentation. You know how long it takes? Forever, because it never happens.
You will never become a good programmer simply by copying other people’s code and praying that it works right for you. But even more importantly, investing time into learning is what it takes to become good. Taking the time now will make you a much faster programmer later. If you spend a lot of time reading up on stuff for the first three months that you’re learning a new technology, you’ll probably be 10 times faster with it for the next 10 years than if you’d just dived into it and then never read anything at all.
I do want to put a certain limiter on that, though–you can’t just read for three months and expect to become a good programmer. First of all, that’s just too boring–nobody wants to just study theory for three months and not get any actual practice in. Very few people would keep up with that for long enough to become programmers at all, let alone good programmers. So I want to point out that understanding comes also from practice, not just from study. But without the study, understanding may never come. So it’s important to balance both the study and the practice of programming.
This is not an attack on any programmer that I’ve worked with personally, or even an attack on any individual programmer at all. I admire almost every programmer I’ve ever known, as a person, and I expect I’d admire the rest were I to meet them, as well. Instead, this is an open invitation to all programmers to open your mind to the thought that there might always be more to know, that both knowledge and practice are the key to skill, and that it’s not shameful at all to not know something–as long as you know that you don’t know it, and take the time to learn it when necessary.
-Max
“But it still baffled me that the world’s universities, technical schools, and training programs could turn out such terrible programmers, even with all of the decades of advancement in software development techniques.”
Well there’s your mistake. You see what we do as professional programmers is “software engineering” (or just “development” if you thing we don’t deserve the “engineer” label), whereas what schools teach is “computer science”. To make an analogy to the hard sciences this is like the difference between physics and (real) engineering. Knowing a bunch of equations will be helpful in designing a machine or structure, but it’s not enough. Similarly, knowing about data structures and algorithms will be helpful in designing software, but it’s not enough.
90%+ of those graduating from formal education in computer science don’t know how to work in a team, don’t know how to maintain software or write software that will be maintained, have never used source control or bug tracking, have never written to a third-party API, and have never had occasion to write a program that was actually useful for anything. All these things, they have to learn on-the-job from other people who in turn had to do the same, in an unending chain of ignorance. None of those “advancements in software development techniques” make it into the classroom, nor do most of the tools that real programmers use.
It would be possible to improve things, by unseating the ivory tower PhD crowd from control over university computer departments and replacing them with ace programmers from the 10%. But A) good luck with that and B) why would a really great programmer choose academia over industry or open source?
“It would be possible to improve things, by unseating the ivory tower PhD crowd from control over university computer departments and replacing them with ace programmers from the 10%.”
I’m not so sure about the “replacing” though. Some of the stuff in Max’s post falls under computer science rather than software engineering. Is there actually space in the existing courses to do both properly? Or would it be better to suggest (while we’re suggesting things for an ideal world) some further professional qualification, like “real” engineers, doctors, architects, accountants and many others. Such things exist, at least in the UK, but I don’t know much about them, and as far as I know, not many people have them (relative to other professions, at least).
No, yeah, I think you’re right–computer science can’t go away. It’s very important, although it’s a very small part of actual programming. We need somebody who can sit down and figure out the most efficient mathematical process for a particular algorithm, so that somebody who’s a professional programmer can implement that once and then give it to everybody in a library.
The professional qualification thing, though, I’m not so sure about, because if we had something like that, I’d be concerned that people would use it as a system of authority regardless of actual knowledge or skill, somewhat like some medical professions do.
-Max
I don’t mean to say that we should eliminate computer science from university curriculum. What we need to eliminate is the computer science monopoly on programming instruction. I think it would be perfectly fine to have parallel-yet-overlapping tracks in computer science, and software engineering. Folks who want to do academic research, or program in non-software industries, or focus on abstract things like language design, could stick with computer science, while folks who want to write code for a living could be diverted to software engineering.
After all it’s not that courses like Data Structures, Algorithms, and Principles of Programming Languages don’t have any use to nascent professional coders, it’s just that courses like Numerical Analysis and the ubiquitous required Math minor are largely wastes of time. There are also fundamental problems with how courses are taught in general. Students are typically prohibited from collaborating, which is the exact opposite of how real work gets done. Coding projects typically start from blank files, which again does not reflect what actual programmers do on a daily basis. I believe this is largely due to CS professors typically being career academics rather than retired professionals. I expect they would approach the issues I’ve raised by trying to offer a course in tools, to cover source control, bug tracking, and modern IDEs, whereas the right approach would be to embed these things in the fabric of the entire program. That is the main reason I proposed the radical step of eliminating the academic types entirely, to get a fresh start with a different perspective.
“What we need to eliminate is the computer science monopoly on programming instruction.”
Yeah, I think that’s true. (Rest of your comment here is also pretty intelligent and well-stated, too.)
-Max
Cutting edge programming languages like Haskell, ML, or even Scala could be mainstream if people from universities pushed cutting edge technology more. If anything, universities try way too hard to be compliant with whatever is in fashion in the industry, which doesn’t play to their strengths. They should have heavy duty software engineering courses, but using cutting edge technologies especially if they aren’t used in industry. It’s very hard to train junior developers who refuse to learn anything other than Java, or even how to use it well. Understanding programming languages well, whether it be functional or object oriented is not a strength of the industry.
I agree that universities place too much emphasis against collaboration, when it actually makes it hard for them to work independently when they are on a team since they learn much less by not sharing information in that early stage at university.
IDEs are fine for programmers who are already experienced, but I think it held back junior developers that I worked with by making them lazy and/or unwilling to move outside of their comfort zone. They think too much in terms of their IDE being “magic”, when if they actually learned command line ways of doing things, they would ironically understand how to use their IDEs more creatively as well.
I don’t like the “software engineer” title because it excludes passionate hobbyists who work on open source projects, many of whom are quite capable of working independently, yet almost any professional programmer has the title “software engineer”, regardless of skill. Emphasizing programmers to work in groups the way the industry does just encourages a few alpha programmers, some beta programmers (who are crucially important to the industry if they are actually good), and almost entirely a lot of programmers who refuse to take initiative to do anything and cost the team a lot of money and time. It’s also annoying how I’ll spend weeks training a programmer how a subsystem works, yet they require tons of intervention to even get started, and even then they don’t take responsiblity for the program actually working correctly.
Yeah, I think that everything you say here is pretty right, but the problems that I ran into were equally spread among people who did and didn’t go to school–there was little difference between a formal education and lack of one, except that a formal education can help people read certain types of documentation more easily, because they’ll be familiar with certain terms that are otherwise somewhat difficult to explain.
There *are* software engineering schools and classes, and I think that’s mostly what I was thinking of when I wrote the blog here. The fact is that somehow, even though the outside world seems to be developing lots of new and interesting (or old and rediscovered) software engineering methods, people still graduate from technical training (or simply just from reading up on their programming language online) without a single bit of understanding of what it is they’re actually doing.
-Max
I believe what you may be experiencing is obscurification.
http://www.allwords.com/word-obscurification.html
A lot of times in businesses for program code.. and for web code.. they purposely dump extra stuff in there so that it is much harder to figure out what does what.. and what can be taken out or modified for everything to keep working.
Its a stupid practice.. but I know lots of people that have to bunk up their code just to keep their bosses happy.
Hahahaha, I do experience some of that, I’m sure. I wrote a bit about that, sort of, here:
http://www.codesimplicity.com/post/complexity-is-a-prison/
But what I was seeing, I was seeing equally in open source as well as in commercial projects, so there was absolutely no reason (not that there ever is) to create intentionally obscure code, and my experience in reviews tells me that in fact the coders simply didn’t know what they were doing.
-Max
Making software is a science and an art. I think that the stupidity comes from the person not caring enough or as passionate about their craft. They have to have a real interest in it to study and want to know everything about it, as opposed to doing the bare minimum of study (to collect the paycheck at the end of the week).
That’s a very likely source of the issue, yes. 🙂 You’re anticipating one of my future blog posts a tiny bit, here. 🙂
It could also be that they don’t have time to invest, because there is a deadline for their work, explicit or not. It’s a generic problem – do things right, or make something that “works” quickly.
(The solution where the deadline is moved, to make things right, is unfortunately sometimes very hard to sell, because the short term profits usually are seen as more important than the long term potential savings)
Could also be some kind of pride – if you need to study something, that must mean that you don’t know about it, and some people might not be willing to admit that.
But unfortunately in some cases you are right, there are some people who just don’t care, which should not be confused with the people who make a conscious decision about limiting their work on a given task in an attempt to either do less work overall or to touch more tasks.
Yeah, it’s unfortunate that people do set deadlines in such a way that systems can’t be designed properly, and also unfortunate that given that deadline, we often still try to get in all the features that we want to see. Still, I think that the people who are looking at those short-term profits really don’t understand that they’re doing such damage to the long-run value of the system that, unless there’s some terrific emergency, they’d be way better off waiting just a few weeks or months for the system to get done right. Otherwise it’s such a slippery slope–you’re going to end up with unmet deadlines forever as the system becomes harder and harder to maintain.
That’s definitely true, about the pride thing. I think that’s a big problem, is that some people view admitting lack of knowledge to be some sort of “blow to their pride”, when really the only way that they’ll ever gain ACTUAL skill is by admitting that they don’t know things, frequently and easily! 🙂
-Max
I totally agree with both of your posts here.
I have consistently found that when we rushed to meet my boss’s unrealistic deadlines (he would expect huge changes to be done in two weeks, and even let the customer attend design meetings, which he later regretted), we end up spending more time to redesign the subsystems correctly, and the customers would complain like crazy because the problems would be noticeable and we would take an eternity to add small features to problematic areas of the software.
I find that increasingly I come up with solutions that take a really long time upfront to design but address every corner case in an important subsystem, but I’ve found that doing so actually saves time, because the bug reports go down, we can more easily add complicated features that the customer wants, and it reduces maintenance work because that section of code almost never needs to be changed, and is often more extensible since more thought was put into understanding the problem domain.
It’s gotten to the point where my boss just lets me do things the right way, because he’s noticed the dramatic increase in quality, and happy customers.
Yes, motivation is a key factor here.
In my undergraduate course (I studied electronics) a lot of colleagues were fanatic (read nerds) about the subject. But there were also a few, for them it obvious electronics wasn’t their thing. Many of them were highly intelligent, but still quite unsure about career and life direction in general.
Recently I’ve been doing career and further studies counseling to students in high school. I consistently witness teenagers who are confused what to do with their lifes after they finish school. You cannot be a happy person in life unless you’re able to read your inner compass, understand your desires and talents, and then follow up with conscious decisions. Our school system does very little for that and this task shouldn’t be left to parents either. The focus of teaching is too much on facts about the external world and too little about discovering and creating yourself, about relating to others.
I’ve got an idea to make a comprehensive survey of what adults think about during an average day and use the results to form the curriculum for high schools. So, for example, if adults spend 20% percent of the day thinking about sex (I wouldn’t be surpised if that proves a lot more 😉 , then spend 20% time in school to teach how to express yourself sexually, recipes for having a successful date, how to find a suitable parner, etc. I also wouldn’t be surprised if “what’s the square root of minus one” doesn’t appear at the top of this survey 😉 To me this is the solution to making schools relevant and seeing happier grownups in future.
That’s totally fascinating about your teenagers and I think you’re spot-on that you just can’t be happy in life unless you’re doing something that at least sort of fits your purposes, and to do that you have to know what your purposes are. 🙂
And haha, I can’t imagine how different the world would be if people got some basic relationship advice in school! Of course, I wouldn’t trust schools to give out relationship advice, so that’s a double-edged sword there.
-Max
That many grownups themselves, including teachers, have problems with relationships should not be a reason not to entrust schools to deal with this topic. What other more qualified institution could there be? Parents? Well, parenthood has even lower qualification demands than becoming a teacher 🙂 For that you only need certain body organs; intelligence is not required to produce offspring 🙂
For instruction to be credible and for kids to pay attention the teacher must be a living example of it. But this is hard to get. Therefore such classes must not deliver any “pre-packaged” advice at all, but instead focus on games, discussion and sharing experience, so that everyone can reach his individual conclusions – and think independently. This would also remove a lot of pressure from the teacher and not require him to be a relationship guru or be proficient in all kama sutra positions 🙂
This is the way forward.
Well, I think this is getting pretty off-topic, but I do at least agree that it’d be nice to have schools focus on talking about some things that are more relevant to everyday life.
-Max
High School Senior here…I only need two classes to graduate, once forced to be taken as a senior; the other a fine arts credit. Because of this, outside of school, where I claim my external knowledge (along with the Internet, and correspondence with the local librarians [who have personality and lives in this town]), I spend a great deal of time on introspection. I do this by looking at my dreams (something that school says nothing about), reading manuals on games (and fitting myself into the rules), and other similar things. The imaginary number (square root of minus one) comes into my mind often actually, but I’m an academic person in nature…just like those who are athletic think about the game. Still, 90% of the classes are geared towards academia is an issue…and 5% are towards introspection (drama & relationships). Even art is taught in a somewhat scientific manner…and it is annoying. Classes on sexuality (and not just the bad sides of it) is not a unique idea I have ran across. It came into my mind frequently as a sophomore, and there was a book made in 2001 that discussed it. It is a good idea, and should be implemented, but will not because education is in control of the government, and those in control of government dislike teaching sexuality. So, once you have your curriculum, make sure to create a private school. Kind of ranty this is, but hopefully it helps.
I program OSS games in my free time, although, while it is true that many programmers don’t know what they are doing, you forgot that not everyone has the brains or time to dedicate their code to perfection.
For me it is completely enough when I manage to achieve an effect with trial and error. Then again I am no professional and I just want a product done.
I wish I could make excellent, fast and readable code, but it requires a time and effort investment I cannot afford unfortunately, having a full time job and barely 2-3 hours to hack (and draw and compose and design). Please think of such perspectives!
Hey there. I do think of that perspective, actually, and I can assure you that, in the long run, the effort spent at the beginning to design and code well actually leads to faster production in the end. It definitely seems grueling at first, when you’re coding for days or weeks with no actual “product” yet, but once the framework is in place, it’s amazing how fast things can go. Software projects take so long that (depending on the size of the project) massive amounts of framework and design are always worth it.
-Max
You need to provide or link to examples to back up your assertions.
Something like:
The hyperbole in this article is disconcerting. For example “The vast majority (90% or more) of programmers have absolutely no idea what they are doing.” Obviously if they have “absolutely no idea” then they wouldn’t even be programming. So the reader has to guess as to what the author means.
Hah, hah, hah. The only people who can answer “yes” to all of your questions either have written only trivial programs, have written authored all of the software libraries they are using, or are liars.
I’m an EE and write some programs for fun and hacky things for work. My ego isn’t involved here; I have no problem recognizing many others are much better at programming than me.
For a while, before I got sick of it, I wrote a number of applications that made extensive use of the Win32 API. I literally had about four or five linear feet of API books — Microsoft’s, various API “Bibles,” some broad, some focusing on API niches. There is so much complexity there, so many undocumented restrictions, interactions, and side effects, that I spent most of my time just doinking problematic code until I got things to work. Sometimes I could then infer an undocumented truth that would guide me in the future; sometimes it was simply a voodoo incantation that happened to work.
Yup, that sucks; yup, it was subject to failure later. But it seems like conceit on your part to imagine that through your sheer intellect and commitment to excellence that you could somehow divine your way out of this mess. And I had the luxury of working on these projects without any pressure or schedule. I can’t imagine the horror of having to figure out some of these API black holes with a boss breathing down my neck to “just get the damn thing to work!”
Years ago I gave up Win32 and switched to using wxWidgets. Programming became fun again.
Hah, hah, hah. The only people who can answer “yes” to all of your questions either have written only trivial programs, have written authored all of the software libraries they are using, or are liars.
You know, I think that’s pretty much true. 🙂 I was actually just having a conversation about that very fact with a friend of mine who’d read this post, and the only people who could answer those questions with an unqualified “yes” would be people who believed they knew everything–the exact mistake the article warns against. 🙂 However, in the domain that I work in, I can answer “yes” to most of those questions, not in the sense of “I know everything about these”, but in the sense of “I know enough about these to really get my work done well.” So it’s not a black-and-white proposition, it’s something one can know more or less about.
As far as undocumented APIs go, I totally agree with you that no matter how intelligent or committed you were, there’s no way you will ever become an excellent programmer with them. If the understanding isn’t there to be had, you can’t know what you’re doing. 🙂
And hahaha, yeah, I wrote an article a while back that used the complexity of the Win32 API as an example of why Windows was fundamentally flawed: http://www.codesimplicity.com/post/simplicity-and-security/. I’m really glad for you that you got out of that world. 🙂
-Max
I’m sorry for the diversion. This was just my interpretation of the “stack trace” for this particular observation of yours.
That’s totally fine, I just wanted to make sure that we didn’t get too off-topic in the comment threads. 🙂
-Max
You, sir, are an idiot. I happen to know a professional programmer who writes programs that run extensive, highly complex assembly lines. He most certainly answers “yes” to all of those questions, as well as money others.
There are some who are not content with mediocrity, even if you may be.
I wonder if everyone has the same idea about what “no idea what they are doing” means. Yes, in complex systems, even experienced programmers probably have to struggle to get things working to meet a deadline. Often, you just call an API function that seems to do what you need, and hope for the best.
But for me, “no idea what they are doing” means really typing in words they’ve seen elsewhere, like names of variables, without actually realizing that the words were variables or that if you want to access a variable in a different function, you have to get the data there somehow (adding it to the parameters of the function, for example). They’re like “The word ‘color’ was used in that other function, so I’ll just use the same word in this function, and hope for the best. Oh no! To the compiler complains again! Why does it say ‘undefined symbol’? I’ve seen the same word used elsewhere, and it doesn’t throw an error there! This has to be a bug in the compiler.” Later: “I’ll just try to add some parentheses. I’ve seen parentheses used in a lot of other places!”
I wonder if everyone has the same idea about what “no idea what they are doing” means.
Ah, you know, I thought about that a bit myself when I was writing the article. 🙂
But for me, “no idea what they are doing” means…
Hahahahahahahahaha!! Yeah, that’s pretty much what I had in mind, too!
I think it’s relative, though, too–after all, there’s so much about a programming system that one can have “absolutely no idea” about, that a total lack of knowledge in one area effectively means that one has absolutely no idea what one is truly doing.
-Max
When the company I worked for at the time decided that they wanted me to port an in-house application to Foxpro to compare it against the previous database engine that they had been using I therefore read the entire reference manual (including descriptions of all statements and functions) from cover to cover and was able to remember most of it for several months.
Wow, that’s awesome. 🙂
-Max
Warning: results not typical. 😉
I’ve always called that ‘cargo cult programming’: writing code you’ve seen work somewhere else, without actually knowing or caring why. I’ve seen it a few times: over-engineered legacy code written by someone who clearly knew of interesting approaches but had no idea how to build them in a simple/usable way; a systems programmer who insists on writing Python like it were C. I think it’s a symptom of people who are interested only in getting something finished, not in the field unto itself; programming is just a means to an end (which is often a paycheck). Most of the secretaries using Word and bosses using Excel don’t really know what they’re doing, either; just that something similar has worked before.
That 90% figure crops up a lot. Perhaps a corollary to Sturgeon’s Law is in order.
Yeah, I’ve heard people call it that as well, although I think that sometimes that term isn’t as well-defined (or as well-known) as it could be. 🙂
And yeah, it’s often a symptom of that mentality where people don’t care about the craft of the field.
I’ve seen an 80% figure for the difference between “good” programmers and “normal” programmers–that’s the one I see the most often. My experience was a lot more severe, though–not just that the 20% “weren’t quite as good”, but that they only appeared to be competent at all and were in fact largely in the dark as to what they were doing, to some degree or another.
-Max
Sure,
The vast majority (90% or more) of programmers have absolutely no idea what they are doing.
but,
The other 10% are not doing anything interesting.
Programming operates at the limit of human cognition. Programmers will never know what they are doing.
jjb
No, that’s not true. I know quite a few people who would fall into the 10%, and they are almost always working on really interesting projects.
-Max
It simply isn’t feasable for any programmer to look at the code behind every API down through various libc’s, taking into account compiler optimizations for specific processors. Thus, no programmer can be confident about what’s going on with their code unless they’re programming to the metal in assembly. Even then you couldn’t be confident about the microcode on specific revisions of CPUs implementing the target ISA (your crystal ball, future directions point).
Civil engineers aren’t generally concerned with quantum physics, somewhere you have to draw a line. With software, that line is usually under the API’s you’re using in a given project. If the tests run and the code works, programmers move on to the next problem.
It’s inacurate to state that 90% of programmers don’t know what they’re doing. It’s more accurate to say that a small percentage of programmers don’t have a clue (like the guy who wrote the comment plugin you’re using — requires javascript) and the majority don’t have the time to fully study every single API they’ll use in their career. What’s more, programmers who do dive into the layers of abstraction upon which they depend will likely be disgusted by the ugly hacks lurking beneath the surface.
I didn’t say the programmer has to know everything down to that level, just that they should understand the documentation of the functions, keywords, and symbols that they’re using.
If you don’t have the time to study the parts of an API that you will be using…that’s like saying that you don’t have time to learn how to drive a car before you get behind the wheel of one.
-Max
I’m curious as to your proposed alternative to the comments section being written in Javascript. Do you expect all of the comments to be sent from the server in one static request? Do you expect the page to be a completely static document, with no interactivity, without any calls from the client except for the form POST request?
interesting article.
it applies to all fields, though. coming from print design and production, it is RARE to see any graphic designers that haven’t worked in printing companies design something that actually prints right. so – 90% of print designers haven’t a clue either.
kind of frightening when it comes to thinking about brain surgeons and doctors like this though, isn’t it?
That’s pretty interesting, to know about the other fields. I think I’ve seen it happen to some degree or another with doctors and so forth, since sometimes I’ll have friends who talk to one doctor for years, only to switch to another doctor and suddenly have their problem resolved immediately. I suppose it’s somewhat less likely with doctors, but I’d have to guess it still happens.
-Max
Yikes to that. I’ve never trusted doctors. It’s irksome to think they can profit from prolonging your illnesses. And as little as they may know about what they’re doing, they know that their patients know much, much less.
I can answer yes to the first three of those seven questions. Is that good enough, or am I a terrible programmer?
Well, that’s better than most people can say. But I’d assume that’s not “yes absolutely” but more “yes, enough to be good at what I’m doing”, since it would probably be impossible to answer “yes absolutely” to any of the questions.
I think that a “yes” to the other four questions would make you even better than you are, though.
-Max
I programmed for many years, starting in 1981. Yes, I did BASIC and Fortran and all that stuff. Then got really into dBase 2,3 and 4, then Foxpro and finally Cold Fusion (Allaire, then). I learned from the get-go to program as modular a fashion as possible. Reuse code… and keep it compact and simple. When OOP came about, I was already for that style change. Again, I used simple modules to do the work.
One thing I noticed about some Cold Fusion programmers is that they would write code that was fantastically convoluted to the point of absurdity. I finally figured out it was done, on purpose, to achieve: 1) job security; no one can figure out the code 2) impress the management as to your degree of cleverness… A class act that says: “The more complex the code then the better it is”. You can write code tighter than anyone. These type of coders usually get their asses handed to them when there is a major revision in the overall corporate code base or strategy.
I have proven time and time again that a longer, simpler approach always works best. Usually runs as fast or faster than any super-compact code. Besides, if you code kind of loose, then no proposed revision will make you panic. You already have ‘plug-in’ capacity for change.
I think a lot of the problem is that not all programmers are well taught to do the analysis part on a project. To be able to not only design for today but to leave the code base very open to alterations. Anticipate future needs.
It’s an educational problem. Programmers are taught the tools but not always the best application of those tools. That takes time and getting into some serious self criticism.
That’s a remarkable story (mostly because few programmers are that adaptable to new developments, even though they should be), and I think you’re totally right about everything you said.
The “I have to be complex to prove that I know my stuff” bit is actually something I wrote a post about, a couple years ago: http://www.codesimplicity.com/post/complexity-is-a-prison/ — I did it in such a fashion as to attempt to address those sorts of people directly, so maybe they would see another side of things.
I agree that there’s a severe educational problem about analysis and the best application of the tools. That’s actually one reason why I’m writing a book about the basic laws of software design–I think that it’s not the teachers that are necessarily at fault, but actually a total lack of a simple manual that would describe what people should actually be thinking and doing when they design software–what the fundamental laws are, and so forth.
-Max
I have no doubt that you are much better than the average programmer, but too much simplicity often leads to solutions that have problems with the corner cases that could easily be addressed with a slightly more complex solution.
That being said, I’m thinking more so of the perils of letting a junior developer implement a “simple” solution in production code that ends up not addressing most of the issues we would run into, whereas an experienced programmer like you would make good tradeoffs. Balance is important.
Do you know as much as possible about every single word and symbol on every page of code you’re writing?
– IF I don’t it makes me feel unsure of my program. I don’t like that feeling, so I only write what I understand.
Did you read and completely understand the documentation of every single function you’re using?
– Read? yes. Understand? Sometimes. Occasionally I’ll see a function whose use is part of a larger mechanism that I am either unfamiliar with, or isn’t immediately apparent to be working that way.
Do you have an excellent grasp of the fundamental principles of software development–such a good grasp that you could explain them flawlessly to novice programmers at your organization?
– Depending on the preconceived notions of the novice programmer. Some can be hard to overcome and act as a sort of mental pollution.
Do you understand how each component of the computer functions, and how they all work together?
– In some cases, down the the electrical flow of certain chips.
Do you understand the history of computers, and where they’re going in the future, so that you can understand how your code will function on the computers that will be built in the future?
– Yes for the past. But as toward their future I am still uncertain, but would like to think that I am taking part in influencing them in their direction.
Do you know the history of programming languages, so that you can understand how the language you’re using evolved and why it works like it does?
– Yes, some more intimately than others.
Do you understand other programming languages, other methods of programming, and other types of computers than the one you’re using, so that you know what the actual best tool for each job is?
– I understand enough about other programming languages to decide effectively (if only through research) what tool is best for the job. I don’t feel I have to know EVERY language, but must know enough about different types/pros/cons of languages to make that decision.
Those are great answers; I’d guess you’re a pretty darn good programmer. 🙂
-Max
By reading a great book on “Intuition” (°) I learned how and why humans like simplicity.
However, simplicity like intuition can have its powers and perils.
Therefore I ask: why should I learn and try to know more than what I need?
I now spent at least two days to figure how I could benefit from “LINQ to SQL” in my projects.
This was funny but quite useless: I just went back to use the good old “ODBC connection strings” and manipulating the data as I do already since 10 years.
And I still don’t see a use case applicable to my current activities.
Why did I want to learn, why was I so afraid to be ignorant?
It is true that the not-yet-read books on my bookshelf are telling a lot about my curiosity.
But those IT books that I didn’t read and that I throwed away to the garbage bin, they just show how stupid I can be sometimes when I want to inflate my head without purpose.
Maybe to keep code simple, we need to protect our head from useless information.
(°)
Intuition: Its Powers and Perils
Yale University Press, 2002
David G. Myers
Well…I understand what you’re saying, but I pretty much disagree with you. It sounds like you’re arguing for ignorance because some new technologies are hard to understand. You can’t have that attitude and remain a good programmer. It would be like people who refused to drive cars because they were familiar with carriages–it may be your option to make that decision, but it’s just going to inconvenience you, because cars really are much better, even though they do take some learning. In the case of being a programmer, it’s even worse, though, because being tied only to the technologies you already know is going to cause your code to degrade more and more over time, and the people who will be affected are your users, not just you.
-Max
The problem with not constantly updating your self new programming technologies is that it will eventually seem like alot to digest thus leading to more problems for you in the future , i advice any programmer to atleast learn some new technology now and then as technology is changing at a really fast pace
Yeah, that’s totally true. 🙂
-Max
Never hire any programmer who hasn’t written at least one compiler for fun.
Hahahaha! Well, I don’t know if I’d go quite THAT far! 🙂
-Max
Coming to the party last here… 🙂
To me it’s a simple quote I reference often:
‘Programming is hard’
Most of programmer do this because they want to save their work and dont give the libraries good info,and those companies exeggerates their case in order to obtai.
I think it’s more due to the nature of programming. Just like lives, so easily become chaos. Maybe a guideline to programmers also qualifies a title – ‘guideline to life’
Hi, I think your right about knowing what to do! It’s the most important thing every programmer must know. I’m a professional programmer and i also teach programming and i see this a lot, people try to code when they actually have no idea what they are going to do!
Sentence like “know what every line of code does” or “think like a compiler” are the sentences i use over and over to tell my students what is the right pass to become a good programmer.
I think it is time for me to remind myself about that!! Recently i discovered i’m not as good programmer as i though i was. I can say yes to 70% of your questions but i still need to know more. Just few weeks a go i was working on an android application and in order to make my code work i combined many advanced software engineering concepts and design patterns all together. I did that and my code became so complex and buggy! Then i found one particular method in API documentation that freed me of all of that!
BTW thank you for your great posts.
Yeah. 🙂 I’m not sure that “think like a compiler” is always the right thing, unless you are teaching optimization (which I think we teach a bit too much of in programming these days–people often come out of school with an overly-intense desire to prematurely optimize) but it can be good sometimes.
I think it’s always good to realize in any field that there’s more to know. 🙂 I know there are lots of areas of software development and computing where I don’t know anything and I’d love to learn more, myself.
-Max
[…] it might be that there are many things to learn more about, such as the fundamentals listed in Why Programmers Suck. Go through each piece you need to learn until you really understand it, then move on to the next […]
The problem is that these kinds of guys now also slowly start to infiltrate the ranks of PROGRAMMING LANGUAGE DESIGNERS!
Or what’s the explanation for the fact that ECMAScript now has, like, 9823472349 different half-assed approaches at async programming (WebWorkers, Promises, Generators, async, blablabla)? It’s increasing complexity and confusion. And that’s not even counting the 234894932 jQuery-like kludges that try to “emulate” this.
I just wanted to read up on the newest developments on the “async” front out of “boredom” (read: keeping up with the cutting edge), and I’ve decided for now that I’ll just stay with callbacks 🙂
Very good article. Opened my eyes. I am doing exactly what the author tells not to do. After 2 years of programming experience, I am stuck on every challenge I tackle. Time to start reading!
[…] understanding, which I’ve gone over quite a bit in several articles, most particularly “Why Programmers Suck.” It really is the key point which makes or breaks a programmer. There are other skills that […]
[…] the idea that computers suck by: Max […]
I have a professional programmer since 1975, (I am just about ready for retirement thank goodness.) I have seen the industry change dramatically many times and go through hundreds of “FASHIONS”, of which none delivered the claimed benefits.
Programming computers is very inefficient, its far cheaper to build a house than build a moderate size computer application. How can that be so? It sounds ridiculous of the surface.
To me the programming tools are way too complicated. The software companies are driven by features (sales), rather than simplicity and usability. Microsoft’s frameworks have many thousands of objects. So many that no one could understand or even remember them. I am sure that many are hardly ever used by anyone, and should not even be in the framework. This vastness of functionality is overwhelming, unnecessary and makes the framework difficult to learn. There are often many different ways to achieve the same result.
The university courses teach impractical theories to their students. (OOP is a bit like communism, its a nice idea but it doesn’t work. I sometimes think that we should set up a chapel at work dedicated to OOP so the devotees can pray during breaks.) OOP is both slower to build applications and harder to maintain them. (I know many people will disagree.) Many programmers would can me a heretic for saying this.
The rapid expansion, over the last 35 years, in the number of computer projects has resulted in inexperienced personal being promoted prematurely. It also means that junior team members don’t have senior team members or leaders to be guided by. The rapid and continuous change in technology also has the same effect. If the technology is new, or even greatly changed, then there is no one with experience to learn from, and advice from someone with general experience isn’t accepted.
The companies creating the programming languages also suffer from both of the above problems as well. If your a programmer better not take the risk and venture into new version of development software or features that aren’t commonly mainstream.
You don’t have to be super intelligent to program computers, but you do have to be diligent, tenacious and thorough. Many people in the industry don’t have a the necessary aptitude and attitude for the job.
Lastly there is a definite lack of respect for general experience. All programming languages are fundamentally the same. The majority of experience in one language is transferable to a new language. Companies should think more about the long term and be less focused on particular languages.
You make some excellent arguments, its like you took the words right of my head.
I’m at a point where I feel like none of the programming languages are up to standard, their bad tooling, monopolized control, bug laden history track and even beefed up paradigms just makes you want to chop wood instead. On the twist side open source is just as stuck up as well with horrible compilers, tooling, toxic dev teams, inability to change spells recipe for doom. I mean just look at Python amazing language, only quirk is the dev is a stuck up closet freak who refuses to get rid of gil in a post x86 world.
Luckily there is still C or computing in general would be royally screwed.
As I get more and more experience I am finding your observations here truer and truer. Actually, I initially agreed with you, but I didn’t get it at the gut level of intuition.
As a trainer, I always maintain a strong confidence that anyone can be taught anything. But to successfully teach people, it’s necessary to recognize where their current level of understanding IS. That’s actually the most common error that any trainer or teacher makes: to fail to correctly estimate the entry point for teaching their students.
A rule of thumb is that the entry point is ALWAYS lower (more basic, simpler, more fundamental) than you think it is. Except maybe one student out of a thousand. Everyone overestimates it, I think partly because they don’t want to seem condescending.
On a related note—the most popular talk I ever saw at a Linux conference was “Process and Job Control in Linux.” Basic (foundational, fundamental) knowledge. Utterly packed room. At least four other talks occurring at the same time, and over 50% of attendees (probably over 80%) were crammed into the room for that talk.
[…] [1]. https://www.codesimplicity.com/post/why-programmers-suck/ […]
I made it only a handful of classes deep into programming and that was enough to understand the reality of that industry. I had an aptitude for it, but I didn’t enjoy doing it in that, “let’s write a few thousand lines of code for hours on end,” sort of way. I couldn’t see myself sitting at a desk doing that most days for a living. Then when I realized that the competency of my peers was going to be the same or worse as it was in anything else I’d have ever done, that was the final nail in the coffin for this little fleeting dream.
I can’t talk a lot of shit about programmers, how terrible the average program is written and how poor their overall practices are, or how bad management is, etc., having not officially learned to code or worked in the industry, and that’s okay. Simply knowing it exists and having dodged it is enough for me. Guaranteed I’d have been a job hopping son of a bitch that ended up going to the oilfield to do something I don’t want to do for more money than anybody else will pay. So I took a shortcut. The only drawback, if I had finished the degree I could have made more money in the oilfield doing something I don’t want to do in exchange for a lot less work.
Ah, 2009, the good old years.