There’s only one problem with this whole debate: what is privacy, and why would anybody want it? This is rarely defined–most people just seem to assume that “everybody knows” that privacy is, so why would it have to be explained?

Well, I’m not a big fan of “everybody knows.” And in fact, it turns out that privacy actually means two different things, which many people use interchangeably without specifying what they’re actually talking about. So to help clear up some of the debate online, and to hopefully shed some light on how it can all be resolved, here are some clear definitions and discussions of what privacy is, and why people would want it.

Privacy of Space

The first type of privacy is “privacy of space”. This is the ability to control who does and does not enter a particular physical space, probably because you’re in the space and you don’t want certain others in that space. “Enter the space” in that definition includes any method of being able to perceive the space–so, for example, if somebody stands outside the door with their ear pressed to it, they’re violating your privacy. If somebody installs a camera in your room without your consent, they’re violating your privacy.

This form of privacy is not metaphorical. It does not apply to anything other than physical space. It literally means, “I do or do not want you to be perceiving this physical location, and I have the choice and ability to control that.”

The most common reason that we want this form of privacy is that we want to protect somebody or something from harm, most commonly ourselves. This harm can be minor (we don’t want to be annoyed by people walking through our house all the time), it can be purely social (we close the door when we go to the bathroom because we know others don’t want to perceive us going to the bathroom, and we may also not want to be perceived in such a state), or it can be extreme (a man with a mask and a chainsaw should not be in my closet).

One interesting thing about this form of privacy is that we don’t usually consider animals, plants, or material objects to be capable of violating it, even if they enter a space without our permission. It might be annoying if the cat comes in the room when you don’t want it to, but you’re not going to complain that the cat is “violating your privacy”, right?

So, when it comes to computer programs, this is not the form of privacy we’re talking about, since we don’t consider that a computer program being in the same room with us is a violation of our privacy of space. My word processor is not violating my physical privacy of space, even though it’s “in the room” with me, because it does not, itself, perceive. The only exception would be a computer program that was transmitting perceptions (sound or sight) to some location that we didn’t want to send it to–that would be a privacy violation, because could perceive our space through it when we didn’t want them to. When it comes to that sort of privacy, violations are pretty pretty cut-and-dry. If a computer program sends perceptions of my space anywhere without my permission, it is absolutely violating my privacy, it’s not useful to me, and it should stop immediately.

But on the Internet, that’s not usually the type of privacy we’re talking about.

Privacy of Information

The second type of privacy is “privacy of information.” This is the ability to control who knows certain things. When we talk about computer programs and the Internet, this is the most common type of privacy we’re talking about.

So why would somebody want privacy of information? Is it just because they’re doing something that they want to hide from others? Is it just for committing crimes or for hiding harmful acts? Well, sometimes it is, yes. There are many people who use the concept of “privacy” to protect themselves from the law or the moral rejection of others. It is probably because of these individuals that the concept of privacy is a muddy subject–as long as it’s unclear quite what “privacy” is, it’s much easier for those who have have committed harmful acts to invoke “privacy” as a defense.

But is that the only reason that somebody would want privacy of information? What about a normal person, who isn’t doing anything harmful–would they ever want to certain information private?

Well, there is absolutely a rational reason that people would want privacy of information, and interestingly, it’s the same reason that people want privacy of space:

An individual or group desires privacy of information because they believe that other people knowing that information could or would be more harmful than them not knowing it.

Here’s a very straightforward example: I consider that a criminal knowing my credit card number would be harmful–far more harmful than them not knowing it.

In certain countries, the fact that I read a certain website or talked to certain people on the Internet could get me killed or put in jail. So, in that situation, other people knowing my browser history could be very harmful, no question about it.

Of course, if one kept everything private, one could not live. If you pay for a piece of candy with a quarter, the person receiving that quarter now knows that you had a quarter. They may know that you kept it in a waller, or that you pulled it out of your pants. They probably know what you look like, if you’re not wearing a mask. They most likely also know that you have five fingers, and that you were in their store at a certain time. In short, no matter what you do, in order to live, you must exchange information with other people. The more things you do, the more information you will have to exchange.

In fact, usually, the more information that others know about you, the more helpful they can be. The bank knows all the transactions that I made, so they can help me by creating an online system that shows me my transactions and lets me search them. That information can be seen by bank employees, but I don’t consider that to be potentially harmful enough to outweigh the obvious benefits of the bank having it.

The web browsers that I use know my passwords to certain sites, so they can help me by putting those passwords into the box, saving me some typing. Potentially, somebody could steal that information from my computer, but the chance of that happening is small enough, and the benefit is significant enough, so I consider it acceptable to save my passwords in the browser.

The examples like this go on and on–the appropriate use of information is extremely beneficial. The inappropriate use is what’s harmful.

So who decides what what’s an appropriate use and what’s an inappropriate use? What information should be sent and stored, and what information should be kept private? Well, these are the fundamental questions being asked when people debate privacy issues–who gets to choose whether my knowledge becomes somebody else’s knowledge? Should I be asked before my information is sent, or should I just be given the option to opt-out and delete the information? Is there some information that should never be sent? What information is more important to keep private than other information?

Though this is all far less cut-and-dry than “privacy of space” issues, these questions can generally be answered by the “help vs harm” equation. The basic sort of questions one might want to ask would be:

• Will sending and storing this information harm any users, immediately or potentially? (Remember, “potentially” is pretty broad–what happens if somebody with bad intentions steals that information from you? What happens if somebody buys your company and decides to use that information in a way that you think is bad?)
• Would it help your users more than harm them to take this information?
• Taking all the above into account, should sending this information be optional? (This is largely determined by how broadly it could be harmful to collect the information.)
• If sending the information is optional, should it be opt-out or opt-in? (That is, should it automatically be on, and people have to turn it off if they don’t want to send the info, or should it be off and people have to choose to turn it on?)
• If it’s opt-in, will the feature still be helpful to enough of your users to justify implementing it?

There are some people who will claim that no information should ever be sent or stored about the user, that all privacy options should always be opt-in, and that all information is so potentially harmful that no debate about this can be accepted. That is, frankly, a ridiculous proposition. It’s so obviously untrue that there’s almost no way to argue with it, because it’s such a shocking irrationality. Just like the fact that somehow, liquids could harm somebody (so you can’t bring liquids on an airplane in the USA) it’s true that there are situations in which almost any piece of information could be dangerous. That doesn’t mean that all information is dangerous, though.

My martial artist friends have frequently joked that they shouldn’t be allowed to bring any object on an airplane, because they could kill somebody with any of them. Similarly, given almost any piece of information, somebody could do something harmful with it, somewhere, at some point. If I know you have a quarter in your pocket, I’m sure there’s some situation in which I could use that information to get you in some serious trouble. But that doesn’t make that information realistically harmful, even potentially.

Even the idea of “every single piece of information should be opt-in” is ridiculous. Do you want the web browser to ask you, “May I send this page your IP address?” every time you load a web page? Well, if you’re a spy in a hostile country, maybe you do. But if you’re like most people, that would probably just annoy you–you’d stop using that web browser and switch to another one. And if you are a spy or a resistance fighter, then you probably know how to use Tor to avoid being tracked.

So when we’re talking about privacy, it’s not an issue of “in some incredibly unlikely situation, this information could be very harmful,” it’s an issue of balancing help vs. harm in real-world situations. Real-world situations can be pretty strange and unexpected, but they at least are real, and can be balanced and talked about. Doing so, you can make good decisions about how to protect your users’ privacy–how much information to take, how you inform them about the information you’re taking, and what you do with that information when you have it.

So no, this is not a casual issue or something that we should brush-off and just ignore the dangerous implications of, but it’s also not an extreme unsolvable situation where we have to decide to keep everything private because we can’t make up our minds about it. Privacy is simply something that we should be able to analyze factually, based on real-world situations and data, and come to some practical and useful decision about.

-Max

Comments: 22

What I didn’t know at the time was why programmers did this. It was obvious that they did do it, but why would the software development industry produce so many crazy, complex masses of unreadable code? Why did it keep happening, even when developers should have learned their lesson after their first bad experience? What was it that made programmers not just make bad code, but keep on making bad code?

Well, this was a mystery, but I didn’t worry too much about it at first. Just the revelation that “bad programs are caused entirely by bad programmers”, as simple and obvious as it may seem, was enough to fuel an entire investigation and study into the field of programming, one which had some pretty good results (that’s mostly what I’ve written about on this blog, and it’s also the subject of a book that’s in development). The problem had been defined (bad programmers who create complexity), it seemingly had a solution (describe laws of software design that would prevent this), and that was enough for me.

But it still baffled me that the world’s universities, technical schools, and training programs could turn out such terrible programmers, even with all of the decades of advancement in software development techniques. Sure, a lot of the principles of software design hadn’t been codified, but a lot of good advice was floating around, a lot of it very common. Even if people hadn’t gone to school, didn’t they read any of this advice?

Well, the truth was beyond my imagination, and it took almost five years of working on the Bugzilla Project with a vast number of separate contributors until one day I suddenly realized an appalling fact:

The vast majority (90% or more) of programmers have absolutely no idea what they are doing.

It’s not that they haven’t read about software design (though they likely haven’t). It’s not that the programming languages are too complex (though they are). It’s that the vast majority of programmers didn’t have the first clue what they were really doing. They were just mimicking the mistakes of other programmers–copying code and typing more-or-less meaningless incantations at the machine in the hope that it would behave like they wanted, without any real understanding of the mechanics of the computer, the principles of software design, or the meanings of each individual word and symbol they were typing into the computer.

That is a bold, shocking, and offensive statement, but it has held up in my experience. I have personally reviewed and given feedback on the code of scores of programmers. I have read the code of many others. I have talked to many, many programmers about software development, and I’ve read the writings of hundreds of developers. The number of programmers who really understand what they are doing comprise only about 10% of all the programmers I’ve ever talked to, worked with, or heard about.

In open source, we get the cream of the crop–people who want to program in their spare time. And even then, I’d say only about 20% of open source programmers have a really good handle on what they are doing.

So why is this? What’s the problem? How could there be so many people working in this field who have absolutely no clue what they’re doing?

Well, that sounds a bit like they’re somehow “stupid.” But what is stupidity? People are not stupid simply for not knowing something. There’s a lot of stuff that everybody doesn’t know. That doesn’t make them stupid. That may make them ignorant about certain things, but it doesn’t make them stupid. No, stupidity, real stupidity, is not knowing that you don’t know. Stupid people think they know something when they don’t, or they have no idea that there is something more to know.

This sort of stupidity is something that can be found in nearly every field, and software development is no exception. Many programmers simply don’t know that there could be laws or general guidelines for software development, and so they don’t even go looking for them. At many software companies, there’s no attempt to improve developers’ understanding of the programming language they’re using–perhaps simply because they think that the programmers must “already know it if they were hired to do it”.

Unfortunately, it’s particularly harmful to have this sort of mindset in software development, because there is so much to know if you really want to be good. Anybody who thinks they already know everything (or who has a “blind spot” where they can’t see that there’s more to learn) is having their ability to produce excellent code crippled by a lack of knowledge–knowledge they don’t even know exists and that they don’t even know they lack.

No matter how much you know, there is almost always more to know about any field, and computer programming is no exception. So it’s always wrong to think you know everything.

Sometimes it’s hard to figure out what one should be learning about, though. There’s so much data, where does one start? Well, to help you out, I’ve come up with a few questions you can ask yourself or others to help figure out what areas might need more study:

• Do you know as much as possible about every single word and symbol on every page of code you’re writing?
• Did you read and completely understand the documentation of every single function you’re using?
• Do you have an excellent grasp of the fundamental principles of software development–such a good grasp that you could explain them flawlessly to novice programmers at your organization?
• Do you understand how each component of the computer functions, and how they all work together?
• Do you understand the history of computers, and where they’re going in the future, so that you can understand how your code will function on the computers that will be built in the future?
• Do you know the history of programming languages, so that you can understand how the language you’re using evolved and why it works like it does?
• Do you understand other programming languages, other methods of programming, and other types of computers than the one you’re using, so that you know what the actual best tool for each job is?

From top to bottom, those are the most important things for any programmer to know about the code they’re writing. If you can truthfully answer “yes” to all those questions, then you are an excellent programmer.

It may seem like an overwhelming list. “Wow, the documentation for every single function? Reading that is going to take too long!” Well, you know what else takes a long time? Becoming a good programmer if you don’t read the documentation. You know how long it takes? Forever, because it never happens.

You will never become a good programmer simply by copying other people’s code and praying that it works right for you. But even more importantly, investing time into learning is what it takes to become good. Taking the time now will make you a much faster programmer later. If you spend a lot of time reading up on stuff for the first three months that you’re learning a new technology, you’ll probably be 10 times faster with it for the next 10 years than if you’d just dived into it and then never read anything at all.

I do want to put a certain limiter on that, though–you can’t just read for three months and expect to become a good programmer. First of all, that’s just too boring–nobody wants to just study theory for three months and not get any actual practice in. Very few people would keep up with that for long enough to become programmers at all, let alone good programmers. So I want to point out that understanding comes also from practice, not just from study. But without the study, understanding may never come. So it’s important to balance both the study and the practice of programming.

This is not an attack on any programmer that I’ve worked with personally, or even an attack on any individual programmer at all. I admire almost every programmer I’ve ever known, as a person, and I expect I’d admire the rest were I to meet them, as well. Instead, this is an open invitation to all programmers to open your mind to the thought that there might always be more to know, that both knowledge and practice are the key to skill, and that it’s not shameful at all to not know something–as long as you know that you don’t know it, and take the time to learn it when necessary.

-Max

Comments: 47

It’s not anything complicated, and it’s not something that’s hard to know. It’s not something that you can only do if you’re born with a special talent or a “magical ability to program well.” There is nothing about the nature of the individual that determines whether or not they will become an excellent programmer or a poor one.

There is only one, singular fact:

The better you understand what you are doing, the better you will do it.

Rockstar programmers understand what they are doing far, far better than average or mediocre programmers. And that is it.

All you have to do in order to become an excellent programmer is fully understand what you are doing.

Some may say that they already understand everything. The test is whether or not they can apply it. Do they produce beautifully architected systems that are a joy to maintain? Do they plow through programming problems at a rate almost unimaginable to the average programmer? Do they explain everything clearly and in simple concepts when they are asked for help? Then they are an excellent programmer, and they understand things well.

However, far more commonly than believing that they “know it all”, many programmers (including myself) often feel as though they are locked in an epic battle with an overwhelming sea of information. There is so much to know that one could literally spend the rest of his life studying and still come out with only 90% of all the data there is to know about computers.

The secret weapon in the epic battle, the Excalibur of computing knowledge, is understanding. The better that you understand the most fundamental level of your field, the easier it will be to learn the next level. The better you understand that level, the easier the next one after that will be, and so on. Then once you’ve gone from the most fundamental simplicities to the highest complexities, you can start all over again and find amazingly that there is so much more to know at the very, very bottom.

It seems almost too simple to be true, but it is in fact the case. The path to becoming an excellent programmer is simply full and complete understanding, starting with a profound grasp of the basics and working up to a solid control of the most advanced data available.

I won’t lie to you–it sometimes is a long path. But it is worthwhile. And at the end of it, you may find yourself suddenly the amazing senior engineer who everyone comes to for advice. You may be the incredible programmer who solves everything and is admired by all his peers. You might even come out a “rock star” with millions of dollars and a fantastically successful product. Who knows? I can’t tell you what to do or what to become. I can only point out some information that I’ve found to be truthful and valuable. What you do with it is up to you.

-Max

Comments: 12

I can solve this problem the right way.

Whatever the problem is, there’s always a right way to solve it. The right way can be known, and it can be implemented. The only valid reason ever to not implement something the right way is lack of resources. However, you should always consider that the right way does exist, you are able to solve the problem the right way, and that given enough resources, you would solve the problem the right way.

The “right way” usually means “the way that accounts for all reasonably possible future occurrences, even unknown and unimaginable occurrences.”

A bridge that could stand up to any reasonably possible environmental condition or any reasonably possible amount of traffic without constant maintenance would be built the “right way.”

Software code that maintained its simplicity while providing the flexibility needed for reasonably possible future enhancements would be designed the “right way.”

There are lots of invalid reasons for not solving a problem the right way:

• I don’t know the right way. Often this just requires more understanding or study, to figure out the right way. When I run into this situation, I walk way from the problem for a while, and then often I’ll come up with the solution when I’m just out walking, or the next day when I come back to it. I try not to compromise on something that isn’t the right way just because I don’t know what the right way is yet.
• The group cannot agree on what the right way would be. Sometimes a group of people have argued about what would be the “right way” and the subject has gotten very confused. Groups are not very good at making decisions. As we all know, you don’t design software by committee, and I suspect that “design by committee” in other fields of engineering is just as bad. The solution here is to assign an experienced and trusted engineer who understands the basic laws of the subject you’re working in to determine the right way by himself or herself, probably after carefully studying the existing arguments and collecting relevant information, following standard, valid engineering procedures.
• I am too lazy/tired/hungry/discombobulated to do this the right way, right now. This happens to everybody from time to time. It’s 1 in the morning, you’ve been working on the project for 15 hours straight, and you just need the damn thing to work, right now! Give it a rest, though, and come back later. The world isn’t ending, and the problem will still be here and solvable later. Go to sleep, go eat something, take a walk–do whatever it takes to get into a mental space where you’re willing to solve the problem the right way, and then come back. If you’re in a state where you can’t solve the problem the right way, then it’s really time to take a break. You’re not being delinquent in your duties if you do so–you’re actually correctly taking responsibility for the success of the project by saying “this needs to be done right, and the way to do it right, right now, is to take a break and come back later”.

Mostly, it all just takes the constant and continual belief in yourself that you can solve the problem the right way.

-Max

Comments: 37

Well, some of it’s really obvious. You press a button and the program takes 10 minutes to respond. That sucks pretty bad. You get 100 complaints a week about the UI of a particular page–okay, so that sucks.

Usually there are one or two HUGE things that really suck, and they’re really obvious–those are the things to focus on first, even if they require a tremendous amount of work. For example, before Bugzilla 3.0, Bugzilla had to compile every single library and the entire script it was about to run, every time you loaded a page. This added several seconds to each page load, on slower machines, and at least 1 second on faster machines. So performance was one big obvious thing that sucked about Bugzilla. But even more importantly, the code of Bugzilla sucked. It was being read by everybody–because people were frequently customizing Bugzilla at their company–and it was an unreadable, garbled mess.

Thankfully, both of those problems had the same solution. The performance problem was solved by allowing people to run Bugzilla in a way that would pre-compile all our scripts when the web server started, instead of every time somebody loads a page. And to enable that pre-compiling, we had to do enormous amounts of refactoring. So, we actually ended up handling our performance problem by handling our code problem.

However, it took four major releases (Bugzilla 2.18, 2.20, 2.22, and finally 3.0) to get all this done! We fixed a lot of little issues for each release along the way, too, so each release really did suck less than the previous one. But handling the major issues was a tremendous effort–it wasn’t just something we could code up in one night and have it be done with.

I think sometimes the big issues in a software project don’t get handled because they do require that much work to fix. That doesn’t mean you can ignore them, though. It just means that you have to plan for a long project, and figure out how you can keep getting releases out in the mean time.

After all that was fixed, then we could turn our attention elsewhere, and wow! It turned out that elsewhere, there were still a bunch of things that sucked! All of the sudden, there were a new batch of “totally obvious” things to fix–things that had been there all the time, but were just overshadowed by the previous set of “totally obvious” things.

Now, we could have just gone on like this forever–fixing one set of “totally obvious” problems and then going on to the next set of “totally obvious” problems. But we ran into an issue–what happens when suddenly, you get to the point where there there are fifty “totally obvious” things that need fixing, and you can’t get them all done for one release? Well, that’s when you suddenly need some method of prioritizing what you’re going to fix.

For the Bugzilla Project, there were two things that we did that really helped us prioritize: the Bugzilla Survey and the Bugzilla Usability Study.

With the survey, the most important part was allowing people to respond in free-form text, to questions asked to them personally. That is, I sent the questions from me personally to Bugzilla administrators personally, often customizing the message for their exact situation. And there were no multiple-choice questions, only questions that prompted them to tell us what was bothering them and what they wanted to see. They were actually really happy to get my emails–lots of them thanked me for just doing the survey.

Once they had all responded, I read everything and compiled a list of major issues that were reported–overall a surprisingly small list! We’re focusing on those issues pretty heavily nowadays, and I think it’s making people happier with Bugzilla in general.

With the usability study, surprisingly the most helpful part was when the researchers (who were usability experts) just sat down in front of Bugzilla and pointed out things that violated usability principles. That is, even more valuable than the actual research they did was just their observations as experts, using the standard principles of usability engineering. The fact that they were fresh eyes–people who’d never worked on Bugzilla and thus didn’t just think “well that’s the way it is”–also was important, I think.

So we took all this data, and it really helped us prioritize. However, it’s important that we did the survey and research when we did them and not earlier. Back before we fixed the top few major issues, the usability and survey results would have just been overwhelming to us–they would have pointed out a million things we already knew, or a lot of things that we just didn’t have the time to work on at that point, and we would have had to re-do the survey and research again later, making it all a bunch of wasted time. So we had to wait until we were at the point of asking ourselves, “Okay, what’s most important now?”, and that was when gathering data became tremendously important and incredibly useful.

So overall, I’d say that when you’re trying to make things suck less, first go with what you know are the top few big obvious issues, and handle those, no matter what it takes. Then things will calm down a little, and you’ll have a huge pile of stuff that all needs fixing. That’s when you go and gather data from your users, and work to fix whatever they tell you actually sucks.

-Max

Comments: 4

But eventually, thanks to a bunch of new members in the Bugzilla community, we released Bugzilla 2.18. Hooray! Bells rang, birds sang, and there was much rejoicing.

However, in the space between Bugzilla 2.16 (which was before my time) and Bugzilla 2.18 (which was the first release that I helped get out), something very strange happened–we developed serious competition. All of the sudden there were a bunch of new bug-tracking systems, some of them open-source and some of them not, that people were actually using.

At first it wasn’t too worrisome. Bugzilla was pretty dominant in its field, and it’s hard to lose that kind of position. But as time went on, there was more and more competition, and some people were predicting doom and gloom for Bugzilla. We were a tiny group of completely unpaid volunteers, and some of these competing products were being made by companies whose marketing and development resources absolutely dwarfed us.

And yet, with every release, our download numbers kept going up. And always significantly–30-50% more than the previous release, every time.

And then we hit Bugzilla 3.0, and our download numbers nearly doubled. And they’ve kept going up with every release from there. Nowadays we get over 10 times the number of downloads per release that we did in 2004.

So how did we pull this off? Well, as far as I can tell:

All you have to do to succeed in software is to consistently suck less with every release.

Nobody would say that Bugzilla 2.18 was awesome, but everybody would say that it sucked less than Bugzilla 2.16 did. Bugzilla 2.20 wasn’t perfect, but without a doubt, it sucked less than Bugzilla 2.18. And then Bugzilla 3.0 fixed a whole lot of sucking in Bugzilla, and it got a whole lot more downloads.

Why is it that this worked?

Well, when people are deciding at first what software to use, they have varying criteria. Sometimes they just use what’s presented to them by default on the computer. Sometimes they have a whole list of requirements and they do lots of research and pick the software that has all the features they need. But once they’ve picked a program, they will stick with it unless there is some compelling reason to leave. It’s not like people constantly are looking for new software to replace yours–they only start looking when your software just won’t stop sucking.

As long as you consistently suck less with every release, you will retain most of your users. You’re fixing the things that bother them, so there’s no reason for them to switch away. Even if you didn’t fix everything in this release, if you sucked less, your users will have faith that eventually, the things that bother them will be fixed. New users will find your software, and they’ll stick with it too. And in this way, your user count will increase steadily over time.

You have to get out releases frequently enough that people believe that you really will suck less, of course. If your new release never comes out, then effectively, your current release never stops sucking.

But what happens if you do release frequently, but instead of fixing the things in your software that suck, you just add new features that don’t fix the sucking? Well, eventually the patience of the individual user is going to run out. They’re not going to wait forever for your software to stop sucking.

I remember a particular piece of software that I used every day for years. It had a lot of great features and a nice user interface, but it it would crash two or three times a week. I really liked the software in general, but man, the crashing sucked. I reported a bug about it, and the bug was ignored. I kept using it through 10 new releases, and it still crashed. The upgrades brought lots of new features, but I didn’t care about them. Remember, the feature set only mattered to me when I first picked the software. Now I just needed it to suck less.

But it never did.

So eventually, I went and looked for another piece of software that did the same thing, switched over, and I was happy with that one for a while. But guess what? It had a bug that really sucked. It didn’t happen very often, but when it did, boy was it a problem. But it sucked less than my old software, so I kept using it. Until one day, my patience ran out (after maybe 7 upgrades of the software), and I switched again.

Now I’m using a program that has half the feature set of either of the previous two programs. But, as a user, I’m the happiest I’ve ever been with this type of software. Because you know what? My new program sucks hardly at all. I mean, there are little things about it that suck, but supposedly a new release is coming out soon that will fix some of that sucking, and so I’m okay with it for now.

Would I have guessed this secret of success before I started working on Bugzilla? No. I would have told you the traditional wisdom–that a product succeeds or fails based on its feature set and user interface. But after 5 years on this project, managing our releases and watching our download count, I can tell you from my factual experience this strange thing: all you have to do to succeed as a software project is to suck less with every release. It doesn’t matter how much competition you have, or how many buzzword-friendly features you can cram into your interface. Just suck less, and you’ll succeed.

-Max

Comments: 57

10. Dudes at Silicon Valley parties will think you’re, like, cool.

    Impress people you don’t know! You can say, “I work on an open source project,” and nod your head like you’re cool. But no, more seriously, about 50% of all the people I meet at Silicon Valley parties are totally amazed that I’m one of the primary developers of Bugzilla, something that they use every day.

    This doesn’t work so well with the ladies, though, usually. “Yeah, I’m an extremely intense programmer, oh yeah.”

    No, but more seriously: How many girls did you see at OSCON? If anybody has a scheme to get more girls involved or interested in open source, the whole open-source world (the current girls included, I’m sure) would be thrilled.

9. It totally has nothing to do with whether or not you’re a “hottie.” Just be yourself, man, just be yourself.

   Open Source is definitely one place where you’ll be respected for your intelligence and ability, not how expensive your clothes are or how much you paid for your haircut. Nobody cares what you look like. We only care how good your ideas are.

8. It’s hella sick when you’re interviewing!

   When you’re interviewing and you worked on some open-source project, it’s completely valid job experience. Sometimes it even makes you more valuable than normal job experience, if the project is well-known. Also, if you worked on open-source in your spare time, that shows the kind of passion for software engineering that employers are really looking for.

7. You can, like, totally freelance. Seriously? Ya, like totally seriously.

   There’s a lot of need for contractors for certain open-source projects. If you become a prominent contributor to a project and get your name on its Consultants List, then you can make a living doing consulting for people who use the software!

6. Dude, we’ll like, laugh at your jokes and stuff, dude.

   I have laughed harder while listening to some conversations on IRC than I have at almost anything else in the world. There’s something about shared experience and understanding that makes things much funnier than jokes that “everybody” is supposed to get. And where else can you make jokes about programming languages and have multiple people actually get them?

5. You can say how, like, stuff goes and people will actually listen to you. Whoa.

   When you write a feature, to a large degree you’re the one who’s going to decide how it works! Don’t like how a particular program works? Well, maybe you could have been the one who wrote that feature instead! Don’t like some documentation? What if you had written it instead?

   And if you don’t like how something works now, the more you contribute to an open-source project, the more say you’ll have in fixing that thing. Have an itch to just make things work right? Open Source is the place to be.

4. You learn, like, so much stuff, like seriously.

   Are you looking for something new, some way to expand your horizons and learn something new instead of just mechanically doing the same thing over and over at work? No matter what your interest is, there’s going to be some open-source project out there that uses the things you want to learn about. And it won’t just be some tiny project just for yourself, but something that people really use!

3. You can like, belong to something, and stuff. That’s some cosmic stuff, man.

   When you contribute to an open-source project, you’re not just a cog in a great machine, or just a worker at a job. You become part of a larger community, with its own in-jokes, culture, and people.

   I used to think that was just some marketing gibberish, but it’s really true. It may not be the best way to find people who you can “hang out” with every day, but you’ll get to know a lot of new people and become part of a group in a very definite way.

   It’s even more true if you go to conventions like OSCON or the more specific ones for the various open-source projects, where you can meet and hang out with lots of other developers “In Real Life”, most of whom are really great (and intelligent) people.

2. You get to feel like a revolutionary (a revolutionary nerd, but…hey, s’all good, s’all good).

   Yeah! Down with The Man! We don’t need any stinkin’ proprietary software!

   You can even get to protest things, like software patents! You’ll be almost as cool as your parents were in the 60’s. Kind of.

   No, but seriously: Open Source is still a relatively new thing in the world, and you can be a part of blazing its trail. It’s not the “normal” way to do things quite just yet, so if you like being a little different than the swarming masses, open source is the place to be.

1. It seriously helps out, man.

   Working on open-source software helps out a lot of people:

   • The people who use the software. Millions of people use open-source software around the world every day to do their job, handle their problems, or just have fun! You could be affecting the lives of all those people.
   • The people who write the software. Open-source projects almost all really want your assistance! If a project is at all popular, they probably have more feature requests than they can handle. Don’t come to an open-source project and say, “This is what I’m going to do for you,” but do come and say, “How can I help you guys out?” We all need some help, and competent help is much appreciated.
   • All the people who won’t have to write the software that you write. Sometimes you just want to download a program to do a task for you. You don’t want to have to write a program for everything you want to do. You’re saving the time of millions of users and computer programmers, by creating something that everybody can use and modify!

And that’s my list! Hope you had totally chill time, brah.

-Max

Comments: 30

Everybody wants to “innovate.” The news talks about “who’s being the most innovative.” Marketing for companies insists that they are “innovating.”

Except actually, it’s not innovation that leads to success. It’s execution.

It doesn’t matter how good or how new my idea is. It matters how well I carry it out in the real world.

Now, our history books worship the inventors, not the executors. We are taught all about the people who invent new things, come up with new ideas, and plough new trails. But look around you in present time and in the recent past, and you’ll see that the most successful people are the ones who carried out the idea really well, not the people who came up with the idea.

Elvis didn’t invent rock and roll. Ford didn’t invent the automobile or the assembly line. Apple didn’t invent the GUI. Webster didn’t invent dictionaries. Maytag didn’t invent the washing machine. Google didn’t invent web searching. I could go on and on and on.

Granted, sometimes the innovator also is an excellent executor (Alexander Graham Bell being an example), but usually that’s not the case. Most inventors don’t turn out to be the most successful people in their field (or even successful at all).

So stop worrying about “coming up with something new.” You don’t have to do that. You just have to execute an already existing idea really, really well. You can add your own flair to it, maybe, or fix it up a little, but you don’t have to have something brand new.

There are so many examples that prove this that it’s hard not to see one if you move your eyes anywhere. Just look, you’ll see.

Now, I’m not saying that people shouldn’t innovate. You should! It’s fun, and it advances the whole human race a tiny step every time you do. But it’s not the path to long-term success for you or for any group you belong to. That’s all in execution.

-Max

Comments: 39

One of the features of this web browser is its JavaScript engine, called v8, which is designed for performance.

Designing for performance is something that Google does often. Now, designing for performance usually leads to complexity. So, being a major supporter of software simplicity, I’m opposed, in a theoretical sense, to designing for performance.

However, Google is in an interesting situation. Essentially, we live in the Bronze Age of computing (or perhaps the Silicon Age, as I suspect future historians may call this period of history). Our computers are primitive, compared to what we are likely to have 50 to 100 (or 1000!) years from now. That may seem hard to believe, but it’s always hard to imagine the far future. Google is operating on a level far exceeding our current hardware technology, really, and so their design methods unfortunately can’t live in a theoretical fairy-land where hardware is always “good enough.” (However, I personally like to live in that land, when possible, because hardware will improve as time goes on–an important fact to understand if you’re going to have a long-lived software project).

What is it about our computers that makes them so primitive? Well, usually I don’t go about predicting the future in this blog, or even talking about too many specifics, because I want to keep things generally applicable (and also because the future is hard to predict, particularly the far future). But I will talk about some of my thoughts here on this, and you can agree with them or not, as you please.

Our Current Computers

First, you have to understand that to me, anything that could be running this web browser right now, that could be showing me my desktop, and that I could be typing into right now–I’d consider that a computer. It actually doesn’t matter what it’s doing underneath, or how it works. A good, offhand definition then of a computer would be:

Any piece of matter which can carry out symbolic instructions and compare data in assistance of a human goal.

Currently computers do this using math, which they do with transistors, digitally. The word “digital” comes from “digit”, a word meaning, basically, “fingers or toes.” For most people, your fingers and toes are separate, individual items. They don’t blend into each other. “Digitally” means, basically, “done with separate, individual numbers.” You know, like 1, 2, 3, 4–not 1.1, 1.2, 1.3. People (normally) have 1 or 2 fingers, not 1.5 fingers.

Said another way, current computers change from one fixed state to another, very fast. They follow instructions that change their state. I don’t really care if we say that the state is “transistor one is on, transistor two is off, transistor three is on…” or “Bob has a cat, Mary has a dog, Jim has a cat…” it’s all a description of a state. What we care about ultimately is the total current state. If there are 1,000,000 possible states (and there are far, far, far more in a current computer), then we can say that “we are at state 10,456″ and that’s all we really need to know.

The problem with current computers is Moore’s Law. We don’t need computers that are twice as fast. We need computers that are about a million times faster than the ones we have. We need computers so fast that software engineers never have to worry about performance ever again, and can just design their code to be the sanest, most maintainable thing possible. With that kind of performance, we could design almost any software imaginable.

The problem is that with Moore’s Law, we’re not going to get computers 1000 times faster for about 20 years. We’re going to get to 1,000,000 times faster in about 40 years. And there’s a chance that the laws of physics will stop us dead in our tracks before that point, anyhow.

Future Computers

So, let’s stick with the idea of a machine that changes states for the near future, because I can’t think of any other clever way to make a computer that would follow my definition from above. There are three problems, then:

1. How many states can we represent?
2. How many physical resources does it require to represent all those states (including space, power, etc.)?
3. How quickly can we change between states?

And then “How many states can we represent at once?” might also be a good question–we’re seeing this come up more and more with dual-core and quad-core processors (and other technologies before that, but I don’t want to assume that everybody reading my blog is an expert in hardware architecture, and I don’t want to explain those technologies).

So the ideal answers are:

1. We can represent an infinite number of states.
2. It requires no physical resources to represent them.
3. We can change between them without time.

And then also “We can represent an infinite number of different states at once.”

Currently we theoretically could represent an infinite number of states, we’d just have to add more transistors to our chip. So really, the question becomes, “How many states can we represent with how many physical resources?” Currently we can fit two states into 32 nanometers. (That’s one transistor.)

My suspicion is that the future is not in fitting two states into a continually smaller space, but in fitting a near-infinite number of states into a slightly larger space. Electricity and other force waves can be “on” or “off”, but they also have lots of other properties, many of which are sufficient to represent an infinity (or near-infinity). Frequency of any wave represents an infinity, for example–you can have a 1 Hz wave, a 1.1 Hz wave, a 1.15 Hz wave, a 1.151 Hz wave, etc. So, that basically answers Question 1 ideally–you can have an infinite number of states, you just have to have some device which is sufficiently small, in which a wave can have its properties modified and measured by electronics, optics, or some other such technology.

You’ll notice that we’ve also conveniently answered our bonus question, because we can represent quite a few different states at once, once each individual component of our system can represent an infinity all by itself.

If we want to look a bit further into the future, our second question can be answered by the fact that waves take up essentially no space (only the medium that they vibrate takes up space). Our understanding of physics is not (as far as I know) currently good enough to create structures out of pure force just yet, but such structures would come quite close to taking up “no physical resources.”

And beyond that (how we get the state changes to happen without time), I have no idea. That question may be unanswerable, and may only be resolvable by changing computers to being something other than mathematical devices. (That is, not be involved with states at all, but some other method of following instructions and comparing data.) But the better our components become, the closer we can get to “no time.”

The Roundup

So there’s my thoughts for the day on the future of computing. Sometimes designing software for performance is a necessary evil (but really only in the case where it’s an extreme issue, like with Google’s products, or the great new need for speed in JavaScript nowadays, or in other low-level places), but I hope that future changes in the fundamental architecture of computers will obsolete that necessity.

-Max

Comments: 14

My policy on projects that I control is that we never add a feature unless the design can support it simply. This drives some people crazy, notably people who have no concept of the future. They start to foam at the mouth and say things like, “We can’t wait! This feature is so important!” or “Just put it in now and we’ll just clean it up later!” They don’t realize that this is their normal attitude. They’re going to say the same thing about the next feature. If you give in to them, then all of your code will be poorly designed and much too complex. It’ll be Frankenstein’s monster, jammed together out of broken parts. And just like the friendly green giant, it’ll be big, ugly, unstable, and harmful to your health.

Adding a tiny little piece and refactoring it afterward is fine. Landing a huge feature that the architecture can’t support and then trying to clean it up afterward is a terrible task. Size matters.

The worst situation, however, is when you let people keep adding features with no design for months or years, and then one day you wake up and realize that something is not right. Now you have to fix your whole codebase. This is a terrible task, because just like adding a new feature, it can’t be done all at once, unless you want to re-write. If you want to start doing things the right way, you have to start doing things the right way. And that means that you have to fix the design piece by piece, in simple steps. That usually requires months or years of effort–totally wasted effort, because you should have just designed from the start. You should have thought about the future.

In your project lacks a strict design, and it continues to grow, then you will eventually end up over your head in complexity. I understand that this is hard for some people to imagine. Some folks can’t imagine that there is a future beyond lunch. Other folks just haven’t had enough experience to understand how complex things can get. And I understand that there can be a corporate culture that says, “Oh, we just hack in new features, and we should do things the right way, but we can’t because blah blah blah.” But one day your project will fail. And no matter how many reasons you can give for that failure, it won’t change the fact that your project failed.

Often, when you’ve done your design right, there’s not a whole lot of credit that comes your way. Catastrophic failures in design are big and noticeable, small increments of work toward a good design are invisible to people who aren’t intimately connected with the code. So this can make it difficult–handling a big failure gets you a lot of thanks, preventing one in the first place, well, nobody noticed.

So I’ll congratulate you myself. Did you design from the start? That was awesome. You absolutely did the right thing. Have you started designing now? Well, you should have started earlier, but congratulations on starting to move in the right direction. Your users and fellow developers will see the benefits–working software, on-time releases, and a clear, understandable codebase. Will they know how much work it took to get it that way? Maybe not. But that’s OK. Sometimes doing things the right way is really its own reward.

-Max

Comments: 8

Why has The Sims sold 100 million copies, while Second Life, an ostensibly much more flexible and powerful universe, only has about 2 million active users? They look pretty similar, and you might guess at first glance that they’d have somewhat similar audiences. But, although 2 million users is nothing to scoff at, 100 million absolutely trounces it. So why the big difference?

Well, of course, The Sims has EA Games behind them, who have a massive distribution channel and a lot of marketing power, but the Internet buzz and general promotion of Second Life is pretty good too, so although EA has the edge, that doesn’t explain a 50-to-1 difference in sales. There must be something actually different about the products themselves.

Well, at first glance, The Sims is very user-friendly and Second Life is (from what I’ve heard) hard to use. The Sims does a limited scope of things very well, and Second Life does an unlimited number of things through a difficult interface, with mediocre results.

But fundamentally, why is it that products like The Sims succeed so much more than things like Second Life? And why does The Sims have a better interface, why do people want to play The Sims more than they want to play Second Life? Well:

It’s easy to make a system that does something specific, and hard to make a system that does everything.

When something is easy to make, you can spend a lot more time focusing on the little details–the polish. When something is hard to make, you spend all your efforts just making it work, and there’s no time left to sand off the rough edges. Specific solutions allow you to handle a problem with a level of grace, efficiency, and quality that could never be achieved by a generic, do-it-all solution.

A great example is the Colossus computer, built in 1943 to break encrypted German radio messages in World War II. That was all the Colossus did–it was fundamentally incapable of doing any other task. However, it would have taken a 5 MHz general-purpose computer to break that code at the same rate as the Colossus–a level of speed that desktop computers didn’t reach until the 1970’s, 30 years later.

This is something that I have been trying to get across to programmers for quite some time–you don’t need to solve all the world’s problems with one piece of code, you only need to solve the problem you’re solving. Whether you’re designing a whole system or just a tiny piece, your code doesn’t need to do any more than is called for by the known requirements. Sure, keep it extendable for the future–that’s part of making a quality solution. But all solutions should be specific solutions to known problems.

The Sims satisfies a specific, known need–the desire of people to play out specific types of fantasy lives. It succeeds because it’s not infinitely flexible. Second Life is a nightmare of complexity because it tries to be “everything to everyone”–a situation in which you almost always end up being not enough to anyone.

When people have problems with complexities or confusions in their software, I encourage them to be The Sims. Be Notepad. Be the Google main page. Don’t try to anticipate every need in the world, just solve the ones you know exist. Everything tends to fall into place when you really know your requirements and just go to solve them.

-Max

Comments: 4

In the US, it’s probably a symptom of a general cultural problem–if an American can’t see an instant result from something, they think it doesn’t work. This leads to fast food, french fries, and fat people. The healthy way to eat (protein and vegetables) has a delayed effect on the body (you don’t get the energy for over an hour), and the bad way to eat (endless carbohydrates without nutritional value) has an instant result–immediate energy.

Software is always a long-term process. I wrote the first version of VCI in about three weeks, and that was insanely fast. Any actual application (VCI’s just a library) takes months or years of person-hours, even if you keep it small. So you’d think that organizations would be far-sighted about their development strategies, right?

Unfortunately, it just doesn’t happen. Competitor X comes out with “Shiny New Feature” and The Company says “we must have Shiny New Feature RIGHT NOW!” That’s not a long-term winning strategy, that’s just short-sighted panic. If you have users, they’re not all going to get up and go away in the next five minutes just because somebody else has one feature that you don’t. You should be looking at trends of how many users you’re gaining or losing, not just responding mindlessly to the immediate environment.

So what’s a good long-term strategy? Well, refactoring your code so that you will still be able to add features in the future, that’s a good one. Or spending some extra time putting some polish on your features and UI so that when the product is released, users are actually happy with it. Not adding features that you don’t want to maintain, if they’re not important enough–that’s another one.

Remember that Mozilla did poorly for years, only to finally start gaining dominance in a market that Netscape had lost, because they had a long-term plan. Granted, Mozilla made some decisions early on that caused some things to take longer than they should have, but they still won out in the long term, despite failing in the short term.

Of course, it can be hard to convince people that your long-term plan is right, sometimes, because it takes so long to show results! When I started refactoring Bugzilla about four years ago, there was pretty constant resistance, particularly when I would review patches and say, “You need to wait for the new architecture before this can go in,” or “This needs to be fixed to not be spaghetti code.” But once the refactoring really got rolling (after about two and a half years), it suddenly became way easier to add new features and nearly all the developers became big supporters of refactoring.

I read so much “advice” on “how to run your software business” that just focuses on instant gratification–what you can get done right now. “Add features!” “Get millions of dollars instantly from VCs!” Unfortunately, the way the universe seems to work is that you can destroy something in an instant, but it takes time to create something. So in reality, the closer you get to “instant gratification”, the closer you get to destruction of your product, your business, and your future.

If you want a good plan, pick one that admits that creation takes time. It doesn’t have to take forever, but it’s never instant.

-Max

Comments: 11

These mathematicians are the people we would consider the modern founders of “computer science.” Computer Science is actually the mathematical study of information processing. It is not, as some people believe it to be, the study of computer programming. In fact, there is no science of computer programming. To understand how that could possibly be true, and what I mean, you have to know the history of programming.

The earliest computers were built under the supervision of computer scientists by highly skilled electronic engineers. They were run by highly-trained operators in tightly-controlled environments. They were all custom-built by the organizations that needed them (mostly governments, to aim missiles and crack codes), and there were only one or two copies of any given model.

Then, along comes UNIVAC and the whole notion of “commercial” computers. Now, there’s only so many advanced theoretical mathematicians in the world. If you start shipping out computers to everybody, you can’t ship a mathematician along with each one. So although some organizations, such as the United States Census Bureau, almost certainly had some highly-trained operators for their machinery, other organizations undoubtedly got their machine and said, “Okay, Bill from Accounting, this is yours! Read the manual and have at it!” And there went Bill, diving into this complex machine and doing his best to make it work.

Bill there is our first “working programmer.” He might have studied math in school, but he almost certainly didn’t study the sort of advanced theory needed to conceive and design the machine itself. But he can read the manual and understand it, and by trial and error, make the machine do what he wants.

Of course, the more commercial computers you ship, the more Bills you have and the fewer highly-trained operators you have. And if there’s one thing that Bill has, it’s job pressure. He has demands from management to “Get that task done now!” and “We don’t care how it’s done, just do it!” He figures out how to make the thing work according to its manual, and it works, even if it crashes every two hours.

Eventually Bill gets a whole team of programmers to work with him. He has to figure out how to design a system and split up the tasks between different people. Instead of being studied and mapped out like a standard science, the whole art of practical programming grows organically, more like college students teaching themselves to cook than like NASA engineers building a space shuttle.

So there’s this hodge-podge system for software development, and it’s all very complex and hard to manage, but everybody gets along somehow. Then along comes The Mythical Man Month, a book by a guy who actually looked at the process of software development and pointed out some things about it–most famously that adding more programmers to a project doesn’t necessarily make it faster. He didn’t come up with a whole science, but he did make some good observations about programming and managing software development.

Of course, then came a flurry of software development methods: the Rational Unified Process, the Capability Maturity Model, Agile Software Development, and others. None of these claim to be a science, just a way of managing the complexity of software development.

And that, basically, brings us up to where we are today: lots of “methods,” but no real science.

A science is composed of observations, experiments, and laws. Although we have hundreds of books of observations about practical programming, and “the world is our laboratory”, all of that has resulted in very few laws, which are the most important part of a science. That is, instead of coming up with methods to work around complexity, there ought to be some fundamental rules to follow that would lead to simplicity–ways to avoid complexity entirely and explain why what “works” in software development does work.

In reality, there are two missing sciences here. The first one is being worked on actively, and includes the various methods I mentioned above. That’s the science of managing software development. The fact that conflicting, equally-valid “opinions” seem to exist within the field indicates that the fundamental laws of software management have not been worked out. However, there is at least attention being given to the problem.

The other science, though, gets very little attention in the practical world of programming: the science of writing software. Very few people are taught that there is a science to writing software, in school. Instead, they are just shown “This is how it works in this programming language, now go write some software!”

The science I’m talking about is not Computer Science. That’s a mathematical study. I’m talking about a science for the “working programmer”–something that gives fundamental laws and rules to follow when writing a program in any language. One that’s as reliable as physics or chemistry in telling you how to create an application.

Some people might say that such a science is not possible, that software development is too variable to ever be described by simple, fundamental laws. Some people also once said that understanding the physical universe was impossible because “it is the creation of God and God is unknowable.” So unless you’re telling me computers are unknowable, I think that making such a science would be entirely possible.

Actually, I’ve started to collect some basic hypotheses for such a science, and I’m writing a book about it. I might post some of them here, but actually, all of what I’ve written in the blog so far actually comes from those hypotheses. That is, I haven’t really posted any of my hypothetical “laws” yet, but I have been posting data that I’ve derived from them.

Anyhow, I think that the primary source of complexity in software is actually the fact that this science is lacking. If programmers actually had a science for simplicity in software, there wouldn’t be nearly so much complexity, and we wouldn’t need crazy processes to manage that complexity.

Whenever there’s a problem with something, my instinct is to go philosophically above the level of the problem and look at it from there. If there’s some “unsolvable” effect happening to something, there must be some unknown cause on a higher level. So there you go–if the problem is complexity, then maybe it’s because there was no science of simplicity in the first place.

-Max

Comments: 11

For example, imagine a program that accepts only the numbers 1 and 2 as input and rejects everything else. Even a tiny variation in the input, like adding a space before or after “1″ would cause the program to throw an error. That would be very “strict” and extremely simple to write. All you’d have to do is check, “Did they enter exactly 1 or exactly 2? If not, throw an error.”

In most situations, though, such a program would be so strict as to be impractical. If the user doesn’t know the exact format you expect your input in, or if they accidentally hit the spacebar or some other key when entering a number, the program will frustrate the user by not “doing what they mean.”

That’s a case where there is a trade-off between simplicity (strictness) and usability. Not all cases of strictness have that trade-off, but many do. If I allow the user to type in 1, One, or " 1" as input, that allows for a lot more user mistakes and makes life easier for them, but also adds code and complexity to my program. Less-strict programs often take more code than strict ones, which is really directly where the complexity comes from.

(By the way, if you’re writing frameworks or languages for programmers, one of the best things you can do is make this type of “non-strictness” as simple as possible, to eliminate the trade-off between usability and complexity, and let them have the best of both worlds.)

Of course, on the other side of things, if I allowed the user to type in O1n1e1 and still have that be accepted as “1″, that would just add needless complexity to my code. We have to be more strict than that.

Strictness is mostly about what input you allow, like the examples above. I suppose in some applications (like, say, a SOAP library), you could have output strictness, too–output that always conforms to a particular, exact standard. But usually, it’s about what input you accept and what input causes an error.

Probably the best-known strictness disaster is HTML. It wasn’t designed to be very strict in the beginning, and as it grew over the years, processing it became a nightmare for the designers of web browsers. Of course, it was eventually standardized, but by that time most of the HTML out there was pretty horrific, and still is. And because it wasn’t strict from the beginning, now nobody can break backwards compatibility and make it strict.

Some people argue that HTML is commonly used because it’s not strict. That the non-strictness of its design makes it popular. That if web browsers had always just thrown an error instead of accepting invalid HTML, somehow people would not have used HTML.

That is a patently ridiculous argument. Imagine a restaurant where the waiter could never say, “Oh, we don’t have that.” So I ask for a “fresh chicken salad”, and I get a live chicken, because that’s “the closest they have.” I would get pretty frustrated with that restaurant. Similarly, if I tell the web browser to do something, and instead of throwing an error it tries to guess what I meant, I get frustrated with the web browser. It can be pretty hard to figure out why my page “doesn’t look right”, now.

So why didn’t the browser just tell me I’d done something wrong, and make life easy for me? Well, because HTML is so un-strict that it’s impossible for the web browser to know that I have done something wrong! It just goes ahead and drops a live chicken on my table without any lettuce.

Granted, I know that at this point that you can’t make HTML strict without “breaking the web.” My point is that we got into that situation because HTML wasn’t strict from the beginning. I’m not saying that it should suddenly become strict now, when it would be almost impossible. (Though there’s nothing wrong with slowly taking incremental steps in that direction.)

In general, I am strongly of the opinion that computers should never “guess” or “try to do their best” with input. That leads to a nightmare of complexity that can easily spiral out of control. The only good guessing is in things like Google’s spelling suggestions–where it gives you the option of doing something, but doesn’t just go ahead and do something for you based on that guess. This is an important part of what I mean by strictness–input is either right or wrong, it’s never a “maybe.” If one input could possibly have two meanings, then you should either present the user with a choice or throw an error.

I could go on about this all day–the world of computers is full of things that should have been strict from the beginning, and became ridiculously complex because they weren’t.

Now, some applications are forced to be non-strict. For example, anything that takes voice commands has to be pretty un-strict about how people talk, or it just won’t work at all. But those sorts of applications are the exception. Keyboards are very accurate input devices, mouses slightly less so but still pretty good. You can require input from those to be in a certain format, as long as you aren’t making life too difficult for the user.

Of course, it’s still important to strive for usability–after all, computers are here to help humans do things. But you don’t necessarily have to accept every input under the sun just to be usable. All that does is get you into a maze of complexity, and good luck finding your way out–they never strictly standardized on any way to write maps for the maze.

-Max

Comments: 13

A great example of the backwards-compatibility problem is Perl. If you read the perl5-porters summaries with any regularity, or if you’re familiar with the history of the Perl internals in general, you’ll have some idea of what I mean.

Perl is full of support for strange syntaxes that really, nobody should be using anymore. For example, in Perl, you’re supposed to call methods on an object like $object->method(). But there’s also a syntax called the “indirect object syntax” where you can do method $object. Not method($object)–only the case without the parenthesis is the indirect object syntax.

Really, nobody should be using that syntax, and it’s not that hard to fix applications to call their methods the right way. And yet that syntax is maintained and supported in the Perl binary to keep backwards compatibility.

Perl is full of things like this that block forward progress because of historical problems.

Now obviously, this is a balancing act. When there are a huge number of people using something, and it would be really difficult for them to change, you pretty much have to maintain backwards compatibility. But if maintaining that backwards-compatibility is really stopping forward progress, you need to warn people that the “old cruft” is going away and ditch it. The alternative is infinite backwards-compatibility and no forward progress, which means total death for your product.

This also gives one good reason why you shouldn’t just add features willy-nilly to your program. One day you might have to support backwards-compatibility for that feature that you added “because it was easy to add even though it’s not very useful.” This is an important thing to think about when adding new features–are you going to have to support that feature forever, now that it’s in your system? The answer is: you probably are.

If you’ve never maintained a large system that’s used by lots of people, you might not have a good idea of: (a) How many people can be screwed if you break backwards-compatibility, and (b) How much you can screw yourself over by having to maintain backwards-compatibility. The ideal solution there is: just don’t add features if you don’t want to support them for many, many future versions to come. Sometimes it takes a lot of programming experience to make that sort of decision effectively, but you can also just look at the feature and think, “Is it really so useful that I want to spend at least 10 hours on it in the next three or four years?” That’s a good estimate of how much effort you’re going to put into backwards-compatibility, QA, and everything else for even the smallest feature.

Once you’ve got a feature, then maintaining backwards-compatibility is generally the thing to do. Modern Bugzilla can still upgrade from 2.8, released in 1999. But it can do that because we wrote the upgrader in such a way that old upgrader code doesn’t require any maintenance–that is, we get that backwards-compatibility for free. We only have to add new code as time goes on for new versions of Bugzilla, we almost never have to change old code. Free backwards-compatibility like that should always be maintained.

The only time you should seriously consider ditching backwards-compatibility is when keeping it is preventing you from adding obviously useful and important new features. But when that’s the case, you’ve really got to ditch it.

-Max

Comments: 4

The dictionary just defines it as a combination of “over” (meaning “too much”) and “engineer” (meaning “design and build”). So per the dictionary, it would mean that you designed or built too much.

Wait, designed or built too much? What’s “too much”? And isn’t design a good thing?

Well, yeah, most projects could use more design. They suffer from underengineering. But once in a while, somebody really gets into it and just designs too much. Basically, this is like when somebody builds an orbital laser to destroy an anthill. An orbital laser is really cool, but it (a) costs too much (b) takes too much time and (c) is a maintenance nightmare. I mean, somebody’s going to have to go up there and fix it when it breaks.

The tricky part is–how do you know when you’re overengineering? What’s the line between good design and too much design?

Well, my criteria is this: When your design actually makes things more complex instead of simplifying things, you’re overengineering. An orbital laser would hugely complicate the life of somebody who just needed to destroy some anthills, whereas some simple ant poison would greatly simplify their life by removing their ant problem (if it worked).

This isn’t to say that all complexity is caused by overengineering. In fact, most complexity is caused by underengineering. If you have to choose, the safer side to be on is overengineering. But that’s kind of like saying it’s safer to be facing away from an atom bomb blast than toward it. It’s true (because it protects your eyes more), but really, either way, it’s going to suck pretty bad.

The best way to avoid overengineering is just don’t design too far into the future. Overengineering tends to happen like this: “Okay, I need some code to reverse a string. Well, might as well make a whole sytem for rearranging and modifying the letters in a string, since we might need that some day.” Essentially, somebody imagined a requirement that they had no idea whether or not was actually needed. They designed too far into the future, without actually knowing the future.

Now, if that developer really did know he’d need such a system in the future, it would be a mistake to design the code in such a way that the system couldn’t be added later. It doesn’t need to be there now, but you’d be underengineering if you made it impossible to add it later.

With overengineering, the big problem is that it makes it difficult for people to understand your code. There’s some piece built into the system that doesn’t really need to be there, and the person reading the code can’t figure out why it’s there, or even how the whole system works (since it’s now so complicated). It also has all the other flaws that designing too far into the future has, such as locking you into a particular design before you can actually be certain it’s the right one.

There are lots of common ways to overengineer. Probably the most common ways are: making something extensible that won’t ever need to be extended, and making something way more generic than it needs to be.

A good example of the first (making something extensible that doesn’t need to be) would be making a web server that could support an unlimited number of other protocols in addition to HTTP. That’s kind of silly, because if you’re a web server, then you’re sending HTTP. You’re not an “every possible protocol” server.

However, in that same situation, underengineering would be not allowing for any future extension of the HTTP standard. That’s something that does need to be extensible, because it really might change.

The issue here is “How likely it is that this thing’s going to change?” If you can be 99.999% certain that some part of your system is never going to change (for example, the letters available in the English language probably won’t be changing much–that’s a fairly good certainty) you don’t need to make that part of the system very extensible. (Even so, it’s still good to leave a tiny little room to expand, in the very rare chance that somebody adds something like, say, the Euro symbol to the language.)

There are just some things you have to assume won’t change (like, “We will never be serving any other protocol than HTTP”)–otherwise your system just gets too complex, trying to take into account every possible unknown future change, when there probably won’t even be any future differences. This is the exception, rather than the rule (you should assume most things will change), but you have to have a few stable, unchanging things to build your system around.

The second way (making something too generic) goes like this: Imagine that the Bugzilla Project suddenly went insane, and instead of saying that Bugzilla was a “bug tracking system”, we decided to make it into a “generic system for managing data in a database through forms.” It would become terribly complex, and it would also stop being a very good bug tracker, because it would be trying to be “everything to everyone” instead of just focusing on adding good bug-tracking features. That would definitely be overengineering–we’re just trying to track bugs, but suddenly we’re a generic form system? Yep, sounds like “orbital lasers” to me.

In addition to being too generic on the whole-program level, individual components of the program can also be too generic. A function that processes strings doesn’t also have to process integers and arrays, if you’re never going to be getting arrays and integers as input.

You don’t have to overengineer in a huge way, either, to mess up your system. Little by little, tiny bits of overengineering can stack up into one huge complex mass.

Good design is design that leads to simplicity in implementation and maintenance, and makes it easy to understand the code. Overengineered design is design that leads to difficulty in implementation, makes maintenance a nightmare, and turns otherwise simple code into a twisty maze of complexity. It’s not nearly as common as underengineering, but it’s still important to watch out for.

-Max

Comments: 11

Well, of course, simplicity is relative. But even so, you can still be more or less simple. From the relative viewpoint of your user, your product can be hard to use, easy to use, or somewhere in between.

So, how simple do you have to be?

Honestly?

If you really want to succeed?

Stupid, Dumb Simple.

Apple knows this, one hundred percent. They make products that are so simple they’re practically idiotic. They aren’t the most technically advanced products (there are lots of phones that beat the pants off the iPhone in terms of features, for example), but they’re all usable by total morons. Big bright buttons, no clutter, straightforward interfaces, big text that explains everything–it’s like an elementary school textbook, but prettier and more expensive.

That’s the level of simplicity I’m talking about–the kind where you’d say, “Hey, my whole family could use this thing!”

But often, people really just don’t understand how stupid, dumb simple they have to be, to get to that level. For example:

When I’m at the mall, there are maps that tell me where everything is. On these maps, I want a huge red dot, with the words “YOU ARE HERE” in gigantic letters right there. Usually, though, I get a tiny red (or green, or yellow) dot (or triangle, or square) in the middle of the map that I really have to search for to see it. And then, off to the side there’s some text that explains, “The tiny green triangle means ‘You are here!’” Add this up to the general confusion of trying to find anything on these maps, and I could be spending five or six minutes just standing in front of this thing, trying to figure out how to get where I’m going.

To the guy that designed the map, this is all totally reasonable. He spent lots of time designing this map, so it was important enough to him that he would spend several minutes on it, learn all about it, figure it out, etc. But to me, the map is a very, very minor part of my existence. I just want it to be as simple as humanly possible, so that I can use it quickly and get on with my life!

Many programmers are particularly bad about this with their code. They assume that other programmers will spend a lot of time learning all about their code, because after all, it took a lot of time to write it! The code is that important to them, so isn’t it that important to everybody?

Now, the programmers I know are generally an intelligent bunch. There’s always a few exceptions, but mostly they’re very bright. But it’s still always a mistake to think, “Oh, other programmers will understand everything I’ve done here without any simplification or explanation of my code.” It’s not a matter of intelligence–it’s a matter of knowledge. That programmer who is new to your code doesn’t know anything about it. He has to learn! The easier you make it for him to learn, the faster he’s going to figure it out, and the easier it will be for him to use it.

There’s lots of ways to make your code easy to use–simple documentation, simple design, step-by-step tutorials, etc.

But, if your code isn’t stupid, dumb simple to use, people are going to have trouble with it. They’re going to use it wrongly, create bugs, and generally muck things up. And when all this happens, who are they going to come ask about it? Yes, you! You are going to be spending time answering all their questions…. (Mmmmmmm, sounds fun, doesn’t it?)

I know that none of us like being talked down to, or treated like we’re idiots. And sometimes that leads to creating things that are a little complicated, so that we feel like we aren’t “talking down to” to the user or the other programmer. We throw in some big words, make it a little less than simple, and people respect our intelligence but feel kind of stupid because they don’t get it. They might think we’re way smarter than they could ever be, and that is kind of flattering. But really, is that helping them?

On the other hand, when you make things stupidly simple, you’re allowing the user to understand your product. That makes them feel smart, lets them use the program, and doesn’t reflect badly on you at all. In fact, your users will probably admire you more if you make things simple than if you make them complex.

So when this question comes up of “How simple do we have to be,” you might as well ask yourself, “Do we want people to understand this and be happy, or do we want them to hate us and be frustrated?” And if you pick the former, then there’s only one level of simplicity that will assure your success:

Stupid, Dumb Simple.

-Max

Comments: 4

• Somehow they’re not demonstrating how intelligent they are, or how valuable they are, to their managers, or
• The project will become so simple to work on that anybody can just steal their job!

It’s almost as though if they actually did their job right, then they’d lose it. Now, stated that way, that’s obviously a nonsensical viewpoint. But, if you’ve ever worried about it, here’s something to think about:

What if your code is so complex that you’ll never be able to leave your job?

What if you made something so complicated that nobody else could understand it? Well, then you personally would be tied to that project forever and ever. If you wanted to work on some other project at your organization, your managers would protest, “But who else will maintain this code?” Whoever was assigned after you to work on your code would constantly be walking into your new office, saying, “Hey, how does this part work?”

Maybe you have no conscience, and you’ll just leave the code to some hopeless replacement and ditch the company. However, I’m guessing that most people would feel tied to a project if they were sure that nobody else could ever take it over successfully. And really, even if you just take off and leave, somebody’s going to be calling you up and saying, “Um, hey, you know that one piece of code where…” You’ll get emails from “the new guy”: “Hey, I hear you wrote this code, and I have this problem…” If you can’t make somebody else understand your code and have them truly take it over, then you’re going to be stuck with a piece of that job forever.

In the Bugzilla Project, I am doing the best I can to work myself out of a job. I love working on Bugzilla, but I don’t want to be tied to it every moment of my life. I want to go on vacation sometimes. I want to write music! I want to be able to leave my computer for a month, and not have the whole world collapse. So I work to make Bugzilla simple enough and well-designed enough that somebody else can take over the parts I work on, some day. Maybe then I can go on and work on other things in Bugzilla, or some other programming project that I have, or maybe I could go make an album! Who knows! I just don’t want to be imprisoned by my own code.

If job security is so important to you that you’d tie yourself to a single job forever just to get it, then maybe you should re-evaluate your priorities in life! Otherwise, when you’re making decisions about your project, one thing to remember is this: complexity is a prison; simplicity is freedom.

-Max

Comments: 6

For example, what’s the purpose of a web page? To give and receive information. Mostly to give information, and then some websites also take information, such as when you’re buying something.

How many complicated web pages have you seen that think they’re doing something else than giving or receiving information? Maybe they think they’re being entertaining, or something. I don’t know what their designers were thinking, but they probably weren’t thinking about giving or receiving information. Instead they’re hiding their information in a complicated mass of pictures and shapes.

Usually, the basic purpose of any given thing you’re working on is pretty simple. But if you add to that purpose, things can get complex pretty fast! For example, the basic purpose of Bugzilla is to store and organize bug reports. If we suddenly made Bugzilla also able to read your email, it would get ridiculously complicated. (Not that Bugzilla is the simplest program in the world, but we’re working on it.) Can you imagine what the UI would look like? Where would we put all the buttons? That would be a violation of Bugzilla’s purpose.

It’s also important to think about user’s purpose. Your user is trying to do something. Ideally, the purpose of a program should be very close (in the exact words you’d use to describe it) to the user’s purpose. For example, I’m trying to blog, and Wordpress is blogging software. That’s a pretty good match. Wordpress is good about not trying to do everything under the sun–it’s just blogging software, and that’s it, which I like.

Now, if your purpose and the user’s purpose don’t match up, you’re probably making their life difficult. For example, they want to read their email, but your manager’s purpose is to show them ads. Those don’t sound like quite the same purpose to me.

Want to see a user get angry really fast? Make it difficult for them to accomplish their purpose. Pop up windows in their face when they’re trying to do something, add so many features to your program that they can’t find the right one, use lots of cute little icons that have no meaning–there’s all kinds of ways to do it. But they all boil down to interfering with the user’s purpose or violating the basic purpose of the program itself.

Sometimes, marketers or managers have goals for a program that are not really aligned with the basic purpose of the program: “Be cute”, “Have an edgy design”, “Become popular with the news media”, “Use the latest technologies”–on and on. But marketers are not the people who should be deciding what your program does! As an engineer or technical manager, it’s your job to see that the program stays on track and never violates its basic purpose. Nobody else is going to hold that responsibility. Sometimes you might really have to fight for it, but it’s well-worth-it in the long run.

And it’s not like you’d come to a marketing failure with that philosophy. I mean, look at the iPod. That’s a device that sticks to a simple purpose, and it doesn’t have any marketing difficulty. You don’t have to have a complicated product to have good marketing. You just have to have good marketing.

There’s no need to get fancy and complex and try to do five hundred things at once in a single program. Users are usually happiest with a focused, simple product that never violates its basic purpose.

-Max

Comments: 0

This is why in VCI I put a big, obvious section of documentation near the top called “New To VCI?” And then it contains some simple, obvious steps to take. It’s written as if the reader knows nothing about VCI, because if you’re new to something, you probably don’t know anything about it.

I see way too many software projects mess this up. You go to read the documentation, and you’re presented with a huge mass of links and no direction. This is simple to the long-time developer of the project, because a page with lots of links lets that developer quickly go to the part they’re looking for. But for the new person, it’s complicated. On the other hand, for the long-time developer, adding a page with big, simple buttons and eliminating that list of links would add to the complexity of his task, because he’s just trying to find a very specific thing very fast in the documentation.

The only thing worse than complex documentation is no documentation, where you’re just expected to figure it out for yourself or “already know.” To the developer, the way his program works is obvious, but to the new user, it’s totally unknown.

With software, there are all sorts of different viewpoints. Just a few would be: Programmer, QA Engineer, Manager, Support Technician, User, Power User–to all of these people, “simple” will be different.

Context has a lot to do with this too. For example, in the context of a program, object-oriented architectures usually lead to simplicity, if done right. But imagine displaying that structure directly on a web page as the only interface to your program! That wouldn’t be that simple in that context, even if you were the developer!

You can be a little less bad than that, and still be complex. Some programmers design their user interfaces in a way that’s very closely tied to the backend architecture of the program. There’s three fields in the database, so they display three fields on the page, even if, for the user, one field containing three values would be simpler. This is “simple” to the developer, because (a) it’s easy to write and (b) it closely maps to how he or she imagines the program working internally. But when you go and try to actually use the program, it would be simpler for everybody if it was all just one field instead of three. (A classic example of this is how we display “status” and “resolution” as separate fields in Bugzilla’s user interface, when users couldn’t care less that they’re separate fields.)

Also, sometimes what seems complex is actually simple in the right context. On a billboard by the side of the road, a lot of explanatory text would be very complex! There’s just no time to read all that text, it would be stupid to put it there. But in a manual for a computer program, lots of text would probably be a lot simpler than just a one-sentence description of something. That’s why people don’t just write one-line blogs. It wouldn’t really be all that simple to just say something and then not give any description of it.

So…man! All these different viewpoints and contexts! Does this mean that simplicity is impossible, and that everything is hopeless? No! Not at all. There are specific target audiences for everything, and the context of any indvidual thing you’re doing is usually pretty limited. The problem is always solvable. It’s just important to take these things into account, so that when your work comes out the other end and it’s actually being used by somebody, it really is simple for that person.

-Max

Comments: 4

The brighter somebody is, the better they can accurately predict the future.

However, no matter how smart somebody is or how much they know, there are some things about the future that you can not know.

In programming, the biggest thing you can’t know is how your program will change in the future. You can know that it will change–that’s guaranteed. But how it will change five, ten, or twenty years from now, we just don’t know.

A common mistake that developers make is designing too far into that unknown future, making too many assumptions about it. Everybody’s done it, probably. I’ve done it. You set up some huge, rigid framework and plan, and you get started setting it up, and either:

• It doesn’t work and you have to re-write it all, or
• Large parts of it never get finished, and then there are little useless pieces sitting around in the code that were “going to be used” but never were, or
• After years of work you find that you’ve designed yourself into a hole and have to re-write the whole thing.

I did that once, in Bugzilla, in a simple but stupid way. In Bugzilla, there are drop-downs where the user can make selections. The list of choices can be customized by the Bugzilla administrator. I added an is_active column to the database, which was eventually going to allow the administrator to disable the selection of certain choices.

Well, that was three years ago, and you still can’t disable those choices in Bugzilla. But that database column is still sitting there, confusing everybody.

The right way to go about development, particularly in an open source project, is to design for the specific requirement (singular) that you know is broadly needed, and implement that on top of the system you already have. If that can’t be done, or can’t be done with simple code, then you do re-architecture. Rinse and repeat.

Violations of that principle lead to crazy, messy code that nobody wants to maintain after a few years. People design massive changes that do everything in some extremely specific way, based on a nearsighted design. Then, much later when somebody comes up with some feature that you never thought of but would be extremely useful, you can’t implement it because you’ve got this unwieldy very-specific design that handled only what you needed at the time, and can’t be extended to do what you need now.

What do I mean by “nearsighted design?” Basically, code should be designed on what you have, not on what you think you’ll have in the future. It should be designed for the requirement that you have right now, without excluding the possibility of future requirements. If you know for a fact that you need it to do X, and just X, then just design it to do X.

A general example of this problem is Bugzilla’s BugMail.pm file. It does everything and the kitchen sink, and was pulled out of an old script called “processmail” which basically did the same thing. Instead of being carefully designed and engineered piece by piece, it was designed way back when as a single, massive whole. As a result, nobody fully understands it today, and it’s pretty difficult to add new features there. Want to make Bugzilla send HTML mail? Ohhh, good luck. As a result, I’m going to have to redesign the module, when it should have just been designed in the first place.

The original authors could have designed it simply, piece by piece, the first time, and entirely saved me the time I’m going to spend fixing it. The amount of extra time required to get it right the first time would have been less than the time it will now take me to fix it.

It’s fine to design for what you need at the time, as long as it’s a small design. So, I just make all designs into small designs, and we’re fine. That’s why I’m so adamant about “one bug report for one issue.” Because it encourages small designs, and it keeps code maintainable.

Maybe for some developers it’s just a misunderstanding of what the word “design” means in programming:

When you have a brand-new product that you haven’t even made yet, first you find out what its requirements are. Then you write a program that fulfills those requirements. You write it in a way that it can be updated in the future for unknown requirements. That’s where design falls in. You look at an individual requirement and you say, “Okay, how can we make this so that it works perfectly, and we can also extend it for unknown requirements in the future?” If you’re at the point of the initial product design (that is, you don’t have a product yet), you’ll probably also want to look at the other requirements and make sure that your design isn’t going to impede those. However, if you actually design any individual requirement correctly (or any piece of your program correctly), it’s unlikely that you’ll interfere too much with other requirements.

This is how many, many people can be working on an open source project simultaneously without knowing what the others are doing. They design their code so that it doesn’t matter what’s going on anywhere else. The code just works, and can be modified and extended easily by anybody. There are lots of ways to do this–they mostly fall under the rules for good object-oriented design. You can find those elsewhere, so I’m not going to go into them here.

Some people think design means, “Write down exactly how you’re going to implement an entire project, from here until 2010.” That won’t work, brother. That design is too rigid–it doesn’t allow your requirements to change. And believe me, your requirements are going to change.

Planning is good. I should probably do more planning myself, when it comes to writing code.

But even if you don’t write out detailed plans, you’ll be fine as long as your changes are always small and easily extendable for that unknown future.

-Max

Comments: 8

Computers have created a major societal change. The reason is that you can do more work with fewer people. That’s really the entire value of a computer—it can do a lot of work, really fast. If a person was to do, by hand, all the math that a computer does just when it starts up, it would probably take the rest of that person’s life.

So that’s great.

Problem is, they break. They break all the time. If anything in my house broke as frequently as my computer, I would return it. Most of the people that I know, their computer crashes at least once a day. Almost every day, I see a computer break in a way that I’ve never seen before. That’s been pretty much every day since I was about eight years old, so I’ve probably seen a computer break over 41,000 different ways, now.

That’s not great.

Why do computers break so much? For software, there’s one reason, and one reason only. Bad programmers.

Now, I didn’t used to be a programmer, and so I wondered about this sort of thing. I suspected that there were bad programmers, but it was sort of like blaming “witches” for a bad crop harvest. I didn’t really know anything about the subject, so there was some reasonable doubt.

Now that I am a programmer, and I have worked for a long time in a professional setting, and have talked extensively to other people who have been professional programmers for a long time, I can confirm that it really is bad programmers.

So, what is a bad programmer and why would somebody be one? This term, “bad programmer,” is pretty ambiguous. Also, most of the people I’ve ever met aren’t totally illogical, so there must be some reason why they would do “bad” programming.

Basically, it all revolves around complexity.

A computer is probably the most complex device that we can make in a factory today. It does billions of things a second. It has hundreds — if not thousands — of tiny parts that all must function correctly in order for it to work.

A program written on that computer is equally as complex. For example, when it was written, Microsoft Windows 2000 was the longest program ever created, at somewhere around 20 million lines. That’s something like writing a book of 150,000,000 words—nearly four times the size of the Encyclopedia Britannica.

The complexity of a program can be particularly confounding, because there isn’t anything to put your hands on. When it breaks, you can’t pick up something solid and look around inside it. It’s all abstract, and that can be really hard to deal with. In fact, the average computer program is so complex that probably no person could actually comprehend the entirety of how all the code works, all at once. The bigger they get, the more this is the case.

Thus, programming has to become the act of reducing complexity to simplicity. Otherwise, nobody could keep working on a program after it reached a certain level of complexity. The complex pieces have to be organized in some simple way so that a programmer can work on it without godlike mental abilities.

That is the art and talent involved in programming—reducing complexity to simplicity.

A “bad programmer” is just somebody who fails to reduce the complexity. Many times this happens because people believe that they are reducing the complexity of just writing in the programming language (which is definitely a complexity all in itself) by writing things that just “work,” without thinking about reducing the complexity for other programmers that they are working with.

It’s sort of like this:

Imagine you have an engineer, who, in need of something to pound a nail into the ground with, invents a device involving pulleys, strings, and a large magnet. Well, you’d probably think that was pretty silly.

Now imagine that I tell you: “I need some code that I can use in any program, anywhere, that will communicate between any computer and any other computer, using any medium imaginable.” That’s definitely harder to reduce to something simple. So, some programmers in that situation (and I might even say most programmers, but I don’t know all of them, so I can’t say) come up with something that involves strings and pulleys and a large magnet, that is only barely comprehensible to other people. They’re not irrational, and there’s nothing wrong with them. They were faced with a really difficult task, and they did what they could in the short time that they had. And what they made works, as far as they’re concerned. It does what it’s supposed to do. That’s what their boss wants, and that’s what their customers seem to want, as well.

But one way or another, they still failed to reduce the complexity to a simplicity. Then they passed this device off to another programmer, and that programmer also added to the complexity by using that device as part of his/her device. The more people who don’t reduce complexity, the more incomprehensible the program becomes.

As the program approaches infinite complexity, it is impossible to find all the problems with it. Jet planes cost millions or billions of dollars because they are close to this complex and were “de-bugged.” But software only costs about 50 or 100 bucks, usually, and is more complicated. There’s no way to spend time shaking all of the problems out of this infinitely complex device that’s being created.

So, a “good programmer” will do everything in his/her power to make what he writes as simple as possible to other programmers. A “good” programmer creates things that are easy to understand, so that it’s really easy to shake all the bugs out.

Now, sometimes this simplicity is misunderstood to mean that programs should not have a lot of code. Or that programs shouldn’t use advanced technologies. But that’s not true. Sometimes a lot of code actually leads to simplicity; it just means more writing and more reading, which is okay. You have to make sure that you have some short document explaining the big mass of code, but that’s all part of reducing complexity. Also, usually more advanced technologies lead to more simplicity, even though you have to learn about them first, which can be some trouble.

Some people believe that writing in a simple way takes more time than quickly writing something that “does the job.” Actually, it turns out that it’s faster to spend a little more time making things simple, as compared to writing things quickly at the beginning and then spending a lot of time trying to understand them later. That’s a pretty big simplification of the issue, but it turns out to be the case by programming-industry history. Many great programs have stagnated in their development over the years just because it took so long to add a feature to the complex beast that they had become.

And that is why computers break so often, because in most major programs out there, many of the programmers on the team failed to reduce the complexity of the part they were writing. Yes, it’s difficult. But that doesn’t make it okay to write complex software.

Comments: 2