Designing Too Far Into The Future

There is only a certain amount of the future you can know. You can know with some certainty that you’ll be breathing in the next few minutes. You can know with some certainty that if you go out to your car and push the gas pedal, the car will probably go somewhere.

The brighter somebody is, the better they can accurately predict the future.

However, no matter how smart somebody is or how much they know, there are some things about the future that you can not know.

In programming, the biggest thing you can’t know is how your program will change in the future. You can know that it will change–that’s guaranteed. But how it will change five, ten, or twenty years from now, we just don’t know.

A common mistake that developers make is designing too far into that unknown future, making too many assumptions about it. Everybody’s done it, probably. I’ve done it. You set up some huge, rigid framework and plan, and you get started setting it up, and either:

  • It doesn’t work and you have to re-write it all, or
  • Large parts of it never get finished, and then there are little useless pieces sitting around in the code that were “going to be used” but never were, or
  • After years of work you find that you’ve designed yourself into a hole and have to re-write the whole thing.

I did that once, in Bugzilla, in a simple but stupid way. In Bugzilla, there are drop-downs where the user can make selections. The list of choices can be customized by the Bugzilla administrator. I added an is_active column to the database, which was eventually going to allow the administrator to disable the selection of certain choices.

Well, that was three years ago, and you still can’t disable those choices in Bugzilla. But that database column is still sitting there, confusing everybody.

The right way to go about development, particularly in an open source project, is to design for the specific requirement (singular) that you know is broadly needed, and implement that on top of the system you already have. If that can’t be done, or can’t be done with simple code, then you do re-architecture. Rinse and repeat.

Violations of that principle lead to crazy, messy code that nobody wants to maintain after a few years. People design massive changes that do everything in some extremely specific way, based on a nearsighted design. Then, much later when somebody comes up with some feature that you never thought of but would be extremely useful, you can’t implement it because you’ve got this unwieldy very-specific design that handled only what you needed at the time, and can’t be extended to do what you need now.

What do I mean by “nearsighted design?” Basically, code should be designed on what you have, not on what you think you’ll have in the future. It should be designed for the requirement that you have right now, without excluding the possibility of future requirements. If you know for a fact that you need it to do X, and just X, then just design it to do X.

A general example of this problem is Bugzilla’s BugMail.pm file. It does everything and the kitchen sink, and was pulled out of an old script called “processmail” which basically did the same thing. Instead of being carefully designed and engineered piece by piece, it was designed way back when as a single, massive whole. As a result, nobody fully understands it today, and it’s pretty difficult to add new features there. Want to make Bugzilla send HTML mail? Ohhh, good luck. As a result, I’m going to have to redesign the module, when it should have just been designed in the first place.

The original authors could have designed it simply, piece by piece, the first time, and entirely saved me the time I’m going to spend fixing it. The amount of extra time required to get it right the first time would have been less than the time it will now take me to fix it.

It’s fine to design for what you need at the time, as long as it’s a small design. So, I just make all designs into small designs, and we’re fine. That’s why I’m so adamant about “one bug report for one issue.” Because it encourages small designs, and it keeps code maintainable.

Maybe for some developers it’s just a misunderstanding of what the word “design” means in programming:

When you have a brand-new product that you haven’t even made yet, first you find out what its requirements are. Then you write a program that fulfills those requirements. You write it in a way that it can be updated in the future for unknown requirements. That’s where design falls in. You look at an individual requirement and you say, “Okay, how can we make this so that it works perfectly, and we can also extend it for unknown requirements in the future?” If you’re at the point of the initial product design (that is, you don’t have a product yet), you’ll probably also want to look at the other requirements and make sure that your design isn’t going to impede those. However, if you actually design any individual requirement correctly (or any piece of your program correctly), it’s unlikely that you’ll interfere too much with other requirements.

This is how many, many people can be working on an open source project simultaneously without knowing what the others are doing. They design their code so that it doesn’t matter what’s going on anywhere else. The code just works, and can be modified and extended easily by anybody. There are lots of ways to do this–they mostly fall under the rules for good object-oriented design. You can find those elsewhere, so I’m not going to go into them here.

Some people think design means, “Write down exactly how you’re going to implement an entire project, from here until 2010.” That won’t work, brother. That design is too rigid–it doesn’t allow your requirements to change. And believe me, your requirements are going to change.

Planning is good. I should probably do more planning myself, when it comes to writing code.

But even if you don’t write out detailed plans, you’ll be fine as long as your changes are always small and easily extendable for that unknown future.

-Max

10 Comments

  1. Max,

    Great article. I’m working on a project right now and find myself spending way too much time on certain aspects. I have to sit back for a minute and ask myself if I should really be doing this and if it’s absolutely necessary.

    I guess I’m just lazy… but in the good way. I want everything to be perfect right away so that I don’t have to spend much time maintaining it in the future. I’ve got the end goal in mind, but like you say, perhaps it is just a little too far into the future.

    Trevor

    • Yeah, I’ve heard that before and it’s quite an amusing way of expressing part of the concept. 🙂 I think YAGNI is a slightly more limited concept than everything I’m talking about, but I pretty much agree with it completely in theory. 🙂

      -Max

  2. In programming, everything we do is a special case of something more general – and often we know it too quickly. —Alan Perlis, “Epigrams on Programming”

    (If you haven’t read the epigrams, you may want to look for a copy on the net. Every programmer should see them at least once, preferably periodically.)

    • Hey, somehow I didn’t notice this comment until just now (almost a year later)! I wouldn’t mind if you put it in your design wiki, at all! 🙂 Just credit me somewhere visible, and link back to the original article here.

      -Max

Leave a Reply