Complexity and the Wrong Solution

Often, if something is getting very complex, that means that there is an error somewhere far below the level that things are getting complex on.

For example, it’s very difficult to make a car move if it has square wheels. You’re going to be spending lots and lots of time figuring out how to make the car work, when really it should just have round wheels.

Any time there’s an “unsolvable complexity” in your program, it’s because there’s something fundamentally wrong with it. If the problem is “unsolvable” at one level, maybe you should back up and look at what’s underlying the problem. Maybe you put square wheels on the car, and now you’re trying to figure out how to make it go fast.

Programmers actually do this quite often. For example, “I have this terribly messy code, now it’s really complex to add a new feature!” Well, your fundamental problem there is the that code is messy. Clean it up, make the already-existing code simple, and suddenly adding the new feature will be simple.

What Problem Are You Trying To Solve?

If somebody comes up to you and says something like, “How do I make this pony fly to the moon?”, the question you need to ask is, “What problem are you trying to solve?” You’ll find out that they really need to collect gray rocks. Why they thought they had to fly to the moon, and use a pony to do it, only they know. People do get confused like this.

So when things get complex, back up and you look at the problem that you’re trying to solve. Take a really big step back. You are allowed to question everything. Maybe you thought that adding 2 and 2 was the only way to get 4, and you didn’t think about adding 1 and 3 instead, or just skipping the addition entirely and just putting “4” there. The “problem” is “How do I get 4?” Any method of solving that problem is acceptable, so figure out what the best method would be, for the situation that you’re in.

Discard your assumptions. Really look at the problem that you’re trying to solve, and think about the simplest way to solve that problem. Not “How do I solve this problem using my current code?” Not “How did Professor Bob solve this problem in his program?” No, just how, in general, in a perfect world, should that problem be solved? From there, you might see how your code needs to be re-worked. Then you can re-work your code. Then you can solve the problem.

-Max

18 Comments

  1. Sounds familiar. When dealing with customer requests, I frequently find that their ‘requirement’ is that a particular feature is changed in a particular way, typically without any clear explanation of why. On investigation, it’s often found that the request wouldn’t actually solve their problem, and if they’d explained the reasons in the first place, a better solution would present itself.

  2. My favorite is that ‘google is giving me the wrong results’ … just how do they expect me to fix that? Break into google and recode their software?

    monk.e.boy

  3. In my experiences, this often leads to the classic power struggle between gold plating and refactoring, which I find is surprisingly sparse covered in our industry. Often when something is working, we are told to ship and it can be quite a battle convincing management that the code base is a mess, that unit tests are required or that perhaps a complete rewrite is in order to ensure a truly sound and future proof base.

  4. @Casper – ask any developer on a project (especially one new to the project) whether they’d prefer to fix the codebase or rewrite it. The majority of the time you’ll get ‘rewrite’ as the answer. Rewrites are expensive, time consuming, and rarely result in a “truly sound and future proof base”. You can’t future proof code any more than you can predict the future. Future proofing is the reason people later ask “Why is this code so bloated and ridiculously complicated?”

    Besides, think about what you are telling management: we’ve spent all your money making a piece of crap and now we’d like you to spend it again so we can make a different piece of crap that does the same thing. Good luck! A better request would be “please spend 2-3 times what you just spent so we can analyze our practices and determine how we can avoid making the mistakes that got us here in the first place.”

    Max — great entry, I wholeheartedly agree. However, you haven’t touched on the core of this problem I believe. You see, while the customer frequently asks for things that are unnecessarily complicated, it is the architects and developers who more often produce the complexity that eventually brings down the ship. Technical folks tend to love complex things. It’s part of their nature. Even things which are born of simplicity eventually become — if left to their natural progression — too complicated. Layering, “future proofing”, complicated callback schemes, etc.

    What’s nice is that simple solutions are often the answer that survives — much like a game of Life where stronger children survive while weaker ones die out. Take Google, for example, which greatly simplified search. It focused on the primary need — finding things — and did that well without all the bloat. Most of Google’s products fall into this pattern of simplifying things and attempting to stay simple, while other companies take the opposite approach, cramming as many features as they can into a product and then attempting to make it scriptable, embeddable, etc.

    Business folks and developers alike could do with a big dose of YAGNI (You Aren’t Going to Need It). Focus on the simple need at hand, solve it in a simple way, and move on to the next real requirement. Refactor, repeat. This will often prevent the problems Casper alluded to, and if you iterate in small pieces (an agile philosophy), you’ll end up with what the customer needs rather than a nest of spaghetti code and spaghetti requirements.

    • One of the goals of the book that I’m writing (of which this is a chapter, along with various other blogs that I’ve posted) is to give people the knowledge they need to avoid re-writes. It is possible to future-proof something in a way that doesn’t overengineer. I’m a pretty strong proponent of refactoring. If there was any code base that “looked like it should be re-written”, it was Bugzilla, and I successfully led a refactoring effort there.

      Ah, yes, I agree with the complexity problem! I touched on it two earlier blogs, and it’s pretty much one of the primary focuses of my book.

      And you’re completely right in the last paragraph, too. 🙂 Reminds me of another blog I wrote.

      -Max

  5. @Dan – The fact is that during development, lessons are learned and only then are the actual requirements often nailed down. Furthermore, our world does not stand still, especially in the Java space where I am. One month it might be IDE A, plugin B, tool C, framework F, library E, F and G but the next it will have been replaced or deprecated by something else entirely.

    Most of us do not have the luxury of Google’s in-house, bottom-up driven approach with beta periods lasting years but have real customers with vague requirements and feature fever. I wholeheartedly agree with you regarding KISS, but I also find it to be somewhat unrealistic in the typical top-down complex environment most of us are subjected to. I continue to try to push for the right blend of “simplexity” in my organization but the problem is that we all view that differently. For example, I find the new LINQ stuff and the RoR conventional style to be vastly superior to the monster JPA/EJB/JSF black-box stacks we’re using but that’s not exactly a popular sentiment.

    • Ah, I’ve seen that all many times, yes!

      Changing requirements can be dealt with by keeping things simple. I can’t go into it in detail here, but that’s the basic idea.

      Vague requirements should be dealt with by clarifying them. I write requirements documents all the time, and that is how I deal with it. You can go back to the customer and ask them what they mean.

      Simplicity is not a popular sentiment because many developers don’t understand the basic principles behind it, or why you would do it. People also tend to reject technologies that they don’t already know about, particularly if they know one technology very well already.

      -Max

  6. Nice entry,

    but sometimes complexity is out of your control and not of technical nature.
    If you for example face a complex business model, you may have to come up with a model that reflects that complexity even if you adopt sound design principles (e.g. loose coupling, …), methods (e.g. communicating code) or whole paradigms (OO, DDD, …).

    Of course, complexity shouldn’t lie in the applied programming techniques or design decisions itself, but often comes from the underlying enterprise (i think we all have seen not only one company that describes a business model where their ‘pony’ have to fly to moon … and that’s their right since they will make money with that business model and pay their employees :o)).

    as someone said correctly: technology shouldn’t be the problem, its the business that matters … and sometimes that business is complex per se …

    Greetings

    Mario

    • Hm, I’ve never encountered a system whose legitimate requirements couldn’t fit into the fourth law.

      Also, just because some complex products survive in the short term (like 10 years) doesn’t mean that they have actual staying power. I think complexity can be reduced from requirements as well as from the design.

      -Max

  7. Max,

    thanks for your reply – i think we may speak about different issues. I’m completely on your side when it’s about introducing (or better not introducing) unnecessary complexity by technology. So i’m not talking about under- or over-engineering a technical solution.

    My comment is rather about complexity that’s driven by forces you can’t control from the technological point of view (and even may be not under control of the company). Take the field of ‘compliances’ in the financial market (Basel II, MiFID, SOX, AML to name a few) for example.
    Fullfilling those compliances will introduce ‘some’ complexity that you have to face. And by reflecting those rules, some ‘pieces’ (refering to your fourth law) of your code will reflect those complexity.

    Of course you don’t want to add additional technological complexity by choosing appropriate design decisions (considering the given forces), but the non-technological complexity is inherent.

    Of course you could argue that complexity is bad and won’t pay of in the long term per se, but unfortunately you’ll sometimes have to accept it, because it’s not under your control (even with self made complexity by the company – as long as their business model is successful, they’ll go along with it. Even for short terms of 10 or 20 years. Business will drive the requirements and thus sometimes complexity, IT have to follow).

    Greetings

    Mario

    • Hey Mario. Yeah, granted, some tasks are complex. Compliances are a very good example, because they’re a sort of “ridiculous thing you can’t eliminate,” since as you’ve pointed out, the requirements are beyond your control. Now, at that point I’d argue that the “wrong solution” was made during the designing of such compliances–that is, the principle still holds, but it is unfortunately beyond your control (though sometimes there are standards bodies that it can be discussed with).

      Such complexities almost certainly wouldn’t hold up to the “What problem are you trying to solve?” test if put to the authors of the complexities. 🙂 They might argue that their complexities are necessary, but in that case I’d argue that they’re solving their problems on the wrong level, and that they ought to be the ones backing up a few steps from their problem and looking at it.

      However, you’re right that given the scope of control of the average software designer, those things are out of control. Usually what I do (and what I’d imagine most designers would do, in such a situation), is to somehow “isolate” those complexities from the rest of the system, so that the pieces of the system itself remain simple, and the complex parts don’t “infect” the whole design. 🙂

      -Max

  8. […] Code Simplicity » Complexity and the Wrong Solution “If somebody comes up to you and says something like, “How do I make this pony fly to the moon?”, the question you need to ask is, “What problem are you trying to solve?” You’ll find out that they really need to collect gray rocks. Why they thou (tags: programming design) […]

  9. […] Code Simplicity » Complexity and the Wrong Solution “If somebody comes up to you and says something like, “How do I make this pony fly to the moon?”, the question you need to ask is, “What problem are you trying to solve?” You’ll find out that they really need to collect gray rocks. Why they thou (tags: programming design) […]

  10. […] Code Simplicity » Complexity and the Wrong Solution If somebody comes up to you and says something like, “How do I make this pony fly to the moon?”, the question you need to ask is, “What problem are you trying to solve?” You’ll find out that they really need to collect gray rocks. Why they thought they had to fly to the moon, and use a pony to do it, only they know. People do get confused like this. […]

  11. On the internet, I see tremendous abuse of “What problem are you trying to solve?” I see it all the time on discussion forums where a person has a very specific difficult problem that they want to solve. People don’t know the answer to the posters question, so they try to appear intelligent by asking “What problem are you trying to solve?” where they could then proceed to pontificate and pronounce on much better ways of solving the underlying problem. Instead they just show me that they can’t answer the poster’s question, and to me they appear stupid, not smart or insightful. It is trivial to often ask “What problem are you trying to solve?” I think there is also an air of smug superiority in asking that question. Something along the lines of, “Of course I can help you with the current problem that you are trying to solve, but you are so stupid that you are trying to solve the wrong problem. Why don’t you let me use my superior intellect to tell you what problem you should really be solving.”

    I’m not saying don’t ever ask the question, but I am saying be smart about using it, or you are just being a jerk.

Leave a Reply