On Technical Debt and the Psychology of Risk Taking

Posted on October 6, 2013 by Hayim Makabee

roulette-wheel-320x180I recently read the following message in a developers’ forum (rephrased here):

“I’m facing a dilemma. I must deliver the product by the end of this quarter, but there are still many tests I would like to execute. The product seems to be working, but even so I would like to run additional tests. However, if I do so I will miss the deadline. What should I do?”

Every programmer has been faced with this dilemma. And we all know what always happen in this case: The product is delivered on time, even with the risk of having some bugs. Meeting the deadline is considered more important than assuring 100% correctness.

There are many other situations in which a developer may make sacrifices to meet the deadline. This happens because the developer’s work is Multi-Objective. He must:

  • Deliver on time.
  • Deliver correct code.
  • Deliver code with good design.
  • Deliver well-documented code.

Frequently, there is not enough time to invest in all objectives, so some of them are assigned lower priority. This is the root of Technical Debt: A developer decides to postpone some task he should do now, because he does not have enough resources to satisfy all his objectives.

Now the question is: How do programmers decide what to postpone? How do they assign priorities to their conflicting tasks?

We would like to imagine that these decisions are being taken consciously and systematically. However, there are several unconscious and psychological influences. One of them is known as Loss Aversion.

Loss Aversion

Loss Aversion and its consequences are discussed by Daniel Kahneman in his book “Thinking, Fast and Slow”.

“Loss Aversion refers to people’s tendency to strongly prefer avoiding losses to acquiring gains. Some studies suggest that losses are twice as powerful, psychologically, as gains.”

In one experiment people are presented with the following options:

  • A certain loss of $100.
  • A 50% risk of losing $200, with a 50% chance of not losing anything.

The first option represents the sure loss of a smaller amount, and the second option represents a risk of losing a higher amount. It was proved that most people prefer the second choice. In other words, in this situation most people would take the risk of losing a higher amount. As Kahneman’s writes:

“In bad choices, where a sure loss is comparable to a larger loss that is merely probable, diminishing sensitivity causes risk seeking.”

Let’s now see how this applies to software developers and their multiple objectives.

Risk Taking in Software Development

Going back to the programmer’s dilemma we presented in the beginning of this article, suppose a software developer is presented with the following choices:

  • Continue testing the system and miss the deadline.
  • Deliver the system as-is, with the risk of a serious bug being found afterwards.

The first option is a sure loss: The deadline is missed. The second option represents the risk of a higher loss: Perhaps the system has a serious bug, but perhaps not. Thus, because of Loss Aversion, the second option will be chosen by most programmers.

What makes things worse is that, for some objectives, quality is not being measured. So when a programmer decides not to invest in a task, he is not only taking a risk on the consequences of this decision. Actually, programmers gamble with the probability that their lack of investment will never be detected.

For example, it is hard to measure design quality. Very often, code-reviews are not systematic and technical debt is considered an acceptable “fact of life”. Technical debt may never be “paid”, and when it is paid, it will probably be paid by someone else.

It is also hard to measure documentation quality. There are not appropriate tools, and sometimes documentation is considered “nice-to-have”. Some programmers believe that documentation is superfluous and that code should be self-explainable.

Regarding testing, traditional development uses QA teams and it is expected that code will have bugs. Developers are not supposed to deliver 100% bug-free code.

However, in Test-Driven Development (TDD), developers are responsible for unit testing, and if the tests don’t run the code cannot be delivered. The programmer has an immediate feedback on the quality of his code, so there is no place left for risk taking.

Conclusions

In order to improve the quality of software systems, we should avoid risk-taking behaviors. Because of Loss Aversion, if we want to avoid risk-taking, each software development task should have its own tools and methods that allow programmers to receive immediate feedback on the quality of the code they are writing.

Posted in Agile, Efficacy, Psychology of Programming, TDD | Tagged , , , | 27 Comments

The Etrog, Idealism and Concrete Choices

Posted on September 16, 2013 by Hayim Makabee

etrogIn the holiday of Succot, Jews make a blessing over the Four Species, including a special fruit called the Etrog. The Etrog is similar to a lemon, but it has a different shape and delicious smell. In the past the Etrog was considered a rare fruit, and it could be very expensive. When observant Jews chose an Etrog for the blessing, they looked for a beautiful one, which is called an “Etrog Mehoudar”.

There is an old tale about an Etrog dealer who arrives in a small village in Eastern Europe. The habitants of this village wanted to buy beautiful Etrogs to say the blessing, but they were not sure about how to choose a good one. So they would ask the Etrog dealer to give them a beautiful Etrog, and then go ask the Rav if it is really a good one.

When they arrived in the Rav’s house with the Etrog, they would ask:

“Rav, is this Etrog beautiful? Is it Mehoudar?”

Then the Rav would examine carefully the Etrog and answer:

“I’m sorry, but this Etrog is not Mehoudar.”

Then the person would return the Etrog to the dealer, claiming that the Rav said it was not Mehoudar.

After this happened several times, the dealer understood that he would not be able to sell any of his Etrogs, and then he decided to talk to the Rav. He went to his house and said:

“Rav, you must help me, no one wants to buy my Etrogs.”

The Rav answered:

“I’m sorry, but if they ask me if the Etrog is Mehoudar, I must say the truth.”

Then the dealer insisted:

“But Rav, if no one buys my Etrogs I will lose everything, and no one will have an Etrog to say the blessing.”

The Rav analyzed the situation and then proposed:

“Instead of giving a single Etrog to each person, you will give two Etrogs. Then, when they come to me, they will ask: ‘Which one among these two Etrogs is the most Mehoudar?’ And in this case I will always have a positive answer.”

Idealism and Concrete Choices

The Etrog Mehoudar represents our ideals. It is a symbol of perfection, of the satisfaction of all our desires. It is extremely important to have ideals, but we must always remember that we are constrained by the concrete choices we have at a particular moment.

It is important to have an Etrog Mehoudar, but when the holiday of Succot starts it is more important to have some Etrog to say the blessing, any Etrog, even if this Etrog is not Mehoudar.

As in the quote: “A goal is a dream with a deadline.” – Napoleon Hill

If the Etrog Mehoudar is the dream, then the beginning of Succot is the deadline. The goal is to have some Etrog to be able to say the blessing.

In the solution proposed by the Rav, the two Etrogs offered by the dealer are the unique concrete choices. The two choices are a limitation, but also part of the solution.

At any specific point in time, we must be wise to understand what our concrete choices are, and then select the best one based on our ideals.

Idealism and Software Development

Software developers are proud of the products they create, and they always want them to be as good as possible. A good piece of software should have many quality attributes: It should be well designed, should be efficient, and should not have bugs. But in our quest of the perfect software, it is easy to miss the deadlines.

One symptom of excessive perfectionism in software development is Analysis Paralysis: “The state of over-analyzing (or over-thinking) a situation so that a decision or action is never taken, in effect paralyzing the outcome.” In this case, software developers may spend too much time writing documents and drawing diagrams instead of writing code.

Another symptom is Featuritis: “The ongoing expansion or addition of new features in a product.” In this case the product will never be ready, because there will always be a new feature which may be added to make it better. Just another feature to make it perfect…

A solution to avoid these mistakes is to focus on a product that is both small and has very concrete purposes. This is the idea behind the concept of Minimum Viable Product, proposed by Eric Ries in his book “The Lean Startup”.

In summary:

There is no contradiction between idealism and pragmatism, as long as:

  • We are able to identify our concrete choices.
  • We use our ideals to select the best choice.

If you celebrate Succot, I wish you a Chag Sameach!

Posted in Efficacy, Jewish Sources | Tagged , | 8 Comments

Simplicity in Software Design: KISS, YAGNI and Occam’s Razor

Posted on August 5, 2013 by Hayim Makabee

paperclip“Perfection is achieved, not when there is nothing more to add, but when there is nothing left to take away.” – Antoine de Saint-Exupery

Simplicity is a desirable quality attribute in any software system. In this article I try to collect the main software design principles that address the importance of simplicity.

KISS

KISS means “Keep it simple, stupid”. It is probably one of the oldest principles of software design (but we keep forgetting it).

“The KISS principle states that most systems work best if they are kept simple rather than made complex; therefore simplicity should be a key goal in design and unnecessary complexity should be avoided.”

Software systems must be maintained by human developers with limited capabilities, thus any increase in a system’s complexity also augments the difficulty to maintain it.

“Debugging is twice as hard as writing the code in the first place. Therefore, if you write the code as cleverly as possible, you are, by definition, not smart enough to debug it.” -Brian Kernighan

YAGNI

YAGNI means “You Aren’t Gonna Need It”.  It is a principle of Extreme Programming (XP) that states that a programmer should not add functionality until really necessary.

“Always implement things when you actually need them, never when you just foresee that you need them.” – Ron Jeffries

The YAGNI principle is related to the “Last Responsible Moment” concept of Lean Software Development:

“Decide as late as possible: Delaying decisions as much as possible until they can be made based on facts and not on uncertain assumptions and predictions.”

Occam’s Razor

Occam’s Razor: “A principle of parsimony, economy, or succinctness used in logic and problem-solving. It states that among competing hypotheses, the hypothesis with the fewest assumptions should be selected.”

Michael Lant has written an interesting article about Occam’s Razor applied to software development. He says:

“Modern interpretation is often quoted as ‘The simplest solution is almost always the best’. The modern interpretation, however, misses the important point that ‘simplest’ as meant by Occam means the ‘fewest assumptions’.”

Thus, we should as much as possible avoid basing our design on assumptions. In this sense the principle of Occam’s Razor is aligned with YAGNI.

Complex Designs

When the simplicity principles above are not observed, the resulting software systems may have complexity in the form of:

  • Unneeded features.
  • Over-engineering.

The first case is that of systems with more features than necessary:

Feature Creep: “The ongoing expansion or addition of new features in a product. Extra features go beyond the basic function of the product and so can result in over-complication rather than simple design.”

Feature Creep is particularly common in software systems, in which it is relatively easy to add new features that were not part of the original design. One of the consequences of this feature expansion is known as Software Bloat.

Software Bloat: “A process whereby successive versions of a computer program become perceptibly slower, use more memory or processing power, or have higher hardware requirements than the previous version whilst making only dubious user-perceptible improvements.”

The second case of high complexity is that of systems which are over-engineered:

Over-engineering: “The designing of a product to be more robust or complicated than is necessary for its application.”

A typical example of over-engineering is that of having too many abstraction layers.

“There are two ways of constructing a software design. One way is to make it so simple that there are obviously no deficiencies. And the other way is to make it so complicated that there are no obvious deficiencies.” – C.A.R. Hoare

Simplistic Designs

Of course we want to eliminate complexity from our designs. But when trying to do a simple design, we must be very careful to avoid the trap of Simplistic Designs. This is the definition of “simplistic” from a dictionary:

Simplistic: “Treating complex issues and problems as if they were much simpler than they really are.”

A simplistic software design may be one that focuses only on present requirements and ignores future needs such as maintainability, extensibility and reusability. As I’ve discussed in a previous post, wise developers foresee the consequences of their design.

I think that the danger of simplistic design is very well expressed by this quote:

“Keep it simple, as simple as possible, but not simpler.” – Albert Einstein

What do you think? How do you cope with complexity in software systems? Please share your experience in the comments below.

Posted in Agile, Software Architecture | Tagged , | 32 Comments

IASA Israel meeting – Daniel Gross on Architectural Description Languages

Posted on July 29, 2013 by Hayim Makabee

The International Association of Software Architects (IASA) in Israel organized a special event with the participation of Dr. Daniel Gross, who talked about “Architectural Description Languages”.

Synopsis: In this talk Daniel reviews main findings of a recent international survey on the utility and adapted use of Architectural Description Languages in industry, and proposes some requirements for architectural design and documentation tools to support next generation ADLs.

Bio: Daniel received his PhD from the Faculty of Information at the University of Toronto, Canada. Daniel’s research focuses on developing socio-technical modelling and analysis techniques to support business and architectural decision-making in organizational settings. Daniel currently works as a knowledge representation researcher for a European FP7 R&D project (Risk and Cost Management during Open Source Software Adoption), and as a freelance Requirements Analyst. Daniel previously worked at the IBM in Belgium and Austria, in the areas of object-oriented methodology development and insurance business modelling, and in areas related to Workflow management. He holds UK, US and Canadian software patents to support recipient controlled communications mechanisms dealing with messaging related information overload.

These are the original slides of Daniel’s presentation:

Here is the video of the entire talk (in Hebrew):

This meeting was sponsored by SAP Labs Israel. Special thanks to Dima Khalatov who helped with the organization.

To participate in our future meetings, please join the IASA IL group on LinkedIn.

Feel free to share your comments below. Thanks!

Posted in IASA Israel, Software Architecture | Tagged , | Leave a comment

IASA Israel meeting – Hayim Makabee on Adaptable Design Up Front

Posted on July 29, 2013 by Hayim Makabee

The International Association of Software Architects (IASA) in Israel organized a special event with the participation of Hayim Makabee, who talked about “ADUF – Adaptable Design Up Front”.

Synopsis: This talk tries to answer the question: “How much Design Up Front should be done in an Agile project?” Hayim presents his approach of Adaptable Design Up Front (ADUF), describing it’s rationale, applications in practice and comparison to other approaches such as Emergent Design.

Bio: Hayim Makabee was born in Rio de Janeiro. He immigrated to Israel in 1992 and completed his M.Sc. studies on Computer Sciences at the Technion. Since then he worked for several hi-tech companies, including also some start-ups. Currently he is a Research Engineer at Yahoo! Labs Haifa. He is also a co-founder of the International Association of Software Architects (IASA) in Israel. Hayim is the author of a book about Object-Oriented Programming and has published papers in the fields of Software Engineering, Distributed Systems and Genetic Algorithms. Hayim’s current research interests are Data Mining and Recommender Systems.

These are the original slides of Hayim’s presentation:

Here is the video of the entire talk (in Hebrew):

This meeting was sponsored by SAP Labs Israel. Special thanks to Dima Khalatov who helped with the organization.

To participate in our future meetings, please join the IASA IL group on LinkedIn.

Feel free to share your comments below. Thanks!

Posted in Adaptable Design, Agile, IASA Israel, Software Architecture, Software Evolution | Tagged , , , , | Leave a comment

Adaptable Design Up Front and the Open/Closed Principle

Posted on July 4, 2013 by Hayim Makabee

The Open/Closed principle is one of the SOLID principles of software design:

“The Open/Closed Principle states that the design and writing of the code should be done in a way that new functionality should be added with minimum changes in the existing code. The design should be done in a way to allow the adding of new functionality as new classes, keeping as much as possible existing code unchanged.”

camera with lensesThe picture on the right illustrates the application of this principle to an object we know very well: Professional cameras can be attached to several types of lenses. The camera itself does not need to be changed, it is simply extended. It is “closed” but also “open”.

The Open/Closed principle was originally proposed by Bertrand Meyer: “Software entities (classes, modules, functions, etc.) should be open for extension, but closed for modification.”

strategyA simple example of the Open/Closed principle is the Strategy design pattern. In this pattern, the Strategy is a component, and the same composite class may be associated to several types of strategies. The composite class using the strategy never needs to be changed to use a new type of strategy, and in this sense it is “closed”. But it is also “open”, because its behavior can be
extended through associations to new concrete strategies.

It is easy to understand the meaning of the Open/Closed principle in the context of relationships between pairs of classes, such as inheritance and composition. It is also relatively simple to use it in low-level design, and actually many design patterns apply this principle. But it is interesting to ask ourselves how the Open/Closed principle may affect the high-level design.

The Open/Closed Principle Applied to High-Level Design

The approach of Adaptable Design Up Front (ADUF) is an application of the Open/Closed principle to high-level design. It focuses on the early definition of the “closed” aspects of the design while at the same time allows easy evolution by making this design open to extensions.

The design is “closed” when it captures the essential aspects of the system: The main entities and relationships. These are the aspects that are not expected to change.

The design is “open” when it can be easily extended without changes. Through these extensions, the closed part of the design can be adapted to changing requirements.

ADUF starts with a domain model defining the main entities and relationships. We then design a framework for this model, in which the relationships among entities are implemented as associations among classes.

The associations must be closed: An object of type A must have a reference to an object of type B. But polymorphism allows this association to be also open: An object of type A can be associated to an instance of a subclass of B, even subclasses that did not exist when A was created.

ADUF also promotes the usage of Plug-ins as an application of the Open/Closed principle to high-level design. The same module may be associated to diverse plug-ins without any need of change. The module associated to the plug-ins may appear to be completely opaque and monolithic from the point-of-view of the plug-in developer.

Enough Design Up Front

Question: How much design should be done up-front?

Answer: The design up-front should capture all the “closed” aspects of the system. Besides that, we should invest to make this design “open”, in order to allow it to evolve through specialization. This may be done with appropriate Design Patterns.

Thus, there are several kinds of mistakes when doing the design up-front:

  • Too much design up front: Capturing aspects that are not essential as if they were such. The consequence is that the design may need to change, and if it needs to change then it is not really “closed”. This also happens when specific details are not generalized. If the details change, the design must be changed.
  • Not enough design up front: Leaving away some of the essential aspects of the system. The consequence is that these aspects will need to be added to the design when other parts of the system are already implemented. If the original design was not “open” enough, the new additions may cause conflicts.
  • Not enough openness: The design captures all essential aspects, but there are not mechanisms that allow this design to be easily extended. For example, if there are Interfaces or Abstract classes that represent the main entities, there should be also creational patterns such as Factories to instantiate specific subclasses.
  • Too much openness: The design is over engineered, resulting in complexity not really necessary. For example, it is always possible to reduce the coupling between modules through additional layers of indirection. However, if there are many such layers, it becomes more difficult to locate the places in the code where the business logic is actually implemented.

In summary:

According to the ADUF approach, the design up front should be:

  • Closed: Capturing the essential aspects that are not going to change.
  • Open: Allowing the initial design to be easily extended and adapted.

What do you think? Have you used similar approaches? Please share your experience in the comments below.

Posted in Adaptable Design, Design Patterns, OOD, Software Architecture, Software Evolution | Tagged , , , , | 11 Comments

The Myth of Emergent Design and the Big Ball of Mud

Posted on June 17, 2013 by Hayim Makabee

eiffel-planSoftware developers following Agile methodologies often ask themselves how to adapt the traditional Design phase to an iterative software lifecycle. The basic question is how much design should be done before starting the implementation.

There seems to be a consensus that it is not possible to do all the design before starting to code, called Big Design Up Front (BDUF), because at the beginning of the project the requirements are usually incomplete or partially defined. Another reason to avoid BDUF is our understanding that even the requirements that were defined are likely to change.

Some people have proposed the idea of Emergent Design:

“With emergent design, a development organization starts delivering functionality and lets the design emerge. Development will take a piece of functionality A and implement it using best practices and proper test coverage and then move on to delivering functionality B. Once B is built, or while it is being built, the organization will look at what A and B have in common and refactor out the commonality, allowing the design to emerge. This process continues as the organization continually delivers functionality. At the end of an agile release cycle, development is left with the smallest set of the design needed, as opposed to the design that could have been anticipated in advance.”

I personally do not believe in this concept of Emergent Design. I think it may work only when implementing small projects or prototypes. My experience is that the development of large software systems requires a lot of effort to keep the design consistent when successive changes are made. It is a fallacy to imagine that the design will tend to naturally re-organize and re-structure itself in an orderly fashion. The reality is that software tends to break, to become more complex and deteriorate over time.

This divergence from the desired order is called Software Entropy:

“The second law of thermodynamics, in principle, states that a closed system’s disorder cannot be reduced, it can only remain unchanged or increase. A measure of this disorder is entropy. This law also seems plausible for software systems; as a system is modified, its disorder, or entropy, always increases.”

Lehman has also captured this behavior in two of his Laws of Software Evolution:

  • A computer program that is used will be modified.
  • When a program is modified, its complexity will increase, provided that one does not actively work against this.

A famous article that discusses the complexity of software systems is the Big Ball of Mud. It says:

“No one really goes into any project blindly. The groundwork must be laid, the infrastructure must be decided upon, tools must be selected, and a general direction must be set. A focus on a shared architectural vision and strategy should be established early.

Unbridled, change can undermine structure. Orderly change can enhance it. Change can engender malignant sprawl, or healthy, orderly growth.

The biggest risk associated with Piecemeal Growth is that it will gradually erode the overall structure of the system, and inexorably turn it into a Big Ball of Mud.”

So the question is: How do you design a system to support orderly changes that actually enhance its structure? My answer to this is Adaptable Design Up Front.

Posted in Adaptable Design, Agile, Refactoring, Software Architecture, Software Evolution | Tagged , , , , | 26 Comments

Meta-Decisions: Agreeing on the criteria before discussing the alternatives

Posted on June 4, 2013 by Hayim Makabee

choicesIn a working environment, meetings may consume lots of time. In general, the goal of meetings is to make decisions based on the input provided by several people. There are normally many different ways to solve a problem or address an issue. When choosing among several alternatives, the discussion is normally based on people presenting their personal opinions.

We would like to be as close as possible to a consensus, because we may not be able to claim that one person’s opinion is more important than others. But very often the participants in a meeting try to convince each other that their opinion is the right one, instead of trying to reach an agreement.

In general, the difficulty in reaching an agreement is caused by the fact that different people are forming their opinions using different criteria. Thus, the solution to reduce the time spent in discussions and to make decisions faster is to first agree on the criteria that should be applied when choosing among the several options.

For example, suppose your team is planning how to develop a new product. One person thinks that first of all you should build a prototype. Another person thinks that you should immediately start implementing the final product. A third person believes that you first need to look for existing components that may be used to build the product.

How is it possible that each person has a different opinion? After all, they are all experienced professionals that have developed such products before. The explanation is that each person is using a different criterion to form his opinion:

  • The person proposing a prototype probably wants to collect feedback from real users before he starts developing the product.
  • The person who wants to start the implementation immediately probably is concerned with the time-to-market.
  • The person who wants to reuse existing components probably wants to reduce the total development cost.

So before this team agrees on the right alternative, they must agree on the most important criterion. What is more important? User feedback? Time-to-market? Cost? After agreeing on the right criteria, choosing the right alternative is much easier.

Deciding about the criteria is what I call a Meta-Decision: It’s a decision about how to make a decision. In other words, a Meta-Decision is the choice of the rules that should be applied when making decisions about some subject or issue.

focusImagine making decisions as a dart-throwing game. If people have different criteria for making their decisions, it’s like they are all throwing their darts in different directions, as if they had different targets. In contrast, when people agree on the criteria for making a decision, they are actually agreeing on the target they want to hit.

Meta-Decisions, Trade-Offs and Software Quality Attributes

In the field of Software Engineering, developers generally need to make their decisions analyzing trade-offs.

Trade-Off: “A situation that involves losing one quality or aspect of something in return for gaining another quality or aspect.”

For example, in many situations it is possible to reduce the execution time of a function by increasing the usage of memory. This is called a Space-Time tradeoff. Therefore, when choosing the algorithms and data-structures to implement a system, the developers must make Meta-Decisions: Do we want to minimize CPU time or memory?

Meta-Decisions are also needed when choosing the focus on Software Quality Attributes. Some attributes refer to the quality of the design or the quality of the code itself, such as Maintainability, Extensibility and Reusability. But there are also dynamic attributes that refer to the system execution, such as Scalability, Throughput and Resilience. Clearly, when comparing several design alternatives it is necessary to focus on a subset of all possible quality attributes.

Advantages of Meta-Decisions

As discussed above, Meta-Decisions are useful to make discussions more effective and thus help us avoid wasting time in long meetings. But applying Meta-Decisions has several other benefits.

Awareness: When you choose an option based on a Meta-Decision, you are conscious. You know the reason you are making this decision. You are following a process.

Systematic Thinking: Using Meta-Decisions actually activates different parts of the brain. It requires Logic and Reasoning vs. Intuition and Personal Bias.

Justification: When you apply Meta-Decisions, you can justify your decisions. You can say “We chose option A because we want to optimize X. If we wanted to optimize Y, then we would have chosen option B.”

So next time you are in a meeting and people are not being able to reach an agreement, the best thing to do is to just interrupt the discussion and bring it to the Meta-level. Say: “Before we discuss the options, let’s discuss the criteria we will use to select an option.”

Do you like the idea of Meta-Decisions? Do you already use it? Please share your experience leaving a comment below.

Posted in Efficacy, Software Architecture | Tagged , | 3 Comments

Adaptable Designs for Agile Software Evolution

Posted on May 5, 2013 by Hayim Makabee

bird beaks“It is not the strongest of the species that survives, nor the most intelligent that survives. It is the one that is the most adaptable to change.” – Charles Darwin

Big Design Up Front (BDUF) is considered a very bad practice in Agile software development. BDUF is problematic because it assumes frozen requirements. The Agile approach is that software must be developed iteratively since the requirements will change and the system will evolve. So we need some Design Up Front, but this must be Adaptable design that supports change: ADUF.

Adaptable Design Up Front

We should never work on the design of a specific system. Instead, we should generalize, designing for a family of systems. Applying object-oriented concepts, our design should represent a class of systems, and each version during the system evolution should be an instance of this class. Thus each new version may have modifications and extensions when compared to previous versions, but if it still fits the same generic design, then it is still an instance of the same class.

Adaptable Software Design: A generic software design for a family of systems which does not need to be changed to accommodate new requirements.

The challenge of creating an adaptable design is making it change-resilient. Adaptable designs should evolve and accommodate new requirements through specialization. When designing for a family of systems, there are two main such approaches: The framework approach and the platform approach.

Frameworks

Framework: “A software framework is an abstraction in which software providing generic functionality can be selectively changed by additional user-written code, thus providing application-specific software.”

The framework defines the system structure: The main system components and the relationships among them. Each one of these framework components should have an abstract interface, and several possible implementations for these interfaces. If the design is sound, the components implementation may evolve over time, but the relationships among them should be preserved.

Each component in a framework should be designed to be highly cohesive, and the framework itself should be designed so that components are weakly coupled. If the component interfaces are indeed abstract and if the coupling among these components is weak, each type of component in the framework can be seen as a kind of plug-in.

Plug-in: “A plug-in is a software component that adds a specific feature to an existing software application. When an application supports plug-ins, it enables customization.”

If we are able to define such framework with diverse plug-ins, the system evolution occurs through the implementation of new versions of components. Thus, instead of having a new version of the entire system, we may have new versions for each one of the plug-ins. Building the system means simply combining these plug-in versions in an appropriate configuration.

Platforms

Platform: “Technology that enables the creation of products and processes that support present or future development.”

A platform should support the basic services which are required by the system to provide its functionality. When these basic services can be well-defined and implemented independently of the system which will use them, we have reached an ideal environment for adaptability:

  • The platform may evolve independently of the systems that are built on top of it. If the interfaces are kept intact and if the behavior is preserved, the platform developers are free to improve its efficiency and other non-functional attributes.
  • Several systems may be built on top of the same platform, using its services to provide diverse functionalities. These systems may each evolve independently of the others, and independently of the evolution of the platform itself.

Building platforms is a very effective approach for experimentation. If an essential part of the system behavior can be captured as basic services being provided by the platform, then each version of the system may be seen as an experiment and the cost of failure is low. The resources invested in the development of the platform are not lost in the case one of the system versions is not deployed.

Adaptable Software Design in Practice

Question: How do you identify the components in your framework or the services in your platform?

Answer: We want to think generally about a family of systems, and avoid defining requirements for a specific system. To identify the main entities that should compose any system in this family, we should apply domain modeling.

Domain Model: “A domain model is a conceptual model of all the topics related to a specific problem. It describes the various entities, their attributes, roles, and relationships, plus the constraints that govern the problem domain.”

Question: How do you decouple the components in your framework?

Answer: To reduce coupling, one should look for appropriate Design Patterns. Most of these patterns are related to avoiding the direct links between concrete classes, through the introduction of interfaces and abstract classes.

For example, when instantiating an object, one should avoid creating an instance of some specific concrete class. This can be done using the Factory pattern.

Summary

Software systems must be able to evolve over time. This evolution should be planned and supported trough Adaptable Software Designs. There are two main approaches for creating such designs: Frameworks and Platforms. Both require the identification of domain entities and the usage of abstract interfaces to minimize coupling between components.

What do you think? What is your personal experience with software evolution, changing requirements and adaptable designs? Please share your thoughts in the comments below.

Posted in Adaptable Design, Agile, Design Patterns, Software Architecture, Software Evolution | Tagged , , , , | 37 Comments

Outliers? The Myth of the 10,000 Hours Rule

Posted on April 4, 2013 by Hayim Makabee

neutralFollowing the “Outliers” book by Malcom Gladwell, some people are convinced that if they “just” invest 10,000 hours in something they will become really good at it. I think that this is not always the case. I don’t believe any person is able to invest 10,000 hours in anything he/she decides. I believe that challenging activities require special aptitude. And I think there are situations in which someone may spend lots of time stuck in some place, doing the same mediocre job for years. Like driving in neutral while pressing down the gas…

Question:  Can you just decide to invest 10,000 hours in anything you feel attracted to?

I believe you can only invest a lot of time in some activity if you really enjoy doing it. And I also believe that you can really enjoy an activity only if you got some talent in performing it. In other words, if you don’t have the innate ability to do some activity you will not be able to enjoy it, and thus even if you try you will simply quit much earlier than you expect.

You certainly know people who have tried all kinds of activities and then decided to quit after a relatively short period of time. And you probably also have your own personal experience of trying to do something and then deciding to quit. That may be trying to learn a foreign language, trying to play some musical instrument, trying to exercise more frequently or even trying to write your own blog.

If you had these experiences, you can ask yourself why you decided to quit. Was learning a foreign language more difficult than you expected? Was playing a musical instrument less fun than you thought? Did you enjoy exercising more frequently? Did you feel satisfaction from your blog writing?  If you decided to quit, this probably means that for these particular activities you did not have enough talent to enjoy and persevere.

One case in point is that of College drop-outs. It’s clear that if a person chose some profession is because he/she felt attracted to it, and was ready to invest a lot of time learning it, believing that with the right amount of dedication and effort success was guaranteed. However, statistics show that 33% of the students drop-out of College. Of course some of these students may have personal reasons to drop-out, such as health problems, but for most of them the reality was different from their expectations, and they simply decided to quit.

Question: May you invest 10,000 hours and still see no improvement?

I believe people can get stuck in some position, for example performing the same job for years without any personal progress or improvement in their competencies. Unfortunately, this probably happens more frequently than it should, since after a certain age people may have very limited opportunities to try something new. So they simple continue doing the same mediocre job they did in the past, because this is the only thing they know.

This phenomenon is known as the “Peter Principle”:

“In a hierarchy, members are promoted so long as they work competently. Eventually they are promoted to a position at which they are no longer competent (their ‘level of incompetence’), and there they remain, being unable to earn further promotions.”

The Peter Principle has two corollaries:

“In time, every post tends to be occupied by an employee who is incompetent to carry out its duties.”

“Work is accomplished by those employees who have not yet reached their level of incompetence.”

These people who have reached their levels of incompetence are probably not enjoying their jobs anymore. But they will continue working because they are being paid and they may not have any other alternative. So in this case they may actually spend 10,000 hours doing something, not enjoying it and not improving.

In Summary:

  • Aptitude is required: To invest 10,000 hours doing some activity you must enjoy it, and to enjoy it you need talent.
  • No guarantee of success: If people reach their incompetence level they may spend 10,000 hours doing the same mediocre job, not enjoying it and not improving.

People like to believe that if they work hard enough they can become very good at anything they want. I think that people should first understand what are the few things they are already good at, and then work hard to become even better.

What do you think? Do you agree or do you have different experiences?

Posted in Efficacy | Tagged | 9 Comments