Software developers following Agile methodologies often ask themselves how to adapt the traditional Design phase to an iterative software lifecycle. The basic question is how much design should be done before starting the implementation.
There seems to be a consensus that it is not possible to do all the design before starting to code, called Big Design Up Front (BDUF), because at the beginning of the project the requirements are usually incomplete or partially defined. Another reason to avoid BDUF is our understanding that even the requirements that were defined are likely to change.
Some people have proposed the idea of Emergent Design:
“With emergent design, a development organization starts delivering functionality and lets the design emerge. Development will take a piece of functionality A and implement it using best practices and proper test coverage and then move on to delivering functionality B. Once B is built, or while it is being built, the organization will look at what A and B have in common and refactor out the commonality, allowing the design to emerge. This process continues as the organization continually delivers functionality. At the end of an agile release cycle, development is left with the smallest set of the design needed, as opposed to the design that could have been anticipated in advance.”
I personally do not believe in this concept of Emergent Design. I think it may work only when implementing small projects or prototypes. My experience is that the development of large software systems requires a lot of effort to keep the design consistent when successive changes are made. It is a fallacy to imagine that the design will tend to naturally re-organize and re-structure itself in an orderly fashion. The reality is that software tends to break, to become more complex and deteriorate over time.
This divergence from the desired order is called Software Entropy:
“The second law of thermodynamics, in principle, states that a closed system’s disorder cannot be reduced, it can only remain unchanged or increase. A measure of this disorder is entropy. This law also seems plausible for software systems; as a system is modified, its disorder, or entropy, always increases.”
Lehman has also captured this behavior in two of his Laws of Software Evolution:
- A computer program that is used will be modified.
- When a program is modified, its complexity will increase, provided that one does not actively work against this.
A famous article that discusses the complexity of software systems is the Big Ball of Mud. It says:
“No one really goes into any project blindly. The groundwork must be laid, the infrastructure must be decided upon, tools must be selected, and a general direction must be set. A focus on a shared architectural vision and strategy should be established early.
Unbridled, change can undermine structure. Orderly change can enhance it. Change can engender malignant sprawl, or healthy, orderly growth.
The biggest risk associated with Piecemeal Growth is that it will gradually erode the overall structure of the system, and inexorably turn it into a Big Ball of Mud.”
So the question is: How do you design a system to support orderly changes that actually enhance its structure? My answer to this is Adaptable Design Up Front.