I recently read the very interesting Ph.D. thesis of Michael Waterman on the topic of Agile Software Architecture. Michael investigated how professional software engineers in the industry are applying Agile principles to software architecture and design. His conclusions are that there are five main strategies:
- Respond to Change
- Address Risk
- Emergent Architecture
- Big Design Up-Front
- Use Frameworks and Template Architectures
These are Michael’s definitions for these five strategies:
“Responding to change is a strategy in which a team designs an evolving architecture that is modifiable and is tolerant of change.”
“Addressing risk is a strategy in which the team does sufficient architecture design to reduce technical risk to a satisfactory level.”
“An emergent architecture is an architecture in which the team makes only the bare minimum architecture decisions up-front, such as selecting the technology stack and the high-level architectural patterns. In some instances these decisions will be implicit or will have already been made, in which case the architecture is totally emergent.”
“The big design up-front requires that the team acquires a full set of requirements and completes a full architecture design before development starts.”
“Using frameworks and template architectures is a strategy in which teams use standard architectures such as those found in development frameworks, template or reference architectures, and off-the-shelf libraries and plug-ins. Standard architectures reduce the team’s architecture design and rework effort, at the expense of additional constraints applied to the system.”
In his thesis Michael provides many details about each one of the strategies above. Reading his work provided me some very interesting insights. The most important of them I would like to present and discuss below.
Emergent Design vs. Framework-Based Design
I personally do not believe in Emergent Design, and thus I have always been surprised to see so many people saying that they were very successfully doing Emergent Design. But after reading Michael’s thesis, I understood that most people who claim they are doing Emergent Design in practice are doing Framework-Based design.
Today there are many categories of software systems for which there are very sophisticated and rich frameworks that already implement most of the hard work. For example, most people building Web sites do not need to understand anything about the underlying communication protocols; they just need to define the site structure, its content and some simple business logic. As a consequence, we all have heard stories about some very nice sites that were built in a few days by people with no degree on Computer Sciences or Software Engineering.
The same is true about applications for smartphones. There are frameworks that make the task of creating a new app extremely simple, to the point that the developers may completely ignore the underlying operating system and use the same code for Android or iPhone apps. There is no need for much development experience or sophisticated programming skills, and as a consequence there are teenagers creating apps and games that are used by millions of people worldwide.
This same reality applies to many domains: Software developers are not doing much design, but the reason is not that design became an obsolete activity. The reason is that developers are adopting ready-made designs; they are following design decisions that were already made by someone else and thus they don’t need to make their own decisions. They are not conceiving neither implementing their own architecture, they are basically “filling holes” with small pieces of relatively simple business logic.
Programming in the Large and Programming in the Small
The distinction between “programming in the large” and “programming in the small” was coined in the 70’s. At that time, it represented the difference between developing software systems and implementing simple programs. The term “programming in the small” characterized tasks that could be done by a single programmer. But at that time there were not sophisticated application frameworks that could be extended.
Today it is important to take in consideration the ratio between “new code” and “reused code”. It is a characteristic of modern software systems that this ratio is relatively small, meaning that most of the code is being reused from existing frameworks, platforms and libraries. In this case, it is possible to create complex systems with small amounts of new code. This is sometimes called “glue code” because it just connects existing components.
For example, Apache is the most popular HTTP server, with a 60% market share, and its implementation has more than 1.7 million lines of code. In contrast, the average Web page has only 295 kB of scripts. Assuming that each line of script has in average 30 characters, this means that the average page has 10,000 lines of scripts. Of course a Web site has multiple pages, and there is also code running on the server side. But on the other hand most sites also use other platforms such as MySQL. Thus, for the average Web site the ratio between new code and existing code is very small.
In another example, an analysis of the top 5000 most active open-source software (OSS) projects has shown that OSS written in Java has in average almost 6 millions lines of code. In contrast, the average iPhone application has in average just a few hundred thousand lines of code. In this sense, OSS is “programming in the large” and iPhone apps are “programming in the small”. But thanks to advanced frameworks, even small iPhone apps can have a very sophisticated and impressive user interface.
Adaptable Design Up Front
What should be done in situations in which there is no framework ready to be used for a particular application? Of course in this case the software developers should be responsible also for defining the architecture, and in my opinion this always requires a certain amount of design up front.
I personally adopt the “Respond to Change” strategy, which I call Adaptable Design Up Front (ADUF). In my opinion, in an Agile environment in which the requirements are constantly changing, this is the only strategy that guarantees the successful evolution of software systems. The idea is basically: If you don’t have a framework to build on top of it, first create your own framework and then extend it.
In one of my previous articles I describe how the ADUF approach combines Domain Modeling and Design Patterns. In another post I explain how ADUF is an application of the Open/Closed principle to high-level design. I’ve also prepared a set of slides to introduce the ideas behind ADUF.
In Summary
Most people claiming that they are doing Emergent Design in practice are doing Framework-Based design in which they don’t need to make high-level design decisions, because they are adopting decisions that were already made by someone else.
In modern software systems the ratio between new code and reused code is quite small. Thanks to advanced frameworks, platforms and libraries, software developers may build sophisticated applications with a relatively small effort.
For the situations in which there are no existing frameworks that can be easily extended, the approach of Emergent Design does not work. It is necessary to do some amount of design up front. My suggested approach is called Adaptable Design Up Front (ADUF).
What do you think? Do you know projects that claimed to do Emergent Design and were not based on existing frameworks? Please share your experience in the comments below.
During the First Israeli Conference on Software Architecture, Hayim Makabee gave a talk about “The Role of the Software Architect”.
Effective Software Design 