“It is not the strongest of the species that survives, nor the most intelligent that survives. It is the one that is the most adaptable to change.” – Charles Darwin
Big Design Up Front (BDUF) is considered a very bad practice in Agile software development. BDUF is problematic because it assumes frozen requirements. The Agile approach is that software must be developed iteratively since the requirements will change and the system will evolve. So we need some Design Up Front, but this must be Adaptable design that supports change: ADUF.
Adaptable Design Up Front
We should never work on the design of a specific system. Instead, we should generalize, designing for a family of systems. Applying object-oriented concepts, our design should represent a class of systems, and each version during the system evolution should be an instance of this class. Thus each new version may have modifications and extensions when compared to previous versions, but if it still fits the same generic design, then it is still an instance of the same class.
Adaptable Software Design: A generic software design for a family of systems which does not need to be changed to accommodate new requirements.
The challenge of creating an adaptable design is making it change-resilient. Adaptable designs should evolve and accommodate new requirements through specialization. When designing for a family of systems, there are two main such approaches: The framework approach and the platform approach.
Frameworks
Framework: “A software framework is an abstraction in which software providing generic functionality can be selectively changed by additional user-written code, thus providing application-specific software.”
The framework defines the system structure: The main system components and the relationships among them. Each one of these framework components should have an abstract interface, and several possible implementations for these interfaces. If the design is sound, the components implementation may evolve over time, but the relationships among them should be preserved.
Each component in a framework should be designed to be highly cohesive, and the framework itself should be designed so that components are weakly coupled. If the component interfaces are indeed abstract and if the coupling among these components is weak, each type of component in the framework can be seen as a kind of plug-in.
Plug-in: “A plug-in is a software component that adds a specific feature to an existing software application. When an application supports plug-ins, it enables customization.”
If we are able to define such framework with diverse plug-ins, the system evolution occurs through the implementation of new versions of components. Thus, instead of having a new version of the entire system, we may have new versions for each one of the plug-ins. Building the system means simply combining these plug-in versions in an appropriate configuration.
Platforms
Platform: “Technology that enables the creation of products and processes that support present or future development.”
A platform should support the basic services which are required by the system to provide its functionality. When these basic services can be well-defined and implemented independently of the system which will use them, we have reached an ideal environment for adaptability:
- The platform may evolve independently of the systems that are built on top of it. If the interfaces are kept intact and if the behavior is preserved, the platform developers are free to improve its efficiency and other non-functional attributes.
- Several systems may be built on top of the same platform, using its services to provide diverse functionalities. These systems may each evolve independently of the others, and independently of the evolution of the platform itself.
Building platforms is a very effective approach for experimentation. If an essential part of the system behavior can be captured as basic services being provided by the platform, then each version of the system may be seen as an experiment and the cost of failure is low. The resources invested in the development of the platform are not lost in the case one of the system versions is not deployed.
Adaptable Software Design in Practice
Question: How do you identify the components in your framework or the services in your platform?
Answer: We want to think generally about a family of systems, and avoid defining requirements for a specific system. To identify the main entities that should compose any system in this family, we should apply domain modeling.
Domain Model: “A domain model is a conceptual model of all the topics related to a specific problem. It describes the various entities, their attributes, roles, and relationships, plus the constraints that govern the problem domain.”
Question: How do you decouple the components in your framework?
Answer: To reduce coupling, one should look for appropriate Design Patterns. Most of these patterns are related to avoiding the direct links between concrete classes, through the introduction of interfaces and abstract classes.
For example, when instantiating an object, one should avoid creating an instance of some specific concrete class. This can be done using the Factory pattern.
Summary
Software systems must be able to evolve over time. This evolution should be planned and supported trough Adaptable Software Designs. There are two main approaches for creating such designs: Frameworks and Platforms. Both require the identification of domain entities and the usage of abstract interfaces to minimize coupling between components.
What do you think? What is your personal experience with software evolution, changing requirements and adaptable designs? Please share your thoughts in the comments below.
Effective Software Design 
Great post, Hayim. I’ve often said that the highest virtue is flexibility. The definition of “perfect” will likely change before you can implement it, but with a good design, you can pivot toward the new target without reworking everything. That sort of flexibility doesn’t come by accident nor by trying to design down to the lowest level – just enough design is the key.
Thanks, Gene. Unfortunately a common mistake in the “just enough design” approach is not understanding the need for adaptability. The result is a design that is simple but not flexible.
I have read the conclusion. It is very good. Adaptable Requirements / Design are OK but Agile looks too shallow and fragmented to me. I prefer Adaptable without Agile. I will read the rest later. Best wishes,
Much of what you said is welcome but I recall the objectives of OOAD are also the same though they did not talk only of design. For this principle to operate Agile need not be brought up (Iterative Incremental Development of Unified Process covers it adequately).
It would be nice to know what the available platforms and frameworks are.
AA: I have been waiting for programming free software configuration and assembly facility (some kind of Software Lego).
BB: ReUSE is the foundation for the above.
For AA and BB those who are interested may see the pdfs Software Lego and ReSAR on http://www.slideshare.net/putchavn
05MAY13
Thanks, Putcha. I agree that adaptability is always necessary, independently of the software development methodology. But in the case of Agile there are too many people fighting against BDUF but not offering any alternative. My goal was to propose ADUF as an alternative to BDUF.
I think your suggestions make a lot of sense, but have some fundamental challenges:
Designing, frameworks, platforms and domain models is very very hard to get right. The common advise is to build frameworks (or platforms), interatively, while abstracting from specific examples/solutions rather the other way around. This allows over time, and with gained experience, to identifying what the core “stable” features of a framework are, and what not.
Its really comes down to experience. Someone who designs the 5th remote access software already knows what the commonalities are in the domain, and can drive the design to adopt useful framework abstractions.
Thanks, Daniel. I agree that experience is essential to model the domain, and for this reason I believe that Agile teams must have the support of SW architects.
Pingback: Topics, related to Agile | Youry's Blog
> Applying object-oriented concepts
You lost me so early. This is not the only way to design. Certainly not in a rapid and adaptable development context. One might review available techniques first, then consider applying the best approach to a problem.
Many agile languages (static or dynamic) allow terse map definitions, with namespaces to hold generic functions that manipulate them. It can clearly describe your domain, but you can use OOP, functional, hybrid,… to quickly define that domain and manipulate it in working code.
Being adaptable means only doing as much design as is required, but no more. NB clean adaptable designs do not imply use of one software paradigm, just the right one to bridge/intersperse design and code, provide flexibility, and enough development speed to deliver regularly and of sufficient quality.
I like what you say in general, but it sounds a little like a synopsis of other peoples ideas!
Thanks, Alex. I used OO concepts to say that an adaptable design should describe a class (or family) of systems and that each version of the system during its evolution should be an “instance” of this design.
The idea is interesting, but form my experience building adaptable systems is a very, VERY complex task. Furthermore, it usually takes longer because of the levels of abstractions you have to add to your design. This takes you away from the whole point of being agile – which at the end is to do things faster. I prefer the idea of Emergent Design (http://en.wikipedia.org/wiki/Emergent_Design#Emergent_design_in_agile_software_development) which matches agile more. Your idea seems good when you already have a set of software components that can be transformed into one extendable platform with a number of plugins/extensions.
Thanks. I agree that producing adaptable designs is very challenging. But my experience is that having plugable components allows the software to evolve faster.
This happens to be close to my dissertation research topic. A key paper in my research points out that agile teams achieve “agility” by “being agile” and “doing agile” (McAvoy et al., 2012). Doing agile refers to following methods to achieve agility, while being agile refers to engaging in effective teamwork to achieve agility. Think of being agile as being flexible, while doing agile as being efficient. What McAvoy et al. found in their study of one organization’s “agility” was a large degree of collective mindlessness. This agrees with leading research into agile teams that finds that when team follow agile methods “by the book” (rote), it deprives them of some agility.
What does this mean for adaptable designs? The Agile Manifesto states that “the best architectures, requirements, and designs emerge from self-organizing teams” (Beck et al., 2000). A self-organizing team is an effective team, so being agile implies being an effective team. The rub comes in because architecture, requirements, designs (and people, politics, etc.) create emergent complexity, or complex system, that co-evolves (Benbya & McKelvey, 2006). So if architecture is BUFD, then there is a dilemma in applying it in emergent fashion. That is, you cannot have a BUFD and evolving architecture, right? Wrong. We do this all the time. The short answer is that architecture and agile development can co-exist (Falessi et al, 2010). The price that Felessi et al. found was that project complexity was the primary reason to focus agile development (teams) on software architecture, based on a survey of agile developers. Their results showed that agile developers perceived software architecture was necessary to support agile values. One way to look at this is if you don’t have software architecture, complexity will likely get out of control. This is precisely what many practicing architects observe in industry (including me): e.g., Grasping the complexity of development (Lee and Xia, 2004). There’s growing literature on this problem .
Architecture is a necessary mindfulness exercise, in my opinion. Without it, you mindlessly follow agile method rules, like Scrum sprints, and generate code complexity that will more than likely rapidly decay your software architecture. The general rule of thumb is to balance some degree of being and doing agile. How? I generally recommend an architectural alignment sprint, although I’m often overruled that business owners that over value speed of features in balance with moving the architecture in the strategic business direction. This is the recurring trade-off that we’ve been making in software designs and architectures for many years, only we are sacrificing strategy for tactical advantage. Some agility research calls this learning to be agile by exploiting or by exploring. Many immature “agile” organizations are stuck in exploiting mode: e.g., Scrumfall.
Beck, K., Beedle, M., van Bennekum, A., Cockburn, A., Cunningham, W., Fowler, M., & et al. (2001). The Agile Manifesto.
Benbya, H., & McKelvey, B. (2006). Toward a complexity theory of information systems development. Information Technology & People, 19(1), 12–12–34.
Falessi, D., Cantone, G., Sarcia, S. A., Calavaro, G., Subiaco, P., & D’Amore, C. (2010). Peaceful Coexistence: Agile Developer Perspectives on Software Architecture. IEEE Software, 27(2), 23–25. doi:10.1109/MS.2010.49
Xia, W., & Lee, G. (2004). Grasping the Complexity of IS Development Projects. Communications of the ACM, 47(5), 68–74.
McAvoy, J., Nagle, T., & Sammon, D. (2012). Using mindfulness to examine ISD agility. Information Systems Journal, 1–18. doi:10.1111/j.1365-2575.2012.00405.x
@Bryan are you at a point in your research where you have further findings that you can share? regards Alex
Thanks for your comments and references, Bryan. I like the distinction between “doing Agile” and “being Agile”.
@Bryan Thanks for the references. Itis great to find more information on the topic, particularly peer reviewed research.
Interesting comment Bryan! Especially the `architectural alignment sprints`. Could you share more regarding this? Is there any scientific literature regarding that topic?
Pingback: Adaptable Design Up Front | Chris F Carroll
Although most of what u said is highly pertinent, Agility at times has a cost. Because systems today go from somewhere and from something, we cannot consider agility to be a key and fondamental factor when it comes to designing the system. technology has given us a great helped in this sense and we should add it on the balance. By the way architecture is about reusability, who says reusability also says loosely coupled and from that building blocks. This said agility shouldn’t add a question mark even on our building blocks. Business processes allows us to be flexible meanwhile SOA gives us reusability and all the buzz around it. when this are put together we deliver a solution that is aligned to business.
As to me I don’t share the point of view stating that the Agile methodology could ameliorate the design of EA in the sense that business processes when well orchestrated gives us full agility and an unlimited reuse potential. and because we must always go from somewhere (web services wrappers for legacy systems and composite application) we can’t actually reconsider even our building block for the sake of agility. Let us focus on understanding the business. the problem today in EA is the need for architects to be extremely competent and rigorous when approaching EA problems but once a good foundation is set…
Pingback: The Myth of Emergent Design and the Big Ball of Mud | Effective Software Design
Pingback: Adaptable Design Up Front and the Open/Closed Principle | Effective Software Design
Great post! I can relate to it as I naturally evolved to develop this way. One downside though that I sometimes forget to account for is that this type of design can add a lot of overhead (boilerplate) to the project and this sometimes is not practical due to the scope of a project (s) or short deadlines.
The added value of a good adaptable design can sometimes work against ROI requirements, and that, to me is the key to choose whether or not use ADUF.
Pingback: Emergence versus Evolution | Form Follows Function
I like the name “Adaptable Design”. One of the biggest challenges with architectures, particularly large ones, is their assumed immutability and commitment. Design patterns and tactics that facilitate change help to give architects and design value and relevance. Like the term “debt” in “technical debt”, which uses a financial metaphor to explain the concept to non-technical stakeholders, “adaptable” emphasizes the intentional, reactive changes it allows.
However, as others have said, adaptable design is only effective if changes are predictable. For example, providing a platform to support changes in one area does not help if there are substantial changes elsewhere.
Similarly, some changes may be uneconomical to accommodate. Billing systems, for example, are notoriously complex and it may be simpler to just reengineer the system when changes are required rather than creating a generic system.
Pingback: Big Design Up Front versus Emergent Design | Random Acts of Architecture
You might say I’m one of those Agile extremists, but also an architect. :-). The problem in discussing the stuff you mention guys is your use of academic argumentation, which makes it unnecessarily complex and still flawed. Bryan McKinley, with his impressive reply, concludes with “architectural alignment sprint” and concerns about strategy, while “business owners” don’t really get this and only care about the speed. Hmm, interesting. Somehow I doubt that, specially because of my experience. It sometimes happens to me too….when I start talking “expensive” and use words like “strategy” often. Somehow, anything similar to “Business-IT alignment” seems failed by definition. Why don’t deliver just deliver what business needs and asks. Why do we even think they want a big mess in long term? The problem is communication at the side of architects.
Back to the original post. @Hayim: Probably the most disliked architecture words by Agile guys are flexible, generic, platforms, and frameworks. You use all of them. If we as architects want to be heard seriously, you should start talking about preventing mess, sustainable delivery of value for business, lowering TCO with architecture and so on.
So, what is the problems in your statements:
1. “A generic software design for a family of systems which does not need to be changed to accommodate new requirements.” is more fantasy than reality. The problem here is way too simplistic way of looking at how difficult this is and how expensive it is when we deliver much more expensive “generic” system. It will not be generic at correct level, it will compromise on specific business need because it must be generic. Any successful reused systems go through many iterations of usage and changes before really being generic at correct level.
Besides, in case of custom systems for specific customer. It is really insane when architects are making generic design for their specific company, with specific needs and specific situation. Usually without even really knowing the needs and domain.
2. Frameworks: I thought we had this discussion around year 2000 and finished it. Frameworks are not bad, it is just that the best emerge after some time after being used by multiple companies in many situations and refined many times. Also, the best are delivered by open source guys contributing from all over the world with many different experience and knowledge. After some time, the big vendor companies see the money in it and ruin it. Open source guys simply move on, and create even better one.
3. Platforms: Great example here is evolution of Enterprise Java. First versions were a mess, with EJB 2.1 as really bad. In the past years, the evolution of platforms moves towards simplification. In other words, less platform is often better. It makes us less limiting in creating perfect solution for the business. Anyway, it seems to me really bad to focus on creating platforms within context of a single company. It would limit experimentation instead of unleashing it. Never heard developer saying: “I need platform in order to experiment”, but rather “Give me internet connection so I can download something to start with.”.
4. I do like “domain modeling”. Domain-Driven Design by Eric Evans is must read for any architect. In your explanation “To identify the main entities that should compose any system in this family, we should apply domain modeling.”. But the definition says: “A domain model is a conceptual model of all the topics related to a SPECIFIC problem…”. Not family of problems.
This does not prevent us from learning by doing and adjusting application landscapes towards more reusability AFTERWARDS. As crazy as it might sound, this costs less money and time.
5. In the past 15+ years mainly XP practices have brought us much more than simply “apply design patterns”. I would mention works from Michael Fathers, Uncle Bob, Kent Beck, Martin Fowler, Jim Coplien ( with Lean Architecture). These guys really make sure we create great designs on “high or low level” without using fancy words.
Thanks for your comments, Viktor. I think you address some important points:
– Do we really need to develop for a family of systems instead of a specific system?
– Do we really have enough experience to be able to generalize our solutions?
– When is the right time to try to build more generic or reusable solutions?
I think these are great questions which I intend to explore in my future posts, because they certainly deserve a much more detailed discussion than what I was able to do in this initial article.
I will also try to discuss alternative approaches from the sources you mentioned and others, such as Neal Ford’s “Evolutionary Architecture” and Alistair Cockburn’s “Incremental Rearchitecture”.
Your feedback helps me a lot in understanding which points require further elaboration and clarification, and I believe that in practice we have more agreements than disagreements.
You’re welcome. Exactly, these are the core questions. To be honest, I’m also still in research mode and don’t always have clear answers for these question. I do think it’s really worth the effort to experiment with completely opposite approach (as I already do in past 5+ years in quite big projects) from what architects generally do or compromise with Agile movement. Why? Simply because it fails too often and I don’t believe any more in “let’s try harder doing the same”.
Pingback: Avoiding Technical Debt: How to Accumulate Technical Savings | Effective Software Design
Pingback: Coping with Change in Agile Software Development | Effective Software Design
Pingback: The End of Agile: Death by Over-Simplification | Effective Software Design
Pingback: Emergence versus Evolution | Iasa Global
Pingback: “Emergence versus Evolution” on Iasa Global Blog | Form Follows Function
Pingback: Re-Post: The End of Agile: Death by Over-Simplification | Youry's Blog
Pingback: On Agile Architecture, Emergent Design and Framework-Based Design | Effective Software Design
Pingback: Manifesto for Adaptable Software Development | Effective Software Design
Pingback: Indefinite Optimism: the Problem with the Agile and Lean Mindsets | Effective Software Design