In the last decade we have seen a decrease of interest in Software Design and a sharp increase of interest in Agile methods. The first graph below compares the trends between Software Design and Agile. The second graph compares the trends between Design Patterns and Scrum. They are very similar. And in my personal opinion, they are very sad.
I am a firm believer in the importance of Software Design. In particular, I strongly believe that Software Design is essential in the context of Agile and Lean software development. In a series of previous posts I defined my Adaptable Design Up Front approach that is focused exactly on supporting Agility: having a software architecture that embraces changes through adaptation.
Thus I am trying to understand this phenomenon that has caused so many software developers to lose their interest in Software Design and get so enthusiastic about Lean and Agile methods. I think one of the reasons is the current predominance of Indefinite Optimism.
Definite vs. Indefinite Optimism
The difference between Definite and Indefinite Optimism is described by Peter Thiel in his book “Zero To One, Notes on Startups, Or How to Build the Future“.
Definite Optimist: has concrete plans for the future and strongly believes that the future will be better than today.
“To a definite optimist, the future will be better than the present if he plans and works to make it better.”
Indefinite Optimist: has great expectations about the future but lacks any design or plan for how to make such a future a reality.
“To an indefinite optimist, the future will be better, but he doesn’t know how exactly, so he won’t make any specific plans.”
Indefinite Optimism, Agile and Lean
If we go back to the graphs above, in my opinion it is clear that they depict a transition in software development from definite to indefinite optimism. Agile and Lean methods are characterized by the lack of planning and design, by the emphasis on the process and by a tendency to postpone decisions.
For example, the popular YAGNI principle from eXtreme Programming instructs developers to avoid implementing things that they are not sure if they will need. In practice, programmers adopt the optimistic guess that they will be able to implement anything when it is really needed. Then, they not only avoid the implementation of new features, they also avoid preparing for these features. In other words, YAGNI causes people to avoid designing for extensibility.
In another example, the Last Responsible Moment principle from Lean Software Development instructs developers to postpone their decisions as much as possible. In practice, programmers adopt the optimistic guess that this Last Responsible Moment will always happen in the future. Then, they not only avoid making premature decisions, they also avoid thinking about the consequences of these decisions. In other words, the Last Responsible Moment causes people to treat decisions as if they were independent of each other, and not part of a cohesive design.
Finally, I am sincerely amazed about the absolute emphasis the Agile community places on the process, focusing on project management instead of software engineering. Apparently having a Sprint retrospective to discuss how to improve the process is considered by many Agile practitioners to be much more important than conducting proper code an design reviews. In the words of Thiel:
“Arguing over process has become a way to endlessly defer making concrete plans.”
In recent years it became almost a consensus that startups should adopt the Lean Feedback Loop, based on evolution through experimentation and incremental improvements. However I think that this evolution requires a software architecture that supports extensibility and adaptability. In the absence of such an Adaptable Design, the only alternative are endless mutations in the form of changes and patches. To conclude with a final quote by Thiel:
“Darwinism may be a fine theory in other contexts, but in startups, intelligent design works best.”
What do you think? Please share your experience in the comments below.