Nothing is more effective than enthusiasm

Posted on March 28, 2011 by Hayim Makabee

We all know that some programmers are much more effective than others. In one study, it was shown that the best programmers are ten times more efficient than the worse programmers. This degree of difference has not been observed in any other kind of industry, and this makes effective software development a unique challenge.

Now the question is: Why are some programmers much better than others? Can this difference be explained in terms of intellectual capacity, aptitude or experience?

In fact, it was observed that some programmers are much more effective than others even if they are very similar on all other aspects, such as IQ, problem solving capacity, math and logic orientation, and years of experience as a professional software developer.

I believe that the main difference is the enthusiasm! Some people simply love programming, more than any other intellectual activity. If these people weren’t assigned any coding task, they would just use their imagination and invent some new program to write, probably as challenging as possible.

I think this is the origin of all open-source projects, which are very successful and sometimes go against the intuition that complex software development requires centralized management. Actually, these passionate programmers are not only able to produce high-quality code in their free time, they are also able to organize themselves in very effective development teams.

So, do you want to find a real programmer? It’s easy: They can’t stay too long without writing a line of code. It’s kind of an addiction to programming!

Posted in Efficacy, Hiring, Psychology of Programming | Tagged , , | 9 Comments

Effective Software Reuse

Posted on February 28, 2011 by Hayim Makabee

For many years Software Reuse has been seen as a solution to both reduce the costs of software development and improve the quality of the systems being developed. However, in practice, most companies today have the feeling that they are not enjoying the full potential of software reuse.

In general, the most successful type of reuse today is the one exploring third-party components, in the form of libraries and frameworks. It is hard to imagine that someone would implement an application to parse some standard file format like XML, or to implement some standard communication protocol, without first looking for libraries that already provide the required functionality.

In contrast, a more difficult type of reuse is the one of components developed inside an organization. This happens because very often the components being implemented as part of an application are not planned to be reusable.

The effective development of reusable components requires a serious investment, including:

  • The design of the component’s interface, that must be both rich and generic.
  • The implementation of the component’s functionality, that must be efficient and independent of the application.
  • The full testing of the component’s behavior, that must take in consideration a great diversity of contexts.
  • The detailed documentation of the component’s usage, that must guide its potential users providing all required details.
  • The indexing of the component in a shared repository where it can be easily found.

In my opinion, this can be achieved only by a team that is dedicated to the development of reusable components. This team should receive requests from the product teams in the organization, with the description of components they would like to have in their applications and that are considered good candidates to become reusable components. Ideally, the communication between these teams should include the participation of a software architect, who should be responsible for approving the requests.

The biggest challenges of the Reusable Components Team (RCT) are:

  • Understanding the requirements from different product teams so that the components developed are really reusable across different applications.
  • Creating new components according to the time schedules of the product teams, respecting their deadlines and avoiding any delays in their plans.
  • Being responsive fast enough  when the product teams ask for changes or extensions in existing reusable components.

Therefore, the product teams are the clients of the RCT. The product teams should treat the components developed by the RCT as-if they were third-party reusable components, and never duplicate the code or change it by themselves.

From my experience, if the RCT is not able to meet its challenges, being responsive enough and fast enough, the product teams will “take ownership” of the reusable components, creating their own versions and ruining the very idea of software reuse.

Good luck with your reuse plans, I will be glad to hear about your own experience.

Posted in OOD, Software Reuse | Tagged , | 6 Comments

Effective Remote Work

Posted on January 20, 2011 by Hayim Makabee

After reading the book “The 4-Hour Workweek” by Tim Ferriss, I was eager to test his concept of Mini-retirements:

“To distribute recovery periods and adventures throughout life on a regular basis and recognize that inactivity is not the goal. Doing that which excites you is.”

I work for Yahoo! Labs in Israel, and I discovered that Yahoo! also has offices in Brazil. Then I decided that in my next trip to Brazil I would combine my two-week vacations with a period of remote work, so that in the total I would be able to stay there more than a month and have many additional weekends to enjoy being abroad.

My main doubt was: “How effective will be my remote work?”

I’m still in Brazil, but after two weeks working remotely I think that overall the experience was very positive. Here are the main points:

Communicating with co-workers: I didn’t feel any disruption in my formal communication with other members of my team, especially because there is only a 4-hour difference between Brazilian summertime and Israeli time. We even didn’t have to change the schedule of our weekly meeting, that was done via audio-conference, in the early afternoon in Israel and the morning in Brazil. We continued to communicate normally through Email and IM, and the only difference was a slight increase in the volume and frequency of messages, but nothing disturbing.

Performing my tasks: We are now in the early stages of our project, and I’m doing basically investigation of technologies and programming tasks. At this stage there are few dependencies between my tasks and the ones being performed by my co-workers. Actually, the fact that I’m abroad has drastically reduced the number and frequency of interruptions of my work, so that I’m certainly more productive here. I intentionally don’t have my mobile phone activated here, and that’s a great change from being at home.

Access to resources: When I arrived in the Yahoo! offices in Brazil, I was immediately given a desk with a phone and network connection. I brought my laptop with me from Israel, so that in a few minutes I was already connected and able to continue my work. Since we store everything in the company network that is globally shared, I have access to all resources I need, including Wikis, documentation, source code repositories and bug-tracking tools. Thus, despite being very far from home I’m working seamlessly.

The only thing I really miss are the social interactions with my co-workers, like the jokes in the corridors and the heated discussions at lunchtime. But this is compensated by the wonderful opportunity to meet and make new friends with my colleagues at Yahoo! offices in Brazil.

Posted in Efficacy, Yahoo! | Tagged , | 1 Comment

Identifying Anti-Patterns

Posted on December 22, 2010 by Hayim Makabee

In my previous post, I’ve discussed the importance of identifying domain-specific Design Patterns in an organization. Equally important is the need to identify Anti-Patterns.

This is the definition of Anti-Patterns from Wikipedia:

“An anti-pattern is a pattern that may be commonly used but is ineffective and/or counterproductive in practice.”

The first interesting aspect in this definition is that anti-patterns are commonly used despite being ineffective or counterproductive. There may be two main reasons for this clear contradiction:

  • The programmers using the anti-pattern are inexperienced, so they are unaware of the fact that these patterns are ineffective or counterproductive.

  • The programmers using the anti-pattern have experience on a different kind of systems, which were less demanding or based on older architectural approaches. Thus, they are not aware that the patterns they have used in the past are no longer appropriate for the new category of systems under development.

The second interesting aspect in the definition is that an anti-pattern is ineffective or counterproductive, but it still implements correct functionality, so the use of an anti-pattern is not a bug. Most often the anti-pattern provides a poor implementation that fails to satisfy non-functional requirements. Here also there are two main cases:

  • Bad design: The usage of the anti-pattern causes the code to be hard to understand and hard to maintain.

  • Bad performance: The usage of the anti-pattern causes the code to be inefficient, not scalable or non-robust.

Normally, inexperienced programmers use anti-patterns that cause the system to have both bad design and bad performance. It is necessary to show them the problematic code and teach them better alternatives.

Experienced programmers, on the other hand, may use anti-patterns that implement very elegant designs, but that provide bad performance. This happens because these patterns were good enough for the systems they have implemented previously, but they are no longer able to satisfy the more demanding non-functional requirements of current systems, such as higher scalability and increased robustness. Another common problem is that patterns that were once appropriate for older architectural approaches are no longer suitable for systems with modern architectures.

In an organization, there are three main opportunities to identify anti-patterns: During design reviews, during code reviews and during refactoring of legacy code.

Design reviews are a great opportunity to find anti-patterns. A software architect should be present in each review, and be responsible for indicating if there is any problematic aspect in the design being proposed, either because it doesn’t fit the overall system architecture or because it doesn’t satisfy the non-functional requirements.

Code reviews are the right place to locate bad code, either because it is inefficient or because it is hard to understand and to maintain. They are an excellent opportunity to teach inexperienced programmers about better alternatives.

During the refactoring of legacy code it should be easy to locate anti-patterns, simply because the system requirements have evolved since the code was written. In this case, the person responsible for the refactoring should have a plan for converting each anti-pattern into appropriate code that satisfies the current requirements. Very often this involves adapting old code to fit the new architecture.

So good luck finding your anti-patterns, there will be only benefits!

Posted in Design Patterns, OOD | Tagged , | 3 Comments

Collecting Design Patterns

Posted on November 30, 2010 by Hayim Makabee

Today we expect every professional Software Engineer to have a good knowledge of Design Patterns. In particular, the names of the patterns in the Gang-of-Four (GoF) book became part of the common vocabulary of software developers. Any good programmer should know the meaning of a Visitor, Singleton or Strategy.

These patterns are part of the general knowledge shared by most professionals in the software industry, independently of the specific fields they are working on. People implementing systems as diverse as games, banking and telecommunications can still apply the same patterns. The GoF Design Patterns widespread usefulness is derived from their generic and abstract properties.

However, to be able to perform our jobs effectively, besides generic knowledge we also need domain-specific knowledge. People developing games have to solve some particular problems that are different from the ones in banking and telecommunications applications. With time, the experts in games programming will have acquired valuable knowledge about how to approach the challenges specific to their field.

Imagine a professional has two set of tools:

  • The first set has the common tools used by all other people in the same profession. These are multi-purpose tools that you can buy at any store and are easy to find.
  • The second set has very specialized tools that the professional himself has invented. The creation of these tools was the result of many years of experience solving some very specific kinds of problems.

Of course other professionals could benefit from this second set of tools. But then you need ways to share these tools with them.

In the case of software development, we are very good at applying the generic and abstract design patterns. But we still need better methodologies to:

  • Identify design patterns specific to our problem domain.
  • Classify domain-specific patterns according to their purpose.
  • Document domain-specific patterns so that they can be easily understood.
  • Collect domain-specific patterns in a repository so that they can be easily shared.

I think that every software development organization should invest resources to create its own collection of design patterns. Every developer in the organization should be aware that he is expected to identify new patterns, classify and document them, and then add them to the shared repository.

I believe that the benefits for the organization are clear. They include:

  • Sharing and preserving knowledge in an effective format.
  • Creating a common vocabulary.
  • Helping new developers learn about the domain.
  • Increasing developer productivity by providing reusable solutions.
  • Improving software quality by using solutions that were already tested.

So good luck collecting your own design patterns!

Posted in Design Patterns, OOD | Tagged , | 2 Comments

Effective Design Reviews

Posted on October 24, 2010 by Hayim Makabee

This is the definition of Design Review from Wikipedia:

“A design review is a process whereby a design is tested against its requirements prior to implementation.”

There are at least two problems with this definition:

The first problem is that it says we are going to review “a design.” In other words, there is a single design alternative being presented. In this case, the designer is probably expecting to convince all other participants that he did a good design. At most, he is willing to hear a few comments about how his solution could be possibly improved. It is very unlikely that his design will be rejected, because almost certainly none will be able to suggest a different option.

The participants of a design review should not be expected to imagine alternative solutions. Only if the feature being implemented is very simple it would be possible to propose entire new designs during the review. Any interesting feature requires a considerable amount of effort to obtain a good design. The consequence is that in this “single design” approach the solution being “reviewed” will almost always be approved with minor changes.

Therefore, effective design reviews must include the presentation of several alternatives. The discussion should not be if a particular design satisfies or not the requirements. Correctness is the most fundamental attribute of a solution, so the diverse design choices must differ on some other attributes that can be compared. In other words, each option has its advantages and disadvantages, and by choosing one of them we are actually deciding which attributes are the most important ones, but all options must be correct.

In this “multiple-choice” approach, the goal of the design review is to analyze several alternatives and understand their different implications. Then, if there is a solution that is clearly better than all others, according to some criteria, it should be the chosen one. Otherwise, we should try to look for additional options.

But here is the second problem with the previous definition of design review: It says that the “design is tested against its requirements.” In most cases this means only the functional requirements, the specific demands related to the feature being implemented. This definition assures that the design must be correct, but it does not address the question of how the new feature being implemented will affect the existing system architecture. In other words, this definition does not take in consideration the non-functional requirements.

Diverse design alternatives will probably have very different implications for non-functional requirements, and this should be used as the basis for comparison when selecting one of them.

For example, one design alternative could have the user’s session data stored on memory, while another would store the same data in a database. The first alternative will probably allow all operations to be performed more efficiently, thus reducing latency and increasing throughput. However, the second alternative improves resilience and fault-tolerance, because if a process crashes no data will be lost and it will be possible to restore the user’s session from the database. In this example, the participants in the design review must decide which attributes are more important, and select the best solution accordingly.

In summary, if there are no alternatives, there is no real review being done. This can be best illustrated by a story of Alfred P. Sloan, then CEO of General Motors:

At one meeting of his top executives, Sloan said: “Gentlemen, I take it we all are in complete agreement on the decision we’ve just made.” Everyone nodded.

“Then,” said Sloan, “I propose we postpone further discussion until our next meeting, to give ourselves time to develop disagreement and perhaps gain some understanding of what the decision is all about.”

So let’s hope we will have more disagreement in our next design reviews!

Posted in OOD, Software Architecture | Tagged , | 4 Comments

The Liskov Substitution Principle and Test-Driven Development

Posted on October 5, 2010 by Hayim Makabee

The Substitution Principle, first defined by Barbara Liskov, says that:

“Let q(x) be a property provable about objects x of type T. Then q(y) should be true for objects y of type S where S is a subtype of T.”

In other words, an instance of the subclass must always behave like an instance of its superclass, so that in every situation in which an instance of the superclass can be used it could be substituted with an instance of the subclass.

In practice, it is difficult to assure that, because most languages do not provide an explicit mechanism to define the semantics of a method. At most we can define the method’s signature: Its name, the types of its parameters, its return type and the exceptions it may raise.

A special case is the Eiffel programming language, created by Bertrand Meyer, in which there are pre-conditions, post-conditions and class invariants. Together, they define a kind of contract between a class and its users, and hence this approach is known as Design by Contract:

  • Pre-conditions are the conditions required by a class before a method is called. The user of the class is responsible for assuring the pre-conditions.
  • Post-conditions are the conditions required by the user after a method returns. The class is responsible for assuring the post-conditions.
  • Class invariants are the properties that are always true regarding the state of an object. The class assumes that invariants are true before a method is executed and guarantee they remain true after its execution.

The question now is: What can we do if there is no way to define the contract between a class and its users? How can we help the developers of new subclasses to make sure that their classes behave correctly, satisfying the Substitution Principle?

This question is particularly relevant for people using frameworks, when it is common to extend the framework through the definition of new subclasses. In this case, a programmer will write new classes that inherit from existing classes that were developed by someone else, and he must be sure that these new classes will work properly when inserted in the framework.

Fortunately we do have a methodology that allows us to define how a class must behave even before we write this class: Test-Driven Development (TDD).

According to the TDD approach, a developer should write unit tests that check the correct implementation of all important functional aspects of a class. It is common to have multiple tests for each method, checking what happens when there are different parameters or when the object being tested has a different state. In general the unit tests can be executed automatically and they include assertions that either return true on success or false on failure.

Now, if we already have a set of unit tests that check the behavior of some class, we can simply apply the same tests on the new subclass we have created and observe if there is anything broken. This is the Substitution Principle in practice! The set of unit tests defines the semantics of the class being tested. It defines the contract between this class and its users. When we run the unit tests replacing an instance of this class with an instance of the new subclass, we actually are checking if the subclass satisfies the Substitution Principle.

Therefore, a framework should be considered complete only if it includes an extensive collection of unit tests for all the classes intended to be inherited from. This will allow developers extending the framework to easily check that their new subclasses are respecting all the existing contracts.

Posted in OOD, OOP, TDD | Tagged , , | 3 Comments

When a square is not a rectangle

Posted on September 20, 2010 by Hayim Makabee

In my previous post, I’ve used the example of squares and rectangles to illustrate the importance of definitions. It is clear that, from a geometric point-of-view, a square is indeed a kind of rectangle. However, as my friend Arnon has noticed, this may be problematic from an object-oriented perspective.

Let’s say that we organize the classes Square and Rectangle in a hierarchy so that Square inherits from Rectangle. This reflects exactly the geometric relationship between them, since inheritance is commonly used to represent the is-a-kind-of relationship.

Now suppose that we have a photo-enhancement application with a feature to take a picture and place a border around it. It would be natural to use a Rectangle as the border, and have a function that would receive as parameters the original Picture and a Rectangle and return a new Picture with a border:

Picture addBorder(Picture picture, Rectangle border)

This function would be responsible for adjusting the height and width of the border to fit the dimensions of the picture.

What is the problem now? Because Square is a subclass of Rectangle, we could pass an instance of Square as the border parameter. But then, if the picture is not a square, it should not be possible to change the border’s height and width, or otherwise the border would no longer be a square.

The conclusion is that a square cannot be used in every situation in which a rectangle can be used, and therefore the class Square should not be a subclass of Rectangle.

But what is the root of the problem?

In the world of geometry, figures are static objects with unchangeable properties. But in the world of object-oriented programming (OOP) objects also have functionality that implement a well-defined dynamic behavior.

Structurally, Squares are a specialization of Rectangles, since both have four sides and all internal angles right. A Square is special because it has its height equal to its width.

Functionally, Squares do not behave like Rectangles. A Rectangle should have independent methods to change its height or its width. Clearly, these methods would not be appropriate for a Square.

Inheritance should be used only if the is-a relationship is both structural and functional.

Posted in OOD, OOP | Tagged , | 4 Comments

Is a square a rectangle? – On the importance of definitions

Posted on September 15, 2010 by Hayim Makabee

Before you continue reading this post, please answer the following question:

 “Is a square a rectangle?”

This is apparently a very simple question, to which we should be able to answer almost immediately, without even thinking about it.

However, if you try asking different people, you will discover that they disagree. Some are absolutely convinced that a square is a kind of rectangle, while others are sure that squares and rectangles are completely different geometric figures. It may be particularly interesting to ask children, and see that most probably they will be very surprised with this question.

Now let’s say that you think that a square is a rectangle, while a co-worker believes that a square is not a rectangle. How will you be able to convince him that you are right? How can you explain to him why do you think that a square is a rectangle?

The answer is that you must first agree on the definitions of what is a square and what is a rectangle. These are some possible definitions:

  • Rectangle: A four-sided plane figure with four right angles.
  • Square: A quadrilateral having all sides equal, and its interior angles all right angles.

From these definitions, it is clear that a square is a special kind of rectangle that happens to have all sides equal. Therefore, if you and your co-worker agree with these definitions, you will also agree that a square is a rectangle.

It happens very often that people working together are not able to agree on some decision simply because they did not make an effort to clarify and agree on the basic definitions of the problem being discussed. Of course this happens in software development projects as well.

Having clear and precise definitions is particularly important in the system requirements specification phase. Most systems have two types of requirements: functional and non-functional.

Functional requirements describe the business logic that the system must implement. Today we have learned how to define functional requirements in details, using relatively formal tools such as use-cases and sequence diagrams.

Non-functional requirements describe the expected system performance. They include technical parameters such as latency and throughput, as well as system attributes such as scalability and robustness. Unfortunately, it happens frequently that non-functional requirements are not given the attention they deserve.

For example, suppose that you are developing a server-side system, and that the specification requires it to be able to handle at least 400 transactions per second (TPS). At first glance, this seems to be a well-defined requirement: It gives you a precise number. However, it is in reality a very problematic definition.

In any interesting system there are different kinds of transactions with diverse complexities. In client-server systems there will probably be login and logout requests. Logout should be a simple request, implemented by updating the user status and doing some clean-up. Login, in contrast, may be much more complex, requiring authentication and authorization. The required TPS is only meaningful if you know how the different types of transactions are mixed together.

Assume that this is the expected transaction distribution of your system:

  • Login: 5%
  • Create record: 10%
  • Read record: 40%
  • Update record: 30%
  • Delete record: 10%
  • Logout: 5%

Now the requirement of 400 TPS is much more meaningful. You know exactly how many transactions of each kind you should expect. If for some reason the system is not efficient enough to handle 400 TPS, you know that the two more frequent types of requests are responsible for 70% of the traffic, so that they should be the first candidates for optimizations. The QA team should now be able to develop automatic tools that reproduce real world load, and test the system under stress.

Without clear definitions it is very easy to implement a rectangle when you should be implementing a square…

Posted in OOD, Requirements Specification | Tagged , | 2 Comments

Efficiency vs. Effectiveness

Posted on September 12, 2010 by Hayim Makabee

This is a blog about Effective Software Design, so it is important to first make clear what do we mean by “effective”.

Peter Drucker defined the difference between being efficient and being effective:

• Efficiency is the capacity to do things right.
• Effectiveness is the capacity to do the right thing.

A manual worker is expected to be efficient. He must perform his task following the orders he was given by his superiors. His work is judged in terms of the quantity and quality of items he is able to produce in some unit of time. The manual worker must do things right.

A knowledge worker is expected to be effective. His work includes making decisions, and therefore he cannot be directly instructed by others about how to perform his task. He produces knowledge, ideas and information that must then be applied to become products. The knowledge worker must do the right thing.

In the past, programmers were more like manual workers. The programming languages were very simple, with just a few elementary constructs. The programmer’s task was to assemble instructions in order to obtain the desired effect. Programmers had to be efficient, and the efficiency of the programmer could be measured by the number of lines of code (LOC) that he could produce in some unit of time.

Today, with the advent of object-oriented programming languages, the nature of programming tasks has changed. For any possible problem being resolved there are certainly libraries of classes that can be easily reused. For many complex systems there are frameworks that already provide most of the required functionality, and that need only be extended with a relatively small development effort.

In this new reality, measuring the LOC produced by the programmer is most likely the wrong thing. Because if he is writing too much, he is probably reusing too few. It is common to see less experienced programmers constantly “reinventing the wheel”, creating again classes that are ready to use in some library. As Peter Drucker said:

“There is nothing so useless as doing efficiently that which should not be done at all.”

Therefore, programmers today should be expected to be effective. They should know about the available frameworks and class libraries, and their work should consist mainly in deciding how to extend and reuse them. These decisions are design decisions, and the effective programmer must be able to compare design alternatives and select “the right one”.

For example, Design Patterns can be described as effective software design decisions for several abstract problems that were exhaustively studied and classified.

I hope the articles in this blog will help you develop your skills and make you more effective as a software designer, so that you will be able to “do the right thing”.

Posted in Efficacy | Tagged | 6 Comments