Planning Poker: Avoiding Fallacies in Effort Estimates

Posted on August 5, 2012 by Hayim Makabee

Many years ago I was working as a software developer in a team with three other programmers. We once had a meeting in which our Team Leader said:

“You are late again! All of you are late! Actually, you are always late! We cannot continue like that!”

Then I replied to him:

“If all of us are late, and if we are always late, then there must be a problem in the way we do effort estimates…”

In this case, the Team Leader himself was the one doing the estimates. He did it alone, based only in his own judgment, without consulting us, the developers. When he assigned new tasks to us, it was something like:

“You now must do this task, and it must be ready in X days.”

Certainly this is not a very nice way to talk to people working with you, but the main problem here is that it is not effective. For any person, it is very difficult to do precise effort estimates. Thus, we should always avoid doing estimates alone and without asking for other opinions.

Planning Fallacy: Why Estimates Are Hard

The difficulty of people to make good effort estimates is a well-known phenomenon, and as such it was the subject of research. Scientists have called it the Planning Fallacy:

“The Planning Fallacy is a tendency for people and organizations to underestimate how long they will need to complete a task, even when they have experience of similar tasks over-running.”

The Planning Fallacy is discussed by Daniel Kahneman, Israeli Nobel Laureate in Economics, in his book “Thinking, Fast and Slow”. According to him, plans and forecasts suffering from this effect have two main characteristics:

  • “are unrealistically close to best-case scenarios”
  • “could be improved by consulting the statistics of similar cases”

Psychologists have tried to explain the causes for the Planning Fallacy. Some claim that the main reason is that people have a tendency to focus on the most optimistic scenario, actually believing that this optimistic scenario is more probable than it really is. Others claim that this is just another example of wishful thinking. This means that people think about what they would like to happen, and not about what is likely to happen.

I think that in the workplace there are some other reasons for the Planning Fallacy:

  • If the manager is the one doing the estimates, he has an interest to make them hard to achieve, in order to push people to work more productively. Managers always want to see the work completed as fast as possible.
  • If the worker is the one doing the estimates, he will be ashamed to assign too much time to himself. People are afraid of doing pessimistic estimates for their own tasks, because they may appear to be lazy or inefficient.

Thus, it is quite clear that there should be other people participating in the estimates, and not only the manager and the worker who will perform the task. People estimating from the outside, who do not have a personal interest in the task and will not be responsible for it, can be more objective.

Planning Poker

The methodology of Agile software development has addressed the problem of poor estimates through the concept of the Planning Poker, in which all members of the team participate and contribute. Each member of the team has a set of numbered cards, with the numbers representing the effort to implement a task. For each feature in the backlog, each person makes an independent estimate using the cards. Then, the cards are simultaneously shown and people can compare and discuss their estimates.

The goal of the Planning Poker is to reach consensus, in which all participants agree with an estimate after they have discussed it and understood each other opinions. There are several reasons that cause this process to provide more accurate estimates:

  • There are more people participating in the estimates, including team members that will not be directly responsible for performing the task.
  • People are not influenced by others when making their original estimate, since they do that simultaneously and independently.
  • People have to justify their estimates, especially if they are too low or too high.

I think this is a great approach for addressing the Planning Fallacy, and it also has the advantage of being fun and strengthening the bonds among team members.

Do you have experience with the Planning Poker? Please share your thoughts in the comments below.

Posted in Agile, Efficacy, Psychology of Programming | Tagged , , | 20 Comments

Industrial Projects at the Technion

Posted on August 1, 2012 by Hayim Makabee

The faculty of Computer Sciences at the Technion has an Industrial Projects initiative in which undergraduate students work under the supervision of software professionals. I think this is a great idea, a real win-win situation:

  • The students have the opportunity to gain experience working on real projects, under the supervision of industry veterans.
  • The companies proposing the projects gain access to talented students to develop prototypes or proofs-of-concept at no cost.
  • The CS faculty gains access to many industry professionals each semester that effectively become part of their training ranks, also at no cost.
  • The early connection between companies and students may help the students to find a job in the future, as well as help the companies to find good candidates.

In the last semester I had the great pleasure to be the supervisor of a project entitled “Movie Quotes Search Engine”, together with my friend Oren Somekh. We were very lucky to “hire” for this project two brilliant students: Meytal Bialik and Zvi Cahana. The results of their work were very impressive, and above our expectations.

You can check the Movie Quotes Search Engine project to learn more about their work. I hope this serves as inspiration for other students looking for practical experience. It should also serve as motivation for other software companies to cooperate with the Technion in this important initiative.

What do you think about that? I will be happy to have your comments below.

Posted in Yahoo! | Tagged | 6 Comments

An Event-Driven Approach for the Separation of Concerns

Posted on July 4, 2012 by Hayim Makabee

Last week I presented a paper at the International Conference on Evaluation of Novel Approaches to Software Engineering (ENASE 2012), at Wroclaw, Poland.

Title: “An Event-Driven Approach for the Separation of Concerns”

Abstract: “This paper presents an event-driven approach for the separation of concerns in software systems. We introduce the EventJ framework that provides an event-driven extension to the Java programming language. The paper describes a general methodology that can be used to identify the cross-cutting concerns and separate them from the main functionality using events and event handlers. We discuss the pre-requisites to perform this change and illustrate it with a concrete example. Finally, we make a comparison between the event-driven approach and the aspect-oriented one, and conclude that the use of events to separate concerns has a positive effect on software quality attributes such as maintainability, extensibility and reusability.”

You can freely download a full-copy of the paper.

You can also view the slides:

An Event-Driven Approach for the Separation of Concerns

View more presentations from Hayim Makabee

I hope you find that interesting. Feel free to leave a comment below.

Posted in AOP, OOD, OOP, Research | Tagged , , , | 4 Comments

Self-meetings: Avoiding distractions in the workplace

Posted on June 15, 2012 by Hayim Makabee

In the modern workplace, in which we are always connected, there are simply too many distractions. You need lots of discipline to really use your time effectively, besides the capacity to clearly define your priorities. You must be able to identify the reasons your valuable time is not being spent in the most effective way, and then develop methods to fix that.

The chart below, called “How to Focus in the Age of Distraction” has wonderful tips about effective time management and increased productivity. It’s certainly worth more than a thousand words, so I suggest you take a good look at it:

One form of distraction that is not mentioned in this chart are the meetings in the workplace. If we really want to “work in focused bursts”, we must assure that our work is not interrupted by a meeting.

In my experience, this is particularly important for software developers. If a programmer is focused, he can accomplish in two consecutive hours much more than in two periods of one hour each. Every time a programmer starts working on a task, he needs considerable time to “enter the zone”, reaching a state of “flow” and working with maximum productivity.

Too many meetings are certainly a productivity-killer for any professional. But, for a software developer, even a small number of meetings randomly distributed during his work day can reduce drastically his output.

The question is: How can I avoid my work to be interrupted by a meeting?

Self-Meetings

I’ve adopted a simple technique that works very well: When I know that I need time to work on an important programming task, I just go to my calendar and reserve this time for myself, marking it as “busy”. In other words, I create a “meeting” in which I’m the only participant, a meeting with myself. If I know I will need to work for several days on a task, I may just reserve all the mornings in a week. For example, I may create self-meetings, every day, from 8:00 to 12:00.

Thus, when someone tries to invite me for a meeting, he will see that I’m busy, and then he will need to look for an alternative time in which I’m free. Better yet, he may decide that I’m not really needed in this meeting.

Of course you must always leave a certain amount of free time in your calendar. After all, there are meetings in which you need to participate, and you do not want to make it extremely difficult for the meeting organizer to find a suitable time slot.

This technique will not work if your organization has a really bad meeting culture. There two types of common problems regarding meetings:

  • Too many meetings: People are creating meetings that are not really needed.

  • Too many participants: Too many people are being invited to the meetings.

In both cases it is extremely difficult for the meeting organizer to find a time slot in which all the participants are free. Then, you will eventually be invited to meetings during your “busy” time. If your company has such problematic meeting behavior, you must discuss that with your management and your peers. That’s not something that you can solve personally using any individual technique.

Finally, you should avoid as much as possible to use the “suggest alternative time” option when replying to a meeting invitation. This should be done only in unforeseen situations, for example when there is an emergency and you will no longer be able to participate in a meeting that you had already accepted. If you know that you will be busy at a certain time, you should make it explicit in your calendar as early as possible.

So good luck with your self-meetings, I hope this technique will help you become more productive.

You may also want to read my previous post about Super-Productivity.

I will be happy to hear your comments!

Posted in Efficacy | Tagged | 4 Comments

The Seven Rules of Effective Communication

Posted on June 3, 2012 by Hayim Makabee

In every professional environment, it is essential to communicate effectively with your peers. This is particularly true during meetings, in which people expect to discuss a subject, reach conclusions and make related decisions in a short time. Here are seven rules of effective communication:

1. Respect Experience and Seniority

As I’ve discussed in a previous post, there is a very important difference between knowledge and experience. Sometimes it’s hard to believe, but most people reach senior positions through real competence and accomplishments. Thus you should always listen carefully to what the veterans have to say, even if it appears to be wrong. Most probably they can see something that you can’t, so if you disagree with their opinions first ask them to justify their point.

2. Don’t Interrupt the Speaker

If you are participating in a meeting, you should be very interested in understanding what the other participants have to say. But by interrupting others you are not giving them the opportunity to clearly communicate their opinion. This is first of all a sign of disrespect that may cause bad feelings. But interrupting others also gives them the right to interrupt you, creating a chain-reaction with very negative consequences.

3. Think Before You Answer

Of course we must think if we are not sure about the answer, but we must also think even if we could answer immediately. When participating in a meeting, we are not only interested in expressing our views, we must also be able to explain and justify our opinions. Especially when we know in advance that someone may disagree with us, we need to think about how to present our ideas in a clear and solid way.

4. Focus on the Subject

For meetings to be effective we must focus on the topic being discussed. There is a limited amount of time available to reach a conclusion, and we do not want to waste this time with parallel issues. If there are other important topics that need to be discussed they should deserve their own meeting. Thus when answering  to someone always address directly the question, and never ask questions that are not related to the discussion.

5. Organize Your Speech

When answering a question you should define your priorities. Say the most important things first, and add more comments only in the case they are necessary. If you were asked several questions, answer them in the same order they were asked. Also, answer each question separately. It will be much easier for the other participants to understand you if you present your thoughts in a clear order.

6. Accept that You Don’t Know Everything

If you are asked a question for which you don’t have a good answer, it’s perfectly natural to say that you simply don’t know. Of course you should know the answer for any question directly related to your personal tasks. But when the topics being discussed are related to the work being done by teams of people, you’re not expected to know all the details.

7. Don’t Argue the Facts

You should have your personal opinion about all the important issues being discussed. This is the contribution that the other participants expect you will bring to the meeting. You should know how to explain and justify your opinions. But you should also know how to stop debating when you are presented with new facts that clearly contradict your view.

Conclusion

What do you think about these rules? Do you agree with them? Do you think that if people followed these simple rules their communication would be more effective? And what if I told you that these seven rules where written more than 2000 years ago, in a Jewish book of ethics called Pirkei Avot?

From Pirkei Avot, Chapter 5:

“There are seven things that characterize a boor, and seven that characterize a wise man. A wise man does not speak before one who is greater than him in wisdom or age. He does not interrupt his fellow’s words. He does not hasten to answer. His questions are on the subject and his answers to the point. He responds to first things first and to latter things later. Concerning what he did not hear, he says “I did not hear.” He concedes to the truth. With the boor, the reverse of all these is the case.”

Yes, technology has changed a lot in the last 2000 years, but it seems that human nature remains the same.

I hope you have enjoyed this post and will appreciate to hear your opinion, please leave your comments below.

Posted in Efficacy, Jewish Sources, Psychology of Programming | Tagged , , | 4 Comments

Code Swarm: Visualizing the Evolution of Software Systems

Posted on May 24, 2012 by Hayim Makabee

Software systems grow over time. They evolve through additions, extensions and modifications. A successful application is not one that satisfies all the original requirements and then is delivered to its users and remains unchanged. If people are using a system, they will require it to be changed. They will have ideas about new features and capabilities that were not originally planned, and will feel that they really need them. Of course users also find bugs that need to be fixed, but this kind of maintenance is actually a small effort when compared to the addition of new functionality. Thus, if a system is not evolving it is because it is not being used, and if a system is being used it certainly will need to evolve.

Now the question is: How do systems evolve?

For a particular system, we can ask specific questions:

  • What parts of the system are being changed more frequently?
  • How the total number of files is growing over time?
  • How the work is being distributed among the software developers?
  • Are there developers that contribute more than others?
  • Is there an “ownership” relationship between developers and software components?

The answers for these questions can be obtained by analyzing the history of the software repository. Every commit represents a change done by a particular developer in a specific file. A commit also means the existence of a relationship between a programmer and a file at a particular moment in time. This relationship may continue for a period of time, if the same person makes several successive changes on the same file.

But for big systems it may be difficult to summarize all this information in order to understand how the system is growing and being changed. Thus, it would be very nice if we could visualize graphically the evolution of software systems. And this is exactly the concept behind Code Swarms, as idealized by Michael Ogawa:

“This visualization, called code_swarm, shows the history of commits in a software project. A commit happens when a developer makes changes to the code or documents and transfers them into the central project repository. Both developers and files are represented as moving elements. When a developer commits a file, it lights up and flies towards that developer. Files are colored according to their purpose, such as whether they are source code or a document. If files or developers have not been active for a while, they will fade away. A histogram at the bottom keeps a reminder of what has come before.“

For an example, see below the Code Swarm for Twitter:

It is interesting to note how a small number of developers play a very central role in the evolution of Twitter, changing a large quantity of files during a long period of time. It is also interesting to see that other developers join the project, give a relatively small contribution, and then “fade away”. Of course here we can only observe the number of files being changed, and not the size of the contribution in lines-of-code (LOC) neither the complexity of the code being added or modified. But even so it is clear that a few programmers are “hubs” while others are in the periphery.

I really find this useful, and intend to run Code Swarm for the systems I’ve developed recently with my colleagues at work.

I will appreciate to know what you think about this idea of Code Swarms, please leave a comment below.

Posted in Software Evolution | Tagged | 2 Comments

Communication Problems in Software Projects

Posted on April 23, 2012 by Hayim Makabee

The cartoon above is the best illustration I know for the so usual communication problems in software projects. This difficulty to capture the customer needs and correctly translate them in the form of requirements, design and working code is one of the main motivations for the methodology of Agile software development.

As I discussed in a previous post, I believe in iterative methods and fast prototyping. I have been applying this approach in all my projects in the last years, and I’m very happy with the results. Among the benefits, I have been able to deliver more frequently and adapt more easily to changing requirements.

Thus, if you are frustrated with your current development lifecycle, you should know that there are alternatives, and try them. Then, perhaps this cartoon will become a thing of the past…

Posted in Agile, Requirements Specification | Tagged , | 2 Comments

How to find the best candidates without interviews

Posted on March 28, 2012 by Hayim Makabee

Interviewing candidates for a job is a very expensive process. If your intention is to hire only the best people, as it should be, you will probably have to interview dozen of candidates until you find one good match. And this of course is after the screening process, in which the potential candidates were already selected through careful analysis of their skills and experience, and perhaps other filtering mechanisms such as phone interviews.

What is the cost of each interview? There is the obvious cost of the interviewers’ time, including their preparation before the interview and the time spent writing their conclusions after it. Companies differ in their selection methods, but in general the entire hiring process will always include a series of interviews with people with rising levels of seniority. Thus, to find one good match, the people involved with the hiring process need to invest dozens of work hours, many of them dedicated to candidates that will be rejected. Clearly, we would like to make this process more effective, both to save money and to make it less frustrating for all the people involved, including the interviewers and the candidates.

There is an additional cost if the candidates are coming through job placement agencies. In this case, the agency will normally get one full month of salary for each candidate that is hired. Of course it would be much more cost-effective if we could attract good candidates without the help of these agencies. Better yet, it would be great if we could get the interest from people who are not actively looking for work. We want to hire the best people, but the best people normally already have good jobs and are happy with them. Unfortunately, the truth is that the pool of people actively looking for work is below average, either regarding their skills or experience.

To summarize, we would like a hiring process that:

  • Is extremely cost-effective.
  • Requires a minimum number of personal interviews.
  • Is able to attract good candidates without the help of placement agencies.
  • Is able to attract good candidates that are not actively looking for a new job.

This is exactly what was idealized by Hola, an Israeli Internet start-up. They have created a promotion in which software developers have to solve a programming challenge in order to win money prizes. A programmer submitting a “perfect” solution gets the top prize of 2000 Shequels (~500 US dollars). Their idea is viral since people can also win prizes if they forward the challenge to their friends, and if their friends solve it. If you are a programmer, you can try their challenge.

According to Ofer Vilenski, Hola’s CEO, several hundreds of Israeli programmers already tried to solve the challenge, but only a few were able to do it perfectly. These few ones that won the prize are considered by Hola to be top programmers, exactly the kind of people they would like to hire. The entire process so far cost to them only a few thousand Shequels, plus the time required to check the solutions, which they claim is very small for each solution. The prize winners are being invited for interviews, even if they were not actively looking for a job.

I’m personally very impressed with the efficacy of this model, which is also very original. But I’m not sure if I would consider adopting it in the future. As I said in a previous post, I believe in the importance of soft skills, but that cannot be measured in a programming challenge.

What do you think? I will be glad to hear your opinion about this idea. Please leave a comment below.

I also invite you to check my previous posts about the hiring process. And if you are a candidate looking for a job, I wish you good luck!

Posted in Hiring | Tagged | 4 Comments

An Interview Question for Fresh Graduates

Posted on March 15, 2012 by Hayim Makabee

Hire Me!It is extremely difficult to evaluate a candidate in a job interview, and even more difficult if this candidate is a fresh graduate. In this case we can talk about his grades, the courses he took and the projects he did during his studies. But we cannot learn from his previous experience. Of course we can focus on his soft skills or his problem-solving abilities, but we still need to predict his performance doing a job that requires specific technical skills.

In the case of software developers, we would like to predict if the candidate will be an effective programmer and a good team member. As I’ve written in a previous post, I believe that nothing is more effective than enthusiasm. Then, the question becomes: Will this candidate enjoy being a programmer? Will he be enthusiastic about his tasks?

I believe this question is extremely relevant today, because software development became a popular profession with very high salaries. In the past, people wanted to study Computer Sciences because they really were interested in working with computers; they were very technical-oriented and liked Math. But unfortunately today it is very common to meet students that chose Computer Sciences just because they know that it will be easy to find a good job with a great salary. And I say “unfortunately” because if these students do not really enjoy programming they will not be happy in their future jobs.

Thus, when interviewing the fresh graduate, we want to know: Is this guy of the first kind, the one that always loved computers and has a passion for programming? Or is he of the second type, the pragmatic that chose his faculty with the goal of finding a good job afterwards, but without any special attraction to algorithms and data structures?

Malcolm Gladwell, in his book “What the Dog Saw”, has a chapter entitled “Most Likely to Succeed” in which he discusses the challenge of hiring the best candidate. He mentions one corporate recruiter who used to ask the question “Give me a typical day”. I think this question is excellent, especially for fresh graduates, and I can imagine it in an interview:

Interviewer: “Please describe your typical day.”

Candidate: “I wake up at 7:00. I arrive at my first class at 8:00 and then I’ve classes until 12:00. I eat lunch with my friends, and then I spend the afternoon doing my home works or projects. Sometimes I need to go to the library or to a computers farm in the faculty. Around 18:00 I go either to the gym or to the swimming pool. When I come back home I have dinner and then I watch TV or surf the Web until I go to sleep around 11:30.”

Interviewer: “And when do you code?”

Candidate: “Well, I normally have projects in which I have to code, and some of my home works also require some programming.”

Interviewer: “Sure. But when do you write your own programs?”

If a candidate presents himself as a programmer, I expect him to have his own pet projects, besides home works or projects he had to do during his studies. Today there are many platforms that allow programmers to develop interesting ideas with relatively small effort: Web sites, Facebook games, iPhone or Android apps. These are examples of applications that can be implemented by a single person working during his free time. And of course there is also the option of contributing to a much greater project as part of an Open-Source effort.

If you are a fresh graduate, perhaps you got a nice diploma, but if you do not have any personal project I doubt you truly enjoy programming. And since I believe that you must really have fun coding to be a great programmer, I’m sorry, but if you do not have pet projects I would not hire you.

You may want to take a look in my other posts about the hiring process.

Feel free to leave a comment or question, and good luck!

Posted in Hiring | Tagged | 7 Comments

IASA Israel meeting about TRIZ

Posted on February 19, 2012 by Hayim Makabee

The first activity of the International Association of Software Architects in Israel in the year of 2012 was a meeting about TRIZ – The Theory of Inventive Problem Solving.

We had an introductory talk by Ido Lapidot, TRIZ expert at Intel Haifa, followed by a panel with other TRIZ specialists to discuss the applications of TRIZ in the industry.

Our invited speakers were:

Ido Lapidot: Directed Evolution and Systematic Innovation Expert (TRIZ) at Intel Strategic Planning Group.

Vladimir Petrov: TRIZ Master, President of the Israeli TRIZ Association and VP R&D at Innovation Technology.

Shadi Saba: Senior Research and Design Automation Team Manager at Intel Corporation.

This meeting was sponsored by SAP Labs Israel.

If you would like to be informed about our next meetings, please join IASA Israel group on LinkedIn.

You can watch below Ido Lapidot’s talk about TRIZ and its applications (in Hebrew):

Introduction to TRIZ: Theory & Practice from Michael Kalika on Vimeo.

Posted in IASA Israel | Tagged | Leave a comment