Programming Without an Internet Connection

Posted on August 29, 2011 by Hayim Makabee

Are you able to program without an internet connection?

In my case the answer is definitely “No.” In most of my programming tasks I need to get some help from the internet, generally in the form of documentation and examples. I’m currently programming in Java, and frequently using new libraries, frameworks and APIs. Even in the Java standard libraries there are so many classes that it is very hard to know how to use them correctly. This situation reminds me a quote:

“Knowledge is of two kinds. We know a subject ourselves, or we know where we can find information on it.” – Samuel Johnson

A few months ago I was working on a Web-based demo, which included the use of HTML, CSS and JavaScript. I’m not an expert on these technologies, but still I wanted to be sure I was implementing things correctly. So, for each problem I had, I asked myself: “Is it possible that none had this problem before?” Then I looked for solutions and with enough patience and a bit of luck I was always able to find an example similar enough to what I needed to do. I never came back of a search with empty hands.

When I program in Java I’m always concerned about not reinventing the wheel. If I need some specific function not yet provided by the classes I know, first of all I search if there is some well-known public library that can be useful. Then I download it and write some short program to test it. Sometimes I find several such libraries that appear to be potentially adequate for my needs. In this case I will spend enough time comparing them, and perhaps even testing all of them.

Therefore, the internet is much more than a source of information in the form of documentation and examples. It is the ultimate reusable code repository, from which millions of developers worldwide are frequently downloading the latest libraries. This level of global reuse certainly saves us a huge amount of development effort, making our work as programmers much more productive and even more pleasurable.

If possible, I would like to hear about your personal experience. Please leave a comment answering these questions:

  1. How frequently do you access the internet when you code?
  2. For what purposes do you access the internet when you code?
  3. Do you feel you are less productive when there is no internet connection?

Thanks!

Posted in Efficacy, Software Reuse | Tagged , | 7 Comments

How Decision Fatigue Affects the Efficacy of Programmers

Posted on August 23, 2011 by Hayim Makabee

In the past, programming was an activity based on assembling low-level instructions using a small numbers of structures: there were loops, conditional statements and function calls. Systems were built through the application of relatively few patterns at different scales. Even the design was very simple: Dataflow Diagrams (DFDs) consisted of only processes, data flows and data stores; Entity-Relationship Diagrams (ERDs) had only entities and relationships.

Today, programmers work on a much higher level of abstraction, applying the mechanisms of Object-Oriented Programming, adopting Design Patterns, reusing sophisticated class libraries and extending powerful frameworks. Besides writing new code, developers are also expected to perform parallel tasks such as Refactoring and writing Unit Tests. The design phase requires a great diversity of complex diagrams, like the ones defined in UML.

The consequence is that today developers are making very important decisions at each step of their work. They must choose between composition and inheritance. They must decide if a class in a library is appropriate for their needs. They must analyze the trade-offs involved in the adoption of different Design Patterns. They must plan how to Refactor their code and how to assure good coverage through Unit Tests. Many of these decisions are not really programming issues in the traditional sense; they are in fact design decisions.

Then, if programming today can be defined as a series of high-level decisions, the question is: How can we assure that the developers are making these decisions effectively?

Decision Fatigue

A recent article at the New York Times addresses the issue of Decision Fatigue. The article mentions several researches that prove that people who are required to make many decisions become mentally exhausted, until they reach the point in which they are no longer able to make effective decisions. In its words:

“No matter how rational and high-minded you try to be, you can’t make decision after decision without paying a biological price. It’s different from ordinary physical fatigue — you’re not consciously aware of being tired — but you’re low on mental energy. The more choices you make throughout the day, the harder each one becomes for your brain, and eventually it looks for shortcuts, usually in either of two very different ways. One shortcut is to become reckless: to act impulsively instead of expending the energy to first think through the consequences. The other shortcut is the ultimate energy saver: do nothing. Instead of agonizing over decisions, avoid any choice.”

One concrete example is Copy-and-Paste programming. It is very common for a programmer to need a piece of code that is very similar to something he has already written in the past. The best way to reuse this code would be to Refactor it so that it can be shared, but this involves many decisions about how to do it correctly without breaking existing code. Thus, if the programmer is tired, he will probably just copy-and-paste it, because it is the most impulsive action and requires minimal mental effort.

The problem of Decision Fatigue is particularly important when the person needs to make trade-offs. As we know, most design decisions require evaluating different options and analyzing their advantages and disadvantages. In the words of the article:

“Part of the resistance against making decisions comes from our fear of giving up options. Once you’re mentally depleted, you become reluctant to make trade-offs, which involve a particularly advanced and taxing form of decision making. To compromise is a complex human ability and therefore one of the first to decline when willpower is depleted. If you’re shopping, you’re liable to look at only one dimension, like price: just give me the cheapest. Or you indulge yourself by looking at quality: I want the very best.”

I believe that every developer has experienced this single-dimension reasoning, making decisions that appeared to be good at the moment, but then proved to have some very serious negative consequences. For example when a programmer implements a new data structure and forgets about the need to keep it synchronized in a multi-threaded environment. Sometimes these bad decisions are just discovered when the system starts exhibiting some very strange bugs and unexpected behavior.

So, to assure that developers are making decisions effectively, we must either avoid Decision Fatigue or provide ways to recover from it. The article recommends consuming sugar, for example in the form of soft drinks. I would recommend making a short break whenever you feel tired, and spend a few minutes in some activity that does not require much mental effort. And have a Coke.

The most important conclusion is that, if a developer is making important decisions, he should not be required to work long hours. It is absolutely impossible for a human being to continue making effective decisions after many hours of work, no matter how smart or experienced he is.

Posted in Efficacy, Psychology of Programming | Tagged , | 5 Comments

The Wisdom of Agile Development

Posted on July 24, 2011 by Hayim Makabee

What is wisdom? This is a definition from the Ethics Resource Center:

“The ability to make good judgments based on what you have learned from your experience.”

According to this definition, a wise man is someone with vast knowledge and experience, a person who can teach others.

However, the Jewish sages had a different view of wisdom, as expressed in the ancient book “Ethics of the Fathers” written almost 2000 years ago:

“Who is wise? One who learns from every man.”

Thus, to become wise, a person needs to have contact with many others, to be exposed to their opinions, and to learn from their knowledge and experience.

I believe that Agile software development, among its many advantages, creates many such opportunities for sharing knowledge and learning from others. Let’s consider a few examples of situations in which this may occur:

Code Reviews

Agile methodologies give emphasis to the importance of Code Reviews. This is a great chance for each developer to receive some feedback on the quality of the code he is writing. My experience is that Code Reviews have the potential to significantly and quickly improve the programming skills of the members of the team. They create opportunities to discuss design alternatives and analyze the implications of the adopted algorithms and data structures. Of course there is no contradiction between Code Reviews and traditional development methodologies, but in practice Agile teams dedicate more time to this activity.

Pair Programming

Pair Programming is one of the most fundamental concepts of Extreme Programming (XP). The idea is that programmers should not work alone, but in pairs. Then, at the exact moment one programmer is typing the code the other programmer is reviewing it. It is like an immediate code review, with the advantage that the developer always has someone to discuss his decisions at the moment he is writing his code. Of course this is considerably more expensive than traditional development, but experience has proved that Pair Programming has a great positive impact on the quality of code.

Stand-Up Meetings

During the daily Stand-Up meeting in Scrum, people can share what they are doing. The meeting itself is not the appropriate place to discuss technical issues, but it strengthens the connections among the team members, removing communication barriers and creating new opportunities for the exchange of knowledge and experience. After Stand-Up meetings each developer understands what the others are doing, so that he knows from whom he should ask for help on a particular issue.

Retrospective Meetings

At the end of each Sprint, the team organizes a meeting to evaluate its own performance. This is a great opportunity to discuss what was done right as well as what were the mistakes, thus learning from the shared experience. This kind of feedback mechanism is essential in a self-management approach such as Scrum. Its main goal is to constantly improve the process, but another consequence is that it forms more effective team members.

In summary, Agile methods introduce many new opportunities to learn from others by giving emphasis to team work. According to the ancient Jewish sages, this is the key to real wisdom.

Posted in Agile, Jewish Sources | Tagged , | 1 Comment

Who is an Expert? – On Rules and Exceptions

Posted on July 18, 2011 by Hayim Makabee

It is relatively easy to find very young people who are brilliant programmers, who seem to know every detail about a particular language, platform or technology. Some of them will be able to implement a nice Android app in a few days, others can write cool Facebook games in a week.

These bright young programmers are frequently very quick learners as well. Give them a new programming language and they will grasp its syntax in a couple of hours. Present them a new API and they will very soon understand all the differences among similar methods and the roles of each parameter.

Now the question is: Do their knowledge and ability to learn transform them into expert software developers?

Whenever I meet these young programmers, I remember this quote:

“The young man knows the rules, but the old man knows the exceptions.” – Oliver Wendell Holmes

So my answer to the previous question is “No”. Knowing the rules of a programming language or platform can make you very effective when working in a particular environment. But to really be an expert you must know the exceptions. You must know when to violate the rules.

1st Example: Composition versus Inheritance

I once worked with a good Java programmer that followed the rule “favor composition over inheritance”. The problem is he was fanatic about it: In all situations he used only composition. In his programs you could not find a single class extending another class. If he wanted to reuse a class, he would just encapsulate it in a new class and forward most method calls. A rule with no exceptions allowed…

2nd Example: Const-Correctness

I once participated in the development of a very large C++ system. I noticed that almost none was using const methods, and as a consequence they could not pass const parameters, even if an object was not supposed to be changed by a function.  I was surprised; it seemed that most people were not aware of the importance of const-correctness.

But then I discovered another reason for this mystery: Most programmers didn’t know about the possibility of declaring a data member as mutable. Then, many classes that used synchronization mechanisms had not const methods, even when the state of the object was not being changed. Even the accessor methods could not be declared as const, because they were changing some lock.

The rule says that const methods should not change the values of data members. But the real intention is that they should not change the object state. Hence the exception to the rule is that Semaphores and Mutexes can be changed by any method, because they are not part of the object state. They should be declared as mutable.

Conclusion: It is relatively easy to learn the rules, but only experience can teach you the exceptions. If you know the rules and the exceptions, then you are an expert.

I will be happy to hear in your comments more examples of rules and exceptions. Please share your experience. Thanks!

Posted in Efficacy, OOD, OOP | Tagged , , | 4 Comments

Design Patterns and the Tower of Babel

Posted on July 11, 2011 by Hayim Makabee

This is the story of the Tower of Babel from Wikipedia:

According to the biblical account, a united humanity of the generations following the Great Flood, speaking a single language and migrating from the east, came to the land of Shinar, where they resolved to build a city with a tower “with its top in the heavens… lest we be scattered abroad upon the face of the Earth.” God came down to see what they did and said: “They are one people and have one language, and nothing will be withholden from them which they purpose to do.” So God said, “Come, let us go down and confound their speech.” And so God scattered them upon the face of the Earth, and confused their languages, and they left off building the city, which was called Babel “because God there confounded the language of all the Earth.” (Genesis 11:5-8).

There is a very clear message in the story of the Tower of Babel: When people speak a common language, they may succeed in very challenging projects. If, however, everyone speaks a different language, the project will surely fail.

The catalogue of Design Patterns in the Gang-of-Four book was a very important contribution to the software development industry worldwide. Many programmers improved their object-oriented design skills, learning from the examples collected in this work. But, in my opinion, the book’s main gift to us was the creation of a common language on a higher level of abstraction.

Before this book, software engineers tried to express their ideas using the basic concepts of Object-Oriented Programming (OOP): inheritance, composition, encapsulation, dynamic binding and abstract methods. There were very few names to capture a portion of the design, i.e. the relationships among several classes in a system. One of the few examples was Model-View-Controller (MVC).

Then, when the Design Patterns book became popular, as an immediate consequence a common vocabulary was created. Suddenly people were able to talk about Singletons, Visitors, Factories and Strategies. Today, when you mention any of these names, you expect other developers to understand exactly what you are talking about, without any further explanation.

We know that it is a characteristic of each profession to have its specific vocabulary, which allows work to be done more effectively. In this sense, I believe that the Design Patterns book was a big step for the maturity of software engineering as a profession.

Posted in Design Patterns, OOD | Tagged , | 2 Comments

The Facebook Killer: Lack of Reciprocity

Posted on July 3, 2011 by Hayim Makabee

Some people claim that Facebook will be replaced by some other social network such as Google+. I think that the real Facebook killer will be the Lack of Reciprocity:

If you are constantly posting about your personal life, you will eventually discover that your hundreds of “friends” know everything about you, but most of them don’t want to tell you about their lives.

This is my classification of Facebook users, according to the way they use it and the kind of information they publish: 

    • Naïve Users: Are very active users, publishing lots of personal information and family pictures. They will let you know everything about their lives: In which restaurants they eat, what clubs they frequent, where they are planning their vacations. They have many albums with lots of pictures, some of them very personal. They also frequently comment and like the posts of others.
    • The Jokers: They like to post statuses that they think are very funny and/or smart, such as: “How are turkeys called in Turkey?” They will never publish anything personal. If they upload a photo, it will probably be something bizarre they saw in the street. They don’t comment or like the statuses of others, but it is very important for them to have as many people as possible liking their own posts.
    • The Linkers: They provide lots of links to things they think are interesting and should be shared, like videos on YouTube, TED talks and news articles in digital journals. They also like to be the first to share some breaking news like “Michael Jackson is dead!” The Linkers never post any kind of personal information.
    • The Gamers: They spend most of their time playing games at Facebook, and their many posts are all notifications of the kind “Nancy has found a lost cow in his farm!” The Gamers don’t post personal information, but they will frequently invite you to play with them.
    • The Fans: They use Facebook very actively to follow everything their friends are publishing. They frequently comment and like other people’s posts, but never write about themselves. They make a true effort to be nice and friendly, but do not share about their own lives.
    • The Wishers: They only use Facebook to wish a “Happy Birthday” to their friends. They connect every day, check who has a birthday, and then post a congratulation message on his/her wall. If there are many such friends, they use “copy-and-paste”. The Wishers rarely write anything on their own wall.
    • The Scanners: They use Facebook to always keep up-to-date about what is happening to his many contacts. They are very interested in other people’s lives, so they read every status and see every picture. If there is not enough activity in the News Feed, the Scanners will start checking who the friends of their friends are. As the name suggests, they never write anything, so they cause you the impression they are not using Facebook. But they are, and a lot!

It is obvious from this classification that the only people publishing any personal information are Naïve Users. Moreover, people using Facebook to read their friends’ posts are interested mostly in the personal information provided by the Naïve Users. The Jokers’ jokes, Linkers’ links and Gamers’ notifications are different types of garbage that only pollute the News Feed. There is nothing really “social” about them, so they do not contribute much in a social network.

I believe that eventually the Naïve Users will understand that they are sharing too much personal information with too many people. They will discover that details of their lives became public domain, while the people consuming this information, their “friends”, do not really reciprocate. Then I doubt they will continue sharing that much.

At the moment that Facebook will lack the personal information provided by the Naïve Users, its social aspect will be lost, and this will greatly affect the motivation of all other types of users.

Technology can provide us new tools to communicate, but it cannot change the inherent defects in human nature.

Posted in Social Networks | Tagged | 2 Comments

The Agile Architect

Posted on June 28, 2011 by Hayim Makabee

The International Association of Software Architects (IASA) in Israel was honored to host a talk by Johanna Rothman last week (June/21/2011).

Johanna Rothman is a consultant, author and speaker with vast experience with project management.

She was visiting Israel and generously agreed to give us a presentation.

In this special meeting, Johanna Rothman talked about “The Agile Architect”.

These are some of the interesting issues that she addressed:

  • What is the role of the Software Architect in Scrum?
  • In Agile, should the architecture be done in advance or incrementally together with implementation?

Johanna was very kind to publish the slides of her presentation.

To complement this material, you can read an interview with Johanna Rothman about “The Role of the Software Architect in Agile Development”.

We at IASA Israel would like to thank Johanna for this very interesting talk.

We would also like to thank SAP Labs Israel for hosting the event and providing the accommodations and refreshments.

If you are interested in future events of IASA Israel, please join our LinkedIn group.

Posted in Agile, IASA Israel, Software Architecture | Tagged , , | Leave a comment

The Single Question Interview

Posted on June 9, 2011 by Hayim Makabee

The routine of a software developer normally turns around design, implementation and testing. But hopefully there will be times in which the team needs to grow, and then he will be asked to do something extraordinary: Interviews.

In the software industry, it is natural to ask technical people to conduct interviews: After all, the company needs to evaluate the professional skills of the candidate, sometimes in a very specific field or technology, and this surely cannot be done by a HR person.

But planning a successful interview is a real challenge. The interviewer must conclude in a short time if the candidate is suitable or not to the open position. Thus, the time of the interview cannot be wasted on questions that do not really contribute to a decision, or, worse yet, questions that may cause the interviewer to make the wrong decision.

A recent article at TechCrunch claims that the most important thing to consider in the recruiting process is the candidate’s achievements. The author recommends:

“Don’t interview anyone who hasn’t accomplished anything. Ever. Certificates and degrees are not accomplishments; I mean real-world projects with real-world users.”

Similarly, Joel Spolsky says that the ideal candidate must be Smart and Get Things Done. According to Joel:

“The only way you’re going to be able to tell if somebody Gets Things Done is to see if historically they have tended to get things done in the past.”

I agree with these views, and I always start my interviews with a question about the candidate’s previous professional experience. I ask the candidate to select one of the projects he has participated in the past, and then describe me its requirements, high-level design and some implementation details. If the project is too big or complex, I may ask the candidate to talk about a particular module, or about a particular aspect, such as scalability or robustness.

In any case, if the candidate really contributed to this project and knows what he is talking about, this first question is just the opening for many interesting ones:

  • What was the biggest challenge in this project?
  • Did this project have any conflicting requirements?
  • What were the trade-offs you had to consider in your design decisions?
  • What were the non-functional requirements and how they affected the design?
  • What were the design patterns that you used in this project?

Thus, by talking about the candidate’s previous projects, we can learn a lot about his real skills and experience. In my opinion, this is much more valuable than discussing generic questions about software engineering or asking the candidate to solve logic puzzles.

But we must pay attention not only to what the candidate says, we must observe how he talks about his experience:

  • Does the candidate talk with enthusiasm?
  • Is he proud of his previous work?

Ideally, the candidate should be so enthusiastic when talking about his contributions to past projects that he should forget that he is in an interview. He shouldn’t feel he is in a kind of exam, but that he is talking to a fellow developer, enjoying the opportunity to describe all the smart things that he did and that he is proud of.

As I said in a previous post, I believe there is nothing more effective than enthusiasm. So we want to hire someone who is smart and gets things done, but with enthusiasm. Achievements alone are not valuable if the candidate is not proud of his achievements.

Good luck hiring the best people; to find them you don’t need more than a single question.

Posted in Hiring | Tagged | 4 Comments

Divide-and-Conquer: Coping with Complexity

Posted on June 6, 2011 by Hayim Makabee

Sometimes a problem is simply too complex for us to solve. Our only chance seems to be breaking it into smaller parts that we know how to deal with.

This strategy of reducing the complexity of a problem by dividing it into simpler sub-problems is known as “Divide-and-Conquer”. In this case there are two assumptions that are normally true:

  • A problem can be divided in several parts, so that each part can be handled independently.
  • Handling each one of the smaller parts that compose a problem is less complex than handling the whole problem, and thus we can “conquer” it.

There are several algorithms that are based in this strategy. For example, the Quicksort algorithm works by recursively dividing the list of values to be sorted into two smaller lists, in which one sub-list has the smaller values and the other the bigger values.

Another example is the Motion Planning problem in robotics, in which the robot needs to find a path to cross a room avoiding several obstacles. It is possible to adopt a divide-and-conquer approach to solve this problem, so that each obstacle is treated independently: The robot first finds possible paths between each pair of obstacles, then merges these smaller steps to create a single plan to pass all obstacles.

Given a complex problem, these are the steps that must be followed to solve it using a divide-and-conquer strategy:

  1. Define a method to decompose the complex problem into smaller parts.
  2. For each part, find a way to solve it.
  3. Combine the solutions of the parts to obtain the solution of the original problem.

Some problems can be solved recursively, as in the example of Quicksort. In this case all sub-problems are solved in the same way in step 2, and it is relatively easy to combine their solutions in step 3.

In the case of a problem that cannot be decomposed recursively, each sub-problem in step 2 may require a different solution, and it may be more difficult to combine the solutions in step 3. This would be the case of Motion Planning if the robot needs to do other types of tasks besides only crossing obstacles. For example, in the middle of its path the robot may be required to recharge its batteries or to carry some object.

In any case, a divide-and-conquer strategy will provide a systematic way to cope with complexity, making difficult problems more tractable by our limited human capabilities.

Posted in OOD, OOP | Tagged , | 2 Comments

On Bubbles and the Value of Data

Posted on June 1, 2011 by Hayim Makabee

In a recent article at Newsweek, Daniel Lyons claims that:

“Data, by itself, is worthless. Facebook is betting it can create software algorithms to extract value from that data—essentially to perform a kind of techno-alchemy and turn zillions of meaningless bits into billions of dollars. Nobody knows if the company can really pull this off…”

I couldn’t disagree more. Today, creating software algorithms to extract value from data is a science, applied by all big companies on the Web.

For instance, the research field of Data Mining focuses on discovering hidden patterns in large data sets. When these patterns are found, what was previously raw data becomes very useful information and knowledge. One example would be an E-commerce company that sells a large diversity of items on the Web. Data mining tools can be used to predict future buying trends and forecast supply demands.

Another important research field is Recommender Systems, whose goal is to recommend items of interest to users based on preferences they have expressed, either explicitly or implicitly. Recommender systems increase both the user satisfaction and the revenue of the companies using it, because the user is exposed to the items that interest him most and thus has a higher probability of acquiring them. The best-known example is Amazon, which is very successful in recommending books to their customers based on their previous purchases.

In the case of Facebook, at this moment they have almost 700 million users, and continue to grow fast in new markets. For each one of these users they have very valuable data, including detailed user profiles and their connections. These data can be easily used to select precisely targeted advertisements, what as a consequence increases the price they can ask for each such advertisement.

We just recently saw the IPO of LinkedIn, and it is now valued at more than U$10bn. This value is not derived from paying users, because most services are free. The real value of LinkedIn comes from more than 100 million very detailed professional profiles, including precious personal information that no other company has. Again, this allows them to display advertisements that have a high probability of being clicked, and thus a high probability of generating revenue.

Facebook and LinkedIn are not bubbles. I and my friends at Yahoo! Labs are working hard devising new ways to extract value from data. We know how valuable data can be. None should underestimate it.

Posted in Data Mining, Recommender Systems | Tagged , | Leave a comment