Grey Swan and Chicken Run

The trigger for this blog was the recent bbc article ”Black swans’ busting IT budgets” that you can find here. The article starts with a fact that one in six big IT projects go over-budget by an average of 200%, more than in any other industry. What got my attention was not that figure but the proposed remedy for it, which you can find at the end of the article: “Managers are very likely to run into black swans. They need to be able to identify them and prevent them”. Oh, my God…

So, let’s start first with Taleb, and his famous book Black Swan. In his retrospective about the book in this New York Times article, he made a nice summary of what he ment by Black Swan event: “First, it is an outlier, as it lies outside the realm of regular expectations, because nothing in the past can convincingly point to its possibility. Second, it carries an extreme impact. Third, in spite of its outlier status, human nature makes us concoct explanations for its occurrence after the fact, making it explainable and predictable. ”

What often you can find in the management books about the risk management is one rather simple formula: Exposure to the risk  = probability of the event * negative impact of that event.

To be frank, that is all you need to know about the risk management: you create a plan, make list of all issues that you THINK need your attention, sorted by the exposure, make the action points register, and regularly monitor each item till you finish the project. This is not Black Swan management, this is a simple risk management tool – and it always works (for what is supposed to remedy). That combined with iterative development, which deals with other risks such as changing requirements is what most of the agile teams do these days.

What I liked about the article though, was that it recognized the fact that this approach is not good enough, and that the biggest problem you will encounter in IT projects are often the ones that suddenly happen. So, what we can do about it? We can do something, but definitely not plan for uknown, this is rubish.

Let’s first try to define types of Black Swans that I have encountered so far. Here I am walking on a slippery road, since unknown events are of course of the unknown types. So, I will call my swans grey swans (see picture below). I think this can get me out of the trouble…

  1. People leaving the team: It often comes in waves where more people leave at once, and it can be either result of the bad management, HR policy, new sexy start-up in the neighborhood… In one of my previous posts, I have proposed few ways to look into this problem. The most important thing though, no matter how it happened, you have lost the most valuable part of the project – people who know how to run it. When this happen, you need to staff new team members, train them on the product, which will obviously keep the rest of the original team busy as well. To the outside world, nothing is really happening. Therefore, you as the project manager, might be called into endless meetings, where people want to see actions, while the only thing you can ask for is patience.
  2. New framework/platform failure Any time you introduce a new database version, application server or similar platform upgrade, you really need to be careful. The first thing to do is to run few spikes and test technology, check what other people say about these products by reading blogs and forums and go over your maintenance contracts with 3rd parties. Even when you believe that you did everything necessary, be ready for surprises. And if happens that you need yet another upgrade, or rollback to previous platform, you are in trouble. Therefore, you as the project manager, might be called into endless meetings, where people want to see actions, while the only thing you can ask for is patience.
  3. Nasty bugs, I mean really nasty bugs There are only two things certain in your life. You will die one day, and your application will have bugs. But here I am not talking about any bugs, here I am talking about concurrency bugs, almost random bugs, the one that come and go, the ones that leave complete team in disbelief, the ones that challenge your atheism and make you read stack traces on your blackberry. Therefore, you as the project manager, might be called into endless meetings, where people want to see actions, while the only thing you can ask for is patience.

Chicken Run

If you don’t stick to this message, and let other people manage your team and project, every day there will be more crazy stuff, till everyone runs around like a chicken. Chicken run trajectory is the one in which chicken in any given time slot completes always the same distance and always in random direction (unlike frog from the previous post). Therefore, the only effect of doing ’emergency actions’, escalations and quick fixes results in more troubles. In my next post, I will talk about things you need to do when you are hit by the grey swan – how to provide a proper feedback to the management, how to deal with your team, and most importantly how to keep yourself sane and healthy.

Leave a Reply