The time has come – to destroy agile

bomb

I still remember my first “talk” on agile – it was inspiration for this site in the first place. The first thing that I said at my “course” was: “I have come here to destroy waterfall”. But, I also said, “Don’t trust me a word. The time will come, when what I say today will not be true any more.”

You see, the problem with methodologies and ideologies is that you sell them differently than medicines or food. When you buy milk in the shop, the first thing you check is the expiration date. There is no such a thing when you sell, or buy a methodology. And there is a time, when people who come after us have to say enough is enough, and destroy the bullshit.

bestBefore

 

The beginning of the end came as soon as agile have become an industry, as soon as agile courses, gurus and Dalai Lamas started pitching and preaching agile. People who never wrote single line a code, who never cursed others for breaking the build, or who never had to wake up in the middle of the night because something got broken. Soon, agile was to be fucked up. All things I wrote on this blog was my silly attempt to correct that, and I hope you had enjoyed it, but now the time has come to destroy it.

My trigger was this post today:

shitAgile

First thing I wanted to do was to write a comment to the guy saying: “man, if  the only problem you have is one QA guy missing, fuck yourself and start testing.” Then, the things became more intriguing. Suddenly the problem was in semantics, that people actually did kanban, while they were saying they were doing scrum, so, could be that the solution was just to name the whole mess kanban? Finally, after reading the post completely, I realised that nobody actually gives a shit about what they are doing, so they just hired one agile scrum moron to go around and whine??

Agile has become a joke, and that hurts me, as I worked so much in the past to make it happen.

I say, fuck it and destroy it. People who wrote agile manifesto long way back deserved better way to be remembered.

P.S.

The day after writing this post I got a tweet from @somatik : @pizuricv you’re not alone 🙂 https://vimeo.com/110554082  , by Erik Meijer.

Oh my god, you must watch it! It is exactly to the point. But please don’t stop watching after the first rant, you must watch it till the end! Promise me, please. Erik is saying the same thing, but he also is talking about other, very important aspects of the software development and management.

  • need for coders to code
  • need for hierarchy
  • need for leadership, but distributed
  • need for different opinions
  • need for feedback loop – but with people who have conflicting arguments

In a way, all things I was writing about in this blog. Amazing.

Propecia is probably the generic viagra which was approved by Food & Drug Administration.The viagra penetrates into the blood and acts only in the vessels of the penis.
Posted in Agile|Agile | Comments Off on The time has come – to destroy agile

Leadership

Leadership is not given, it is taken. Leadership is not management. Leaders are born. With managers, it is a little bit different. Some managers are born, some are trained. Every person can be a good project manager if he can manage a big shopping on Saturday: buying all goods in one go, covering minimal distance and picking up only the stuff which was written on the paper. Testing for leadership is different. Whether someone is leader or not you can only feel – the answer is based on the instinct alone. It is the same in the animal world, and in that respect, leadership is a million years old job description.

What kind of leadership we know about? According to one Chinese philosopher, there are 4 types of leaders: Ones you don’t notice, next to them are the ones you admire, the ones you fear, and finally leaders you hate. Make no mistake, all four types of leadership work. Motivation to avoid pain is many times higher than motivation to seek pleasure. If you are a leader – you are one of these 4 types. Which one is your choice, not others people choice. Like Seneca said once “You are your choices”.
Does any place need a leader? Yes, but that doesn’t mean it has one. In most cases, the office place has a manager, but as I said, the same person is not necessarily a leader. It might be that a given place has both, and it can be that these roles are not owned by the same person. Something else is also possible, that the place has more than one leader, even though with some of the 4 leadership archetypes is harder to imagine that.
Does every place pretend it has one? Sure it does, as every place needs a leader and people naturally seek to have one. Does your place have one? Well, as I said, you know it already.
So, what it has to do with agile development you may ask? Nothing really. Just it helps to have one – if you are into agile development.

Posted in Agile|Agile, blog, project management | Leave a comment

Are you really happy at work?

Jim Collins, in the book Good to Great suggests the following exercise:

“Suppose you woke up tomorrow and received two phone calls. The first phone call tells you that you have inherited $20 million, no strings attached. The second tells you that you have an incurable and terminal disease, and you have no more than 10 years to live. What would you do differently, and, in particular, what would you stop doing?”

Interesting read, it makes me think… I always wanted to be a football player, but it is too late now….

If you are an engineer, and you feel that this is exactly what you were born for, let me add one more test to the list. You receive a third call asking you to pick up the money only if you take a flight in the plane that was designed, implemented and tested by you. Would you still pick up the money?

Making stuff is not just a fun; it is not only about feeling cool, or geeky. It is about being proud of the thing you have created. Do you remember the first drawings you showed to your parents? How proud you felt while they were smiling looking into your creation? In the end, we like to create things that other people can use and appreciate. If you have lost that drive, or you never had it, think one more time about the stuff you do.

Don’t worry be happy? No, worry and be happy. Care about things you do.

With this post, I am closing the chapter on agile development, and starting a new serial. In the next set of blogs, I will be talking about innovation, research and development, chaos theory and s-curves, startups and big corporations, operations and support – all things I’ve learned – hard way. Stay tuned.

Posted in blog, Uncategorized|Ostalo | Leave a comment

“It all depends“ my friend

There are two kinds of people, people that believe that everything can be divided into two categories and the other ones – which make it hard for you not to belong in one of two categories, even if you don’t believe in the two categories theory.

There is a religious theory about good and evil. And there is a theory which says that people are either religious or not, or whether they are destined to be religious or not. And there is a theory that we are all born and pre-wired to be conservatives or liberals – left or right guys.
And there is a left and right brain theory. Left part used for analysis, the right for the synthesis. Some people are more of the left brainers; some people are the right brainers.

And there is a theory about waterfall and one about agile. About writing tests before coding or doing it after. About OO and about functional programing, having mutable things or not, threads or no threads, strong types or not, SYNC or ASYNC.

And how about people in the middle? These are, according to the two categories theory, folks that can be split in either:

  • people that are wise, who have tried both and got to a conclusion that nothing is black or white, OR
  • people who have tried nothing in their life, and they just mingle around, pretending they are smart (or wise), and do “Dalai Lama like” talks, without having a clue what are they talking about.

Whenever you are new into something, and ask other people for advice, the best advice you can get is: “It all depends my friend”. And, indeed, if you are sure that it came from the person that has tried some of the options that your answer depends on – that’s the best answer you can get. Otherwise, he will be just wasting your time.
It is one of these two my friend, the same question, the same answer, two options.

Posted in Uncategorized|Ostalo | Comments Off on “It all depends“ my friend

Agile Development or how many balls you think you can juggle?

Most of the time, when people talk about project management, they only consider a phase between the official start of the project M1, and beta release date M2, as a “real work”. As explained earlier, this is a wrong approach, and you should avoid skipping very important preparation/workshop phase, which although sounds like a waste of time, is extremely important step in overall process.

We often associate this time frame with “busyness syndrome”, stress, heavy workload and stuff like that. In this blog post, I will try to make an overview of tasks that people should be doing in this time slot, and explain one way of organizing teams around various parallel activities.

But before I move on, I would like you to imagine the following:
You are the captian of the ship, and you need to get your guys from point A to point B in couple of months. On this journey, there will be some storms, troubles with a ship, and sometimes, issues with a crew. Before you start the journey, make sure that every crew member is good at something – and willing, otherwise, should not be on the ship in the first place. You need to explain the purpose and destination of the trip, and not to forget, what is the role and expected contribution of every person. During the journey, you will have the chance to adjust your direction at fixed time intervals. The actual length of one time interval will depend on the inertia of the system – the ship itself. Bigger the ship, longer the intervals. The end of a time interval (iteration or sprint) – is your correction point. At that moment, you position the ship for the next round, and till the next point arrives, the ship must go in the same direction. Once you reach the end of the interval, you adjust the path, and move on to the next point. Within one interval, nothing can disturb the crew – that is your task too. If you pictured this story right, this will be kind of zig-zag path journey, going from A to B. (If you are a guy that hates blog posts that are longer than few lines, you can as well stop here, that is all you need to know 🙂 )

And here we go, well almost….few more checks first…

  • First do your journey preparation as described before
  • Choose your iteration span – (you start iterating from M0, but you may as well decide later to have a different span between M0-M1 and the rest of the project lifecycle- although this is not a good practise). I personally prefer short iterations, fast feedback is always good (imagine taking shower and having to adjust hot and cold water – while the time between your action and the tap actually responding is 4-5 seconds – this is always fun, I had it few times). If you think you need long iterations (longer than 4weeks), you either manage a space shuttle program, or your processes are really lousy.
  • Decide on the branching strategy (see my post on the maintenance), and check this site  (thanks Joachim @joachimvda for the link) and awsome CRISP folks for one more great article.
  • Find a good build manager, setup jenkins, bamboo, git etc …
  • Find a good Test Scrum Master (this is something new, no?)
  • And fasten your seat-belt…..hmmmmm, i just realised I am on the ship actually, anyway …GO!!!

Juggling intro (now I am almost there…)
In all scrum books you can find something that is called “definition of done”. Idea behind this is that you don’t close the iteration till everything is developed and TESTED wiithin the same iteration. Some years back my test manager and I have decided to tweak this a little bit, by developing things in one iteration, and testing the same load in the next iteration, while dev guys would already work on something new at that time. If you have dev/test teams that are not on the same location – not even on the same continent (like it was in my case), this approach seemed more optimal to us. I also must say that this only works if your iteration are short. BTW, if you are familiar with kanban, this is exactly what I think these guys tried to fix as well (more about this in another blog post). One not that pleasant artifact of this choice is that the time between the code freeze and the time that the beta load is fully tested is also delayed by one/two iterations.

Finally, let’s do some Juggling!!!

Correction point C0: scrum kick off meeting (discussing/work split for iter1)

Iteration 1 (1 ball):

  • RD guys develop new stuff in ITER_1

Correction point C1: scrum kick off meeting (demo’s iter1, work split iter2), the current load is labelled (LAB_1) and handed over to the test team

Iteration 2 (3 balls):

  • RD guys develop new stuff in (ITER_2)
  • TEST guys test load (LOAD_1)
  • TEST guys fire bugs against LOAD_1

Correction point C2: scrum kick off meeting (demo’s iter2, work split iter3, assign issues of iter1), the current load is labelled (LAB_2) and handed over to the Test team

Iteration 3 (4 balls):

  • RD guys develop new stuff in (ITER_3)
  • RD guys fix bugs from (LOAD_1)
  • TEST guys test load (LOAD_2)
  • TEST guys fire bugs against LOAD_2

Correction point C3: scrum kick off meeting (demo’s iter3, work split iter4, assign issues of iter1,2), the current load is labelled (LAB_3) and handed over to the Test team

Iteration 4 (5 balls):

  • RD guys develop new stuff in (ITER_4)
  • RD guys fix bugs from (LOAD_1, LOAD_2)
  • TEST guys test load (LOAD_3)
  • TEST guys fire bugs against LOAD_3
  • TEST guys validate tickets from (LOAD_1)

Correction point C4: scrum kick off meeting (demo’s iter4, work split iter5, assign issues of iter1,2,3), the current load is labelled (LAB_4) and handed over to the Test team

Iteration 5:

  • RD guys develop new stuff in (ITER_5)
  • RD guys fix bugs from (LOAD_1, LOAD_2, LOAD_3)
  • TEST guys test load (LOAD_4)
  • TEST guys fire bugs against LOAD_4
  • TEST guys validate tickets from (LOAD_1, LOAD_2)

Correction point C5: scrum kick off meeting (demo’s iter5, work split iter6, assign issues of iter1,2,3,4), the current load is labelled (LAB_5) and handed over to the Test team

Iteration 6,7,8 ….

For simplifications, I have omitted few more tasks , such as:

  • Bug fixing of the issues of the previous releases (if you have them already out)
  • Validation of these issues in the next maintenance load that goes to the customer
  • Automation work (by both test and dev team) on simulators, test environment etc.
  • Test document update (you need to keep use cases and test execution in sync all the time)
  • Support for the field issues (L3/L4)
  • Sales support …

This would be already 9-10 balls in the air after iteration4! Once you do so many activities in parallel, the best thing you can do is to split test and dev teams (yeah, I know what you will say about cross-functional teams, but this is not the same thing) and let the test manager do the same thing that dev manger does – organize test teams in separate sprints and let him become a “Test Scrum Master”, it sounds really cool!

So, let’s put all things together and write it this way:

COUT(“ARE YOU READY?”);
FOR(I=1;I<NUM_ITER+1;I++){
ALL(KICKOFF_I);
DEV(ITER_I);
DEV(FIX(FIELD ISSUES);
TEST(VALIDATE(FIXED(FIELD ISSUES)));
TEST(UPDATE TEST SPECS);
ALL(WORK_ON_AUTOMATION);
if(I>1){
TEST(LOAD_I-1);
TEST(FIREBUGS(LOAD_I-1));
if(I>2){
DEV(FIXLOAD(2..I-2));
}
if(I>3){
TEST(VALIDATION(LOAD(I-2)));
}
}
}
COUT(“THANKS FOR READING!!”);

In my next blog post, I will mockup one JIRA project and explain the same topic via dashboard graphs and greenhopper screenshots. 

 

Posted in Uncategorized|Ostalo | Leave a comment

Agile Project Management – the project follow up

XP was a first development framework in which the customer feedback was embedded in the complete project lifecycle (see Beck Kent’s books, Ron Jeffries blog and articles, or check this list). In the picture below,  you can find “one slide explains it all” that I often used back in 2002-3, when people were asking me about “extreme stuff” we were doing:

(Side note, I can’t remember any more where I found this drawing)

User story: “As a <role>, I want <goal/desire> so that <benefit>”

Early on, I have noticed few interesting things about user stories:

  • First, it is an excellent way of introducing customer into the picture, similar to the “elevator test” idea – and I liked that.
  • It is excellent way of preparing demos after every iteration (and fast feedback which comes as the result of it)
  • It is the best way to do acceptance testing (check this guy out and his blogs/books on this link)
  • It is not always that great as the tool for project/release tracking (one interesting observation, in XP days, we used to talk about  “tracker” role for someone that was responsible for story/planning tracking – later replaced by my favorite term “scrum master” )
  • If the feature is rather complex, simple story splitting is not the best way to capture the work that needs to be accomplished.

While working on my first XP projects, I was a fanatic about data gathering, everything you can think of. We estimated tasks in days, hours, even minutes. We were all booking 8 hours a day – no exceptions. We would book coffee time, meetings, coding, demo’s… I kept doing this for years. No matter what we were doing, who was doing it, I always found similar observations:

  • 30% of all time in project is “wasted” on meetings, coffee breaks and kick-off meetings. (one more reason why I got so “freaky” about making agile meetings more efficient)
  • Almost all stories of the small size were implemented faster or on time.
  • Most of the big stories (or epics as they like to call them these days) and some small features were experiencing a strange distribution, and this is a “funny part”, the range varied from 20-30% to about 3-4 times of the original effort (in both direction, although majority of estimations proven to be too optimistic). It didn’t necessarily correlate to any point in time of the project, or how good the original estimation/analysis was done. Just one more thing for the record: I’ve had a pleasure working with some of the most experienced developers you can imagine, which makes these graphs even more interesting.
Here is a an example of two releases:

That was at the time I didn’t bother reading about “Cone of Uncertainty” or “Iron Triangle”. I was more like a chef in the kitchen, tweaking ideas rather than estimations, trying to figure out what’s the best way forward…

From then onwards, any time I see a project where the burn down chart behaves nicely, I get suspicious.  It tells me one of the following: either the estimation of all work was done with great accuracy and all developments is happening smoothly (with no grey swans around) OR someone is too busy making burn down charts look nice.

While I was occupied with my projects: planning, tracking, coding, support, bug fixing and studding the data that was in my hands, I must admit, I completely missed out on scrum movement, which was to become a dominant agile force… What surprised me at first was how much some of my conclusions differed from slides you can find in any scrum training.

One thing I realized, and that is what I have written in one of the previous blogs, is that trying to be too much smart upfront really doesn’t work – even if you have your stories. You need to code, as soon as you can, but not before that (you may hear me now talking about “invisible lines”, this is the moment when other people’s eyes start rolling when I talk like that). I also found another interesting thing, burn down charts don’t work for me, but burn up charts do the magic! Let me show you two graphs, and some important explanations that will follow:

(Graphs are created using JIRA dashboard feature – these two graphs are must for all my projects)

  • Developers must have subtasks for everyhing they do, and they are free to add more tasks any time they find it necessary. Every subtask is assigned at kick off.
  • At the beginning, all FTPET’s are estimated in either days or weeks
  • If FTPET is deemed to be complex, we address it early on (sometimes even before M0). During this process we do light analysis which results in set of subtasks. These subtasks are than estimated separately.
  • About 15% of effort is kept for refactoring. This is necessary to keep the code clean and robust in long term. Oh yes, I hear you saying: Refactoring should happen always and it should not be given a “task”. Well, I am saying the same, with one exception – I know that people will not estimate this effort when they estimate their work. I don’t really know why, but that’s the way it is. So I book it anyway (some nasty readers might call it my secret buffer – at least it is not that secret any more 🙂 )
  • All technical challenges are addressed in separate “spikes” and most of the time are not part of any release planning
  • Estimations are only about the work, it doesn’t include any overhead (meetings, maintenance, holiday, coffee time …)
  • Since R&D logs every day 8 hours, information about overhead and other non-dev tasks can be taken into account in the future planning (about 30% of R&D effort goes to overhead).
  • The team is closing subtasks with steady pace – follow the green line (interesting note: subtasks are not all of the same size, but why they look as they are? This is XP trick, every item should be finished within one or two iterations, not more than that).  Since my iterations are normally 1 week, it turns out that “story point” size in this graph on average is about 3-4 days! Multiply this by number of dev guys and …Hey, you have your velocity?!! Sounds like scrum, no? 
  • Some FTPTET’s and other big features are divided into subtask only later in the project (we have a “good idea” how much they would take, no time to split them all – Pareto rule). That you can see by following the red line.
  • Number of subtasks doubled between M1 and GA, but all FTPET’s were closed on time .
  • Most of the FTPTET’s were known to the R&D team 3-4 months prior to M0
  • Any time you add something, you make sure nothing else it broken (automation, TDD, whatever). That makes you adjust better on the major question- are we hitting the date, or the feature content?

So, what you do with these two graphs and how I do project follow up in 5 minutes every day? Is it rather simple:

You monitor the list of new subtasks that comes after M1 – this is the red line (at M0 you should have about 30%-50% of subtasks identified – check the previous post). The curve should start slowing down after a while. If it doesn’t, you are in trouble, good news – you know it.

By looking into green line, you only see that people are working, and you can make “projection” in your head when it should meet the red line – but again, this is not exactly as working with burn down chart. Burn down chart makes managers freaking, and instead of being focused on the work, they spent big effort trying to make them look nice. More over, it makes them feel like they need a better estimation process – next time, the trap I talked about in this blog Estimation Fallacy and Estimation (only) Driven Project Management. With the burn down chart, ideally, the red line stays constant – since it expresses the cumulative effort of that release. In my case, it only represents new subtasks that are created in time. I can stay focused on adding value, I am flexible to add and remove things as long as we are all on the same page (PO, me and the team) during the project lifecycle, and it is brutally transparent – to everyone.

Good, this was a long blog post, and by now you should know how you can do the project follow up in 5 minutes. But be cautious, the project management is not about task follow up ONLY! In my next blog, which I might call “How many balls you think you can juggle?” I will be talking about some other aspects of the project management during the development phase. Stay tuned.

 

Posted in Agile|Agile, project management | Tagged , | Leave a comment

Agile Release Planning – the preparation phase

Before I start, I want you to read the following statement:

“The most important role of PO, project manager and the team is to create a product that someone wants to buy, from which company can earn some money, and in return pay you something back. In process of creation, you need to respect the law and make sure that people are treated with respect and dignity. Anything else is a second priority”.

Some time back, I spoke about “Iron Triangle” and how critical is to know upfront which project lever (functionality, date or release) are at your disposal for adjustment as you start with the planning. Failure to ask this to stakeholders is a sign of immaturity or lack of confidence, or both.

Once that is clear, you look into the feature list which is presented by PO at the kick start of the project. That list can’t be longer than 10-15 items. This list must survive “so-called” elevator test (from Geoffrey Moore’s “classic” Crossing the Chasm), and it goes like this:

  • For <target customer>
  • Who <statement of the need or opportunity>
  • The <product name> is a <product category>
  • That <key benefit, compelling reason to buy>
  • Unlike <primary competitive alternative>
  • Our product <statement of primary differentiation>

These items are not “epics”, these are your FTPET’s (I just made that name now, features that passed elevator test). For the stakeholders and the outside community, these are only sellable items, and you need to track them separately throughout the project lifecycle, next to all other working items (stories, epics, tasks, subtasks, whatever – these are too technical terms for the rest of the folks).

In the first phase of the project (between M1 and M2 from the project milestones blog), you need to organize a workshop. For more info, I recommend a book “Practices for Scaling Lean & Agile Development“ by Craig Larman and Bas Vodde.  Short version: go for a separate location with key guys, make sure you have enough white boards around (and camera to capture drawings and discussions) and do the following (extract from the book):

  • Envision business case, strategy, high-level list of features (major scope), and constraints for the release.
  • Identify ten or twenty percent of the features that deserve deep analysis immediately.
  • Choose a subset that will yield broad and deep information about the overall release work. There are always some key features that, if deeply analyzed, give you overarching information about the big picture.
  • These may be the most complex, the most architecturally influential, the highest-priority features, or features with the most obscurity. From the viewpoint of information theory, they represent a subset that, if analyzed, is most likely to have lots of surprising information—the most valuable kind (end of ref)

Once the workshop is over, create a small sketch plan on A4 paper. If you need A3, you are already in trouble. Divide it into 4 quarters (Q1 to Q4) and draw the activity diagram, similar to story mapping, but taking into account only major items, including FTPETs and most important non-feature tasks (what ever that is).  Small tip: for god’s sake don’t use week numbers instead of quarters, most of the people outside

of project circles can’t actually tell you in which w## are they. It would also give a wrong impression that your precision will be in weeks – and it is not. This paper will be your basis for a discussion with R&D folks, key architects with whom are you going to make a more detailed plan. Right now, you need this paper as a tool to check the dependencies, and get a first feedback from R&D. After few sanity checks on the white board, you will know whether this plan makes sense, nothing more. If people can’t tell you this at least, you either have not done a great workshop, or you are surrounded by wrong people (next tip in later case: if you can’t change the team around you, change the team around you).

Depending on the team size, and whether the team is collocated or not, you will need to split development to one or more feature teams. Each feature team must have a feature owner, call it a scrum master, what ever. He should be leading feature discussions with PO (to come up with more detail user stories), lead technical discussions with R&D folks, and make sure that all ideas are captured in the wiki, with drawings, mockups and design choices. Here is one example of feature mockup in the wiki:

Once the key items passed this phase, you need to ask scrum masters and devs to add all KEY user stories, FTPETs and major tasks in the system (in our case JIRA). By then, about 30%- 50% of all features are well understood. Reminder: after workshop you tackled 20% of the features, few weeks later you are at 20% of the time span of the release. This is your milestone M2. 20% of time required for analysis is rule of thumb, BUT, you should not cross it too much.  If the time is the most important dimension, everything that has not been discussed at this stage and it still looks complex to solve, must be out of that release (or at least be given very low probability of success in that release). Here is what you can do next, create a feature list with the following items:

  • Key (JIRA or something similar)
  • Summary
  • Probability
  • Rationale (Grow, Harden, Customer etc)
  • Priority (P1, P2 or P3)
  • Comment
  • Size (relative) in story points

In my previous blog, I talked about story points estimation. Now you can see how I use them (R&D still can use them for other things). The size of the feature  is a story point given as a relative number. I can sum all stories together and provide a quick overview to PO on where the majority of the effort of this release will be spent (like in the picture above, which is made by one simple wiki macro). It also provides me with a tool to give a honest/professional feedback to the stakeholders about probability of the success of any given feature. Please also note that for the test and doc effort estimation I use T-Shirt estimation sizes (S,M,L,XL).

Rationale category is important information for PO and stakeholders, as based on this info we can all see in one pie chart whether we are making this release as the result of the push effect (bring a new functionality that is unique to the product), the pull effect (customers ask for new features) or we are busy with tasks such as improved scalability, some major redesign, or refactoring (resulted from “Demo Driven Development” syndrome).

Once you have your skeleton plan and 30%-50% of all features well understood (including the most complex in this bucket), you are ready to go – you have your milestone M2. Now the fun starts! Stay tuned for the next blog, it’s coming….

 

P.S. Comment  on the first paragraph:

When I became a manager some years back, I was told that I will be measured by “Whether the project is delivered on time, with agreed budget and all features implemented”. I believe now, as I did at that time, that this is the worst possible message (and metric) you can ever give to a young manager.

 

Posted in Agile|Agile, project management | Tagged , | Leave a comment

Effort Estimations Fallacy or Effort (only) Driven Project Planning

Countless books, blogs and studies have been written about the effort estimation process. Unless we talk about repetitive, so called “blue collar” jobs, the estimation of the time required to complete any complex activity has proven something hard to do right (these are my favorite studies: “Research suggests people underestimate numerical guesses when leaning left“, “Impact of irrelevant and misleading information on the effort estimate.”). My take on this is that the problem is not easy to solve as our work includes solving non-linear problems (or that the estimation of the effort to solve a problem is non-linear problem to solve). Before joining the big company, I had fun dealing with some of these problems in another field, and to my mind, from early days of working as a developer in a dev team, everything that had to do with the planning and the effort estimation was similar to the simulating annealing process, where the convergence to the global maximum requires sometimes taking random (not to say wrong) steps. In my previous blog, I talked about the “Cone of Uncertainty” that should shrink over time, which in a way is similar to the idea of the cooling in the simulating annealing algorithm.

I went to this length of explaining my “academical view” on this, as I would like you to remember two things: First, if you ever find yourself reading or trying to find a magic formula for effort estimation – you are wasting your time. Techniques, such as “functional points analysis”, have been invented by folks that never wrote single line of code. Secondly, the fact that the estimation is not a simple thing to do, doesn’t mean you should not do it, I am only saying, what ever you do, be careful about the data you get out of this process.

Many agile gurus, like Craig Larman, rather talk about a need for trust and transparency between stake holders and R&D, in order to make a project planning process more transparent and more engaging. More over, as the burden of success or failure of the project are often solely responsibility of the project manager after initial roadmap phase (which is wrong in many ways), discussions about the effort estimation becomes the central point of the project planning – generating mistrust between R&D and the rest of community at early stage of the project. WRONG!!

First of all, like mentioned  in my previous blog, project roadmap planning must be first about the product strategy and activities which are required to make the release success. Once that is clear, and first few iterations are completed, people will have better idea what is probably achievable in the current release. Before that phase is completed, any estimation or the feedback from R&D must be taken with a caution. At very beginning of the project, the focus of the  team, including R&D and PO, should not be on the effort estimation, but rather on understanding of the requirements, complexity of the problem and different possible paths that can lead to solution. Once this is clear, we are ready for the planning phase.

But before I explain how I do it (in the end, this is a blog about agileME), I still feel like introducing a little bit of methodology, sorry for that. When it comes to getting a number from the estimation process, they are today two main schools of thoughts: you either do the estimation in time, or in story points.

Some time back, Mike Cohn came up with a story point planning strategy: check out book or video link from google talk, or slides. In short, first you go with iterative estimation strategy, called a poker game. Here are the steps:

  • Each estimator is given a deck of cards, each card has a valid estimate written on it (Fibonacci series)
  • Customer/Product owner reads a story and it’s discussed briefly
  • Each estimator selects a card that’s his or her estimate
  • Cards are turned over so all can see them
  • Discuss differences (especially outliers)
  • Re-estimate until estimates converge

Finally fill the backlog with tasks, subtasks and story points. Once you have done so, as Mike says, you continue like this:

  • Determine how many iterations you have
  • Estimate velocity as a range (number of story points team can finish in one iteration)
  • Multiply low velocity × number of iterations  – These are “Will Have” items
  • Multiply high velocity × number of iterations and substruct from this the number of “Will Have” items in story points  – These are “Might Have items”

Picture taken from Mike Cohn’s presentation on the iteration planning

It sounds simple, but there are few “gotchas” that have hit every single scrum team I have seen so far:

  • it makes people focused too much on poker games too early in the project (you miss a big picture).
  • it feels like project management is all about story point estimation and simple counting (to be frank, few agile folks warn about  this trap, but still it happens way too often)
  • if you make your planning only based on the total number of story points, then you need to estimate every single little thing in story points. I hate this, and this is in contrary to Pareto rule from the previous blog. More over, you may wonder whether to estimate “framework things”. Then you have academic discussions on whether to estimate refactoring, etc. endless. Before you know it, people would spend weeks doing estimation without doing any real work, contrary to everything agile should bring. If you have inexperienced scrum master that needs to provide a roadmap planning, you will be doing poker games like crazy and end up with 800 stories in your backlog before you even start doing anything (flat backlog blackout syndrome)
  • people don’t learn from their own estimations. As mentioned earlier, estimations are always wrong, but if you do it in time, at least after a while you know better whether you are a pessimist or an optimist.
  • you can’t make that much out of the story points if the team is new in this
  • we are told that story points are about the complexity number of the feature, BUT everyone in his mind has the time dimension anyway (strange how few people are willing to admit this, don’t know why). For a fun, check out this TED video “Why we think it’s OK to cheat and steal (sometimes)” from Dan Ariely, and pay attention on the part of the experiment which includes tokens as a intermediate currency before being exchange for a real money.
  • if someone tells me that he will need 5 points to make something, I have no clue when this stuff will be done, more over, the guy doesn’t feel like making any commitment to the rest of the team. (You may wonder what this has to do with my previous statement of avoiding getting in situation “commitment from R&D only” to the stakeholders). This is different, it is about a guy having the best guess how much is going to take him to finish a task within one iteration. If there is no time information present, you never know whether the guy is in trouble during the development, whether he knows what he is talking about, and if you like, it is hard to make a peer pressure working (I am a hard core XP guy on this, to be frank).

 

Now, you may wonder why project managers like story points planning. I think for the same reason they liked waterfall, it gives them a fake feeling of having things under control once you have your numbers. Like by magic, story points remove all uncertainty of making the project a success.

If you by now think that I am not using story points at all, you would be wrong :). I do use them, but in a different way. First of all, story points discussion is very good way of discussing complexities – indeed. It is engaging. It also provides people with a “token platform” to discuss different aspects of the problem, and that level of indirection at that point of time is a good thing. The only real question is what you do with these discussions and the numbers that come out of them? Here is a little tip, while listening on discussions, I am interested in two things:

  1. How much opinions differ
  2. How complex the feature is

Complexity number gives you an idea about the nature of the probability distribution of the estimation for a given feature, while different opinions are giving you the range (or sigma) of the “Bell Curve” – if and only if, the complexity seems to be low (in that case we can talk about the Bell curve).

If you are faced with a project that is not evolving that much, and I guess this is the place where most of the “story points guys” are writing their studies and books, you are faced with rather predictable environment, and story point only project planning might work well. BUT, if you are faced with projects in which features exhibit high level of skewness, you better do things differently. One thing that strikes me the most about story point articles is how they tend to plot the distribution of story points per iteration (like in picture above) – and come with a bell curve always, while I would be more interested to plot estimated vs. actual points required to finish any given story. Well, the point is, you can’t, as this is never captured. Again, you may think, why the hell would someone need that, since you come up with the global distribution that is bell curved anyway? Again, that is only true, if all the features are of the same distribution, and that is not the case in more complex projects! Just ask yourself, how many times you have actually seen burn down  chart working? Really?

By now, you probably know why most of the project managers are either depressed, lunatics or alcoholics, as they need to deal with quite some uncertainty every single day, and provide always accurate planing. But there is a way out of this, more about it in my next blog post.

Posted in Agile|Agile, project management | Tagged , | Leave a comment

Agile Planning

In my previous post I introduced some basic principles of project management, so if you skipped that part, please read it first 🙂 ). Here are my rules of Agile planning:

  • Rule1: Ask PO to present the vision of the new release to the complete team. This is milestone M1. Everyone in the team deserves to know what is the ultimate goal of that release.
  • Rule2: Define “rules of the engagement” for the complete team, including PO. That includes iteration span (heartbeat of the project), kick off dates (day in a week and the exact hour), standup meetings, scrum of the scrum meetings, format of the kick offs and demos etc (link to my previous post on agile meetings). This needs to be presented by project manager at very first iteration to the complete team. There might be some suggestions and arguments – that is good as we need to work as a team, but once that is all cleared – no more discussion on that. Important part of a project manager role is to show consistency by example, and insist other people follow the same.  I am true believer in self organized teams as long as the rules of game are known to everyone.
  • Rule 3: Don’t wait longer that 20-30% of the full release cycle on scoping and effort estimation. More than this is a waste – less than this is a chaos.  The most important part of the work within first few weeks is to run workshops and understand underlying complexity. Use whiteboards and avoid any UML tools or ppt slides. This is also the time you can tests new frameworks. Do iterations and coding(spikes) from day1, more you wait, longer the uncertainty (“Cone of Uncertainty” is not shrinking).
  • Rule4: Once you reached high level of understanding of the project functionality, create the project plan (something like a skeleton planning discussed before). Skeleton plan should be a joined effort and known to everyone. Once that is ready, you have your next milestone – M2
  • Rule 5: After every iteration, re-adjust one of the parameters of “The Iron Triangle”: your release valves – functionality vs. effort.  This is done after every iteration (between PO, scrum master and some key developers). During the project lifecycle these two items will become more and more accurate.

Presentation of the project organization to the complete team – real example

Once all features are implemented – you have your “beta” release. How long it takes you from “beta” till GA date, depends on various things. Typically, agile folks hate to talk about the “hardening” of beta releases and preach GA quality at every iteration. In on of my future blog posts, I will spend more time on this topic.

Project milestones

What does this mean, and what not?

  • In Agile, you still need a good overall planning, and you need to have a product backlog
  • Not ALL functionality needs to be known upfront, but R&D has to have a good understanding of all use cases. Agile saves time by having detailed discussion as part of iteration planning, and you are making discussions more efficient, simply because you have accumulated knowledge (through the previous iterations) which you did not have at M1
  • PO can ask for a feature to be extended/changed during a project – but he always needs to take into account  “Iron Triangle Principle” (hence PO/R&D collaboration is important throughout the project lifecycle)
  • Later requirements can be added to the release, but be careful – closer to “beta”  date, bigger the risk of adding new things.
  • PO defines the ‘What’, The Scrum Team define the ‘How’ and they work together to agree on ‘When’.
  • Testers can’t know upfront all detail requirements, but should know all use cases. They need to be on board from day1.
Why and when is agile so successful? – my philosophical view
  • Any methodology that invites people to collaborate and motivates people to excel must work.
  • Frequent inspection and adaptation is the most effective concept in any complex and high risk activity
  • R&D and PO in agile context will eventually collaborate, because they are forced to play TIT for TAT with fast feedback cycle. Another phenomena – longer they work together, more they are willing to collaborate. This is well known fact from gaming theory.
  • R&D folks gets motivated by feeling truly in charge of things they know the best
  • “Control” is achieved by clear accountability for everyone’s work
  • Openness allows constructive peer pressure and unbiased norm for the complete team
Posted in Agile|Agile, product management, project management | Tagged , , | 2 Comments

Project Management Basics

In this and my next blog post I will be talking  about rules of agile planning, based on my experience in agile development and project management. But before I get there, I will need to introduce some fundamental concepts of project management first:

  • “Iron Triangle” (a model of the constraints : time, money, scope)
  • “Cone of Uncertainty” (accuracy of effort estimation vs. time)

At very begging of a release, it is  important to decide which project lever — time, cost, OR functionality—cannot be compromised to launch a successful product.  These three levers are also referred to as the “Iron Triangle.” Fixing time, budget, and functionality is not possible; at least one has to act as a release valve. Decision which dimension is less critical is not the choice of a project manager, but failing to ask PO and stakeholders before the start of the project this question is either sign of lack of maturity or courage. As a project manager, you must have both.


“Iron triangle” and “Cone of Uncertainty” 


Some project management cynics would point out that there is always one dimension more you can “play with” if you get in trouble – and that is quality. Therefore, the best way to evaluate teams and project managers is by checking the quality of the consecutive releases of the same project. (that is also the reason why one-ff CDE projects lack trust in relations –  such projects are always full of SLAs and metrics).

Even with a shared product vision in place, the product’s exact functionality is not always well known up front due to:

  • Technical challenges
  • Customer feedback (which might impact design choices or add extra functionality)
  • Sometimes we need to dig into the surface to learn the true nature of things
  • Sudden change in “market requirements” – which often means new development in the current release (that largely depends on the release lifecycle)

Three main project management practices differ only in how they see a change of the functionality during the project lifecycle.

  • The waterfall model (“mechanical view”, no changes allowed)
  • Spiral development (can cope with change, via prototypes and risk analysis)
  • Agile development (“designed” to allow changes, higher value on adaptability than on predictability)

In agile, it is recommended to fix the time and “flex” the functionality.
Identifying the launch date is facilitated by the product vision – a roadmap, which is something that PO needs to bring to the table. It sketches the future product and describes its target customers. Vision allows us to determine the window of opportunity, the time frame in which the product must be launched to achieve the desired benefits.

Note that choosing the launch date based on the work in the product backlog is difficult, as it forces the team to freeze requirements at an early stage. If that is done too early, it  results in extremely poor estimates. The accuracy of the software estimate depends on the level of refinement of the software’s definition. The more refined the definition over time, the more accurate the estimate – which is a definition of the “Cone of Uncertainty”.

“Walking on water and developing software from a specification are easy if both are frozen.” Edward V Berard

This is exactly where the waterfall guys got this problem wrong, as they have tried to reduce the uncertainty by insisting on detailed requirements too early in process – which resulted in lack of willingness to adopt to later changes, poor estimates and huge waste at the begging of the projects. One more problem which came with waterfall, which I particular dislike was covered in one of my previous posts.

Fixing functionality is a good strategy only when the product lacks some of the key features that competitors already have or if you want to bring something new and really cool on the market.

For instance, the best way to describe two different release planning strategies is to imagine yourself trying to bring a new smartphone on the market:
Either you have to have the phone in the shops before Christmas, as this is the time when kids get new gadgets, or you need to build a new phone with two antennas, as the phone with only one is not good enough.

When it comes to the cost, using stable scrum teams makes determining a budget straightforward—assuming that labor is the decisive cost factor. If you have to scale your project, accurately forecasting the budget is more difficult, particularly for new-product development projects.

If the budget is in danger of getting overrun, the product owner has to make a choice: Either ship with less functionality, or increase the cost by asking more people to join the project—as long as there is enough time for the new project members to increase productivity. But beware of Brooks’ Law: “Adding manpower to a late software project makes it later.”

OK, now that this is covered, let’s get to the real thing, here is my post on agile planning.

 

Posted in product management, project management | Tagged , | Leave a comment