Agile Development or how many balls you think you can juggle?

Most of the time, when people talk about project management, they only consider a phase between the official start of the project M1, and beta release date M2, as a “real work”. As explained earlier, this is a wrong approach, and you should avoid skipping very important preparation/workshop phase, which although sounds like a waste of time, is extremely important step in overall process.

We often associate this time frame with “busyness syndrome”, stress, heavy workload and stuff like that. In this blog post, I will try to make an overview of tasks that people should be doing in this time slot, and explain one way of organizing teams around various parallel activities.

But before I move on, I would like you to imagine the following:
You are the captian of the ship, and you need to get your guys from point A to point B in couple of months. On this journey, there will be some storms, troubles with a ship, and sometimes, issues with a crew. Before you start the journey, make sure that every crew member is good at something – and willing, otherwise, should not be on the ship in the first place. You need to explain the purpose and destination of the trip, and not to forget, what is the role and expected contribution of every person. During the journey, you will have the chance to adjust your direction at fixed time intervals. The actual length of one time interval will depend on the inertia of the system – the ship itself. Bigger the ship, longer the intervals. The end of a time interval (iteration or sprint) – is your correction point. At that moment, you position the ship for the next round, and till the next point arrives, the ship must go in the same direction. Once you reach the end of the interval, you adjust the path, and move on to the next point. Within one interval, nothing can disturb the crew – that is your task too. If you pictured this story right, this will be kind of zig-zag path journey, going from A to B. (If you are a guy that hates blog posts that are longer than few lines, you can as well stop here, that is all you need to know 🙂 )

And here we go, well almost….few more checks first…

  • First do your journey preparation as described before
  • Choose your iteration span – (you start iterating from M0, but you may as well decide later to have a different span between M0-M1 and the rest of the project lifecycle- although this is not a good practise). I personally prefer short iterations, fast feedback is always good (imagine taking shower and having to adjust hot and cold water – while the time between your action and the tap actually responding is 4-5 seconds – this is always fun, I had it few times). If you think you need long iterations (longer than 4weeks), you either manage a space shuttle program, or your processes are really lousy.
  • Decide on the branching strategy (see my post on the maintenance), and check this site  (thanks Joachim @joachimvda for the link) and awsome CRISP folks for one more great article.
  • Find a good build manager, setup jenkins, bamboo, git etc …
  • Find a good Test Scrum Master (this is something new, no?)
  • And fasten your seat-belt…..hmmmmm, i just realised I am on the ship actually, anyway …GO!!!

Juggling intro (now I am almost there…)
In all scrum books you can find something that is called “definition of done”. Idea behind this is that you don’t close the iteration till everything is developed and TESTED wiithin the same iteration. Some years back my test manager and I have decided to tweak this a little bit, by developing things in one iteration, and testing the same load in the next iteration, while dev guys would already work on something new at that time. If you have dev/test teams that are not on the same location – not even on the same continent (like it was in my case), this approach seemed more optimal to us. I also must say that this only works if your iteration are short. BTW, if you are familiar with kanban, this is exactly what I think these guys tried to fix as well (more about this in another blog post). One not that pleasant artifact of this choice is that the time between the code freeze and the time that the beta load is fully tested is also delayed by one/two iterations.

Finally, let’s do some Juggling!!!

Correction point C0: scrum kick off meeting (discussing/work split for iter1)

Iteration 1 (1 ball):

  • RD guys develop new stuff in ITER_1

Correction point C1: scrum kick off meeting (demo’s iter1, work split iter2), the current load is labelled (LAB_1) and handed over to the test team

Iteration 2 (3 balls):

  • RD guys develop new stuff in (ITER_2)
  • TEST guys test load (LOAD_1)
  • TEST guys fire bugs against LOAD_1

Correction point C2: scrum kick off meeting (demo’s iter2, work split iter3, assign issues of iter1), the current load is labelled (LAB_2) and handed over to the Test team

Iteration 3 (4 balls):

  • RD guys develop new stuff in (ITER_3)
  • RD guys fix bugs from (LOAD_1)
  • TEST guys test load (LOAD_2)
  • TEST guys fire bugs against LOAD_2

Correction point C3: scrum kick off meeting (demo’s iter3, work split iter4, assign issues of iter1,2), the current load is labelled (LAB_3) and handed over to the Test team

Iteration 4 (5 balls):

  • RD guys develop new stuff in (ITER_4)
  • RD guys fix bugs from (LOAD_1, LOAD_2)
  • TEST guys test load (LOAD_3)
  • TEST guys fire bugs against LOAD_3
  • TEST guys validate tickets from (LOAD_1)

Correction point C4: scrum kick off meeting (demo’s iter4, work split iter5, assign issues of iter1,2,3), the current load is labelled (LAB_4) and handed over to the Test team

Iteration 5:

  • RD guys develop new stuff in (ITER_5)
  • RD guys fix bugs from (LOAD_1, LOAD_2, LOAD_3)
  • TEST guys test load (LOAD_4)
  • TEST guys fire bugs against LOAD_4
  • TEST guys validate tickets from (LOAD_1, LOAD_2)

Correction point C5: scrum kick off meeting (demo’s iter5, work split iter6, assign issues of iter1,2,3,4), the current load is labelled (LAB_5) and handed over to the Test team

Iteration 6,7,8 ….

For simplifications, I have omitted few more tasks , such as:

  • Bug fixing of the issues of the previous releases (if you have them already out)
  • Validation of these issues in the next maintenance load that goes to the customer
  • Automation work (by both test and dev team) on simulators, test environment etc.
  • Test document update (you need to keep use cases and test execution in sync all the time)
  • Support for the field issues (L3/L4)
  • Sales support …

This would be already 9-10 balls in the air after iteration4! Once you do so many activities in parallel, the best thing you can do is to split test and dev teams (yeah, I know what you will say about cross-functional teams, but this is not the same thing) and let the test manager do the same thing that dev manger does – organize test teams in separate sprints and let him become a “Test Scrum Master”, it sounds really cool!

So, let’s put all things together and write it this way:


In my next blog post, I will mockup one JIRA project and explain the same topic via dashboard graphs and greenhopper screenshots. 


Agile Project Management – the project follow up

XP was a first development framework in which the customer feedback was embedded in the complete project lifecycle (see Beck Kent’s books, Ron Jeffries blog and articles, or check this list). In the picture below,  you can find “one slide explains it all” that I often used back in 2002-3, when people were asking me about “extreme stuff” we were doing:

(Side note, I can’t remember any more where I found this drawing)

User story: “As a <role>, I want <goal/desire> so that <benefit>”

Early on, I have noticed few interesting things about user stories:

  • First, it is an excellent way of introducing customer into the picture, similar to the “elevator test” idea – and I liked that.
  • It is excellent way of preparing demos after every iteration (and fast feedback which comes as the result of it)
  • It is the best way to do acceptance testing (check this guy out and his blogs/books on this link)
  • It is not always that great as the tool for project/release tracking (one interesting observation, in XP days, we used to talk about  “tracker” role for someone that was responsible for story/planning tracking – later replaced by my favorite term “scrum master” )
  • If the feature is rather complex, simple story splitting is not the best way to capture the work that needs to be accomplished.

While working on my first XP projects, I was a fanatic about data gathering, everything you can think of. We estimated tasks in days, hours, even minutes. We were all booking 8 hours a day – no exceptions. We would book coffee time, meetings, coding, demo’s… I kept doing this for years. No matter what we were doing, who was doing it, I always found similar observations:

  • 30% of all time in project is “wasted” on meetings, coffee breaks and kick-off meetings. (one more reason why I got so “freaky” about making agile meetings more efficient)
  • Almost all stories of the small size were implemented faster or on time.
  • Most of the big stories (or epics as they like to call them these days) and some small features were experiencing a strange distribution, and this is a “funny part”, the range varied from 20-30% to about 3-4 times of the original effort (in both direction, although majority of estimations proven to be too optimistic). It didn’t necessarily correlate to any point in time of the project, or how good the original estimation/analysis was done. Just one more thing for the record: I’ve had a pleasure working with some of the most experienced developers you can imagine, which makes these graphs even more interesting.
Here is a an example of two releases:

That was at the time I didn’t bother reading about “Cone of Uncertainty” or “Iron Triangle”. I was more like a chef in the kitchen, tweaking ideas rather than estimations, trying to figure out what’s the best way forward…

From then onwards, any time I see a project where the burn down chart behaves nicely, I get suspicious.  It tells me one of the following: either the estimation of all work was done with great accuracy and all developments is happening smoothly (with no grey swans around) OR someone is too busy making burn down charts look nice.

While I was occupied with my projects: planning, tracking, coding, support, bug fixing and studding the data that was in my hands, I must admit, I completely missed out on scrum movement, which was to become a dominant agile force… What surprised me at first was how much some of my conclusions differed from slides you can find in any scrum training.

One thing I realized, and that is what I have written in one of the previous blogs, is that trying to be too much smart upfront really doesn’t work – even if you have your stories. You need to code, as soon as you can, but not before that (you may hear me now talking about “invisible lines”, this is the moment when other people’s eyes start rolling when I talk like that). I also found another interesting thing, burn down charts don’t work for me, but burn up charts do the magic! Let me show you two graphs, and some important explanations that will follow:

(Graphs are created using JIRA dashboard feature – these two graphs are must for all my projects)

  • Developers must have subtasks for everyhing they do, and they are free to add more tasks any time they find it necessary. Every subtask is assigned at kick off.
  • At the beginning, all FTPET’s are estimated in either days or weeks
  • If FTPET is deemed to be complex, we address it early on (sometimes even before M0). During this process we do light analysis which results in set of subtasks. These subtasks are than estimated separately.
  • About 15% of effort is kept for refactoring. This is necessary to keep the code clean and robust in long term. Oh yes, I hear you saying: Refactoring should happen always and it should not be given a “task”. Well, I am saying the same, with one exception – I know that people will not estimate this effort when they estimate their work. I don’t really know why, but that’s the way it is. So I book it anyway (some nasty readers might call it my secret buffer – at least it is not that secret any more 🙂 )
  • All technical challenges are addressed in separate “spikes” and most of the time are not part of any release planning
  • Estimations are only about the work, it doesn’t include any overhead (meetings, maintenance, holiday, coffee time …)
  • Since R&D logs every day 8 hours, information about overhead and other non-dev tasks can be taken into account in the future planning (about 30% of R&D effort goes to overhead).
  • The team is closing subtasks with steady pace – follow the green line (interesting note: subtasks are not all of the same size, but why they look as they are? This is XP trick, every item should be finished within one or two iterations, not more than that).  Since my iterations are normally 1 week, it turns out that “story point” size in this graph on average is about 3-4 days! Multiply this by number of dev guys and …Hey, you have your velocity?!! Sounds like scrum, no? 
  • Some FTPTET’s and other big features are divided into subtask only later in the project (we have a “good idea” how much they would take, no time to split them all – Pareto rule). That you can see by following the red line.
  • Number of subtasks doubled between M1 and GA, but all FTPET’s were closed on time .
  • Most of the FTPTET’s were known to the R&D team 3-4 months prior to M0
  • Any time you add something, you make sure nothing else it broken (automation, TDD, whatever). That makes you adjust better on the major question- are we hitting the date, or the feature content?

So, what you do with these two graphs and how I do project follow up in 5 minutes every day? Is it rather simple:

You monitor the list of new subtasks that comes after M1 – this is the red line (at M0 you should have about 30%-50% of subtasks identified – check the previous post). The curve should start slowing down after a while. If it doesn’t, you are in trouble, good news – you know it.

By looking into green line, you only see that people are working, and you can make “projection” in your head when it should meet the red line – but again, this is not exactly as working with burn down chart. Burn down chart makes managers freaking, and instead of being focused on the work, they spent big effort trying to make them look nice. More over, it makes them feel like they need a better estimation process – next time, the trap I talked about in this blog Estimation Fallacy and Estimation (only) Driven Project Management. With the burn down chart, ideally, the red line stays constant – since it expresses the cumulative effort of that release. In my case, it only represents new subtasks that are created in time. I can stay focused on adding value, I am flexible to add and remove things as long as we are all on the same page (PO, me and the team) during the project lifecycle, and it is brutally transparent – to everyone.

Good, this was a long blog post, and by now you should know how you can do the project follow up in 5 minutes. But be cautious, the project management is not about task follow up ONLY! In my next blog, which I might call “How many balls you think you can juggle?” I will be talking about some other aspects of the project management during the development phase. Stay tuned.