Scheduling the baby out with the bathwater

When it comes to true innovation, try you must or do you will not. Whilst the larger companies have whole R&D departments dedicated to experimentation with new techniques and technologies, the smaller outfits or individuals have to innovate as they go along. One of the first casualties of war when it comes to tight schedules is the wistful experimentation that leads to great leaps forward. Playing and fiddling with new technologies, new ideas or just exploring whacky thoughts is what ultimately invents the new stuff that’s really exciting. It also exercises the brain, develops new knowledge and expands horizons. When you lose the chance to play in this fashion, then you’ve damaged your ability to develop the quality of software that, ironically, your clients would really love you for.

As Douglas Adams once said, “I love deadlines. I like the whooshing sound they make as they fly by”. Other than the detailed specification, never has a more comprehensive pack of lies been produced than the software project schedule. They don’t all start out that way, but arrive there via a collective negotiation of errors. Amongst the bottomless pit of reasons why schedules are so difficult to get right are the yin and yang of incorrect estimations:

  1. You are fundamentally scheduling invention and
  2. There is heavy pressure to keep the duration and costs low

The first one is obvious: unless you are purely assembling simple parts, then your estimate is, at best, going to be an educated guess based on your experience. The second is where the pain usually comes from. Whoever is paying for the work wishes to pay the least amount possible and wants the project delivered as fast as possible, which is only reasonable given that money doesn’t grow on trees and nobody likes been bent over a barrel for a thorough ripping off. This creates an enormous amount of pressure to take your first best-guess schedule and shrink it in order to make everyone happy.

The problem was that you were already 50% out with your estimate and any further shrinkage is going to make a bad situation even worse. It’s like receiving a punch in the face whilst being kicked in the balls. Let’s visualise the problem using one of the scheduler’s best torture tools, the Gantt Chart!

Schedules

You see that red bar? Unless it's in your spare time, that never gets done, regardless of its benefit. Irregular Pigeon is here to point out just how irregular this should be. Little does he know, eh?

If you’ve been involved in software development then this might have given you a cold chill of familiarity. Note from the overflow that there is a substantial miss, usually by at least 50%. If you work at a games company and you’re not towards the top end of the chain o’ command, then this massive schedule overrun is usually compressed at your expense using the medium of “crunch time”. This genius process is where employees are guilted into working shudderingly long weeks (80 hours plus are not unknown) for months (often without any additional pay) in order to cover up for management’s piss-poor, over-optimistic scheduling, under resourcing or whatever else contributed to the error. Of course the full blame is assigned to the technical people who were pressured to deliver such outrageous estimates in the first place when, in fact, they probably only deserved a third of the blame for not realising why their estimates are grade-A bollocks:

  • Gantt Charts encourage a task oriented rather than problem oriented schedule. It is utterly impossible to describe the entire problem’s task list in advance of doing it
  • Your client’s specification is a variable, not a constant. This variable is tweaked by them and you. They change their requirements as they learn and you change the requirements as you learn – it is part of being smart enough to change to meet changing needs. This will absorb most of the rest of the contingency. The act of doing clears the mists
  • Things never go right all the time. Half of your contingency will be used by things that you could never possibly have predicted and many will be totally out of your control (what’s that? Another massive incompatible SDK update?)

The approach I like to believe that I take is the Scotty Approach1. I am asked how long something takes, I estimate then I double it. That way, when I am inevitably talked down, I can remove a smidgen of the contingency and still have time to exceed everyone’s expectations and deliver on time. This would not have worked for me twenty years ago because I did not have the experience to make any of these estimates accurately. Now, though, generally I have the knowledge to schedule invention without being light-years away from reality (even if those schedules are generally ignored). Even with this approach, there isn’t much room to manoeuvre. The actual formula I use to turn an estimate into something half-accurate is this:

  • Create best possible estimate, let’s call this X.
  • Multiply X by 1.5.
  • Then add another 10% for good measure and round up to the nearest month.

Example: you estimate that a piece of software is going to take 12 months to develop. X = 12. Now multiply by 1.5 leaving you with 18. Now add 10% leaving you with a shade under 20 months and round up to 20. Crikey! 20 months! But it’s surely only a 12 month project! No, no it isn’t. It’s a 20 month project when you add everything else in.

Total time = ((estimated time) x 1.5) x 1.1)

Ok, you can simplify this formula but I prefer to see it broken down like this. The 50% is all the extra stuff that you and the client didn’t consider at the outset. It’s the stuff that was forgotten and the stuff that becomes important as the project develops in order to deliver something that is suitably excellent. The 10% is the contingency. To be honest, I’d like it to be 20% as it’s genuine extra time that dictates the quality of the cake’s decoration. It also provides proper time to review and test the software — something which is usually the first thing to get cut.

The net result with my normal formula is that if it is stuck to, you deliver high-quality software on time and on budget and are in a position to exceed expectations of all stakeholders whilst delivering excellent value for money… so long as you are a half decent developer.

Then, sometimes, despite all your experience you make one or both of these errors:

  1. You agree times and budgets without a decent or full specification: the “gentleman’s agreement error”
  2. You end up quoting X, or worse still, slightly less than X under immense pressure to reduce time and cost

Either way, you are now set up to crash and burn. You end up delivering the full formula’s worth of work, but you can usually only bill for less than half of any over-run, if at all. Professional pride ensures that you run it out to the line but the lateness is a cross you bare: further projects will inherit the bad estimates and you run the risk of being locked into a continual nightmare of under billing for increasingly large amounts of work.

All words, no answers

I have no idea what the answer to this is. Unless you’re EA squeezing out another iteration of one of their evergreens or are in such a position that you can simply say “this will take as long as it takes”, then scheduling is always going to be a tough exercise in educated guesswork: travel past the mysterious specification of incompleteness, through the woodland of over-optimistic estimates and then wade knee-deep in the swamp of unrealistic expectations. The larger, more evolved, better resourced companies can mitigate and control the risks through process: proper change-control and staff who’s job it is to deal with all this stuff. Whilst you procedure your way out of a whole stack of creativity, you do create a situation where slippage is minimised and controlled with the added bonus of potentially being able to avoid it altogether by simply managing the situation professionally. For the small or individual developers this kind of management overhead is simply unrealistic: you either do the work or manage the work but you can’t do both effectively without some compromise somewhere.

I make the scheduling mistake all the time. I ignore my gut feeling, bow to pressure and agree to schedules that are going to be tough to achieve. That isn’t to say that they can’t be achieved, but I condemn myself to additional work either paid or unpaid that should be spent rocking backwards and forwards in a hammock with a gin and tonic in one hand and a good book in the other. Take a look at my independent iPhone game: I’ve already ignored my own formula for calculating its duration. It should be a seven month project but fortunately, I don’t care. It’s for the fun so whether I deliver it or not is not really the point but I accept that I am a special kind of hypocrite for lecturing on the subject of bad scheduling decisions whilst making those bad decisions.



Why task-orientated, tight schedules for games don’t work 101:
Take this snake-oriented game. The snakes indicate the area of the specification that is unchanged. Oh! What’s that? Giraffes? That’s right. Turns out that adding vertical snakes with giraffe patterns on them made a better game: but not a single giraffe existed in the documentation when the project started. The original task-oriented tight schedule contained less than half of the tasks that were eventually implemented. Thus, it was whoppingly out from the start. This is on-going specification dilution. It’s normal. It’s good. It shows you’re smart enough to not flog dead horses: fail to build this into the schedule and terrible things happen with milestones. They end up like concrete boots.

Personal hatchet job aside, though, “accurate” scheduling is done only if the parties involved accept that;

  1. the up-front knowledge of the scope of the project will be under by a good 30%,
  2. the act of doing will change the project scope,
  3. scheduling by detailed individual tasks is doomed to failure,
  4. unexpected surprises are so normal that they’re really expected surprises,
  5. testing always takes longer than anticipated and needs to be factored in as a continuous process and finally,
  6. you’ll still need contingency to experiment, learn and try: schedule this out, you lose the little touches that allow your solutions to really stand out

To spot the expert, pick the one who predicts the job will take the longest and cost the most”

For the work that pays the bills, though, pressure to deliver is high and over the years I’ve delivered a shocking amount of work that I am not able to charge for for one reason or another. Other than being truly independent financially, it is difficult to place yourself in a position where you can quote realistic times and budgets without someone else being prepared to undercut you knowing that they won’t deliver, but figuring that getting their foot in the door is more important than delivering quality. Winning contracts by quoting the least amount of time normally means that QA and contingency have been scheduled out of a schedule that was probably already utter fiction anyway.

Quality, well structured, reliable software development is hard to come by. Endless warranties that stretch to infinity are even rarer. It surprises me that the impression is that this stuff is easy to do – it is not. Quality, as it does in all areas, comes at a cost. That is not to say that there isn’t a sliding scale of value, but it’s amazing to me that so many people expect first class service for third-class costs.

In the meanwhile, though, yeah, sure: I can shave a couple of months off that schedule…


1 I say “like to believe” because this ideal situation exists purely in my dreams. I dream that I estimate it’s a year’s work, quote two years and they say “OK, Mr Cobras, that’s great, but we’d love it if you can do it in, say, 22 months.” I say “Yes, I can manage that” and then love, happiness and general excellence washes over the world at large.

This entry was posted in Software development and tagged , , , , . Bookmark the permalink.

11 Responses to Scheduling the baby out with the bathwater

  1. Montaigne says:

    So, essentially, you suggest under promising and over achieving :)

    The ultimate problem is competition and the only way out from under this yoke (lol I originally spelt this as yolk) is by building up a reputation to the point that you can charge high and long on the basis that your clients know you will deliver a superior product, which is why I’m happy to wait for Diablo 3 to be released, at some point in the distant future!

    I work as a broker for portable appliance testing firms. A number of them are new starters and they’re competing in a marketplace that is saturated with competitors, where they have no work history, no unique selling points at all and where 10 firms compete for every quote, yet they are unable to understand that they cannot charge a premium price for their services. The only thing they have going for them is cheap prices but I have a constant battle trying to explain this.

    Surely it is the fault of individuals within the gaming industry if they’re (I’m sorry to say) stupid enough to accept 80 hour weeks for months on end without extra pay? I used to work for free in Sales for unpaid overtime because I wanted to (and I did it as extra work ultimately meant extra commission) but doing 2 month’s of work for 1 month’s pay just seems pretty crazy.

    “That is not to say that there isn’t a sliding scale of value, but it’s amazing to me that so many people expect first class service for third-class costs.”

    Another problem with software development I imagine is that it is so easy to outsource it overseas. Highly skilled Indian programmers can do the same thing for a fraction of your costs so the only protection you have I would of thought was to fall back on reputation and quality.

    • Chief Cobra says:

      It is absolutely a complex situation. One of the things that I have found is tough to outsource factory-style is creativity, design and architecture as-you-go and the flexibility to manage to continue to play a good game of football even when the goalposts are moving faster than a cheetah on speed. It is this stuff that creates an environment where I can provide something that is tough to sell in a six-pack. I have seen (and have had to pick up the pieces from) the situation I mention where the young and inexperienced win through undercutting everyone else and then make such a whipping hash of the situation that their contribution is a write-off, but not before they have billed for a catastrophe and then fled screaming. It is these customers that I have helped who gain a fresh appreciation of the value of quality.

      I could not agree more with you about the 80 hour week situation if I sat here with a thesaurus. Having BEING stupid enough to work 80 hour weeks I know that it is pointless, thankless and ultimately is just a collection of hours of my life that I will never get back again.

  2. Montaigne says:

    “Having BEING stupid enough to work 80 hour weeks I know that it is pointless, thankless and ultimately is just a collection of hours of my life that I will never get back again”.

    Doh, Montaigne removes foot from mouth lol.

    But is that you working for your own firm or working for someone else? If it’s for your own business then it makes sense. I work those sorts of hours every week as a matter of course and I currently earn a third of what I was earning when I was working for someone else but the long hours are an investment in the future.

    I’m a bit militant when it comes to being an employee. I’ve worked for too many employers who take advantage of their staff and staff, not knowing their rights, are afraid to say or do anything.

    From your above post it strikes me that Game staff working huge unpaid hours to get a game finished is the norm rather than an isolated incident but the only reason it is the norm is because the employers know they can get away with it.

    One of my ex employers instigated a fining system on his employees where if they made a mistake he would dock their wages 5p. That doesn’t sound much but when PAT testing you enter a site code for a job and then each test that is done automatically adds the site code to the test and then the test number. You may be regularly testing 1000’s of tests per site and getting charged 5p a test because each one is a mistake is ridiculous.

    This small company waited until I had left as they knew I would fight back, then set up the new fining system without telling anyone. In the first month one engineer had £450 docked out of his wages due to a fining system he didn’t even know about, another had £20 docked for a single incident because the MD “didn’t like her attitude”.

    All of this is categorically illegal. When my mate told me I printed off the relevant legislation and pointed out to him the law and told him to raise an official grievance procedure, threaten an industrial tribunal and watch the fining system disappear. He was too scared to do it for fear of being fired and so an illegal system is still in place 2 years later (and which keeps getting added to in order to claw back more money from the staff).

    • Chief Cobra says:

      Done it for both my own company and for companies I worked for. As for people being scared to raise official grievances: I’ve seen that too. It’s amazing how easily people are spooked by underlying threats, guilt and the fear of suddenly finding themselves out of work. Employers can afford better legal representation than employees and in tough times it can be alarming to assert one’s legal rights just in case.

      As for game companies: it’s not at all of ’em, but it’s a huge problem across the industry with some pretty high profile legal cases at some pretty big firms when staff tried to fix these things.

      That fining system is incredible. Just when I thought I’d seen some pretty amazing examples of employees getting taken for a royal ride you have raised the bar by a substantial level by mentioning that.

  3. Montaigne says:

    “It is absolutely a complex situation. One of the things that I have found is tough to outsource factory-style is creativity, design and architecture as-you-go and the flexibility to manage to continue to play a good game of football even when the goalposts are moving faster than a cheetah on speed. It is this stuff that creates an environment where I can provide something that is tough to sell in a six-pack.”

    Surely there are Indian versions of you however? :)

    It’s just that clients need to know where to look.

    I’m not trying to be derogatory in terms of your skillset. It just must be incredibly difficult starting out in such an industry but I would assume that people start at the bottom and if they’re any good they build up a reputation and a network of contacts and this is what enables them to combat cheaper options.

    “I have seen (and have had to pick up the pieces from) the situation I mention where the young and inexperienced win through undercutting everyone else and then make such a whipping hash of the situation that their contribution is a write-off, but not before they have billed for a catastrophe and then fled screaming. ”

    Yes, I occasionally inhabit freelancer forums and that tends to be a common complaint. I signed on to a freelance site once in India for doing proof reading for Indian firms. It was a waste of time as people were quoting stupidly insane prices for work which worked out as less than a dollar an hour. These forums were just full of people offering dirt cheap services and there was no guarantee of quality.

    Especially for design/web design freelancers they find that customers want the cheapest price they can find and it’s amusing to read the horror stories of one freelancer quoting high, the client going with the dirt cheap price and then coming back 2 months later after losing their initial payments and having no result for their efforts and asking the higher priced freelancer to pick up the pieces.

    • Chief Cobra says:

      “Surely there are Indian versions of you however?”

      I’m sure there are, but I BET that they can’t draw giraffes, snakes and germs like I can.

      And you’re right: clients just need to know where to look, it’s a big planet. It gets marginally easier when it is specific experience that is required as some software experience isn’t as common as a lot of stuff.

      I’m kinda excited by things like the iPhone game that I am, in theory, creating. That is a case of being partially in control of everything: if it’s not a crap game, then, well, it might do OK.

      But ifs and buts don’t pay the bills, feed Baby Cobra or keep me in wine and I guess I’m lucky enough to have found such exciting work that matches my somewhat odd range of skills :-)

  4. Montaigne says:

    Because of my militancy lol I have many bizarre stories like that. I always try and defend my colleagues when they’re being bullied.

    On one occasion I was officially disciplined by a sales manager ( who I didn’t get on with. When I asked him for sales training once he sent me an email that suggested I don’t try to aim a duck to death. End of training) for “interfering” i.e. helping my colleagues out when they asked for help.

    I asked what I should do when people ask me for help. I can’t just ignore them? He said that everything they need to know is in the employee handbook so point them to that.

    So, a colleague a couple of weeks later disagreed with a decision the sales manager made so asked my advice. I explained that he would need to look in the employee handbook where it explained that if you disagreed with a decision from your boss you could raise a grievance procedure and it would be heard by the manager of the person you disagreed with.

    Later that day I was informed that I needed to attend another disciplinary procedure as following my sales manager’s instructions, to the letter, was also a disciplinary offence.

    This same manager put me on a disciplinary procedure for not spending an average of 1 minute 30 seconds or more on the telephone. I explained that everyone else just listened to recorded messages whereas I hit zero to bypass them to get straight to receptionists so my calls were much shorter (I was also top salesman of the team). He ignored this and told me I had to increase my call times.

    I explained that this would lead to me selling less but he insisted, so in a formal disciplinary with minutes being taken I told him that I would listen to recorded messages, take my time leaving messages on answerfones and this would lead to a drop in my sales; the meeting ended and we were to reconvene in a month.

    I dropped my talk speed by about 30%, listened to recorded messages all month and generally did the opposite of a good salesman i.e. I played the system and my sales dropped accordingly.

    At the meeting a month later the sales manager examined my call stats which had increased dramatically, ignored my drop in sales and congratulated me on the vast improvements I had made in my performance even though a month earlier I had said to his face that I was going to game the system :)

  5. Montaigne says:

    “I’m kinda excited by things like the iPhone game that I am, in theory, creating. That is a case of being partially in control of everything: if it’s not a crap game, then, well, it might do OK.”

    That’s why I am loving being self employed generally. I have nobody stopping me from doing what I want. I don’t have to wait 6 weeks just to get a new letter template approved!

    It really feels like I have been on holiday for 16 months even though I’m probably earning less than minimum wage at the moment given the number of hours I work.

    • Chief Cobra says:

      Hahaha! And no-one wants to wait 6 weeks for a letter template! I’ve had some fun with starting a business but there is something about being able to work with/for a bunch of great people to help achieve something really cool that I quite like. There’s the security, for sure, and that’s important but also it’s being part of a group of people all fighting to do something special. Being self-employed in the games industry is tough unless one strikes it lucky! And this is why I have set my sights so low with my iOS game: it’s for the fun and for the sense of achievement (and also to learn some new stuff and play with some amazing devices). And I get the excuse to draw snakes. Rattle rattle rattle.

  6. Montaigne says:

    And it was 2 years I had to wait for the new letter template but I didn’t think you’d take it seriously! At one point the sales manager deleted all the templates from the CRM package without backing them up and then couldn’t re-install them.

    The IT department said it was impossible to re-install them and insisted on this even when it was pointed out to them that as there were templates on the system originally, in order for them to be deleted, then surely this demonstrated the falsity of the statement that it was impossible.

    In a sales meeting about improving sales stats a financial director asked an entire sales team, with a straight face, whether they had ever considered just asking for an order!

    I’ve worked for some really stupid people.

  7. Pingback: Book Club episode II: Wool and the art of software documentation | Cobras Cobras

Feel free to say hello