How does agile development fit with budgets and financial controls?

For the last 12 months I have been immersed in an agile development environment which really follows the agile mantra, e.g. stand ups, pair programming, test driven development, 2 week iterations, retrospectives, refactoring, and the development world is pretty harmonious and delivers. The fun starts when you spill into other domains of the organization that has other objectives to fulfil.  For instance we have continual friction between developers and the project managers over cost and effort on future projects.  Left to our own development agile-centric world we would estimate for the next iteration and refuse to speculate on the future.   The industry, and company in particular, I am currently working in, is very dynamic and time-to-market is critical and development cost is just one component of the project make–up, and for the decision makers they drive for an understanding of likely costs and timescales. I’m sat in the middle trying to pacify developers who really don’t believe in estimating beyond the near future, and a business that insists on a full picture.  “Change the company’s ways” is the cry I hear from the developers but at best this is a far distant result, and in truth, it may never happen. 

And my wife can tell you why.

I ran an experiment with my wife over the weekend, which convinced me there has to be another answer to this dilemma.  We want to have a conservatory built, and we have a pretty good idea of what we want.    To boot I have lined up a really good builder who I trust and is able to start work fairly soon. All that is left is to convince my wife to adopt an agile approach to the work.

So I say to my wife I would like the work to start, and that she will work closely with the builder (she will be the product owner – I didn’t even suggest that might be a role for me!) and every couple of weeks we review how much we’ve spent, how much the next piece of work will take, and over time we know when we’ll finish. Her response to my very sensible agile approach left me a little nonplussed but it had a lot to do with her not living in a building site at Christmas when her family visits (because I can’t tell her when my favourite builder will be gone) and if it costs over 15K she doesn’t want it anyway. For the sake of my marriage I will not be advancing agile techniques on home projects in the near future.

Going back to the business example – it remains a real problem. Having good velocity and timescales on previous projects would possibly help but, to date, we have been poor in collecting these consistently.  If anyone has any suggestions I would love to hear them as longer-term planning and agile development are not happy bedfellows, at least not where I am, and certainly not at home.