MAG:RAILS 2011

Videos Are Out!

So at long last our videos are online! You can check them out on our main page or go over to our YouTube channel.

Enjoy, folks!

Old mistakes in a new guise

Introductory thoughts, or what triggered this article

I was at a team retrospective yesterday and we were grappling with some really difficult issues around a lack of tests on a ruck of badly-designed legacy code.

The first point I feel I need to make is that the individuals who wrote this code didn’t fail, it was the system that commissioned the work and oversaw its construction that did. In our Western, individualistic, culture critical comments get taken personally and they shouldn’t be. For me, one of the most powerful things that came out of Magrails was Bob Marshal’s quote from Deming that 95% of an individual’s performance is down to the system they are forced to work with, and only 5% down to the the person themselves. So, if you aren’t in a high-performing team that has goals which should result in well-crafted, considered output, you won’t be able to produce it (in general). And you may not be able to deal with it either, except by going somewhere else. This sucks, but at least you know it’s not your fault and your frustration is justified.

The team want full test coverage and to deal down the non-fatal error count we get from tools like Airbrake so that it’s actually reporting real errors we can do something about, instead of noise about nils that comes from bad design and implementation.

If you talk to a development team, particularly if they inherited the code base, the usual solution to this is to rewrite whatever it is that causes you so much pain. Five years in, with a functioning business who can do that what they need, would be throwing your money away – like that’s going to happen! Instead you need to approach things by following the ideas outlined by Michael Feathers in Working Effectively with Legacy Code spiking new functionality into a new code base and making sure that it’s fully tested, plus refactoring into tests any code you have to touch to add that functionality. Michael’s approach is incremental, and painstaking, but it does give you a solid base over time without a pointless, massive, upfront investment. Read Michael’s book – it’s full of considered detail explaining how to do this. His twitter handle is @mfeathers and blog is here – I met him at Scottish Ruby Con earlier this year and he’s a really nice guy too. Looking forward to his next book, which I believe will be called Brutal Refactoring.

I was thinking about where the development effort is going over the next few months and realised that at least some of the core functionality is being rewritten and repurposed for an internal business function (can’t give details, sorry). The lead developer on this is one of the best devs I’ve ever worked with (you know who you are) and I know that this stand-alone module will be done to the highest standards and have full test coverage. It’s not up for debate, that’s how he rolls.

So, the speculative part of me started thinking instead of trying to polish the current turd, why not build outward from the new shiny? This code is repurposing a lot of customer-facing code – so why not pull that back into the customer facing area and rip out the cruft? It sounds like a good plan, and it is, but also it isn’t.

Where does bad code come from?

I did a talk at Agile North 2011 on YAGNI and there’s a blog post – one of the things that I talk about is how working in silos gives you really maladaptive code. Each silo’s code is probably fine for the silo, but in terms of the whole business it’s rubbish and you have to spend a ton of effort joining the disparate areas together. Not necessarily so bad it kills the business (although I’m sure that this has happened) but in terms of the flow. Using kanban, and reading around the systems thinking ideas (plus John Seddon’s excellent talk ), I’ve come to realise that building outwards from this particular shiny is probably a bad idea.

One of the reasons we’re faced with so much pain is the system was built to run the call centre, and sell our services. Unfortunately, the finance and marketing functions weren’t in the conversation, and there’s a whole heap of nasty compromises and (quite frankly) incoherent crazy stuff, that works, but makes your eyes bleed when you try and change it and has potentially lethal non-local effects. This of course drives the effort to fix or change things up through the roof. But it’s not unusual in a lot of businesses. I have to say I’ve been here before dozens of times.

So, the cost is in the flow. It’s not in the tasks themselves, it’s in handing them off to someone else. For example, Seddon talks about how removing the annoying IVR call centre crap (“press 3 to shoot a pony, press 2 to ride a unicorn, press 0 to speak to someone who is being treated like a robot”) can actually increase capacity by 25% by meeting customer needs straight away and avoiding hand off and customers giving up and calling back etc. He says a lot of capacity is taken up with failure demand. He also says managing by cost makes your costs go up – good eh?

Of course, not managing the team, not doing code reviews, not starting from a place where quality is built in (prevention being far cheaper than cure), is most definitely part of the problem too. But that’s water under the bridge now, and the current team are more than capable of “blowing the bloody doors off”. They also have the full support of the CTO, who is willing to go into battle over the quality issue.

So, ok Francis, what the hell should we do?

This is where I should try and sell you the new book I haven’t written yet, but I’m not going to do that. I’m still thinking about how to do this kind of stuff. My initial thoughts are that the new shiny should incorporate some end-to-end functionality that exposes holes in the existing architecture. Or better yet, starts constructing a new architecture without the existing problems and faults. Then you can build outward from it, by starting with something that crosses all of the functional areas.

I don’t know if this will happen, and I’m moving on to a different job at the end of the week, so I will probably never know. I sincerely hope that it does, because the opportunity to start from a better place is there with a little bit of investment, and the excellent talent doing the work.

Guest blog: The Agile Waiting Game

We have a special guest blog today from Ash Moran over at Patchspace.co.uk

The Agile Waiting Game

Agile practices (pair programming, TDD, continuous integration, etc) and methodologies (Scrum, XP etc) are intended to increase the productivity of teams that adopt them. In one sense you can think of it as “team + practice = improved team”. The bit that gets the most focus is “practice”; what we will explore here is the nature of “+”.

Filling up

If you want to take a bath, initially the bath will be empty. You make an intervention to fix this undesirable state: put the bath plug in the plughole and turn the taps on. Is the bath ready now? I mean, now? Unless you’re reading this over a modem from 1983 and it took you an hour to get this far, the answer is no: the bath hasn’t had time to fill yet. There’s a delay between turning the taps on and the bath being full, somewhere in the region of 10 minutes. This is significant: it means you can go and make a cup of tea without fear of flooding the house. It also means that if the bath isn’t full in half an hour, something is wrong. (Perhaps the plug is leaking slightly, or the flow through the taps is too slow.) Whatever happens, we intuitively know how to use this delay to manage “bath + running taps = full bath”.

The same idea is equally - if not more - important in software teams. Without a sense of the delay between the original state and the desired state given a certain intervention, it is easy to over- or under-manage.

Say your team currently integrates and releases on a 6-month cycle, at the end of which usually follows a month of mad scrambling and tail-chasing to deploy the code. You want to improve this, so you intervene by introducing fortnightly iterations, where code is supposed to be fully integrated at the end of each iteration. If after a month of this (2 iterations), the team’s productivity is down, changes are not being fully integrated, and not all integrated features are fully working, has the intervention failed? At this stage, there isn’t really enough evidence to suggest this. An understanding of the nature of the intervention and its inherent delays helps here.

Moving to frequent iterations reduces the batch size of work being integrated. A manual integration process twice a year may be fine; the same process every 2 weeks will show enormous waste. Therefore the team will need to automate its integration process, but this will take time. This effort will take capacity away from existing development, so in the short term you would expect measured productivity to go down. (I’ll ignore any debate on actual productivity for the purpose of this example.) Also, with more rapid integration, team members will also be forced to communicate more frequently, and meetings will need to be made more focused and resolve conflict faster.

If 3 months later, the same pain is still being felt, has the intervention failed? Now the answer looks more like yes: it should not take most teams three months to achieve significant improvements in a deployment process. However, the the real answer is context-specific, or “it depends” as it’s known in the consulting trade.

There is no need to pick just one or the other out of the two (arbitrary) time-points: reacting at 1 month or 3 months with no information in between is still less than ideal. When filling a bath, you don’t need to wait until the end to see if it’s ever going to fill: by checking occasionally, you can see if it filling as expected. You may be able to see water leaking as it becomes half-full, or you may see it filling slowly and realise you only half-opened the taps. This extra feedback lets you manage more effectively. But if the bath takes 15 minutes to fill under ideal conditions, and you’re only 10 minutes in, don’t expect it to be full yet.

You wouldn’t blame a bath for not filling quickly enough, so don’t blame a team for not improving fast enough, unless they are actually goofing off. Slow progress may be due to a new practice coming into conflict with existing policies or mindsets (which you should watch for), but even under ideal conditions will take time.

Draining out

The reverse situation is also important. When you take the plug out of a bath, it takes time to empty. Imagine you filled the bath and got in, but half an hour later - to your bemusement - you realise the water level has dropped significantly. What gives? It may be that there’s a tiny crack you didn’t notice when filling it, which has allowed water to seep out. Again, the inherent delay and your own perception means it took a while for the problem to become apparent.

Software development teams can have leaks too. Any time a process which maintains long-term productivity of is dropped or reduced, the team’s performance will also start to fall - but it will take time for the problem to appear. Reducing TDD will not cause the defect rate to rise sharply, nor the ease of adding features to fall sharply. Instead, what will happen is that 6 months down the line, customers are complaining about the increasing number of bugs, and developers are complaining about the increasing difficulty of fixing them. The delays between different people’s responses will not be uniform, either, so watch for the early complainers - far from being a nuisance, they’re the system giving you a hint of what the future holds.

There’s a particularly insidious situation that can emerge when practices are dropped, known in systems thinking as “shifting the burden”. Here, some team members will pick up the slack, increasing the delay before the fundamental problem becomes apparent - which it will do so dramatically.

Again, you wouldn’t pull the bath plug out of a bath and exclaim “The bath isn’t empty! We didn’t need the bath plug after all!”, so don’t let a team reduce or drop practices it knows to maintain productivity, only to act surprised when things fall apart later.

A few delays

Start looking for delays in your team and software development process. As a starter, here are some examples: * learning a new language or framework * changing the practices in use (including learning a new one) * changing the members of a team * changing suppliers * taking on or dropping clients * learning to look for delays

If you’re too busy in the day to think about the delays around you, think about it next time you’re running a bath. What are your thoughts? Feel free to leave your comments below. I’m happy to wait for them.

My name is Ash Moran. I’m a software developer and agile coach, and owner of PatchSpace Ltd (Twitter). If you have any feedback, questions, or would like to know more about my services, feel free to contact me at ash.moran@patchspace.co.uk. I’ll be at the MagRails 2011 Conference, so please say hi if you see me in the corridors, and feel free to add me on LinkedIn if you run into me there.

Linode and MagRails

We're very happy to be able to offer a free 1-year Linode 512 server instance to one lucky attendee of Magrails!

Linode are a VPS hosting company built upon one simple premise: provide the best possible tools and services to those that know what they need. As a Linode user, you get everything from the kernel and root access on up and a wide range of modern Linux distributions to keep your server running tip-top.

This is a really wonderful prize and we're very grateful to Linode for their generosity. Keep your eyes peeled for details on how to win!

Apollo - Get. Things. Done.

We're really pleased to have Apollo on board with Magrails 2011. They have very generously offered a lifetime discount code for each conference attendee!

So what is Apollo? Apollo is project and contact management done right. Using Apollo, you will realise that it's built to help you get things done, quickly and efficiently. With Apollo, you will always know where your projects, your contacts and your life are at and you will feel on top of everything — regardless of how hectic your schedule is.

We can testify to this here at Magrails HQ as we've been using Apollo to manage our conference task lists and it's been completely invaluable. Well worth checking out to help you manage your projects, your schedule and your own personal to-do list.

Apress Discount!

Apress has very generously offered a 50% ebook discount to all attendees of Magrails!

They offer a wide selection of technical texts, both in print and digital form and provide a reliable and readable place to start learning something new. Developers can rely on Apress Pro books to provide the no-fuss, honest approach that is needed for solving everyday problems and coverage of topics that is broad enough to support their complete career development.

Check your conference bag to learn about how to claim your ebook.

Twilio Joins Magrails!

We're happy to welcome aboard Twilio to the Magrails 2011 Sponsorship. Twiio provide excellent APIs for voice and text messaging and boast an impressive range of high profile clients including eBay, SalesForce and LinkedIn.

Check out how it all works over at twilio.com/api where a free tour and trial is available.

Twilio's Stevie Graham will be delivering a talk at the conference concerning good RESTful API design and elegant consumption. We're very excited to hear what he has to say!

Melbourne Is Buying You Lunch!

We're proud to announce that the Manchester-based Melbourne Hosting has joined the Magrails sponsors and is officially buying you lunch!

Food on the day will include coffee, cookies and muffins in the morning and afternoon with a buffet lunch of sandwiches, salads, soups, pasta and cold meats. Halal, vegetarian and wheat-free options will be available. If you have any special dietary requirements then please use the Contact Us function on the main page and let us know what you need.

Melbourne host around 3,000 servers on a variety of different platforms, each offering its own array of managed services. Launched by Daniel Keighron-Foster, a 29 year old entrepreneur, the company is one of the only hosting specialists in the North West to be in control of its own infrastructure boasting two data centres and a dedicated disaster recovery facility.

Also, they assure us that they Will Beat You At Pool.

We Love Manchester

We've said it many times in the course of launching Magrails but we're very proud to be a part of this city. Manchester is a wonderful place to live and work and we hope everyone who visits for the conference will see what a vibrant, fun, charming place it is. To show our support and love for the city, we've teamed up with the I Love MCR campaign to give all our attendees a free memento.

FreeAgent will be at MagRails

FreeAgent will be at MagRails to let you know just how great their App really is. Every attendee will get a useful gift from FreeAgent to help you experience what they have to offer. I use FreeAgent almost everyday and think it really is a tool that every Dev should not be without. FreeAgent launched in 2007, hewn from the frustration that managing company finances was just too damn hard for most small businesses and freelancers. Since then they have doggedly stuck to their mantra of demystifying accounting and redefining the relationship people have with their finances. Look forward to seeing you their FreeAgent.

FreeAgent are hiring at the moment so check out their jobs page or ask at their stand on the Conference Day for more information.