On Monday 6 April 2009 GOVIS hosted a forum exploring the opportunities for—and issues facing—the adoption of Agile development methodologies in the government space. The forum featured a panel of four people representing several different viewpoints: Mike Lowery, Agile evangelist; Liesle Venter-Wagner, developer; Mark Pascall, vendor; and Andy Neale, client. The forum was introduced and managed by Mark Leicester, Senior Technical Analyst with the Strategy and Innovation Team, GCIO, State Services Commission.

After a short overview of Agile the panel discussed Governance of Agile projects, The Vendor/Agency relationship, and Getting started with Agile. Then the audience was then invited to vote for three topics from a longer list. The audience chose Being Agile, Agile myths and legends and Reconciling Agile with government procurement processes. The session was recorded and is summarised below.

Please accept our apologies for the occasional short gaps in the audio during the first couple of sections and the background hum throughout.

Welcome and Introductions

Mike Lowery [ML]

Mike is an Agile evangelist with over four year’s experience at the BBC and has presented in the USA on Agile practices. Mike is introducing Agile to the ACC and is helping them to engineer a culture shift, wrestle with the government procurement process, and avoid what he calls “badgile”.

Liesle Venter-Wagner [LVW]

Liesle is Applications and Database Manager at the Department of Conservation. Liesle is specifically interested in Application Development Methodologies and how Agile can be used in the government environment.

Mark Pascall [MP]

Mark is Director and co-founder of 3months.com, one of the vendors responsible for the development of Digital New Zealand. He has over 16 years experience in software development and 9 in web development. Mark presents regularly at conferences throughout New Zealand on the topic of Agile Development.

Andy Neale [AN]

Andy is Web Manager for the National Library of New Zealand and Technical Lead for Digital New Zealand, developed the Agile way. For Andy, Agile was a “very successful way of working and it enabled us to complete our development with in 16 weeks from go to whoa. It was fast, furious and an absolute blast!”

Photo from Wikipedia

Agile Overview

Mark Leicester presents an overview of Agile. This presentation was adapted from Agile Overview by Balachander Swaminathan and Naresh Jain (licensed under Creative Commons).

You can download the presentation in Microsoft Powerpoint, or Open Office Impress formats. Please read the license terms before re-using.

Governance

ML: One of the least explored aspects of project failure (including Agile) is Governance. Weekly reports, issue logs, appropriate project plans are important regardless of methodology. Regardless of language, it’s all about understanding: education not confrontation. [30-40% of audience indicates that they use Agile]. Terms like sprint, backlog, release plan, velocity, impediments may not mean something to someone familiar with Gantt charts. Avoid confrontation over terminology. Instead, get your product to understand the product they want; get teams to work together.

MP: Often stakeholders want more certainty than is available. Acknowledge this lack of certainty: build trust, transparency and performance. Governance must aid this process.

ML: [Mike draws an example burndown chart]. Translate for stakeholders as necessary. You can represent an Agile project in a Gantt chart if necessary.

MP: Agile projects cannot be governed from afar. A product owner must be tightly ingrained in the process. The focus is on tangible results (working software), not documentation or Gantt charts.

LVW: It’s hard for stakeholders to move to Agile. Stakeholders traditionally need to understand where we are today, tomorrow etc. Trust is hard establish.

ML: Focus on making your processes work, instead of the documenting the plan.

AN: DigitalNZ had a project kick-off with governance and stakeholders, checking expectations. The role of governance was made clear. Governance is often engaged at the beginning and the end. With DigitalNZ the development team presented to governance every two weeks. Governance was invited to provide incremental feedback and appreciated the opportunity. The principles for Agile came out of successful projects: common-sense principles.

Audience: What expertise did you consult to introduce Agile ideas into DigitalNZ?

AN: We brought in an external ScrumMaster.

The Vendor/Agency Relationship

MP: Trust can be a problem. Agile is all about trust. With a new vendor/agency relationship trust must be established first, perhaps with a small waterfall project. Co-location aids communication. Agile methodologies encourage co-location. The product owner must buy into the Agile process, and must be available throughout the process. Agile is about breaking big projects into many mini-projects: billing can be per iteration, making it easy for clients to go elsewhere. Agencies must be certain of budget. To develop trust the costs of a planning iteration (establishing a cost, timeline, number of story cards) might be shared (an “iteration zero”).

AN: The relationship must be based on trust, not contracts. When things go wrong in a relationship based on contracts you might talk to lawyers, but that’s really a last resort. Agile acknowledges the up front need for trust: you need a shared understanding of the principles by which a project will be managed.

ML: A government agency choosing an Agile agency must work in an Agile way. Ensure the agency and vendor are talking the same language and understand the methodology.

AN: Agencies using Agile must work to Agile timeframes.

ML: Agencies using Agile must sell Agile across the agency.

Getting Started

AN: Agile is not the cure for all issues. Agile won’t solve lack of funding, lack of stakeholder engagement, lack of vision etc. However, it can help manage the issues arising from these problems. A good candidate first Agile project might be one with a fixed time frame or fixed cost. Scrum is a project management methodology suited to web development (but not only). To increase chance of success start with a single, co-located team. Choose vendors with Agile experience. Bring in some help (e.g. an experienced ScrumMaster). We now use Agile for non-technology project management. Get the support of your project sponsor. We decided to take hosting outside of the organisation because hosting was not able to be responsive to the Agile approach.

ML: Don’t pick the biggest, most expensive project. Don’t choose something that you can’t iterate over. Don’t work in an area where business ownership is mixed, or fought over. Choose a project where failing quickly isn’t a problem. Try Agile internally first if possible, otherwise choose a good vendor and bring in a coach. The BBC employed dedicated Agile coaches 3 days a week for a year to ensure processes are established and sustained. Take your time to make the transition.

MP: Establish a culture of courage to make mistakes, learn and move on.

ML: Don’t be afraid to change your team.

LVW: Match the vendor, business and development team carefully. Ensure they are prepared to work together. Choose the right project: small, but with enough business priority to build credibility in the methodology.

ML: Don’t rush introduction of Agile.

Audience: How do you find a ScrumMaster?

ML: Try the Agile Professionals Network.

AN: Find a certified ScrumMaster (remember that Scrum is only one of many Agile methodologies)

The Audience Decides

During the interval the audience was invited to vote for one of five topics by affixing a sticker to a wall chart under the title of their choice. After Mark Leicester announced the winners, Mark Pascall explained to the audience how the exercise related to Agile methodologies.

MP: The audience is the business. The ‘project’ is to learn about Agile in government. The ‘development team’ is the panel. Time is fixed to 45 minutes. The budget is the panelists’ hourly rate. A traditional methodology would have us spending 35 minutes exploring what we wanted to learn about Agile and documenting the requirements. This would leave just 10 minutes for the doing. We’d agree to extend the project and meet again next Monday. However in the interim you, the audience, might have learned more about Agile. As a result the questions would change, and the documentation would have to be rewritten… With Agile the project is first broken down into story cards. In our case the story cards (i.e. the topics) are of equal difficulty, each taking 10 minutes each. The business chooses the highest priority stories and works down the list. After 45 minutes we draw a line, and the audience (i.e. the business) decides whether or not you’ve learned enough (i.e. the project is complete). Either way the business has completed the most important tasks first.

“Being Agile”

LVW: For us [at DoC] Agile is a way of thinking completely different from traditional methodologies. We take the task and split into functions. The vendor quotes per function, splitting work into 10 day time boxes with specific deliverables. Analysis for each 10 day cycle happens 3 days before the cycle begins. The business decides when it has enough functionality, or has run out of budget.

AN: DigitalNZ is a web application. There was no requirements documents or specification. We began with a list of features, turned into story cards. Each feature was turned into a wireframe. Users tested the wireframes. This was complete after one week. After two weeks back-end developers had converted the wireframes into a working example. Graphic design came later.

ML: Scrum: a process and a lifestyle. The process is straightforward. The lifestyle is harder. Everyone is a team member. The team plans its own project. The product owner (”the truth” or “the single wringable neck”) must be involved, guiding, helping the product to succeed. Each increment must be ‘potentially shippable’: put quality in the process all the way through. With Agile, scope is the only variable: you don’t compromise quality.

Audience: How does ‘potentially shippable’ work for typical projects made up of parts e.g. a database component and front-end interaction?

MP: Build a whole vertical slice at a time.

ML: After the first iteration of the BBC iPlayer we had a fixed web page simulating a download: the XML schema was designed but copied manually. Potentially shippable quality, but not saleable.

Audience: What is a good size for an Agile team?

AN: There were four teams working on DigitalNZ. One team of five at National Library, along with three separate vendors (two with five developers and one with two). A total of 15-20 people.

ML: Team theory suggest 7±2. There were 90 people working on the BBC iPlayer, divided up into 9-10 teams of varying size.

MP: Under 3 is too small, over 7 is too big.

Audience: With 90 people do you have parallel increments?

ML: Yes, synchronised sprints, interdependency mapping, Scrums of Scrums.

AN: Yahoo! has 200 Scrum teams running at once.

Agile Myths and Legends

ML: A list might include: lack of process; lack of documentation; lack of planning; teams do what they want; no testing; no quality control; projects never end; it’s only for simple software or the web; it’ll solve all your problems; it’s easy; lack of architecture. Taking ‘lack of planning’ as an example, a 6 month project involving 8 people on a one-month cycle included 424 hours of planning. Product demos (taking feedback) is planning. Taking ‘architecture’ as another example, enterprise architecture sets constraints. Bad Agile projects ignore these constraints.

AN: With a critical deadline for a deliverable, perhaps you can ignore enterprise constraints, but ensure you have the conversation first.

ML: Another Agile myth is that Agile teams don’t test. Testing is continuous on every level. Users see the deliverable every two weeks. Everyone can test. A “done statement” defines completion. Testing is built in all the way through and cannot be cut from the schedule.

Audience: Is a development/test/UAT/production environment appropriate for an Agile project?

ML: Yes, no different from other methodologies.

Audience: With a test team does Agile presuppose you need automated testing?

AN: With time pressures tools like automated testing help make projects more successful but are not a requirement.

Audience: What is the next-best alternative to Scrum?

MP: Scrum is a project management focus, XP is more about engineering.

ML: DSDM focuses more on governance. Waterfall is fine if it’s working for you. Choose an approach that works for you.

MP: All Agile methodologies share concepts like iterations etc. Some minor details differ.

Reconciling Agile with Government Procurement

AN: The RFP process wasn’t much different between Agile and non-Agile. In both cases we went to GETS. The RFP contains the requirements, including how you want to work. You evaluate the responses on this basis.

ML: How do you tell the good Agile vendor from the bad?

AN: You look for a track record and referees.

MP: As a vendor, I prefer an RFI first. RFPs ask for fixed timeline, fixed quality and fixed scope. With the unpredictability involved you’re asking the vendors to lie. Check a vendor’s willingness to take a share of the risk. The process should permit an agency to meet with the vendor to determine compatibility.

Audience: Few organisations are ready for the cultural shift that Agile brings. Agile brings transparency, highlighting dysfunction. Can you comment?

ML: Yes. Projects have succeeded at the ACC internally. There is reluctance to open up externally.

LVW: Organisations like the Agile sales pitch. However, organisations find it hard to open up.

AN: Once you’ve had success with Agile an organisation may start to talk about Agile without really doing it. There’s a whole ecosystem with Agile. Be careful not to pick out pieces.

ML: We chose Scrum to avoid mixing and matching. Be careful to procure Agile only when you’ve made the necessary changes internally.

AN: The organisation doesn’t have to change entirely.

ML: But it is important.

Audience: With a transitional project, where you want to use Scrum can it work where the rest of the organisation is traditional?

ML: Outputs from Agile can be translated into waterfall until other parts of the business can understand Agile reporting.

MP: Start with a small project first, perhaps it needs to be isolated in an un-Agile culture.

ML: Champions who can clearly articulate advantages to management help.

AN: If you have a fixed budget then you should be open to different outcomes.

Audience: Should the stakeholders then also not have a fixed idea of the outcome?

AN: Regardless of Agile, the role of governance should be to ensure appropriate process and monitoring and that outcomes are achieved. Wireframes, designs and the details of the project are not the business of governance.

Audience: Individuals and interactions come before processes, but some process are required and some are optional. How do you decide which Agile processes are appropriate for your environment?

ML: If you adopt Scrum, don’t change it. Don’t change elements of the process of delivery. However, engineering practices are about an organisation’s quality choices.

Audience: Without co-location can you do Scrum?

ML: Scrum doesn’t require co-location. Ideally you should co-locate, but be pragmatic and do your best. However, don’t have variable length stand-ups, or sprints.

Audience: Scrum has failure modes. Retrospectives, time boxing are considered hard and fast rules.

MP: Agile methodologies are high-level. Experience is important.

Thanks to the panel, the audience, GOVIS and the State Services Commission for hosting the forum.

Technorati Tags: agile, project management