Demystifying Agile

A colleague and I had an interesting discussion via Twitter a few days ago after someone tweeted a link to an article on agile billing. Having a company who has aligned its contracts and billing cycles with our agile methods, I found it nice to find someone else had been thinking the same things as we’ve done for some 18 months now. But the conversation that followed led to a few very interesting questions, some of which  I wanted to address in a space greater than 140 characters.

Agile is just a developer tool. Agile only works with developers – that’s what its made for. Anything else is a hack.

Scrum didn’t start its life in software development. The modern Scrum process was based on an article in the New New Product Development Game in the 1986 Harvard Business Review. In this article, Hirotaka Takeuchi and Ikujiro Nonaka looked at high performing companies like Fuji-Xerox, Honda and Canon and how they’ve adapted their old processes to innovate and create new products:

“This [agile] approach is essential for companies seeking to develop new products quickly and flexibly. The shift from a linear to an integrated approach encourages trial and error and challenges the status quo. It stimulates new kinds of learning and thinking within the organization at different levels and functions” – Takeuchi & Nonaka, 1986.

The article describes processes and approaches used to produce a Honda City, photocopiers, and in the example of Hewlett Packard, software. The processes that became Scrum are also product focussed, as are many agile methods, and works well for any product-based work, including financial products, Internet products, and medical products. Most of these processes pre-date the modern use of agile methods to develop software. The difference, therefore, in the use of Scrum, is a focus on creating smaller, high-value products in short, fixed time intervals, rather than a lengthy sets of tasks (research, analysis and design before building and testing) that only produce a product months after the initial research.

Agile is … not possible for consulting

So long as the outcome of consulting work is a product then, certainly in my experience, Scrum is very effective in creating it. The challenge for me, though, has always been breaking down the large, single consulting report that would often span tens if not hundreds of pages, into smaller products that can be delivered in succession each Sprint.

We recently did a business case for a large government department over a 4-week period. We chose 2 Sprints each of 2 weeks in length for the project with the first product being a skinny solution. This took the form of a document outline with the bare essentials in terms of background, project vision and high-level solution, and an outline of the costs involved. At the beginning of the Sprint we met with the client to talk about the document’s acceptance criteria and then worked to deliver this by the end of the two-week period. At the end of the Sprint, we walked the client through this document in preparation for planning for the next Sprint. The product was certainly of high-quality, and could certainly stand on its own, but there was more that the client wanted to see.

In planning for Sprint 2, we met with the client to discuss what new elements they wanted, where they wanted more ‘fat’ in the document, and the acceptance criteria for these elements. Then we set to work.

At the end of Sprint 2, again, we walked the client through the final draft based on the acceptance criteria they had agreed to. There were, of course, a few minor things to add to this final draft, but our contract noted that this tidy-up work is at time & materials. This clean-up is about the only deviation we use from our Scrum processes because there’s often not enough in this effort to comprise a whole Sprint by itself. Alternatively, if the client did want more though, then they could pay for an additional Sprint.

You can’t … do agile if the contract is short term (few weeks)

Most of the strategy work that I do at Zen Ex Machina is very short term — sometimes it’s only a few weeks in length. Sometimes this work is days of effort back-to-back and others are those same few days spread over a few months. For the former, particularly where the team is comprised of 1 or 2 consultants, I’ve found Scrum doesn’t really work very well, so in these instances I tend to use Kanban.

A personal Kanban board I used to create a concept of operations for an EDRMS

I list every product (not task) that needs doing, the order in which the client wants to see it (with acceptance criteria/definition of done), and then I define the tasks that will be required to deliver the work. This is really Sprint Planning, but rather than having a team of 7 +/- 2, it’s often just me. I then use the board to track the flow of those products through their various stages, even attaching notes as I go that summarise its journey from inception through to sign-off and delivery.

I’ve found personal Kanban techniques to work very well as an adaptation and extension of my normal Scrum processes. I still like the use formal planning, grooming and demo/review processes within my Kanban as I like the formality of up-front planning and formal points in time to engage with the client. I find Kanban very useful to keep work-in-progress visible to myself and anyone who might pass by (including the client if I am working on-site). It also serves as a point of discussion and collaboration about work because I can clearly articulate what I’m doing and ask for clarification about whether something else is more important to do instead because, at the end of the day, there is a limit to how much I can do at once. I can then stop doing one product in order to work on something else of greater importance/priority.

How can you work on product that you have no idea what it will be or if its viable?

Scrum starts with a Product Backlog where the top 20% of the products are very well known. By the time the first Sprint starts then the team know exactly what is being asked for. But how is the Product Backlog created? Scrum doesn’t care. Sometimes there’s a business case. Sometimes there’s a conversation with the client. Sometimes there’s a conversation with end-users and business stakeholders. Insofar as agile methods are concerned, this process lies outside of the first Sprint.

Some people like, though, to do a Sprint Zero to create the Product Backlog. Sprint Zero is more of a concept than a process in which the scope is articulated and the top 20% of Products are defined in sufficient detail to start work. User’s needs are identified, sometimes some user research is undertaken, and the Backlog produced.

If I’m lucky enough to kick-start a project, I’ll do Sprint Zer0 in four Sprints each of two weeks as the project’s initiation phase, but again, I use Scrum. The process works a little like this:

  • Sprints 1-2: I create the business case within 4 weeks (i.e.: in two Sprints. This creates the scope and defines things as Epics. Sometimes the business case will also contain an Experience Vision through engaging users and stakeholders by conducting a workshop using design studio methodology or initial user research. With modern web analytics available to most of the clients I work with it’s pretty easy to understand wants and needs and infer behaviour. Couple this with freely available research from the likes of Forrester (they tend to blog about most of the important stats) and you’ve got a great starting point.
  • Sprint 3: Do project initiation and turn the Epics into User Stories in the next Sprint.
  • Sprint 4: Engage stakeholders and end-users in the next Sprint to rank the Backlog and create a 20/20/60 gradation in the User Stories. I typically create “skinny Personas” as a product in this Sprint which contain just enough information to help make design decisions when the project gets going.
A typical Sprint Zero process
A typical Sprint Zero process

This process leads to the first ‘real Sprint’ for the project and its team. At this time, the products are reasonably well defined. The first activity, though, in each Sprint is Sprint Planning in which the team spend the whole day with the Product Owner to help define what is being asked for in each User Story. If the team still have no idea what’s being asked for, then the team can’t commit to delivering that product, and it returns to the Product Backlog for the Product Owner to work a bit more on (including with the Team during Backlog Grooming) for next Sprint.

Do you include research in … Sprints? What if there is no product at [the end of a Sprint]?

In the “old days”, when UX was called “information architecture” and HCI, I used to do a lot of user research. Sometimes it would take months. In one project, there was 6 months of user research, surveys and phone polls. Today, I don’t find I’m doing these sorts of activities as much. Paul Bryan has reflected on this trend in an article titled “Are Personas Still Relevant to UX Strategy?” He notes that there is a lot of value already had in existing knowledge embedded in web analytics and I agree with this notion. I’ve not known of a project I’ve done in the last few years where most of what is known about end-users hasn’t come from existing trusted sources.

Overall, though, in an agile project, research is just a task that occurs in a Sprint to underpin the design of products that need to be produced at the end of that Sprint. Developers do research as well in order to understand the architecture or best approaches to take for their code. Focussing on the here and now, with research linked explicitly to products for the current Sprint, keeps everyone from wasting time with doing tasks that doesn’t add any value to the end-product.

“UX work is mostly about discovery … discovery without delivery isn’t very valuable” – Jeff Patton, 2008.

What if research results in the product. Can a research outcome be a product?

I sometimes do research, though, as a product in the form of a document or report. In a recent UX engagement, we had a number of small products or deliverables, to create. We used Scrum processes and, in collaboration with the client, created an order in which each small product was created. The products were:

  • Sprint 1 — Personas.
  • Sprint 2 — Core wireframes.
  • Sprint 3 — Ancillary wireframes and a final document that related the rationale for the UX design.

In Sprint 2, we also allowed for some rework of the Personas, and included a buffer of 1 days effort to take care of this should it arise. We also did this for Sprint 3 in case we needed to rework the Personas again and do any iterations of the core wireframes. Not much rework was required, though, because during Sprint Planning with the client and his team, we made sure we got their acceptance criteria and worked exclusively to that. At the end of each Sprint we simply showed them we had delivered what they had asked for in accordance with that acceptance criteria.


Much of my working life has been spent with my boss watching what I’m doing because being (seen to be) busy was of high importance. I’ve also seen some managers micromanage others and tell staff what tasks to do rather than let them work out the best way to deliver an outcome. Traditional project work is similar with the focus tending to be the project manager putting what people are doing in to a big schedule. I’d rather not work in this way.  I’d rather outline what I’m producing and produce it in small, bite-sized pieces, in regular, fixed intervals. It’s also how I want to run my teams. These are some of the important reasons why agile methods and their underlying simple philosophies are so attractive to me — autonomy, self-management, product (outcomes) focused. Today, I wouldn’t work any other way.