What are the key considerations when choosing to adopt agile methods in a government context?
This whitepaper by Zen Ex Machina is aimed at government senior executives and managers. It examines the benefits of adoption of Scrum to both initiate new projects and rescued stalled ones within a government context.
In this whitepaper, Zen Ex Machina articulates:
- The ANAO’s factors in planning and approving projects as they relate to using Scrum.
- Why projects using Waterfall are more likely to fail than Agile projects.
- The benefits/outcomes can be realised through adopting Scrum.
- How Scrum aligns with Prince2 requirements.
- How Scrum’s key processes can increase transparency and reduce risk.
- Impediments to adopting Scrum.
Importantly, this whitepaper articulates how to get started with Scrum, including planning, governance and building requirements.
If you work in government, you can receive your copy of the whitepaper by completing the the following details:
As an Agile Coach and Chief Scrum Master responsible for 8 teams in a service delivery program, when things went wrong, I found it really tempting to get in and try to patch and fix things, direct the teams and make decisions for the Scrum Masters and Product Owners. In this type of role, I drew heavily upon my Program Manager background and found that when things were going wrong, I would fall back into directing teams what to do, sorting out issues for them and making decisions. I spend my life as a soccer mum outside work hours and now I found at work I had ultimately become the “Scrum Mum”.
At the recent Scrum Australia conference in Melbourne, I presented my lessons learnt on how to evolve from this Scrum Mum anti-pattern towards Agile coaching using a contingency based approach depending on the environment of the team/s. This approach helps guides the team through coaching and mentoring towards self organisation, empowerment and cultural change.
In this presentation I highlighted different scrum master maturity models and demonstrated how at an enterprise level, the Scrum Mum is an anti-pattern that isn’t sustainable or scaleable and whilst the team/s may be successful in the short term due to the heroic efforts of the Scrum Mum and a few good individuals, this anti- pattern will ultimately lead to stagnation by the teams
When making the decision to adopt an agile method like Scrum, invariably the question about existing roles arises. In particular, I find that the question of “where do my project managers fit in” is one asked most often.
Comparing Project Manager and Product Owner/Product Manager roles
While the Project Management and Product Owner roles are quite different in their focus, there are many similarities between them.
|Project Manager||Product Owner/
|Primary concern||Create “a temporary endeavour undertaken to create a unique product, service, or result” ||Create a framework built on the need for a product to “satisfy a want or a need” |
|Focus||When will it be delivered?||·Why are we delivering it and to whom?|
|Schedule driven||·Value driven|
|Plan creates the cost, schedule, estimates||Valued features drive estimates and then schedule|
|Try to predict what users need||Find out what users want and adapt to their evolving needs|
|Optimise utilisation of team members through phase based and sequential activities||Team self-organises to optimise its work Sprint to Sprint supported by a Scrum Master|
The function of good project management doesn’t go away
In Scrum, the gaps between a Project Manager and a Product Owner are filled by other roles.
Scrum projects eliminate the role of the project manager, but that doesn’t mean a team can get rid of the work and responsibilities of that role. Since self-organizing teams are at the core of Scrum, a great deal of the responsibility previously shouldered by the project manager is transferred to the Scrum team.
Without a project manager to assign tasks to individuals, team members assume the responsibility of selecting tasks themselves.
– Mike Cohn 
Whether you’re title is Project Manager or Program Manager the tasks you normally do day-to-day don’t go away when your organisation transitions to Scrum, but the focus of why the project is being done does change. Suddenly, delivery becomes about continuous delivery of value. For business, stakeholders and end-users this transformation provides better outcomes for everyone.
– – –
 Project Management Institute (2004). A Guide to the Project Management Body of Knowledge: PMBOK Guide. 3rd Edition. Newtown Square, Pennsylvania, Project Management Institute, p. 5.
 Kotler, P., Armstrong, G., Brown, L., and Adam, S. (2006) Marketing, 7th Ed. Pearson Education Australia/Prentice Hall.
 Cohn, M (2016) Which course is right for me? Online at: https://www.mountaingoatsoftware.com/training/roles/project-manager
“But isn’t that just doing mini-waterfalls?”, asked a recent attendee at one of our Agile Essentials courses.
Scrum requires the team do all the same type of activity they used to do in a Waterfall environment — analysis, design, build and testing their product. I showed the person the faster feedback cycles and the anti-patterns of waterfalling Sprints, but it still wasn’t enough for her.
What I needed was a side-by-side comparison that illustrated how each of these two methods handled different aspects of project management and delivery.
|Schedule driven||Value driven|
|Plan creates the cost, schedule, estimates||Valued features drive estimates|
|Development is phase based and sequential||Development is iterative and incremental|
|Focus is predictive||Focus is adaptive|
|Demonstrate progress by reporting on activity and stage gateways||Demonstrate progress by delivering valued features every two weeks|
|Product quality at the end after extensive test/fix activities||Quality is built in with upfront standards|
|Batches are large (frequently 100%)||Optimises smaller, economically sensible, batch sizes for speed of delivery of valued features|
|Critical learning applies on one major analyse-design-build-test loop||Leverages multiple concurrent learning loops|
|Process is tolerant of late learning||Work is organised for fast feedback|
|Handovers between analyse-design-build-test phases with knowledge stored in documents||Cross-functional team with knowledge of the product invested in the whole team through shared experiences|
For me, the main difference in comparing the two methods is:
- Scrum is value driven, where the plan is formed around the question “what is the most valuable item we can deliver today”
- Waterfall produces a plan from which costs, schedule and estimates are created
If I wanted to deliver value to clients and stakeholders, Scrum is the method I would choose.
In my previous post on Business Analysts as Specilaist Team members within a Scrum team noted, individual team members may have specialised skills and areas of focus, however accountability to deliver the Sprint work belongs to the Scrum development team as a whole.
The whole team is responsible – whether its support for testing built into the features or more people testing, it is not done until it is done. Testers within a Scrum development team help ensure that the right product is getting created and that the product is being made right.
The tester would support the Product Owner by:
- Writing/reviewing acceptance criteria for each product backlog item (user story)
- Write/review testing scenarios for the user stories
- Meeting with clients and business to establish what is needed, what they are wanting to achieve and how that may impact current systems, processes and users
- Creating test cases
- Automating test cases
- Executing acceptance criteria
- Smoke test UI/regressions (previous sprint).
- Exploratory testing to find bugs that no automated test will ever find
- Pairing on development to improve understanding of the code
Scrum teams are intentionally small. Most have a high ratio of developers to testers. Even though that can account for most of your testing time, successful teams have flexible members and testers should have a willingness to step outside their circle of comfort.
Before the Sprint begins, testers can help the team by validating the product (looking at the design and talking through the feature to make sure it makes sense), identifying product risks and system risks before they become issues and making sure that the feature will support test automation.
During implementation, testers make sure the tests are getting written from the start. The best way to do that is to meet with the developer and product owner before any work is done so that the tester can provide their input for what will be tested and also start working on BDD/TDD scenarios for testing to make sure all acceptance criteria are verified and reflect end user’s behaviour and interaction with the system.
Testers help the team to understand what the system state is at the beginning, the trigger for the test, and the criteria for the test with enough detail to know which business rules are being tested. Writing the code behind the tests, such as step definitions for “Given, When, Then” cucumber scenarios, will help the team get to ‘done’ but also help them understand what is happening when the step runs.
Pairing on development and testing strengthens both team members. With people crossing disciplines, they improve understanding of the product, the code, and what other stakeholders find important.
Whilst individuals within a Scrum development team may have specialist skills and areas of focus, accountability to deliver the Sprint work belongs to the Development team as a whole.
In Scrum, there is no business analyst role; instead the business analyst works as part of the development team and utilises their specialist skills to help their peers refine the product backlog. As a team member within the Scrum development team, business analysts play an important role: They usually act as the link between the business and IT, helping to discover the user needs and the solution to address them.
As backlog refinement is a team effort in Scrum, analysts working on the team take on additional responsibilities, for instance, working closely with the testers or the technical writer. As a business analyst on the team, they would be expected to pick up new skills, broaden their expertise, and be open to work in new areas.
The business analyst would support the Product Owner by:
- helping write/define product backlog items (user stories)
- researching the background for user stories
- meeting with clients and business to establish what is needed, what they are wanting to achieve and how that may impact current systems, processes and users
The business analyst would contribute to the development team by:
- hunting out buried documentation about the system being modified
- assisting with the preparation of wireframes
- writing and quality assuring test cases
- writing copy and getting sign off for it
- Completing design and functional documentation
Business analysts can be instrumental team members on an agile project as they can help shift the emphasis from writing about requirements to talking about them. On traditional projects, analysts tend to be intermediaries between team members, but on a Scrum project, the analyst as a development team member is the facilitator of the team and product owner discussion.
As a team member, an analyst’s first priority is to achieve the goals of the current sprint. This is in contrast to looking ahead and forecasting as he or she is used to. An analyst on a Scrum team will assist in testing and answering questions (or tracking down answers) about features, and participating fully in all regular sprint meetings.
The Team chooses how best to accomplish their work, rather than being directed by others outside the team. Business Analysts are often included in Scrum cross functional teams as these teams will have all the competencies needed to accomplish the work without depending on others not part of the team in order to optimise flexibility, creativity and productivity.
Whilst Scrum Teams are self-organising and empowered, at scale, the Master Scrum Master (i.e. Chief Scrum Master) provides coordination and alignment to support teams’ delivery. In Government, this role is crucial to ensure that alignment not only across the teams, it also needs to be in line with program and portfolio strategic intentions for that government agency and in line with wider whole of government initiatives as such as the digital transformation to provide simpler, clearer and faster public services.
The Master Scrum Master role is a recent concept in government. I have been a Master Scrum Master on a number of transformation projects across large government agencies and have had to explore some of the unique challenges of scaling in government including:
• Scaling from a team of eight backend developers to eight teams of 100 people across the three platforms with six external vendors on separate government contracts/SLAs
• Introducing scrum roles into a government structure that are traditionally based on levels of hierarchy with associated HR and PMO reporting
• Dealing with expectations that Agile teams will still meet work schedules driven by internal and external Waterfall teams outside of the sprint planning process
• Working within Gateway signoff processes driven by upfront detailed level design
So, how can we address these challenges?
The Master Scrum Master plays a vital role in addressing challenges of scaling in government in three critical ways:
1. Coordination of Integration to meet shared goals: As a contractor or vendor into government agencies, often it is hard to navigate the landscape and projects and programs of work can become caught up in “noise”. This is where the Master Scrum Master as owner of the process and integration can help as their focus on tactical delivery of the implementation, can help those new to government or the agency, to understand the integration points and then work with these vendors and contractors to gain a better understanding of lead times, dependencies, capabilities and capacities external to the team, to work together to support the small batch sized product increments.
2. Facilitating collaboration: As a Master Scrum Master of a government service delivery teams, I found patterns such as “Scrum of Scrums” and its counterpart the “Product Owners Council” were useful in ensuring collaboration and integration coordination across the teams and facilitate an effective escalation process. For example, these patterns were key to cementing the Scrum Masters and Product Owners relationship to put the focus on business value and working together towards achieving the goals for the Product Increment rather than working in team silos or formal sections/divisions.
3. Focus on Continuous Improvement: The Master Scrum Master requires a very different mindset to a traditional program manager as the Master Scrum Master needs to be a servant leader who coordinates all the team’s efforts to facilitate program level processes and execution, escalate impediments, manage risk, and drive program-level continuous improvement through Inspect/Adapt feedback loops.
I’ve been working with a number of government agencies as they look to Agile methodologies to help them implement their digital transformation strategies. Governance is key to these implementations as the governance framework needs to describe the functional lines of responsibility and accountability.
It is important to recognise that, because a Release train is a construct, the governance framework is explicitly designed to support the outcomes of that construct. As such, it is independent of the organisational hierarchy and therefore must also contextualise these new Agile roles and the responsibilities of all those involved in the Release Train.
Team members need to know what decisions are within their domain and which need to be escalated. In our Release Trains, we want to optimise for fast, decentralised decision making as any decision that must be escalated to a higher level introduces delay (waste) and inhibits flow however within the government agencies we also needed to acknowledge that some decisions are strategic, critical and far reaching across the enterprise and therefore need to be centralised.
Decentralised Decision Making
Decentralised decisions tend to be those decisions that need to happen frequently, are often time-critical (e.g. acceptance of a Sprint Goal at the end of Sprint Planning or acceptance of User Stories as “Done” in Sprint Review), and do not have significant economies of scale. The responsibility for these team-level decisions typically resides with the Product Owner (for decisions regarding the team’s work, scope and quality) or with the Scrum Master (for decisions regarding execution of work aligned with the Scrum process).
Centralised Decision Making
Strategic decisions, those that are infrequent, long lasting and have significant economies of scale, are made at a higher level of management. In the Release Train, the responsibility for these decisions typically resides with the Product Manager (for decisions regarding the scope and boundary of the work for the Agile Release Train and its quality) and Release Train Engineer (for decisions regarding execution of work across the Agile Release Train), with the ultimate level of accountability residing with the Program Director and then the Portfolio Executive.
- A single point of responsibility for the success of the Train – This provides clarity of leadership, management and timeliness of decision-making during the product increment.
- Service delivery is focused on business value – Placing the delivery of business value at the heart of the product increment.
- Separation of delivery ownership and product ownership – To focus on delivery of business outcomes and prevent decision-making forums, such as the Release Management Group (RMG), from becoming just a status report meeting focused on technical issues.
- Separation of project governance and organisational structures – To reduce the number of project decision layers, since the project decision path does not follow the organisational line of command.
To achieve its goals, this Release Train is comprised of the following three structures:
|Directing||Release Management Group||Accountable for the direction and management of the Train within the constraints set out by the Program’s mandate.|
|Managing||Release Train Engineer and Product Manager||Responsible for the day-to-day management of the Train within the constraints set out by the RMG, typically encompassing time, cost, quality, scope, risk and benefits.|
|Delivering||Scrum Team/s||Responsible for delivering the products to an appropriate quality within a specified timescale and cost.
Responsible for planning and producing products.
When scaling Agile at an enterprise level, coordination and alignment across multiple teams is challenging. Whilst Agile teams are self-organising and empowered, someone needs to steer the train to keep it on the tracks to facilitate program level processes and execution, escalate impediments, manage issues, mitigate risks, and drive continuous improvement.
This is my presentation form the LAST (Lean Agile System Thinking) Conference in Melbourne where I shared my experiences of being a Release Train Engineer (RTE) on a digital transformation project across a large government enterprise and explored the challenges and lessons learnt. In particular, I focused on the Scrum of Scrums and how the RTE is essentially the Master Scrum Master of the Release Train and demonstrated how to ensure Scrum Masters are working together towards achieving the goals for the Train’s Product Increment.
Who should be your (Scrum) Product Owner?
I’ve had a number of these conversations recently with small, boutique software development agencies and even design companies with UX expertise. Typically, the conversation goes something like this:
- The business stakeholder or client is the owner of the product, so they should be the Product Owner
- We’ve got a Product Manager, so they should be the Product Owner?
- The Development Team is supposed to self-manage, so they can just share the Product Owner role
- Decisions about the scope and what the Development Team should work on is far too important for just one person, so we should have a committee make the decision.
The Scrum Guide (2013) is pretty specific about what the role of Product Owner is and what it does:
“The Product Owner is responsible for maximizing the value of the product and the work of the
Development Team. How this is done may vary widely across organizations, Scrum Teams, and
“The Product Owner is the sole person responsible for managing the Product Backlog… [the role requires] is one person, not a committee. The Product Owner may represent the desires of a committee in the Product Backlog, but those wanting to change a Product Backlog item’s priority must address the Product Owner .”
The implications are significant. Ultimately, it means managing the Product Backlog in such a way so as to achieve balance between optimisation at the delivery (Team) level and the vision of the product.
Different Product Owners
Client as Product Owner
In my experience, when push comes to shove, and the end of funds is near or the project is running out of time, a client won’t throw Scrum aside and push for more features. I’ve often heard them say, “you’re agile, so that means you just have to learn to adapt”.
- Advantage: Clear view of the vision
- Disadvantage: Doesn’t care about your Scrum process. Won’t truly understand balancing technical features and technical debt over features and functionality.
- Result: Features! Features! Features!
Business Analyst as Product Owner
A senior B.A. is a whiz at managing scope and interpreting clients’ needs. Unfortunately, they are often too low down the pecking order to be given responsibility for delivering an outcome.
- Advantage: Product Backlog Lumberjack!
- Disadvantage: Often not empowered.
- Result: Wait until I go ask someone and then I can tell you the decision.
Scrum doesn’t say where that vision needs to come from, though. It could come from the client, a Product Manager, a Program Manager or the executive leadership team in a large enterprise.
Project Manager as Product Owner
A Project Manager’s focus is not too dissimilar from a Product Owner’s. He maximises the ROI of a project, manage stakeholders and communication, its finances and changes to its scope.
- Advantage: Often have the power to make decisions
- Disadvantage: May not be technical enough to create balance between the technical needs of a product and the needs of the requirements of clients. May tend to blur the line between the “what” and the “how” and micromanage the Development Team to deliver an outcome. Scrum key meetings may fall by the wayside as a result to make more time to code.
- Result: You should be coding not being in meetings so much!
UX Designer/Architect as the Product Owners
I came from a UX background. It influences my Scrum practice quite heavily in terms of optimising the Product Backlog so that it delivers value to end-users. As such, when I perform the role of Product Owner, I’m always conscious of building a product with a user-centred vision, and balancing good UX with technical features and the “nice to haves” that clients ask for.
- Advantage: User centred perspective.
- Disadvantage: Like a BA, may be too low level to be empowered. May “gold plate” the UI and not care how long it takes the Development Team to deliver.
- Result: This would be good for users, but the Project/Product Manager/Client has overridden me and wants more features.
Product Managers as Product Owners
I find that Product Managers, like Program Managers, are just too high level and too concerned with larger things to get their hands dirty at a local, Development Team level. Their responsibilities often mean there is little time to spend in key meetings relating User Story level details to a Development Team.
- Advantage: Have the vision and are empowered to achieve it
- Disadvantage: May not know how to translate that vision into items for the Product Backlog. May not have the time to dedicate to collaborating with the Development Team to deliver the outcome because they are spending more of their time with senior stakeholders.
- Result: Here is the big picture, now just do it!
Line Managers as Product Owners
In an organisational context, Managers can be quite effective Product Owners. They already have experience telling their teams what to do. Empowerment is an important key aspect of being a good Product Owner, and in environments with an existing hierarchical structure, this is one of my favourite patterns.
- Advantage: They are empowered to make decisions.
- Disadvantage: May be too used to micromanaging their team.
- Result: Agile aside, what do you need to deliver?
So, who could be the Product Owner
My favourite Product Owners are senior BAs with enough referent power on the client side to help shepherd them to good decisions regarding the vision, enough expertise to manage the scope of the Product Backlog themselves to deliver a high-value product, and with experience with Scrum so that they respect the process.
Ultimately, there is no best person for the role other than someone who is empowered to make decisions, balance the short and longer term vision of delivery, and has experience with Scrum sufficient not to throw it away when things get tough.
Hierarchy and Product Owners
But sometimes one Product Owner is not enough. And when there is a whole program of work and many teams, what happens to the Product Owner? Can he make the same decisions independent of other Product Owners?
Program Management and Product Owners
In SAFe, the high-level vision comes from the Program Manager who collaborates with a number of Team-level Product Owners.
This model is quite logical in an enterprise with an existing management hierarchy, particularly where the vision of the product is made at a level higher than the Team.
Chief Product Owners and Local Product Owners
In LeSS, that superordinate view of the direction of the product as a whole comes from a (Chief) Product Owner who then engages Area Product Owners.
In more decentralised Team environments, an Area Product Owner (APO) specialises in a customer-centric area and acts as Product Owner in relation to one or more teams for that area. An Area Product Owner does the same work as a Product Owner but with a more limited, yet still customer-centric, perspective.
Understanding the governance between the strategic, tactical and delivery aspects of a program of work is key to understanding who is the best person to be the Product Owner. Ultimately, it can be anyone, so long as “the entire organization must respect his or her decisions. The Product Owner’s decisions are visible in the content and ordering of the Product Backlog ”.
The Development’s Team’s focus is often to build the product right, making sure it works and the code is of good quality. The Product Manager’s focus is on achieving the vision, but not necessarily on the detail to make it work. The clients’ focus is all about features features features!
Ultimately, the role of the Scrum’s Product Owner is the focus point of all these conversations. It has to be one person, not many, otherwise decisions may take too long, the focus might lean too much toward the higher (product management) or lower perspective (team’s delivery), or the governance watered down so that no one is the “go to gaol” person when things go poorly.
Who ever you decide on making your Product Owner, ensure they are:
- Empowered to make decisions.
- Can be pragmatic in creating a balanced outcome between the vision and delivery.
- Know and respect Scrum, so they make it all work through collaboration with the Development Team, the Scrum Master and the customer (who ever that might be).
 Schwaber, K. & Sutherland, J. 2013. The Scrum Guide™ The Definitive Guide to Scrum: The Rules of the Game. p5.
The team I was working with had a “great problem” – more work than we could deliver. However this success brought mixed blessings as the strain of growing so quickly was starting to show. We had a backlog of work, process issues, resourcing and quality issues and a lot of knowledge residing with one or two of the original start-up team who were now single points of failure.
The innovative, “can do” attitude of the start-up company was still there but we were having growing pains. We knew that what we were experiencing in our market (Australia) would eventually be seen in our USA market if we didn’t find a solution to our growing pains.
We looked to Lean and Agile as a multidisciplinary approach to achieving an effective product strategy, development and delivery capability that could be scaled to the whole organization.
As a result of our optimization for lean delivery we had an empowered team, improved quality, know velocity and capacity and an organization that is learning to deal with changes and project complexity. In the short term we achieved greater transparency of projects, increased collaboration and knowledge sharing across teams and the team now had a better understanding of the “what” and definition of done and a process to handle unknowns and respond to changes. Projects are now managed in programs of work and we now have a product and service delivery roadmap to ensure enterprise aims for product development are linked to implementation strategy and delivery.
Adopting Scrum helped to improve delivery capability and enhance product co-development partnerships and decreased the risk of failure to deliver the Program of work in a timely fashion. This was done through employing a number of methods and techniques, including working collaboratively with stakeholders and end-users, simplifying governance and increasing accountability and employing iterative methods, rather than a ‘big bang’ approach. Encouraging frequent inspection and adaption, teamwork and collaboration between the project’s members ensure that knowledge and skills were shared within and across teams as the process scaled from project delivery to program and portfolio level delivery.
The artefacts and learnings from the Australian pilot were adopted by the global PMO and we are now helping to roll out this framework to other regions. Once clients become a program of work, we had a scalable and proven method to ensure we continue to deliver a quality outcome.
An important part of our aim was to optimize for lean delivery. Scrum’s processes are explicitly engineered to identify and decrease waste and the organization was able to ensure that through the PRINCE2 governance framework the project was empowered to be responsive to changes in its environment so that delivery aligns with changing user, stakeholder and organizational needs. This ensured that all of the Program’s features delivered were useful and of high-value.