Feedback Loops
Let’s take a look at a few of the prescriptive components of Scrum, the most popular agile framework:
- Iterations and frequent releases
- Backlog grooming
- Standups
- Functionality demos
- Retrospectives
A team that implements these components is arguably doing “most of Scrum” (I acknowledge that there are many other important aspects to Scrum as well). That said...what do these items have in common? You guessed it – they’re all feedback loops! So what's a feedback loop? This InfoQ article says that “a process has a feedback loop when the results of running the process are allowed to influence how the process itself works in the future”. We’ll take this as a reasonable definition and run with it.
Agile is in many ways a reaction against traditional waterfall style methodologies, which draw at least some inspiration from established construction practices such as engineering blueprints. However, as we've all witnessed, the only possible blueprint for a software system is a working codebase…our very end goal! Agile embraces this by providing a set of fault tolerant principles which embrace change during construction – through feedback loops.
Agility provides many benefits which allow a project to absorb and react to change. These changes come in many forms – changes to team composition, incomplete requirements, organizational shifts, market conditions, emerging technologies, and good old-fashioned human indecision. Feedback loops represent an infinitude of decision points at which an individual, team, or organization can refine direction based upon the best available current information. The interesting part of this process is in how we implement different kinds of feedback loops.
Characterizing Feedback Loops
In general, feedback loops can be characterized by the parts of the system which they target (purpose), and by their length. For example, standups provide team-level feedback on a daily basis. Retrospectives provide process-level feedback on a per-iteration basis. Demos provide end-user feedback on a per-story or per-release basis. Unit tests provide code-level feedback on a per-build basis. Feedback loops can vary greatly in length and purpose.
Loops such as those listed above are interesting in that the scope of the loop (how much of the process it is allowed to affect) is often proportional to its length. Sweeping process changes are more likely to be the result of retrospectives than code reviews. Tiny code tweaks are more likely to come from pair programming than from demos, but major wireframe revisions are more like the result of the latter. This makes a lot of sense; making many sweeping process changes in rapid succession is likely to be unsustainable, causing more disruption than benefit. On the other hand, commenting your code only once a year may not be a good idea either.
So what balance of feedback loop type-and-size is most effective? Scrum doesn't tell us! That's because embracing agile (and not just following rote instructions) means learning to adjust the knobs to match your team and tempo. I like to think that retrospectives, one of the explicitly provided features, are a great opportunity to focus specifically on examining the existing balance of feedback loops and trimming or adding as necessary.
How Do We Use Feedback?
The issue of how to effectively leverage feedback loops is as important as having them in place. If individuals and teams are not empowered to take actions as a result of the feedback they encounter, then that feedback simply becomes noise, aggravating and incessant (just like microphone feedback). Yet, it is reasonable to want to limit the scope of change resulting from a given process. To that end, it seems intuitive that the scope of change resulting from feedback should be proportional to the size of the loop, and limited to the domain at which the feedback is targeted. Use standups to affect the course of the team’s actions over the next day; use retrospectives to affect process over the course of the next iteration; and use project debriefs to affect the course of the organization’s next project. Empower your teams to affect how they work within the scope of their responsibility, and give them frequent opportunity to do so, and they will gravitate towards the so-called “path of least resistance”.
Of course, if you want the path of least resistance to also be the path of decent quality, within-budget cost, and business value, you’ll need engagement from all sides of the project…you never know when we developers will decide we can deliver faster if everyone just uses a Python REPL as a front end. And there’s the rub – for feedback loops to work at full effectiveness, every stakeholder in that process must be available as often as the feedback loop closes. You don't need the product owner during code reviews, but get them involved in demos and retrospectives. As for how to drive involvement…well, I leave that as an exercise to the reader.
Some Additional Loop Taxonomy
Another interesting dichotomy is that of fixed-interval loops vs. variable-interval loops. We can roughly characterize the former as those which close on a pre-determined interval, and the latter as those which close as-often-as-necessary. For example, Scrum is rife with fixed-interval feedback loops, such as standups and iterations. On the other hand, many common construction practices are variable-interval loops. Pair programming gives an opportunity for frequent feedback as often as necessary, although you may occasionally spend several minutes in silence. Integration tests provide structural feedback whenever necessary (i.e. when something changes), but won't generally need to run without reason. This is an interesting distinction, because the fixed-interval loops provide cadence, but may not always be necessary, whereas the variable-interval loops tend to supply less discipline but may provide more efficient value. I get the sense that nascent moments towards lean, continuous integration, and other similar practices are in some sense a push towards the trimming of fixed-interval loops (which can be portrayed as process overhead). Personally, I find a certain beauty in a process that exerts itself exactly as often as it's needed, but it definitely asks a lot from the team's individuals.
One final critical concept I’d like to explore is that of on-demand feedback loops (get ready to file this one under "ivory tower agile", although I promise I've actually seen it happen). The contrast with variable-interval loops as described above is that team members can pull an on-demand feedback loop whenever they want, rather than it being pushed to the team members by the process. I have been fortunate to work within a project where there were several regular facilities in place for reaching the product owner, testers, business stakeholders, and end users on an as-needed basis. As with Scrum's prescribed loops, these manifest in various forms and sizes ranging from asynchronous communication (email, chat clients) to longer meetings (weekly placeholder meetings with flexible agendas, business-centric retrospectives, even office hours). I think that in some ways, this is the ideal realization of agile – allowing team members to dip into the communication stream, get the time and type of communication they need, receive cogent feedback, and get back to work, confident and empowered by early buy-in and up-to-date information. This sort of culture is rare and difficult to achieve, but the benefits are real and tangible. In this environment, you can feel the manifesto at work – individuals and interactions, customer collaboration, and responding to change. Working software…well, I leave that as an exercise to the reader.
Conclusion
Almost everything that you do in a software development project can be framed in the context of feedback loops, whether walking over to the senior developer's cube for advice (on-demand small-scale code-oriented feedback), needing timely answers to an email, or being able to get the right bodies in the room for that quarterly meeting. Feedback loops are just another way of thinking about our constituent processes so that we can organize these processes to address our pain – both in domain and in scope – in a fashion that keeps agile principles ever at the forefront. The short of the long? Maximize your feedback loops, from the continuous (open email channels, pair programming) to the frequent (standups, demos) to the over-arching (retrospectives, backlog grooming, forecasting). Use retrospectives to identify pain points, and implement (or remove) feedback loops to address that pain. Empower us with responsibility and information, and we will steer towards the light. And never forget the unheralded fifth pinnacle of agile development – Caffeine over Sleep. #freecoffee
