On scaling engineering teams; a theory of how effective teams leverage autonomy, strategy and change management to grow

On scaling engineering teams; a theory of how effective teams leverage autonomy, strategy and change management to grow

Tags
ManagementCulture
Published
April 23, 2024

Context

At one point in my career as an Engineering Manager, I went from managing a small team of engineers, all within the same discipline, to managing twice as many engineers across several disciplines - accompanied by an expanded team mandate. Here are some reflections on how we were able to produce a productive and successful team.

Disclaimer: As always, all views expressed herein are my own and not meant to be representative of or endorsed by any employers past or present

Evolving in leadership

In order for this new team to be successful with its expanded roster and scope, I believed that I had to be intentional about adapting my role.

Challenges

Some of the challenges of the new arrangement were that

  • Whereas previously I had a background in my team’s primary engineering discipline, I didn’t have as much background in the respective domains of the new engineers.
  • Even if I had the domain expertise, there isn’t enough time in the day to be a blocking contributor and/or reviewer on work output. Not that I was before but now that simply wasn’t an option at all.
  • Along with the new engineering disciplines, there were entirely new problem spaces to learn about and support.

From those challenges, I identified the following guiding principles

Principles

Be intentional about what you put down and what you pick up

Operating in the old way doesn’t scale and it is much better to decide what to stop doing, do later, or delegate than to have things fall through the cracks. So what I decided was to

  • Provide the same or more
    • Expectations around ways of working, especially communication and performance
    • Clear priorities
    • Guiding questions around stated intentions (e.g. factors and alternatives considered, stakeholders consulted, etc)
    • Guidance on challenges and ambiguity
  • Provide same or less
    • Technical review (initially)
      • Existing folks know at least as much as I do
      • Folks who were new to the team and already experts in their fields know more about their craft than I do; I'll be leaning on and learning from them
      • Frame: tech review is Priority 1-2, people are Priority 0 (0 being higher than 1)
  • Ask for
    • Clear communication of intentions, sequencing, timing, and needs
    • Flagging of risks and concerns
  • In exchange, offer
    • Ownership over work, discretion on how to solve problems
    • Autonomy within the bounds of established ways of working
    • License to influence ways of working

Foster ownership and autonomy

Fostering an environment in which engineers and team leads communicate in ways imbued with ownership and rich in context allows people to take ownership of entire problem spaces. This has the benefits of

  • Allowing the experts to do what they do best without imposing myself as a bottleneck
  • Cultivate meaningful growth for the engineers as they are empowered to find the best solutions with the right context as guidance

Examples of ownership-imbued language in day-to-day operations might looks like

  • "Based on {rationale/discovery/outcome}, I intend to do X this week"
  • “I think risk Y could impact feasibility/timelines of our plan. I intend to investigate/I could use some help in investigating while I focus on Z.”
  • "I'm semi-blocked on team Y's input on my proposal and I haven't heard back yet. I intend to 1. Keep operating under the assumption that their suggestions won't be major 2. Reach out again on (day). If communication continues to be slow, it may put the delivery estimate at risk, so I'll let you know.”’

This may sound familiar to readers of Turn This Ship Around by David Marquet. I found it quite fortuitous that I ended up reading that book around the time that I in the process of drafting these principles, so it is certainly an influence and I’d recommend reading it.

Ownership in practice: It's a muscle 💪🏾

This style of communication is a muscle. It takes intentional effort to break out of the habit of immediately providing solutions or suggestions and, instead, ask directed questions that move the game plan forward.

  • Given what you know, how do you / would you plan to move forward?
  • From there, based on the response, you may branch into
    • What information/context is missing to determine what action is best?
    • What do you need (from me or otherwise) to carry out that plan?
    • Provide feedback on the plan, especially factors you considered that they may not have and vise versa and how weight each factor.
      • This process trains each other's decision-making models, giving each party new, updated, or reinforced perspective.

This has some similarities to the GROW model in coaching, which stands for Goal → Reality → Options/Obstacles → Way forward.

Ownership in practice: example: prioritization vs sequencing

I mentioned that engineers and the team’s leads (i.e. EMs, PMs, etc) need to communicate in this way because it doesn't work as a one-way street. It's a muscle that needs to be exercised by the whole team.

For example, let’s say we have Project A, which has a deadline of X weeks in the future and is higher priority than Project B, which has a flexible delivery window but is already in progress. Given that context, depending on how much longer it may take to complete Project B and the estimates for completing Project A, it may make sense to wrap up B before starting A (high confidence in completing both means there’s low benefit to context switching). It may also make sense to shelve B to start A sooner (chalk it up to suboptimal planning, but there’s risk of not getting A out on time, pay the context switching cost).

I’d almost certainly have an opinion on how to approach that situation. But the muscle to develop here, as a leader invested in empowering contributors, is to provide the context and guidance to the engineers involved to let them arrive at the same or better conclusion, using their hands on context.

💡
Share, and encourage your team to ask for, the information that you are factoring into your decisions and suggestiosn. And reciprocate - encourage them to identify the variables they are considering.

The end result is that they should be able to articulate a path forward that lets them own the sequencing while you’ve retained accountability for the priorities.

The default temptation is be to dictate, as a capital-L Leader, which path to take. But the style of leadership I aim to practice is one in which those being led are themselves leaders because they understand the goals and the constraints and figure out the path to achieving them.

Meaningfully integrate members and mandates

I’ve found group development lifecycles to be a helpful guide for thinking about what team’s need at their various states of (non-linear) maturity. In this case, given the magnitude of the changes, I operated on the premise that we were a Forming/Stage 1 team, meaning the corresponding actions would be to

  • Host a team kickoff, including mission, members, initial ways of working, and core values
  • Welcome new perspectives on existing ways of working - any process that doesn’t serve its intended purpose in the new environment is fair game for iteration.
  • Build relationships and working agreements with new stakeholders
  • Work with engineers to define tech strategy focused on the first 6 months

Useful framework: Good Strategy

My lasting takeaway from the book, Good Strategy Bad Strategy, is a definition of “strategy” that serves as a useful mental framework for problem solving.

A good strategy is a cohesive set of

  • Feasible goals - ideally with a clear sense of the “why” in addition to the “what”
  • Acknowledged challenges/diagnosis of issues that stand between you and achieving the goals
  • Guiding principles/policies
  • Focused actions to surmount the challenges and achieve the goals

The aforementioned challenges and principles were the results of applying this through process, which is how I determined both what to invest in as well as what to divest from, as alluded to earlier. A more tech focused example is,

  • Goal: Improve and maintain the health of systems we own while still delivering against our road map
  • Challenges: Some systems inherited as part of scaling up were unreliable to operate, presenting a drag on productivity and morale
  • Principle: We care about the long term quality and effectiveness of the things we ship and invest accordingly.
  • Actions: Foster a culture where system observability (monitoring and alerting) are baked into the definition of done. Ensure on-call rotations are well staffed and have clear responsibilities and support for incident handling, including prioritizing mitigations. Identify systems that fall short of that standard and either invest in them or decommission them based on their corresponding value.

Evolving teams logistics

Challenges

  • Many people working on many different things that don’t necessarily overlap.
  • It is necessary to start building shared understanding of the new landscape. This has to be balanced with maintaining focus on existing focus areas.

Thought prompts

  • Do we do planning together? Do we do stand-ups together?
  • Will everyone focused on the same project(s) or work stream(s)?
  • Might people end up flexing across pre-existing boundaries, especially as there are those interested in working in new domains?
  • Even if everyone is dispersed across different focus areas, is there value in maintaining some amount of shared context?

Principles

My conclusions I came to were

  • We plan together and we plan by work stream
    • We can make this manageable by implementing the right guardrails
  • We stand up together
  • We retrospect together

We plan together at a high level.

Outcomes: prioritized focus areas, pulling in and refining pre-defined work, clear sense of where immediate or near term follow ups are needed.

Guardrail: Time-box: 1 hour per week (with up to 15 minutes dedicated to ice breakers). If it can’t fit within an hour, it’s time to refine the scope.

We plan by work stream at a granular level.

Outcomes: deep dives between contributors, work breakdowns and distribution, definitions of done.

Guardrails: Cadence: this should happen at as frequently as needed to make sure that right information gets to the right people at the right time. Depending on the complexity of the project and the cohesion of the contributors, I’ve seen one per week, twice per week, or purely asynchronous cadences work. The needs here can also change throughout the lifecycle of a project and the cadence can be scaled up or down to match.

Ownership in practice: if the contributors on a work stream can gel without a standing project sync, then there doesn’t have to be one. To enable that, the context I, as a technical leader, should check for and provide when necessary, is the information that I think is necessary for the initiative to succeed (some combination of scope, timing, resources needed and stakeholders involved) and alignment on when and where to expect that information.

We stand up together.

Outcomes: Standing checkpoint for risks, progress, seeking and offering support

Guardrail: Time-box: 30 minutes. If it takes significantly more than 30 minutes to go through the core round-robin (not counting subsequent follow on convos), then consider off-loading even more to dedicated work stream sessions and/or one-on-ones.

We retrospect together.

Outcomes: Insights into what is working well, what to keep an eye on, and what to adjust. Accountability for the adjustments to be made.

Guardrails: Cadence: Too frequently and it becomes a drag on time. Too infrequently and you miss out gleaming insights while occupancies are still in the collective consciousness.

Time-boxes and clear participants

In order for team-wide forums to not become low-value time sinks, cap the length and depth of any discussion that any subset of participants don’t need to be a part of. Flag it for a later follow up and keep it moving. For those follow ups, be sure to state who should be involved and give other folks to explicit license to skip out, giving them the autonomy to balance focus vs context. That might looks like, “since we’ve wrapped the main agenda, if you’re working on X, stick around so we can follow up on topic Y that came up. Otherwise, feel free to dip out.”

Wrap Up

  • Autonomy enables scale. Lean on your experts and leverage the language of ownership.
    • Provide context: Make sure contributors understand what we are trying to achieve and why. Let them guide how it gets done.
    • Offer input in ways that develop, rather than diminish, contributors’ thinking (including your own). You can, and often should, have opinions on how things should be done. In practice, the best way I've found to offer input in ways that guide thinking is to state the purpose behind solutions. What am I optimizing for? What trade offs am I intentionally making? That added context allows the approach to be understood independently of and cohesively with the goals. That creates the opportunity to flag approaches that better serve the goals or gaps in the goals. I don't need to be right, I need to increase the likelihood of the right thing being done.
  • Be intentional about what you pick up and put down in order to create capacity.
  • Ceremonies: Leverage what has worked where you can; be willing to audit and adjust anything. Processes are meant to serve the team, not the other way around.
    • Remember to find your fences when managing change. Try to understand the goals of a system before deciding that the goals aren’t being met.