The One Where Executives React to Their First Metrics Dashboard

🍿 Welcome to Episode 2 of our Managing Metrics series, where Mojtaba Hosseini of Zapier is reflecting back on his metrics adventures at previous companies as he guides Zapier in their current journey of becoming a more metric-driven engineering organization!

Previously on Tales From a Metrics Journey…

In Episode 1 of our Managing Metrics series, I wrote about a previous organization where the Dev and Ops teams’ measurement of Deployment Frequency and Lead Time uncovered a well-known but not-well-articulated problem: with a lead time of 75 days, the new features and bug fixes from the Dev team were not making it into production - which meant we were not delivering value to our customers!

Executives Taste The Power of Metrics

Showing the very large (and growing) Lead Time metric and explaining what this meant for our customers, the Business Unit VP succinctly understood the problem, had a visual and simple way to track the progress of its solution, and so empowered the teams to tackle it (Dev team’s sprint would be dominated by helping Ops overcome the deployment backlog). This was not the end, however.

Seeing how powerful measuring metrics could be be, our Business Unit VP now wanted metrics for everything. A dashboard to rule them all! He wanted the ability to see how we were measuring against our goals, and how our initiatives impacted the metrics all in one place.

Panic? Yes. Go ahead and panic!

For the line managers of the various teams in the business unit, we had opened Pandora’s box. We had invited the Eye of Sauron to gaze upon each feeble kingdom - and so a good degree of panic and depression set in.

“How can we quantify to a simple dashboard all the nuance, complexity, and intricacy involved in our work?”
“How can we prevent perverse incentives from setting in where everyone is working to game these metrics?”
“How can we work to create and maintain and review these metrics when we are already running at 100% capacity?”
“Will we be punished if a metric has gone down for a month/quarter and it wasn’t even our doing?”

And so on…

There were many objections, worries, and trepidations about our executive’s newfound desire for metrics. In fact, most if not all of these worries were well-founded and speak to the many pitfalls of implementing metrics.

So what do you do about them?

On the Next Episode

Tune in to see how our plan to tackle the problem of embedding metrics in our daily work ran into a vicious cycle and completely stalled!

Lessons Learned

Having taken the first step and learned how incredibly powerful metrics can be, we then learned:

• Being metric driven is a journey, not a final destination. It involves many many steps, each with its own challenges, and it’s important to move forward and not be fixated on the “ideal” solution.
• Metrics can be a double-edged sword that needs careful wielding.
• Metrics come with context and typically don’t stand fully on their own - they need humans to ask questions in order to understand what they really mean.
• Executive demand for metrics is rooted in a very legitimate need to succinctly see how the business is doing, to measure success, and to assess decisions.
• Executive demand for metrics, when an organization early in its journey, can cause much panic
• There are many pitfalls to using metrics, and it’s good to be aware of them and mitigate them.

Stay tuned for Episode 3 of Managing Metrics to see what vicious cycle awaited on this metrics journey!

[ Guest post from Mojtaba Hosseini ]