Technology
May 4, 2020

Choosing The Right Metrics

Making sound business decisions depends on inputs like a calculated risk, timing, a bit of luck, execution, and a myriad of other components. These factors make deciding ‘what to do’ more overwhelming than it has to be in most cases. In theory, prioritization should be as simple as choosing the most valuable thing weighed against what’s achievable in the given period and resources. We’ve found the easiest way to ensure that such decisions have the most substantial impact is to pick the right measures beforehand.

A Driving Metric: What actually moves the numbers?

A driving metric is a measure that contributes profoundly to the result or goal. It has a ‘causal’ effect on the outcome implying that if that metric goes up or down, the outcome changes.

Take driving, for example. There are numerous measures presented to you while driving your car. How fast you’re going is the primary measure. In Massachusetts, we usually measure speed by how fast the guy we just passed seems to be going. This practice happens in bumper-to-bumper traffic too, and it’s much more enjoyable to think about in retrospect than in practice.

If we made the speed our key measure and set a goal of going faster, this gives us an ethical framework for figuring out which metrics drive that outcome. Immediately next to our speedometer (in many vehicles), we have a gauge called a tachometer. The tachometer measures the working speed of your engine and is conveyed in RPMs (rotations per minute).

Most cars have multiple gears, which change the relative torque of the engine at different stages to help both increase and decrease speed. Without getting too much into the specifics, lower gears generally make it easier for your engine to move your car forward and accelerate while at low speeds. However, lower gears need to rotate many more times per minute to sustain the same speed as the taller gears. This is why, as you go faster, you tend to shift up.

So if our goal is to go faster, we cannot use the RPMs as a driving metric. Why? Because they fluctuate based on the gear your vehicle is using at any given time, not the speed at which you travel. In theory, your car could downshift to first gear, and go 40MPH at 5,000 RPMs – a speed it could easily maintain in third or fourth gear at 1500-2000 RPMs.

A driving metric, in this case, would be much more specific. It can be how much fuel your vehicle was consuming while operating. Or how much oxygen was let into your engine while operating?

In bumper-to-bumper traffic, you can consume much fuel without going very far. But if the controlled environment is while the car is moving: generally increasing the amount of fuel and oxygen you consume produces higher speed. This is a driving metric.

A Vanity Metric: What is everyone else doing that we feel like we should do?

Vanity metrics are standard and everywhere. They dominate every market, every industry, everything. They’re unavoidable because they are simple, easy to understand and appear to translate across a broad majority of things seamlessly.

In software, vanity metrics are things like “active users.” They sound good and are easy to line up side by side to compare against other businesses, but they are rarely indicative of real value. This is evident in every company using different inputs to calculate active users. On Facebook, it’s how many logs in every day, and on LinkedIn, it’s how many users clicked on any LinkedIn link over 90 days. These don’t make sense to compare side by side.

They also don’t change meaningful outcomes because they are generally standardized and broadly accepted. They are not specific to your business or organization. So while you might create a public-facing measure that ties into one, trying to use a vanity metric as a chief measure to something critical to your business is often counterproductive.

For example, in football, there are hundreds of organizations that track an endless stream of player and team statistics.  It’s difficult now to watch an in-season game without hearing a commentator bring up some obscure record that is about to be broken.  It makes for exciting TV, but more often these ‘records’ are set because no one was tracking them before and our data pool is limited.  One key metric is yards per season for wide receivers.

It’s an exciting stat to share and discuss (we do often).  But it’s a vanity metric in its purest form. If a player is thrown the ball 100 times in a season (for example), they are more likely to have more yards per season than a player who is thrown the ball 10 times.  If your goal outcome was a higher yardage total, then pass attempts per season would be a driving metric.

Scope & Practicality: What is feasible for us to achieve in the given time period?

Every metric’s weight and impact vary based on the time and resources available for your exercise. If your business could aim at 40% market penetration, but with no proper resources and time to achieve it, success is nearly impossible. Your metrics are influenced in the same way.

Another way to think about it is in the context of moving homes. The type of home doesn’t matter. The objective remains consistent: to move all of our stuff into the new place. Assuming the amount of stuff we move from A to B remains the same during the move, we can look at how scope and practicality influence the metrics we choose to influence our outcome.

If we have much time to move from one place to the other, an excellent driving metric would be the number of trips we’ve taken between the two places. Assuming we could commit to delivering more of our things from one place to the other on each trip, more trips would = more stuff moved. This would mean that increasing the number of trips we took would accelerate the pace at which we achieved our objective: all things moved into the new place.

On the other hand, if we had limited time to move, the frequency of trips is neither practical nor a good indicator of success. In this case, moving capacity or cargo space becomes a key driving metric. Increasing cargo space decreases the number of trips needed to move all of our things into the new place. It also increases our likelihood of getting it all done in a short time. Considering key metrics without the context of the objective is a recipe for disaster.

Considering key metrics without the context of the objective is a recipe for disaster.  This example is written from personal experience. Trying to move an entire home in one week with two sedans is neither practical nor possible.  Fortunately, we had the resources to hire moving help last minute and got 80% of the work done in 5 hours. Increasing cargo space was the metric we should have focused on from the get-go.

Choosing the measure first: Understanding success indicators before anything is done

The most critical mistake that could occur preceding any attempt is failing to define success criteria clearly. This guarantees the confusion and frustration of all parties involved and limits any organization’s ability to learn from what happened. Even with the perfect driving metrics, plenty of time and resources, failing to define what qualifies as ‘good’ makes the entire effort subject to interpretation.

There are thousands of examples of this we could walkthrough. Their most common defining characteristic is when a responsible party defends the effort by saying, “it could have been worse.” This statement is impossible to prove without time travel and a second shot at it.

If success is not defined and transparently communicated, then the only clear outcome of any effort is increased expense and lost time. If an organization cannot come to a consensus on success criteria, we recommend avoiding the project altogether.

By clearly defining success criteria and adequately considering the context of your project, your organization can simplify the exercise of choosing the right metrics. We recommend focusing on driving metrics, those specific to your organization that contributes directly to the success or failure of your clearly defined goal. Vanity metrics are lovely to share and promote to external parties, but avoid bringing them into focus on a project they don’t directly affect.

This simplifies your prioritization of activity, and guarantees some success in every project:

First, by meeting your goal, you can discover key driving metrics you can build an evergreen program around improving.

Second, if you miss your goal, you’ll have a clear understanding of the activity that didn’t work, and how that activity influenced your key metrics. If you missed your goal but considerably improved the key metrics you chose, you’ll recognize that they were vanity metrics and not driving metrics.

latest news from us

Related Posts

view all
Technology

Choosing The Right Metrics

May 5, 2020