Are Your Business Metrics Meaningless?

If data drives decisions, measurements drive data. That's why managers and business owners love metrics. We use -- and sometimes overuse -- metrics to quantify profitability, return, productivity, efficiency, quality, etc.

The metrics we use should provide an objective scorecard of results.

Should provide an objective scorecard, but not always. Here's an example.

I was in charge of manufacturing at a book production facility. Productivity was all-important, driving a number of performance and financial results. But productivity could be measured in many different ways.

Take throughput, a commonly used productivity metric. Throughput reflects the number of units produced per some period of time. If you produce 29,000 widgets in 8 hours, your throughput is 3,625 widgets/hour. Clean and simple.

Also misleading. In our business, throughput was almost meaningless as a metric.

Why? Say you manage a call center and the number of calls a sales rep makes is a key metric. (You assume the more calls made the more sales since in your business sales is in large part a numbers game.) Because your reps are cold calling prospects, call duration ranges from seconds to twenty minutes or more. Measuring call throughput over a long period of time -- say, the average number of calls made per month -- could help smooth out natural variation in call length and provide a relatively accurate reflection of a salesperson's call productivity.

Maybe.

As a short-term metric, though, throughput results could be very misleading.

The same was true at our book plant. We ran thousands of titles a year of widely different run lengths. Some run quantities were under 2k, others over 500k. Running a number of jobs with small quantities during a shift automatically decreased throughput, since changing from job to job took time. The more jobs run in a day, the lower the throughput -- even if the crew was incredibly efficient.

So we measured run and makeready results separately, using two simple formulas:

• Makeready average = total makeready time / number of makereadies

• Run average = total books produced / total run hours

Sound good? Not so fast. Comparing results, either between crews or over time, was a problem. Which is better:

• 3,800 books per hour and a 24 minute makeready average?

• 3,750 books per hour and a 22 minute makeready average?

Worse, the crews knew that, so they would manipulate results, writing down slower or longer makeready times as they saw fit to manipulate results. While a shift always added up to 12 hours, how time was actually spent during that 12 hours was open to manipulation.

Instead of attempting (and probably failing) to enforce more accurate reporting, we created a metric we called adjusted books/hour (ABH).

Here's how it worked:

Our makeready goal was 20 minutes, so every time a crew performed a makeready they got credit for a 20-minute makeready, regardless of how long that makeready actually took. If they had 3 makereadies during a 12-hour shift they got 60 minutes of makeready time. The remaining time was considered run time and was divided into the total books produced to determine ABH.

Here's the formula:

ABH = total books / (total run time - (number of makereadies times 20 minutes))

The ABH metric gave us a straightforward way to evaluate performance trends and compare crew-to-crew performance. If crews performed makereadies faster than 20 minutes, great -- they had more run time available and could run more books. If their makereadies took longer their ABH suffered since they had less actual time time to run books.

(Keep in mind we tracked actual performance on makereadies so we could spot areas for improvement; we just used ABH as an apples to apples metric.)

Take a look at what you measure:

• Do some metrics leave room for interpretation?
• Can some metrics be manipulated by how and when data is collected?
• Do your metrics truly measure what is important to your business?

Measuring is important, but measuring what you need to measure -- and measuring it the right way -- is critical.

Note to Readers: Check out my BNET series The 11,500 Foot View, the chronicle of my attempt to accomplish an impossible goal, remain in the good graces of my family, run a business, stay sane, and blog about it. If you like to watch people struggle, you'll love this.