Metrics in Lean – Part 1 – Requirements

Ever since childhood, we all want to know what grades we get on our report cards, and what these grades mean in terms of how well we are doing. We want to be evaluated based on parameters we understand and we can affect by our efforts.

A key issue in manufacturing is consistency as we go from a shop to a department to an entire plant and to the company as a whole. We don’t want to use parameters in terms of which excellent local performance can aggregate to poor global performance. Once performance measures are selected, the next challenge is to use them as a basis for management decisions that are in the best interest of the company while being fair and nonthreatening to employees. In particular, actions taken to improve one aspect of performance must not degrade another. In addition to these issues, in a lean environment, we need to consider the impact of improvement projects, before and after they are carried out.

Measuring process compliance or results?

One possible approach to performance evaluation is to measure how closely our practice matches a standard of how things should be done. This is how you will be evaluated if you apply for the Malcolm Baldridge award or for ISO-900x certification. If matters little whether your outgoing quality is any good, as long as you follow the “right” processes. Iwao Kobayashi’s “20-keys” approach follows the same logic. The keys have names like “Cleaning and organizing,” or “Quick changeover,” and each key has 5 levels of achievement. By definition, a plant that is at level 5 in all 20 keys is excellent.

The advantage of process measures is that the corrective action for bad performance is always to bring the plant closer to compliance. But is it impossible for a plant to be at level 5 in all 20 keys and still chronically lose money? Don’t some of the keys matter more than others? The world would be simpler if a process existed such that compliance guaranteed excellence.

In fact, all the stakeholders in a factory care much more about the results it achieves than the processes by which it does. Most commonly used are the five dimensions of Quality, Cost, Delivery, Safety, and Morale. More generally, Harvard’s R. Kaplan has proposed a “balance scorecard” to measure multiple aspects of business performance, as opposed to just manufacturing performance.

Requirements on metrics

Metrics should be focused on results rather than process compliance. The Malcolm Baldridge award criteria, ISO-900x, or Kobayashi’s “20-keys to workplace improvement” promote performance measurement based on check lists of how close actual shop practices are to some norm. The problem with this approach is that it is possible to score high on any of these check lists and still go bankrupt. In other words, it’s not what you do that counts but what good it does. The key requirements for metrics are as follows:

1. A good metric is immediately understandable. No training or even explanation is required to figure out what it means, and the number directly maps to reality, free of any manipulation. One type of common manipulation is to assume that one particular ratio cannot possibly be over 85%, and redefine 85% for this ratio as “100% performance.” While this makes performance look better, it also makes the number misleading and difficult to interpret.
2. People see how they can affect the outcome. With a good metric, it is also easy to understand what kind of actions can affect the value of the measurement. A shop floor metric, for example, should not a be function of the price of oil in the world market, because there is nothing the operators can do to affect it. Their actions, on the other hand, can affect the number of labor-hours required per unit, or the rework rate.
3. A better value for the metric always means better business performance for the company. One of the most difficult characteristics to guarantee is that abetter value of a metric always translates to better business performance for the company. Equipment efficiency measures are notorious for failing in this area, because maximizing them often leads to overproduction and WIP accumulation.
4. The input data of the metric should be easy to collect. Lead time statistics,for example, require entry and exit timestamps by unit of production. The  difference between these times then only gives you the lead time is calendar time, not in work time. The get lead times in work time, you then haveto match the timestamps against the plant’s work calendar. Lead time information,however, can be inferred from WIP and WIP age data, which can be collected by direct observation of WIP on the shop floor. Metrics of WIP, therefore, contain essentially the same information but are easier to calculate.
5. All metrics should have the appropriate sensitivity. If daily fluctuations are not what is of interest, then they need to be filtered out. A common method for doing this is to plot 5-day moving averages instead of individual values– that is, the point plotted today is the average of the values observed in the last five days. Daily fluctuations are smoothed away, but weekly trends clearly show.