Metrics in Lean – Chart junk in performance boards and presentations

Manufacturing professionals who read Edward Tufte‘s books on visualization may be stunned to discover that their 3D pie charts, stacked bar charts, and green safety crosses are chart junk. These charts are common, both on shop floor performance boards and in management presentations. But they are information-poor, and their decorative elements distract, confuse, and occasionally mislead.  The purpose of plotting is not to dazzle, but to discover patterns, understand the underlying phenomena, and communicate with people whose livelihood is affected by these patterns.

For details, see:

1. Pabulum pies

Following is an example of what Microsoft Support considers to be a pie chart with a “spiffy, professional look,” and a few comments about it.
This chart not only uses buckets of ink to display five data points, it also violates other good design principles:
  1. The legends are remote from their objects, forcing the reader to look in two places to understand each item.
  2. The chart begs a question that it could answer but doesn’t: percentages of what total amount?
  3. The shadow and the light reflection convey no information.
  4. The 3D effect makes the 21% slice appear larger than it actually is.
If we put the legends on their objects, add the total amount as a subtitle, get rid of the vinyl-cushion look, and take a top view so that the wedge sizes are not distorted, we get the following:
Microsoft pie chart example - improved

One feature that now stands out is that the wedges are ordered by decreasing size, except for the last two: “Desserts” is larger than “Beverages” but comes after. It is the example Microsoft uses for training and they don’t explain the sequence. While this chart is an improvement over the previous version, but do we actually need it? A picture may be worth a 1,000 words but, if what you have to say fits in 10 words, a picture may be overkill. Compare the pie chart with the following table of the data in it:

Microsoft pie chart example - data table

If you compare a pie chart with a sorted table of the data in it, you see that the chart uses orders of magnitude more ink in multiple colors than the table, without telling you anything that isn’t already obvious from the table. In Edward Tufte’s book, it makes it the kind of chart junk that you should banish from your materials.

3D pie charts are actually worse, because they are not only useless but misleading in that the perspective distorts the apparent size of the wedges. This is an illustration of another of Tufte’s principles, that a graphic display should not have more dimensions than the data. In 3D pie charts, the height of the pie is meaningless. The only circumstance in which showing pie thicknesses would be useful if when showing several pie charts side by side, and both pie diameter and thickness would represent parameters of each pie.

2. Stacks astray

Vertical bar charts, also known as column charts, usually serve to show the evolution of a quantity for which data is available for a sequence of periods. It is used when there are few periods and interpolation between periods does not make sense. If you plot daily sales over a year in a vertical bar chart, the columns will be so densely packed that you will only see the line formed by their tops anyway, and you might as well just use a line plot.
Bar charts - Daily sales as bar versus line plot
On the other hand, if you are plotting monthly sales, you use vertical bars, because interpolation between two points does not give you meaningful intermediate numbers. If you plotted temperature readings taken at fixed intervals, for example on a hospital patient, you would interpolate to estimate the temperature between readings, and therefore you use a line plot rather than vertical bars.
Bar chart of monthly sales

Bar chart of monthly sales

There is nothing objectionable to the ordinary, vertical bar chart. It is simple to produce and easy to read, and provides information that is not obvious in a table of numbers. Stacked bar charts, however, are another matter. They attempt to show the evolution over multiple periods of both an aggregate quantity and its breakdown in multiple categories, and do a poor job of it, especially when a spurious 3rd dimension is added for decoration.
We can retun to the Microsoft support web site for a tutorial on stacked bar charts. The building-block look of the Microsoft example may be appropriate for an audience of preschoolers.

Another oddity of the Microsoft example is that it does not follow the convention by which vertical bars are used when the categories on the x-axis represent consecutive time buckets. When they are not ordered categories, you usually prefer horizontal bars, not only because we are used to the x-axis representing time in performance charts but also because this chart has horizontal category labels, readable without tilting your head.

Based on the preceding section, the first question we might ask is whether we need graphics at all, and the answer is yes. When we take a look at the data in table format, even though there are only 16 points, no pattern is immediately apparent:

Stacked bar Microsoft example source table

If we forget the spurious 3rd dimension and the building-block look, and toggle the axes so that the x-axis represents time, we get the following:

Stacked bar Microsoft example 3D removed

From this chart, it is obvious that total sales collapsed from the 2nd to the 4th quarter; it was not obvious from either the table of number or the 3D chart Microsoft presented as a model. Even in this form, however, the stacked bar chart is ineffective at answering the immediate follow-up question of whether the decline in sales was more pronounced in some regions than others. For example, if we try to isolate the Northeast on this chart, we find bars floating at different heights. On the other hand, the answers become visible if you de-stack the bars, as in the following:

Stacked bar Microsoft example unstacked

For example, you can see that sales actually grew from the 1st to the 2nd quarter in both Eastern regions, while declining throughout the year everywhere else. Yes, it takes up more space than the stacked bar, but is that really a concern when, even in a simple example like this one, it lets you see better?

3. Safety crosses to bear

The safety performance of a plant matters, and not only to the potential victims of accidents. Injuries must be rare events, occur with ever decreasing frequency, and have both immediate countermeasures and permanent solutions to prevent recurrence. In light of this, what kind of graphic summaries would be appropriate to represent the safety performance of a whole plant, or of a shop within it?

A common metric is the number of consecutive days without a lost-time accident. It is a relevant measure, but imperfect in that its use has been known to lead thoughtless managers to blame victims for hurting their department’s performance and pressure them not to report injuries. It is also necessary to show detailed information about each accident, and to categorize injuries, for example, in terms of whether they affected hands, shoulders, feet, etc. and where exactly they occurred.

In light of these considerations, what is industry using? In the US, the National Safety Council uses a green cross in its logo and awards a Green Cross for Safety Medal to one company each year. That makes the green cross a symbol of safety, and it has motivated some to make it the basis of a safety performance tracking chart. You start with a cross shape subdivided into rectangles numbered 1 through 31 and, on each day of the month, you place a magnet on the corresponding spot, which is green if nothing happened, yellow if a minor incident happened, and red for a lost time accident.

It is difficult to think of the use of a chart in the form of a symbol of safety as anything but a gimmick. The green cross has no connection with any requirements we can think of for a chart of safety performance. It does not make it particularly easy to count consecutive days without incident, nor does it bear any information about the nature or location of the accidents. The green cross shape evokes safety but has nothing to do with the key questions about employee safety.

To visualize a sequence of rare events, a technique that comes to mind is the timeline. You arrange events around a centerline that shows the passage of time, as in the following example that summarizes 10 years of the history of the iPhone.

Timeline 10 years of iPhone history

Safety-related events in a factory do not need a 10-year timeline, but possibly a six-month timeline along the following model:

Timeline of sports events january-june 2010

Note that the time between events is immediately visible, and that each event has some explanation and photographic documentation. For location information, you can pin injury locations on an outline of a human body and a map of the shop floor. While these may not be pleasant charts to consider, they are a means of starting a conversation in the team on what the safety issues are and the means of preventing injuries.

There are, of course, other safety issues, like repetitive stress, that are not associated with discrete events and do not appear on these charts, but they do not appear on the green crosses either.

4. Recommendation on performance board design

Performance boards come in all sorts of shapes, as in the following examples:

As a performance board for a shop floor team, I recommend the following template:

Performance board template

Performance board template

This template has one column per dimension of performance and one row for each type of information, as follows:

  1. The top row is for the latest news, what happened this shift or last.
  2. The second row is for historical trends in aggregate performance.
  3. The third row is for a breakdown of the aggregate into its constituent parts, such as the most common injuries, defect categories, most frequently made products, or the employee skills matrix.
  4. The bottom is about actions or projects in progress in each of these areas.

5 comments on “Metrics in Lean – Chart junk in performance boards and presentations

  1. Comment by David J (Joe) Armstrong on LinkedIn Updates:

    Nice tutorial. I’ve followed Tufte’s work for years and concentrate hard on getting rid of chart junk including down to the exasperating default gray background that is or was standard in Excel graphs. As Tufte says, maximize the information to ink ratio. One tool I selectively use is what I call multi-charts, similar to your example of the plant metrics storyboard or Tufte’s T-shirt example. In multi charts, I group many related charts on a page. Once the eye and the brain familiarize themselves with the information and layout, it is possible to shift rapidly between the charts to gain insight and understanding.

    For an example of David Armstrongs multi-charts, see his Tracking Financials page.

  2. The lessons here about data vs. information are especially important in companies struggling with lean transformation. Collaborative problem-solving by cross-functional groups is a common element of successful transformations. When confronted by unclear information or piles of raw data, groups of diverse thinkers struggle to find consensus. Evidence-based analysis gives way to interpretations and bias, root causes are never reached and precious resources are wasted on the resulting frivilous actions. Too often we believe the ‘data speaks for itself’ when, in reality, nothing could be further from the truth. When faced with the chart junk and visually conflicted information that is so common, we owe it to one another to speak up despite the risk of being seen as the stupid one in the room. “Um, pardon me, but I’m not sure I understand what that chart/graph is saying. Please explain.”

  3. Thanks for the interesting examples of information panels. These are in my opinion very clear and easy to read by those not involved directly.

  4. Pingback: Where Have The Scatterplots Gone? | Michel Baudin's Blog

  5. Pingback: What You Can And Cannot Learn About Manufacturing From Books | Michel Baudin's Blog

Please share your thoughts: