May 1 2013
Metrics in Lean – Chart junk in performance boards and presentations
Manufacturing professionals who read Edward Tufte‘s books on visualization may be stunned to discover that their 3D pie charts, stacked bar charts, and green safety crosses are chart junk. These charts are common, both on shop floor performance boards and in management presentations. But they are information-poor, and their decorative elements distract, confuse, and occasionally mislead. The purpose of plotting is not to dazzle, but to discover patterns, understand the underlying phenomena, and communicate with people whose livelihood is affected by these patterns.
For details, see:
- The legends are remote from their objects, forcing the reader to look in two places to understand each item.
- The chart begs a question that it could answer but doesn’t: percentages of what total amount?
- The shadow and the light reflection convey no information.
- The 3D effect makes the 21% slice appear larger than it actually is.
One feature that now stands out is that the wedges are ordered by decreasing size, except for the last two: “Desserts” is larger than “Beverages” but comes after. It is the example Microsoft uses for training and they don’t explain the sequence. While this chart is an improvement over the previous version, but do we actually need it? A picture may be worth a 1,000 words but, if what you have to say fits in 10 words, a picture may be overkill. Compare the pie chart with the following table of the data in it:
If you compare a pie chart with a sorted table of the data in it, you see that the chart uses orders of magnitude more ink in multiple colors than the table, without telling you anything that isn’t already obvious from the table. In Edward Tufte’s book, it makes it the kind of chart junk that you should banish from your materials.
3D pie charts are actually worse, because they are not only useless but misleading in that the perspective distorts the apparent size of the wedges. This is an illustration of another of Tufte’s principles, that a graphic display should not have more dimensions than the data. In 3D pie charts, the height of the pie is meaningless. The only circumstance in which showing pie thicknesses would be useful if when showing several pie charts side by side, and both pie diameter and thickness would represent parameters of each pie.
Another oddity of the Microsoft example is that it does not follow the convention by which vertical bars are used when the categories on the x-axis represent consecutive time buckets. When they are not ordered categories, you usually prefer horizontal bars, not only because we are used to the x-axis representing time in performance charts but also because this chart has horizontal category labels, readable without tilting your head.
Based on the preceding section, the first question we might ask is whether we need graphics at all, and the answer is yes. When we take a look at the data in table format, even though there are only 16 points, no pattern is immediately apparent:
If we forget the spurious 3rd dimension and the building-block look, and toggle the axes so that the x-axis represents time, we get the following:
From this chart, it is obvious that total sales collapsed from the 2nd to the 4th quarter; it was not obvious from either the table of number or the 3D chart Microsoft presented as a model. Even in this form, however, the stacked bar chart is ineffective at answering the immediate follow-up question of whether the decline in sales was more pronounced in some regions than others. For example, if we try to isolate the Northeast on this chart, we find bars floating at different heights. On the other hand, the answers become visible if you de-stack the bars, as in the following:
For example, you can see that sales actually grew from the 1st to the 2nd quarter in both Eastern regions, while declining throughout the year everywhere else. Yes, it takes up more space than the stacked bar, but is that really a concern when, even in a simple example like this one, it lets you see better?
Safety crosses to bear
The safety performance of a plant matters, and not only to the potential victims of accidents. Injuries must be rare events, occur with ever decreasing frequency, and have both immediate countermeasures and permanent solutions to prevent recurrence. In light of this, what kind of graphic summaries would be appropriate to represent the safety performance of a whole plant, or of a shop within it?
A common metric is the number of consecutive days without a lost-time accident. It is a relevant measure, but imperfect in that its use has been known to lead thoughtless managers to blame victims for hurting their department’s performance and pressure them not to report injuries. It is also necessary to show detailed information about each accident, and to categorize injuries, for example, in terms of whether they affected hands, shoulders, feet, etc. and where exactly they occurred.
In light of these considerations, what is industry using? In the US, the National Safety Council uses a green cross in its logo and awards a Green Cross for Safety Medal to one company each year. That makes the green cross a symbol of safety, and it has motivated some to make it the basis of a safety performance tracking chart. You start with a cross shape subdivided into rectangles numbered 1 through 31 and, on each day of the month, you place a magnet on the corresponding spot, which is green if nothing happened, yellow if a minor incident happened, and red for a lost time accident.
It is difficult to think of the use of a chart in the form of a symbol of safety as anything but a gimmick. The green cross has no connection with any requirements we can think of for a chart of safety performance. It does not make it particularly easy to count consecutive days without incident, nor does it bear any information about the nature or location of the accidents. The green cross shape evokes safety but has nothing to do with the key questions about employee safety.
To visualize a sequence of rare events, a technique that comes to mind is the timeline. You arrange events around a centerline that shows the passage of time, as in the following example that summarizes 10 years of the history of the iPhone.
Safety-related events in a factory do not need a 10-year timeline, but possibly a six-month timeline along the following model:
Note that the time between events is immediately visible, and that each event has some explanation and photographic documentation. For location information, you can pin injury locations on an outline of a human body and a map of the shop floor. While these may not be pleasant charts to consider, they are a means of starting a conversation in the team on what the safety issues are and the means of preventing injuries.
There are, of course, other safety issues, like repetitive stress, that are not associated with discrete events and do not appear on these charts, but they do not appear on the green crosses either.
Recommendation on performance board design
Performance boards come in all sorts of shapes, as in the following examples:
As a performance board for a shop floor team, I recommend the following template:
This template has one column per dimension of performance and one row for each type of information, as follows:
- The top row is for the latest news, what happened this shift or last.
- The second row is for historical trends in aggregate performance.
- The third row is for a breakdown of the aggregate into its constituent parts, such as the most common injuries, defect categories, most frequently made products, or the employee skills matrix.
- The bottom is about actions or projects in progress in each of these areas.
David J (Joe) Armstrong
May 1, 2013 @ 4:08 pm
Comment by David J (Joe) Armstrong on LinkedIn Updates:
For an example of David Armstrongs multi-charts, see his Tracking Financials page.
May 2, 2013 @ 8:31 pm
The lessons here about data vs. information are especially important in companies struggling with lean transformation. Collaborative problem-solving by cross-functional groups is a common element of successful transformations. When confronted by unclear information or piles of raw data, groups of diverse thinkers struggle to find consensus. Evidence-based analysis gives way to interpretations and bias, root causes are never reached and precious resources are wasted on the resulting frivilous actions. Too often we believe the ‘data speaks for itself’ when, in reality, nothing could be further from the truth. When faced with the chart junk and visually conflicted information that is so common, we owe it to one another to speak up despite the risk of being seen as the stupid one in the room. “Um, pardon me, but I’m not sure I understand what that chart/graph is saying. Please explain.”
June 16, 2013 @ 7:23 am
Thanks for the interesting examples of information panels. These are in my opinion very clear and easy to read by those not involved directly.
Where Have The Scatterplots Gone? | Michel Baudin's Blog
May 14, 2015 @ 2:37 pm
[…] and stacked bars charts, with a few time series and heat maps. Other than being loaded with graphic junk, these charts have in common that each shows only one variable, as you can see in the following […]
What You Can And Cannot Learn About Manufacturing From Books | Michel Baudin's Blog
December 5, 2016 @ 3:45 pm
[…] check out the illustrations in the book, how many there are and of what kind. If all you see is pie charts, stacked-bar charts and assorted staples of business slideware, you are probably better off skipping the book. It is […]
Updating the 7 tools of QC - Michel Baudin's Blog
January 28, 2020 @ 12:55 pm
[…] For more on chart junk, how to avoid it and what to replace it with, see Metrics in Lean – Chart junk in performance boards and presentations – Michel Baudin’s… […]