Data, information, knowledge, and Lean

cuneiform

Terms like data and information are often used interchangeably. Value Stream Mapping (VSM), for example, is also called Materials and Information Flow Analysis (MIFA) and, in this context, there is no difference between information and data. Why then should we bother with two terms? Because  “retrieving information from data” is meaningless unless we distinguish the two.

The term knowledge is used to call a document a “body of knowledge” or an online resource a “knowledge base,” when their contents might be more aptly described as dogmas or beliefs with sometimes a tenuous relationship with reality. Computer scientists are fond of calling  knowledge anything that takes the form of rules or recommendations. Having an assertion in a “knowledge base,”  however, does not make it knowledge in any sense the rest of the world would recognize. If it did, astrology would qualify as knowledge.

In Lean, as well as for many other topics, clarity enriches communication, and, in this case, can be achieved easily, in a way that is useful both technically and in everyday language. In a nutshell:

  1. Data is what is read or written.
  2. Information is what you learn from reading data.
  3. Knowledge is information that agrees with reality.

Authors like Chaim Zins have written theories about data, information, and knowledge that are much more complicated and I don’t believe more enlightening than the simple points that follow. They also go one step further, and discuss how you extract wisdom from knowledge, but I won’t follow them there. The search for wisdom is usually called philosophy, and it is too theoretical a topic for this blog.

Data

In his landmark book on computer programming, Don Knuth defined data as “the stuff that’s input or output.” While appropriate in context, this definition needs refinement to include data that is not necessarily used in a computer, such as the Wright Brothers’ lift measurements (See Figure 1). If we just say data is the stuff that’s read or written, this small change does the trick. It can be read by a human or a machine. It can be written on paper by hand or by a printer, it can be displayed on a computer screen, it can be a colored lamp that turns on, or even a siren sound.

Figure 1. Data example: the Wright Brothers’ lift measurements

More generally, all our efforts we make a plant visible have the purpose of making it easier to read and, although it is not often presented this way, 5S is really about data acquisition. Conversely, team performance boards, kanbans andons, or real-time production monitors are all ways to write data for people, while any means used to pass instructions to machines can be viewed as writing data, whether it is done manually or by control systems.

What is noteworthy about reading and writing is that both involve replication rather than consumption. Flow diagrams for materials and data can look similar, but, once you used a screw to fasten two parts, you no longer have that screw, and you need to keep track of how many you have left. On the other hand the instructions you read on how to fasten these parts are still there once you have read them: they have been replicated in your memory. Writing data does not make you forget it. This fundamental difference between materials and data needs to be kept in mind when generating or reviewing, for example, Value Stream Maps.

Information

Information is a more subtle quantity. If you don’t know who won the 2010 Soccer World Cup and read a news headline that tells you Spain did, you would agree that reading it gave you some information. On the other hand, if you already knew it, it would not inform you, and, if you read it a second time, it won’t inform you either. In other words, information is not a quantity that you can attach to the data alone, but to a reading of the data by an agent.

If you think of it as a quantity, it has the following characteristics:

  1. It is positive. You can learn from reading data, but reading data cannot make you forget. As a quantity, information can therefore only be positive or zero, and is zero only  if the data tells you nothing you didn’t already know. In addition, data in a news story about an outcome that you know to be impossible add no information.
  2. It is maximum for equally likely outcomes.A product order gives you the most information when you had no idea what product it might be for. Conversely, if you know that 90% of all orders are for product X, the content of the next order is not much of a surprise: you will lose only 10% of the time if you bet on X. The amount of information you get from reading data is maximum when you know the least, and therefore all possible values are equally likely to you.
  3. It is subadditive. If you read two news stories, the information you acquire from both will be at most the sum of the information you acquire from each. If you read about independent topics like the flow of orders on the company’s e-commerce website and lubrication problems in the machine shop, the information from both stories will be the sum of the information in each. If, however, the second story is about, say, dealer orders, then the two stories are on related subjects, and the total information received will be less than the sum of the two.

The above discussion embodies our intuitive, everyday notion of what information is. For most of our purposes —  like designing a performance board for Lean daily management, an andon system, a dashboard on a manager’s screen, or a report — this qualitative discussion of information is sufficient. We need to make sure they provide content the reader does not already know, and make the world in which he or she operates less uncertain. In other words, reading the data we provide should allow readers to make decisions that are safer bets about the future.

In the mid 20th century, however, the mathematician Claude Shannon took it a step further, formalized this principle into a quantitative definition of information, and proved that there was one only one mathematical function that could be used to measure it. He then  introduced the bit as its unit of measure. Let us assume that you read a headline that says “Spain defeats the Netherlands in the Soccer World Cup Final.” If you already knew that the finalists were Spain and the Netherlands and thought they were evenly matched, then the headline gives you one bit of information. If you had no idea which of the 32 teams that entered the tournament would be finalists, and, to you, they had all equal chances, then, by telling you it was Spain and the Netherlands, the headline gives you an additional 8.9 bits.

Over the decades, his theory has had applications ranging from the design communication networks to counting cards in blackjack, more than to help manufacturers understand factory data. It has a use, however, in assigning an economic value to the acquisition of information, and thereby justify the needed investment.

Knowledge

On November 3, 1948, the readers of the Chicago Tribune received information in the “Dewey defeats Truman” headline (See Figure 2). None of them would, however, describe this information as knowledge, just because it was not true. It should not need to be said, and, outside the software industry, it doesn’t. Software marketers, however, have muddied the water be calling rules derived from these assertions “knowledge,” regardless of any connection with reality. By doing so, they have erased the vital distinction between belief, superstition or delusion on one side and knowledge on the other.

Figure 2. When information is not knowledge

As Mortimer Adler put it in Ten Philosophical Mistakes (pp. 83-84), “it is generally understood that those who have knowledge of anything are in possession of the truth about it.  […] The phrase ‘false knowledge’ is a contradiction in terms; ‘true knowledge’ is manifestly redundant.”

When “knowledge bases” were first heard from in the 1980’s, they contained rules to arrive at a decision, and only worked well with rules that were true by definition. For example, insurance companies have procedures to set premiums, which translate well to “if-then” rules. A software system applying these rules could then be faster and more accurate than a human underwriter retrieving them from a thick binder.

On the other hand, in machine failures diagnosis, rules are true only to the extent that they actually work with the machine; this is substantially more complex and error-prone that applying procedures, and the rule-based knowledge systems of the 1980’s were not successful in this area.  Nowadays, a “knowledge base” is more often a forum where users of a particular software product post solutions to problems. While these forums are useful, there is no guarantee that their content is, in any way, knowledge.

The books on data mining are all about algorithms, and assume the availability of accurate data. In real situations, and particularly in manufacturing, algorithms are much less of a problem than data quality. There is no algorithm sophisticated enough to make wrong data tell a true story. The key point here is that, if we want to acquire knowledge from the data we collect, the starting point is to make sure it is accurate. Then we can all be credible Hulks (See Figure 3).

Figure 3. The Credible Hulk (received from Arun Rao, source unknown)

Metrics in Lean – Part 3 – Equipment

The aggregate metric for equipment most often mentioned in the Lean literature is Overall Equipment Effectiveness (OEE). I first encountered it 15 years ago, when a client’s intern who had been slated to help on a project I was coaching, was instead commandeered to spend the summer calculating the OEEs of machine tools. I argued that  the project was a better opportunity than just taking measurements, both for the improvements at stake for the client and for the intern’s learning experience, but I lost. Looking closer at the OEE itself, I felt that it was difficult to calculate accurately, analyze, and act on. In other words, it does not meet the requirements listed in Part 1.

The OEE is usually defined as follows:

OEE = Availability × Performance × Quality

A perfect machine works whenever you need it, and is therefore available 100% of the time. It works at its nominal speed, and therefore its performance is 100%, and it never makes a defective product, so that its yield is 100%, and so is its OEE. The OEE of a real machine is intended reflects the combination of its failures to live up to these ideals.

Availability

The first problem is the meaning of availability. When we say of any device that it is available, we mean that we can use it right now. For a production machine that does one operation at a time, it would mean that it is both up and not in use. The main reason for it to be unavailable is that it is busy, which really shouldn’t count against it, should it? In telecommunications, availability for a telephone switch is synonymous with the probability that it is up. This is because it is supposed to be up all the time, and to have the capacity to handle all incoming calls. In principle, it could be unavailable because of saturation, but the availability formula does not even consider it. It is only based on uptime and downtime,  or on time between failures and time to repair.

But a lathe doesn’t work like a telephone switch it at least two ways:

  1. It is rarely expected to work all the time: it may work two shifts a day, five days a week, and, whether it is down the rest of the time has no effect on performance.
  2. If you have one work piece on a spindle, you can’t load another one at the same time, and the spindle is unavailable.

In the OEE context, we are not talking about the machine being available in the sense being up and ready to take on a new task  but instead of time available to a scheduler to assign it work in the course of a planning period, which may be a shift or a day, or whatever time interval is used for this factory.

If, in a 480-minute shift, a machine stops during a 30-minute break and has up to 60 minutes of unscheduled downtime and setups, then the planner can count of 480 -30-60 = 390 minutes in which to schedule work, which yields a ratio of: Availability = 390/480 = 87%.

This assumes that the machine’s ability to do work is proportional to the time it is up. My first moped as a teenager was a hand-me-down from a relative that had been garaged for 7 years. It started fine when cold, but the spark plug started malfunctioning once it was warm, about 15 minutes later. It would stay up for 75% of a 20-minute ride but that didn’t mean it completed 75% of the rides. It actually left me stranded about 100% of the time;  it was unusable. Likewise, your link to a server may work 99% of the time while uploading a large file and break every time you try to save it. The formula makes it look as if it has 99% availability when in fact it is 0%.

There is also an issue with deducting setups from available time, because, unlike breakdowns, it is not just an issue of the technical performance of the machine but is directly affected by operating policies. The planner can influence the amount of time used for setups, reducing it by increasing the size of production runs or, if setup times vary with all pairs of from- and t0-products, by sequencing them so as to minimize the total setup time.

This is not to say that the formula is wrong but only that it commingles the effects of many causes and that its relevance is not universal. There may be better  ways to quantify availability depending on the characteristics of a machine and the work it is assigned. Companies that calculate OEEs often do not bother with such subtleties and simply equate availability with uptime.

Performance

Performance is a generic term with many different meanings. As a factor in the OEE, it  is the ratio of nominal to actual process time of the machine. If the machine actually takes two minutes to process a part when it is supposed to take only one, its performance is 50%. The times used are net of setups and don’t consider any quality issue, because quality is accounted for in the last factor. This factor is meant to account for microstoppages and reduced speeds, and it is a relevant and important equipment metric in its own right.

Quality

As discussed in Part 2, Quality is not a metric but a whole dimension of manufacturing performance with many relevant metrics. In the OEE, this factor is just the yield of the operation, meaning the ratio of good parts to total parts produced. It is not the First-Pass Yield, because reworked parts are still counted as good.

Conclusions on the OEE

While the OEE summarizes metrics that are individually of interest, not much use can be made of it without unbundling it into its different factors. Since the meaning and the calculation methods for its factors vary across companies, it cannot be used for benchmarking. Within a company and even within a plant, it is not always obvious that raising the OEE of every machine enhances the performance of the plant as a whole.

In principle, it should. Who doesn’t want all machines to be always available, perform to standard, and make good parts? The problem is that, in practice, increasing the OEE is often confused with increasing utilization, and that there are machines for which  it is not a worthwhile goal. Such machines may be cheap auxiliaries to expensive ones, like a drill press following a large milling machine in a cell, or they may have been bought for their ability to take on a large variety of tasks on demand.

Unbundling the OEE into its component factors yields a more easily understandable  set of equipment metrics that is less likely to mislead management. While these metrics can be collected on each piece of equipment, management must then be wary of aggregating them over machines that are intended to be used differently.

Metrics in Lean – Part 2 – Quality

My previous post on this subject listed some requirements on metrics to be useful on the shop floor. Here, we take it one level further, first by establishing that shop floor metrics should be in the language of things rather than the language of money, and then by using the requirements to evaluate examples of quality metrics. In later posts, I will do the same for other dimensions of manufacturing performance.

The language of things and the language of money

The language of the shop floor is that of things, not money. It is a world where you speak in numbers of units made, time spent, or weight consumed, and it is several organizational layers removed from the boundary of the company where money changes hands. Metrics posted on the shop floor must therefore not be financial. This does not mean that financials should be hidden from shop floor personnel, just that they should not be the basis for metrics these people review everyday.

To be effective, the metrics used on the shop floor should be expressed in the language of things, and translated into the language of money by accountants for communication up the management chain. This is not to say that financial information should be hidden from shop floor people, only that it should be made available on request rather than broadcast or posted on bulletin boards.

Metrics based on financials are useful, but at a higher level of aggregation than a cell and for other audiences than teams of operators. In some areas, it is also difficult to arrive at financial metrics that accurately represent the business impact of activities and events occurring on the shop floor. Cost of Quality (COQ) metrics, for example, are limited to the direct costs of failure, appraisal and repair, which do not even begin to account for the sales impact of the company’s reputation for quality. In the Quality Control Handbook, Frank Gryna explicitly excluded such items from the COQ  as being to complex and too controversial. The problem is that what he did include is negligible with respect to what he excluded. Just ask Firestone about tread separation, or even Toyota about its 2010 recalls. As a result, COQ cannot be used to justify improving quality. Staying on the subject of quality, let us examine what metrics can be effective.

Metrics of Quality

Many companies use a scoring system, as illustrated in Figure 1, to measure outgoing quality. Starting from an agreed score for a perfect unit, points are deducted for every defect observed. The quality department publishes averages of these scores by shift, day, or month, looking for an upward trend.

FIGURE 1. A scoring system for cars

While such metrics are frequently a source of pride for those who develop them, they are unintelligible to the rest of the organization. Higher scores are understood to be better than lower scores, but the relation of the score to competitive position and the sources of its variations are difficult to interpret. In addition, the penalty assigned to each type of defect can become a matter of endless debate. Quality is more effectively measured by using multiple, simpler metrics, covering different subtopics, such as:

  1. Ratings by external agencies for consumer goods. The JD Power and Associates Initial Quality Surveys rate products based on the number of problems reported by buyers within the first three months of purchase, which reflect  manufacturing quality. Consumer Reports publishes reliability and maintainability ratings for cars that have been on the road for several years, which are more indicative of design quality.
  2. Counts of customer claims. For parts sold to OEMs, Quality Problem Reports (QPR) are the equivalent of traffic tickets issued by customers. They require failure analysis by the supplier’s quality department and a report to the customer in a prescribed format, such as Ford’s “8D” or Honda’s “5P.” The rate at which such claims are issued is clearly a metric of the customers’ perception of your product quality.
  3. Rejection rates. Defectives are weeded out at Receiving, at various points inside the manufacturing process, at final test, and at customers’ Receiving. Rejection rates are calculated at each of these points and fed back to the managers in charge. There may, however, be too many of these rates to be usable by top management.
  4. First-pass yield. The first-pass yield may be the most useful measure of in-process quality for top management. It is the percentage of the production volume that makes it through the entire process unscathed — that is, without failing any inspection or test, and without being waived, reworked, or scrapped (See Figure 2).

Figure 2. The First-Pass Yield

Metrics in Lean – Part 1 – Requirements

Ever since childhood, we all want to know what grades we get on our report cards, and what these grades mean in terms of how well we are doing. We want to be evaluated based on parameters we understand and we can affect by our efforts.

A key issue in manufacturing is consistency as we go from a shop to an department to an entire plant and to the company as a whole. We don’t want to use parameters in terms of which which excellent local performance can aggregate to poor global performance. Once performance measures are selected, the next challenge is to use them as a basis for management decisions that are in the best interest of the company while being fair and nonthreatening to employees. In particular, actions taken to improve one aspect of performance must not degrade another. In addition to these issues, in a lean environment, we need to consider the impact of improvement projects, before and after they are carried out.

Measuring process compliance or results?

One possible approach to performance evaluation is to measure how closely our practice matches a standard of how things should be done. This is how you will be evaluated if you apply for the Malcolm Baldridge award or for ISO-900x certification. If matters little whether your outgoing quality is any good, as long as you follow the “right” processes. Iwao Kobayashi’s “20-keys” approach follows the same logic. The keys have names like “Cleaning and organizing,” or “Quick changeover,” and each key has 5 levels of achievement. By definition, a plant that is at level 5 in all 20 keys is excellent.

The advantage of process measures is that the corrective action for bad performance is always to bring the plant closer to compliance. But is it impossible for a plant to be at level 5 in all 20 keys and still chronically lose money? Don’t some of the keys matter more than others? The world would be simpler if a process existed such that compliance guaranteed excellence.

In fact, all the stakeholders in a factory care much more about the results it achieves than the processes by which it does. Most commonly used are the five dimensions of Quality, Cost, Delivery, Safety, and Morale. More generally, Harvard’s R. Kaplan has proposed a “balance scorecard” to measure multiple aspects of business performance, as opposed to just manufacturing performance.

Requirements on metrics

Metrics should be focused on results rather than process compliance. The Malcolm Baldridge award criteria, ISO-900x, or Kobayashi’s “20-keys to workplace improvement” promote performance measurement based on check lists of how close actual shop practices are to some norm. The problem with this approach is that it is possible to score high on any of these check lists and still go bankrupt. In other words, it’s not what you do that counts but what good it does. The key requirements for metrics are as follows:

  1. A good metric is immediately understandable. No training or even explanation is required to figure out what it means, and the number directly maps to reality, free of any manipulation. One type of common manipulation is to assume that one particular ratio cannot possibly be over 85%, and redefine 85% for this ratio as “100% performance.” While this makes performance look better, it also makes the number misleading and difficult to interpret.
  2. People see how they can affect the outcome. With a good metric, it is also easy to understand what kind of actions can affect the value of the measurement. A shop floor metric, for example, should not a be function of the price of oil in the world market, because there is nothing the operators can do to affect it. Their actions, on the other hand, can affect the number of labor-hours required per unit, or the rework rate.
  3. A better value for the metric always means better business performance for the company. One of the most difficult characteristics to guarantee is that abetter value of a metric always translates to better business performance for the company. Equipment efficiency measures are notorious for failing in this area, because maximizing them often leads to overproduction and WIP accumulation.
  4. The input data of the metric should be easy to collect. Lead time statistics,for example, require entry and exit timestamps by unit of production. The  difference between these times then only gives you the lead time is calendar time, not in work time. The get lead times in work time, you then haveto match the timestamps against the plant’s work calendar. Lead time information,however, can be inferred from WIP and WIP age data, which can be collected by direct observation of WIP on the shop floor. Metrics of WIP, therefore, contain essentially the same information but are easier to calculate.
  5. All metrics should have the appropriate sensitivity. If daily fluctuations are not what is of interest, then they need to be filtered out. A common method for doing this is to plot 5-day moving averages instead of individual values– that is, the point plotted today is the average of the values observed in the last five days. Daily fluctuations are smoothed away, but weekly trends clearly show.