Data science

Dec 10 2017

Is SPC Obsolete? (Revisited)

Six years ago, one of the first posts in this blog — Is SPC Obsolete? — started a spirited discussion with 122 comments. Reflecting on it, however, I find that the participants, including myself, missed the mark in many ways:

  1. My own post and comments were too long on what is wrong with SPC, as taught to this day, and too short on alternatives. Here, I am attempting to remedy this by presenting two techniques, induction trees and naive Bayes, that I think should be taught as part of anything reasonably called statistical process control. I conclude with what I think are the cultural reasons why they are ignored.
  2. The discussions were too narrowly focused on control charts. While the Wikipedia article on SPC is only about control charts, other authors, like Douglas Montgomery or Jack B. Revelle, see it as including other tools, such scatterplots, Pareto charts, and histograms, topics that none of the discussion participants said anything about. Even among control charts, there was undue emphasis on just one kind, the XmR chart, that Don Wheeler thinks is all you need to understand variation.
  3. Many of the contributors resorted to the argument of authority, saying that an approach must be right because of who said so, as opposed to what it says. With all due respect to Shewhart, Deming, and Juran, we are not going to solve today’s quality problems by parsing their words. If they were still around, perhaps they would chime in and exhort quality professionals to apply their own judgment instead.

Continue reading…

By • Data science • 2 • Tags: , , ,

Oct 23 2017

There Is More To Data Than Just Numbers

Don Wheeler’s Understanding Variation starts with a chapter entitled “Data are random and miscellaneous” that contains no discussion of any part of its title. Implicit in Wheeler’s book, however, is the view that data consists of tables of numbers, representing either measured variables — lengths, weights, densities,… — or event occurrence counts — defective units, defects, machine failures,…

Many times, I have quoted computer scientist Don Knuth on this subject, saying that data is “the stuff that’s input or output,” meaning anything that can be read or written, and it includes much more than tables of numbers. The data we work with today includes, for example, the following:

  • Unstructured text, like 25,000 incident reports written by maintenance techs all over the world in their versions of English about problems with jet engines, or thousands of product reviews posted by consumers on e-commerce sites
  • Images, like photographs of visual defects on products, or electron-microscope images of integrated circuits.
  • Videos recordings of operations.
  • …

Analyzing data about a manufacturing process today means extracting information from all sources. The state of the art, based on automatic data acquisition and databases includes analytical techniques that were unthinkable in Shewhart’s day, known under the labels of data science, data mining or machine learning.

Continue reading…

By • Data science • 2 • Tags: , , ,

Jan 3 2017

Probability For Professionals

dice In a previous post, I pointed out that manufacturing professionals’ eyes glaze over when they hear the word “probability.” Even outside manufacturing, most professionals’ idea of probability is that, if you throw a die, you have one chance in six of getting an ace.

2000 years ago, Claudius wrote a book on how to win at dice but the field of inquiry has broadened since, producing results that affect business, technology, science, politics, and everyday life.

In the age of big data, all professionals would benefit from digging deeper and becoming, at least, savvy recipients of probabilistic arguments prepared by others. The analysts themselves need a deeper understanding than their audience.

With the software available today in the broad categories of data science or machine learning, however, they don’t need to master 1,000 pages of math in order to apply probability theory, any more than you need to understand the mechanics of gearboxes to drive a car.

It wasn’t the case in earlier decades, when you needed to learn the math and implement it in your own code. Not only is it now unnecessary, but many new tools have been added to the kit. You still need to learn what the math doesn’t tell you: which tools to apply, when and how, in order to solve your actual problems. It’s no longer about computing, but about figuring out what to compute and acting on the results.

Following are a few examples that illustrate these ideas, and pointers on concepts I have personally found most enlightening on this subject. There is more to come, if there is popular demand.

Continue reading…

By • Data science • 1 • Tags: , , , ,

Nov 22 2016

If Talk Of Probability Makes Your Eyes Glaze Over…

Few terms cause manufacturing professionals’ eyes to glaze over like “probability.” They perceive it as a complicated theory without much relevance to their work. It is nowhere to be found in the Japanese literature on production systems and supply chains, or in the American literature on Lean. Among influential American thinkers on manufacturing, Deming was the only one to focus on it, albeit implicitly, when he made “Knowledge of Variation” one of the four components of his System of Profound Knowledge (SoPK).

Continue reading…

By • Data science • 1 • Tags: , , , ,

Jul 9 2016

The Value Of Surveys: A Debate With Joseph Paris

Joseph Paris and I debated this issue in the Operational Excellence group on LinkedIn, where he started a discussion by posting the following:

“Riddle me this…

If the Japanese way of management and their engagement with employees is supposedly the best, yielding the best result, why is there such a lack of trust among employment across the spectrum; employers, bosses, teams/colleagues. From Bloomberg and EY.

Japanese Workers Really Distrust Their Employers preview image

Japanese Workers Really Distrust Their Employers

Lifetime employment sounds like a great thing, but not if you hate where you work. That seems to be the plight of Japanese “salarymen” and “office ladies.” Only 22 percent of Japanese workers have “a great deal of trust” in their employers, which is way below the average of eight countries surveyed, according to a new report by EY, the global accounting and consulting firm formerly known as Ernst & Young. And it’s not just the companies: Those employees are no more trusting of their bosses or colleagues, the study found.

Continue reading…

By • Data science • 2 • Tags: , , , ,

May 14 2015

Where Have The Scatterplots Gone?

What passes for “business analytics” (BI), as advertised by software vendors, is limited to basic and poorly designed charts that fail to show interactions between variables, even though the use of scatterplots and elementary regression is taught to American middle schoolers and to shop floor operators participating in quality circles.

But the software suppliers seem to think that it is beyond the cognitive ability of executives. Technically, scatterplots are not difficult to generate, and there are even techniques to visualize more complex interactions than between pairs of variables, like trendalyzers or 3D scatterplots. And, of course, visualization is only the first step. You usually need other techniques to base any decision on data.

Continue reading…

By • Data science • 4 • Tags: , , ,

1 2 3 4 5
%d