data science

null

Jun 25 2025

Update on Data Science versus Statistics

Based on the usage of the terms in the literature, I have concluded that statistics has been subsumed under data science. I view statistics as beginning with a dataset and ending with conclusions, while data science starts with sensors and transaction processing, and ends in data products for end users. Kelleher & Tierney’s Data Science views it the same way, and so do tool-specific references like Gromelund’s R for Data Science, or Zumel & Mount’s Practical Data Science with R.

Trevor Hastie
Bradley Efron

Brad Efron and Trevor Hastie are two prominent statisticians with a different perspective. In the epilogue of their 2016 book, Computer Age Statistical Inference, they describe data science as a subset of statistics that emphasizes algorithms and empirical validation, while inferential statistics focuses on mathematical models and probability theory.

Efron and Hastie’s book is definitely about statistics, as it contains no discussion of data acquisition, cleaning, storage and retrieval, or visualization. I asked Brad Efron about it and he responded: “That definition of data science is fine for its general use in business and industry.” He and Hastie were looking at it from the perspective of researchers in the field.

 

Continue reading…

By • Data science, Uncategorized • 0 • Tags: , ,

null

Sep 23 2023

Orbit Charts, Revisited

Data visualization is not just the art of presenting data to an audience. Upstream from this, you use visualizations in data cleaning to identify defective points, and in exploratory analysis, to identify patterns of interest. Then, you validate these patterns with a more formal analysis. Once confident that you have findings of value to communicate, you worry about making a compelling presentation.

Nick Desbarats and I had a long exchange on LinkedIn prompted by his article Connected Scatterplots Make Me Feel Dumb in Nightingale, the Data Visualization Society journal, on 8/29/2023. What he called Connected Scatterplot is what I call orbit charts, and I have found them helpful, particularly in analysis.

Continue reading…

By • Data science • 3 • Tags: , , ,

Pavimento_di_siena,_ruota_della_fortuna_small

Jun 12 2022

Perspectives On Probability In Operations

The spirited discussions on LinkedIn about whether probabilities are relative frequencies or quantifications of beliefs are guaranteed to baffle practitioners. They come up in threads about manufacturing quality, supply-chain management, and public health, and do not generate much light. Their participants trade barbs without much civility, and without actually exchanging on substance.

The latest one, by Alexander von Felbert, is among the more thoughtful, and therefore unlikely to inspire rants. I do, however, fault it with using words like “aleatory” or “epistemic” that I don’t think are helpful. I am trying to discuss it here in everyday language, and to apply the concepts to numerically specific cases, with an eye to operations.

While there are genuinely great and not-so-great ideas, the root of the most violent disagreements is elsewhere, with individuals generalizing from different experience bases. You may map probability to reality differently depending on whether you are developing drugs in the pharmaceutical industry, enhancing yield in a semiconductor process, or driving down dppms in auto parts. The math doesn’t care as long as you follow its rules, and it doesn’t invalidate other interpretations.

Continue reading…

By • Data science • 0 • Tags: , , ,

satellite-image-National-Oceanic-and-Atmospheric-Administration-August-28-2005.jpeg

Dec 25 2021

Always the Hurricanes Blowing

Atlantic hurricanes hurt people and destroy property around the Gulf of Mexico every year. Whether climate change is increasing their frequency is a serious question. Don Wheeler just had a column on this subject in the latest Quality Digest. It’s about Torturing the Data. He argues that we should be careful about not force-fitting models to arrive at pre-ordained conclusions.

His way of not torturing the history of hurricanes in the Atlantic is plotting yearly counts on, what else,  an XmR chart. It’s just as he would for hole diameters in metal plates coming off a production line in 1945. Hurricanes and holes in metal plates, however, have different backstories.

Continue reading…

By • Press clippings • 2 • Tags: , , , , , ,

Dec 25 2019

Your Lean Six Sigma Belt Program Is the Problem | Dan Markovitz | Industry Week

“I visited a company a few weeks ago that asks all of their employees to do a green belt project. It’s not mandatory, but completion of a project is part of their annual review. Not surprisingly, the management boasts that nearly everyone does a project. You know how many people do a second project? Less than 5%. This company is doing okay, but they definitely don’t have a culture of continuous improvement.”

Source: Industry Week

Michel Baudin‘s comments: Dan’s article is spot on, except in his assessment of statistical tools. Depending on the company’s situation, none of the ones he lists may be needed. Other tools, like SMED, cell design, mistake-proofing or JKK may be more relevant. Data science is needed in semiconductors and pharmaceuticals but the statistical tools Dan describes as “advanced” are not. ANOVA, regression, and t-tests go back 100 years; Design Of Experiments (DOE), a good 50. As for Ishikawa’s “7 tools of QC” from the 1960s,  I have never seen them used as advertised anywhere. They are sorely in need of an update in every respect, from data acquisition to analysis and presentation.

#leansixsigma, #blackbelts, #datascience,#7toolsofqc

By • Press clippings • 2 • Tags: , , , ,

QCtrackingToyota1950sArtOfLean2

Jul 16 2019

Updating the 7 tools of QC

A conversation with Franck Vermet about problem-solving tools for factory operators caused me to revisit the 7 tools of QC from 50 years ago and ponder how they should be updated with current data science.

Data Science for Operators, as a book, remains to be written. If you google this phrase today, what comes up is training courses offering to “change your career” by attending a “data science bootcamp.” TIBCO Spotfire has “Workflow Operators” but these are programs, not people.

So the following are tentative answers to questions that haven’t been asked before.

Continue reading…

By • Data science • 11 • Tags: , , , , , ,

1 2 3