Michel Baudin‘s comments: I am sure many have had similar experiences to Becky’s with customer service in many companies. They tell you their product is used by millions and it’s the first time anyone reports this problem. You are probably using it wrong, or misreading its output,… This being said, it’s not really related to the concept of statistical significance.
Whenever journalists or politicians want to say that a change of any kind is meaningful, they borrow the phrase statistically significant from data science without realizing that, taken out of context and stripped of quantitative levels of significance, it means nothing. Significance is a concept that statisticians have invented to support decisions when outcomes are unclear.
Some experiments have obvious outcomes: as you turn on the ignition the car starts or it doesn’t; others experiments aren’t so clear. It the test plot with fertilizer A yields 3% more carrots than the one with fertilizer B, how firmly can you conclude that A is more effective than B? Whichever call you make, you run the risk of either ranking A as better than B when it’s not or ranking them as equal when A is better.
The statisticians’ levels of significance help you quantify these risks. For example, you may set a threshold in differences between A and B so that a yield above that threshold has a 1% probability of occurring if both fertilizers have the same effect. Then you say that the difference is significant at a 1% level of significance. It matters whether the level is 1% or .001%. Unless you specify the level, statistically significant has no meaning.
Statistical significance is a matter of probability, not volume. In the celebrated case of the Cuckoo’s Egg of the 1980s, Clifford Stoll, the system manager at the Lawrence Berkeley Lab in California, noticed a 75-cent error in the computer center’s books. Since the books were kept using the center’s computers, this small error could only be consistent with normal operations if the computers couldn’t add. Stoll imputed it to hacking instead, and eventually traced it to a German spy ring working for the KGB that had penetrated numerous US military databases.
“Not statistically significant” cannot be used to justify a lack of attention to details or rare events. Within a company, attention to detail is a matter of engagement. It withers where employees are expected to just do as they are told because details at any level are invisible to higher levels in the hierarchy. It thrives where employees are called upon to put all their abilities to use on the job, including their senses and their intelligence, in other words where management shows respect for their humanity. What is a minor detail to their bosses is major to them, and they will take the initiative to address it.
May 11 2017
Not statistically significant does NOT mean insignificant | Rebecca A. Morgan | AME Target
Sourced through AME Target
Michel Baudin‘s comments: I am sure many have had similar experiences to Becky’s with customer service in many companies. They tell you their product is used by millions and it’s the first time anyone reports this problem. You are probably using it wrong, or misreading its output,… This being said, it’s not really related to the concept of statistical significance.
Whenever journalists or politicians want to say that a change of any kind is meaningful, they borrow the phrase statistically significant from data science without realizing that, taken out of context and stripped of quantitative levels of significance, it means nothing. Significance is a concept that statisticians have invented to support decisions when outcomes are unclear.
Some experiments have obvious outcomes: as you turn on the ignition the car starts or it doesn’t; others experiments aren’t so clear. It the test plot with fertilizer A yields 3% more carrots than the one with fertilizer B, how firmly can you conclude that A is more effective than B? Whichever call you make, you run the risk of either ranking A as better than B when it’s not or ranking them as equal when A is better.
The statisticians’ levels of significance help you quantify these risks. For example, you may set a threshold in differences between A and B so that a yield above that threshold has a 1% probability of occurring if both fertilizers have the same effect. Then you say that the difference is significant at a 1% level of significance. It matters whether the level is 1% or .001%. Unless you specify the level, statistically significant has no meaning.
Statistical significance is a matter of probability, not volume. In the celebrated case of the Cuckoo’s Egg of the 1980s, Clifford Stoll, the system manager at the Lawrence Berkeley Lab in California, noticed a 75-cent error in the computer center’s books. Since the books were kept using the center’s computers, this small error could only be consistent with normal operations if the computers couldn’t add. Stoll imputed it to hacking instead, and eventually traced it to a German spy ring working for the KGB that had penetrated numerous US military databases.
“Not statistically significant” cannot be used to justify a lack of attention to details or rare events. Within a company, attention to detail is a matter of engagement. It withers where employees are expected to just do as they are told because details at any level are invisible to higher levels in the hierarchy. It thrives where employees are called upon to put all their abilities to use on the job, including their senses and their intelligence, in other words where management shows respect for their humanity. What is a minor detail to their bosses is major to them, and they will take the initiative to address it.
#StatisticalSignificance, #AttentionToDetail, #RespectForHumanity
Contents
Share this:
Like this:
Related
By Michel Baudin • Press clippings • 0 • Tags: Attention to Detail, Respect for Humanity, Statistical Significance