The fallacy of maturity assessments | Chris Hohmann

“Maturity assessments are a kind of qualitative audit during which the current ‘maturity’ of an organization is compared to a maturity reference model and ranked accordingly to its score.[…] The maturity assessment is usually quite simple: a questionnaire guides the assessment, each maturity level being characterized by a set of requirements. It is close to an audit.

The outcome of such an assessment is usually a graphic summary displaying the maturity profile or a radar chart, comments about the weak points / poor scores and maybe some recommendation for improvement. […]

Maturity assessments are not a bad thing per se, but their practicality and simplicity are often misused to assess more than just maturity (or awareness). This is most often misleading because of the false underlying assumptions and promoting wrong behaviors and practices.”

Sourced through Chris Hohmann’s blog

Michel Baudin‘s comments:

I agree with Chris’s analysis, but my conclusions are blunter. Scoring an organization in terms of compliance with a set of practices is like judging a chess player by the number of pawns moved per game. It’s doable but irrelevant, and a distraction from the real work of improvement. The record of this approach is that you have organizations scoring top marks on every axis while going bankrupt and low scoring organizations that prosper.

The reason this approach is popular, particularly with the supplier support groups of large companies, is that it doesn’t require the auditors to know any specifics of the business, management, and technology of the organization they are scoring. They can see whether all the visible signs of 5S are in place, each team has a performance board and each machine an andon light,… You can even check how many Kaizen events they have run in the past 12 months.

The suppliers learn how they must appear to humor the auditors and engineer their facilities to look right. To them, it’s a cost of doing business, not an opportunity to improve, which explains why they don’t perform any better than competitors. If instead of complying with external mandates, an organization is driven by its management to improve its practices at all times, in pursuit of broader goals than certification by a customer, it will not follow a generic script and perhaps undertake projects that are not as spectacular to auditors but are more useful. For example, instead of putting

If instead of complying with external mandates, an organization is driven by its management to improve its practices at all times in pursuit of broader goals than certification by a customer, it will not follow a generic script and perhaps will undertake projects that are not as spectacular to auditors but are more useful.

For example, instead of putting andon lights on every machine, they may pursue setup time reduction on one of the machines and later propagate the new method to all similar machines. Or they engineer retrofits to old machines that double their capacity or eliminate an entire category of defects.

Having mental or physical check lists when visiting a plant is useful. You want to notice the type of containers used on the side of an assembly line, idle machines, or empty tool pockets in machining centers,… to use as input to an analysis, not just to summarize as a simplistic score.

#MaturityAssessment, #RadarChart, #20Keys, #ShingoModel, #ExternalMandate, #ChristianHohmann