Aug 22 2017
The fallacy of maturity assessments | Chris Hohmann
“Maturity assessments are a kind of qualitative audit during which the current ‘maturity’ of an organization is compared to a maturity reference model and ranked accordingly to its score.[…] The maturity assessment is usually quite simple: a questionnaire guides the assessment, each maturity level being characterized by a set of requirements. It is close to an audit.
The outcome of such an assessment is usually a graphic summary displaying the maturity profile or a radar chart, comments about the weak points / poor scores and maybe some recommendation for improvement. […]
Maturity assessments are not a bad thing per se, but their practicality and simplicity are often misused to assess more than just maturity (or awareness). This is most often misleading because of the false underlying assumptions and promoting wrong behaviors and practices.”
Sourced through Chris Hohmann’s blog
Michel Baudin‘s comments:
I agree with Chris’s analysis, but my conclusions are blunter. Scoring an organization in terms of compliance with a set of practices is like judging a chess player by the number of pawns moved per game. It’s doable but irrelevant, and a distraction from the real work of improvement. The record of this approach is that you have organizations scoring top marks on every axis while going bankrupt and low scoring organizations that prosper.
The reason this approach is popular, particularly with the supplier support groups of large companies, is that it doesn’t require the auditors to know any specifics of the business, management, and technology of the organization they are scoring. They can see whether all the visible signs of 5S are in place, each team has a performance board and each machine an andon light,… You can even check how many Kaizen events they have run in the past 12 months.
The suppliers learn how they must appear to humor the auditors and engineer their facilities to look right. To them, it’s a cost of doing business, not an opportunity to improve, which explains why they don’t perform any better than competitors. If instead of complying with external mandates, an organization is driven by its management to improve its practices at all times, in pursuit of broader goals than certification by a customer, it will not follow a generic script and perhaps undertake projects that are not as spectacular to auditors but are more useful. For example, instead of putting
If instead of complying with external mandates, an organization is driven by its management to improve its practices at all times in pursuit of broader goals than certification by a customer, it will not follow a generic script and perhaps will undertake projects that are not as spectacular to auditors but are more useful.
For example, instead of putting andon lights on every machine, they may pursue setup time reduction on one of the machines and later propagate the new method to all similar machines. Or they engineer retrofits to old machines that double their capacity or eliminate an entire category of defects.
Having mental or physical check lists when visiting a plant is useful. You want to notice the type of containers used on the side of an assembly line, idle machines, or empty tool pockets in machining centers,… to use as input to an analysis, not just to summarize as a simplistic score.
#MaturityAssessment, #RadarChart, #20Keys, #ShingoModel, #ExternalMandate, #ChristianHohmann
August 23, 2017 @ 10:20 am
Comment on LinkedIn:
August 23, 2017 @ 10:28 am
Comment on LinkedIn:
August 23, 2017 @ 10:29 am
Comment on LinkedIn:
August 23, 2017 @ 10:40 am
Comment on LinkedIn:
August 24, 2017 @ 12:39 am
Excellent post. I agree with both of you. And the situation gets worse when factories or business units are compared against one another – especially when the results are used in reviews/ratings. It’s always good to know where you are in terms of your vision, but a subjective assessment is not the way to do it. Enlightened leadership teams know where their problems are and, if they possess even a basic kaizen mindset, can get more out of working on closing the gaps than wasting time conducting and arguing about a maturity assessment.
August 25, 2017 @ 5:22 am
It is hard to disagree with you.
However, how should a big company have an idea of how good its suppliers are? Imagine a supermarket chain, for example, with zero understanding of what drives good performance in production.
August 25, 2017 @ 9:44 am
Please feel free to disagree.
You don’t need to understand what drives good performance for your suppliers in order to monitor them in terms of delivery, quality, and price but you can’t help them improve unless you do understand it.
A common mistake for companies is to staff supplier support groups with rookies and give them audit checklists as crutches. To suppliers, it is arrogant and disrespectful.
Supplier support groups should be comprised of seasoned professionals who have been successful in internal projects and are intellectually nimble enough to understand the differences between their company’s business and that of each supplier.
They must be willing and able to dive into each supplier’s specific issues deep enough to hold meaningful discussions with its leaders. They must come across as respectful and credible.
And then, not only must you do without their help on internal projects, but you have the challenge of retaining them.
August 27, 2017 @ 1:09 am
That makes sense and matches my observations!
To most suppliers, “assessment” means “audit”. And they feel they have to look good. They keep the important data to themselves. Don’t expect them to disclose and break down their costs of quality, for example.
Outside the auto industry, I have seen very few organizations doing a good job at helping their suppliers improve. Ikea does a good job. Apple sends a lot of resources to set up processes & lines correctly. I am sure there are others but not many. And in each case the supplier’s top management is open to help/deep involvement from their customer.
August 27, 2017 @ 6:49 am
Big companies, like Ikea, Apple, or Toyota can, unless they act with restraint, be perceived as bullies by their suppliers. That happens even if they don’t intend to, and they must be especially careful. But the roles are reversed when the customer is a small company buying from larger suppliers.
Can a small customer provide supplier support? I know at least one case where it was successfully done.
September 16, 2017 @ 10:59 am
I’d say, it all depends how maturity assessments are used. Like with any other tool, if it is applicable and used with focus on an actual problem, then it can be powerful by creating new insights and initiating the right actions.
I am using the Lean Factory Audit and the Lean Office Audit (https://www.leanmap.com/benchmarking/lean-audit/) to engage the management team into an (often intense) discussion, identifying the weak areas that contribute or even cause the problem the team is trying to solve.
Because the outcome is a score, and performance driven people like scores, they stay engaged and interested. The process is then viewed as a sporty challenge to improve the score. New energy is created that can now be directed towards the problem. If properly steered, any gain in maturity will then correlate well with the performance gain that team is going after, e.g. increasing productivity to reduce the cost base.
And I agree, a maturity assessment is not very helpful when just used in a compliance context (not much learning created), while it can be very effective when used as an anchor point and integral part of an improvement system – with focus on a well-defined business problem. Just my five cents.
Joerg Muenzing