Nov 19 2011

Is it true that you get what you measure?

The article Lean Manufacturing: Measuring To Get Results by Gerald Najarian lists a number of useful metrics. It also opens with the saying, or cliche, that “you get what you measure.”

The implications are (1) that people will always do whatever it takes to maximize their metrics, and (2) that, if you put the right metrics in place, improvement will take care of itself. While I agree that we need good metrics, we should not overestimate their impact. Peer pressure and personal ethics, among other factors, drive most people more than their performance metrics. And even when employees do their utmost to maximize their scores, they often do not have the necessary skills, and performance targets will have no effect unless backed up by some form of training, coaching and support.
Via ezinearticles.com

By Michel Baudin • Metrics, Press clippings • 35 • Tags: Lean, Management, Performance

35 Comments

Jim
November 26, 2011 @ 12:24 pm

Albert Einstein—“Not everything that counts can be counted and not everything that can be counted counts.”

With that said, use facts and data wisely to support gut feel and use gut feel wisely to support facts and data. There is a place for both in decision making.

Reply
Dan
December 13, 2011 @ 3:51 pm

The last letter of KPI is for Indicator. Measurements are shadows, not real objects. My first lesson as a Supervisor was ‘what you pay attention to, gets better’. If you only pay attention to metrics, only metrics will get better. Measurement is a form of paying attention, certainly not the only or best.

Reply
Bruce Weeks
December 31, 2011 @ 6:34 am

Bruce Weeks made the following comments in the LinkedIn Continuous Improvement, Six Sigma, & Lean Group:

The truth is, yes, you get what you measure. Simply because that is what employees can see, touch and effect. To them, if it is not measured then it must not be important. I’ve had my supervisors tell me that I am not being measured on that, go work on what makes my KPI’s go up.

That is why it is so critical to get the right measures in place. They must align with the organization’s goals and objectives, be rolled up at a fairly high level from lower level measures that are directly observable by the employee that touches that process.

There are a couple articles available on Wiremold who has done this very successfully. Just Google Wiremold and you will find them.

Reply
Dave Medrano
December 31, 2011 @ 6:38 am

Dave Medrano made the following comments in the LinkedIn Continuous Improvement, Six Sigma, & Lean Group:

30 years ago someone said it best, “You can only expect, what you inspect.” If you can’t surround it with metrics of some type, or a “go look, go see” then it’s highly unlikely to get the expected results.

Reply
Steven Borris
December 31, 2011 @ 6:40 am

Steven Borris made the following comments in the LinkedIn Continuous Improvement, Six Sigma, & Lean Group:

I think you can “track” what you measure and use the information to guide your resources.

For example, if a production line have a high rate of defects, how do they know there is a need to reduce them?

If they are trying to reduce them, how do they know they are being successful?

If the company receives a high number of customer complaints, does this information get recorded? Does anyone investigate the causes or introduce solutions? Do the folk that make/provide the the product/service know that this is happening?

We are utilising the technique of Short Interval Control. Measuring does not guarantee an improvement. It does provide the data you need, however, to recognise if improvement is needed. Not measuring will probably mean very little improvement.

Steve

Reply
Johnny Davis
December 31, 2011 @ 7:11 am

Johnny Davis made the following comments in the LinkedIn Lean & Kaizen Group:

To a large degree this is true. But the measurements involved have to be followed-up on by a truly interested management and coupled with written objectives by those responsible for effectively improving results. Otherwise measurements are somewhat meaningless.

Reply
Patrick Spagon, Ph.D.
January 2, 2012 @ 12:03 pm

Comment in the Lean Six Sigma discussion group on LinkedIn:

It depends on the quality/reliability of the complete measurement system. Also do you really have the right metric for what you are trying to achieve?

Reply
PAOLO B-- USETTI
January 2, 2012 @ 12:05 pm

Comment in the Lean Six Sigma discussion group on LinkedIn:

I agree with Patrick: you can get what you measure depending on the approximation related to the effectiveness (R&R) of the measurement system you get. In case of objective measurements and metrics this can be even calculated. But this become much more difficult when you face subjective measurements. Perhaps in this case we could speak about evaluations instead of measurements: consequently it would be more difficult to answer to your question.

Reply
MARK A. KINSEY
January 2, 2012 @ 12:08 pm

Comment in the Lean Six Sigma discussion group on LinkedIn:

Not sure…but I do know this…pretty much “what gets done, is what gets rewarded” (of course the federal government is the exception). Best small book I read in my whole life. The Greatest Management Principle. It is soooo true!!! I believe it is a very important parallel to your question.

Reply
Scott Hoepfl
January 2, 2012 @ 12:10 pm

Comment in the Lean Six Sigma discussion group on LinkedIn:

@Mark – I agree, people do what they are rewarded to do. This makes the issue of what gets rewarded very important and critical to the success of the company. The reward program must be viewed in a cross functional way so as many unintended consequences as possible are identified and countered before the program is implemented.

As for the government, it is very difficult to work for a board of directors (congress) who hates your very existence. I think that is why there are so many detailed procedures. They must always be able to justify e

Reply
David Bacon
January 2, 2012 @ 12:11 pm

Comment in the Lean Six Sigma discussion group on LinkedIn:

I agree that Metrics will always drive behaviour : If you don’t measure it, you can’t manage it! If you do measure it, but don’t know how good or bad your measurement system is, then you aren’t managing it!
Metrics will always focus attention and priorities and should be selected wisely, have personal accountability and be regularly challenged and reviewed.

Reply
R. Steven Schmidt
January 2, 2012 @ 12:13 pm

Comment in the Lean Six Sigma discussion group on LinkedIn:
“Tell me how you are going to measure me, I’ll tell you how I will behave.” Dr. Goldratt

Reply
- Brian Waild
  January 6, 2012 @ 7:51 am
  
  I agree- Measurements are critical because they guide behavior. A good leader strives to find the balance between effective metrics and analysis paralysis. if your team spends more than 5% collecting and analysis it should be a red flag. If there are discussions about accuracy of data or doubt about what’s really going on more than once per week that’s another red flag. Immediate action by a senior leader should be taking to influence corrective action.
  
  Reply
V.B. Manian
January 2, 2012 @ 12:44 pm

Comment in the Lean Six Sigma Canada discussion group on LinkedIn:

In simple business parlance METRIC is the measurement of a particular characteristic of the performance or efficiency of a process. Now the definition (operational/working/practical) of what the METRIC should be would depend on the question, “The metric is needed to measure what ?”. Most often it is seen that what (ie, the problem) needs to be measured is NOT clear or is fraught with ambiguity, which further makes the process of understanding the METRIC for measuring it (the problem) difficult. So clarity is needed FIRST in understanding what needs to be measured before going ahead with understanding the metric for measuring it.

In your Six Sigma project, you first need to determine the Key Process Input/Output Variables (KPOV/KPIV), perform an MSA (very important) & then measure them. Never select a metric simply because it is easier to measure or more readily available. Select a METRIC because it is IMPORTANT, then measure it. You will often find that the important metrics are the ones that are not measured, and the metrics that are measured were selected because they were easy to obtain, not necessarily because they were important.

the following also characterise a good definition of Metric:

* Necessity/Requirement – The data required must be necessary per the situation of investigation.
* Source – From where the data needs to be collected.
* Frequency – Frequency of data collection.
* Accuracy – Uncertainty of data accuracy makes the investigation cumbersome & may be also a waste. (Do not confuse this with 1st point (Necessity/Requirement). A particular data may be necessary/required but may not be accurate)
* Reliability – If the data to be collected is unreliable then it defeats the purpose.
* Sufficiency – Unless there is sufficiency of data further progress becomes uncertain.
* Effectiveness – The data must be effective/strong enough to indicate the situation being investigated.

Also, many a times different work situations for doing the same job become the deciding factor in arriving at the definition of Metric.

An example in this regard is consideration of various factors for assessing the performance of call center agents wrt time taken on calls, viz, AHT, talk time & the like, wherein the dynamics of one process may differ from the other & accordingly the metric for performance. For instance, an outbound calling process for collections is different from an inbound calling process for customer service & critieria for judging performance in each, though wrt time, have a different bearing per the call situation in each & cannot be done at the same level. This (metric) becomes all the more complex when two different operations are a part of the same process. For instance, outbound calling for collections & also inbound calling for collections (ie, receiving calls of customers who call to inform about payments made, deferred payment for any reason or calling in response to a left message by the outbound caller). In the latter (inbound calling for collections) the person receiving the call has to do the same job of collections but with a customer service approach where again the criteria for measuring performance wrt time factor would be different than an outbound collections call.

Another important point is to consider the CUSTOMER perspective as well while we are choosing and defining our metrics in a Six Sigma project. We should also choose the metrics that reflect important customer aspects apart from those that are only understandable and countable by the business/organization.

Further thoughts are invited from one & all.

Reply
Paul Hoernschemeyer
January 2, 2012 @ 12:47 pm

Comment in the Lean Six Sigma Canada discussion group on LinkedIn:

You don’t always get what you measure, but it improves the odds. It helps you decide the skills, tools, and motivations that need to be employed.

On the other hand, if you don’t measure what you want, odds are you won’t get it, and you surely won’t know it even if you do.

Reply
Robert Gordon
January 2, 2012 @ 12:48 pm

Comment in the Lean Six Sigma Canada discussion group on LinkedIn:

You get what you reward. But you have to prove by consistency that you will reward it. And to be consistent, you have to measure it. So, you get what you measure and consistently reward.

But people need to understand. You have to clearly communicate the point of the measuring, if you want to have a controllable response. The scale of increase will depend on the scale of reward. The stability will depend on the consistency of the reward.

You can get something just by saying you will measure it, even if you don’t. But that effect will not last long when you are seen not actually to be measuring. And you can get something for a little longer than that merely by measuring and not rewarding it.

The above remarks assume that you are trying to increase the output of something. Obviously, if you are measuring waste, you are likely to get less of that.

And you need to take time to be sure you will be happy with the increased output of what you are measuring. That is: Be careful what you ask for. As V.B. put it with technical precision above.

Reply
Daniel Lang
January 2, 2012 @ 2:49 pm

Comment in the PEX Network & IQPC – Lean Six Sigma & Process Excellence for Continuous Improvement discussion group on LinkedIn:

It is wrong to suppose if you cannot measure it, you cannot manage it – a costly myth. – Dr W Edward Deming

Reply
- Valerie Johnson
  January 2, 2012 @ 2:51 pm
  
  Comment in the PEX Network & IQPC – Lean Six Sigma & Process Excellence for Continuous Improvement discussion group on LinkedIn:
  
  aniel Lang, please elaborate. can you give an example of something you can manage without measuring? Would it be true that you could manage it but you would have no way of telling how well or if whatever you are measuring has improved?
  Sorry for the typos above, my keys are sticking. I agree with Daniel that the concept of measuring can be costly, but I think its because sometimes we become obsessed with measuring and metrics an focus on measuring and managing things that are not critical to quality. Time and money can be eaten up with those things.
  
  Reply
  - Daniel Lang
    January 2, 2012 @ 2:53 pm
    
    Comment in the PEX Network & IQPC – Lean Six Sigma & Process Excellence for Continuous Improvement discussion group on LinkedIn:
    
    There are many things that are unmeasurable. Some examples:
    * Loss of a prized customer. (this was Dr Deming’s favorite)
    
    * Customer satisfaction when you are losing marketshare
    
    * The loyalty of customers
    
    * The cost of not investing enough in R&D
    
    * The cost of leaving a known defect in a product
    
    * The cost of downsizing customer service or sales.
    
    Each of these issues must be managed for sustainability despite being unable to be measured.
Michel Baudin
January 2, 2012 @ 2:54 pm

Some might argue that, if you try hard enough, you can quantify these items. The real question is whether you should spend your energy on that or on solving the problems.

Reply
- Daniel Lang
  January 2, 2012 @ 2:55 pm
  
  Comment in the PEX Network & IQPC – Lean Six Sigma & Process Excellence for Continuous Improvement discussion group on LinkedIn:
  
  I would say trying to spend energy and time on unmeasurables would be a waste of time, and take you away from finding outside knowledge (knowledge that is not within a organization) and thus, problem solving
  
  .
  
  Reply
Roland Johnson
January 2, 2012 @ 2:56 pm

Comment in the PEX Network & IQPC – Lean Six Sigma & Process Excellence for Continuous Improvement discussion group on LinkedIn:

Just because you can measure (count) a characteristic does not mean you can control it. Suppose that you are counting the number of yellow cars going by? Is there any reason to expect that you can increase the percentage? No.

Conversely, If you don’t measure the characteristic there is no way to control.
Not all measurements are equal. for example, low tire pressure can show up as an observation of one tire being different from the others, it can show up as a tendency to pull to the left or right, or you can use a pressure gage and measure it directly. All are measurement; but all are not equal.
So, If you do measure you may not be able to control a characteristic but, if you do not measure you can not.

Reply
Gary McBain
January 2, 2012 @ 2:58 pm

Comment in the PEX Network & IQPC – Lean Six Sigma & Process Excellence for Continuous Improvement discussion group on LinkedIn:

Absolutely Michel……
Daniel.. not just so… especially in service organisations and especially public service organisations… These type of producers do not have the same cultural commitment to measurement and management as the private sector. Thus there is lots that could be measured that isnt, but should be! I have spent 15 years delivering operational research to the public sector, performance measures etc…. and there isnt the management commitment yet to deal with all of the issues.

So….. some extra effort in the area of measurement and subsequent management could deliver vast improvements in quality, cost reduction and waste reduction….and customer satisfaction!!

Roland… many many people try to control things without measurement. If you dont believe me then think of women to try to manage their weight without measurement, my wife is in that category bless her!!

Then think of all the people who try to buy healthy food without ever resorting to proper measurement. People do apply control without measurement…. although the engineer in me says its not an effective system.. but hey!!!

The yellow cars could be changed…. why couldnt it…. i think you mean that people measure without any of intention of managing the output or outcome…. ie pretend to be doing performance management………

Reply
Roland Johnson
January 2, 2012 @ 3:01 pm

Comment in the PEX Network & IQPC – Lean Six Sigma & Process Excellence for Continuous Improvement discussion group on LinkedIn:

Gary:
First, your wife has a measurement system – visual feed back from the mirror.
And Second, yes, counting yellow cars on the road has been used to generate real random inputs. Actually better than the random number generator in MS Excel. And Third, No, you can’t change or manage the output, the car color is not your choice and the decision to drive that particular morning is not yours. Again, measuring / counting yellow cars does not give you the ability to change the output.

The most important item here is that I am considering any feed back as a form of measurement.

You are right though, counting yellow cars is not performance management or process control. For it to be process control you must be in a position where you can take action to effect the process. Otherwise you are a witness and not a manager.

Reply
Robert Reid
January 2, 2012 @ 3:03 pm

Comment in the PEX Network & IQPC – Lean Six Sigma & Process Excellence for Continuous Improvement discussion group on LinkedIn:

@ Michel, self fulfilling prophecy is a strong element. The concept of be careful what you wish for comes into play when upper level managers send a signal downline of what numbers they want to see system/process alignments will often occur to provide exactly what is asked for…but the question is at what costs?

It is human nature to want to succeed, so when the goal is set the nature takes over. If the culture is driven by management by measure and the measure is of ill design, adding pressure to the mix of a flawed measure based leadership style merely increases a subculture that will often sub-optimize, lie, cheat, and do whatever it takes to deliver the demanded number/metric.

If your organization has a fairly mature CPI culture and you want to see this in documentation, look back on your organizations early wins (Low Hanging Fruit, LHF). What you are likely to discover is waste was eliminated by closing the gap of what is expected to happen to get a job done and how the job is executed. Specifically, LHF is achieved by shrinking the effects of a subculture that gets the job done at any costs, which is a cultural response to management by bad metrics.

Of course, there are significant advantages in using a CPI embedded control plan that monitors and prompts further analysis and action based on valuable metrics/measures. Often, the organization will best discern what those metrics are and how/when to gain them, as well as how to respond to them, based on the work of cross functional and bottom up/top-down hybrid team approach. One other caveat is the aforementioned team needs a highly qualified facilitator and sustained champion support (from a champion that can implement the changes required and lead cultural acceptance).

Reply
- Gary McBain
  January 2, 2012 @ 3:46 pm
  
  Comment in the PEX Network & IQPC – Lean Six Sigma & Process Excellence for Continuous Improvement discussion group on LinkedIn:
  
  What is wrong with “low hanging fruit”! Who would want to start picking apples from the top of the tree?
  
  Reply
  - Robert Reid
    January 2, 2012 @ 3:48 pm
    
    Comment in the PEX Network & IQPC – Lean Six Sigma & Process Excellence for Continuous Improvement discussion group on LinkedIn:
    
    LHF is great! I am stating to take a walk down memory lane if your organization has already harvested the LHF…be sure to see where your organization was before its current state if the organization has had a maturity progression in CPI.
William Heeb
January 2, 2012 @ 3:50 pm

William Heeb
Comment in the PEX Network & IQPC – Lean Six Sigma & Process Excellence for Continuous Improvement discussion group on LinkedIn:

I like to say nothing changes until you measure it. How do know if what you did worked? Simple by measuring it.

Reply
- Gary McBain
  January 2, 2012 @ 3:53 pm
  
  Comment in the PEX Network & IQPC – Lean Six Sigma & Process Excellence for Continuous Improvement discussion group on LinkedIn:
  
  William – Things do change if you don’t measure them!! Both intentional and unintentional changes…………
  
  Secondly you can know if its changed without measurement (well without precise, repeated and standard measure measurement), its called sensing (though i agree its not very good).
  
  Reply
Tina Columbine
January 2, 2012 @ 3:55 pm

Comment in the PEX Network & IQPC – Lean Six Sigma & Process Excellence for Continuous Improvement discussion group on LinkedIn:

I recall the correct quotation to be “What gets measured, gets done”. The question is how do you know if it has been improved if you don’t measure from the original state to the current one. At the very least an improvement must be shown to upper management so they can be knowledgable about business decisions. There is a small percentage of the time that the reciprocal will work. Would be an interesting research project using six sigma theories.

Reply
Alessandro Gambara
January 2, 2012 @ 8:17 pm

Alessandro Gambara
Comment in the Schlumberger discussion group on LinkedIn:

I guess it is much probable to “measure what you get” instead :).

Reply
- Michel Baudin
  January 2, 2012 @ 8:19 pm
  
  I agree that it is much more reasonable to quantify the results you are getting than to try to drive activities through measurements.
  
  Reply
  - Alessandro Gambara
    January 2, 2012 @ 8:19 pm
    
    Comment in the Schlumberger discussion group on LinkedIn:
    
    Now, I must say Michel, that things sounds different. It is quite reasonable in fact to drive activities thru measurements but the point stay that you can only “measure what you get”.
Kevin Fellon
January 9, 2012 @ 9:50 am

Comment in the APICS The Association for Operations Management discussion group on LinkedIn:

In my experience, you start by measuring what you believe to be critical to your success. Then your teams will begin to push to meet expectations on the metrics you are using. It is a bit of a self fulfilling prophecy. If you accurately assessed the importance of the metrics, you are good to go. If not…

Reply
Laxman Marathe
January 9, 2012 @ 9:54 am

Comment in the APICS The Association for Operations Management discussion group on LinkedIn:

Hello Michel – Actually all work as they do but the moment a particular measure is made then we emphasis on this metrics and in a way tell all that this is what we are looking for. So, if your measure is intelligently formulated you can expect correct response else may prove counter-productive.

Let me give you an actual case: We are a commercial printer. We initially thought the output of any Press should be what has finally been created at the end of the shift or day – basically what is the net on pallet (NOP) output of the press. So, NOP became a measure. As you know operators world over are smarter than Managers who claim to control them. They said OK if that is what you are looking for then I give you a high NOP. so, anything & everything that could quickly & easily add increase NOP was printed irrespective of whether it was required or not. So, what was the en result we landed up with huge work-in-progress.

Morale: measures are important to know the health of operations but they must be very carefully selected.

Reply

Is it true that you get what you measure?

Like this:

Related

35 Comments

Leave a Reply Cancel reply

Follow Blog via Email

Recent Posts

Categories

Is it true that you get what you measure?

Share this:

Like this:

Related

35 Comments

Leave a Reply Cancel reply

Follow Blog via Email

Recent Posts

Categories

Social links

My tags