"When a measure become a target, it ceases to be a good measure"

"When a measure become a target, it ceases to be a good measure"

By Christian Fournier,  former Head of Finance Europe at Orange Business Services

FP&A Tags: 

measurementHere are two laws that should question a few paradigms in company management and finance.

Goodhart's law "When a measure become a target, it ceases to be a good measure"

Campbell's law "The more any quantitative social indicator is used for social decision-making, the more subject it will be to corruption pressures and the more apt it will be to distort and corrupt the social processes it is intended to monitor."

Applied to company management those two laws bring few questions. In practice, there are quite a number of measurements that are used to measure “recognition” both externally i.e. shareholders, finance analysts, investors … and internally i.e. Management and sales in particular but not only. If we believe in those laws, it implies that we invite fundamental flaws in the company ecosystem. It means that each time a form of recognition is in play (i) the way it is measured tend to be “corrupted” and the measure becomes… less valuable (ii) the ways recognition systems works are polluted by the way you measure it.

We need both tools

It is difficult to exclude motivation as a driver for better performance individually and collectively and thus recognition systems. Organisations such as administrations that have no or little recognition systems usually demonstrate lower performances. The more employees and managers are in some form of recognition system the better is the company performances (all other things being equal). Obviously, some extreme systems of recognition have medium or long-term human consequences that are negative. This is the opposite of no recognition system.

Still, measurement tools need to be kept as meaningful as possible in order to manage the company effectively. Corruption or manipulation of figures are “fraud” even if (in most cases) kept under the threshold where they become punishable. Furthermore, it creates an unethical culture.

We need both so how do we practically ensure that the effects of those laws are reduced and kept under control? How do we organise data collection and administration that are “resistant” to human manipulation? How to organise a recognition system that is both humanly acceptable and promote ethical behaviour?

What needs to be done on the measurement side

Accounting and audit regulations, practices and compliances offer when correctly applied a reasonable framework (we still see from time to time GAPP/IFRS changes to cover some wide potential deviances). Still, performance measurements are not limited to pure finance parameters. Order or contract booking (for sales in particular), quality in manufacturing or customer services are non-finance figures that do not necessarily come with the same level of compliances but are a basis for objectives. Furthermore, there are elements that are not measured through figures (strictly speaking). Customer and employee satisfaction are good examples.

Not only but in particular if they are used by some form of recognition systems, all form of measurement (more exactly all data used by) needs to be documented, controlled and managed by a team that is different than the one that are measured e.g. If salespeople collect, control, register and report their own orders whereas orders are the base for their commissions and their managers bonuses, there is “an open door“ to have the salespeople … “managing” their commissions and their managers expectations while defining targets. If a separate sales administration department manages the orders the risk is largely reduced in particular if the rules applied by this function has been “synchronised” with finance/internal audit.

Separation of responsibility and proper documentation on process and data management and quality are a key element to limit the two laws effects.

What needs to be done on the recognition side?

The design and organisation of the recognition systems is also an area where strong documentation is needed. Not just the system as such but also how it is supposed to work (measurement, arbitration, what… if/hypothesis/scenario documentation, business as usual versus exceptional events, evaluation and evaluation control, …). It ideally defines the ethics to be applied both by the manager and the person concerned. One element of it is whether the objective reflects an obligation of means or an obligation of results or a mix. In other words, is it the result that is rewarded or the effort developed to reach it or a mix? It shall be clear and understood that it will induce quite different behaviours.

More importantly, it shall be “tested” i.e. whether it shall bring the expected results and behaviours and what are the possible deviances. An objective that would be to “achieve an order intake of X € and revenues of Y €” might favour revenues against profitability. One way to balance it would be “an order intake of X € with a maximum average discount of x% and revenues of Y € with an average margin of y%”. Those tests shall also ensure that you are not creating Simpson paradoxes. See my article in FP&A trend. https://www.fpa-trends.com/article/fpa-analytics-and-simpsons-paradox

This test is not only at the individual level but also at the collective one. The sum of distributed objectives needs to drive (i) in the same desired direction (ii) cover all fields/aspect of the company in a coherent way.

A focus that is limited to too few items generates distortions at the medium and long term. Deviances examples would be:

  • Revenues versus profitability,
  • Delays/speed versus quality,
  • New versus mature products,
  • Cash versus investments and/or inventories,
  • Profitability versus customers and employee’s satisfaction,

Each example can be seen the other way i.e. profitability versus revenues, etc.

The fine tuning is then extremely important and not obvious. Generally, it works top down i.e. Each N+1 is a break down of the N level. Unfortunately, N0 or N0 and N1 targets/objectives do not necessarily capture all necessary objectives. The break down of those objectives to N2, N3, N4, … levels may leave aside objectives that are important at the medium and long term.

Target definition: the key issue?

If measuring the actual performance may be an area of issue, target definition is even more sensitive. If your system of recognition is strictly linked to the forecast tools used, it will have strong impacts on the way forecast are built. If they are somewhat disconnected, impacts might be less important. Still at the end let us be clear, a target means a forecast of some sort and a measurement. This applies to both figures and KPI’s.

The natural individual attitude would be to minimize the forecast in order to improve the chances to achieve or overachieve the recognition whereas the collective attitude whole be to maximise forecast in order to maximise the potential output. The whole forecast process instead of focusing on business and competitive intelligence focus on managing expectations in order to maximise recognition. In extreme case, it can turn into endless target negotiations and definitively delay the forecast finalisation (increase its length).

Solution?

I do not think there is a single and simple solution. It is highly dependant on the company business and situation but also on its culture.

So, I can only give common sense recommendations.

  • Have a strong (process and system wise) data management not just on finance but in all domains, do not accept “compromises” with those rules,
  • Implement as far as possible separation of duties/responsibilities between those who manage the (actual and forecast) data and those who are incentivised,
  • Document and test your recognition systems (in addition to the general recommendations concerning targets i.e. be fair, achievable, ….),
  • Keep a reasonable balance between fix and recognition,
  • Make sure your forecasts are driven by business and competitive intelligence.
  • Banish wishful thinking. Make sure that scenarios and hypothesis are clearly documented.
  • Ensure high coherency in the management suite (actuals, analysis, reporting, forecasting) whether it is financial or none financial information. Ensure it is fitting to your own business natural complexity.
  • Keep in mind Goodhart's and Campbell's laws when designing and managing your suite and recognition system. Regularly analyse or audit those different processes, identify deviances, communicate on results and implement corrective actions.

In summary, do not mess with those laws but provided those recommendations are implemented they are not a fatality.

The full text is available for registered users. Please register to view the rest of the article.