IndexesBecomePolicyTargets

ThoughtStorms Wiki

Very important correlate of WhatYouCanMeasureYouCanManage.

When you start trying to understand a system, you start trying to measure it. And you create indexes ie. summaries of the state of the system.

Obviously, these indexes don't include all the information. Nevertheless, as you start trying to control the system, you measure your success by the changes in the indexes. Your indexes become policy targets.

Adjustments which improve the indexes will be selected over ones that don't.

The danger is, that sometimes the indexes can be improved by techniques that don't actually improve the underlying system. They may actually be covering up something going wrong underneath.

Basically follows from TheMapIsNotTheTerritory

Bad examples of indexes becoming policy targets :

yield becomes the measure of agricultural success (GeneticModification, GreenRevolution)

exams become the measure of education which is suppressing intelligence (JohnTaylorGatto)

Counter

CriticalRationalism : all knowledge consists of conjectured models. Nothing you can do about that.

Counter-counter

Exactly, we have to live with this situation, and that's why we should be especially attentive to criticisms of this kind : "my indexes show things are OK", "well let's look beneath your indexes" should be a standard move in many arguments, and equally acceptable to both sides.

Why does it happen?

deliberate misleading : "let's make up indexes that say what we want"

mistakes

more importantly : SuccessChangesTheGame, yield was useful compared to other indexes in the past, now the game is changed and it's failures (to record the degredation of BioDiversity or promote mutually supportive complementary species) starts to outweigh it's virtues.

Peacock's tail / HandicapPrinciple : sometimes an index gets LockedIn due to several actors : the male peacock needs the tail to appeal to the female. The female takes it as a genuine sign of fitness in the male. Neither wants to change their use of it if the other hasn't. The HandicapPrinciple compounds the problem. The costly signals are more succesful than non-costly ones. If they turn out to be bad indexes which we're locked into, we are continuously incuring this cost.