Goodhart's Law

Goodhart’s Law: Once a measure becomes a target, it ceases to be a good measure. ^def

Optimizing for a single metric or set of metrics often leads to tradeoffs and shortcuts when it comes to everything that isn’t being measured and optimized for (a well-known effect on Kaggle, …

… where winning models are often overly specialized for the specific benchmark they won and cannot be deployed on real-world versions of the underlying problem).
In the case of AI, the focus on achieving task-specific performance while placing no conditions on how the system arrives at this performance has led to systems that, despite performing the target tasks well, largely do not feature the sort of human intelligence that the field of AI set out to build.

Link to original

You achieve what you target, at the expense of everything else.

Examples

See:

• Goodhart’s Law
• Too much efficiency makes everything worse - overfitting and the strong version of Goodhart’s law