Proxy Health Check
Is this signal still measuring what we think it's measuring?
Quick diagnostic
Run any signal your organization treats as load-bearing — a credential, a review score, a proposal — through these four questions.
| 1 | Does this signal substitute for something too expensive, slow, or far in the future to observe directly? |
| 2 | Did producing the signal use to require possessing the thing it indicated? |
| 3 | Can the signal now be produced without that underlying thing — has something made it cheaper to fake? |
| 4 | Is there an actual scalable way to check the real thing, or are you still relying on the same instrument and hoping it holds? |
Confirmed case
- ✓ Question 1 — Yes
- ✓ Question 2 — Yes
- ✓ Question 3 — Yes
- ✗ Question 4 — No scalable alternative
Result: the signal is no longer reliably measuring what it claims to measure.
If any answer breaks that pattern — a "no" on 1–3, or a real scalable alternative on 4 — this isn't a confirmed case. Treat it as a maybe, not a verdict.
Response
Substitute
A scalable alternative exists or can be built. Replace the signal with direct observation.
Accept
No scalable alternative exists, but the signal still discriminates well above some threshold. Accept the limit explicitly. Some institutions have responded to exactly this by formally randomizing decisions at the margin the signal can no longer resolve — an honest admission of where judgment runs out, not a failure to plan for it.
Rebuild
No scalable alternative exists, and the signal no longer discriminates at all. Detection alone — catching fakes — treats the symptom. Rebuilding the instrument around a different bundle treats the cause.
Why this works
Organizations rely on signals to stand in for things too expensive, too slow, or too far in the future to check directly — a review score for research quality, an essay for analytical skill, a credential for capability. These substitutions work for a specific reason that's easy to overlook: producing the signal used to require possessing the real thing. The difficulty of faking it was doing the verification work, for free, without anyone having to design for it.
That arrangement can fail silently. Nothing about the signal's appearance changes when it stops working — which is exactly why organizations keep relying on a degraded instrument long after they know it's degraded. Not from denial. From the absence of anything scalable to replace it with.
The question isn't whether your organization has a signal like this. Almost every organization evaluating anything at scale does, and always has. The question is whether you know which of your signals were ever actually bundled to what they claimed to measure — and whether you'd notice the day that stopped being true.