The Only Metric That Matters: Are We Re-Explaining Less?

When you build a tool meant to preserve project memory, the obvious metrics to reach for are activity metrics: how many decisions got logged, how many searches got run, how many handovers got written. They're easy to measure and they go up and to the right in a way that feels reassuring. They're also, I've come to think, close to useless as a signal of whether the tool is actually doing its job — because a project can log plenty of decisions and still be failing at the actual point of the exercise, which is not having to explain the same thing twice.

The metric that actually matters

The signal I trust now is much simpler to state and much harder to measure cleanly: how often are we re-explaining something we've already explained before, to a tool or a person who should already know it? Every time that happens, it means the memory failed at its one job, regardless of how many entries are sitting in it. A project memory with a thousand recorded decisions that nobody actually consults before repeating a settled argument is not succeeding by the metric that matters, no matter how impressive the entry count looks on a dashboard.

This reframes what "working" means. It's not about the volume of memory captured. It's about the reduction in redundant explanation — a quieter, harder-to-graph, but much more honest signal.

Where re-explanation actually shows up

Once I started paying attention to this specifically, instead of to activity counts, it turned up in a handful of recurring situations, each worth naming separately because each has a slightly different fix:

Onboarding. When someone new joins a project — a person or a fresh AI session — how much do they need to be told before they can contribute usefully? If the answer hasn't shrunk over time despite a growing body of recorded project memory, the memory isn't actually reducing onboarding cost, whatever else it might be doing.

Recovery after interruption. A project gets set aside for a few weeks and picked back up. How much re-orientation does it take before work can resume at the same quality as before the gap? This is one of the clearest tests, because the gap makes the cost of poor memory impossible to hide behind momentum.

Decision re-litigation. Someone proposes something that was already decided and explicitly rejected, with reasons, months ago. If that keeps happening, the reasoning behind past decisions isn't actually reaching the people — or tools — making new proposals, regardless of whether it's technically recorded somewhere.

Defect investigation. When something breaks, how much of the investigation is spent reconstructing what the system was supposed to do and why, versus actually diagnosing what went wrong? A project with genuinely working memory spends its investigation time on the second question, because the first is already answered and easy to find.

Why this is hard to measure cleanly, and why I'm doing it anyway

I'll be direct about the limitation here: "are we re-explaining less" is not a clean, automatically-collectible number the way "entries logged" is. It requires someone to actually notice when re-explanation happens and track it, which is itself a discipline that can slip. I don't have a fully automated way to measure this, and I'm suspicious of anyone who claims they do without describing exactly how they're distinguishing genuine repeated explanation from a legitimately new question that merely sounds similar to an old one.

What I've done instead is treat it as a qualitative check, revisited deliberately rather than tracked continuously: periodically asking, honestly, whether recent friction in the project traces back to information that existed somewhere but didn't reach the person or tool that needed it, versus friction from something genuinely new. That's a much softer signal than a dashboard number, and it depends on being honest with yourself in a way that's easy to fudge when you want the tool you built to be working.

Why activity metrics actively mislead, not just fail to help

It's worth being direct about why I moved away from activity metrics rather than just adding this one alongside them. A rising count of logged decisions can happen for a reason that has nothing to do with the memory working — it can simply mean more decisions are being made, or that logging got easier so people log more of the same underlying behaviour, or, less charitably, that the metric itself became a target and people started logging things performatively because the count was being watched. None of those explanations mean the project is re-explaining itself less. They just mean a number went up.

This is a fairly ordinary version of a well-known problem — once a proxy measure becomes the target, it stops reliably tracking the thing it was a proxy for. I don't think activity metrics are useless; they're a reasonable signal that the tool is being used at all, which is a real prerequisite for it helping. But "being used" and "reducing redundant explanation" are different claims, and conflating them is exactly how a team ends up confidently pointing at a healthy-looking dashboard while everyone on the project can feel, if they're honest, that the same conversations keep happening anyway.

What I'd still call a hypothesis rather than a proven result

I believe the re-explanation metric is the right one to chase, based on what it's felt like to work inside projects where it's improving versus projects where it isn't. I don't have a rigorous, quantified before-and-after study proving that belief — what I have is direct experience and a conviction strong enough to keep building around it. That's a meaningfully weaker claim than a proven result, and I'd rather state the difference plainly than let "the metric that matters" read as more validated than it currently is. The measurement problem itself — how to track something as a quiet absence rather than a countable presence — is still open, and it's the part of this whole approach I think about the most without yet having a satisfying answer.