The Resolution Library

On what actually goes into the learning loop

A few people wrote back after the last post asking a version of the same question: okay, the learning loop but what goes into it?

Nadella describes the architecture beautifully. Private evals, fine-tuning environments, a retrieval layer over institutional memory. The system that compounds over time, the one that stays yours even when you swap the underlying model. All of that is real and worth building.

But there’s a step before all of it. Before the system, before the retrieval layer, before any of that infrastructure - someone has to capture something worth capturing. And that part is harder than the tooling, because it can’t be automated. It might be the one thing in this whole conversation that genuinely can not.

What actually gets lost

I’ve been working in consulting long enough to watch a lot of knowledge disappear.

A project closes. The people on it move to the next engagement. The lessons, the ones that actually mattered, the client dynamic that almost tanked it, the architecture choice that turned out to be right for reasons nobody fully understood at the time - none of that makes it into the retrospective. The retrospective covers what shipped, not why things went the way they did.

Months later, a similar project starts. Similar client, similar configuration, similar early warning signs. Nobody connects the dots, because there are no dots to connect. The knowledge was there. It just wasn’t in a form anyone could use.

I don’t think this is a consulting problem specifically. I think it’s what happens by default when organizations are moving fast and capture is always someone’s lowest priority.

The decisions that don’t reduce to process

There’s a specific category of knowledge that’s hardest to hold onto.

It’s not the process knowledge: how to run a discovery workshop, how to structure a go-live plan. That gets documented, eventually. It’s the judgment knowledge. The calls that weren’t obvious. Whether to push back on a client scope request or absorb it. Which early signal in a project actually predicts trouble six months out. Why a particular customer relationship held through a difficult moment when others didn’t.

Ask someone who made one of those calls well why it worked. You’ll get an answer, but it won’t be an algorithm. It’ll be a story with context: what was happening at the client, what the options actually were, what they decided to discount and why. That story is the thing.

A language model can summarize a decision after the fact, but it can’t reconstruct the judgment that was present at the moment of it - the uncertainty, the alternatives that were seriously considered and dropped, the information that was available but turned out not to matter. That exists only in the person who was there, and only briefly.

The idea I’ve been testing

So I’ve started doing something I’m calling a Resolution Library, though “library” makes it sound more organized than it is right now.

The practice is simple: when a significant decision closes, write it down while it’s fresh. What was happening, what the options were, what got chosen and what got rejected, and crucially, the reasoning at the time. Not the polished version, but the actual thinking.

Then, when the outcome is clear, sometimes a month later, sometimes two years, go back and close the loop. What happened? Where the reasoning held and where it didn’t.

That’s it. No special tool. It’s a habit, not a system.

The honest reason I’m writing about this rather than presenting it as a solved method is that it’s genuinely hard to keep up. There’s no immediate return, the discipline it requires is dull, and there’s always something more urgent. I’ve missed entries I wish I had. I’m not holding this up as something I’ve perfected, more as something I’ve become convinced matters enough to persist with despite the friction.

Why this is Nadella’s argument taken to its source

The previous post ended with the question: how does your company accumulate experience through AI in a way nobody else can copy?

The Resolution Library is one attempt at an answer, specifically for the kind of experience that’s hardest to commoditize. The judgment knowledge. The record of what happened at the actual decision points, before and after.

Nadella’s test for whether you have a real learning loop is whether you can swap out the model without losing the company-veteran expertise built into the system. That expertise has to come from somewhere. It doesn’t emerge from the volume of transactions or aggregated usage data. It comes from the reasoning behind the calls that actually shaped outcomes and captured at the moment they were made, in a form that can be used.

There’s a piece by Usman Sheikh at Framebreak that frames the same stakes through Federer: he won 54% of the points he played and nearly 80% of his matches. A small edge, consistently held at the right moments, produces a dramatically different aggregate result. Not every point matters. Win the ones that do.

That’s the selection problem the Resolution Library is trying to solve, capturing the decisions that actually determine outcomes, which is a much shorter list.