← back to the rack
// learning from failure · how we done do that AI

you can be bad at this

an honest account of AI-assisted work, that reviews itself as you read it.

reading along: the peer · checks the facts the bastard · catches the shape the meta · catches it twice

I've worked with AI tooling daily for a few months, cross-repo features, developer tooling, docs. the most useful thing I can tell you about it is that you can be bad at it. what follows is the system I use to be less bad, deliberately.

a talk about catching your own blind spots that doesn't catch its own is just hype. so this page does the thing it describes: the two reviewers I ran on the files that produced it are reading along with you. their notes land in the margins. i didn't soften them.

01I keep notes on what works and what doesn't

the core of it is a mistake log. append-only, seven weeks old, 54 entries, two working cycles of rule promotions. alongside it sit 45 memory files, each naming a specific failure mode with a why and a how-to-apply.

seven weeks isn't a long-running practice. that's the point, it paid off this fast because the logging is disciplined, not because i've been at it for years.

02the loop: plan → check → review → do

most of my time now sits at the planning end of that loop. i plan maybe 90% of the time now; the implementation falls out of a good plan and has become the cheap part.

the counter-example keeps me honest: the one time i skipped the planning step on a rewrite, i paid a whole wrong-architecture round for it. planning isn't immunity to a bad assumption. it's just where the leverage is.

so how do we actually do it? two reviewers. one of them a bastard. watch.

03the bastard review

that "review" stage is two independent passes on the same work: a competent peer who checks every fact, and an annoying colleague briefed to catch what i'm hiding. neither sees the other's notes. here's that pair, run live on the four files that produced this talk.

exhibit a · a post-mortem
self-review · draftv1

across 32 daily notes this quarter i consistently caught my own mistakes early. this isn't a highlight reel, i included the failures too. the process is working as designed.

③ meta-review · the reckoning
the peer caught 2 things that are wrong: a count, a word. the bastard caught 3 things that aren't wrong, they're hollow.
they overlapped on 0
↳ near-zero overlap is usually the signal, each catches what the other can't. the margins of this page are that same pair, running on me.

the value is usually the gap between them, each catches what the other can't. but sometimes the peer and the bastard land on the same finding, and that isn't the process failing. it's the loudest signal there is: the thing is so plainly broken that both of them caught it. fix that one first.

it cuts both ways, though: on one dated morning my own bastard pass got a few things wrong, and the fix was eight grounding gates, now baked into the skill.

04where the methodology stops

if a methodology talk has no section on where it stops working, it's a sales pitch. mine has one because the data has one. even hook-blocked rules recur: one habit fired eight times in five weeks despite a hook that catches every single instance. the hook fixes the symptom; the reflex underneath is unchanged.

the meta · it demonstrates itselfthree of those eight fired while this talk was being written. the claim proved itself three times during prep. about as clean as evidence gets.

05what I haven't measured

cost in tokens and money: unmeasured. a real non-AI baseline: directional only. the false-negative rate of the mistake log: unknown, five self-catches went unlogged on one day alone.

i'd rather name the gaps than have the room name them for me. and the deepest gap is the one this page is built on, an audit produced by the method can only ever catch what the method already sees. saying that out loud is what turns the weakness into the frame.

you can be bad at this

and then you can get less bad, deliberately, with a system that catches what you missed.

↳ that's the whole talk, including the part where it caught itself, on this page, while you watched.

snap