30 days later: numbers, mistakes and next steps

Published: November 16, 2026
Reading time: 7 min
Series: lemon-code in 30 days, part 13 of 13

No model was promoted. That's the most important line in this wrap-up, and it comes before any nice number.

The snapshot is October 16, 2026. Commit counts and dates come from each repository's HEAD; file counts reflect paths present in the local trees that day. These commands reproduce the Git figures:

git rev-list --count HEAD
git log --reverse --format=%ad --date=short | head -1
git log -1 --format=%ad --date=short
git log --format=%s | grep -c '^feat'
git log --format=%s | grep -c '^fix'

TL;DR

• Numbers pulled from git and docs on 2026-10-16.
• CLI: 448 commits, 161 test files and 548 .ts/.tsx files, version 0.1.45. App: 152 commits. Model lab: 268 commits, 34 specs, 64 evidence docs and 144 test files.
• Recorded mistakes: duplicate ADR numbers and an evidence failure that became a fix.
• Next: run the coding eval in lemon-model before claiming a better model.

The intuition: the bank statement, not the credit card bill

Think of a bill versus a statement. The bill shows what you choose to pay now. The statement shows everything that went through the account, including what you'd rather forget. Project retrospectives are usually bills. This one tries to be a statement.

The numbers

Commit counts and dates come from HEAD; file counts are a snapshot of paths present in the local trees on October 16, 2026.

61 CLI commits landed since September 16. By message prefix, 177 start with feat and 78 with fix; the other 193 start with neither.

The decisions

The CLI has 67 numbered ADR documents across 57 distinct IDs, up to 0060. There are collisions: ID 0049 appears in six documents; 0050, 0051, 0052 and 0055 repeat too. Parallel authors each looked at the folder and picked the next number they saw as available.

The model lab evidence

• The reward-hacking detector zeroes answers that game tests: 33 tests passing.
• The holdout is scored once per candidate: 10 tests passing.
• The forgetting guard blocks drops beyond tolerance: 20 tests passing on example answers, not real model outputs.
• The promotion gate requires 200 paired tasks and a statistical test. Its tests use fixtures; no trained candidate has passed it.
• The current holdout has two tasks toward the 200 required; no real model has been evaluated on the coding suite.
• In the QLoRA pilot, validation loss went from 0.893 to 0.881 over six steps on one validation row. Base versus adapter on that row was 1.0660 to 1.0482; neither result measures generalization.
• Source verification for the training data remains pending; its license is NOASSERTION, and the data and adapters remain private, local and ineligible for promotion.

The README still says it: coding-quality evaluation and a production model are pending.

Recorded mistakes

1. Duplicate ADR numbers. Fixing means renumbering and updating references. Numbering needs a single source, not "look at the folder".
2. Incomplete commit convention. 193 of the 448 CLI commits start with neither feat nor fix.
3. Pilot result on a minimal sample. One comparison example isn't evidence of a better model.
4. An important ADR still proposed. The sandboxed tool execution ADR (0024) is still proposed.

Next steps

• Complete the holdout to the required 200 paired tasks and evaluate a real candidate; record the result without assuming promotion.
• Resolve duplicate ADR numbering.
• Review proposed ADRs such as 0024 and accept or drop them.
• Keep measuring the app and CLI together; their integration deserves its own attention.

Checklist: is your retrospective a statement or a bill?

• Every number has a command that reproduces it.
• The snapshot date is written down.
• Mistakes get the same prominence as wins.
• What isn't validated yet is said in those words.
• Next steps come from mistakes, not wishes.

Conclusion

Thirteen posts later, lemon-code is a 448-commit CLI, a native app on top and a model lab with more gates than models. The numbers don't say it worked. They say exactly what was done, what broke and what's missing.

Thanks for following the series.

If you want to build a coding agent, an app over a CLI or a model evaluation process with this level of record keeping, I help with that work at LEMON.DEV.

References

• git rev-list
• Conventional Commits
• Documenting Architecture Decisions, Michael Nygard

Related reading

• Why I built my own coding agent
• ADRs as project memory for multiple agents
• Evals before hype