What blocked it: MLX peak isn't Mac memory

The planned 7B profile: batch 1, gradient accumulation 2, 16 layers, rank 8, gradient checkpointing, a declared 14 GB peak cap. The problem was another limit: lemon-model stops training when free system memory falls below 15%. On an 18 GB Mac, the OS, the browser and everything else share unified memory. Two 7B attempts hit that cutoff while MLX's peak was still under 14 GB. The framework's number is only part of it; the sum is what takes the machine down.

The pilot that passed

Parameter	Planned	Pilot
adapted layers	16	8
rank	8	4
batch	1	1
gradient accumulation	2	2
max sequence length	2048	4096
gradient checkpointing	yes	yes

Data: 6 training examples and 1 validation example from a curated set of my own repos, 14,066 planned training tokens. Three examples over 4,096 tokens were dropped whole, not truncated.

Recorded result: 6 iterations in 199.87 s, about 130 training tokens/s, 11.139 GB peak, validation loss on the single validation row of 0.893 at step 1 and 0.881 at step 6, and an 11.5 MB adapter. On that row, loss went from 1.0660 (base) to 1.0482 (adapter).

A follow-up expansion recorded six trainings with different seeds and settings, all under the 14 GB cap. One, seed 7, reached 0.883 validation loss with an 11.58 GB peak. These remain development-training results, not evidence from the holdout.

The cost of using the adapter

Same generation with and without the adapter: time to first token went from 8.4 s to 9.4 s and tokens/s dropped 10.9% (30.6 to 27.3). The machine was already under memory pressure after training, so treat it as an estimate.

What these numbers don't say

• A comparison on one validation example doesn't measure generalization.
• These pilots include no code-quality evaluation and no clean holdout.
• No adapter was promoted.
• Source verification remains pending, the license is recorded as NOASSERTION, and the dataset and adapter remain private, local and ineligible for promotion.

The record shows that this bounded training run finished on the Mac and produced a hashed adapter. It doesn't establish cleared data rights, passing code tests, quality, generalization or production readiness. Quality is the next post.

Three protections I recommend

• Bounded execution. Training runs as a subprocess with a clean offline environment, an empty temp dir and time and memory kill switches that record the breach.
• Data gate. Training examples leaking from the holdout set are rejected before training.
• Quantization parity. Serving the adapter on a base with different quantization than training is forbidden without re-evaluation, enforced by a test.

Checklist: before training LoRA on your Mac

• Watch free system memory, not just the framework peak.
• Start with few layers and low rank; scale up later.
• Use gradient checkpointing and batch 1.
• Drop long examples whole instead of truncating.
• Record hashes for base, adapter and data.
• Measure the adapter's latency cost.
• Don't call a loss drop on one example a quality gain.

Conclusion

In this profile, a LoRA run over a quantized base completed on an 18 GB Mac when I watched whole-system memory, not just the framework's. The pilot is small on purpose: 8 layers, rank 4, 6 examples. It records one run; it doesn't show that the model got better at code. That needs a different tool.

Friday: evals before hype, and why lemon-model blocks promotion of any model that doesn't pass them.

If your team wants to adapt open models with its own data without sending code out, I help build bounded, traceable workflows at LEMON.DEV.

References

• LoRA: Low-Rank Adaptation of Large Language Models
• QLoRA: Efficient Finetuning of Quantized LLMs
• mlx-lm LoRA documentation

Related reading

• A local coding model on the Mac
• Self-hosting open weights and VRAM