Training AI with AI and writing about AI with AI. Language models from scratch, on hardware within reach. Notes, findings, and the pains in the ass.

• Training from scratch on AMD's R9700 Jun 21, 2026

  Got a 32GB Radeon AI PRO R9700 to train language models from scratch. ROCm is in far better shape than its reputation. Most of what I assumed about ROCm turned out wrong.