66 Tokens Make a Diffusion Language Model Look Easy

A diffusion language model generates text by starting from masked or otherwise corrupted tokens and iteratively restoring them. In this MacBook Air M2 demo, that idea shows up in its smallest, most hackable form: a toy character-level model that learns to recover missing characters from Karpathy's t