LanguageDiffusionModels

Apply DiffusionModels to chunks of language. Eg. take a chunk of language, mask multiple tokens with a special masking character, and then have a diffusion-like model turn the masks back to tokens.

This is as opposed to the typical Transformers which are trained on predicting the next tokens in a sequence based on the earlier ones.

https://arstechnica.com/ai/2025/02/new-ai-text-diffusion-models-break-speed-barriers-by-pulling-words-from-noise/

InceptionLabs are making MercuryCoder : https://www.inceptionlabs.ai/news

CodeFusion was an early attempt

Embedding failed with embed type :twitter

and data
{:type :twitter, :url "https://x.com/karpathy/status/1894923254864978091", :title "", :caption ""}
java.lang.Exception: JSON error (unexpected character): <

Contrast :

• TransfusionModel where a language model drives image generation.