LanguageDiffusionModels

Apply DiffusionModels to chunks of language. Eg. take a chunk of language, mask multiple tokens with a special masking character, and then have a diffusion-like model turn the masks back to tokens.

This is as opposed to the typical Transformers which are trained on predicting the next tokens in a sequence based on the earlier ones.

https://arstechnica.com/ai/2025/02/new-ai-text-diffusion-models-break-speed-barriers-by-pulling-words-from-noise/

InceptionLabs are making MercuryCoder : https://www.inceptionlabs.ai/news

CodeFusion was an early attempt