Between Diffusion Language Models and Autoregressive Models
Exploring the latest research on diffusion language models, data efficiency, and hybrid approaches with autoregressive models.
LLM
Do AI Systems Dream of Diffusion Language Models?
Explaining the differences between autoregressive models and diffusion language models from a B/F (Bytes per FLOP) perspective in LLM inference.
Do You Know About NanoGPT Speedrun?
And do you know about Muon?
When Will the Era of 1-bit LLMs Come, Or Will It?
Exploring the latest research trends in 1-bit LLMs and the accuracy reversal phenomenon through extreme quantization.