Lei Yang's DDN reached ICLR 2025 by keeping the closest of many samples

Mainstream generative models approximate a distribution with one output per input, which makes their machinery indirect: iterative denoising, adversarial training, or elaborate conditioning. Lei Yang's Discrete Distribution Networks (DDN) start from a different position — network features already carry distributional information, so a network that emits multiple sample points at once may represent a distribution more directly, including continuous ones.

DDN stacks Discrete Distribution Layers. At each layer the network generates K candidate outputs, but the loss is computed only on the one closest to the ground-truth sample; that winner feeds the next layer, so depth expands the output space exponentially and each layer refines the previous guess. A Split-and-Prune optimizer prevents dead nodes and density shift. At inference the 'pick closest' sampler is replaced by random choice, which generates new samples and gives zero-shot conditional generation without gradients — text-to-image via a black-box CLIP model — plus a 1D discrete latent.

External validation followed quickly: ICLR 2025 accepted the paper with reviews calling it 'novel and elegant' and 'a whole new method, very different from any of the existing generative models', and experiments covered CIFAR-10 and FFHQ. The model was built in under three months, mostly by one person, and the launch page with code, demos and an interactive coloring demo reached #2 on Hacker News's daily rankings.

Emit many samples; feed the closest winner to the next layer

Training only the closest of K candidates focuses every update on the output that matters, so the gradient cost stays bounded.

Feeding the winner to the next layer turns depth into exponential refinement: coarse-to-fine accuracy without extra supervision.

Swapping the guided sampler for random choice at inference delivers zero-shot conditional generation with no gradient, a capability other models need separate machinery for.

Validation was external and cheap: ICLR peer review, public code and demos, and a top-2 Hacker News launch rather than only the author's own benchmarks.

Instead of forcing one network output to be perfect, let the model propose many candidates, keep the best, and iterate: selection turns cheap guesses into precise results.

DDN was published at ICLR 2025, and the author's launch page reached #2 on Hacker News's daily rankings. The first citing paper, DDCM, has already borrowed DDN's 1D discrete-latent construction for diffusion models, and Yang's roadmap covers tokenizer-free language modeling (combining DDN with GPT), robotics policies and conditional imaging. The author concedes the experiments were preliminary — under three months of development, mostly alone — and that DDN still produces blurry samples when asked to cover a distribution beyond its capacity; scaling to ImageNet-level complexity remains open.