Show HN: I invented a new generative model and got accepted to ICLR

Summary

DDN: Discrete Distribution Networks 🥳 Accepted by ICLR 2025🚀 The code has been released Discrete Distribution Networks A novel generative model with simple principles and unique properties Lei Yang Details of density estimation This GIF demonstrates the optimization process of DDN for 2D probability density estimation: Left image: All samples that can currently be generated Right image: Target probability density map For demonstration purposes, the target probability density maps switch periodically. Names and sequence of target probability maps: blur_circles -> QR_code -> spiral -> words -> gaussian -> blur_circles (same at beginning and end, completing a cycle) Therefore DDN continuously optimizes parameters to fit new distributions Optimizer: Gradient Descent with Split-and-Prune This only shows experimental results with 1,000 nodes; for a clearer and more comprehensive view of the optimization process, see the 2D Density Estimation with 10,000 Nodes DDN page The experiment code is in sddn/toy_exp.py, and the experimental environment is provided by the distribution_playground library, feel free to play with it yourself Contributions of this paper: We introduce a novel generative model, termed Discrete Distribution Networks (DDN), which demonstrates a more straightforward and streamlined principle and form. For training the DDN, we propose the Split-and-Prune optimization algorithm, and a range of practical techniques. We conduct preliminary experiments and analysis on the DDN, showcasing its intriguing properties and capabilities, such as zero-shot conditional generation without gradient and distinctive 1D discrete representations. Left: Illustrates the process of image reconstruction and latent acquisition in DDN. Each layer of DDN outputs distinct images, here , to approximate the distribution . The sampler then selects the image most similar to the target from these and feeds it into the next DDN layer. As the number of layers increases, the generated images b...