An Efficient Adaptive Latent Diffusion Framework with High-Order DPM-Solver for High-Fidelity Image Generation

Diffusion-based models have emerged as the latest framework for generating data with high fidelity, showing great success in image synthesis, medical image reconstruction, video generation, simulation of scientific processes, and multimodal content generation. While producing extremely high-quality results, current diffusion models require hundreds to thousands of denoising iterations, leading to long computation times, high computational costs, large GPU memory usage, and high energy costs. This makes them unusable in real-time applications and resource-limited settings such as edge devices, mobile platforms, health care applications, and self-driving systems. While recent advances such as DPM-Solver++, DDIM, Consistency Models, and Flow Matching improve sampling efficiency, they tackle individual problems without providing a complete solution to minimize computation cost and maintain generation quality and stability. The present study introduces a new model, the Adaptive Compute-Optimal Diffusion Algorithm (ACODA), that offers a unified approach to accelerating diffusion models by leveraging adaptive timestep selection and efficient inference. This new framework consists of the Greedy Adaptive Timestep Selection approach for choosing the best informative denoising steps, an Efficient U-Net/Diffusion Transformer for predicting noise accurately, state-of-the-art numerical solvers like DPM-Solver++ and UniPC for fast sampling, and efficient approaches like knowledge distillation, pruning, quantization, and mixed-precision training to make the model lightweight.