[Linkpost] Generalization in diffusion models arises from geometry-adaptive harmonic representation

post by Bogdan Ionut Cirstea (bogdan-ionut-cirstea) · 2023-10-11T17:48:24.500Z · LW · GW · 3 comments

Contents

3 comments

This is a linkpost for https://arxiv.org/abs/2310.02557. 

we show that two denoisers trained on non-overlapping training sets converge to essentially the same denoising function. As a result, when used for image generation, these networks produce nearly identical samples. These results provide stronger and more direct evidence of generalization than standard comparisons of average performance on train and test sets. The fact that this generalization is achieved with a small train set relative to the network capacity and the image size implies that the network’s inductive biases are well-matched to the underlying distribution of photographic images.

 

Here, we showed empirically that diffusion models can achieve a strong form of generalization, converging to a unique density model that is independent of the specific training samples, with an amount of training data that is small relative to the size of the parameter or input spaces. The convergence exhibits a phase transition between memorization and generalization as training data grows. The amount of data needed to cross this phase transition depends on both the image complexity and the neural network capacity (Yoon et al., 2023), and it is of interest to extend both the theory and the empirical studies to account for these. The framework we introduced to assess memorization versus generalization may be applied to any generative model.

3 comments

Comments sorted by top scores.