236610 – Diffusion Diffusion Diffusion – Winter 2022/2023

Diffusion Diffusion Diffusion (236610)

Winter 2022/2023

Given as an “Advanced Topics in CS” Course

Note: This course may be given in English

 

Lecturer Prof. Michael Elad
Credit 2 points
Pre-requisites 236201 (Introduction to Data Processing and Representation) or 046200 (Processing and Analysis of Images)
Parallel Courses 048954 (Statistical Methods in Image Processing) is NOT considered as parellel, but students should avoid double-deeping their projects
Course Description A fascinating machine learning topic in recent years touches on the grand challenge of synthesizing content out of thin air. This includes the creation of images, paragraphs, short videos, music pieces, and more. In most cases, this synthesis goes hand in hand with the notion of GAN’s – Generative Adversarial Networks, although other techniques for such synthesis do exist, such as VAE, energy-based methods, Normalizing flow, and more. The entry of diffusion techniques as a competitive alternative to all the above brought a revolution. This¬†fresh line of work that already grows exponentially offers iterative algorithms that start with a Gaussian iid noise and end with the desaired synthesized content. A key to the success of these techniques is an access to the score function of the data distribution, which can be obtained by simple denoisers. This makes diffusion techniques quire appealing and effective. In this course we will study this field by surverying the existing literature, exploring its extensions and prospects, and by diving into this field’s mathematical foundations.
Course Structure After 2-3 lectures by the lecturer, we will concentrate on students’ lectures on their assigned projects, covering various recent papers in this domain. Each student participating in the course will be assigned with a paper (or more, depending on the content) as the grounds for their project. The project itself will include the following activities:

  • Reading the paper(s) and understanding their content;
  • Implementing the algorithms suggested;
  • Exploring possible extentions to the work suggested;
  • Preparing a slide show to present all the above – the slides should be cleared by the lecturer before they are presented; and
  • Issuing a short report to summarize all the above.
Grading Policy (0-100) based on the quality of the project on all its facets.
Tentative Syllabus
  • Diffusion methods – Mathematical foundations (Stochastic Differential Equations, Score Approximation, Langevine and other samplers, relation to RED)
  • Diffusion Methods – Variations of algorithms (one-sided flow, two sided flow, excellerations)
  • Diffusion Methods – Building blocks (MMSE Image Denoisers, GAN-based Score approximation, alternatives to the score)
  • Diffusion Methods – Applications (Image synthesis, Audio synthesis, Solving inverse problems, Conditional sampling, IMAGEN and related methods, Image Compression, …)

 

List of Resources: TBD