Fine-tuning Stable Diffusion with KerasCV 🔥

Hi folks,

I hope y’all are doing well. Happy new year!

Delighted to present a project that @deep-diver and I have been working on for a while now – Fine-tuning Stable Diffusion with KerasCV. The project shows how to fine-tune the encoder part of Stable Diffusion on a custom image-caption dataset. It supports model checkpointing, mixed-precision training and is fairly customizable. Following are the main links:

We hope y’all will like the project and have fun playing with it.


Thanks @Sayak_Paul !

There is also:

Amazing stuff :+1:

You missed this one :stuck_out_tongue:

1 Like

Can this model be used for Inpainting by simply adding init and mask images?

Yup. There is already a class for that.