Text-to-image diffusion based generative AI models can offer up copyrighted work of artists. DALL-E, Stable Diffusion and Midjourney are all facing lawsuits from artists who allege that the models produce art similar to their works. A new generative AI tool draws inspiration from artwork instead of replicating them.
Examples of images generated by the Ambient Diffusion model. (Image Credit: UT).
New Delhi: Machine learning models with the capabilities of turning images into prompts are trained on thousands of pairs of images and descriptive captions, that are not publicly disclosed. At times the artwork can be close to the styles of artists, leading to lawsuits against DALL-E, Stable Diffusion and Midjourney, the most popular models with generative capabilities. Researchers from the University of Texas at Austin have developed a framework to train AI on images corrupted beyond recognition.
The newly proposed framework uses only data sets with corrupted images. Preliminary experiments reveal that the model is capable of producing high-quality samples without having trained on anything that can be recognised as the original source images. The model was originally presented at NeurIPS, a machine-learning conference in 2023.
The model has now been adapted and expanded, with the follow-up paper being accepted for the 2024 International Conference of Machine Learning. Initially, the images were corrupted by masking pixels. Now, the team has used other types of noise to corrupt the images used for training, and leveraged larger data sets as well.
One of the study authors, Adam Klivans says, “The framework could prove useful for scientific and medical applications, too. That would be true for basically any research where it is expensive or impossible to have a full set of uncorrupted data, from black hole imaging to certain types of MRI scans.” During the initial effort up to 90 per cent of the pixels in the images were randomly masked. The system has been tested on images of celebrities.
Sure, but what exactly is ‘Diffusion’?
Diffusion models are a class of generative AI models that first add random noise to existing data, and then gradually transform the random noise to a structured output. These models work by either adding Gaussian noise to available training data, or recovering the original data by removing the noise.
