The use cases of artificial intelligence (AI) are somewhat endless – and this has been recently shown by Shane Guffogg, an artist who has synesthesia, meaning he “hears colour.” The artist watches paintings and colours used in them, and uses AI to recreate the art into music.
Guffogg says he likes to listen to Indian classical music, Gregorian chants, and other composers while he paints as music sparks his creativity. He, however, also says that he wondered how his paintings would sound if it was music.
“The AI revolution pushed me to search for experts who could help me. My first point of contact was Radhika Dirks, an AI and quantum computing expert. We had a couple of Zoom sessions, and she told me — to the best of her knowledge — that no AI program could help me. Instead, she suggested I create a visual alphabet that matched the musical chords I heard in my mind to colours,” he was quoted as saying by Business Insider.
Musicians and AI experts created visual alphabet
Guffogg met with musicians and AI experts to create a visual alphabet – which means that he associated a sound with different colours in paintings.
“We would sit down and examine my paintings together. I would zoom in on a colour in Photoshop, look at it, and sensorially feel the musical note,” he said.
As per the artist, he has been able to chart chords corresponding to 40 colours. Then an AI researcher developed a program to “read” the paintings and correspond the colours with the chords. Apart from the colours used, the software took into account the movement of Guffogg’s hand movements.
The resulting data was fed into a neural network that has been trained on the last 500 years of keyboard music. The AI model converted the data, and essentially paintings, into music.
“When I heard that music played back to me, it brought tears to my eyes,” he said. He said the music created by AI was close to what he was listening to while painting.
Guffogg says he likes to listen to Indian classical music, Gregorian chants, and other composers while he paints as music sparks his creativity. He, however, also says that he wondered how his paintings would sound if it was music.
“The AI revolution pushed me to search for experts who could help me. My first point of contact was Radhika Dirks, an AI and quantum computing expert. We had a couple of Zoom sessions, and she told me — to the best of her knowledge — that no AI program could help me. Instead, she suggested I create a visual alphabet that matched the musical chords I heard in my mind to colours,” he was quoted as saying by Business Insider.
Musicians and AI experts created visual alphabet
Guffogg met with musicians and AI experts to create a visual alphabet – which means that he associated a sound with different colours in paintings.
“We would sit down and examine my paintings together. I would zoom in on a colour in Photoshop, look at it, and sensorially feel the musical note,” he said.
As per the artist, he has been able to chart chords corresponding to 40 colours. Then an AI researcher developed a program to “read” the paintings and correspond the colours with the chords. Apart from the colours used, the software took into account the movement of Guffogg’s hand movements.
The resulting data was fed into a neural network that has been trained on the last 500 years of keyboard music. The AI model converted the data, and essentially paintings, into music.
“When I heard that music played back to me, it brought tears to my eyes,” he said. He said the music created by AI was close to what he was listening to while painting.