Film Created by AI Won an AwardOctober 2022
If you’re impressed by the recent spate of text-to-image generators, get ready for the next step in AI artistry: text-to-video. While the huge compute costs and scarcity of text-to-video datasets have stunted the technique’s growth, recent research has brought the promise closer to reality. A computer artist called Glenn Marshall has given a glimpse at the potential. The Belfast-based composer recently won the Jury Award at the Cannes Short Film Festival for his AI film The Crow.
Marshall had previously earned plaudits for an AI-generated Daft Punk video, but he applied a different approach to The Crow. While his earlier technique turned text into random visual mutations, The Crow uses an underlying film as an image reference. “I had been heavily getting into the idea of AI style transfer using video footage as a source,” Marshall told TNW. “So every day I would be looking for something on YouTube or stock video sites, and trying to make an interesting video by abstracting it or transforming it into something different using my techniques. “It was during this time I discovered Painted on YouTube — a short live action dance film — which would become the basis of The Crow.” Marshall fed the video frames of Painted to CLIP, a neural network created by OpenAI. He then prompted the system to generate a video of “a painting of a crow in a desolate landscape.”