“GPT3 sees” is a project which explores how or what a language model like GPT3 sees. Through code generation and guided creation, we tried to find out what does a text-only language model imagine. Surprisingly one of its first creations from the code-generation technique was what resembled a Floppy Disc, giving an impression of nostalgia in a machine. A subsequent exploration was a collaboration between the language model and humans. Finally we also tried to create a movie sequence using a story created by GPT3 played out using GIFs.
The GPT3 sees project consisted of 3 major works:
Image (static and moving) generation:
With a prompt of SVG code for 2D primitives, we asked GPT3 to create a new SVG code. It created a code which translated to the image of something like a Floppy Disc.
Then we asked GPT3 with prompts to explain how to draw common objects and animals with shapes. These were drawn by a human with the steps by the machine.
Finally we took it one step forward and asked GPT3 to animate a shape using CSS code.
Assisted painting:
The second work involved asking GPT3 to construct a scene from its imagination. The text was then used step by step by a person to draw the scene. A beautiful scene was created and of particular interest were the last few lines where GPT3 reflects on the meaning of the creation to the person creating the art.
NFT on Async Art : https://async.art/art/master/0xb6dae651468e9593e4581705a09c10a76ac1e0c8-1459
Movie creation :
In the final work, we used a story created by GPT3 and fed it sentence by sentence to a GIF search API from Tenor. This resulted in the creation of a GIF/Meme movie from the story that GPT3 created. This suddenly asks many questions about the future of movies and movie-making. We took it one step forward to create an OTT interface called ”Meme Flix” which automatically generated such ”Meme movies” every time we load the interface. Thus, on demand, one can get access to fresh new stories and movies, in this case with the expressiveness and surprise factor that comes with Memes.