Share
Explore

Generative Sol

Recreating Sol LeWitt's Wall Drawings with AI
Sol LeWitt was a pioneer of conceptual art. His Wall Drawings emphasized art as a concept; the idea behind the art is as important as the art itself. LeWitt created the instructions to produce a wall drawing, while a team of drafters followed those instructions to accurately reproduce his vision. Generative art can trace its roots to conceptual art, and particularly to LeWitt’s work. Code is fundamentally a set of instructions that a computer follows to generate an output: in this case, art.
So what would happen if we asked computers to create LeWitt’s Wall Drawings using only his instructions? Let’s find out, using AI! I used OpenAI’s DALL·E 2, with comparisons to an older technique, VQGAN+CLIP.
From OpenAI’s announcement: “ is a new AI system that can create realistic images and art from a description in natural language.”
is a type of Generative Adversarial Network that can synthesize high resolution images.
(Contrastive Language–Image Pre-training) is a neural network that identifies images with very high accuracy via natural language processing. Combine the two techniques and you have an AI program that can take a text prompt and spit out an image.
I used instructions found in MASS MoCA’s as source text for my input. However, not every instruction will work properly. Below are a few selected drawings which have reasonably formatted instructions and produced recognizable results.
Wall Drawing 396
A black five-pointed star, a yellow six-pointed star, a red seven-pointed star, and a blue eight-pointed star, drawn in color and India ink washes.
Search
Unknown-15.png
VQGAN+CLIP (ImageNet)
DALL·E 2022-07-13 16.33.09 - A black five-pointed star, a yellow six-pointed star, a red seven-pointed star, and a blue eight-pointed star, drawn in color and India ink washes..png
OpenAI DALL·E 2
The GAN seemed to focus on “star” and the various colors but not much else. I was surprised to see “India ink washes” translate to the ImageNet training model. Definitely much more painting-like than anticipated. Meanwhile, DALL·E 2 lands closer to the original style, but can’t quite get the number of points and associated colors right.
Wall Drawing 766
Twenty-one isometric cubes of varying sizes, each with color ink washes superimposed.
Search
Unknown-17.png
VQGAN+CLIP (ImageNet)
DALL·E 2022-07-13 16.19.13 - Twenty-one isometric cubes of varying sizes, each with color ink washes superimposed..png
OpenAI DALL·E 2
Now this is the good stuff! VQGAN doesn’t seem to know how to treat the word “superimposed.” It’s trying to nest the “cubes” within each other. “Color ink washes” translates well though, as each image has a lovely sketched quality to it. DALL·E 2 nails the isometric perspective, and picks a lovely color palette, but doesn’t really vary the size of each cube.
Wall Drawing 439
Asymmetrical pyramid with color ink washes superimposed.
Search
Unknown-29.png
VQGAN+CLIP (ImageNet)
DALL·E 2022-07-13 16.53.48 - Asymmetrical pyramid with color ink washes superimposed..png
OpenAI DALL·E 2
I find it interesting that the simpler prompts can produce such vivid images. It’s almost like both systems feel less constrained and go a little nuts. Also, I am terrified of their creativity.
Wall Drawing 415D
Double Drawing. Right: Isometric Figure (Cube) with progressively darker graduations of gray on each of three planes; Left: Isometric figure with red, yellow, and blue superimposed progressively on each of the three planes. The background is gray.
Search
Unknown-24.png
VQGAN+CLIP (ImageNet)
DALL·E 2022-07-13 16.19.38 - Double Drawing. Right_ Isometric Figure (Cube) with progressively darker graduations of gray on each of three planes; Left_ Isometric figure with red,.png
OpenAI DALL·E 2
VQGAN couldn’t really decide what the word “figure” was supposed to represent. Is it a shape? A diagram? Or a person? Why not all at once? Whereas DALL·E 2 is very close to the original design. This is starting to get a little scary.
Wall Drawing 610
Isometric figure with color ink washes superimposed.
Search
Unknown-37.png
VQGAN+CLIP (ImageNet)
DALL·E 2022-07-13 17.08.48 - Isometric figure with color ink washes superimposed..png
OpenAI DALL·E 2
Again, VQGAN has some confusion with the word “figure,” represented here as a hybrid between a humanoid sculpture and cubic abstraction. DALL·E 2’s output is once again remarkably close to the original.
Wall Drawing 552D
Tilted forms with color ink washes superimposed.
Search
Unknown-31.png
VQGAN+CLIP (ImageNet)
DALL·E 2022-07-13 16.56.32 - Tilted forms with color ink washes superimposed..png
OpenAI DALL·E 2
Forms? Tilted. Colors? Yep, they’re all there. Neither system had much to work with on this one.
Wall Drawing 692
Continuous forms with color ink washes superimposed.
Search
Unknown-35.png
VQGAN+CLIP (ImageNet)
DALL·E 2022-07-13 17.05.43 - Continuous forms with color ink washes superimposed..png
OpenAI DALL·E 2
Very similar to 552D given the prompts are almost identical but “continuous” seems to translate as curvier shapes.
Wall Drawing 527
Two flat-topped pyramids with color ink washes superimposed.
Search
Unknown-19.png
VQGAN+CLIP (ImageNet)
DALL·E 2022-07-13 16.24.25 - Two flat-topped pyramids with color ink washes superimposed..png
OpenAI DALL·E 2
The tops of the pyramids are flat! There are colors! Good job, AI!
Wall Drawing 386
Stars with three, four, five, six, seven, eight, and nine points, drawn with a light tone India ink wash inside, an India ink wash outside, separated by a 6-inch (15 cm) white band.
Search
Unknown-28.png
VQGAN+CLIP (ImageNet)
DALL·E 2022-07-13 16.39.48 - Stars with three, four, five, six, seven, eight, and nine points, drawn with a light tone India ink wash inside, an India ink wash outside, separated .png
OpenAI DALL·E 2
VQGAN decided "stars” are both geometric figures and giant hot balls of gas. Given there was no specific color indicated in the prompt, it was interesting to see VQGAN veer towards a space scene whereas DALL·E 2 looks much more like a wall drawing, albeit totally missing the “points” in the stars, turning them into numbers inscribed in the stars!
Wall Drawing 579
Three concentric arches. The outside one is blue; the middle red; and the inside one is yellow.
Search
Unknown-13.png
VQGAN+CLIP (ImageNet)
DALL·E 2022-07-13 16.27.41 - Three concentric arches. The outside one is blue; the middle red; and the inside one is yellow..png
OpenAI DALL·E 2
What is an arch? VQGAN struggles a bit with literalism by trying to contort stone arches into an impossible shape. The colors are a bit of an afterthought. Meanwhile, DALL·E 2 absolutely nails it.
Wall Drawing 681C
A wall divided vertically into four equal squares separated and bordered by black bands. Within each square, bands in one of four directions, each with color ink washes superimposed.
Search
Unknown-21.png
VQGAN+CLIP (ImageNet)
DALL·E 2022-07-13 16.22.02 - A wall divided vertically into four equal squares separated and bordered by black bands. Within each square, bands in one of four directions, each wit.png
OpenAI DALL·E 2
VQGAN tries to take “wall” very literally by drawing a stone wall of sorts. DALL·E 2 gets the vertical divisions and equal spacing, but gives us 7 squares instead of 4. Come on, DALL·E 2!
Wall Drawing 901
Color bands and black blob. The wall is divided vertically into six equal bands; red; yellow; blue; orange; purple; green. In the center is a black glossy blob.
Search
Unknown-33.png
VQGAN+CLIP (ImageNet)
DALL·E 2022-07-13 16.59.16.png
OpenAI DALL·E 2
Pretty much every element of the drawing is present at some level, especially “black glossy blob.” VQGAN picks the most terrifying interpretation to show to us, whereas DALL·E 2 is so close to the original it could very well pass as a proper Sol LeWitt wall drawing.


Want to print your doc?
This is not the way.
Try clicking the ⋯ next to your doc name or using a keyboard shortcut (
CtrlP
) instead.