Explore

Generative Sol
Generative Sol

Recreating Sol LeWitt's Wall Drawings with AI

Recreating Sol LeWitt's Wall Drawings with AI

Sohan Murthy

Sol LeWitt was a pioneer of conceptual art. His Wall Drawings emphasized art as a concept; the idea behind the art is as important as the art itself. LeWitt created the instructions to produce a wall drawing, while a team of drafters followed those instructions to accurately reproduce his vision. Generative art can trace its roots to conceptual art, and particularly to LeWitt’s work. Code is fundamentally a set of instructions that a computer follows to generate an output: in this case, art.

So what would happen if we asked computers to create LeWitt’s Wall Drawings using only his instructions? Let’s find out, using AI! I used OpenAI’s DALL·E 2, with comparisons to an older technique, VQGAN+CLIP.

From OpenAI’s announcement: “

DALL·E 2⁠

is a new AI system that can create realistic images and art from a description in natural language.”

⁠

VQGAN⁠

is a type of Generative Adversarial Network that can synthesize high resolution images.

CLIP⁠

(Contrastive Language–Image Pre-training) is a neural network that identifies images with very high accuracy via natural language processing. Combine the two techniques and you have an AI program that can take a text prompt and spit out an image.

I used instructions found in MASS MoCA’s

Sol LeWitt Wall Drawing retrospective⁠

as source text for my input. However, not every instruction will work properly. Below are a few selected drawings which have reasonably formatted instructions and produced recognizable results.

⁠

Wall Drawing 396

A black five-pointed star, a yellow six-pointed star, a red seven-pointed star, and a blue eight-pointed star, drawn in color and India ink washes.

⁠

Original | MASS MoCA⁠

⁠

VQGAN+CLIP (ImageNet)

DALL·E 2022-07-13 16.33.09 - A black five-pointed star, a yellow six-pointed star, a red seven-pointed star, and a blue eight-pointed star, drawn in color and India ink washes..png

OpenAI DALL·E 2

⁠

The GAN seemed to focus on “star” and the various colors but not much else. I was surprised to see “India ink washes” translate to the ImageNet training model. Definitely much more painting-like than anticipated. Meanwhile, DALL·E 2 lands closer to the original style, but can’t quite get the number of points and associated colors right.

Wall Drawing 766

Twenty-one isometric cubes of varying sizes, each with color ink washes superimposed.

⁠

Original | MASS MoCA⁠

⁠

VQGAN+CLIP (ImageNet)

DALL·E 2022-07-13 16.19.13 - Twenty-one isometric cubes of varying sizes, each with color ink washes superimposed..png

OpenAI DALL·E 2

⁠

Now this is the good stuff! VQGAN doesn’t seem to know how to treat the word “superimposed.” It’s trying to nest the “cubes” within each other. “Color ink washes” translates well though, as each image has a lovely sketched quality to it. DALL·E 2 nails the isometric perspective, and picks a lovely color palette, but doesn’t really vary the size of each cube.

Wall Drawing 439

Asymmetrical pyramid with color ink washes superimposed.

⁠

Original | MASS MoCA⁠

⁠

VQGAN+CLIP (ImageNet)

DALL·E 2022-07-13 16.53.48 - Asymmetrical pyramid with color ink washes superimposed..png

OpenAI DALL·E 2

⁠

I find it interesting that the simpler prompts can produce such vivid images. It’s almost like both systems feel less constrained and go a little nuts. Also, I am terrified of their creativity.

Wall Drawing 415D

Double Drawing. Right: Isometric Figure (Cube) with progressively darker graduations of gray on each of three planes; Left: Isometric figure with red, yellow, and blue superimposed progressively on each of the three planes. The background is gray.

⁠

Original | MASS MoCA⁠

⁠

VQGAN+CLIP (ImageNet)

DALL·E 2022-07-13 16.19.38 - Double Drawing. Right_ Isometric Figure (Cube) with progressively darker graduations of gray on each of three planes; Left_ Isometric figure with red,.png

OpenAI DALL·E 2

⁠

VQGAN couldn’t really decide what the word “figure” was supposed to represent. Is it a shape? A diagram? Or a person? Why not all at once? Whereas DALL·E 2 is very close to the original design. This is starting to get a little scary.

Wall Drawing 610

Isometric figure with color ink washes superimposed.

⁠

Original | MASS MoCA⁠

⁠

VQGAN+CLIP (ImageNet)

DALL·E 2022-07-13 17.08.48 - Isometric figure with color ink washes superimposed..png

OpenAI DALL·E 2

⁠

Again, VQGAN has some confusion with the word “figure,” represented here as a hybrid between a humanoid sculpture and cubic abstraction. DALL·E 2’s output is once again remarkably close to the original.

Wall Drawing 552D

Tilted forms with color ink washes superimposed.

⁠

Original | MASS MoCA⁠

⁠

VQGAN+CLIP (ImageNet)

DALL·E 2022-07-13 16.56.32 - Tilted forms with color ink washes superimposed..png

OpenAI DALL·E 2

⁠

Forms? Tilted. Colors? Yep, they’re all there. Neither system had much to work with on this one.

Wall Drawing 692

Continuous forms with color ink washes superimposed.

⁠

Original | MASS MoCA⁠

⁠

VQGAN+CLIP (ImageNet)

DALL·E 2022-07-13 17.05.43 - Continuous forms with color ink washes superimposed..png

OpenAI DALL·E 2

⁠

Very similar to 552D given the prompts are almost identical but “continuous” seems to translate as curvier shapes.

Wall Drawing 527

Two flat-topped pyramids with color ink washes superimposed.

⁠

Original | MASS MoCA⁠

⁠

VQGAN+CLIP (ImageNet)

DALL·E 2022-07-13 16.24.25 - Two flat-topped pyramids with color ink washes superimposed..png

OpenAI DALL·E 2

⁠

The tops of the pyramids are flat! There are colors! Good job, AI!

Wall Drawing 386

Stars with three, four, five, six, seven, eight, and nine points, drawn with a light tone India ink wash inside, an India ink wash outside, separated by a 6-inch (15 cm) white band.

⁠

Original | MASS MoCA⁠

⁠

VQGAN+CLIP (ImageNet)

DALL·E 2022-07-13 16.39.48 - Stars with three, four, five, six, seven, eight, and nine points, drawn with a light tone India ink wash inside, an India ink wash outside, separated .png

OpenAI DALL·E 2

⁠

VQGAN decided "stars” are both geometric figures and giant hot balls of gas. Given there was no specific color indicated in the prompt, it was interesting to see VQGAN veer towards a space scene whereas DALL·E 2 looks much more like a wall drawing, albeit totally missing the “points” in the stars, turning them into numbers inscribed in the stars!

Wall Drawing 579

Three concentric arches. The outside one is blue; the middle red; and the inside one is yellow.

⁠

Original | MASS MoCA⁠

⁠

VQGAN+CLIP (ImageNet)

DALL·E 2022-07-13 16.27.41 - Three concentric arches. The outside one is blue; the middle red; and the inside one is yellow..png

OpenAI DALL·E 2

⁠

What is an arch? VQGAN struggles a bit with literalism by trying to contort stone arches into an impossible shape. The colors are a bit of an afterthought. Meanwhile, DALL·E 2 absolutely nails it.

Wall Drawing 681C

A wall divided vertically into four equal squares separated and bordered by black bands. Within each square, bands in one of four directions, each with color ink washes superimposed.

⁠

Original | MASS MoCA⁠

⁠

VQGAN+CLIP (ImageNet)

DALL·E 2022-07-13 16.22.02 - A wall divided vertically into four equal squares separated and bordered by black bands. Within each square, bands in one of four directions, each wit.png

OpenAI DALL·E 2

⁠

VQGAN tries to take “wall” very literally by drawing a stone wall of sorts. DALL·E 2 gets the vertical divisions and equal spacing, but gives us 7 squares instead of 4. Come on, DALL·E 2!

Wall Drawing 901

Color bands and black blob. The wall is divided vertically into six equal bands; red; yellow; blue; orange; purple; green. In the center is a black glossy blob.

⁠

Original | MASS MoCA⁠

⁠

VQGAN+CLIP (ImageNet)

OpenAI DALL·E 2

⁠

Pretty much every element of the drawing is present at some level, especially “black glossy blob.” VQGAN picks the most terrifying interpretation to show to us, whereas DALL·E 2 is so close to the original it could very well pass as a proper Sol LeWitt wall drawing.