Share
Explore

Generative Sol

Recreating Sol LeWitt's Wall Drawings with VQGAN+CLIP
Sol LeWitt was a pioneer of conceptual art. His Wall Drawings emphasized art as a
concept
; the idea behind the art is as important as the art itself. LeWitt created the instructions to produce a wall drawing, while a team of drafters followed those instructions to accurately reproduce his vision. Generative art can trace its roots to conceptual art, and
particularly
to LeWitt’s work. Code is fundamentally a set of instructions that a computer follows to generate an output: in this case, art.

So what would happen if we asked computers to create LeWitt’s Wall Drawings using only his instructions? Let’s find out, using VQGAN+CLIP! But first, some background:
(Contrastive Language–Image Pre-training) is a neural network that identifies images with very high accuracy via natural language processing.
is a type of Generative Adversarial Network that can synthesize high resolution images.
Combine the two techniques and you have an AI program that can take a text prompt and spit out an image. So far, we’ve seen some
with this technique.

I used instructions found in MASS MoCA’s
as source text for my input. However, not every instruction will work properly. Some instructions are too long and cannot be processed by CLIP, while some are too vague, and others simply lack text-based instructions altogether.

Below are a few selected drawings which have reasonably formatted instructions and produced recognizable results after 200 iterations of running the GAN. I included outputs trained on both ImageNet and WikiArt.


Wall Drawing 396

A black five-pointed star, a yellow six-pointed star, a red seven-pointed star, and a blue eight-pointed star, drawn in color and India ink washes.

Unknown-15.png
imagenet_16384
Unknown-16.png
wikiart_16384

The GAN seemed to focus on “star” and the various colors but not much else. I was surprised to see “India ink washes” translate to the ImageNet training model. Definitely much more painting-like than anticipated. Overall, quite pleasant to look at!


Wall Drawing 766

Twenty-one isometric cubes of varying sizes, each with color ink washes superimposed.

Unknown-17.png
imagenet_16384
Unknown-18.png
wikiart_16384

Now this is the good stuff! The neural network doesn’t seem to know how to treat the word “superimposed.” It’s trying to nest the “cubes” within each other. “Color ink washes” translates well though, as each image has a lovely sketched quality to it.


Wall Drawing 439

Asymmetrical pyramid with color ink washes superimposed.

Unknown-29.png
imagenet_16384
Unknown-30.png
wikiart_16384

I find it interesting that the simpler prompts can produce such vivid images. It’s almost like the GAN feels less constrained and goes a little nuts. Also, I am terrified of its creativity.


Wall Drawing 415D

Double Drawing. Right: Isometric Figure (Cube) with progressively darker graduations of gray on each of three planes; Left: Isometric figure with red, yellow, and blue superimposed progressively on each of the three planes. The background is gray.

Unknown-24.png
imagenet_16384
Unknown-25.png
wikiart_16384

The version trained on WikiArt really enforced the “double drawing” element, but also couldn’t decide what “figure” was supposed to represent. Is it a shape? A diagram? Or a person? Why not all at once?


Wall Drawing 610

Isometric figure with color ink washes superimposed.

Unknown-37.png
imagenet_16384
Unknown-38.png
wikiart_16384

Again we see some confusion with the word “figure,” represented here as a hybrid between a humanoid sculpture and cubic abstraction.


Wall Drawing 552D

Tilted forms with color ink washes superimposed.

Unknown-31.png
imagenet_16384
Unknown-32.png
wikiart_16384

Forms? Tilted. Colors? Yep, they’re all there. This one is probably the most accurately reproduced drawing of the bunch.


Wall Drawing 692

Continuous forms with color ink washes superimposed.

Unknown-35.png
imagenet_16384
Unknown-36.png
wikiart_16384

Very similar to 552D given the prompts are almost identical but “continuous” seems to translate as curvier shapes.


Wall Drawing 527

Two flat-topped pyramids with color ink washes superimposed.

Unknown-19.png
imagenet_16384
Unknown-20.png
wikiart_16384

The tops of the pyramids are flat! There are colors! Sure, there are more than two, but that’s ok! Good job, AI!


Wall Drawing 386

Stars with three, four, five, six, seven, eight, and nine points, drawn with a light tone India ink wash inside, an India ink wash outside, separated by a 6-inch (15 cm) white band.

Unknown-28.png
imagenet_16384
Unknown-27.png
wikiart_16384

The AI decided "stars” are both geometric figures
and
giant hot balls of gas. Given there was no specific color indicated in the prompt, it was interesting to see roughly the same color scheme in both models.


Wall Drawing 579

Three concentric arches. The outside one is blue; the middle red; and the inside one is yellow.

Unknown-13.png
imagenet_16384
Unknown-14.png
wikiart_16384

What is an arch? Our AI struggles a bit with literalism by trying to contort stone arches into an impossible shape. The colors are a bit of an afterthought.


Wall Drawing 681C

A wall divided vertically into four equal squares separated and bordered by black bands. Within each square, bands in one of four directions, each with color ink washes superimposed.

Unknown-21.png
imagenet_16384
Unknown-22.png
wikiart_16384

Again, our AI tries to take “wall” very literally by drawing a stone wall of sorts. We see a tiny inkling of color bands in the image trained by WikiArt, but not much else to speak of.


Wall Drawing 901

Color bands and black blob. The wall is divided vertically into six equal bands; red; yellow; blue; orange; purple; green. In the center is a black glossy blob.

Unknown-33.png
imagenet_16384
Unknown-34.png
wikiart_16384

Pretty much every element of the drawing is present at some level, especially “black glossy blob.” As usual, the GAN picks the most terrifying interpretation to show to us.



Want to print your doc?
This is not the way.
Try clicking the ⋯ next to your doc name or using a keyboard shortcut (
CtrlP
) instead.