Creating Artworks by Adversarial Evolution and Deep Learning

Artist-Critic Coevolution

"All in all the creative act is not performed by the artist alone; the spectator brings the work in contact with the external world by deciphering and interpreting its inner qualifications and thus adds his contribution to the creative act."

- Marcel Duchamp

Imagine, if you will, a situation where a critic collects a few dozen images - perhaps of a famous landmark - and asks an artist for a painting of that landmark, based on the supplied images. The fumbling artist eventually creates a painting that satisfies the critic, but the fickle critic then becomes dissatisfied and asks for another painting of the same landmark, noticeably distinct from the first painting. This process continues, adding successive paintings one by one to a virtual gallery, with each painting designed to resemble the original photographs while at the same time being distinct from all previous images in the gallery, as judged by the critic.

The process described above is known as artist-critic coevolution. The artist can be automated in the form of a genetic program (GP) or a convolutional neural network (CNN); the role of the critic can be played by a human, or it can also be automated as a GP or CNN.

One popular variant of artist-critic coevolution which has gained popularity in recent years is Generative Adversarial Networks (GANs) in which a convolutional neural network is used for both the generator (artist) and discriminator (critic).

Hercule LeNet

In the case of Hercule LeNet, the critic is a GAN-style convolutional neural network modeled on the LeNet architecture, while the role of the artist is played by particular kind of GP known as Hierarchical Evolutionary Re-Combination Language (HERCL). Compared to other GP paradigms, HERCL has the advantages that

  1. the number of competing individuals is kept small, while still preserving diversity (important because the CNN fitness evaluation can be computationally expensive)
  2. the code for previous images in the gallery can be stored in a library and re-used in a natural way, allowing the artist to revisit and further develop earlier themes, and
  3. HERCL does a good job of applying selective pressure to encourage shorter programs, and therefore images of lower algorithmic complexity

Previous approaches to computer generated art have often involved either feedback from a human critic, or a database of human artworks whose style the system is intended to copy. Our system, in contrast, is provided only with photographs of the intended subject, and is not asked to mimic the style of any previous artist. Instead, the art emerges from the creative tension between selective pressure for images of low algorithmic complexity and the imperitive of fooling the critic into thinking the images are real.

Training Paradigm

At the beginning of each round, a new critic is trained by graident descent to output a number close to 0 for the real images and close to 1 for the images generated in all of the previous rounds. After that, a new artist is evolved, with the objective of producing an image for which the new critic will assign as low a number (cost) as possible. The HERCL code for the artist acts as a function which inputs the (x,y) coordinates of a pixel and outputs the red, green and blue intensity of that pixel. Evolution proceeds until either a fixed threshold in achieved, or a predefined maximum number of images (typically, 200000) have been evaluated. The image achieving the lowest cost is then added to the gallery, ready for the next round.

Resolution and Upscaling

When initially generated, images are fed to the critic at a low resolution such as 28×28, 32×32, 48×48 or 64×64. Afterwards, the same HERCL code can be used to render the images at higher resolution, as illustrated in this figure.

Very occasionally, it may happen that the HERCL code crashes or fails to produce an output for one of the pixels in the upscaled image, in which case image generation will fail.

Genetype to Phenotype mapping

The HERCL code for Venice 6.214 is shown below, along with equivalent pseudocode. Note that x runs from left (-1) to right (+1), while y runs from top (-1) to bottom (+1). Color intensities are output in the order blue, green, red; outputs less than 0 or greater than 1 are treated as 0 and 1, respectively. In order to avoid floating point exceptions, HERCL adheres to certain safety conventions, such as sin-1(α) = -π/2 for α < -1, which in this case ensures a uniform color for the sky.

HERCL code: 

 0[!qatcz]
 1[capwwwo.]
 2[%]
 3[is.32#>sg:1j|c>xg:hp2j|+a{>cpa%.4338#p>g~<:0j|xww.88#wo]

Pseudocode:

  scan(x,y)       // -1 ≤ x ≤ +1, (upper) -1 ≤ y ≤ +1 (lower)
  if y ≥ 0.32       // water
        return (sqrt(y2+ (sin-1y)2), atan2(y, sin-1y), x)
  else
        if y > x           // obstacle
              u = sin-1(x + y)
        else                // ship
              r = sqrt(y2 + tanh(x)2), θ = atan2(y, tanh(x))
              u = sin-1(⌊θ/r⌋ + (θ mod r))
        end
        θ = (π/4)(-1 + 2 sgn(u)),     ρ = sin-1(√2u)
        z = atan2((φ mod ρ), 0.4338), s = sqrt((φ mod ρ)2 + 0.43382)
        if s ≤ z            // sails
              return(z, s, 0.88)
        else                // hull
              v = sin-1(√z) cos(⌊φ/ρ⌋)
              return(v, ⌊φ/ρ⌉, 0.88)
        end
  end

PickArtSo.com