“Make it new”: AI, Modernism, and database art

This essay explores the parallels between contemporary generative AI art and earlier modernism. Although generative AI and modernist art appear to be opposites (one was focused on “making it new”, the other is based on training data of already existing art), they are actually similar. While modernist artists explicitly opposed traditions, they achieved innovation by reinterpreting and incorporating older art forms from other cultures. Similarly, generative AI tools allow the creation of new works because they are trained on massive databases of existing art and media. AI art, thus, fits into a long-standing tradition in modern and contemporary art that involves creating new art from the accumulation of existing artefacts. This tradition encompasses modernist collage and photomontage, post-modern bricolage, Net art and the pioneering media work of artists like Nam June Paik. Contemporary AI artists, such as Refik Anadol and Lev Pereulkov, exemplify the practice of using AI models trained on specific datasets to produce novel artworks that engage in a dialogue with historical art while introducing new aesthetic possibilities. Their projects, such as Unsupervised and Artificial Experiments 1-10, illustrate how AI artworks can creatively engage with older art, creating an ongoing dialogue between new media and historical art traditions.

The current generation of generative AI systems, such as ChatGPT, Midjourney and Stable Diffusion, have been trained on large and diverse datasets consisting of trillions of individual texts or billions of images and their text descriptions. However, many creators working with generative AI chose to either fine-tune existing AI models on their own data or train models only on such data. It is also very interesting to limit the training data set to a narrower area within the larger space of human cultural history or to a specific set of artists from a specific historical period. One such project will be a starting point for my discussion.

Unsupervised (2022) by Refik Anadol Studio is an AI art project that exemplifies these possibilities. The project uses an AI model trained on the image dataset of tens of thousands of artworks from the MoMA (Museum of Modern Art, New York) collection. MoMA’s collection, in my opinion, is one of the best representations of the most creative and experimental period in human visual history – the one hundred years of modern art between 1870 and 1970. It captures modernist artists’ feverish and relentless experiments to create new visual and communication languages and “make it new”.

On the surface, the logic of Modernism appears to be diametrically opposed to the process of training generative AI systems. Modern artists desired to move away from classical art and its defining characteristics, such as visual symmetry, hierarchical compositions and narrative content. In other words, their art was founded on fundamentally rejecting everything that had come before it (at least in theory, as expressed in their manifestos). Neural networks are trained oppositely by learning from historical culture and art created up to now. A neural network is analogous to a very conservative artist studying in a “meta”-museum without walls that only houses historical art.

But we all know that art theory and art practice are not the same thing. Modernist artists did not wholly reject the past and everything that came before them. Instead, modernist art developed by reinterpreting and copying images and forms from much older art traditions, such as Japanese prints (van Gogh), African sculpture (Picasso), and Russian icons (Malevich). Thus, artists only rejected the dominant “high art” paradigms of the time (Realism and Salon Art), but not the rest of human art history. In other words, Modernism was deeply historicist: rather than inventing everything from scratch, it innovated by adapting certain older aesthetics to contemporary art contexts. In the case of Geometric Abstract Art created in the 1910s, these artists used images that were already widely used in experimental psychology to study human visual sensations and perception.

When it comes to artistic AIs, we should not be blinded by how these systems are trained. Yes, artificial neural networks are trained on already existing human art and culture artefacts. However, their newly generated outputs are not mechanical replicas or simulations of what has already been created. In my opinion, these are frequently genuinely new cultural artefacts with previously unseen content, aesthetics, or styles. In other words, I want to suggest that modernist projects and AI art phenomena are often more similar than they may appear.

Of course, simply being novel does not automatically make something culturally or socially interesting or significant. Indeed, many definitions of creativity agree on this point: it is the creation of something that is both original and worthwhile or useful.

However, estimating what percentage of all novel artefacts produced by generative AI is also useful and/or meaningful for a larger culture is not a feasible project at this time. For one thing, I am not aware of any systematic effort to use such systems to “fill in”, so to speak, a massive matrix of all the content and aesthetic possibilities by providing millions of specifically designed prompts. Instead, it is likely that, as in every other area of popular culture, only a small number of possibilities are realized over and over by millions of users, leaving a long tail of other possibilities unrealized. So, if only a tiny fraction of the vast universe of potential AI creations is being realized in practice, we cannot make broad statements about the originality or utility of the rest of the universe.

Some AI artists, such as Anna Ridler, Sarah Meyohas and Refik Anadol, have utilized neural nets trained on specific datasets in their works. Many other artists, designers, architects, and technologists use networks released by other companies or research institutions already trained on large datasets (e.g., Stable Diffusion) and then fine-tune them on their own data.

For example, artist Lev Pereulkov fine-tuned the Stable Diffusion model 2.1 using 40 paintings by well-known “non-conformist” artists who worked in the USSR starting in the 1960s (Erik Bulatov, Ilya Kabakov and others). Pereulkov’s image series Artificial Experiments 1-10 (Pereulkov, 2023), created with this custom AI model, is an original artwork that captures these artists’ aesthetic and semantic worlds without closely repeating any of their existing works. Instead, their “DNAs” captured by the model enable the production of new meanings and visual concepts.

Most of the millions of everyday people and creative professionals who employ generative media tools use them as is and do not customize them further. This may change in the future as fine-tuning these tools to follow our aesthetic preferences becomes more commonplace. But regardless of these specifics, all newly created cultural artefacts produced by generative AI have a common logic.

Unlike traditional drawings, sculptures, and paintings, generative media artefacts are not created from scratch. They are also not the result of capturing some sensory phenomenon, such as photos, videos or sound recordings. Instead, they are built from a large archive of other media artefacts. This generative mechanism links generative media to certain earlier art genres and processes.

We can compare it with film editing, which first appeared around 1898, or with composite photography, which was popular – even earlier – in the 19th century. We can also consider specific artworks that are especially relevant, such as the experimental collage film A Movie (Bruce Conner, 1958) or many Nam June Paik installations that feature edited fragments of TV footage. Seeing projects like Unsupervised or Artificial Experiments 1-10 in the context of this media-making tradition and its historical variations will help us understand these and many other AI artworks as art objects engaged in dialogues with art from the past rather than as purely technological novelties or works of entertainment.

I see many relevant moments and periods when I scan the history of art, visual culture, and media for other prominent uses of this paradigm, such as making new cultural objects from collections of existing ones. They are relevant to the current generative media not only because many artists in the past at different moments in media history used this approach but also because the motivation for its periodic reoccurrence seems to remain the same. A new accumulation and accessibility of masses of cultural artefacts led artists to create new forms of art driven by these accumulations. Let me describe a few of these examples.

The Net and digital artists created some works in the late 1990s and early 2000s in response to the new rapidly expanding universe of the World Wide Web. Health Bunting’s _readme (1998), for example, is a web page containing the text of an article about the artist, with each word linked to an existing web domain corresponding to that word. Mark Napier’s Shredder 1.0 (also 1998) presents a dynamic montage of elements that comprise numerous websites – images, texts, HTML code and links.

Going earlier to the 1980s, we also find artists reacting to the accumulation of historical art and cultural artefacts in easily accessible media collections. This paradigm is known as “Postmodernism”. Postmodern artists and architects frequently used bricolage to create works that included quotations and references to historical art, rejecting Modernism’s self-proclaimed emphasis on novelty and breaking with the past.

While there are many possible explanations for the emergence of the postmodern paradigm at that time, one of them is particularly relevant to our discussion. The accumulation of earlier art and media artefacts in structured and accessible collections such as slide libraries, film archives, art history textbooks with many photos of the artworks and other formats – where different historical periods, movements, and creators were positioned together – inspired artists to begin creating bricolages from such references as well as extensively quoting them.

And what about the Modernism of the 1910s-1920s? While modernists claimed they valued originality and innovation, one of the methods they employed to achieve this novelty was the incorporation of direct quotations from the rapidly expanding realm of contemporary visual media. In these decades, the use of large headings and the inclusion of photos and maps made newspapers more visually impactful; new visually-oriented magazines, such as Vogue and Times, were launched in 1913 and 1923, respectively; and of course, the new medium of cinema continued to develop.

In response to this visual intensification of mass culture, in 1912, Georges Braque and Pablo Picasso began incorporating actual newspapers, posters, wallpaper and fabric fragments into their paintings. A few years later, John Heartfield, George Grosz, Hannah Hoch, Aleksandr Rodchenko and a handful of other artists began to develop photo collages, which became another method of creating new media artefacts from existing mass media images.

Contemporary artworks that employ AI models trained on cultural databases, such as Unsupervised or Artificial Experiments 1-10, continue a long tradition of creating new art from accumulations of images and other media. Thus, these artworks create novel possibilities for art and its methodologies, specifically within the realm of what I previously described as “database art” (Manovich, 1999). The introduction of new methods for reading cultural databases and creating new narratives from them is part of this expansion.

Unsupervised, neither creates collages from existing images, as did modernist artists of the 1920s nor quotes them extensively, as did postmodern artists of the 1980s. Instead, Refik Anadol Studio trained an AI model to extract patterns from tens of thousands of MoMA’s artworks. The model can generate new images that have the same patterns as training data but do not look like any specific paintings. However, rather than simply displaying these images separately, the installation presents the viewers with the constantly changing animation. As we watch it, we travel through the space of these patterns (e.g., “latent space”), exploring various regions of the universe of modern art as represented in the MoMA’s collection.

Pereulkov’s Artificial Experiments 1-10 use a different technique to generate new images from an existing image database. He chose only forty paintings by artists who shared certain characteristics. They developed their oppositional art in late communist societies (1960s-1980s USSR). They also lived in the same visual culture. In my memories, this society was dominated by two colours: grey (representing the monotony of urban life) and red for propaganda slogans and flags.

In addition, Pereulkov chose paintings that share something else: “I chose, as a rule, paintings that conceptually relate in some way to the canvas – or to the space on it. For example, I use the image of the painting New Accordion by Ilya Kabakov, which features paper applications on top of the canvas” (my personal communication with Pereulkov, 04/16/2023). Pereulkov also crafted custom text descriptions of each painting used for fine-tuning the Stable Diffusion image generation model. To teach the model the specific visual languages of the chosen artists, he added terms such as “thick strokes”, “red lighting”, “blue background” and “flat circles” to these descriptions.

Clearly, each of these steps represents a conceptual and aesthetic decision. In other words, the key to the success of Artificial Experiments 1-10 is the creation of a custom database with particular art images and specific descriptions added by the author. This work demonstrates how fine-tuning an existing AI model that was trained on billions of image and text pairs (such as Stable Diffusion) can make this network follow the artist’s ideas. The biases and noise of such a massive network can be overcome and minimized and do not need to dominate our own imagination.

Recommended citation: MANOVICH, Lev. “Make it new”: AI, Modernism, and database art. Mosaic [online], October 2024, no. 201. ISSN: 1696-3296. DOI: https://doi.org/10.7238/m.n201.2404