9 Ways AI Can Interact With Culture

Since 2018, Google Arts & Culture Lab has been exploring the application of artificial intelligence (AI) and machine learning (ML) in an artistic and cultural context. Both as a tool and a collaborator, AI — like other emerging and revolutionizing technologies before it — opens up a multitude of possibilities for artists, technologists and cultural institutions, whether that’s within the creative process, to solve a challenge, or as a new way of engaging with audiences.

Many of our collaborations originate from singular questions: What’s it like to hear color? How can an artist’s archive inspire the creative process? Can I find my doppelgänger in a museum? What ways are there to visualize the climate crisis? How can we make Gen Z dig opera singing? How can curators uncover lost stories?

Questions such as these have driven us for years, and more often than not, AI played the key role in developing the formats and experiences to answer them.

So, let’s take a look at how AI has been of service to the world of culture through 9 of our most intriguing projects.

AI as a tool for solving challenges

Preserving and promoting endangered languages with AI

Of the 7,000 languages currently spoken on Earth, more than 3,000 are under threat of vanishing. Woolaroo is an open source tool that helps language communities preserve and expand their language word lists. Today it supports 17 global languages including Louisiana Creole, Māori, Nawat, Tamazight, Sicilian, Yang Zhuang, Rapa Nui and Yiddish. Woolaroo — with the help of AI powered object recognition — identifies objects in your camera’s frame and matches them to its ever growing library of words. More here.

Surfacing women in science with AI

Women on the forefront of science often haven’t received proper credit or acknowledgement for their essential work. That’s why we collaborated with curators of the Smithsonian to develop machine learning tools that can help to uncover the history and contributions of women in science that were previously too hard to locate. The tools provide curators and data scientists with at-scale analysis and visualizations across nearly two centuries of cultural data. Find out more.

Using AI to restore disappeared masterpieces

Gustav Klimt’s three masterpieces, Medicine, Jurisprudence, and Philosophy, were destroyed during the Second World War. Of these so-called “Faculty Paintings” only black and white photos and articles describing the paintings remained. Working closely with Klimt experts from the Belvedere Museum, we digitally restored Klimt’s Faculty paintings using machine learning to what they might have looked like. Discover more here.

These are just three of many projects targeted at solving challenges our cultural partners face. AI has also been used to help identify artworks captured in old photographs of MOMA exhibitions , analyze large datasets from LIFE Magazine, and make NASA’s visual archive understandable with surprising and interactive results.

AI to experiment with new ways to connect with culture online

Blob Opera on stage

Blob Opera is a playful machine learning experiment that takes your musical ideas and transforms them into beautiful harmonious opera singing — no singing skills required. We used the voices of four professional opera singers to train a “neural network,” essentially teaching the algorithm how to sing and harmonize across four different voice types and incorporating singing synthesis, an area of AI focused on how computers can model human singing. A digital toy for everyone regardless of musical skills, Blob Opera has been used in many contexts, from teachers in the classroom to professional artists to amateur creatives. More here.

Find your doppelganger in artworks with AI

Art Selfie provides a doorway into art, surfacing artwork with faces that look like yours. When you take a selfie, a machine learning model compares your photo with faces in artworks that our museum partners have provided. After a short moment, you will see your results along with a percentage to estimate the visual similarity of each match and your face. You can then tap on your lookalike to discover more information about it and its artist, one that perhaps you’ve never heard of before. Read more here.

AI helps you to “hear” an artwork

What if you could hear color? In this collaboration with the Centre Pompidou, you can explore Vassily Kandinsky’s synesthesia and “play” his pioneering masterpiece, Yellow-Red-Blue, with the help of machine learning. Everyone — perhaps for the first time —can now experience what a “Kandinsky” sounds like by listening to a painting. More information here.

Other projects connecting people to culture in new ways include Giga Manga, which helps you create unique manga characters with the help of machine learning. Sketch a few simple lines and add some splashes of color — then let machine learning fill in the rest. And following the success of Art Selfie, we also created Art Transfer and Pet Portrait.

AI as a tool for artists in their creative work

Create your very own dance moves with AI

This tool for choreography, powered by AI and created with renowned choreographer Wayne McGregor, generates original movement inspired by Wayne McGregor’s archive, creating a live dialogue between dancers and his 25 year body of work. Try out this “Living Archive” here.

Create your own unique pollinator-friendly garden

If Pollinators designed gardens, what would humans see? Commissioned by the Eden Project, Cornwall, UK, and created by artist Alexandra Daisy Ginsberg, this one-of-a-kind interspecies artwork called Pollinator Pathmaker, uses an algorithmic tool to help you design and plant gardens for endangered bees and butterflies.

Create a personalized poem, with the help of AI

POEMPORTRAITS is an online collective artwork created by artist Es Devlin in collaboration with Google Arts & Culture Lab and technologist Ross Goodwin that allows visitors to create poems with the help of AI. It is accessible online & currently on tour as an installation as part of Barbican Centre’s AI: More than Human exhibition.

Through our AMI Grants, we’re also supporting contemporary artists working with machine learning in their art practices, presenting possibilities of artistic application of machine learning since 2019. The grants have helped a diverse set of international artists to collaborate with Google creative technologists to explore machine learning techniques in film, poetry, sound art, and interactive storytelling — with more to come.

We’ll continue to explore the opportunities AI offers at the intersection of art and technology.

By: Amit Sood
Source: Google Blog