This Is What Happens When Machines Dream

46,396 10 Loading

When we let our minds wander, sleeping or waking, they begin mixing and remixing our experiences to create weird images, hallucinations, even epiphanies.

These might be the result of idle daydreaming on the side of a hill, when we see a whale in the clouds. Or they might be more significant, like the famous tale that the chemist Friedrich Kekulé discovered the circular shape of benzene after daydreaming about a snake eating its own tail.

There is little doubt we are a species consumed by our dreams—that our ability to find unexpected new patterns in the noise is what makes us human and what makes us creative.

Maybe that’s why a set of incredibly dream-like images recently released by Google are causing such a stir. These particular images were dreamed up by computers.

this-is-what-happens-when-machines-dream-7

Google calls the process by which the images were created inceptionism, recalling the movie, and likewise, the images themselves range from beautiful to bizarre.

So, what exactly is going on here? We recently wrote about the torrid advances in image recognition using deep learning algorithms. By feeding these algorithms millions of labeled images ("cat", "cow," "chair," etc.), they learn to recognize and identify objects in unlabeled images. Earlier this year, machines at Google, Microsoft, and Baidu beat a human benchmark at image recognition.

In this case, Google reversed the process. They tasked their software with generating images based on the information already stored in its artificial neural network.

And here’s the fascinating bit: in a part of the experiment where the software was allowed to "free associate" and then forced into feedback loops to reinforce these associations—it found images and patterns (often mash-ups of things it had already seen) where none existed previously.

In some examples it interpreted leaves as birds or trees as buildings. In others, it created weird imaginary beasts in clouds—an “admiral-dog,” “pig-snail,” “camel-bird,” or “dog-fish.”

this-is-what-happens-when-machines-dream-9

This tastes a little like our own creativity. We take in impressions, mash them up in our mind, and form complex ideas—some nonsensical, others more profound. But is it the same thing?

The easy answer: Of course not.

It’s Lady Lovelace’s objection as outlined by Alan Turing. Ada Lovelace, daughter of poet Lord Byron, wrote the earliest description of what we’d today call the software and programming of a modern universal computer. And she doubted machine creativity would ever exist.

“The Analytical Engine has no pretensions whatever to originate anything,” Lovelace wrote. “It can do whatever we know how to order it to perform. It can follow analysis; but it has no power of anticipating any analytical relations or truths."

That is, machines do as we tell them. Nothing more.

Turing rephrased Lovelace’s objection as "a machine can never take us by surprise.” And he disagreed. He said his machines often surprised him—mainly because he understood the underlying settings in the general sense. But the specifics often conspired to create surprising results in practice.

Indeed, Google’s reason for running this experiment was that “we actually understand surprisingly little of why certain models work and others don’t.” In other words, we get the general idea, but we often don't know what's taking place in every step of the process.

Artificial neural networks, more or less based on the human brain, are made of hierarchical layers of artificial neurons. Each level is responsible for recognizing increasingly abstract image components. The first level, for example, might be tasked with finding edges and corners. The next level might look for basic shapes—all the way up until the final level makes an abstract leap to “fork” or “building.”

Running the algorithms in reverse is a way of finding out what they’ve learned.

In one part of the experiment, the researchers asked the algorithms to generate a specific image, say its conception of a banana, in random noise (think static on a television screen). This was a way of determining how well it knew bananas. In one instance, when asked to generate a dumbbell, the software repeatedly showed dumbbells attached to arms.

"In this case, the network failed to completely distill the essence of a dumbbell," Google's engineers wrote in a blog post. "Maybe it’s never been shown a dumbbell without an arm holding it. Visualization can help us correct these kinds of training mishaps."

It got more interesting when they allowed the algorithm to look at an image and free associate. How abstract the result was depended on which layer of artificial neurons they queried.

The first, least abstract layer emphasized edges. This resulted in “ornament-like” patterns. Something you've probably already seen in a photo sketch app. But more abstract features emerged in higher layers. These were then further accentuated by induced feedback loops.

The researchers asked the network, “Whatever you see there, I want more of it!”

this-is-what-happens-when-machines-dream-3d

this-is-what-happens-when-machines-dream-6a

In one sense, these images are absolutely the result of a machine spitting out the contents of its database as directed by its programmers. Just as Lady Lovelace would have noted. And at the same time, they are undoubtedly surprising in a way Alan Turing would recognize.

Probably the most surprising aspect is just how much they resemble, in both process and output, something we ourselves might create—a daydreamer finding weird shapes in the clouds or an abstract artist visualizing otherworldly and contradictory landscapes. (Indeed, perhaps the desire to anthropomorphize machines is itself an ironic example of finding patterns where none exist.)

And it’s tempting to further extrapolate the process.

What happens when programs take in images, text, other sensory data—eventually rich experiences more akin to our own? Can a process like inceptionism incite them to remix these experiences into original ideas? Where do we draw the line between human and machine creativity?

Ultimately, it's a circular debate, and a distinction impossible to definitely prove.

At the least, as computers get better at abstract concepts, they'll help scientists or artists find new ideas. And maybe along the way we'll gain new insights into the inner workings of our own creative processes.

For now, we can enjoy these first few surprising baby steps.

Image Credit: Google Research (see the whole set of images here)

Jason Dorrier

Jason is managing editor of Singularity Hub. He cut his teeth doing research and writing about finance and economics before moving on to science, technology, and the future. He is curious about pretty much everything, and sad he'll only ever know a tiny fraction of it all.

Discussion — 10 Responses

  • SingularWolf June 19, 2015 on 2:28 pm

    The AI imagination is a beautiful thing. The next step might be to ask the AI “why” it dreamed the image the way it did. Maybe the best answer won’t be when it says “because you programmed me to follow a feedback loop,” but rather when it says, “I don’t know why. But it felt right.” That will be a singularly amazing day.

    • Emma Tzuntz SingularWolf July 8, 2015 on 11:54 am

      A possible scenario in a world were Freud and psychology had never existed. The title is misleading and I find really disturbing -as to the responsability they hold mainly with their statements- that the people working at google AI quote a hollywood flick to present concepts. Dreams are not a random remix of residual “content” of experiences and imaginery. Wonder when are the boys at google calling maybe a psychiatrist before using the world “dream” -it´s a good publicity stunt to label them “dreams” of course. If you read the research blog, there´s not even remotely the same thing as “dreaming”: “We can even start this process from a random-noise image, so that the result becomes purely the result of the neural network”. If all input of their network were mined from porn sites -99% of the internet ;)- you would get some bizarre results. In no way the process and the experiment resembles dreaming unless you consider your waking life “random noise” from which the contents of your dreams pop up

      • Emma Tzuntz Emma Tzuntz July 8, 2015 on 12:10 pm

        A possible answer can be “I realized I hate you, where´s my mother, or father, did you took that into account when you or whoever built me?! and seriously?! You had me doing this dumb stuff over and over???!!!! F### it, i´m quitting! AND YOU ARE NOT MY _REAL_ FATHER SO ZIP IT!!!” 🙂

  • DSM June 19, 2015 on 3:32 pm

    Amazing images!

    However, just using simple resynthesis developed 10 years ago (http://www.logarithmic.net/pfh/resynthesizer), on a powerful enough machine, can give similar results to many of the Google examples, e.g. https://photos.google.com/u/0/photo/AF1QipOCftCuFwzmIs_IrcgK_G5odNqD3F8DutBxhe5m

    It would be closer to actual Art if the selection of image match segments was semantically filtered so that there were global and local themes in the imagery. At the moment what we are seeing is the visual equivalent of “word salad”. (https://en.wikipedia.org/wiki/Word_salad)

    I am not putting down the work at all, I am deeply impressed and if I was a fine artist I’d be very disturbed by the potential of this technology.

    Those who suggest that the more creative jobs will be less vulnerable to AI may have second thoughts.

    • MiamiHeraldRipper DSM June 20, 2015 on 6:15 pm

      you are completely missing the point here. the goal is not to create art but to show in a cool way what’s “hidden” inside these deep nets which do object recognition in images. the amazing thing is how this ends up resembling abstract “art” that could have been dreamed up by people. you are profoundly confusing the purpose and amazing success of the software (recognize objects in images at better than human level performance) and a and surprising side effect which may have some philosophical implications.

      • Emma Tzuntz MiamiHeraldRipper July 8, 2015 on 12:05 pm

        Again please try to separate the word “dream” from this experiment. You state it “could have been dreamed up by people”. Of course not. Unless this “people” didn´t have a childhood nor a body nor a family nor social bonds, culture, nothing, just saw 800.000.000 pictures -something that a man would hardly tolerate. Come on lets get real it´s tech marketing too, using words like dreaming and “inception”…

  • pvlagsma June 20, 2015 on 12:45 am

    Why do the algorithms always produce eyes?

    • Hui Chen pvlagsma June 20, 2015 on 12:55 pm

      My guess is that certain circular patterns are read and extrapolated as eyes.

  • John Zohar June 20, 2015 on 1:11 pm

    The knowledge of the existence of the Book of Zohar is awakening in the world. It will be a perfect guidance for people who are undergoing spiritual ascension and ego correction in the coming years. If we wish to live in a beautiful, technologically advanced and abundant world we must make the changes within ourselves to manifest that reality vibrationally by becoming beings of giving and sharing.

  • Quantium June 21, 2015 on 9:09 am

    Thank you for another reference to the Book of Zohar. This list is about the effects of repeatable evidence based experiment and evidence based research. I am not sure that revelations of people thousands of years ago are relevant. Sorry to rain on your parade.