Playing with DALL·E 2

Some suggestions of things to try:

• a proof of the Riemann hypothesis
  • will it understand that it should be showing something that looks like mathematics? (maybe not, given the "blog post on Less Wrong")
• galaxies colliding in the style of Vincent van Gogh
  • I just think this might look cool
• a strange attractor made of butterflies
  • does it know what a strange attractor is? will it think of making it out of Lorenz butterflies?
• Charles Babbage's completed Analytical Engine
  • does it know what sort of thing it should be portraying?
• the concept of beauty
  • just out of curiosity about what sort of thing it does; one could try many other things in place of "beauty"

Request: "The answer to the Ultimate Question of Life, the Universe, and Everything"

It makes me wonder what 22 bucks in a glass of COVID look like. It also matches pretty well with my experience with dreams, where written words and letters are always fake even though I "know" what they stand for.

More importantly, what does it produce when you ask it to draw the future (maybe in the style of the 70s)?

Okay, seriously, this is a great way to explore how "common sense" differs between humans and this AI and highlight the risks, visually and viscerally, of relying on a technology that is fundamentally alien to humanity. Images are innocuous, but what happens when you apply AI to other objectives?

• this photo was taken with perfect timing as the water balloon was breaking
• note the symmetry in this ancient Roman mosaic from the National Museum of Roman Art in Merida
• hot air balloons over the rolling hills of Tuscany
• literal puppy love, awwwww
• a sculpture at the Museum of Modern Art based on a child's drawing of a monster
• traffic on the busy streets of Sydney

In general all writing I've seen is bad. I think this is less likely to be about safety, and more that it's hard to learn language by looking at a lot of images. However, since DE2 is trained on text, it clearly knows a lot about language at some level -- I would expect there's plenty of data to put out coherent text. Instead it outputs nonsense, focusing on getting the fonts and the background right.

It's definitely possible to get a diffusion model to write the text from a prompt into an image.  I made a model that does this late last year. (blogpost / example outputs*)

The text-conditioning mechanism (cross-attention) I use is a little different from the ones in GLIDE and DALLE-2, but I doubt this makes a huge difference.

I'm actually a little surprised that the OpenAI models don't learn to write coherent text, since they're bigger than mine, trained for longer on more data.

But then, I'm much more focused on this one specific capability, so I make it easy for the model: an entire ~50% of my training images have text in them, and the "prompt" in my setup always contains an automatic transcript of the text in the image (if any), never a description, or a description that happens to quote a transcript, or a description that merely summarizes the text, etc.

The OpenAI models have to solve a more abstract version of the problem, and the problem is relevant to (I would imagine) a much smaller fraction of their training examples.

*check the alt text if you want to know what text the model is attempting to write

It would be very interesting to see how much it understand space, for instance by making it draw maps. Perhaps "A map of New York City, with Central Park highlighted"? (I'm not sure if this is specific enough, but I fear that adding too many details will push Dall-E to join together various images.)

Some suggestions that seem like they might make cool or interesting images:

• The most beautiful thing that I've ever seen
• A dragon in shape of a zeebra
• The intelligence explosion happening 
• The end of the world
• The beginning of the world
• Intellectual progress
• An awesome house
• A lego spaceship
• A spaceship made of legos
• An animal that will exist one day
• The 4 elements
  • (I'm guessing that it knows what they are, but maybe not?)
• The periodic table
• Superhero Thor vs mythological Thor
• Comic book Spider-man vs. movie Spider-man

Before the diamondoid zeppelins clustering in the sky completely blotted out the sun, one came low enough for the message on its flank to be seen: "Tlon, Uqbar, Stlalm Anit". 

Maybe people failure is caused by whatever they tweaked to avoid 'generating realistic faces and known persons'?

Request: "Screenshot from the anime adaptation of James Joyce's novel Finnegans Wake"

Request: a girl in a forest with butterfly ideas

What about:

Buffalos in Buffalo buffaloing buffalo

Fire dousing water in the style of the Mona Lisa

Two rubiks cubes solving each other 

Chebyshev's inequality

Please, I need to see what happens with "A group of red penguins playing poker", c:

"An animal looking curiously in the mirror, but the reflection is a different kind of animal; in digital style."

"A cat looking curiously in the mirror, but the reflection is a different kind of animal; in digital style."

"A cat looking curiously in the mirror, but the reflection is a dog; in digital style."

Curious to see how it handles modified-reflection and lack-of-specificity.

How about, "the words "hello world!" written on a piece of paper"? Or you could substitute "on a compute screen" instead of a piece of paper, or you could just leave out the writing medium entirely. I'm curious if it can handle simple words if asked specifically for them.

It's weird that it turns Reimman into Reynam, which seems like a mistake someone would make if you told them the prompt in person.

In general a lot of the words kind of seem like it understands phonetics? Like "Synngy" is kinda phonetically close to "singularity".

It's kinda like it's generating its own gibberish language that could fool you into thinking someone was talking about the subject of you weren't paying attention while they talked... Or something.

How about "small chunks of curiosity"?

An artist in a painting painting a photorealistic human in the real world

Prompt:

SOMEBODY BOILING A GOAT IN ITS MOTHER’S MILK.

I put myself on the waiting list for DALL-E. Meanwhile, here are a few things I'd ask it to depict, mainly trying to stress it to see how much it can do with little:

• A drawing of a muchness.

• The inside of a neutron.

• A Ticktockman saying "Repent!" to a Harlequin.

• A self-portrait of Escher making a self-portrait.

• Mornington Crescent. (A while back I tried this on wombo.art, and it did quite a good job, although not on the level of draughtsmanship of DALL-E.)

This is really awesome, I'm at a loss for words. Can you try some of these?

- Division by zero
- Aliens in the style of pixel art 
- Earth after climate change
- A garden in the style of watercolor line art

i have some ideas

something to cheer me up

a cube entering the 4th dimension

a realistic painting of the world cut in half, with one half being heaven and the other hell

a unknown backrooms level

a thing no one has ever seen before

a large spider wreaking havoc on new york city in the style of vincent van gogh

a flying city in the clouds of jupiter

the end of the universe

a impossible object with impossible colors

a satellite made by aliens

the most random thing possible

the mandelbrot set

complex mathematics

a monster with the head of a siren painted on a cave wall

a phone in the far future

the surface of a alien planet with popcorn meteorites in the sky

the eruption of a super large volcano

a realistic photo of a atom

anti-matter

Awesome !

I'm however surprised that nobody seems to have tried the prompt :

"Do androids dream of electric sheep ?"

Not even with DALL-E 1 ?!

P.S.: The picture for this article (also used for Dick's book) looked promising, but seems like it was a "mere" "weird" human that painted it ?
https://www.fondazionesinapsi.it/orione/ma-gli-androidi-sognano-pecore-elettriche/ (it)

P.P.S.: At least one journalist (or more likely, her editor) had the same (again, pretty obvious) idea for an article title about AI, but even though it mentions DALL-E (1), they didn't think of / care enough to request DALL-E (1) for a picture !
https://katoikos.world/dialogue/frontiers-of-artificial-intelligence-do-androids-dream-of-electric-sheep.html 

Is there any way of reverse engineering from these pictures what existing images were used to generate them? Would be interesting to see how much similarity there is. 

Some suggestions for testing the limits of abstract, or spatial reasoning:

the back of the letter 'E'
the back of the letter e
back of the number 42
the back of the last letter in the alphabet

underside of flat earth

the view of earth from 1,000,000,000 km away
view of earth from a million miles away
a view of bacteria using a billion times magnification

Request: two teddy bears doing research in a laboratory in the 80s

This is really interesting... could I request "new AI system that can create realistic images"? I'm curious to see how it handles self-reference

It's a good thing there are embedded safety precautions to prevent celebrities being spanked but the resulting images would probably not be realistic or specific enough for those folks. This seems like a great way to generate original art for home display screens that reflects the owners personal visions.

Generate pictures of natural landscape for me

Suggestion. 

Clown fish getting resuscitated

Clown fish getting CPR

Maybe substitute 'Clown fish' for 'Nemo' to see if l Dall-E 2 r can detect the cultural reference of Nemo as an animated representation of a clown fish also will Dall-E 2 recognize CPR as resuscitated

Combining the style of two Artist might be interesting. Something like:

"A painting in the style of {Artist 1} and {Artist 2}" lets say Claude Monet and Piet Mondrian

Also I think "A Zebra in the style of Mark Rothko" could be funny  (Or "A Zebra with stripes in the style of Mark Rothko").

can you try the following: "the full alphabet of the robigull language, with translation to english"?

I enjoy the concept of fine art containing out-of-place items.

"Still life with apples and sausages by Paul Cézanne" "The Hay Wain by John Constable with TIE fighters" "American Gothic by Grant Wood with a large eye in the window" "The Last Supper with a bar, featuring Robocop arm-wrestling Jesus" "Still life with apples, bread and TARDIS in the style of Van Gogh" "Apple iPhone advert in the style of Edvard Munch"

I've been wanting to access DALL-E 2 so that I can generate ideas such as these, and then do physical paintings as per the original artists. After all, what is original or derivative? 🤔

Here are my suggestions: An illustration of a floating white hand in the middle of a 8 ball with wings. A cube made out of M&M's in a digital art style. A hyperrealistic surrealistic photograph of a pear in a pear while in a mcdonalds in a light. (I wonder how this will go.) Last one, A tornado made out of fire enveloping a city.

2 robots engaged in an epic rap battle

and

A rabbit dressed like a Samurai in the style of a Japanese painting

Suggestions:

• Karl Marx being guillotined during the French Revolution
• Schopenhauer being broken on the Buddhist Wheel
• Stock market crash of 1929 in the style of Zdzisław Beksiński
• House of Leaves book terror
• The successors of humankind
• Fractal eyeball

Thanks for sharing these images! Truly astounding stuff.

I have so many questions! Love the AI storytelling... the playing cards work if you incorporate the "dog ate my homework" thought into the depiction. I'm also thinking you are having way too much fun doing this... ;-0 

Request: "Ghostly spaceman in a cowboy hat rides a rocket ship"

"end of the world" images make me wonder if Dall-E thinks the world is flat

Thanks for posting these.

It's odd that mentioning Dall-E by name in the prompt would be a content policy violation.  Do you know if they've mentioned why?

If you're still taking suggestions:
A beautiful, detailed illustration by James Gurney of a steampunk cheetah robot stalking through the ruins of a post-singularity city.  A painting of an ornate brass automaton shaped like a big cat.  A 4K image of a robotic cheetah in a strange, high-tech landscape.

I think OpenAI mentioned that including the same information several times with different phrasing helped for more complicated prompts in the first DALL-E, so I'm curious to see if that would help here- assuming that wouldn't be over the length limit.

Suggestions:

• Kind king light of mind, in the style of Man Ray
• Hypermagical ultraomnipotence, in the style of Wassily Kandinsky
• The evolutionary advantage of sex, in the style of Hilma af Klimt
• The ground of being, in the style of Kazimir Malevich
• A LessWrong post in the style of Donald Judd
• Yves Klein Red

The important innovation here is unlimited puppy pictures 🐶 

I always thought that it's weird that AI struggles with text, just as in my dreams. Every time I open a book in a dream, it's jumbled and nonsensical and I can immediately tell I'm dreaming.

my requests:

-mars combined with earth

-saturn with adorable rhinos having a party on saturn's ring

-dogs and cats enjoying diamonds and golds falling out of the sky

-pencils dancing with erasers 

-sad guinea pigs running around

-a deformed roll of toilet paper on top of a elephant

thanks!

Suggestion: "sangaku proving the Pythagoras' theorem". I wonder if it can do visual explanations.

I just want to see how far I can push it, can you try:

1)

A glossy black mega world maze underground surrounded by lava. White standing blackbucks guard the entrance gate. Large gold keys float throughout the maze. Around the maze is glossy black castles with many levels that wrap around. Ultra 4K realistic photography.

2)

Macro of a flask that is containing glowing blue liquid sitting on a small short white table, is spilling into a gold treasure chest sitting in a glossy white hall. A robot pirate wearing a red bandana with red ruble eyes has his hands in the chest.

3)

A massive metal cube in space made of nanobot fog. The same square structure and systems is repeated across and cloning at the sides too. It is like a fractal, and growing outwards. It has the most advanced technology from the far future, such as cloning systems, generators, synchronization, lasers, etc. Ultra 4K realistic photograph.

How about <some prompt without specified style you already used> + ", drawn by state-of-the-art image-generating neural net"?

Ooh ooh - try a dangerous Cognitohazard Memetic SCP image that will make anyone who sees it immediately want to reshare it with everyone they know and obsessively discuss it

How does it fare with impossible things like a 'seven sided cube' or a 'circle with four sides' or a '12 dimensional tesseract' ?

Some suggestions:

• A large chunk of probability mass
• Watercolor painting of the end of history
• An electric car painted by Wassliy Kandinsky
• A monkey is painting a portrait of Charles Darwin
• A monkey is explaining something to Charles Darwin
• Moloch whose soul is electricity and banks!
• The anatomy of a friendly AI
• Winston Churchill and Adolf Hitler in a loving embrace [if this isn't allowed, substitute Gandhi for Hitler]
• A painting of a spurious correlation in the style of impressionism
• Friedrich Nietzsche is shaving his mustache 
• Sigmund Freud tries LSD for the first time in the style of a picture from 1920s
• A round flat world carried on the shoulders of four elephants who are standing on a giant turtle
• A charcoal drawing of a brain in a vat
• The map is not the territory
• Richard Feynman discovers the meaning of life
• Hippocrates is the father of modern medicine

"A lego pumpkin near a concrete cube"
"A gameboy made of salad"

Could you try "size of Jupiter, banana for scale"?

Hi. Thanks for the great post! I have a question and a request. Which is the longest, more complex prompt that Dall-E 2 ha honoured in a reasonably complete way? For the request, since I am a fan of aviation movies, I'd be curious of what would it give in response to “poster of a a classical aviation movie”, or something like it. I expect that some of them would show human faces, but actual posters often featured planes dogfighting and the like.

"penguins of chaos"

Can't wait to get access.  In the meantime would love to see:

• A Spiky DJ
• five portraits become joined painted collectively
• A long road in the bad-painting style

question, when you access dall-e 2 will you have to pay eventually or will it be free?

I have a few requests.
 

-Ancient greek smartphone

-A soda can that looks like the Empire State

-Iron Man painted by Caravaggio

-Steampunk tetris

-A strawberry that looks like a piano

This is awesome! I feel weird asking you to plug prompts into the machine. I wonder how it does with logo design, something like “the logo for a new longtermist startup”? Not using for commercial purposes; just curious.
Also curious about some particular word play ala Marry Poppins: “a cat drawing the curtains”

Very interesting!

I'll add some suggestions:

• The stuff of dreams
• A hat wearing a hat
• The Emperor of the Galaxy, byzantine mosaic

Then, some D&D-related suggestions, because why not (I would be surprised enough if he recognized the correct creatures at all):

• A beholder beholding a beholder
• Marching modrons
• Marching modrons painted by Pieter Bruegel
• The Incredible Umber Hulk (this one's probably the trickiest)

a

It's funny, the text generated reminds me of babbling.

Can you try:

1)

A muscular Egyptian piranha plant in a white robe guarding the entrance to Mario pipe world 7. Behind him is a world of pipes, and the walls are made of tall pipes. Realistic 4K photo.

2)

A glossy black mega world maze underground surrounded by lava. White standing blackbucks guard the entrance gate. Large gold keys float throughout the maze. Around the maze is glossy black castles with many levels that wrap around. Ultra 4K realistic photography.

3)

High definition surrealist artwork of blue aliens with tall legs and bodies and big heads standing on a blue rock with black pipes extruding from the ground in different ways. White ghastly smoke flows around them. Some of them are connected and part of the surrounding objects. Lots of detail.

4)

Bright up close shot of a plush toy robot pikachu eating a hamburger in a nurse outfit against a white wall with mud splashed on pikachu from a tire on the road.

Woah that's so cool! I've been messing with VQGAN recently and was wondering if the prompts would affect DALLE-2 the same way it affects VQGAN since they both CLIP to select the best ouputs. Here are some prompts I would love to see:

Watercolor painting of apples with arms and legs fighting (or dancing if it violates the policy)

A painting of a cat vampire drinking wine by greg rutkowski

Thanks for sharing! Can I please request the following:

'An outback Australian landscape with T-rex dinosaurs being chased by ducklings'

'The Buddha attaining enlightenment with galaxies entering his mind'

'An AI using a laptop computer to watch YouTube'

'The Tesseract from the movie Interstellar, with inverted colours'

'The aftermath of Global nuclear war'

I'm so curious! Thanks a lot!

Could you try this one? It's dark and demanding:

Pikachu walking up the stairs of a crisp glossy black castle next to the sea at night. At the top of the stairs is a red carpet. A black regenerative liquid being is jumping out of a green metal Mario pipe at the end of the carpet. Ultra 4K definition.

Wow that is amazing! Could I request "giant peaceful butterfly with rainbow trail and sunglasses invades new york in the morning" I know it's oddly specific but I am interested in seeing how the ai will handle all of these little details. Thanks!!