Last week, OpenAI released their newest feature for DALLE 3: in-painting. For those who don’t know, in-painting allows you to highlight a section of an image and use Generative AI to alter just that area. Don’t like the decorations in the background of your family photo? You can highlight a section of the couch and add a statement pillow. Your wedding photographer took the cutest candid of you but there’s a hair tie on your wrist? In-painting allows you to point to that section and remove it in seconds.
I’m pretty familiar with DALLE’s capabilities at this point, so it was cool to see how they’ve introduced in-painting to the conversational interface of ChatGPT. I was already playing around with generating some cinematic images of wolves when I found the feature, and figured the best way to test it was to change the wolf in my image to something equally ferocious: a mountain lion.
Just to see what my baseline was, I highlighted the section around Fido here and gave the simple command “a snarling mountain lion.”
This guy is giving less “wild animal” and more “feral neighborhood cat.” While he is kinda cute, I was hoping for something a little more ferocious. After highlighting him and giving the more specific command, “The front of a large, snarling mountain lion, angry and violent,” I got the following.
I hate this guy so much. Genuinely, in real life, he made me recoil in my seat at the sight of him. It was definitely the “the front” part of my prompt that confused the model, but, out of spite, I refused to engage with mountain lions anymore and tried to shift gears a little. Highlighting the above monster’s freaky floating head, I typed in the much simpler “lion” and got this.
A lion! He looks a little out of place in the image, more like he was photoshopped in that in-painted, but at least I was on the right track. With the prompt, “a lion stalking forward, prowling, one foot perched against a mound of grass on the ground” I finally got something I was satisfied with.
I really like how this looks like an action shot. The lion in the image has this mysterious look to him, and blends well with both the style of the image and the scene itself.
With in-painting, it’s important to be extra specific with what you’re looking for, even if you risk a mountain lion MODOK scaring you at your desk. Using tonal and compositional words while directing the new subject from within your prompt will help you get to a solid, realistic result. These modalities aren’t perfect and are only as good as the prompts you give it. And, clearly, they’re bigger fans of real lions than mountain lions, but I’d still go running after that little feral black cat if I saw him in my yard.
Annika McTamaney
