#ai tutotrial

[deepdreamnights]

How I summoned the Demons

Bringing the Lesser Key of Solomon, Hieronymus Bosch, and a handful of oddballs into video:

Now, since every AI tutorial is someone's first, this was not a simple "type in what you want and get a video out." situation. The push-button-finished project is a myth. Today we're working with Vidu, Midjourney, and photoshop, among a handful of others.

I started with a Suno song that I generated, using parody lyrics I wrote for Carol of the Bells. I wanted to do a project involving a demon Christmas, but not Christmas demons (so no Krampus, Grýla, Yule Lads, Yule Cat, Mariah Carey, or undead pickle boys)

So Barbatos and Yan-Gant-Y-Tan from the Dictionnaire Infernal (which I manually remastered back in 2019) will be our examples.

Midjourney added overpainting through their "edit" feature about a month ago. It's highly limited, has much stricter censorship controls than the standard generator, and you can't iterate the images in the normal way, but it is useful for extending image edges or getting a very specific character design from one style to another.

Starting with Barbatos, I used the "retexture" option, with the following prompt:

photograph for casting, a demon with green skin and a gray beard and hair, he is dressed as a 15th century noble huntsman, in vibrant silken clothes (black, violet, yellow), he carries a long musket, two golden trumpets fly near his head, he is standing on a snowy hillock, a Christmas village in the far distance

The illustrations by Louis Le Breton were not in color, and the text is light on description, giving me room to play with the concepts while still keeping them (hopefully) recognizable.

I liked the idea of Barbatos as a green figure with a gray beard, and one of the results hit very close to what I wanted, but some photoshopping of the gloves was needed.

For an image post I would edit the final version a lot more, but he wasn't going to show up except from the waist up, so giving him proper old timey stockings wasn't high on my priorities list. That, and Vidu's object reference feature is rather broad and doesn't require the polish a finalized image does.

Yan-Gant-Y-Tan, on the other hand, was a more elaborate combo because I had a direct idea for what he'd look like. The demon is a french will-o-wisp spirit (a 'John with the Fire'), with a hand of candle-fingers hand-of-glory style that it uses to lure enemies in. A literal "Candle Jack."

So immediately I wanted something that was made of yellowish candle wax made semi-translucent by heat.

I wound up not having to put him in longjohns like I feared I would, and out of a large stack of results, three had aspects I specifically liked. The heads did not, however, turn out remotely correct, so I clipped out a closeup, and did that separately.

Then came the photoshop compositing and editing, including manually adding candle flames.

Resulting in this handsome devil.

The resulting images served as "actors" that I could drop into Vidu. A lot of video AI systems have a "starting/ending frame" feature (Vidu included), and when you want more control over the shot that's often the way to go (along with longer, more script-format prompts).

But Vidu's object reference makes things a lot easier. Things are still "soft" in a lot of ways. Fine details will change, asymmetry can give it problems, but it lets you use a single character image, or a model sheet, without the exact pose or location starting the shot.

So having Barbatos shop for gifts and then take those gifts to an apartment can be done with the same "actor image."

Other shots, like the Bosch ear-knife, required a starting frame. The robot has no idea how to handle something this weird.

I started by retexturing the ear-knife from the original painting several times, then I composited that together, did color changes, and removed the background, placed another generated background under it, then used that as the starting frame.

Even then the ear monster mostly stayed still because every time it moved it became an unholy(er) abomination.

Which while somewhat cool didn't fit the vibe of the project.

Speaking of Jankenstein's Monster

The stuff that gave me the most problems were most of the bosch-creatures, Buer, and King Charles Ba'al. The sled-riding blob demon and funnelbird were exceptions for Bosch, being generally cooperative, but everything else was a lot of trial and error.

And Ba'al? Vidu didn't know what to do with Ba'al.

To be fair, neither did Midjourney (or Louis Le Breton, for that matter)

Demonology Ba'al is very clearly built from a bunch of creepy things with no real understanding or concern about how they'd mesh together (making him the most biblically angelic demon in the batch, IMHO). In the end I wound up photobashing him together with some AI touchups.

I wound up using a start-frame for Hailuo for him, because Hailuo will default to minimal movement it doesn't understand what's going on.

Anyhoo, here's some Christmas Demons for you to enjoy:

Carol of the Hells (Scary Wicked Deadly Evil Christmas) The DeepDreamNights Robot Band Feat. The Trans-Stygian Orchestra

Even the unfathomable legions of the damned love Christmas, and like any musical act with a general disdain for mankind, the DeepDreamNights Robot Band was way overdue a Christmas Song.

So summon up these Christmas spirits and see how many demons, devils and evil spirits can you identify?