🎥｜animation

[inner crest]

https://www.youtube.com/watch?v=MXRjTOE2v64&t=780s

The guy who created this tutorial and even Parseq is on the Deforum channel

[safe oriole]

yea i have watched the tutorial, i had been pondering learning parseq, was just curious on time cost involved 🙂

[violet bridge]

https://youtu.be/g_J_blTF18U been working on this one for a while. Enjoy 🙂

[grand copper]

https://www.youtube.com/watch?v=C0T0Y5yZlmQ
Shooting Stars amv with ai

[inner crest]

I would say if you know deforum well. It goes fast. But still need a lot of practice. And a lot of parseq users use it for the interpolation mode.

Most of animations I have seen are not really smooth or consistent. They sync everything to a intense beat

And let the chaos generate the animation 😂

Not my taste personally

[safe oriole]

not my taste either, i like being able to see whats going on and not be overwhelmed by randomness

[wispy spire]

What Ai stable app do y'all use for animations, I am a bit new so I don't know better apps

[surreal portal]

Recent improvements to my interpolation algorithm lets this pipeline use a lot more RIFE and results in much smoother motion. Can afford higher steps too now. Should be released to the app store in the next week or so

My AI Friends - literally just the friends theme + a cinematic theme and it spat this out (roughly 6 minutes render and a minute for the GPT direction)

I can probably get these render times down a lot if we get users, there's plenty of opportunity for parallelising stuff

[inner crest]

Deforum

[fallen girder]

"The Rise of the AGI" (Stable Diffusion + SadTalker + Vicuna-13b) [4K] - Workflow Included
https://youtu.be/mT-uwdOleKQ

[stable hawk]

Text to Image in 1.5

[rotund inlet]

Can you share your workflow? thanks for anything.........

[echo kite]

Of course, thank you for asking. https://github.com/sj-data/stable-diffusion-webui-4koma-gif2grid I will be making a more detailed walkthrough soon but basically:

Split a gif into frames, turn the frames into 2x2, 3x3, or 4x4 grid and run through controlnet. Upsample, turn back into gif.

Please star if you find it useful, I am working on automating the process so only the last frame is changed each iteration making it possible to create long animations. Follow the repository for updates.

[stoic sage]

@echo kite

I think it's absolutely beautiful and brilliant

Do you mind if I add you on here

[echo kite]

Please do, also let me know when you encounter issues. This is my first Gradio project.

[stoic sage]

Will do sir

[inner crest]

https://youtu.be/KUOwbcdZozQ

I wonder if I could make a video from this ?

Something mixing parseq + ControlNet + embedding + my genius madness

https://tenor.com/view/i-accept-the-challenge-captain-kirk-star-trek-the-original-series-challenge-accepted-gif-23575421

And obviously 100% anime

[inner crest]

something big is coming...

And I dont talk about the stuff under the belt...

[whole fractal]

https://youtu.be/vJvzPS0fLmA

[oblique ivy]

Here is a modelscope AI animation:

footage of the end of the world

[neon coral]

Deforum

[rugged prism]

You can and should implement gifski

And stop using pil for this

[sage yacht]

first time dipping my toes into animations

[echo kite]

Thank you for the advice I will research this.

Trained a small LoRa to handle gSFW grids, still needs work but I'm very pleased so far.

[earnest sigil]

https://youtube.com/shorts/Wx9k95d7mek

[waxen locust]

My first stabilized video with SD :3

although have a bit error :3

[neon coral]

My New Video 🙂 I try to remove flicker with Resolve (Removedirt, Deflicker) take a half day to render. https://www.youtube.com/watch?v=G-G3B-DMl3Y

[safe oriole]

that is very clean indeed!

[neon coral]

Yes, i like the result. But the rendering time was really way to long.

[rugged prism]

Old aspect ratio formula it seems

The depth warping is acting weird

[neon coral]

I used : "use_depth_warping": true,
"midas_weight": 0.2,
"padding_mode": "border",
"sampling_mode": "bicubic",

What u think i can improve here ?

[stoic frigate]

Using Diffuser's implementation of Modelscope through an add-on in Blender. Interpolated and scaled in Davinci Resolve. Sounds from Freesounds. Music by Mubert:
https://www.youtube.com/watch?v=auHu02KJZQs

[humble dome]

That is really impressive

[dense crater]

https://www.youtube.com/watch?v=uDqnotU18uo

[dark sage]

https://youtu.be/uvEPTYKUw4c

https://youtu.be/-SDbH_Hxzf0

[humble dome]

Custom text2video method. It allows to create animations of unlimited length, at any resolution your GPU can handle, using any SD model as a backbone. Comming soon to SD-CN-Animation.

[earnest sigil]

https://www.youtube.com/shorts/uHWZIUMRVKE

[white pelican]

Hey everyone! Excited to share my new YouTube channel where I'm combining phonk music with AI-powered video creation using stable diffusion. If you enjoy the content, I'd be grateful if you could check out my channel and show your support! https://www.youtube.com/watch?v=Tf0vVHAAqxM

[tight arrow]

holy this looks awesome! it looks like an anime

what other videos are you gonna make

[stable sky]

https://www.youtube.com/shorts/g-MHvHLhtZ8

[little glen]

Cadence, FILM interpolation, and slowmo in Deforum.

[earnest sigil]

Join the video animation community https://t.me/NeuroZone_chennel

[mossy ridge]

#🎥｜animation future

[vapid skiff]

@mossy ridge

FAQ: How do I generate images? Is there a bot on the server?

Currently, there is no bot on the server that generates images. However, there are plenty of other ways such as the official https://beta.dreamstudio.ai/ website or by running Stable Diffusion locally using your own hardware! Check out #1080946152318443610 for more details! You can also stop by #1025467151206854736 for any issues you experience while using DreamStudio or #🤝｜tech-support for any problems you encounter while installing it locally!

[earnest sigil]

https://t.me/NeuroZone_chennel/42

[surreal portal]

A torch port of FILM would be super useful

[burnt fossil]

Hi, I`m using video init with deforum to make a upscaled somewhat different version of animation. When I reduce strength it just becomes a noisy mess, seems to ignore prompts entirely. Any idea whats going on? Or does video init simply not work with prompts? Using webui

[plucky plover]

Peaches - The Presidents Of The USA but with the SM64 soundfont & Deforum https://www.youtube.com/watch?v=veCmXC42ADc

[next knoll]

cctv camera

[little glen]

Used the Automatic1111 depth plugin to create a synthesized depth map and then used it to render a scene in Blender. Started by rendering an image in stable diffusion of a cave scene and then ran it through the depth script. After that, set up a scene in Blender, imported the depth/texture file with a plugin and added point lights + Poly Haven assets, and keyframed a couple of camera shots before rendering it in Eevee. There were no shadows in the final render because I wasn't unable to get the shadow catcher to work with my mesh. The links to the script and the plugin are in this thread. https://twitter.com/Render_Artist/status/1648240901616201729

[indigo garnet]

https://youtu.be/greAWmAWN1A

[neon coral]

Ecstasy Club – Jesus Loves The Acid / made with StableDiffusion+Deforum+Parseq - Deliberate-V2/ Beatsynced to the Break with Resolve https://www.youtube.com/shorts/QLk_srBSRyo

[plucky plaza]

#1072220168534642768 create jesus dancing on the moon

#1072229020520947753

[vapid skiff]

FAQ: How do I generate images? Is there a bot on the server?

Currently, there is no bot on the server that generates images. However, there are plenty of other ways such as the official https://beta.dreamstudio.ai/ website or by running Stable Diffusion locally using your own hardware! Check out #1080946152318443610 for more details! You can also stop by #1025467151206854736 for any issues you experience while using DreamStudio or #🤝｜tech-support for any problems you encounter while installing it locally!

[plucky plover]

Use Deforum + Controlnet to make Pixelated version of the Super Mario Bros 1993 trailer with a Blink182 song made with SM64 Soundfont https://www.youtube.com/watch?v=gxmhgiJhinc

[tired wing]

Hey Animators! Our first animation challenge is live over at #1097851711697203220

Take a look, have fun and win some prizes!

[raven sail]

[warped ridge]

Cool, dude. If you're interested in making your grid dimensions more dynamic, MakeGrid() and BreakGrid() functions are pretty simple to follow: https://github.com/LonicaMewinsky/sd-webui-keyframer/blob/main/Scripts/keyframer.py

[earnest sigil]

https://t.me/NeuroZone_chennel/44

[surreal portal]

Boats and ships are always funky with SD for me

[misty dagger]

Like a multiverse wedding

[humble dome]

SD-CN-Animation can now to text to video, using any SD model as a base
https://www.youtube.com/shorts/zjBX0zyHzQE

[faint rivet]

https://www.youtube.com/shorts/koqS6tkcbNU

[native bison]

/create HEATWAVE ALERT IN STATES, TEMPERATURE NEARS 45°C

[cold trail]

Me, myself and them.
https://youtu.be/JHdgSS9IygQ

[celest fossil]

highly guided character animation, with deflicker for output

the frame dont sync perfectly

because i forgot to delete the first two frame which is not used in the animation for the right side

[dense sonnet]

A single shot animation with a mix of 4 base prompts and added variations, rotation, and zoom. I let Lady Luck go with the flow. 😎
https://www.youtube.com/watch?v=nf1SnSXt2do

[pliant spoke]

собака

dog

[fallen girder]

"Lambada 420" (ModelScope text2video / Animov 0.1) [4K] Workflow Included - https://youtu.be/0cp8NoikbSo

[winter trellis]

Awesome, How did you do that ? You should upload to youtube so I can share x)

Does anyone know about a auto1111 extension taking a mp3 and generating a sequence of denoising strengths based on the beats in the mp3 that can then be used for deforum in order to sync the song and animation ?

[rugged prism]

Framesync.xyz

[winter trellis]

thanks

[white pelican]

Blade Runner by Stable Diffusion || Suffer With Me
https://www.youtube.com/watch?v=PVXHRLcyPoY

[earnest sigil]

https://t.me/NeuroZone_chennel/47 my job

[plucky plover]

Y2K commercials turned pixel. Made with 1.1 Controlnet + Deforum + Hybrid Video 🌟 + Spice Girls MIDI track with Super Mario 64 sounds. https://youtu.be/idBr3gc4Qgs

[surreal portal]

More automated prompt interpolation fun, Chop Suey comes out pretty amusing

[jovial flare]

Made a little short film about how different artforms influence one another. Used Stable Diffusion + Controlnet + Deforum:

https://youtu.be/HNVUPB7KDRA

[earnest sigil]

https://t.me/NeuroZone_chennel/51 ✌️

[dark sage]

FrAiday Mind Dump https://www.youtube.com/watch?v=hFiXSn6f7y4

[bright stream]

Hi! I'm extremely out of the loop on animating, deflicker, and controlnet as it pertains to animation. If I show a snippet of something I'm working on, could somebody maybe briefly tell me if any of the new tools/techniques should be used to improve it?

[earnest sigil]

https://t.me/NeuroZone_chennel/52

[white pelican]

Fight Club by Stable Diffusion || Where Is My Mind?
https://www.youtube.com/watch?v=i_083xxWbxM

[stable hawk]

The Walk made with Stable Diffusion. 2D Animation style Walk cycle. Muah haa haa haa haa. Get on my text to image cohesion level lol. The brag is a joke lol, I am excited though lol.

[whole fractal]

https://youtu.be/UyFS4PGtzd0

[granite flicker]

I love ControlNet 1.1! Here's a video I made with Automatic1111 + Deforum+ ControlNet 1.1 with a custom LoRA and Deliberate_v2 checkpoint:
https://youtube.com/shorts/6R7omF4QHaI?feature=share

[faint rivet]

https://www.youtube.com/watch?v=GGDUm3eQeCA

[fallen girder]

Stable Diffusion + Blender (AI Generated 3D Environment) Quick Animation Test: https://youtu.be/sIVxQeAkme4

[real cliff]

Stable diffusion with deforum plugin, setting used are in the youtube description https://youtu.be/gHu1_0G95tI

[fringe carbon]

Somebody in a local discord posted this odd doggy painting so I decided to help them out

[dense holly]

In my upcoming short animated film I wanted to explore alternative ways of creating professional animation aided by the visual power of SD Here's its trailer, as story inspired by "I, We, Waluigi: a Post-Modern analysis of Waluigi". https://youtu.be/YaJo4EPAtio

[dense holly]

I'll also be sharing the behind the scenes, where I will show how I used SD in my workflow, through fine-tuning, 3D animated characters and a lot of attention to details 😉

[earnest sigil]

https://t.me/NeuroZone_chennel/55

[wicked fjord]

Tried doing the animation Corridor Digital taught. https://www.youtube.com/watch?v=RoKDp8KxXHQ

[dense crater]

https://www.youtube.com/watch?v=A6cX3g5cEmw

[white pelican]

https://youtu.be/i_083xxWbxM

[faint harness]

https://youtu.be/TqYx22vujJ8

[dark sage]

AI-xited My latest contribution to the AI scene. Enjoy! https://www.youtube.com/watch?v=YH0Q8J1NjIA

[smoky terrace]

https://discord.gg/stablediffusion?event=1100084952810405960 👀

[orchid halo]

https://youtube.com/shorts/q74qxAYhUHM?feature=share

[rapid estuary]

https://www.youtube.com/watch?v=D8lCQbyvwWE I made the animation for this song using Deforum, just thought I'd share it here

[stable hawk]

An attempt as some cool transition effects using AI.

[spare goblet]

This is neat!!

[stable hawk]

Used a new song made with Riffusion in this one 😅.

[fervent bloom]

A street in a modern city, a young man wearing headphones, with his head down, playing with a mobile phone, shuttles through the crowds

[broken token]

how do you do this kind of animation

[stable hawk]

I plan out the video, make the different sections with AI, use inpainting to make some changes (like I painted the starting image of the zoom section into the phone with inpainting) then I reverse the zoom and end on the image I started on. I then stitch the videos together 2 by 2, then I extract all the frames. Then I an audio file and the image folder to make the final video.

The audio I painted sort of 😅. I paint a Spectrogram out and watch the patters and etc. I use inpainting and outpainting to make it, then I run it through Riffusion to convert it to audio.

[limber zenith]

Hi Guys, we have released the new music video of the mythic band Magma (from 1969)
We work with a custom stable diffusion made by our Dev in Python.
It took like 5 month to have our definitive renders.
Full film here : https://youtu.be/UmgJkEajQUo
Let me know what you think 🙂

[whole fractal]

https://youtu.be/9ZXCGbtjzSU

[daring warren]

https://www.youtube.com/watch?v=xQ8t3VDQaIw

[daring warren]

https://twitter.com/NoFaceKolektiv/status/1651232110882037766

TheNoFace.ai🥷🏼 (@NoFaceKolektiv)

A little #deforum space walk 🫠🍄🍀

#aiartcommunity #stablediffusion

▶ Play video

Twitter · Apr 26, 2023, 2:29 PM

[ionic zenith]

Work in progress for a song I'm releasing on May 13... Experimenting a lot again 😊

It will come join the others at https://www.youtube.com/playlist?list=PL31dSpUZxYLVy6QPBavTDWtnzvY7q-gIX 😊

[livid cape]

https://www.instagram.com/p/Cmwo-jDhEC-/

machetewestern

🎵Molly Lewis - Oceanic Feeling🎵

Likes

3448

Instagram

https://www.instagram.com/p/CkboHElheYf/

machetewestern

Likes

3451

Instagram

https://www.instagram.com/p/CrWVuqMgSkz/

machetewestern

🎵Sister Owls -🎵 What song should I do next?

Likes

1145

Instagram

[daring warren]

https://www.youtube.com/shorts/tonudcdLaxo

[daring warren]

https://www.youtube.com/shorts/80iXfi35aOM

[dim zealot]

#https://drive.google.com/file/d/1wAkuq6heAyv9irXxnlGePLMT4rRWCQPD/view?usp=drivesdk

#🏞｜general-with-images anime

[stable hawk]

Music and Images made with Stable Diffusion and RIffusion.

[modest delta]

Generate

[daring warren]

https://www.instagram.com/p/CrlWzpmoOwK/

nofacekolektiv

What about some Psychedelic Cumbia?

🫠🍄🌌

🎶Music by the amazing, check him out! 🙌🏻

Instagram

[haughty isle]

Audio synchronisation using some features of the latest version of Parseq: https://www.youtube.com/watch?v=e7_krKpKnXI

[spare goblet]

Tried to get here but just can't figure it out. Any tips on how to do this?

[ionic sandal]

Made this video with the goal of having it be buttery smooth with minimal jitter/flickering while still getting a noticeable effect from AI

[stable hawk]

So close, I'm so close on coherence in animation and music lol. I painted this song in openoutpaint. I match up patterns and use inpainting to make adjustments. It's a weird process but it's fun and if you like patterns you can paint voices along the Spectrogram lol. I cut off the ends and then use swap sides and paint in the area between them to make a middle piece that will loop the whole track because the start and end were the start and end before right, but I cut the ends back off of the middle section so when it all gets stitched together I can just Daisy chain the track 😅.

I call it Nature's Symphony lol. I almost got her to sing the words lol.b

[stable hawk]

Omg, whoever that was before the message disappeared, thanks lol. I had forgotten about the contest 😅 . Whoops lol. Well, this all still helped lol.

[fringe carbon]

Using https://replicate.com/pollinations/stable-diffusion-dance but not quite there yet 😦

[leaden lintel]

Just some early tests - Still working on distinct prompts but its coming along

More refinement of prompts and scales - still testing

[hexed bridge]

Hola!

[cloud parcel]

very nice stuff im new to this and want make similar animations. good work

[ionic sandal]

Thanks, in my experience the lower the denoising and higher weight of controlnet the better the results, but it isn't gonna make them look like an anime girl or anything. I need to experiment more and try out multi controlnet, I've seen some people get some crazy good consistency. There was also a lot of post processing I did with flowframes, davinci resolve deflicker and after effects "force motion blur".

[fading wagon]

So cool! Any tips how to get started iwth this?

[leaden lintel]

Early testing and noodling around

Used some niche prompts and way better results

[haughty isle]

There are a couple of Parseq tutorials here but they don't touch on audio synchronisation very much: https://www.youtube.com/playlist?list=PLXbx1PHKHwIHsYFfb5lq2wS8g1FKz6aP8 . I'm working on the audio sync tutorial right now. 🙂

[earnest sigil]

https://t.me/NeuroZone_chennel/82

[cloud parcel]

wow thats very informative. Thank you for the tips I wish you the best!

[sour gulch]

Infinite zoom I did for a music video. Hopefully TikTok doesn't mute the audio again. https://www.tiktok.com/t/ZTRwD1NTP/

[mossy ridge]

How did you do this ? Looks awesome

[stable hawk]

I use an iterative process I guess. I start with prompt travel and using two seeds within the same latent bubble and set the variation strength to maximum. Then I figure out the number of frames I need, do some quick calculations, and then stack my prompts on top of each other in a way that will force the next generation to be close to logical at the least for movement. It's not perfect, numbers start to jumble when they start getting to big so I have to use my eyes to do the math which is less accurate lol. But if you determine how many frames you need, plan it out logically, you can make text walk! Then when you run it through interpolation you can smooth it. This one's interpolated, the one you replied to isn't.

[stable hawk]

The audio is slightly different. I can see the patterns in the Spectrograms pretty good so I paint them manually in Openoutpaint with inpainting and outpainting. Connecting and building patterns you can find instruments, voices, sound effects, like all kinds of stuff.

[mossy ridge]

Okay so you generate a photo first, generate a second one based on the first one and continue from there with more ?

[stable hawk]

Okay, so yes. First you generate your starting frame. Once you gotten the start you want, go to image to image, save the seed somewhere. Now, generate a few more images in image to image to kind of narrow down your main descriptors for the subject. Save the seed from the final render that you decide was perfect.

Now set the starting seed as the previous you saved, you variant, use the other seed for the second seed and set the variation strength all the way up

You've now created a sort of bubble in the latent space for the style and etc.

[stable hawk]

Next you want to use Prompt Travel and you want to use that main prompt for the subject you made and start stacking it on top of itself in the prompt input window.

[stable hawk]

Think about the number of frames, your fps, and travels between and etc. But add the actions after the descriptor, then the style settings. (Photorealistic)(Anime Style) that kind of stuff should be last so it doesn't alter your subject or scene.

[stable hawk]

I posted this on my page as an example of the prompt stacking one sec.

[stable hawk]

For Structuring the sections I broke it down into 4, one for each verse, did some calculations, refined the concept in each and theorized on how to approach each section of the video and connect them. How many frames did I need for a base minimum before interpolation to get what I wanted. Here is a glimpse at that process and prompts before refinement. This wasn't what I ended up using, but it was the initial base for my refinements. Here is the raw scratch paper if you will from one of the verse blocks I broke it up into... I'm just a part of everything
I stand on two legs and I learn to sing
It's not what were said
It's not what I heard
I walk with my dog and I whistle with a bird

Tree in a forest
Tree Becoming the silohette of a man
Silohette of a man dancing in a forest
Silohette of a man standing and basking in the sunlight
Silohette of a man standing in a forest holding a microphone
Silohette of a Man Standing in a forest singing
Silohette of Man running side by side with wolves
Silohette of Man becoming the Silohette of a Wolf
Silohette of a wolf running with real wolves
Silohette of a wolf become a man standing in a forest
Silohette of a Man singing and Looking around a forest
Silohette of a Man with his arms turning into wings + Flying in the Sky with Birds
Silohette of a man with a microphone Singing with Birds while flying in the air
Silohette of a man landing back on the ground

refrain

Stuff coming out, stuff going in
I'm just a part of everything
Stuff coming out, stuff going in
I'm just a part of everything

fps * seconds = frames

FPS = 15
Seconds = 60
Total Frames = 900

Cutting it slightly short on purpose - 14 Lines - 60 travel steps, successive, Linear, 15 FPS, Lerp.

[mossy ridge]

ah i get it now

so it's a stop motion from multiple images in the same bubble right ?

[stable hawk]

Yeah, I figured it would require the full explanation so I have a post ready lol

[stable hawk]

You only supply the one, the seeds and prompt force the rest.

The one you generated sets your first seed and int basically so you gotta make sure it's perfect to you.

The second one you generate is just for the seed and prompt refinement. You don't actually use the image.

[mossy ridge]

Okay good to know, ty 👍

[stable hawk]

My Facebook page has a breakdown of sine video tools I use and their links. I'm Geeky Ghost Designs.

[ionic zenith]

Well the release is officially on the 13th but I'm making it unlisted for a little bit just to share with some AI enthusiasts such as yourselves 😁 Whaddayathink? If there's any question regarding the process or whatever I'll be glad to answer them 😊
https://www.youtube.com/watch?v=EV7pifgfAWc

[crude solar]

Hi all,
Wav2Lip improvement with ControlNet 1.1
https://www.youtube.com/watch?v=-3WLUxz6XKM

[ionic zenith]

I had people on other channels ask me what it had to do with AI art so I just want to say that every single frame went through SD, either to fully generate or as vid2vid. Sometimes several times (generate +> mask -> insert -> vid2vid).

[daring warren]

https://www.youtube.com/shorts/1O2-h3UXV2Q

https://www.instagram.com/p/Cr0ylUMIhkI/

veiled.vision.studio

Starlight Riders 🐎🌌

Made on Deforum. This one wasn't easy but enjoyed it a lot 🖤

Instagram

[ripe plaza]

https://www.youtube.com/watch?v=UzDmXV7qAcs&ab_channel=StableSwirls

[misty sonnet]

I had a video, which i then shrank in size, then split it into individual frames, and am then feedin them frames into SD with control net, and getting some funky stuff in the output. but each frame is quite different from the last which leads to a jarring effect and you cant really tell whats going on, theres just too much information. Has anyone figured out how to take the previously generated frame as a partial input to the next, to create a smoother animation?

[scenic kiln]

https://youtu.be/4NTweEOYg5A

https://youtu.be/3UMmQg1r4wQ

https://youtu.be/CkQYP6PJGKU

https://youtu.be/AFjvPe5eVvU

[misty sonnet]

This seems like exactly the effect i am going after, whats your process?

[scenic kiln]

Which one?

So first one was simple img2img batch with very specific prompting. Firestarter was as well. Snoop Dogg video is with Deforum and Uberduck which allows Snoop's voice, and Clouded was Deforum with a poem someone I know wrote.

Those first two had interesting videos to start with, the flower one had someone on fire the whole time, so I replaced that with the asian theme

the 2nd vid as a woman dancing, with some flame graphics below, so I just used prompts and high CFG for that one in batch edit img2img

For the Deforum two, here is how I did that
https://aidreams.tech/blog/f/deforum-basics-with-stable-diffusion

[misty sonnet]

oh ok, that seems similar to what i've been doing, however yours seems to flow much better

[scenic kiln]

ok turn FPS down

mine is 10 to 12

also slow movement a little will help

also on deforum set cadence to 2 so you get 2 of each frame

turn denoise down some perhaps as well, high denoise might be allowing too much change and flicker

in fact I think one video I did I had the denoise as low as 0.3

um, for anime

wait wrong one

This one had low denoise

https://youtu.be/mDxraLgXOAk

[misty sonnet]

thank you, I will give it another go with those tips 🙂

[faint rivet]

It's magic, okay

Running at cadence 20, at 30fps... so, it's only rendering 1.5 actual frames per second.

This one is the same, but with a fixed seed.

[misty sonnet]

wow how did you do this...

[faint rivet]

I'm a dev on Deforum Stable Diffusion. I just made optical flow cadence fully functional, and fixed the 3D depth animation routines to work better. I'm also the creator of hybrid video in deforum. But, this isn't video. It's good old 3D animation in Deforum... just much better now

[thick dagger]

Nice animation. 👍

[fallen girder]

"Undercover" (ReV Animated v1.2.2 / Infinite Zoom) [4K]
https://youtu.be/K1zN_ZhmUeI

[earnest sigil]

https://t.me/NeuroZone_chennel/91

[narrow tiger]

does anyone know why i get weird lines and like just verticle jitter whenever i make zoom out animations

[misty sonnet]

How can one use optical flow generation, I can't see anything about it in the docs, Im looking to create somthing similar to your paper dancers

[faint rivet]

Optical flow generation is not recommended for video processing, lol.
The only documentation right now is... in the documentation (follow link from Hybrid Video tab)
Unfortunately, I've come out with so many features so fast that there are no new tutorials yet. I just have the one tutorial I made a few months back for use of normal hybrid video, and if you look at my documentation, there's a link to it. But, more coming soon.

[misty sonnet]

OK, I guess optical flow generation isn't the tool I'm looking for. How did you end go getting so much temporal consistency between frames in the paper people dancing clip? It seems like each frame has knowledge of the previously generated frame, but also I assume you used a video input as your animation style. The strength schedule does not appear to work when I use a video input

[faint rivet]

I developed Hybrid Video in Deforum specifically NOT for Video Input mode (I hate Video Input mode and never use it). Video Input mode in Deforum is roughly the same as batch img2img in auto1111, and has no connection between frames like regular deforum animation has. I made Hybrid Video to address this. It allows for compositing of video into normal 2D/3D animation mode. If you open the held in the hybrid video tab, it's literally the first thing it tells you.

Here's the settings file for that animation I did. It uses hybrid video compositing, as well as one controlnet layer using ip2p model (instruct pix2pix). For my ckpt model I used popupBook from civitai. Replace the video with your own video init, and make sure to uncheck resume if I have it on, and you should be able to do the same thing if you have controlnet with ip2p model and the popupBook model.

But, for the record, this one wasn't my best work. It was actually a quick test. I used a feature I made called Redo generation on this one. https://www.youtube.com/shorts/eHZokSuDnCc

[safe oriole]

very very clean! love it

[mental pecan]

https://youtu.be/BhooB8LS-us

[thick dagger]

I used extension Ebsynth Utility to make THIS https://youtu.be/-ML0J0MEV1w

[last schooner]

what's the best / most universally supported animated image format that doesn't look like garbage?

[hazy parrot]

how did you make that smooth?

Could I know the toold you used ? or the video you watched ?

if you don't mind tell me what you done cuz I want to make animations like this

I have done this with ai

by img2img

with a lot of edit to make the images look the same

[thick dagger]

I used stable diffusion with extension Ebsynth Utility + Ebsynth. The link of original video has been added in the description of the video.

[hazy parrot]

it's not working

the link in the description

[thick dagger]

I will send the google link to you just a moment.

[hazy parrot]

okay

but is there a video I can follow to make like this smooth?

it's really smooth and there is no like changing of clothes or hair or looks flicker

[bronze shard]

looks good

[thick dagger]

Each video can be remade like this (the best is solo dance). But you need a powerful graphic card.

[hazy parrot]

You mean every frame is has been edited ?

and it took a lot of times?

[thick dagger]

Edited and remake keyframe, then use Ebsynth to generate other frames.

[hazy parrot]

oh so it's very hard to get this smooth ?

like it's takes a lot of time

[thick dagger]

Pick up the correct keyframe is important, and you will get smooth video. This video is just a test, it can be made more smooth and without flick.

[hazy parrot]

does it take a lot of time ?

[thick dagger]

The speed is mainly determined by the performance of the graphics card and the number of keyframes.

[thick dagger]

2500 frames video, need at least 4-5 hours.

[hazy parrot]

I mean time by that I need to take a lot of time edit each frame

[thick dagger]

Use batch processing with img2img

[hazy parrot]

oh so it comes smooth without editing ?

[thick dagger]

Yes, the more keyframes, the smoother the animation.

[hazy parrot]

if you have another details tell me please

becuase I want to make anime animation using img2img

[thick dagger]

Please watch this tutorial https://youtu.be/Qph7A3UaVds. If you meet any other questions, @ me 👌

[hazy parrot]

thank you man

Can I really make anime animation high quality using img2img ?

or stable diffusion ?

like this and better ?

[thick dagger]

If there are only 1-2 characters in the video, it’s the best for animation.

[hazy parrot]

yes it's

[hazy parrot]

I sent to you a friend requset so we can talk about it

[thick dagger]

👌

[safe oriole]

coherence from ai getting up there very quickly

[dense holly]

an animated short film fully generated with Stable Diffusion. Not just prompts, but rather a carefully crafted workflow of animation, fine-tuning, noise and wacky characters. Enjoy! https://www.youtube.com/watch?v=antIs2qSg8o

[stable hawk]

Fun equiations, I was trying to make this drunken type of walk for the viewer throughout the video lol, the FPS is a bit high as this was a test, but it was a fun one... Translation X: 0:(50 * (sin(2 * 3.14 * t / (2 * 8)) * (t % 8 < 8 / 2) - sin(2 * 3.14 * (t - 8 / 2) / (2 * 8)) * (t % 8 >= 8 / 2)))
Rotation 3D Y: 0:(2 * sin(2 * 3.14 * t / 24))
Rotation 3D X: 0:(2 * sin(2 * 3.14 * t / 24))

[stable axle]

I have a 2070 super and used SD in the past with good success. If I wanted to make 30fps shorts (20-60s) how long do you think it would take? I estimated for a max length 60s video at 15s per frame generation it would take up to 7.5 hours. Seem right?

[stable hawk]

That's honestly hard to say and even harder when it comes to long form videos. The variables start compounding on each other. Prompts, Prompt weights, transition affects, like all that adds incrementally to the time to complete. Aspect ration is another significant factor. Are you upscaling? Using Restore faces? On a 2070, it seems like a good assumption since I just spent 6 hours on a 2 minute section because it has a significant amount of transitions and other variables. The other section only took 2 hours originally. After tweaks and etc lol. Like the final run took 2 hours. this is the final run on the second half and it took around 6 hours. I'm on a 3070 so similar vram.

[stable axle]

Ok good to know. Thank you! Anything you can show me that you’ve made?

[stable hawk]

Oh I've made a lot lol

Audio, Images, Video, Animation, all made with Stable Diffusion lol.

[daring warren]

https://www.instagram.com/p/Cr59CTmI48a/

veiled.vision.studio

..."And though her journey was fraught with danger and uncertainty, she never wavered in her resolve. For she knew that the future belonged to those who refused to be defeated."

Made on. Thanks tofor make me discover this banger! 🔥

Instagram

[stable axle]

Sick bro!

If I wanted to create stylized animations of myself doing some random activity using Cnet would it be beneficial to train myself in the model or will that not matter?

I wouldn’t think it would matter too much if I was just going for a stylized effect (anime for example). I do have a beard and that worries me as I’ve seen it struggle.

[oak urchin]

Amazing, is that using deforum ?

Just posted this, as early access, hope it will bring on some peeps to create Deforum Videos and Gifs.
https://civitai.com/models/59785/deforum-chatgpt-prompter

using gpt4 tho, will test with 3.5 tomorrow, seems dumb and broken with 3.5

[stable hawk]

Those are actually prompt travel 😅. Using seed variations to stay within a bubble in the latent space and using sequential tends to work really well if you craft your Prompts right. The zoom affect is infinite zoom, but I also use Deforum.

[oak urchin]

I'm just playing with deforum, as a dumb nobody, to understand how to set motion and things.
I would love to dig further to produce more realistic videos like yours.
They are amazing, no idea how to produce this yet. I'm far from it.
gg

[visual salmon]

i has to ask...who i can start to try create animations?

[ripe plaza]

What does redo generation do exactly? I'd like to give it a try. I guess it takes double the time?

[faint rivet]

it does extra generations before the final generation, sending the outputs back into the inputs. It's kind of like doing more steps, but a little different.

it randomizes the seed during redo and restores it coming out

[ripe plaza]

So it applies more style while maintaining the consistency? Correct me if I'm wrong

[faint rivet]

here's default settings, but with 3x redo (so 4 generations instead of 1)

If you're using controlnet, it has the side effect of kind of reinforcing the controlnet conditioning more, since it's regenerating with controlnet too

This is with Optical flow generation, which is actually like 1 redo before final generation. It uses the optical flow from the previous image to the redo generation, and warps it using that flow, before sending for final generation. On this one, I set the optical flow generation's flow factor at 4. You can see things moving around a bit.

mixin it up a bit at cadence 5 with some optical flow cadence with flow factor 4 along with an optical flow generation with flow factor 10

if I remove the optical flow generation and just leave optical flow cadence, it doesn't move around like it did

btw, the last one took me 22 seconds to render. lol

120 frames at cadence 5, so I actually only rendered 24 actual SD frames and the rest was optical flow cadence.

32 seconds to render 480 frames at cadence 20. Try it!

[ripe plaza]

Thanks a lot for the info

I'll give it a try

The video I made yesterday using hybrid video took me 2+ hours

But yeah cadence was 2

[humble dome]

Hello guys. I finally made my SD-CN-Animation project into web ui extension. It allows to generate coherent video2video and text2video animations. You can check it out here: https://github.com/volotat/SD-CN-Animation

[whole pelican]

nice, ill definitely try it tomorrow

[whole pelican]

I get Need to enable queue to use generators. and it's not super clear to me where it comes from, Are there any other requirements to install?

[humble dome]

Please, open an issue on github with a full log. I can't say the reason without more information, but I can say for sure this is not a requirements problem but something related to gradio.

[whole pelican]

I just did, though sadly the traceback doesn't include that much more

[faint harness]

https://youtu.be/9EhdTl2Y4PQ

[hazy parrot]

Made by me

[hazy parrot]

I don't like these tpyes of animations, look weried

[faint harness]

Thanks for your input friend

I love them

The ever evolving imagery is fascinating to me

[hazy parrot]

okay

but what you think about my animation ?

[faint harness]

Dope!

[hazy parrot]

sorry my english a little bad

what you mean by dope ?

oh you mean Excellent?

thanks :D

[faint harness]

yes

[spring basin]

does anyone have the link to that video made by the artificial intelligence of the forum, which the guy had more than millions of views on tiktok?

[stable hawk]

Playing with a walk animation with Prompt Travel and seed variation. This was a decent bubble, almost got the legs moving like I did for my superhero one lol.

[safe oriole]

anyway to prompt it hard to both feet flat on ground vs 1 foot raise in walk motion?

[stable hawk]

Oh yeah, I've actually got the walk cycle down a few ways, I just wanted to try a new way lol. If you scroll up I got a better walk cycle above. This one was an attempt to shorten my prompts and be lazy lol. Sometimes I'll catch a good bubble in latent space and it will handle most of it for me so I can focus more on style than mechanics, sometimes it takes a lot of work lol.

[stable hawk]

I could be super lazy to though lol. I have a ton of 3D models I've made and licensed so I've used those to do my animations before as well. Like a ballet one I did a year or so ago was 3D models.

[safe oriole]

guessing this is where openpose shines with your existing assets?

[stable hawk]

It can, I don't use open pose much. Runway introduced me to automatic alphas which negated the need for it mostly when using models. That works a bit better for existing videos. It was a big part of the ebsynth pipeline. I've just repurchased the idea for inpaintint lol.

Repurpossed

[oak urchin]

hey, updated the deforum prompter to be okay with 3.5, with some example posts to catch some eyes. Still just a draft to improve a lot, especially regarding a more creative content, and the provided example, but the answer begin to be structured. Would love any pro tips here. Early access only, tag me if needed 🙃
https://civitai.com/models/59785/deforum-chatgpt-prompter

[stable hawk]

Does it do text to equations?

[oak urchin]

Ho boy, lmao, trying to get a decent story would be a good start 😆
Joke aside, would love to implement a maximum of settings, in the limited token amount available.

I'm touching the intructions part rn, will be much shorter

[stable hawk]

My dyscalculia won't let me get much more complex than this 😅. Translation X: 0:(50 * (sin(2 * 3.14 * t / (2 * 8)) * (t % 8 < 8 / 2) - sin(2 * 3.14 * (t - 8 / 2) / (2 * 8)) * (t % 8 >= 8 / 2)))
Rotation 3D Y: 0:(2 * sin(2 * 3.14 * t / 24))
Rotation 3D X: 0:(2 * sin(2 * 3.14 * t / 24))

[oak urchin]

omg 😆

didn't set 3D yet

[stable hawk]

I have some patterns I want to do that incorporate fractals.

But I can't work out the equation lol.

[oak urchin]

Basically need to put settings in it, with an example. And then force it to be creative and set coherent and creative motion settings.
I need a very well written example, to exactly show gpt how to do it

Can make things like that, with a basic prompt and no motion. The motion need to be improved, hard to explain how to in 3 lines, especially to gpt 3.5 😆
Of course, should only be considered as a starting point to edit.

Can bring new users to dig further

[stable hawk]

Nice! I Dig it.

[stable hawk]

Nice

[oak urchin]

lol, can't get a response in a codeblock anymore xD Bed is calling me, will see later

🍻 .

repost* code block is back

Let's launch this unedited copy pasta from 3.5, using 540 frames, not 600, to see why this error.

Will post the result later, my 'lil rx570 need time

[stable hawk]

Drifting Into Orbit made with Stable Diffusion, Riffusion and Deforum. Wanted to test out some equations and patterns lol.

[oak urchin]

Result, i was expecting a less better motion. Interesting

[oak urchin]

Awesome

[fallen girder]

"Sakura" (Fking Scifi v2.1f / Infinite Zoom) [4K]
https://youtu.be/5rWchHORlRI

[mental tide]

can u share ur settings please

[oak urchin]

ofc (: This is for the 'Forest Traveler'

[mental tide]

ty

wow , never tried using 2d , it's too much faster than 3d

[oak urchin]

ahah, didn't try 3d yet

[mental tide]

3d with 600 frames 720*1280 takes about 2 hours lol

while this takes 8mins

[oak urchin]

8mins for 600 frames, omg. I only play with an AMD rx570, it takes hours for me, 512*512 👀

[mental tide]

was still using gtx 750ti not from a long time lol . hope u be able upgrade very soon<3

[oak urchin]

Ye need cuda on this

[mental tide]

yes

[oak urchin]

But 8 mins, c'mon 😂 🫠

I need to see some benchmark scores, to have an idea

[mental tide]

and this with a mid card 3060

imagine how much time will 4090 take lool

[oak urchin]

I'm 22 times slower than you, it's a ridiculous difference lol

[mental tide]

lmao

but wait how

rx 570 isn't that far

user benchmark gives me just 100% difference

[oak urchin]

Yeah, didn't dig in this topic, probably 'AI boost' and CUDA things

[mental tide]

so it's double preformance

[mental tide]

oh

[oak urchin]

Let's try another prompt for Deforum from chatgpt. The space traveler, 600 frames, euler a, 20 initial steps then 8

~2 hours

[mental tide]

i don't know how people make frames very connected

my results isn't good as i expect

[oak urchin]

Yeah, it's on my todo list too.
Saw some crazy ones, probably mixing scripts and things to craft those kind of results

[mental tide]

yeah i'll dig into it when i get free time

[oak urchin]

But should definitely start from a 'seed variation', i think this is the thing to look at

[mental tide]

u don't know which seed will give u a better result

it's luck i guess

[mental tide]

can i use this?

[oak urchin]

Well, you can use text2img to create your first pic, to init from.
Then you use the seed variation from the initial seed, and start the video from the pic

[oak urchin]

ofc, i share everything

[mental tide]

oh cool idea

i got some cool frames using 3d yesterday

[oak urchin]

Nice, but 3d in deforum stands for the motion, more than the pics, right ?
You can fake 3d using the prompt, adding things like 'sense of depth' or whatever

[oak urchin]

Just a though

[mental tide]

idk , but i get a very better quality images

[oak urchin]

I'll try. do you use easynegative ?

[mental tide]

here using 2d

same check point

[oak urchin]

Aww, lol okay

[mental tide]

the difference is really huge xd

[oak urchin]

Then I'll should try rn

I'll implement the choice in my prompter, nice

[mental tide]

i used also dpm ++2m

rev animated

[oak urchin]

Hum, my main testing subject rn is the motion created by chatgpt

Shit could be worse, I like the way he do it

Many settings need to be touch, for a better result. I'll add as much as possible

[marble heath]

Made a video, but it's more of a slideshow with a narrator.

[oak urchin]

lmao, the lord of data

great, pics are dope 👍

Yeah, get an error with 3D, probably another amd related issue. Can't generate, i remember now

[mental tide]

sad

[jolly spruce]

Saitama fighting versus goku

[mental tide]

cool

i want to see that

[jolly spruce]

I no understand what i use this for create things

[mental tide]

youtube has good toturials

i'll be back in 4 hours lol

[eager eagle]

kjlij

[jolly spruce]

Kkkkkkk rework this

Any knows some link i can use to generate images ia? Free?

[oak urchin]

You can uses spaces on huggingface

"Spaces" here on the list, find one that satisfy what you want, you can find many different models or setup

for example, the first 'space' in the list above : https://huggingface.co/spaces/stabilityai/stable-diffusion

finally a decent blurry preview

Let's don't look at the final version 😂

not that far, obviously not a manga model. Need some loras here. Switching colors and features from basic prompt

[stable hawk]

More fun with equations. I think I'm almost done with a settings converter lol. Basically adjusting the duration or more precisely adjusts the relevant bits of the equations used based on the desired duration and FPS.

[oak urchin]

Indeed, looks great. I'm jealous

[stable hawk]

Coming up with the settings and equations takes significant effort lol, you're way will likely yield far better results in the long run. Once I'm done experimenting I'll trying to do an education series with different things I've found with the results and the settings txt and etc. in case it can be used for training or etc.

[oak urchin]

I would love to see just some pertinent equation examples, to teach him

Teaching him the default zoom one

Settings for the response above

[oak urchin]

Neat, he can give adapted default equation, even if not asked. Let's see

[stable hawk]

Very nice

[oak urchin]

New form
Angle: 0:(0), 90:(-0.1 * t), 180:(-1 * t), 270:(0.1 * t), 360:(0)

[sly gate]

first job done, Lady Led day Galadriel, Frodo Barbellin and Swollum...

[whole pelican]

which reminds me, does anyone have something of similar quality to D-ID to use locally?

[stable hawk]

My son had a request today, so that's what we did lol.

[stable hawk]

You can just use Daz3D instead of worrying about things like that for now. They have anilips and a ton of other things that the D-ID stuff is based on (based on meaning it works similarly) and it's free. Generate a face with your AI image generator of choice. Use face transfer or one of the others to generate a 3D model, then animate it, you can use Nvidias free text to voice tools if you have an RTX or you can buy the ones in Daz or numerous other places. The D-ID stuff is essentially a limited but simplified version of existing tech slightly augmented by AI.

[whole pelican]

oh nice, I'll try it

though the sick thing about D-ID is that it works on pretty much everything first try, and you are done in like a minute, and this sounds potentially a lot more time consuming

but I'll experiment later today for sure

[stable hawk]

It's definitely more time consuming lol, but the offline software for the things like D-ID and etc are few and far between for a reason 😅. That's where people think the money is, but it's a mistake lol. Voxels will shine soon an make this all moot eventually lol.

Ooh! Metaballs... theyd make perfect canvases since they can merge like nonnewtonian fluids!

[oak urchin]

Nice, just installed seed travel extension, first test. How do you achieve that, for example ?! 🤔

[stable hawk]

Inside the Mind - Made with Stable Diffusion and many extensions lol.

[oak urchin]

Dang

I don't even get how to link seed travel and deforum. Every test take a life, meh

[oak urchin]

this is amazing

[stable hawk]

Thanks! There's a post floating around on facebook about what it's like inside your mind. Figured, I can actually show people now lol.

[stable hawk]

for seed travel, you can schedule seeds similarly to how you can schedule other settings. Try this for see scheduling? It's the subseed and I haven't gotten around to experimenting with it yet, this is my first experiment settings if you want to give them a go.... {
"Enable Subseed scheduling": "True",
"Subseed schedule": "0:1,20:22,40:333,60:4444,80:55555,100:666666",
"Subseed strength schedule": "0:0",
"Resize seed from width": "0",
"Resize seed from height": "0"
}

0: <--- Frame start number 0:1 <--- Seed

You MIGHT be able to schedule seeds lol. I haven't tested it yet, so saying it as an absolute would be facetious lol.

[waxen basin]

Dragon Ball deforum video

[oak urchin]

Awww okay, this is the way to seed travel in deforum, i was trying in txt2img with the extension

[stable hawk]

Oh for the extension, seed travesty weird, the prompt travel one is better for cohesion. Set it to embryo for simple animations,and check variation enter your two seeds and set vatiation to max.

[stable hawk]

See travel works similarly for better for different things than animation.

Seed travel and but better for different things.

[severe cradle]

looks promising
almost done rendering

how tf

take 2

im ok with this

Upgrades

[serene veldt]

https://www.youtube.com/watch?v=7BV7yLDQumI

[safe oriole]

holding the faith strong!

[mental pecan]

https://www.instagram.com/p/CsAZCR-OwV6/

dreamingcomputers

😍AI Reimagined Beautiful Fashion model

Experience the elegance reimagined through AI! 💎👸 We’ve used cutting-edge AI technology to create a stunning transformation of a beautiful fashion model.

Don’t forget to like, comment, and follow for more innovative AI transformations!

Instagram

[haughty isle]

Hi all, I've just published a tutorial on doing audio sync with Deforum & Parseq: https://www.youtube.com/watch?v=M6Z-kD2HnDQ

[faint rivet]

https://www.youtube.com/watch?v=o42TMIhqb4c

[wet dew]

animations

[dull parrot]

Dragon ball defution# animation

#1072220168534642768

[random quartz]

#1072220168534642768

[inner crest]

It is really hard to have consistency, no spaghetti lines when moving the camera and prompt control...

What kind of formulas do you use for fov, Tz, strength, etc... when you audio sync your animation ?

[lethal ridge]

I went wild

[inner crest]

raaah sooo hard

[oak urchin]

from chatgpt, cadence 3, added the default music and checked the box "Make gif", nothing else

[oak urchin]

Repost
Extended version, also from chatgpt-4, to 2280 frames (x4), for deforum (Need 10-12 Hours to run this with my card...):
768*512, cadence 3, 50 initial steps, then 13, 0.75 strenght, Euler A, from 474680868
Should be nice

[proper plume]

No idea what you guys are saying but did you just says 10-12 hours? To what? Make a video?

[oak urchin]

Yeah, my rx570 is a complete joke..😭

[proper plume]

How long would it take on my 4090?

Ive just been doing simple stuff on SD like txt2video.

[oak urchin]

You can install deforum extension from the extension tab. Click Available, load, and install 'deforum'
Then, below the preview in the deforum tab, once restarted the UI, you paste the path to the file i shared and click load all settings.
Then generate

Pretty simple

[proper plume]

What's deforum do? Same as txt2video?

Im still a bit new... just been doing txt2video only at the moment.

[oak urchin]

It's img2img, it does create the video above, from the settings.
Going from one frame to another, from one prompt to another, from one motion to another.
Can do way more

[proper plume]

I wanted to ask, is it not possible to create videos in like 1280x720? Im creating videos in 256x256 with black bars on the sides. The good part is how fast it processes like 20 videos in a batch.

[oak urchin]

Sure you can, more if you card allows it. Mine juste yield errors if I goes over 900

[proper plume]

Exactly.

It gives errors

I think the model is only designed for 256

[oak urchin]

Yeah, it need to fit into your VRAM

[proper plume]

Even when I did manage something else it changes the whole video and doesnt produce good results for some reason.

So I guess even a 4090 isnt good enough for higher resolutions and quality. AI video must need a lot of power

[oak urchin]

The trick will be to use --lowram or this kind of setting, in your user-webui.bat

Depends your card

My arguments line is 40 feet long

[proper plume]

24gb ram on this 4090

[oak urchin]

Aww man

[proper plume]

There's higher than that?

I don't know much about it

[oak urchin]

It should be able to make coffe doing 8K

Yeah, way more higher, but way more expensive.

Google nvidia A100 80gb lmao

[proper plume]

4090 with more vram?

I thought all 4090 is the same

[oak urchin]

No, another ones

anyway, the model isn't made to produce 2048*2048 stuff.
SD 2.1 is trained with 768x768 content.
You need to upscale the low res generated ones

1280*720 is quite honest to begin with, once upscaled x2 or x4

[proper plume]

Okay but anyway, for SD txt2video, I can pretty much just get 256x256 right? I mean it processes quicker so I can imagine higher resolutions would take way too long for like 20 videos.

[oak urchin]

256 is ridiculously low, 512 or 768 min

[proper plume]

Thats the default

[oak urchin]

512 for SD 1.5, 768 for SD 2.1 yeah

Your card can do my 12h work in, like ... 30 min maybe

[proper plume]

30 min for how many seconds lol

I hate waiting

[oak urchin]

2280 at 15fps

a lot

152 secs

[proper plume]

What does txt2video use? SD model or something else?

[oak urchin]

with good motions ^^

[proper plume]

SD15NewVAEpruned.ckpt this?

[oak urchin]

same for all. Except img2img using inpaint/outpaint models

and some extensions needs models too

[proper plume]

Yeah I only use txt2video

I get errors when trying 768 x 768

Exception occurred: CUDA out of memory. Tried to allocate 18.98 GiB (GPU 0; 23.99 GiB total capacity; 3.98 GiB already allocated; 16.83 GiB free; 4.54 GiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation.  See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF```

Usually always happens when I change dimensions

[oak urchin]

This is a model, not sure if a good one. also use safetensors files. ckpt can be malicious

[proper plume]

512 is okay though

Slower to process 24 frames though

SD15 is SD 1.5?

So Im not using the latest?

[oak urchin]

18.98 is a lot lmao for a small pic.
Unfortunatly, my setup with amd is even weirder, can't help much.

what's the size of SD15NewVAEpruned.ckpt

[proper plume]

Sorry Im still new to this lol

[oak urchin]

Should be between 4-7gb

[proper plume]

2gb just

512 looks a lot better already

lol

Might stick with that

And I been making so many videos in 256 already.. damn

[oak urchin]

Anyway, the model need to be trained. The factory ones are really bad.
I use this one, most of the time.
https://huggingface.co/Bulldeawh/merged/blob/main/allinone.safetensors
You can find models on huggingface or civitai dot com

[oak urchin]

Someone else with this card may help you. Your installation may be broken, or you're using bad arguments in your user-webui.bat

[proper plume]

Nah nothing is broken. Im just processing videos in shit resolution thats all

Might try upgrade the model and see what happens

But I doubt I'll achieve 1280x720 videos.. even if I did, it would take too long for my liking

[oak urchin]

Nah it's totally impossible. Update your driver or install from scratch, if you didn't yet.

This is so easy for your card, you can almost play in the same time

The webui seems to be launched on the good device, but you can force it

[oak urchin]

At this point, you're probably doing like 1 iteration/sec with a very low res.
Your card can achieve like 15-20 iterations/sec at high res

Don't stay this way, sad

[proper plume]

The process is really fast on 256

It does 1 video in about 10 seconds

I wouldnt mind talking to someone who knows about all this.

I wanna know if I should be updating my model or whatever

I dont even know what version of SD model Im on

[whole pelican]

it literally says so on the top left

[proper plume]

I thought it updates every time its restarted anyway via the cmd?

This?

[whole pelican]

I dont know what you use to start it, it's possible it upgrades your webui, but I doubt it changes the model

[proper plume]

What is it using to make my videos? I'm lost. Can't be the comicBabes one because the videos are realistic.

[whole pelican]

seems like it is

[proper plume]

Some launcher

[whole pelican]

anyway, all 1.5 models (which this probably is) are probably not really different in speed, it's more likely old GPU drivers, or possibly other stuff in your setup not being updated

[proper plume]

Yeah Im not worried about speeds.

Im just wondering if Im missing out on anything in terms of the latest model or whatever

Man Im kinda pissed off this whole time I been using an old version of SD model 😐

Im wondering if it makes txt2video look better when I use the new one.

[whole pelican]

there's not really 'the latest model' but yes there's new better models to pick from all the time

[proper plume]

This hurts my head lol

[whole pelican]

and if you are doing realistic stuff with a model called comicBabes, I suspect you can get better results with a new popular one

[proper plume]

I cannot believe Im using a model called comicBabes? I dont get it

[whole pelican]

but I wouldn't recommend the base models and tbf wouldnt recommend a 2.1 model

[proper plume]

The images I got from comicBabes were anime

txt2video is literally all realistic stuff

[proper plume]

What do you recommend then?

[whole pelican]

https://civitai.com/, sort by most downloaded and pick one near the top that fits

maybe 'cyberrealistic'

[proper plume]

All this time I could have been doing videos in better quality.. damn

Isnt THIS the model Im using?

[oak urchin]

Show the tabs above that, you should be in the first tab txt2img

[proper plume]

I dont use txt2img though

[oak urchin]

If you're using an extension tab, is it the same in the proper txt2img tab ?

[proper plume]

I dunno what this is.. apparently its the model but the other screenshot literally says Model type

Honestly, I think THIS is for img only

It doesnt make sense if thats my model for videos.

Otherwise my videos would look anime.

I think for txt2video Im using something called ModelScope

Does anyone here use txt2video?

😆

[oak urchin]

I don't know what txt2video is, this tab may be your problem here

[proper plume]

Yeah Im sure thats what it is then

[oak urchin]

What can you do in the txt2img tab, this is the question

[proper plume]

comicBabes is the model for images because I tried it a few times.

I can make images.

Ive done it before

Looks like anime.

Because of comicBabes model.

[whole pelican]

deforum does use whatever is in the top left for your model, too not just txt2img

[oak urchin]

What max res before an error ?

If high, then you have no problem, uninstall this text2video something tab, it's just broken or using very bad/high settings

[oak urchin]

Huh, i can't get chatgpt to eat my new version as well

gpt4 is fine, producing way better prompts

Not bad, need some regeneration, but meh
Deforum ChatGPT Prompter (gpt4 is better) :

I did steal parts from the other prompter on civitai lol, to build better scenes. It works well. Just need a better/perfect example, or mine overwrite the previous instructions.

[inner crest]

experiment made with Parseq

[oak urchin]

Music synchro in great to watch

Okay, 3650 tokens for the prompter, not bad

[oak urchin]

Universal 2D Motion Settings for a ~2280 frames deforum video @ 15fps, 4-16 scenes in the prompt :

zoom: 0:(1.00), 135:(1.025), 270:(1.00), 450:(1.015), 540:(1.00), 675:(1.025), 810:(1.00), 945:(1.015), 1080:(1.00), 1215:(1.025), 1350:(1.00), 1485:(1.015), 1620:(1.00), 1755:(1.025), 1890:(1.00), 2025:(1.015), 2160:(1.00) angle: 0:(0), 90:(0.5), 135:(0), 180:(0), 225:(-0.5), 270:(0), 315:(0.5), 360:(0), 405:(0), 450:(0.5), 500:(0), 550:(-0.5), 600:(0), 650:(0), 700:(-0.5), 750:(0), 800:(0.5), 850:(0), 900:(0), 950:(0.5), 1000:(0), 1050:(-0.5), 1100:(0), 1150:(0), 1200:(-0.5), 1250:(0), 1300:(0.5), 1350:(0), 1400:(0), 1450:(0.5), 1500:(0), 1550:(-0.5), 1600:(0), 1650:(0), 1700:(-0.5), 1750:(0), 1800:(0.5), 1850:(0), 1900:(0), 1950:(0.5), 2000:(0), 2050:(-0.5), 2100:(0), 2150:(0), 2200:(-0.5), 2250:(0) transformX: 0:(0.5), 135:(0.2), 270:(0.5), 405:(0.8), 500:(0), 540:(0.2), 675:(0.5), 810:(0.8), 945:(0), 1080:(0.2), 1215:(0.5), 1250:(0.8), 1385:(0), 1520:(0.2), 1655:(0.5), 1790:(0.8), 1925:(0), 2060:(0.2), 2195:(0.5) transformY: 0:(0.5), 135:(0.8), 270:(0.5), 405:(0.2), 500:(0), 540:(0.8), 675:(0.5), 810:(0.2), 945:(0), 1080:(0.8), 1215:(0.5), 1250:(0.2), 1385:(0), 1520:(0.8), 1655:(0.5), 1790:(0.2), 1925:(0), 2060:(0.8), 2195:(0.5)

TranslationX: 0:(0), 45:(-2), 90:(-3), 135:(0), 180:(1), 225:(2), 270:(0), 315:(-3), 360:(0), 405:(-2), 450:(-3), 500:(0), 550:(2), 600:(0), 650:(1), 700:(2), 750:(0), 800:(2), 850:(0), 900:(-2), 950:(-3), 1000:(0), 1050:(2), 1100:(0), 1150:(2), 1200:(3), 1250:(0), 1300:(-2), 1350:(0), 1400:(-1), 1450:(-2), 1500:(0), 1550:(2), 1600:(0), 1650:(2), 1700:(3), 1750:(0), 1800:(-2), 1850:(0), 1900:(-2), 1950:(-3), 2000:(0), 2050:(2), 2100:(0), 2150:(1), 2200:(2), 2250:(0) TranslationY: 0:(0), 135:(-1), 270:(0), 405:(1), 500:(0), 540:(-1), 675:(0), 810:(1), 945:(0), 1080:(-1), 1215:(0), 1250:(1), 1385:(0), 1520:(-1), 1655:(0), 1790:(1), 1925:(0), 2060:(-1), 2195:(0)

An horizontal camera translation, from left,pause, to right, pause, to left, etc...
A vertical camera translation, up, pause, down, pause, up, etc...
Angle starting and ending with 'TranslationX', like a plane going left and right.
Zoom starting and ending with 'TranslationX', like the same plane accelerating and decelerating.
Like exploring the pictures, everywhere. All synchronized.
Need to test, be should be good. Good base to edit.

[oak urchin]

Settings shared on the civitai page too 👍 First version now public, hope some will post some gifs of their videos. v1.3 in early access
https://civitai.com/models/59785/deforum-chatgpt-prompter

8h33, 86% for a 2280 frames at a diffusion cadence of 3, 152sec vid 😭

[safe oriole]

8479 frames by the looks...

i barely run 2100 frames for 3 min videos

cadence and interpolation go a long way

[oak urchin]

10K steps, not frames. It's deforum. Running with a sh*tty rx570, soooo slow

I'll post the vid in 1 hour

Bad luck with camera moves looking at the pics, but not bad. It was using the last version of the motion settings, pretty incoherent.
I'll run a new one right after, have a lot of similar prompts, on different themes.

[oak urchin]

The ultimate deforum prompt lol
"The Parallel Worlds", 8040 frames, 67 scenes, 120 frames per scene. No positive/negative prompt

I would need more than 2 days to run this, 768*512 at cadence 3

[safe oriole]

your it/s seems good thou

oh wait 😄

im used to seeing it/s not s/it

ok yea thats really slow :\

my 3060ti is like 1it/s - 1.5it/s for video generation

those 4090 , oh boy, would be killer 😄

i think the 8000 frames would take 4 hours

be 12-18 hours for you

[fallen girder]

"Symbolic" (Counterfeit v3.0 / Infinite Zoom) [4K]
https://youtu.be/IV9lSGC9w18

[trim moth]

I may need little bit of help with the correct set-up with Deforum, for using interpolation.

What am I trying to do? I want to take an original image, and change/morph it gradually into something else (say a dog gets changed into a cat in span of 3 seconds). The way I'm doing it currently is that I use an Init Image as the base (in the example, the dog) and set the strength of it to somewhere around 0.7-0.8. Then setting specific prompts for specific frames does the job.

This does work, however I have two problems with it:

A) The new image resambles the original little too much, since the Init Image Strength is fairly high (though going under 0.5/0.4 causes it to be basically ignored), meaning there can't be big differences being made to the original while keeping it somewhat reminiscent of the initial image. What could work here would be the Init Image Strength being decreased as time goes, though I haven't seen an option for that anywhere. Also the "Strength" under "Keyframes" doesn't seem to do nearly anything with the current set-up (it should be an equivalent of the Denoising Strength, but I had no luck with it).

B) The changes don't happen gradually over the frames I set, but rather the most radical changes happen immediatelly and for most of the frames the image doesn't change all that much. How do I solve this?
Am I even doing this the right way, with all the Init Image Strength?

[earnest tartan]

wow how do you make this?

Hello everyone! did anyone experiment with gif2gif? if so have you figured out how to have the same character to appear in every frame? in terms of looks and outfit, like the only thing changing is the animation between frames, thank you!

[oak urchin]

😂 🥲 .

Here is the result, "Forest Travel", 2min30. Partially failed motions. No luck with the camera going into obstacles.

Next one will use the motion settings posted earlier, should be way better.

[daring warren]

https://youtu.be/MECihjQfEyc

[oak urchin]

Blacky blacky, but awesome. Nice music sync

Seems to be hard to sync when guitars goes mad xD

Now running the 'Space Travel'. Same vibes, corrected motion settings, 2280 frames, cadence 3, strenght back to 0.65, 768*512

[misty venture]

How did you made it man?

[oak urchin]

To install Deforum in Stable Diffusion webUI, you can find it here :
https://github.com/deforum-art/deforum-for-automatic1111-webui
or:
Click on the 'Extension' tab, in your webui.
Click on the 'Available' tab, and click on the 'Load from:' button.
Write 'deforum' in the 'Search' form, and click on the 'Install' button.
Click on the 'Installed' tab, and click on 'Apply and restart UI'
Tada ! You can see the Deforum tab now (: 🍻

.
Then, go to the 'Deforum' tab and, below the preview, just set the path to the settings file shared below the video, and click load all settings.

I'm preparing a 10200 frames file. 100 lines of prompts and motions. This is ridiculously long, lmao !

11min+ video @ 15fps

[ripe plaza]

Deforum Hybrid Video + CN + Masking.
https://youtube.com/shorts/cVSIJpe3NrQ?feature=share
https://www.instagram.com/reel/CsEPfxzgdpT/?utm_source=ig_web_copy_link&igshid=MzRlODBiNWFlZA==

[livid wasp]

https://youtube.com/shorts/49gkEk_XRgs?feature=share

[misty harness]

https://www.youtube.com/watch?v=BHt2RXRKCSE

[oak urchin]

Here is the file for the raw extended version of 'The Parallel Worlds'. (not Space Travel, my bad)
10200 frames, 67 scenes, cadence 3, 768*512, Euler A
50 initial steps then 18 per frame, test motion settings extended to 10200 frames long.

😆 .

Need to adjust the speed, to be universal and go forward enough.
And replace the zoom by an equation, for more immersion.

@stable hawk Can I ask you something Sir about that !? (:

[stable hawk]

Sure, what's up?

[oak urchin]

For, for example :
Zoom: 0:(1.00), 60:(1.0025+0.001*sin(1.25*3.14*t/30)), 120:(1.00), 180:(0.998+0.002*sin(1.25*3.14*t/30))), 225:(1.00), 405:(1.00)

How do you adapt the t/30 part

[stable hawk]

t represents the current frame, and t/30 scales the value of t so that it completes a full sine wave every 30 frames. To adjust this value, simply replace the 30 with the desired number of frames for a full sine wave. For example, if you want the sine wave to complete in 60 frames, you would use t/60.

[oak urchin]

Okayy, wanted to be sure before testing with another life.. lol
So, here I would need
Zoom: 0:(1.00), 60:(1.0025+0.001*sin(1.25*3.14*t/120)), 120:(1.00), 180:(0.998+0.002*sin(1.25*3.14*t/105))), 225:(1.00), 405:(1.00)

Thanks ! ((:

Flat zoom is 'lil too flat

[stable hawk]

Yeah, it's hard getting it fluid. I'm working on a heartbeat at the moment. So that it pulses every so often through the animation.

[oak urchin]

ChatGPT talking to me about a 'breathing' movement, ealier. Like the pov in video games

[stable hawk]

Nice, yeah the breathing equations were fun. I'm collecting patterns so I can use them like a dependency library lol.

I want a run cycle, grab this pattern, want a combat scene, grab these lol.

[stable hawk]

Do you mind if I use this and play with it?

[oak urchin]

Take everything buddy !
Here is my last prompter too. Better with gpt4, for now

[stable hawk]

The results lol.

Here is the block I was working on. "animation_mode": "2D",
"max_frames": 120,
"zoom": "0:(1.0)",
"translation_x": "0:(0.001 * t)",
"translation_y": "0:(amplitude * sin(frequency * t))",
"translation_z": "0:(0)",
"transform_center_x": "0:(0.5)",
"transform_center_y":

I just replaced the zoom with yours.

Translation_y I plugged in for the short video I was making. translation_y: 0:(5 * sin(0.05 * t)) <--- substitutions made for this specific video.

[oak urchin]

Hum nice 'lil oscillation

[stable hawk]

It's pretty smooth for a low framerate. I dig it.

[oak urchin]

Where the right and left 'shaking' comes from here. Is it on purpose ? or just my eyes

[stable hawk]

Yes, it's on purpose. These two influence it.

[oak urchin]

Okayy, was trying to visualize it, but i'm stuck. I got it anyway 👍 🍻

[lyric niche]

Kinda of bad guidance video because of the occlusion, but the new animation pipeline I made is showing pretty guide temporal coherence. Some popping, but pretty sure that is from openpose getting confused.

[oak urchin]

Dang, amazing ! Sorcery

[lyric niche]

Running another example, same style different guidance video

[oak urchin]

10 200 frames video incoming lol, at 15fps, a discord friend is running it. Using a RTX3090, wayyy better image quality than me

Wait for it, 57%

[lyric niche]

ugh openpose issues. We need a temporal consist version of openpose that works on videos

[safe oriole]

looks very clean, does adjusting fov push it closer to objects due to lowering depth? (i dunno, asking)

[oak urchin]

All settings are stock, except res, cadence
I'm focusing on the motion

I'm preparing a good version, moving for every 120 frames scene, it should be awesome. I'll post them when ready

[stable hawk]

This is why I went RunwayMLs route. The auto alpha was a brilliant concept.

[stable hawk]

I use a ton of controlnet models, but for subject replacement, the automatic alphas are just better.

[lyric niche]

What is auto alpha?

[stable hawk]

You apply it to the subject in the first frame and it auto tracks through the rest of them and covers up the subject with an alpha mask so you can use image to image, inpainting, and etc.

[lyric niche]

Ah gotcha

[stable hawk]

There's greenscreen and alphas. It was amazing in the early days.

https://youtu.be/oKzAuJRPaIY

[lyric niche]

Yeah, guess "Track Anything" is the opensource version of that

[stable hawk]

Pretty much, I made my own 🤣😅.

[lyric niche]

For anyone not familiar: https://github.com/gaomingqi/Track-Anything

[oak urchin]

This is so lol

But awesome

[inner crest]

I use Parseq

[inner crest]

@waxen basin who post a tutorial on Parseq...
https://www.youtube.com/watch?v=n4zj1lrbIEM

I say it is a good thing. The people needs to unleash their power

gosh he even audio sync the animation...

Hummm....

https://tenor.com/view/so-cool-stephen-sharer-awesome-amazing-gif-20635844

[inner crest]

Another try from me with Parseq

the girl seems almost alive and moving on her own...

damn, I was close to create a living creature !

[oak urchin]

The Parallel Worlds

[safe oriole]

i recommend (masterpiece:1.2),(best quality:1.2)

try out these in negative prompt image tests, see if it makes a difference for you? (worst quality:2, bad quality:2),(low quality:2)

[stable hawk]

Nice work, that came out really good.

[oak urchin]

The model used here is very darkish, lyriel

[stable hawk]

@oak urchin Decided to work on something weird. I'm making an app with python using Tkinter to create a flight path inside of for canvas motion in Deforum. Basically the idea is to create the motion inside the app with controls, then convert that to a json, well the Deforum txt file with the settings and etc. I may, I may make it work with a flight stick. I feel like a flight stick would be best for controlling a theoretically camera that floats and flies lol.

Throttle for speed, flight stick for pitch and angle, zoom in and out and etc.

[oak urchin]

That's an interesting approach

[stable hawk]

It seemed to make more sense than just trying to do all the math in my head. Why not just treat it like an airplane and create flight paths instead lol.

If that works out... I wonder if I can add more "cameras"

[oak urchin]

You would need to see or predict the picture, to know where to go, if not using any trick to force it

I'm not satisfied with my moves, i'll rollback to a simplier version

[stable hawk]

I can't, my prediction models and understanding on how to use them is limited lol. I'm no mathematician lol. I think the AI could excel at predictions, it has to have many baked to learn the way it does.

like the way it operates, it IS a statistical prediction model of sorts, but like the most advanced prediction model ever made with more variables than you could think of covered automatically lol.

[oak urchin]

can't wait to see that

[devout zenith]

did anyone else notice that the modelscope VAE encoder is exactly the same as the SD encoder 🤔

[oak urchin]

The "Sanctuary of Mythical Animals", 2800 frames, 3min, anti-blur at 1

Testing a different motion pattern.
Stop going down, it's weird.
Sync with prompt's scenes.
It will almost stop, to give time to SD to create the full scene without going beside, and for the viewer to 'visually know' when a new scene is here.
Anti-blur is awesome, the default value sux if using any zoom, should be adjusted with zoom usage.

[stable hawk]

Nice, That... is going to take me some time to evaluate lol. Those are some impressive equations.

[safe oriole]

nice motion and the images were very clear

[regal aurora]

https://youtu.be/S7APlrc4q4w

[limber flume]

Hi, I am a newbie ☺️ Don't even have a PC but want to create videos with the stable diffusion. The original video was already created with the stable diffusion, I've just modified it on my smartphone 😌 Hope the cover looks better than the original 🤭

https://youtube.com/shorts/NzBOdGi8p-Y?feature=share

[safe oriole]

thanks for the anti blur tip, trying it out , had been running 0.1 or 0.05

[oak urchin]

https://tenor.com/view/seth-meyers-right-am-i-right-totally-same-gif-9191853

Just discovered the anti-blur param
I'm using a lot of arguments to run SD, I figured out yesterday that the quality is really terrible with an small amd, can't fight this.
But this anti-blur param is underrated, should be a basic param on the first tab, or should move automatically when using a zoom.

Is the deforum dev here ? curious

[oak urchin]

Saw this, my mind blew up
https://www.youtube.com/watch?v=_lzIKlfY-88

How no one mention this param

[oak urchin]

Also, I'm still not an experimented peep here, but this model (discovered only yesterday, need to test) entered directly my top 3
https://civitai.com/models/22922/lyriel

Give it a try, for anything.

[safe oriole]

yea ive run some lyriel 14 i think, its pretty great

[oak urchin]

I have a lot to learn

While you're here, do you have a favorite model ?

[safe oriole]

was enjoying cyber realistic thou it spits out nudity too often even with nsfw prompted and sfw prompted on negative /positive

[oak urchin]

I'm mostly using a random one, let me find it. It's my favorite, general use, not dumb, kind of universal

[safe oriole]

was using deliberate 20 for weeks

[oak urchin]

https://huggingface.co/Bulldeawh/merged/tree/main
This one : allinone.safetensors

nb: i don't like hyper realistic models, meh. Artefact and malformation are way too obvious, or the picture lacking sense

[oak urchin]

this one is merged in the link above

[safe oriole]

oh yup, yea, if doing fantasy videos then hyper realistic a no go for sure , i lean more on sci fi which requires more exact detail

[oak urchin]

I wonder if there is a model that i don't know, and better than others

And why no one put a a freking *chatbot beside or in the model.

[safe oriole]

endless models out there for endless things 🙂 an ocean, can curate your own too, editing models got easier with this automatic1111 addon https://github.com/arenasys/stable-diffusion-webui-model-toolkit

thou i just browse top downloaded on civ and try them out 🙂

[oak urchin]

Okay, i just remember, people are not kind enough. People don't sahre enough. Forget my question lol

[oak urchin]

I did, still wondering why so few out there

Most are just the same trained on differents kind of boobs.

You won't help the community with fine-tuned models, that's not the open-source spirit

[safe oriole]

i just like exploring the coherent space that is offered

[scenic kiln]

One of my piano students became a music producer and pro singer so I made her a video with her song
https://youtu.be/nVhV-hbdR60

Snoop Dog wants to talk to you, well... AI Snoop voice
https://youtube.com/shorts/CkQYP6PJGKU?feature=share

Poem animation, this one came out great
https://youtu.be/AFjvPe5eVvU

[oak urchin]

sryyy bad phrasing !! you wasn't you personally, but 'they' in general

sry again

[safe oriole]

all good 🙂 enjoying your learning journey, might try your long prompt out with some tweaks and see how it turns out in the weekend

testing out different blur

[oak urchin]

🍻 😘

[oak urchin]

my 1 hour research ( 😆 ) conclusion, if zoom not null, then 0.5
If zoom > ~ 1.01 , then between 0.5 and 1

[safe oriole]

my Z is 0:(0.3)

so 0.5 ill try 🙂 danke

[inner crest]

Final try made with Parseq

[safe oriole]

nice! question, if you used init image on each bounce is there a way to diffuse the background a lot so keeping consistent character but diffused background?

[inner crest]

Maybe in using controlnet

[safe oriole]

dunno how to use mask for controlnet (thought its disabled?)

[inner crest]

I would remove the background. Make a video of the character and enable ControlNet

You dont necessary need a mask

[safe oriole]

oh right! yea thatd do it

[inner crest]

If your controlnet model and parameter are well adjusted

Above all if the character move

In this video, the zoom in and out increases. And the character stays the same.

I only use 1 ControlNet Model: hed. No mask.

It was my first video with ControlNet + Deforum. So the bounce is not really heavy.

[safe oriole]

think i seen your video on yt 😄

[inner crest]

Youtube ?

[safe oriole]

yea, youtube

[inner crest]

Yeah I didnt describe how I made it

But Im sure with faster camera movements. The background will change but not the character

You can also use Parseq and put some weight to change the background

[leaden turtle]

Played around with temporal kit

[safe oriole]

what is this magic

[scenic kiln]

https://youtu.be/jZOsxKSW3Nk

[crude solar]

Hi all,

A simple Wav2Lip UHQ Improvement Script with ControlNet 1.1
https://github.com/numz/wav2lip_uhq

result here :
https://www.youtube.com/watch?v=-3WLUxz6XKM

[fallen girder]

Psychedelic Animation (IllusTrip3D) [4K]
https://youtu.be/s9RJRpBqJzE

[leaden turtle]

I made it with stable diffusion 1.5 controlnet and temporal kit and ran the images through Ebsynth. Lemme know if u want me to explain it further!

[ripe plaza]

Hybrid Video + CN + Masking.
https://youtube.com/shorts/iDxpZLEZzUM?feature=share
https://www.instagram.com/reel/CsJl18ug5BB/?utm_source=ig_web_copy_link&igshid=MzRlODBiNWFlZA==

[wintry geode]

If you ever make a how-to, add me to the distribution list lol

[visual mortar]

Hello, I just made the Guardians of the Galaxy clip - Dance Off Bro in 1080p 60fps with controlnet, what do you think? https://youtu.be/jItQJLtTXEU

[vivid nacelle]

@inner cresthow to get the animation thingy

[inner crest]

Im french so what thingy means ? You mean how to make them consistent, neat, audio sync ?

[vivid nacelle]

no

just

how do i install it

the extension

if its one

@inner crest

[inner crest]

Parseq is a webapp

and I use Deforum

[vivid nacelle]

?

[inner crest]

ok which extension you are talking about ?

😄

[vivid nacelle]

any i can make a animation with

[inner crest]

so Deforum

https://www.youtube.com/watch?v=Zvo86Y7hgfY

https://www.youtube.com/watch?v=6bZmwFZItEE

this one is probably better

[vivid nacelle]

how do i make it use the model

[inner crest]

if you install it locally with A1111, it is really easy

[vivid nacelle]

i did.

[inner crest]

do you see the deforum tab ?

[vivid nacelle]

nvm

i was blind

[inner crest]

lol you should watch some tuto about deforum first

[vivid nacelle]

i dont need

i already got it

i do rn.

[inner crest]

yep this one is good !

you need to play with strength and noise

to get better quality

[vivid nacelle]

struggling with 3it/s

🗿

[inner crest]

some examples

[vivid nacelle]

@inner cresthow do i change the video length

max framess?

[inner crest]

Keyframes

Max frames

[vivid nacelle]

ok

[inner crest]

and try to calculate how many frames per second you need

[vivid nacelle]

ik

[inner crest]

ok

10s econds at 24 fps = 240 frames

[inner crest]

yep you get it !

you may add equation like sin/cos, etc... into parameters making them easier to control

allowing you to control camera movements in 3d

[vivid nacelle]

[oak urchin]

Also always increase the anti-blur from 0.1 to 0.5 - 1 when using a zoom

[lavish harness]

This great work are you willing to share how do I get multiple prompts, it seems like I only get background changes

[inner crest]

do you use controlnet + Deforum ?

[lavish harness]

I was just using the sdk

[inner crest]

sdk ?

[lavish harness]

Yes the new animation sdk.

[inner crest]

oh yeah it is the new stuff

I havent used it yet

[lavish harness]

Ok thank you

[faint rivet]

Please enjoy Metropolis 2077
https://www.youtube.com/watch?v=gqh89qEEMTs

[lavish harness]

Thanks for the text file I used your text file but I only wanted to use 80 frames as I am paying for the credit

[inner crest]

lol awesome 😄

[lavish harness]

Lol ... thank you my friend ...

[woven cipher]

I ran the original Bad Apple!! animation through Stable Diffusion/ControlNet to create a colorized version:
https://www.youtube.com/watch?v=WeaBoHKpaws

Details of how I created it are in the youtube description of the video.

[stable hawk]

Tried something different with this one. Just Deforum this time. The scene merges into a single one 👀😅

[tidal brook]

Does anyone have experience with working on a video? Been struggling... This is an example for something I made.

[plucky plover]

https://youtu.be/u49m7_M2gUc

[vapid skiff]

FAQ: Where can I access the DreamStudio beta website?

https://beta.dreamstudio.ai

[vivid nacelle]

i extract the frames and then put frames into img2img batch with controlnet

[tidal brook]

I use controlnet with deforum, how is that different?
Thanks for the response.

[normal meadow]

@tidal brook bro can u tell me where to create these animations

[tidal brook]

What animations?

[normal meadow]

With images

Like everyone is creating a video

[tidal brook]

stable diffusion deforum

[normal meadow]

Where is the link for that?

[tidal brook]

are you using automatic 1111?

web ui?

[normal meadow]

Means?

I never used stability

[tidal brook]

https://www.youtube.com/watch?v=bicPayZDI60

This video explains it pretty well

[normal meadow]

Thank you

[tidal brook]

You need to have Stable Diffusion automatic 1111

If you don't know what it is

follow this first

https://www.youtube.com/watch?v=3cvP7yJotUM

[normal meadow]

So I need to download it in pc or it will work in mobile also

[tidal brook]

PC but you need a strong graphics card

8 VRAM +

[normal meadow]

512mb

[tidal brook]

you can't then

There are paid versions if you don't have the processing powers

You can either use google collab or things like this: https://rundiffusion.com/

DISLAIMER: I never tried them

[normal meadow]

Free?

[tidal brook]

No

Google Stable Diffusion Google Collab

For the most part google lets you use it for free. If you need more processing power or if they're too occupied they ask you to pay

But you won't have the fancy UI

you need to read a little

[normal meadow]

It means I have to pay to create videos

[tidal brook]

You pay for the processing power, yes.

Or you can buy a fancy graphics card

and run as much as you want for free

[normal meadow]

Which GPU do you have

[tidal brook]

Right now RTX 2070 and it takes me forever. I just ordered the RTX 4090 2 days ago.

[normal meadow]

Nice! How much time will it reduce?

[tidal brook]

4090 is very expensive though

Idk, hopefully a lot

[tidal brook]

this took 20 hours

[normal meadow]

How much? 2k-3k

[tidal brook]

maximum steps and resolution though

What I ordered cost me $2.2k

hopefully I will not need to upgrade my PC for it as well

which will cost more zzzz

[normal meadow]

Whaaaaat!

[tidal brook]

on RTX 2070

[normal meadow]

Yes, your pc will become future proof

[tidal brook]

But processed all frames max resolution I could and 200 steps (which is too many)

[vivid nacelle]

img2img

[tidal brook]

Yea I know what you meant, I'm asking what's the difference between your method (batch img2img and control net) vs deforum on a video using control net as well? If you know

[tidal brook]

was this from original frames of a video?

[vivid nacelle]

no

[tidal brook]

which model/checkpoint did you use if you don't mind sharing?

[vivid nacelle]

i forgot

i think

[stable hawk]

Text to special affects lol... I made the pizza smoke/steam lol.

[foggy elbow]

This is great

[faint raven]

https://www.youtube.com/watch?v=U18xah5ItO0

[lyric niche]

Does anyone know of a good solution for adding motion blur? Ideally something open source that runs on linux

I'm trying FlowBlur-App, assuming I can install it. I guess I could also just blur using the optical flow somehow?

Not having a ton of luck with the installation. I could get it to work though, hoping for a easier approach though

[ionic zenith]

My Blender / Deforum hybrid is out!

I just released my third music video assisted by AI. The first one was full AI. For the second one I also filmed scenes and ran them through AI. This time I animated scenes with Blender, ran them through AI, then also made scenes with many layers mixing pure ai and img2img with masks.

Every frame involves some contribution by Deforum.

Anyway, it just came out and I hope you'll enjoy it 😊 Open to any questions.

https://youtu.be/EV7pifgfAWc

[frozen elm]

Hi everyone, I’m an animation masters student studying in the Netherlands 👋 I’ve been writing a paper about how image generators may impact the traditional animation pipeline. I’m all for it, they’ve made my life so much easier and I’ve never felt so creative, inspired and productive.

As part of my paper I’m creating two films, one I wrote years ago about human connection, which has been entirely generated using AI, and one written with the help of AI but animated entirely by hand. I plan to present them together as a package, with a challenge: which one is more human? Which one did I make, which one did the machine make? Do you care about one more than the other because you feel one is more a human creation versus an artificial creation? I want to challenge people to justify why they do or do not respect generated art.

The first film is pretty much complete! But I’m now seeking a collaborator. I used over ten different ‘AI’ to create it, and it looks a bit like a collage as a result. The effect is pretty cool, but I’m wondering what the results could be if this ~4 minute film is fed back into an AI such as Stable Diffusion. Keen to get trippy!

I’ve had a look through the rules and hope I’m not ruffling any feathers posting this call-out - if you’re interested at all in checking out this little project and chat about potentially collaborating on the final step, please reach out. I don’t have any money or funding, and I can’t imagine this will ever make a profit. The goal is to submit the film(s) to festivals, and release them online for free along with the research paper detailing the intention of the project.

Thank you all for your time, and I hope to hear from you 🙂

[oak urchin]

@stable hawk Hey Sir !
https://github.com/Rakile/deforumation
Take a look, found this

from :
https://www.reddit.com/r/StableDiffusion/comments/127y2rm/finally_grasping_deforum_settings_in_comments/

also using a discord bot now. Plug on the webui through api. No conflict
https://github.com/Kilvoctu/aiyabot

[stable hawk]

Dude, that bobbing motion will let you make so many walk cycles!!

You can slightly increase the speed and you'll bee able to generate a run cycle that I've been struggling with because I can't get the right bobbing motion with pure equations.

[oak urchin]

Didn't even read the github yet. Seems there is examples and cie

Just waking up, i'll try to replace my boring motion settings with an infinite equation

making horizontal 8 on the screen

[stable hawk]

Infinity and beyond lol.

[stable hawk]

I've gotten hooked on the final theory currently. Maxwell's Equations were a rabbit hole I wasn't expecting.

But look! Maxwells Equations lead to this! STEAM animation! lol

[oak urchin]

pretty cool one ^^

[stable hawk]

Gauss and Farady are todays focus lol. The differential equations that relate to the electric field could hopefully lead to a ripple animation. Maybe, hopefully. It would be cool if it did.

[oak urchin]

pausing the render to adjust movement

simple but almost perfect

[stable hawk]

I was going to justify doing it my way instead, but in reality you are correct lol. But... this is not how my brain works lol. It really does boil down to me going through all this effort to come up with an equation that barely borrows from the principle and equations I was studying lol. But, on the plus side, who knows, the random physics knowledge and etc might come in handy someday. 😅

[oak urchin]

thought, would be great to be able to set pauses in the process

if my scenes are 120 frames long (or even with duration variations) in my prompt, i would set a pause for frame 120 in the generation, to see what's the generated picture and see where to go

[faint rivet]

https://www.youtube.com/watch?v=LVCppw7k-ss

[daring warren]

https://www.youtube.com/watch?v=ZM2taQ9R6ss

[humble dome]

Something I'm very proud of
https://twitter.com/volotat/status/1657811170508648451

Alexey Borsky (@volotat)

Fallout 1 intro remastered with SD-CN-Animation v0.9. Original video in the end for comparison.
#AIvideo #Fallout #GenerativeAI #stabledifussion #SDCNAnimation

▶ Play video

Twitter · May 14, 2023, 6:12 PM

[ornate glacier]

Hello. Can someone please recommend an animation extension I can add to Stable Diffusion 1111. I found out the hard way Deforum isn't compatible with Mac, and now I am having no luck adding Stable Animation SDK. If you can't tell, I am new to this. Thanks for your help.

[faint rivet]

If you can get automatic1111 working on your mac, I'd be surprised if you couldn't at least do some stuff in Deforum with it too. I mean, it uses auto1111 to render. If you do 3D mode, it will also try to load a depth model into your GPU RAM, and that can push you over the top. While I don't see people using deforum on a mac, I was unaware of specific incompatibility.

[silver lagoon]

what's the best extension for doing videos?

[safe oriole]

testing your text out at 1344x576