#🎥｜animation
faint rivet
safe oriole
huh, you can pause an animation and edit a frame and resume it? sounds amazing
gilded vigil
Goo!
inner crest
huh, you can pause an animation and edit a frame and resume it? sounds amazing
you can even do it now. Deforum iterate images. If you see a flaw in one image. You stop, clean it and resume the animation. All, frames after that are cleaner.
Now I havent test ControlNet + Deforum yet.
But I will probably use the same method with scribble mode in order to get the character. For the background, I may use the segmentation mode, etc...
inner crest
huh, you can pause an animation and edit a frame and resume it? sounds amazing
Quick comparison when you clean an image. At the end of the animation, the armor of the skeleton became something like a weird wall. If I remove it, I can finally see my beautiful princess
it is a stupid trick but let's imagine I can make the same thing with scribble mode
grand zealot
is there anything similiar/close to this AI stabilization?
https://www.artstation.com/blogs/javoraj/71X1/ai-animation-stabilization
vital yacht
New AI Animation 100% Deforum (colab notebook v.07) - "Small Space"
https://www.instagram.com/reel/CpddRfRO5Np/?utm_source=ig_web_copy_link
faint rivet
faint rivet
misty harness
vapid skiffBOT
warped ridge
So I thought, "wouldn't it be easier if it did face detection and landmark drawing for every frame?"
We'll see.
dense crater
inner crest
me trying to help ai to identify what is a face...
Ai is the future, they are clearly smarter than us...
warped ridge
the time it takes to detect and feature-paint a face, i think i might just run GFPGAN and make a mask out of the difference
inner crest
the time it takes to detect and feature-paint a face, i think i might just run G...
No idea how 😂
I always try too complicated stuffs with ControlNet...
But this time I dont think the issue is ControlNet or model
But just because in the original video, the guy wear a mask...
It fuck up completely ai. And it doesnt know if it is a back view or front view...
warped ridge
No idea how 😂
GFPGAN actually painted his face pretty nicely
inner crest
Gfpgan ?
warped ridge
face restore
inner crest
Is it an extension ?
Oh ok
It doesnt work 😂
I tried
It is because the guy is too far from the camera and full body
And sd1.5 model...
That still suck for that
warped ridge
in this case, i didn't care much about the quality of the restore. just the coverage.
i ran the process and found the difference between "normal" and "restored". boom got a painted face. it's not viable over thousands of frames though. that'd be .... rough.
inner crest
Yes exactly. what am I doing now
It is to upscale x4 all frames
With SD Ultimate
And faces are not so bad now
But it is really long
If only I had a better graphic card, I could make anything I want in few hours...
warped ridge
inner crest
Lol
warped ridge
warped ridge
pallid lance
inner crest
I will show you the original video when it is finished, you will see what was the problem 😂
Your videos are good but the difference btw your video and the original is not far. Therefore, hands and face are easy to generate...
Me I chose something almost impossible...
Also movements are so fast...
My goal is to use the poses of the original video and create something new. Also, the original video has an obvious flaw. But I wanted to see if I could make an animation with it. Since I changed almost everything except the movements, it should be possible.
But it is hard, harder than making a waifu...
keen kindle
https://youtu.be/OMbyh7KGPHQ I liking these chill animations. No vid input. Just deforum.
The no vid only applies to the chill fishing guy.
frigid vortex
Love #📹
daring warren
little glen
inner crest
pure beauty
little glen
pure beauty
Thanks 🙏🏼
warped ridge
inner crest
Anotheer experiment with ControlNet. The dancer wearing a mask was really a big error. Since his face is covered Ai couldnt detect it. Otherwise, I learn a lot. I will try something with hands and embedding next time
And Im amazed how ControlNet is able to follow the dancer
This tool is soooo amazing
warped ridge
which model? openpose, hed?
warped ridge
😦
inner crest
Sorry I forget the 1st rule of Stable Diffusion. It is not my animation but OUR animation.
It is a mix between Open Pose,
HED
warped ridge
i thought the first rule was greg rutkowski and emma watson
warped ridge
inner crest
sorry. It was Open Pose + Depth + Normal
Open Pose at 1, Depth at 0.45 and Normal at 0.45
denoising at 0.75
I think more I will make animation with ControlNet, more I will understand how it works and increase the consistency
Because there is no magic formula to get something consistent
it is just a good understanding of the original video, picking up the right models and luck
but it is crazy how ControlNet can detect arms and legs. The pink lady is not really beautiful but her poses are almost identical to the original. I would say at 95%...
near badger
warped ridge
warped ridge
Jimcarrey-fp16
inner crest
What ? Jimcarrey-fp16 ? Is it a controlnet model ? 😂
warped ridge
kidding. HED i think. i just wanted to put jim carrey on the animation as a joke.
dapper geyser
I just tried this as my first one in deforum, I dunno what I did wrong but it just generated images instead lol
daring warren
Hope you enjoy it, uploaded in youtube but appreciate the love on IG since I am shadowbanned 🫠
inner crest
quick test with ebsynth and Control Net. I probably need to add more keyframes...
what is annoying is the keyframes are visible
I wonder how I could blend them
warped ridge
blending aside, one thing you can look at for ebsynth keyframing is doing a "sheet" type generation. put your keyframes on a single image, generate against it, then chop it up.
for example. cnet (or ipix2pix) keeps everything on the rails.
it's not perfect, but images generated in the same latent space are generally more "consistent" between each other.
whole fractal
humble dome
faint rivet
inner crest
blending aside, one thing you can look at for ebsynth keyframing is doing a "she...
lets try
damn, it is crazy how sometimes the problem is not the tool but the way we use them...
warped ridge
I've got an extension for it, kind of, but it creates a series of sheets for ALL frames (rather than keyframing). https://github.com/LonicaMewinsky/Concurrent-gif2gif
It would honestly be more useful if it dynamically spotted keyframes (scene change algorithm?), and just populated the sheet with those.
inner crest
🔥
damn thanks
crazy you can use custom upscaler
ok lets convert my key frames into a gif
I dont want to convert 250frames
warped ridge
or just, you know, plot your keyframes manually. how many could you possibly have
inner crest
12
warped ridge
that's like 30 seconds in PS.
warped ridge
it'd be more useful if it took video files and only plotted keyframes. it was more of an experiment than anything.
inner crest
ok it works
but it is in 1000x1000
therefore, I will need to chop them up. Then upscale
warped ridge
womp womp. still looks sweet though.
versed garnet
https://tenor.com/view/mia-mia-khalifa-hot-big-gif-gif-18638403
oh so lebanese fashion models?
inner crest
I have no idea what is the job of that woman
crazy it works
cool cool cool
I need to adjust the ControlNet parameters
since the grid is not identical
but I think I will be able to get an animation with Mia Khalifa
whoever that woman is
dusky raptor
I was able to make this loop using the prompt "Dungeon crawler hallway" here: https://replicate.com/arielreplicate/stable_diffusion_infinite_zoom
However, I’d like to find a way to add similar animation to my own script which generates endless hallways. I tried looking at the previous link’s code but couldn’t figure it out because I suck at math. Would appreciate any help from someone smarter than me, here is my repo: https://github.com/brick2face/corridor-crawler-outpainting
faint rivet
elder bluff
wanton leaf
@warped ridge which controlnet models are u using to get a render for the grid?
warped ridge
Any aggressive edge detection. Canny or hed work best.
wanton leaf
is it all done through the script or is there a stage i need to pull it into controlnet?
warped ridge
Oh, my bad, context: those are individual frames that are pulled from a gif and aligned through a separate process before being generated against in SD
wanton leaf
yeah im up to the stage of manually plotting them in PS
but now i skip the script right?
these are just the keyframes
warped ridge
if you're manually plotting them there's no magic. just controlnet to keep stuff on rails.
warped ridge
make sure to increase cnet annotator resolution to match, also "just resize"
warped ridge
wanton leaf
oh right
warped ridge
if this is too small, it will not have the detail needed to keep the frames in place. also, if "resize mode" is set to anything other than "just resize," it'll dislodge the frames.
warped ridge
without scripting i don't think you'll be able to get dimensions you need. the ui only does 2048.
wanton leaf
do u ever use canny as well as hed together
warped ridge
yes. every case is a little different.
warped ridge
i can always upscale again later?
https://github.com/LonicaMewinsky/Concurrent-gif2gif/blob/main/Scripts/c-gif2gif.py this code has MakeGrid() and BreakGrid() if you know some python.
those are what i use to plot images and break them apart again.
wanton leaf
niiice
wanton leaf
@warped ridge what about making a gif of just the keyframes and putting that into the script?
mental pecan
New video I just released.. https://youtu.be/OGu5Er4TFTc used deforum to generate this along with blender tools and some other tools to help me work on the key framing the audio with the animations.
faint rivet
hearty reef
Is there any way to batch inpaint exact same results on different images? Like I'm trying inpaint sequence of images and it must be similar inpainting bcoz i want to make an animated gif out of it.
warped ridge
shameless plug: gif2gif ui extension can help. there's an option to blend the frames together in the inpainting window so you can mask it efficiently before generating.
warped ridge
my sides
novel prism
fallen girder
"Dark Rites" | Deforum Animation (Stable Diffusion v2.1) [4K] - Workflow in the video description
https://youtu.be/tgPhbfWlHmQ
warped ridge
<@550184985332547594> what about making a gif of just the keyframes and putting ...
technically, yes, that'll work. but that seems like such an odd roundabout method. like people who begin their workflow description with "first convert your movie to gif, so you can use gif2gif"
ITS FOR MEMES
i'll write up a "keyframer" script later today which does this properly. sounds like people have a use for it.
humble dome
Apparently ControlNet is very good at changing personality on close up face animation. (No deflickering used)
inner crest
yes it is the best for that
I give up on creating a image grid
I try this now
it seems to work well
and probably better to create keyframes
For example, frame00
and at frame 36
I think the problem is now the parameters I use in ControlNet
But the faces are identical
so I could upload only my keyframes. Use the script. Then use ebsynth
Even if with the script it seems working great by itself
test0001 is not so bad. But it doesnt really follow the original animation...
hummm...
what did I do wrong ?
warped ridge
warped ridge
@wanton leaf @inner crest here's an extension that will plot/generate/break keyframes if you're interested. https://github.com/LonicaMewinsky/sd-webui-keyframer
fading turtle
inner crest
<@217213209831145474> <@1006234707375173722> here's an extension that will plot/...
Thank you, I will try it soon
test0002 is another failure...
I wonder why it doesnt follow the animation
inner crest
I need to control every frames !
but thanks 😄
I think I know why
I have a seed number
instead of -1
restive ether
Hello...is there a dreambot text to image in this account?.. i saw a video there were but i dont see them now
warped ridge
but thanks 😄
Post it on reddit with something self-deprecating. Easy karma.
sudden phoenix
Hello...is there a dreambot text to image in this account?.. i saw a video there...
No it's not here anymore see #1072220168534642768
faint rivet
𝑲ₙₑᵣₖ𝓥ᵃᶠᶠˡᵉ?!¿¡ - ( KNERKVAFFLE CyberPunk Tour! ) https://youtu.be/1OOhJUkdV-8
inner crest
test0003 is another failure
test0004 will be the good one. I feel it !
https://tenor.com/view/oakley-sciencedog-oakleyscience-gif-22346803
inner crest
Test004 works !!!!
well well well lets try another one with embedding
then I will try to create my keyframes with it and run ebsynth
faint rivet
hearty reef
shameless plug: gif2gif ui extension can help. there's an option to blend the fr...
I tried this but its generating different results on every single frame, any solution?
wicked nexus
I made my first VR video, had to use Disco Diffusion, does anyone know of an extension or notebook that can make VR videos with Stable Diffusion? This is real ugly and rough but I wanted to make sure the 3D vr would work before I got too far with it and it was easier than I thought it would be. I also just got a VR camera so I am going to try to mix VR video with AI animation and for now will probably make some more Disco Diffusion ones. I am also going to experiment to see if I can use it as video input for SD but it will probably mess up the VR unless I change it very little and there would be not much point to that.
warped ridge
I tried this but its generating different results on every single frame, any sol...
The mask is changing? Or the generations? In terms of generations you're still at the mercy of inconsistency between images.
Unless you unchecked "common seed for all frames" though your results shouldn't be wildly different frame-to-frame.
warped ridge
test0003 is another failure
7/10 needs more melting
hearty reef
Unless you unchecked "common seed for all frames" though your results shouldn't ...
Yeah i haven't unchecked it , the thing is i need them to be exactly same for it to work, and I'm getting similar but different generations, suppose i want a red hat on 5 frames and that hat must be exactly same in all frames so that it looks good in the actual gif
warped ridge
Gif may not be your vehicle of choice, then. It's for memes. There are more ideal methods of getting consistency across frames, such as limited key frames and ebsynth.
inner crest
test0005 work fine also.
It seems embedding and multi Control Net is not a problem for the script
the hair change but the face stay the same. Therefore, it could be easy to get even more consistency
Also, my prompt wasn't so great...
I dont do NSFW but I guess it would be easy to make a animation with Mia Khalifa in swim suit. Something so consistent and real that people will get confused quickly. An animation at million of views
I also see embedding with Belle Delphine and all these women of easy virtue
Therefore, I guess a lot of sinners will make great videos. But I wont be one of them...
Anyway, time to start ebsynth test
inner crest
test0006 with Ebsynth. Keyframes are still visible. But I think the problem is me. Not the keyframes. I dont know how to blend in Davinci Resolve. Therefore, Im sure it must be really easy for a pro to get rid of that. Me I just have no idea how..
But look how keyframes look similar
without the Lora model and better ControlNet parameters, it would be even more accurate
Anyway, time to learn Parseq and become a legend
warped ridge
What does the original look like?
inner crest
ControlNet dont track the movements of eyes
and LoraModels dont care about the smile, eyes, mouth, etc...
but the movement of the head is accurate
OK time to read more about Parseq + Deforum ! I need to be able to create that:
warped ridge
obligatory emma watson
inner crest
lol
smile is good for this one
even eyes ?
a little
what are your COntrolNet parameters ?
I probably need to decrease the Lora weight or ti weight when I use them
warped ridge
hed, 0.55 guidance end. i didn't do anything weird here.
probably do canny and tune that to get eyeballs and smile if that's your focus
inner crest
oh yeah you lower the hed
me I put it at 1
probably, my goal was just to make the script work 😄
But Im sure with better parameters, the result may be awesome
even for ebsynth
We are now able to get insane consistency in our animation
wanton leaf
@warped ridge i found a normalmap weight of 2 was good for the sheets lol
for the dimensions i was using
warped ridge
Oh normal sounds good. Never thought about that. Didn't think it would hold the frames as well.
wanton leaf
the main issue im having is capturing the facial expression and mouth details etc. i think the embedding im using has an aversion to open mouths.
when i put the CFG slider way back unfortunately i cant change much i want to get rid of then
might have to do some intermediate steps first to transfer the things i want and then increase noise
warped ridge
Transfer things you want? Expand on that?
wanton leaf
so i wanna change his shirt and remove the hat
but keep the facial expressions
open mouth, showing teeth etc just how the original is
otherwise my keyframes are all gonna be closed mouth
warped ridge
Yea some loras just legit do NOT do open mouth. Sometimes specifying open mouth works though.
wanton leaf
ye i tried it doesnt work
its not even lora just a ti
think i gonna have to make my own
dont just want open mouth i want to keep the same facial expression
same mouth
basically wanna copy the movements, expressions, etc
but change the visual
pallid lance
I would love to communicate with you all, but I don't understand the language. Is it funny? I'm thinking of a good way to translate it into English.
warped ridge
have you tried combining normal with hed or canny?
or say, guidance end for normal at 0.35 (to get stuff on track) and leaving hed at >0.7 to get facial features
wanton leaf
ill try those weights
i had 5 controlnets enabled at one point but the fault is with the embedding
pallid lance
wanton leaf
720 steps trained one i pulled off civit
pallid lance
😆😆😆
wanton leaf
lol
ไม่ต้องกังวล ฉันพูดภาษาอังกฤษและยังเข้าใจสิ่งที่พวกเขาพูดถึงได้ยาก
the thing is im thinking of deepfaking over the top with deepfacelive as well
so the features dont even need to be exact, just the facial expressions at this point
warped ridge
it really is tough to track eye movement.
mental pecan
https://www.instagram.com/p/CppkhtvOG64/ new post today
nimble lotus
mossy ridge
wanton leaf
metahuman in unreal engine has an option to keep eye focused on viewer
but u would have to video2animation with rokoko studio
then rig to metahuman and enable that option
then animate over the top
wanton leaf
kek
warped ridge
emphasis on "special"
faint rivet
New Optical Flow Cadence! Deforum Stable Diffusion WebUI https://youtu.be/qnxEEBHKqaY
noble quail
hey guys! My stablediffusion based talkshow ColinDiffusion has a new episode! https://youtu.be/Aatfg7CTFF4
mental pecan
https://www.youtube.com/shorts/0LwzdTkGHIo this uses Stable Warp with Control Net..
trail valley
New Optical Flow Cadence! Deforum Stable Diffusion WebUI https://youtu.be/qnxEEB...
how does one put the correct init path for hybrid video
pallid lance
faint rivet
how does one put the correct init path for hybrid video
in the video init path in init tab
link to documentation in the hybrid video tab... llink to tutorial in documentation
trail valley
link to documentation in the hybrid video tab... llink to tutorial in documentat...
i guess i'm trying to find out what path everyone is typing in for it to find the video clip im referencing?
wanton leaf
inner crest
ok I can generate now but no idea how to change anything and what Iam doing 😄
inner crest
I get rid of the camera movements but I still have no idea how to control the interpolation...
inner crest
I can control the interpolation now... Time to see if I can change the prompt...
trail valley
in the video init path in init tab
what does one type in the video init path?
faint rivet
what does one type in the video init path?
the path to the video
inner crest
Not easy to have control of the camera movement. But I start to get it
inner crest
that thing goes wild easily
inner crest
I wont lie Im not a big fan of the audio sync of Parseq. Camera Movements are too wild. I think I will stay with my cos/sin functions for smoother movements
Anyway, I can add Parseq to my infinity gauntlet ! It is insanely powerful and I will probably use it for my next animations in Deforum.
Ok time to learn Deforum + ControlNet
daring warren
Hey there bois and girls, made a little collab with a very cool guy with a very cool art and music, hope you enjoy it.
Some love on the Reel is always appreciated
Instagram
vast cipher
As I play with this technology more I find myself doing two different workflows. 1) I start with a single frame and use image2image generatively, with a relatively high strength setting, letting it wonder off and create imagery. (Typically I'll add in rotation/zooming). 2) The other workflow I use is an image2image on each frame of the video, typically using a relatively low strength value so I don't lose the shape too much. (I am using CoreML locally on my mac, with custom swift code, so don't have any fancy ControlNET or anything).
The interesting thing is that I like some of the flickering, and wouldn't want it to completely go away. I played with ebSynth a bit and its feels to filter-y. I want the flickering randomness. My favorite bit in this video I made is right at about 1:45. my hand turns into a fish-like creature with eyes...so weird. love it. Anyway, enjoy, just made this over the weekend. https://www.youtube.com/watch?v=h93WxBI9izE
pallid lance
400 frame lol 
"Is there anyone who can make a set unchangeable? I think it might be related to LoRa."
"I have made everything static, but I think the only thing I cannot make unchangeable is the set because the LoRa module is not trained well enough."
glossy steeple
ai test animation
faint rivet
@blazing matrix
dapper geyser
Hey there bois and girls, made a little collab with a very cool guy with a very ...
Whoa @tame otter check this out
daring warren
Whoa <@231531230649122816> check this out
humble dome
Made a script that allows to stylize long videos with ok-ish consistency. Gonna opensouce it right after I'm done with this video. Although it takes ages... 7 minutes video estimetad to be processed in 4 days... wooh.
faint rivet
processed in 5 mins
dapper geyser
On what?
round jay
faint rivet
On what?
Well, I have a 4090. But, that means you could do it in less than a half hour on a 3060 12GB.
faint rivet
On what?
The ***"speed" ***is mostly due to my new optical flow cadence in deforum. Instead of just blending between frames while animating, it now morphs while it blends. This allows for much higher cadence (in-between frames), and much less actual rendering. For instance, I'm working on a settings file right now that runs at 30fps, with cadence 20, a total of 1200 frames. But, Stable Diffusion only rendered 60 frames in this entire video. The rest were filled in by deforum's cadence. Only 1 out of 20 frames are "real" here.
60 diffusions stretched out to 39 seconds at 30fps
all the same, each one was 60 actual diffusions
To be clear, at 30fps cadence 20, it's only rendering 1.5 frames per second of video. The great part is that the cadence, even with optical flow, is FAST.
sour gulch
Made with Deforum and Realistic Vision v1.4
dapper geyser
The ***"speed" ***is mostly due to my new optical flow cadence in deforum. Inst...
Neat
faint rivet
Here we go... a whole video
https://www.youtube.com/watch?v=Zm2LAMfHWNc
sour gulch
Here we go... a whole video
https://www.youtube.com/watch?v=Zm2LAMfHWNc
These are super cool.
daring warren
New Video out bois and girls, hope you enjoy it.
Little love on the reel it's always appreciated
faint rivet
mental pecan
Reimagined AI version of this amazing skit with Jenna Ortega - SNL
https://youtu.be/X5q33a5o2-k using stable warp..
mossy ridge
hi, this is my first ever short film and it was made thanks to Stable Diffusion and Deforum. would love to hear your feedback.
https://utkay.io/insomnie
unkempt imp
guys when i use midjourney or something it change a lot the base image
i want to do something cool with that ;
withouth changing too much the picture
like more or less same clothes, same guitar (important)
do you know how i can achieve that ?
unkempt imp
here for exemple the two picture are totally different
do you understand my purpose ?
whole fractal
radiant hedge
Here we go... a whole video
https://www.youtube.com/watch?v=Zm2LAMfHWNc
There is no way you only used stable diffusion here
faint rivet
There is no way you only used stable diffusion here
I'm one of the developers of Deforum.
radiant hedge
Still did you used after effects as well?
faint rivet
Still did you used after effects as well?
I rendered it in one shot and I only did upscaling after.
in a few hours on a 4090
my new optical flow cadence helps a lot...
this is literally the file I made to do it
I was using a combination of keyframe motion and "Affine" hybrid motion which does RANSAC tracking on video and tries to replicate the camera angle - causing the glitchy behaviors. I'm also mixing in a very small amount of video here, but it's nothing like what you're seeing. It's just serving as noise to feed elements of the video. Link to original video is in yt description.
Stable Diffusion can do pure black and white? who knew?
vital yacht
New AI Animation 100% Deforum: "Lilith's Metamorphosis" 🔴 Soft Nsfw
https://www.instagram.com/p/CppkbPEMy6g/
radiant hedge
I can imagine this last one being used for the end credit of some sorta horror/comedy show
red mirage
ai test animation
Curious how you were able to get the face to be consistent at that distance. I keep running into issues with img2img when the person's face is too far away.
haughty isle
New Parseq tutorial covering prompt changes and basic beat sync: https://youtu.be/PvWckT0Pik8
frozen finch
brittle needle
I tried using the multi-controlnet approach to get an ‘anime ish’ style
novel badger
I rendered it in one shot and I only did upscaling after.
Do you have any tutorial of yours on this process? Is it possible to apply your technique in google colab?
faint rivet
Do you have any tutorial of yours on this process? Is it possible to apply your ...
I don't use colab, but I know that some people use deforum webui in a colab. I don't know if it's really possible for me to make a tutorial for my "process"... but, I have a settings file you could review.
I just kind of do stuff.
but, I am going to be making a comprehensive tutorial soon for the features I've added to deforum. I only have one short tutorial right now, for hybrid video, and it's linked from the documentation in the ui.
noble quail
sonic stratus
So have people had any success replicating what corridor crew did with consistency between frames and non flickering? Just curious if the community has been able to achieve what they have
lyric niche
First is deflickered. Second is with no post processing.
The main victory here is the background doesn't change very much
Using a video that drives an openpose control net. Vanilla SD 1.5 otherwise.
Well the model is vanilla, the pipeline is custom. It blends the prior noise depending on the magnitude of the optical flow of the original video.
keen kelp
livid wasp
whole fractal
faint rivet
Harold & Kumar go to NightCastle
https://www.youtube.com/watch?v=4Lz7WdQLQRY
mental pecan
Updated guitar video using deliberate v2 model.. I find this one to be pretty flexible.. just doing some tests with different models.. some work well some dont, of course depends on the style.. this is less cyberpunk style than the original posted.. (stable warp + CN)
crisp marsh
music video made with stable diffusion: https://www.youtube.com/watch?v=aCOyl31q8W8
stuck harness
My band 2 Fat 2 Skydive is releasing our album next month, our first music video dropped today. #deforum
humble dome
Long and relatevely consistent video2video created with this script I made: https://github.com/volotat/Reference-based-SD-CN-Animation
https://www.youtube.com/watch?v=YW1JBJ57YBQ
gilded vigil
Goo!
radiant hedge
Updated guitar video using deliberate v2 model.. I find this one to be pretty fl...
Woahh
dusty copper
Updated guitar video using deliberate v2 model.. I find this one to be pretty fl...
minda doing a miniguide?
this is coherent as HELL
radiant hedge
this is coherent as HELL
prolly used video2video and low image2image strenght
dusty copper
prolly used video2video and low image2image strenght
kinda new to animation, what is v2v?
radiant hedge
kinda new to animation, what is v2v?
more like extracting your video to frames and then batch the images with a filter and put it back together as a video. I am not to familiar with creating animations tho.
mental pecan
minda doing a miniguide?
its on my todo list for my website and YT..
faint rivet
Gorillas in the “Mist” https://youtu.be/PPWdpmrEtDs
knotty dust
dense crater
autumn dove
mossy ridge
ai test animation
How?
cosmic rain
Updated guitar video using deliberate v2 model.. I find this one to be pretty flexible.. just doing some tests with different models.. some work well some dont, of course depends on the style.. this is less cyberpunk style than the original posted.. (stable warp + CN)
mystic cape
Newbie here..just playing arround Deforum with parseq.
Inspired by: Jordan Rudess (Dream Theater)
Music: Ra-Jordan Rudess
Feel free to visit my IG: @mystic cape
slim oak
This is a photo that I turned into an AI video using Stable-Diffusion on Unishape.io. 【Animate】It really looks like a real person. AI is becoming more and more amazing.
https://www.unishape.io/creative
tepid trout
https://www.youtube.com/watch?v=_vL2kGysGIM&ab_channel=ivcoffee just some fun with parseq
faint rivet
Wireframe Journey - Lucy's Cosmic Adventure https://youtu.be/SXXcalsapng
faint rivet
See description for link to original 3D fractal animation:
https://www.youtube.com/watch?v=IoH_4bFUpSo
radiant hedge
This is a photo that I turned into an AI video using Stable-Diffusion on Unishap...
Can this be done for 3D model characters??
lone ember
hey guys, anyone recommend using a cloud for rendering animations? my gpu is old (1070 gtx)
near badger
radiant hedge
How'd do that?
lone ember
any tips of how easliy I can turn footage into animation? I want to be able to take an mp4 video and turn it into animated version
is it possible?
mossy ridge
HELLOOO!!!!
cosmic rain
eagle fly ¯_(ツ)_/¯
tacit spoke
welcome both of you
but there is no bot around, if that's what eagle fly meant #1072220168534642768
still vine
Hey guys, is there a surefire way to avoid Deforum getting blurry after the first frame in its animations? (Running it locally with A1111). First frame runs the number of steps specified in the interface and every frame from then on is generated with a lot less steps and I can't figure out why.
tacit spoke
Hey guys, is there a surefire way to avoid Deforum getting blurry after the firs...
the lower the denoising, the less steps get performed. Adding a little more denoising should fix that, or using the setting :
still vine
the lower the denoising, the less steps get performed. Adding a little more deno...
thank you! unfortunately neither the setting in the image or rewriting values in Deforum settings did not change that but in theory that is what I should be tweaking, I'll work on this
thank you!
tacit spoke
thank you! unfortunately neither the setting in the image or rewriting values in...
Try maybe on deforum discord, they may know best. Hope you make it !
cedar vapor
lilac hound
Looking for feedback on my AI animation. What do you guys think ?
(Full resolution animation here --> https://youtube.com/shorts/ulsfa71qChM)
weak pecan
Finally managed to get ControlNet implemented into my Unreal Engine plugin. This animation is using both the normal map and depth map ControlNet models along with SD 1.5. The background is flickering quite a bit, but I believe it could be from the depth map not capturing enough of the scene depth. I think that I may be losing data by normalizing the depth layer to 8bit RGB before passing it from Unreal to Python
radiant hedge
Looking for feedback on my AI animation. What do you guys think ?
(Full resoluti...
I need to get better in controlnet....
lone ember
which one faster to render a video? png2png or video init?
fallen girder
"Cyborgs" | ModelScope text2video Synthesis [4K]
https://youtu.be/TFOlNB5VabA
humble dome
Some experiments with ModelScope model
https://youtube.com/shorts/3rqkVrH-0dg?feature=share
daring warren
sharp depot
whole fractal
inner crest
Oh normal sounds good. Never thought about that. Didn't think it would hold the ...
no more keyframes visible !!!
damn...
what a script
I just forgot to change the parameters in Ebsynth. Therefore, it is probably possible to get even better result
noble quail
radiant hedge
no more keyframes visible !!!
How do you make it so smooth? Does this exist for full body characters and to make them turn around?
opaque frigate
Please I have a question how can I create my photo as animation.
safe oriole
oh control net is finally running for me, little tests without attempting to use it (same input video as output video) canny - openpose, unsure why the video degrades thou, happened the last day for me, either its model / new settings / something else 😄
slate fulcrum
Hello guys, anyone have an idea how to do this : https://www.youtube.com/shorts/cGSn6N8AIeI ? I can't find any good tutorial about it 😦
tepid trout
https://www.youtube.com/watch?v=lKKY6vmiurw&ab_channel=ivcoffee trying different prompt weights based on pitch values
tepid trout
Hello guys, anyone have an idea how to do this : https://www.youtube.com/shorts/...
have you looked up parseq?
slate fulcrum
have you looked up parseq?
what is parseq ?
tepid trout
just the most amazing tool :d https://www.youtube.com/watch?v=PvWckT0Pik8&t=1306s&ab_channel=RobinFernandes the creator posted it a few days ago here I think
slate fulcrum
just the most amazing tool :d https://www.youtube.com/watch?v=PvWckT0Pik8&t=130...
oh ok, from what I know, the video I sent is made with deforum using init images but I don't know how it works lol
tepid trout
oh ok, from what I know, the video I sent is made with deforum using init images...
for sure you can do that but it seems tedious. You'd have to sync to the beat in a way, and then drop the strength a few frames before the beat, and rise it up after
slate fulcrum
for sure you can do that but it seems tedious. You'd have to sync to the beat in...
it's possible to add inits or do I have to redo the image with parseq ?
tepid trout
it's possible to add inits or do I have to redo the image with parseq ?
honestly I haven't tried using img refs with parseq
shell fiber
frank kiln
music video I made after following the corridor digital tutorial as well as some deforum
warped ridge
It boggles the mind why the modelscope folks used shutterstock images rather than something like http://moments.csail.mit.edu/.
There are a number of contexted and indexed video datasets out there, for free, that don't have an ugly ass watermark
fallen girder
"Shinobi" | ModelScope text2video Synthesis [4K]
https://youtu.be/Ae5n4kgS_ms
inner crest
How do you make it so smooth? Does this exist for full body characters and to ma...
Ebsynth + ControlNet + Scrit in SD. And ebsynth may do full body character as well. But it is really good for face
humble dome
If Terry built an AGI in the TempleOS
summer dove
Why was my video removed? Here all my videos, the last one was removed from here: https://www.youtube.com/channel/UCQNM7wWAP_ggu_iyGyClqNw
Someone don't like spooky monsters here... (video is safe for work btw)
near badger
balmy sinew
Looking for feedback on my AI animation. What do you guys think ?
(Full resoluti...
beautiful! wow! can you outline the process?
plucky plover
Text To Video + IMG2IMG . Workflow in Youtube description! https://www.youtube.com/watch?v=4G8F42hMC0Q
keen kindle
https://youtu.be/GLumpzSASHc Was going to post to reddit but for some reason they took down my account and deleted everything about the Steampunk Circus. No reason given. I didn't break any rules. What a terrible feeling. I put a good lot of effort into providing content. Oh, well. Screw Reddit. Distributed networks that can't crap on users on a whim are where it's headed.
obtuse wave
Sabiduría
glad viper
warped ridge
So I found a dataset with 150k+ pre-captioned animated gifs. "Neat. I wonder.."
Welp. haha
hahahahaha
"text2gif" needs some work.
summer dove
rain anchor
Text To Video + IMG2IMG . Workflow in Youtube description! https://www.youtube.c...
Thanks for this man, saw your Reddit thread and it didn't occur to me to batch the output frames. Really clever idea!
plucky plover
Thanks for this man, saw your Reddit thread and it didn't occur to me to batch t...
amazing work!!! happy to see others doing it as well. really cool workflow!
lilac hound
I need to get better in controlnet....
If I can help in any way, don't hesitate to ask
lilac hound
beautiful! wow! can you outline the process?
Stable diffusion to create the images, deforum to animate with prompt scheduling, ControlNet to keep structure in the animation and visually interesting. If you have other questions, don't hesitate to ask
I finally succeeded in animating photorealistic humans.. .Phew
See the full resolution here -> https://www.instagram.com/reel/CqDr7fToJcO/
Or youtube or whichever you prefer through linktree -> https://linktr.ee/midnightdata
Any feedback ? What do you think ? Trying to improve.
Workflow ===
It's StableDiffusion2 with Deforum (scheduled prompt) animation and Depth based control using ControlNet.
Second example: same workflow. If you're interested I share my settings file. Any feedback ? What can I do better ?
Full resolution here -> https://www.instagram.com/p/CqCEzqdgnSo/
chrome grove
Text To Video + IMG2IMG . Workflow in Youtube description! https://www.youtube.c...
wicked stuff dude 👌 I just got into SD and looking to incorporate it with Blender and Ae. I'll use your method as a guide and hopefully post something good soon
plucky plover
wicked stuff dude 👌 I just got into SD and looking to incorporate it with Blen...
amazing! you’ll benefit a lot from knowing other programs like that. good luck!!
inner crest
well well well, it seems ControlNet allows the basis of my idea for my next animation to work
Now I wonder if I can use Deforum + ControlNet + Parseq...
how to get the same thing in Deforum + Control Net + Parseq ?
inner crest
Second example: same workflow. If you're interested I share my settings file. **...
have you tried Parseq or subseed in Deforum ?
for me your problem is the cadence
clothes change too fast, same for background. You keep consistency of the woman, probably because of the seed schedule and prompt
inner crest
Second example: same workflow. If you're interested I share my settings file. **...
and look how her hair and jewelry change
today we are able to get something with more consistency
Otherwise, good job to get a such perfect face with camera movements
chrome grove
Brought a Blender animation into SD. Slowly getting the hang of batch img2img with controlNet. I'm going to try Deforum + ControlNet next to compare results/control.
The next step is to work on individual layers, preserve the alpha with depthmasks, and composite in Ae.
mossy ridge
knotty rune
faint rivet
“Don't Leave Me Lonely” - Zombie Flowers Version https://youtu.be/iuwdQPHfCc0
summer dove
proven rune
summer dove
humble dome
Alien doing ASMR. Both video and sound are AI generated.
https://www.youtube.com/shorts/MuEa83r4PBQ
tepid trout
I finally succeeded in animating photorealistic humans.. .Phew
See the full reso...
that's really nice.. you can also keep more consistency with seed schedule ..or cycle between seeds
keen kindle
summer dove
Youtube Short https://www.youtube.com/shorts/2NivmNGBkD0
faint rivet
primal elk
What channel should i visit if I have init Settings questions
need to know how "strength" operates
chilly hedge
apple
mossy ridge
I’ve seen videos a lot like these on tiktok and I’ve seen some models on civitai and I was wondering how I could make some myself https://www.tiktok.com/t/ZTRvfkreM/ anybody know how?
summer dove
broken kayak
Image of a child
vapid skiffBOT
@broken kayak
lean shuttle
https://youtu.be/JQx2cKDVsgc
how to make the character do Natu Natu steps?
summer dove
flint sluice
https://www.youtube.com/watch?v=Q08fzjTVKQs Enjoy all 🙂
humble dome
Anybody know any good way to remove watermark from the video? img2img does not work well as it either keep the mask or destroys coherence of the video.
fallen girder
"Zombification" | AI Generated Short Film (ModelScope text2video Synthesis) [4K]
https://youtu.be/AAbN9yFwstE
fallen girder
Anybody know any good way to remove watermark from the video? img2img does not w...
check the AniEraser app: https://anieraser.media.io/app/
humble dome
check the AniEraser app: https://anieraser.media.io/app/
Thank you. By the way, amazing video, music fits perfectly and it feels like a genuine music video.
fallen girder
Thank you. By the way, amazing video, music fits perfectly and it feels like a g...
Thanks! Aitrepreneur explains here how to use the AniEraser for free unlimited, just deleting the cockies each time: https://youtu.be/BhN2XVgUNSg?t=582
humble dome
Thanks! Aitrepreneur explains here how to use the AniEraser for free unlimited, ...
Hey, good trick, but it usually not a good idea to show stuff like that on a large audience as it tend to be quickly fixed.
fallen girder
Hey, good trick, but it usually not a good idea to show stuff like that on a lar...
yeah you're right, possibly there are other free alternatives for when they fix it
flint sluice
does anyone know if there is a discord for modelscope ?
( dont post it here, u can pm me it )
humble dome
I'm also interested
flint sluice
make sure you all update ur extension!!!! im currently generating 384x384 video omg
slowly testing the limit
now for 512x512 😄
whole fractal
humble dome
slowly testing the limit
It tends to generate stuff like this is if I do higher resolution of longer videos then default.
At least for me. Do the fixed it?
humble dome
so 384x384 seems to be my max on 3080 10gb - nothing weird like that tho
Try to do 32 or 48 frames. Does it degrades?
humble dome
Yeah.. It does. It is not so bad as my example, but this is seed dependent, sometimes its ok (like in your result) and sometimes it totally messed up
flint sluice
tbf I wouldnt say that looks any worse than my normal ones
humble dome
The strides became hugely exaggerated. It is what usually happens to me too. It's like.. the texture becomes main part of the video. Try it a couple of times you'll see
flint sluice
yeah i see what you mean
next im trying to do 256 x 256 but longer than 5 seconds
mind if I add you Alex? finally met someone into this as much as me 😄
humble dome
More then happy. Regarding the modelscope discord. There must be one which https://github.com/deforum-art/sd-webui-modelscope-text2video devs use. But I cannot find link anywhere
faint rivet
Deforum - 2D Offset Center Rotation Test w/Random Prompt Engine https://youtu.be/ck84aAUHS2o
faint rivet
More then happy. Regarding the modelscope discord. There must be one which https...
There are channels in the deforum discord for it.
humble dome
There are channels in the deforum discord for it.
Could you send me a link?
faint rivet
Could you send me a link?
No... I mean, it won't let me. It's pretty straightforward though.
humble dome
No... I mean, it won't let me. It's pretty straightforward though.
Yeah, found it
faint rivet
Yeah, found it
The extension is cobbled together using parts of deforum, and it's rapidly evolving.
humble dome
The extension is cobbled together using parts of deforum, and it's rapidly evolv...
It blows my mind how rapidly everything is going.
summer dove
mossy ridge
misty dagger
summer dove
flint sluice
when you're baked 3 in the morning
mums watching their kids open presents on christmas day
flint sluice
flint sluice
tacit spoke
Since those are "very short" shots, did you try the new trick of editing all frames in the same batch ? The results I saw were just perfect in quality
no more jittering
flint sluice
Oh no?? Got a link to explain this?
tacit spoke
I'm not finding one right now, but I'll explain quite simply : when doing X pictures in a single batch, the small details tend to copy, from one picture to the next.
Using this fact, and a high batch size in img2img, you put all your frames in a single batch and process all of them at once.
it results in the bear being the same in each pictures almost for example
warped ridge
There's a "multiframe" approach, a la https://xanthius.itch.io/multi-frame-rendering-for-stablediffusion
But you might be talking about https://github.com/LonicaMewinsky/sd-webui-keyframer, where you place all the frames in a single grid and generate against that
summer dove
summer dove
summer dove
hearty torrent
Hey all. Just uploaded my first deforum video. It's a dark comedy horror clown story... Every aspect is AI, including the script and OST. https://www.youtube.com/watch?v=vg4n71v-VbI&ab_channel=JeffBoerst
tepid trout
https://youtu.be/Y70EGMJ_X-Y tried 12 cadence, 12 fps, spent 6 hours upscaling with 4x-ultrasharp and then interpolated with flowframes. Should have done it at 24 fps but 4x-utrasharp is too slow at ~15sec/frame (4k frames take almost a full day)
faint rivet
safe oriole
https://www.youtube.com/watch?v=2egBsCEIUtA
reminds me of this, which i wish i knew how to recreate, stumbled upon it end of last year https://youtu.be/ldFnVamk6eM?t=78
summer dove
dense crater
faint rivet
humble dome
Has anybody experimented with TemporalNet? Is it actually any good for video stylization?
summer dove
https://youtu.be/Y70EGMJ_X-Y tried 12 cadence, 12 fps, spent 6 hours upscaling w...
Really cool video, what is the prompt for that?
misty dagger
summer dove
wanton leaf
summer dove
tepid trout
Really cool video, what is the prompt for that?
colorful alien planet background, giant mushrooms, waterfall, tropical vegetation, rocks, anime, clean sharp focus, raytraced reflections, even lighting, no contrast, octane render, shiny wet surfaces, warm light, (4 k:1.36), ingame shot by greg rutkowski, rosey fujita ... a mouthful
swift sundial
summer dove
Short with insects... I don't like it mutch: https://www.youtube.com/shorts/BcFIk3PmE7Q
negative prompts are hard with this stuff... insects are deform and have multiple legs... and are ugly.
daring warren
faint rivet
humble dome
https://www.youtube.com/watch?v=dasNNW3IIbI
Stylization strength is a bit too much.
Btw, is it just me, or gen-1 feels like an SD with motion flow and some postprocessing? ModelScope vid2vid seems to look much more consistent.
faint rivet
Btw, is it just me, or gen-1 feels like an SD with motion flow and some postproc...
LOL... You didn't read the description.
I made it sort of lame on purpose... like the original video. haha
I'm the creator of hybrid video in deforum. LOL
The original was 240p 30fps, and I dropped it to 15fps... then didn't mask anything... and made the stylization like that all on purpose. Haha. If I wanted to do a good one, which I may, I'd first enhance the original video, then slow it down (effectively making it 120fps), so the motion could track it, then convert back to like 60fps. But, I wasn't gonna go through all of that. I just wanted a quick funny render!!! 🧌
used deforum w/hybrid video and controlnet
faint rivet
Btw, is it just me, or gen-1 feels like an SD with motion flow and some postproc...
It seems a lot more like Instruct_pix2pix to me actually
summer dove
dusky raptor
Now with smoother animations: https://github.com/brick2face/corridor-crawler-outpainting
swift sundial
steel gyro
whole fractal
summer dove
This double interpolation i made is not bad! https://www.youtube.com/watch?v=5l99VLCanag
safe oriole
testing controlnet out as a scaffold for text to image deforum https://www.youtube.com/watch?v=WH_EHXWrg8s
spring dagger
hearty torrent
NSFW My Second attempt with deforum. I used 100 MJ generated images and laced them together with deforum into a music video for Haddaway's "What is Love". Pretty darned good facial coherence, I must say. Basically, a 100 models dancing on a runway as they mutate/melt into one another, so with heavy horror elements, highlighting the "Model-as-Meat-Mannequin" concept. https://www.youtube.com/watch?v=btwfboWbQk0&ab_channel=JeffBoerst
timber coral
tepid trout
https://www.youtube.com/watch?v=0YtajfSGoB4&ab_channel=ivcoffee i have no patience to figure out camera movement
summer dove
haughty isle
FYI: Parseq v0.1.50 is now live with major performance improvements, zooming & panning on the graph, a new audio reference view so you can see an audio waveform alongside your params, a new rand() function and more. https://sd-parseq.web.app/
lethal ridge
I can't find any good tutorials for this, where did u learn this?
violet hamlet
Just made music video with SD:
https://www.youtube.com/watch?v=ltbTnv53mwg
warped ridge
Had a thought.
What if I trained a LORA on an animation's keyframes before generating?
left, photoreal model. right, photoreal model with LORA trained on 1/10 keyframes from the animation.
obligatory emma watson
dark sage
Deforumation! Live prompting and more, tool for deforum / auto1111 deforum extension. Lets you do LIVE positive and negative prompting and motion control, strenght control, CFG value, Sampler step control. Everything while you watch the frames render in deforum GUI inside auto1111, You also have the ability to paus and redo frame generations from a given frame. Visit https://www.reddit.com/r/deforumation/ for installation and user instructions! Have fun, create and share!
faint rivet
Cyberpunk Flow Factor Demonstrations https://youtu.be/qBwMa0W-cMY
torn dragon
https://youtu.be/jN-2svUgo4k My first decent video after a long line of failed attempts! suggest improvements
summer dove
zinc tide
knotty rune
crisp pumice
Anyone tried any temporal consistency techniques?
I tried text2video-Zero here, but its not working
Theres no temporal cohesion (the face design is the same because of controlnet)
tacit spoke
Anyone tried any temporal consistency techniques?
The main defect is jittering : the details jump around from a frame to the next.
To fight this, multiple techniques have been explored already :
- using the same seed for all img2img pictures tend to reduce the change in details
- training a model on 1/10th of the keyframes leads to a lot more stable composition
- batching ALL FRAMES in a single batch gives a very high fidelity : the details tend to copy almost, this seems the most stable
In any case, post processing is also an option, like professional deflickering options, but this is quite costly usually.
Also, interpolating some frames to help the details morph and not jump around has seen some success
(I answered that not long ago for that same problem)
also in your case, using the "loopback" script in the img2img tab, to have the previous frame be an input of the next frame
crisp pumice
also in your case, using the "loopback" script in the img2img tab, to have the p...
Heres what im actually trying to do
Redo this
In higher res and with better consistency
Higher res is easy, itll just take a while
But the clothes change alot
And for input im working with not alot
Heres a comparison for the input vs output
So im using controlnet
tacit spoke
using the same seed is a good way to fight this for a start.
and loopback seems like your solution here.
Instead of generating frames in txt2img, you would generate in img2img, using your already established controlnet frame and the previous frame, so that most details would just be forced to morph into the controlnet, and it could reduce that full change every frame
earnest sigil
crisp pumice
using the same seed is a good way to fight this for a start.
and loopback seems ...
Im already using img2img
Already using the prev frame as input
Same seed will probably work a little, but I dont like how the same "pattern" stays even when the character moves
ie the lines of the hair stay the same even if the character moves a little to the left, those lines dont move with them
tacit spoke
ie the lines of the hair stay the same even if the character moves a little to t...
the only other new technique that I saw, but I don't get how that would work practicaly because VRAM, is to batch all pictures at the same time. details tend to copy properly then.
But yeah, models trained for video will be the next step, given all the teases we see everywhere, currently that is a hard problem to solve
warped ridge
the only other new technique that I saw, but I don't get how that would work pra...
Each "scene" in bad apple is only a few seconds, which the sheet generation approach might be able to support. It's better-suited for making keyframes for ebsynth and the like.
64 frames at 320x320, 16fps, 4 seconds. Sheet would be 2560x2560. 24GB gpu does that fine.
Pare it back more, 8fps, rely on interpolation. That's still a lot of work for each scene, and that's not considering the transitions, which are delightfully unique and each producing a unique challenge.
faint harness
fallen girder
"Dragon Slayer" | AI Generated Short Film (ModelScope text2video Synthesis) [4K] - Workflow in the video description
https://youtu.be/oh2RGgJ7ni8
soft bone
Music Video! Modelscope, Warpfusion and lots of tinkering in gimp https://www.youtube.com/watch?v=hA6tbSXrPSw
unreal bronze
I used Dreambooth to train 28 keyframes from a YouTube video(1600 frames), and then I used Stable Diffusion web UI to convert the video into an anime-style one. All the operations were performed on Stable Diffusion web UI, and no other software was used.
https://youtu.be/e6-TAtGGTLs
devout delta
This AI "fractal zoom" was made with continuous in-painting with stable diffusion.
https://youtu.be/UQyoK-je2eM
faint rivet
summer dove
burnt glen
faint rivet
icy portal
summer dove
earnest sigil
echo minnow
Alô
whole fractal
young cave
Dancing
summer dove
faint harness
earnest sigil
torn dragon
Can anyone help me? I am getting this error-> Keyerror:None
faint rivet
humble dome
Life on Europa. Took quite a time to make. An attempt of making semi-realistic video from the future. Probably very far future.
https://www.youtube.com/shorts/MtJAqzwGQ50
faint rivet
oblique ivy
Made with Modelscope AI:
grand copper
Shenhe Mommy Animation
dull bison
hi
marsh vortex
Hi! I started a YouTube channel about animation: https://youtu.be/Gj9TU_ZuJ54
unkempt imp
Hi! I started a YouTube channel about animation: https://youtu.be/Gj9TU_ZuJ54
for me, i absolutely can't watch something with a robotic voice like that
even the content is interesting
i quit at the first second
covert gorge
A colorful watercolour underwater featuring various types of marine life such as fish, sea turtles, dolphins, and whales, white background --ar 2:3 --q 2
marsh vortex
for me, i absolutely can't watch something with a robotic voice like that
Thank you for the advice! Next video will be with my funny voice 🙂
unkempt imp
Thank you for the advice! Next video will be with my funny voice 🙂
hey its just me amd my own opinion i don't say the universal truth, wait for other comment
grand copper
Hi! I started a YouTube channel about animation: https://youtu.be/Gj9TU_ZuJ54
I tried your techniques extensively and man i love ebsynth, Though of course if suck at wide range of movemets like most ai animation now a days do but with the stuff out there right now, i think ebsynth is still kicking butt with the non flickering part of it
Kisaki Pose Session 🔥 (also figured out how to do this which is fun hehe)
faint rivet
earnest sigil
mossy ridge
Very nice~
faint rivet
Rendered this epic one overnight at 1024p at 12fps. Used FILM to bring to 24fps (tested RIFE, but it was more wiggly for this). Upsampled to 2160p in smooth mode in Vegas Pro, and unsharp mask. Uses Optical flow redo. I noticed that optical flow redo tends to do nice things with geometric stuff, and anything with symmetries, because the flow is informed by the render, so SD kind of gets to have a feedback with itself on the movement of things. So, I had been wanting to try something with a lot of geometry, and it did not dissapoint!
So please, just 🌀 𝐀𝐬𝐤 𝐃𝐨𝐜𝐭𝐨𝐫 𝐌𝐞𝐭𝐚𝐭𝐫𝐨𝐧❕
https://www.youtube.com/watch?v=-7nS8El8xqQ
whole frost
faint rivet
https://youtu.be/Tuw6hTJneOU
Hybrid video in Deforum?
quasi spire
Hybrid video in Deforum?
nope, just img2img batch with canny controlnet
faint rivet
right on
faint rivet
I made a Color force grayscale mode for deforum, and I thought you were using that.
faint rivet
yeah, but SD is color, and sometimes it turns things slightly color on output, depending on settings.
quasi spire
yeah, true
it will tend to do that if you let it guess mode in controlnet
i dont let it
I like to let style and the source video do it's thing. Trying to learn what works and what doesn't https://youtu.be/BeVAqT1nDVM
faint rivet
My forcing of grayscale in Deforum happens pre-render and post-render. Because, deforum is always feeding image back through as prev_img (init image). So, that way it gets forced on input to SD (so it's hallucinating in B&W as much as possible) and on output, in case it introduced any color.
quasi spire
gotcha. It's always nice to have as much control as possible when you want it
faint rivet
I just try to allow the inner Sith Lord to get out. https://www.youtube.com/watch?v=dasNNW3IIbI
quasi spire
niiiice
faint rivet
It's supposed to be lame and funny... and it's not Gen-1. lol
faint rivet
I made Hobo With a Shotgun trailer into Legos.
https://www.youtube.com/watch?v=ciDnQ3N_PQw
quasi spire
fuck. yes.
this rocks
finally, a person of culture
I still have yet to watch this
i need to put it on my to do list, hobo with a shotgun
faint rivet
finally, a person of culture
I actually haven't watched it. I keep meaning to!
You'd probably like this one if you're into altering video.
https://www.youtube.com/watch?v=ZXsuwbJhpTQ
quasi spire
classic
i am. I can't wait for an animation model to come out... https://youtu.be/r-2r6UtcUbw
quasi spire
I actually haven't watched it. I keep meaning to!
You'd probably like this one i...
nice. nousr model or some other android lora?
faint rivet
i am. I can't wait for an animation model to come out... https://youtu.be/r-2r6U...
Well, for now we have ModelScope. lol
https://www.youtube.com/watch?v=wGs3k-7hu4c
faint rivet
nice. nousr model or some other android lora?
Instruct_pix2pix
faint rivet
modelscope?
it's an auto1111 extension that does cheesy little text2video things
quasi spire
they have an extention for auto to run local?
faint rivet
I run auto local
quasi spire
what model do you run the aniamtions on, any?
faint rivet
I'm the developer of hybrid video for deforum, and I mainly use the deforum webui extension for auto1111
quasi spire
is this the one? https://github.com/deforum-art/sd-webui-modelscope-text2video
faint rivet
there's just the one model for now, and it's trained on shutterstock... and everything had the watermark no matter what you do! It's funny.
faint rivet
is this the one? https://github.com/deforum-art/sd-webui-modelscope-text2video
yeah, you can install it straight from the extension tab if you have auto running
quasi spire
NIce, will bookmark to do later
faint rivet
memory hog though... can't do much without more GPU ram
quasi spire
sad face, only have 8gb
faint rivet
I think it's set up so it will barely work maybe on that... I forget
I have 24gb 😉
it might take a little tiny bit more
quasi spire
I will hold off then until something streamlined comes along for 8gb. Figure I'll have to wait, what, like a month?
faint rivet
I used to have a 3060 with 12GB... worked nice. $310 reconditioned
faint rivet
modelscope sucks, but it's funny
faint rivet
NOT Stable Diffusion
https://www.youtube.com/watch?v=fZQVEm_1AN0
quasi spire
pretty decent for a start
i like when rick steals the food...
course, it's not animatd 😦
i have a BUNCH like that for backrooms
and a xfiles
humble dome
I fucking did it boys.. https://twitter.com/volotat/status/1643444048286318596
Coherent video2video with strong denoising strength. It still not perfect, there are some ghosting and occasional flickering, but you can't imagine how much I spend experimenting with different approaches to achieve this result. This time I do like it. Gonna clean the code today and update the repository.
Btw, each frame has its own random seed. That why style slightly shifted over time.
quasi spire
I fucking did it boys.. https://twitter.com/volotat/status/1643444048286318596
C...
very nice. Now run that through flowframes!
it will smooth it over just a smidge deducing the flicker even more
i used it here, exporting at 6fps and using flowframes to get back up to 24fps #🎥｜animation message
to render 1/4 the frames
humble dome
It does not need it, actually. It should work even better with high fps, but I did not tried this yet. The main insight in this method is to incorparate optical flow into the pipeline.
quasi spire
totally unfamiliar with "optical flow"
humble dome
i used it here, exporting at 6fps and using flowframes to get back up to 24fps h...
Yeah, it does look good too.
quasi spire
you vid is very temporally sound though. Almost a claymation feel
would love to hear the workflow!
I'm just using style to force sstability
humble dome
you vid is very temporally sound though. Almost a claymation feel
Yeah, it is exactly what I tried to do, Matrix as clay animation. The whole "workflow" incorporated into the script, you just feed it the video, set the prompt and it spits animation frame by frame. It's already opensourced here https://github.com/volotat/Reference-based-SD-CN-Animation but I need to update the code.
quasi spire
i havn't really used deform at all
quasi spire
Yeah, it is exactly what I tried to do, Matrix as clay animation. The whole "wor...
yeah, i might not have this script, but the one i do have wouldnt run, so i just used controlnet without a script and just try to hone in the high and low setting
found that out the hard way with te Hermione/ firestarte video when it's too loose it looks a little too arthouse
quasi spire
Yeah, it is exactly what I tried to do, Matrix as clay animation. The whole "wor...
the real fun is taking something like the matrix there and swapping out the characters for someone else. Or really letting loose with the style
even if it was keanu matrix swapped with john wick or ted Theodor Logan
or johnny silverhand, that would fit
johnny silverhand invades the matrix
humble dome
It should be doable. I specifically chose this scene as it is full of action and usual methods cannot handle it at all. With the last iteration of the script it looks like it could handle even action sequences. The video is still processing though.
quasi spire
sounds aweosme, will have to give it a go
quasi spire
It should be doable. I specifically chose this scene as it is full of action and...
what does --api do?
humble dome
web-ui has an internal API but it is turned off by default, --api activates it.
quasi spire
web-ui has an internal API but it is turned off by default, --api activates it.
awesome. I ran the pip install opencv-python numpy tqdm, put the --api, will take a shower and then take a peek. I then look for python 3 script when i'm ready to batch with controlnet?
humble dome
awesome. I ran the pip install opencv-python numpy tqdm, put the --api, will tak...
Sorry, I can’t understand your question 😅
quasi spire
Sorry, I can’t understand your question 😅
I thought I installed it all correctly but I dont s a "script" script in the scripts section.
chrome grove
what is a good tool to animated warps between two images? I only found Stable Warp Fusion
dark sage
Flying out a window, using live prompting, live camera movement.
quasi spire
Here's my newest attempt https://youtu.be/JXU-L0SPNB4
dark sage
dark sage
i like the smile change mid clip
Thanks, i change the prompt live. amongst other settings
dark sage
humble dome
the real fun is taking something like the matrix there and swapping out the char...
Yeap. It works. And it works really well. The script is updated by the way https://github.com/volotat/SD-CN-Animation
Making everyone in the Matrix Will Smith:
quasi spire
Yeap. It works. And it works really well. The script is updated by the way https...
alright, I just ran the git clone again and the dependency, just in case i did it wrong last night. Will give it a try later for sure!! tyvm!
humble dome
The meme potential of this approach is yet to be explored...
https://youtube.com/shorts/be93_dIeZWU?feature=share
faint rivet
gilded bane
Octopus stealing a motorcycle
safe oriole
oh no poor octo
misty dagger
that is..... profile of a woman and woman facing eachother, kissing, in the style of craig mullins and ruan jia and emil melmoth, romantic
also negative embedding is nfixer
quasi spire
The meme potential of this approach is yet to be explored...
https://youtube.com...
i'm really new at all this stuff... can you help me install the script? I must be doing something wrong. Is it installed in the root or do I need to do it from activated vae folder? Once it's installed, will it be in the img2img script menu?
humble dome
i'm really new at all this stuff... can you help me install the script? I must b...
No, this script does not have UI integration. You have to run it from CLI
quasi spire
No, this script does not have UI integration. You have to run it from CLI
ah, this explains. ty. i have no clue.
so, i edit it to my project setting and run it without launching the web gui?
just open cmd from the root?
a tut video would go a long way!
misty dagger
Ya sure
humble dome
so, i edit it to my project setting and run it without launching the web gui?
No, you first run web-ui with —api flag, and then run the script from cli. Just make sure that the video you process and the script in the same folder.
quasi spire
No, you first run web-ui with —api flag, and then run the script from cli. Just ...
put the script in the same folder?? You really need to make a tutorial, I didnt see that listed. Also, do I install this from the root or the activated venv?
i guess the root if im placing that script with the processed frames
humble dome
put the script in the same folder?? You really need to make a tutorial, I didnt ...
I would like to, but my kids scream all the time, so no luck to make any videos with the voice in it 😂
quasi spire
also, i guess i have no clue how to just "run from the client"
doesn't need voice if you show it being done
humble dome
Yeah, maybe you right. I will consider it, but cannot promise just yet. Basically “running the script” comes down to putting “python3 script.py” in the command line and pressing enter
quasi spire
Yeah, maybe you right. I will consider it, but cannot promise just yet. Basicall...
do i open a new cmd? cause i cant seem to type in the one that is launched with auto1111
stable hawk
Text to Image. Music's made with Riffusion.
stable hawk
what is a good tool to animated warps between two images? I only found Stable Wa...
FILM from the Google research team is the best I've come across atm.
faint rivet
faint rivet
FILM from the Google research team is the best I've come across atm.
FILM is available in Deforum webui extension for AUTO1111. You can either do it at the end of an animation, or you can do it to a file as a standalone! Actually, I used FILM x2 on the video directly above.
faint rivet
what is a good tool to animated warps between two images? I only found Stable Wa...
That's not really what WarpFusion is for. lol
stable hawk
FILM is available in Deforum webui extension for AUTO1111. You can either do it...
That makes sense. I didn't even think to check how Deforum handled it 😅.
faint rivet
It's a fairly new addition. We have RIFE and FILM... There are also Upscalers, esrgan, etc
I called that Flower of RIFE... but, I actually used FILM. lol
stable hawk
Nice! Any plans on using DAIN?
faint rivet
But yeah, FILM is generally for more than 2X
I think RIFE is generally accepted as the newer better DAIN
stable hawk
Ahhh!!
faint rivet
But, I used to use DAIN App
You can do up to 20x RIFE... You can do up to 200x FILM!!!
stable hawk
That's, frigging awesome!
faint rivet
I mean, it's not generally recommended, but we made the slider go that high... FILM is better for long slow interpolations
stable hawk
Yeah, but I didn't figure out how to get voices out of Riffusion music by tiling on the Y access and using inpainting by doing what was recommended 😅.
faint rivet
I still gotta try that
chrome grove
That's not really what WarpFusion is for. lol
Had no idea, I only dipped my pinky toe into SD two weeks ago. How would you describe Warp Fusions utility?
I'm a motion designer and looking for ways to incorporate all these new tools into my workflow
faint rivet
I'm a motion designer and looking for ways to incorporate all these new tools in...
Traditional editors can warp between images... That's just not what Stable Diffusion does.
humble dome
do i open a new cmd? cause i cant seem to type in the one that is launched with ...
No you can type there. Yes, You need to open new comand line from the folder you are working with (there should be an option in context menu "Open in comand line")
charred sigil
Made with SD
https://www.youtube.com/watch?v=hByp-zi23C4
noble quail
swift sundial
rancid fossil
start very clear but middle and ending seems filled random things
its deforum stock prompt
This is the result few days ago but today ended like this
fallen girder
"Assault Squad" | Deforum Animation (Stable Diffusion) [4K] - Workflow in the video description
https://youtu.be/tXh-jlq1RnI
warped ridge
true belfry
https://vm.tiktok.com/ZMYpSKaG3/
Hope you enjoy I'm proud of this one ^^ ❤️ !
tawny sable
surreal portal
I have been working on a music video API / app. All prompting is done via GPT from track metadata / lyrics extracted with whisper. Render uses a combination of SD and RIFE and I can get 30s of, er 240p res video in 6-7 minutes. Results are pretty reliably good and the GPT video director even produces director's notes
earnest sigil
quasi spire
I can't believe MrBeast would do THIS https://youtu.be/BLaIPW-Tw1U
earnest sigil
rugged prism
Why this happening? Is it cause the wrong setting?
your model is 768 native
and this is 512-512
up the rez
trail agate
https://www.youtube.com/shorts/Gr7d6iKdoe4 short video
surreal portal
Magma's SD use for possibly their first 'music video' is rather good. Some collaboration with dancers from the Paris opera and professional VFX involved. Particularly amusing as they are not exactly zoomers, have been playing and composing since the late 60s. Remarkable and unique artists using a novel medium, I love them even more now
They are hard for my stuff as they sing in a constructed alien language (not kidding)
marsh hedge
Hi guys. I'm making an animation with deforum using this video as control net, but seems like deforum is not processing my images with control net... Have anyone had this issue?
humble dome
I found out how to fix the blurring issue in 'SD-CN-Animation'. It generates much better quality animations now
https://youtube.com/shorts/1zf04AfXFwE
dire laurel
I found out how to fix the blurring issue in 'SD-CN-Animation'. It generates muc...
That's quite cool! Do you have the original video you're converting from to compare to?
I've been experimenting with video and trying to get things stable. I'm wondering if I could use your script to get a stable more stylized / animated look
Here's a video I finished a few days ago. I ended up forgoing most of the controlnet approach and focusing on seed and embedding interpolation: https://www.youtube.com/watch?v=IU8wHxJhuW4
humble dome
That's quite cool! Do you have the original video you're converting from to comp...
Here it is. There is a glips of it in the video itself.
dire laurel
lol yours honestly looks better
that's crazy
I guess the mouth movement in the original is better, but yours looks somehow less janky to me in general
oh I see how you overlaid the original
humble dome
I've been experimenting with video and trying to get things stable. I'm wonderin...
You could use for stylization for sure. I have an example, but this is an older version of the script, it should work even better now. https://www.youtube.com/watch?v=j-0niEMm6DU&t=33s
Basically this script is an open source alternative to a Gen1. Maybe even better, as you can generate videos in any resolution as long as your GPU can handle it. And in unlimited length.
dire laurel
I'll give it a shot, thanks
the holy grail of what I was originally hoping was possible was to feed in only controlnet images without having an original source frame. Like essentially line animated drawings and letting SD imagine and animate details. I was hoping using img2img with the basis for the next frame being the previous frame could work, but I ended up getting increasingly contrasty / blown out images on followup frames
whole fractal
uncut spear
Cat#
acoustic gulch
how do you animate a photo, do any of you know the basics?
earnest sigil
dark sage
whole fractal
mental pecan
New little YT short drop of my buddy popping.. didnt spend much time since im in the middle of some paid gigs right now.. https://www.youtube.com/watch?v=gp4ERY5E1q0
echo kite
Very happy with the temporal stability I was achieving today.
inner crest
https://youtu.be/8xizO2GbF9U
Kind of thing I try to understand with parseq 😂
dark sage
The Aiwakening https://youtu.be/VLggWlV0fmY
cunning temple
A video I made for some cool Psy chill track https://youtu.be/Et3Y2hpmUGQ
faint rivet
mild geyser
How do I change the Denoise in Deforum when using a video as an input? It's stuck at 0.4
echo kite
Still have a lot to work out but I'm really happy with the stability of the frames.
rough scaffold
How can i create something?
faint rivet
How do I change the Denoise in Deforum when using a video as an input? It's stuc...
It's called Strength in Deforum, but Deforum displays the values being sent to Stable Diffusion, which is Denoising Strength, and that's opposite of Strength. So, your strength is set at 0.6, which is why you see 0.4 Denoise.
mild geyser
Thanks!
faint rivet
Also, steps in Deforum work differently, but if you're using Video Input mode, they work basically the same as img2img steps. But, in 2D/3D animation mode, using hybrid video, the steps are proportional to the strength on all but the first frame. So, most frames are like steps * strength to figure out how many actual steps it will take. 100 steps at strength 0.9 only takes 10 actual steps each frame (besides the first frame).
in animation mode, each frame is informed by the previous frame before it, and the strength is also a measure of how much it conforms to that last frame. In Video Input mode, strength ONLY is a measure of how much it conforms to the video frame, with no temporal connection between frames.
hybrid comp alpha can also help add temporal coherence between frames by mixing the video in at a lower level than full (default is 0.5), so there's some of the previous frame mixed into the new frame before it goes in for the render.
That thing I did above - you can't do that with Video Input mode. With controlnet, you could get close, but not like that.
This was done with hybrid video and controlnet
gilded bane
whole fractal
inner crest
How to explain ai art to your friends:
distant apex
the majestic eagle, the national bird of the United States. hyper realastic
vapid skiffBOT
@distant apex
vital yacht
Trilogy "Baal Resurgence" (parts 1-3). 100% Deforum 2D Animation (A1111 extension version 2.3b)§
Model: Realistic Vision 2.0 | Sampler: DPM++ SDE Karras
🔴 soft nsfw
Part 1: https://www.instagram.com/p/CqlcJV3OimO/
Part 2: https://www.instagram.com/p/Cqu_S_lLCWY/
part 3: https://www.instagram.com/p/Cq6E-9bpU0r/
plucky plover
warning: cursed ❌ 
https://www.youtube.com/watch?v=q2-YqBO0rkU
sharp depot
warning: cursed ❌ <:waluichad:1003413227838914590> <:smugshroom:1064257380453470...
Please stop making such mainstream music.
plucky plover
Please stop making such mainstream music.
Thanks for your input! I enjoy what I do & I'll keep at it🥳
restive silo
warning: cursed ❌ <:waluichad:1003413227838914590> <:smugshroom:1064257380453470...
Love it!
rocky matrix
check out his awesome AI animated video clip! https://www.youtube.com/watch?v=wnMvDjWvC9o&t=206s
inner crest
Parseq + ControlNet
primal reef
safe oriole
Parseq + ControlNet
how long did it take for you to learn to parseq this? its a great format
inner crest
how long did it take for you to learn to parseq this? its a great format
maybe few weeks. There are good tutorials
and a channel on Deforum server