🎥｜animation

[oak urchin]

ahah, so large that it create 2 gates

[safe oriole]

hehe yea, often >1k get duplicate objects 🙂

[oak urchin]

The stops are here to mark the start of a prompt

smoother version would be

don't forget the anti-blur, or not too much steps when not moving

seems blurry

[safe oriole]

heres anti blur strength 2 and threshold 0.5

might try up threshold more, see if it sharpens , unsure its range of use

heres threshold 2 anti blur 2

maybe another setting blurring it too much , i dunno 😄

[oak urchin]

1 is already high i think.
Hum, happen to me when i put too much steps when not moving

not sure why else

[safe oriole]

think its my changed strength schedule, going back to pure .65 instead of a curve

wonder what sort of y transition to go through the door instead of straight through, be pretty sharp 😄

[hazy parrot]

hello guys. is there someone can use sd rn?

I just need upscaling an image,

I can't use sd rn

It need updating

I hope you don't mind.

• it looks everyone gone -

[stable hawk]

Muah haa haa haa! Stokes' Theorem gave me inspiration for the water like flow and the ripples in the water at the end lol. Ripple affect actually worked, flowing water affect needs slight adjustments on initial FPS.

[woven cipher]

I'm still relatively new, but I think parseq is something to strive towards learning. The author has several really good youtube tutorials, here is the first one:
https://www.youtube.com/watch?v=MXRjTOE2v64

If that's too advanced at the moment, the fundamentals that I suggest learning first:
In automatic1111, make sure you understand what is going in the txt2img tab and that you can get a very rough concept of what you are looking to create.

Then work on understanding what is happening in img2img tab, and understand how a source image can influence your final result. This takes a little more research to figure out. For your source image you can use any image you find off the internet. Another good option is to use the 'openpose-editor' extension. You can create a generic pose of a character, and then send that to either txt2img or img2img to see how that influences your final result. With all of these, you will want the 'sd-webui-controlnet' extension, and understand what ControlNet provides.

Hopefully once you have those fundamentals down, then parseq (which at a minimum is a frontend to deforum), will help you in creating videos.

[fallen girder]

They Came From Below (Infinite Zoom extension)
https://youtu.be/QXcjCzGeeoY

[safe oriole]

dang what got the blurryness down to 0 here? very crispy

[fallen girder]

It's a SD v2.1 model rendered at 1408x768 resolution and upscaled x3

[hazy parrot]

hello guys, is there a free way to remove flicker s?

I can't crack programs

[fallen girder]

https://github.com/AbyszOne/Abysz-LAB-Ext check this extension

[hazy parrot]

thank you

@fallen girder how to make png to mp4 ?

If I make it gif, it losses its quality\

[fallen girder]

You can do it in Deforum, in the "Outputs" tab you have the "Frames to Video" option, or using a video editor like Adobe Premiere

[hazy parrot]

is there a tutorial ?

about deforum

[fallen girder]

https://discord.com/channels/1011314943380959313/1087960316769472582

[hazy parrot]

@fallen girder

[stable hawk]

Getting motion and fluids down, slowly but surely lol.

[lyric niche]

reference-only + tile test

[vital yacht]

100% AI Deforum animation - "BEACH PARTY"
https://youtu.be/AiLwHqroaJo

[stable hawk]

water lol. No references, straight text to image.

[near badger]

This is a modified loopbackwave extension.
You can use it if you like.
https://github.com/s9roll7/sd_loopback_music_sync_wave

[leaden turtle]

awesome work! openpose and lineart should fix some of the character deformation and the consistency is great

[lyric niche]

Tile is causing the deformation. I'll try the lineart+openpose combo. Thanks for the suggestion 🙏

One of the problems I find with openpose is it doesn't do a perfect job when driven by real world video. It gets confused and lack temporal consistency, which is too bad because as far as style does, it does the best.

Here is an example I made with openpose and some custom cross-frame attention. The problem is the openpose annotations pop all over the place. #🎥｜animation message

[leaden turtle]

I agree openpose can be finicky when paired with real world video. However I was able to get some decent results with multiple subjects when pairing openpose with lineart and tile.

The results are still a little rough but decently consistent considering how much is going on

And I’m sure you could get even better results by playing around with the settings. However all those controlnet models make processing really slow lol

[fringe carbon]

Your knowledge of physics and the way you've applied it is absolutely amazing. Any chance you could the music vid for this AI-tune I'm working on?
Hopefully it's going to be one of the first nice AI-tunes.

[ripe plaza]

This is cool

[safe oriole]

thanks for the lineart openpose suggestion, whats the combo with tiles? this is lineart openpose

ooo tile_resample depth with lineart openpose is neat effect

[leaden turtle]

Great results! Are you running the frames through Ebsynth?

[leaden turtle]

Yeah I usually put the weight for lineart at .45 and tile at .65 as a good compromise between consistency and quality

[leaden turtle]

Thank you 🙏

[safe oriole]

so is it lineart openpose (model) and tile openpose(model) ? the above is purely deforum with controlnet

[lyric niche]

Tried openpose but got better result with lineart and tile. Thanks for the suggest btw:

[stoic cape]

SD MAKES VIDS ?

[ripe plaza]

Hybrid video + CN p2p model.
https://www.instagram.com/reel/CsS8dbMghsK/?utm_source=ig_web_copy_link&igshid=MzRlODBiNWFlZA==

[tropic lichen]

Is the original video available somewhere for comparison ?

[stable hawk]

And, a long video of a cats surfing made with text to image lol. More water testing lol.

[humble dome]

I'm surprised how well it turned out
https://www.youtube.com/shorts/SgDv6TbYfcw

[safe oriole]

dang your maxwell equations worked great , or did you move to other equations

[stable hawk]

Stokes' equations, the curl in comparison to the vector seems to help with the flow. And it's never a true conversion of their equations. I try to bring the principles over to deforum math and it's equations. So it's more like, if you went to the dollar store to buy physics equations, that's where you'd find my equation conversions 😅.

[stable hawk]

Maxwells did lead to the steam coming from the Pizza, so it had value, just not the water effect lol. I'm working on incorporating quasi fractal patterns now to see if I can make it a repeating cycle with variation so it feels fresh and unique throughout

The current walk cycle I have is great but it repeats in a static way. I want to add those little differences that make it feel organic. But still have it a repeating pattern lol.

[safe oriole]

you sure are exploring the latent space from a unique angle 🙂 its cool

[stable hawk]

Thanks 😅, I'm glad you think it's cool. I definitely ended up on a unique road with my latent bubbles, equations, prompt style, and etc 😅. I'm still trying to figure out how to explain painting spectrograms with inpainting and outpainting to people that can't see the patterns and how they connect 😅.

How did you make that song? Me: Oh, I painted it!

🤣😂

[safe oriole]

synthesesia

[stable hawk]

That's actually pretty probable. 🤔

[stable hawk]

Nooo Maholic! You've changes the trajectory of today's projects 😅😂🤣. Today was decidedly a no music and psychology day. Now it's all I want to do! 😅

[stable hawk]

This extension basically takes the text to image animation approach I've been using and goes about it a little different and then expands on it in ways I didn't think about. Definitely check this one out. No equations, but it seems to have some of the movement principles baked in so you don't have to worry about it. https://github.com/volotat/SD-CN-Animation

[cloud smelt]

Stable Diffusion+ControlNet
https://youtube.com/shorts/1CEDk9XGCbk?feature=share

https://youtube.com/shorts/X_tzddrD320?feature=share

https://youtube.com/shorts/IupcYnVU1_g?feature=share

https://youtube.com/shorts/UqkUCCegpis?feature=share

Stable Diffusion + Deforum
https://youtube.com/shorts/8tZ6rUoNFLw?feature=share

Stable Diffusion + Deforum
https://youtube.com/shorts/JhI-GYIMhCM?feature=share

[stable hawk]

Called it 420 frames. Was listening to music and working on Synthesia research I ended up making this, decided the theme of the video would be the frame count. 420 frames lol. It's more than that after running it through film and using slowmo, but, it did start with 420. So it stayed true to the concept lol. Text to image using the SD-CN- Animation extension.

[neon coral]

Deforum + Parseq https://www.youtube.com/watch?v=sZ_UP07q7W8

[leaden turtle]

No problem, the results look great 👍

[leaden turtle]

Im using lineart realistic, openpose full, and tile_resample. Sorry if im misunderstanding your question

[leaden turtle]

https://youtu.be/B5SdTdYq4Y0

[livid wasp]

https://www.youtube.com/shorts/XSKHZ0TqxMk

[lyric niche]

Oh I didn't realize you were using all three. I'll might try that later

[stable hawk]

Like, I don't know if this is of interest or not but a lot of my animation equations for Deforum come from re-interpreting principles and equations from physics into Deforum math. If you're interested I just found Khan Academy has free physics classes. https://www.khanacademy.org/science/physics/ It's only loosely related to animations lol, but I mean, free physics classes is pretty awesome.

[main saddle]

I wrote a custom python script using open AI completions for multi-prompting. It's a little glitchy still on the animations results. Has anyone had any success in "Write a story" which is story based text prompts parsed into parameters for frames?

[misty trail]

How do i generate a animation

[stable hawk]

Another SD-CN Animation lol. I love this thing lol. Oh, I did run it through slowmo and film.

[stable hawk]

How would you like to?

[terse shell]

Anyone know how to get video like this?

https://www.instagram.com/reel/CsOsJcRIZOC/?igshid=MzRlODBiNWFlZA==

I have some videos which I wanna make look like this, I have stable diffusion but i'm sure you need a bunch of plugins and my version is probably outdated by now so any help would be awesome!

[safe oriole]

testing out SD-CN-animation

[stable hawk]

Very cool. Dude put some work into this extension. I was talking to him a little early. He's got some cool stuff planned. I'm looking forward to seeing what he does.

[pastel obsidian]

anyone know what this means? i'm trying to install sd animation

[knotty gale]

Do u know if there’s a discord for the extension 🤔 Good results so far

[fallen girder]

"Black Rainbows" (Stable Diffusion v2.1 / Infinite Zoom)
https://youtu.be/EPoH2RL9DrY

[stable hawk]

I'm actually not sure, I hit them up on github in the discussion section.

[snow ingot]

https://www.youtube.com/watch?v=T9m3MpZ7_kE

[fringe carbon]

Re: the offer to make us a music video, we are a big budget startup who's gonna be the next Twitter, featuring many of Scandinavia and Southern USA's best musicians. You would get paid a great deal.

My childhood friend and now bitter rival makes real music videos with real people. Whatever AI manages to come up with, it has to be better than https://www.youtube.com/watch?v=X2ynsC-8owU etc.

[stable hawk]

You have no idea how much I wish money was my motivation 😅. You can message me. Let me know about you guys, your mission, your goals. What you want to do with your music. I'll get back to you asap. Just full disclosure, I can't work on something I don't want to work on. I'm very clearly AuADHD even though self diagnosed. If you come at me with noble intentions and cool concepts, I'm liable to do it for free and have it done by the end of the conversation 😅.

[mossy ridge]

hello there, not sure if this is the right place to post it, just made myself a music clip cover using stable diffusion with the model openjourney-v4, enjoy :)

https://www.youtube.com/watch?v=mxpdhtKzCXs&list=PLyu5LukP2mChkTuvMixMwf0NMb4I7J9JJ&index=1

[fringe carbon]

🤣 Cheers mate! I'll be in touch later today. ADHD here too. The entire crew is!

[stable hawk]

Awesome lol

[fringe carbon]

Posted this in Harmonai just now, added a little extra analog for good measure 👍

[stable hawk]

I like how it starts off sounding like a record. Awesome work. I'll hop down to HarmonAI in a bit.

[fringe carbon]

Thanks a lot. Took a long time to figure out how to get it like that.

Hero of mine: HEAT: Dave Hill and the Art of Analog Emulation - https://youtube.com/watch?v=SyqPZcFRkRY

[stable hawk]

I put a little to much spin on the end lol. Almost able to make short cartoons now lol. text to vid with Deforum and some equations. Nothing else. Well and Rife.

[stable hawk]

Better equation lol. Now it kind of works. Now I just need to work on prompt structure and I should be able to make quasi cartoons lol.

[hazy parrot]

hello how to remove flickers from ai animation ?

[stable hawk]

Another text to vid animation test. Deforum same equations as above. Playing with perspective theta instead of angle. No ints. Math and Prompt testing.

[fringe carbon]

Neat!

Hey, I wanted to ask you that

With my lomography script https://gist.github.com/basicfeatures/a083eb1ac1602f583d4993c3aca65358 one can get the exact same results.

But I just uncovered VSCO 01 for Lr 4 & 5.rar through to VSCO 07 containing all the color curves of all the great old analog cams.

Should I just mass-convert it all over to Ruby so it's easily accessible to us?

Camera profiles and develop presetse for Canon, Fuji, Leica, Nikon, Olympus etc.
Doesn't sound too legal though.

But damn can they do wonders to regular bland photos.

[ripe plaza]

Hybrid + Tile CN, Strength 0.
https://www.instagram.com/reel/CsYJgT4gFRU/?utm_source=ig_web_copy_link&igshid=MzRlODBiNWFlZA==

[proud mantle]

I got a little bit further than this but the Webui does not respond it loads but then nothing appears

[upper kayak]

I tried using this one but for some reason can't work for me

does anyone possibly know how to deal with this kind of error?

[stable hawk]

Do you have controlnet?

[upper kayak]

yes

[stable hawk]

It looks like it's skipping the whole optic flow portion.

[stable hawk]

I'd hit up the github page issues section. I'm currently 6 notepads deep into SD's opensource UI at the moment lol.

[upper kayak]

thanks for directing me, I also can't make anything wIth 3d, it always reports some errors 😰

[stable hawk]

Ahh, sorry to hear that. Yeah a lot of this is super complicated. SD is helping us out by making their UI Open-source so people are working on making this easier. Some snags have been hit, but it's going lol.

[stable hawk]

wait, are you using the text2vid or vid2vid input area? The vid2vid one is first and should give you an error if you don't use a vid. It will accept images even though it will give an error. Let me double check.

[stable hawk]

Okay, I think both fields require a vid for input after checking but both let you upload an image. The image will throw an error. Remove it and try again with just text.

[stable hawk]

Then if you really want to use the image as an int, duplicate it a bunch and then concatenate it into a video. Use that as an int and then you can increase denoising strength maybe to get what you're after.

[stable hawk]

Oh Oh Even better. Okay, check it out right. Try to get 3 Good images of your wolf with different angles. You can probably use the character turner lora for this. Take and concatenate those 3 images into a video. Use film to slow it down and interpolate between the 3 images. This will extend your video and give you a bunch of transition images to work with. It's better than noise at this stage I think. This may work real well. I haven't tried it, I just thought of it, but it should work. I do something similar with Hakuimg extension. I layer images and blend them a bit to give guided noise basically lol.

[upper kayak]

wow, thank you for your responses. The wolf is not that important tbh - it was just a test. I would rather work with some videos I have. When I input a video, it starts processing nicely and then stops, reporting: An exception occurred while trying to process the frame: list index out of range

[upper kayak]

and I tried a bunch of different videos, different sizes, and lengths. Is there any sort of comprehensive guide on how to set everything up?

[daring ginkgo]

Tried recreating the infamous reddit post. Turned out pretty good for a first attempt

[glass grove]

Hi guys is there a you tube or tutorial for the heads up on prompting on Deforum Auto 1111 ?

[timber carbon]

This is one of my newest creations. I have played a bit with movement to create better transitions, yet it seems that this model has trouble not enlarging heads during transitions 😄

[visual flax]

i made an AI music video to go with a song i really like https://youtu.be/u3YPuEc6qo4 made with SD and Deforum

[vast vale]

Sir

[vast vale]

I’ve seen some tutorials on youtube but unable to get it properly, can you please 🙏 provide me a tutorial link so i can create something like this (but different scene or concepts)

Anyone?

Easy to understand tutorial is preferred

[daring ginkgo]

Updated version with consistent hair. Gonna try out a workflow based on SD Extension this time

I am still amazed that AI can do really well flowing hair

[snow ingot]

https://www.youtube.com/watch?v=GD-yFn_eIiQ

[vivid nacelle]

earrape

[fallen girder]

Psychedelic gore trip (IllusTrip3D + ControlNet) [4K]
https://youtu.be/Iu-LnDjgkPg

[daring ginkgo]

I think this is my best work yet

Proper background/shadow, smooth movement, and little to no flicker other than the face which had some issues with eyes

[vivid nacelle]

what

[vivid nacelle]

how u do that

img2img?

[vivid nacelle]

ok

[humble grail]

so damn smooth. yeah i'm also wondering like nep does

how did you do that

[vivid nacelle]

snapchat filter~

[charred sigil]

Hey, nice animation. How did you get such non-flickerin/non-ghosting results? Do you mind to share your settings? Thanks

and how to fix the issue? : )

[stable hawk]

My son picked out the prompts for Deforum, each line started with somewhere to go with the ai generated music lol.

[lament orchid]

can we use stable animations by generating images locally? Without using Dreamstudio credits?

[stable hawk]

Made with Deforum and Riffusion. I can make 1 minute and 21 second long clips now (audio) lol. Son picked the prompts. Halloween theme lol. Cute and Spooky.

[daring ginkgo]

@vivid nacelle @humble grail
I used this workflow: https://github.com/s9roll7/ebsynth_utility

It basically makes a masking layer to detect what area of the video frame to denoise

Then the computer makes key frames, denoise it, upscale it, then use those key frames to interpolate with ebsynth

This process does take like hours though

Masking, rendering all key frames, and upscaling all takes so long

[dusk island]

How much VRAM does the whole process use when you use ebsynth?

[humble grail]

thx for sharing!! did try ebsynth tho but not using the utility. i used it with temporal. time to grind this out

[daring ginkgo]

Not sure. I use 4090 but maybe 12gb will be enough?

Will try to monitor vrm use next time

[dusk island]

OK, 12gb ain't bad at all. I'm really conscious of my gpu which is why I've been avoiding animation/video generation all this time. Thanks!

[mystic gazelle]

Does anyone use d-id ?

[vivid nacelle]

whats a workflow

[daring ginkgo]

Just the steps I used to make the animation

It isn't just img2img batch process

[vivid nacelle]

ok

[daring ginkgo]

It involves many steps

[gritty swallow]

hi guys i have a question about controlling camera movement, angles and rotation ... i have generated a realistic figure ... can anyone give me some suggestions ?

[vivid nacelle]

how do i install the extension

[daring ginkgo]

There is a YouTube guide that details the steps

[candid salmon]

um,can you tell which youtube guide

[daring ginkgo]

https://www.youtube.com/watch?v=Qph7A3UaVds

[candid salmon]

ok ty

[daring ginkgo]

You need ControlNet, Ebsynth_utility extensions, you need ffmpeg and ebsynth files on the PC

[candid salmon]

tyty!

[timber carbon]

Does anybody have any idea why there is such glitch between images? It is as if it suddenly skips 5 frames

Not sure which setting could influence this

[stable hawk]

You're equations may have some parts where the transition in the waves aren't smooth.

What shape did you formulate?

I kind of want to do a mobious stripe equation, it's a bit daunting though lol. The math is a bit intense on that one in a latentent space deforum conversion lol.

Strip*

The problem is breaking the equations up among the different axis while simultaneously converting them to deforum math lol.

[safe oriole]

@stable hawk could break it into all iteration components of +x +y +z -y -x -z in a 2x2x2 grid? 3d grid?

[stable hawk]

Hmm, I'll give it a go.

[safe oriole]

be 8 formulas in theory , the merge line of each be the trick 😄

[stable hawk]

Yeah, that's usually what it comes down to lol.

[safe oriole]

i wish it kept the middle in focus, is that a FOV setting?

[stable hawk]

Ah, I was lazy with this one lol. I used angle instead of hopping down to perspective lol. Perspective will tend to keep in focus, angle just kinds of does it's on thing lol.

[stable hawk]

What's I was mostly hoping for was the framing lol. It goes from like a liquid to a lattice shape and back and forth.

[stable hawk]

I think I'd have to do the video, extract the frames, mask the center, then run it through as a batch to get both.

or just do the math and use perspective lol.

[stable hawk]

Theta is the same thing, better, but if you mess up the perspective equations the video gets wonky fast lol.

[stable hawk]

translation_3D_x: 0:(1 + 0.5cos(t/23.14/41))cos(t3.14/41); 41:(1 + 0.5cos(t/23.14/41))cos(t3.14/41)",

translation_3D_y: 0:(1 + 0.5cos(t/23.14/41))sin(t3.14/41); 41:(1 + 0.5cos(t/23.14/41))sin(t3.14/41)

translation_3D_z: 0:0.5sin(t/23.14/41); 41:0.5sin(t/23.14/41)

[stable hawk]

Look good?

[safe oriole]

will try it now 🙂 thank you always!

[stable hawk]

Ah wait, forgot lol. Won't work one moment lol.

[stable hawk]

translation_3D_x": "0:(1 + 0.5cos(t23.14/41))cos(t23.14/41)
translation_3D_y": "0:(1 + 0.5cos(t23.14/41))sin(t23.14/41)
translation_3D_z": "0:0.5sin(t2*3.14/41)

This one should work lol.

Deforum math has it's own rules. Math rules, python rules, Deforum rules, lol. All super specific lol.

[safe oriole]

i converted to this, hopefully its what you implied 🙂 0:(1 + 0.5cos(23.14t/41))**cos(23.14t/41) X 0:(1 + 0.5cos(23.14t/41))**sin(23.14t/41) Y 0:0.5sin(23.14*t/41) Z

oh, it purges * 😄

ill snip

[stable hawk]

Ahh lol. Yes lol

Hmm, that's probably worth remembering that it purges the * lol.

And yes that's all correct lol.

Correction, those are indeed the equations I intended. Now, as for being correct, that's a different story lol.

[stable hawk]

It looks like it's working. I think we did it. Deforum Mobius Strip lol. The video is using the same seed and prompts as before but it's vastly different due to the motion. I made a mistake as I forgot I tweaked the Anti-blur for another test and forgot because this was more interesting lol.

[safe oriole]

test run

[stable hawk]

Oh man, that's going to lead to some great animations.

[stable hawk]

Your model is awesome by the way. I love the color palette there.

[safe oriole]

cyberrealistic30beta currently

[stable hawk]

I've been using my accidental over trained model forever now lol. GeekyGhostOG lol

I read the training instructions and... instead of 10k steps as advised. I accidentally trained for 100k steps lol

Dyscalculia for the win though, because it's my best model lol

I calculated out 615 frames at 15 fps because the audio I'm using is 41 seconds long, then with cadence and etc it's taking a hot minute lol.

[stable hawk]

The anti-blur equation, bad, really bad lol. The Mobius strip equation though, that one turned out good. this will be a fun one lol.

[stable hawk]

You my friend are awesome. Probably wouldn't have done this without our interaction and your assistance. I thought it could lead to an awesome animation, so I paired it with my AI song Somewhere. The video is a Hitchhikers guide homage lol. I present Mobius and the Petunias by Geeky Ghost and Maholic! lol.

[safe oriole]

cool, its all you, i'm filled with mirror notions thou for bouncing 😄 , i have to try make ai songs so i stop relying on the free yt music for my videos 😄 think yt a bit saturated with ai videos now thou 😄

[stable hawk]

I use Riffusion inside of Automatic. I can show you how if you grab the model. I've become a master at Spectrograms lol! Like I can autograph my Spectrograms LOL! I just did this for fun. The major red part was to inpaint out some noise and add the vocals lol.

[safe oriole]

oh woops when i tested out your variables i did them in translation x y z 😄 not rotation3d x y z 😄

will be nice when we have drop down saveable formula presets to inject at frames, might be worth asking if thats on the deforum road map or even useful for others

coordinating 4d is difficult 😄

[stable hawk]

Ah, sort of. There's already a UI repo out there and I have one I'm working out with flight sticks. I keep getting distracted though. I found a way to study physics, phycology, and philosophy all at once using GPT lol. Rene Decartes and David Chamels were some good studies based on the weird hybrid study i've been doing lol. So that keeps pulling me away, then Michio and Neil DeGrasse were chatting today so I had to watch that. SD opensourced their UI so I'm trying to work on that and the HarmoniAI team has some awesome music stuff I have notepads open for. Then, well, then a ton of other projects lol. So many projects lol. That's actually not a bad thing. It is for them. Not for me lol. Then I keep having new ideas for Spectrograms, so that keeps pulling me back in. Then of course the random things like the spur of the moment Mobius Strip equation dive lol.

[safe oriole]

no time for sleep 😄 must keep the iterative journey progress! 🙂 im just testing your formula with the accidental translation instead of rotation ,i'll see what monstosity the translation to the top right does after an hour 😄

[stable hawk]

But yeah, I suggested to the SD-CN dude about preset shape equations. He has a super easy video extension and man is it primed for a shape drop down.

Pick your motion shape, it auto input the equations. It would keep his system super simple like it is, but also offer some robustness for the user.

[safe oriole]

@stable hawk another drop down could be 'movie' stages, aka typical panning, zoom in, whatnot movie motions

be great if the ai can extract an entire movies 'motions' 😄

and you just play it back in pieces that you stich together after

movie camera cloner 😄

[stable hawk]

You know, if you can code and etc. It never hurts to hop into a git hub discussion and offer to help out especially if you got some cool ideas.

Hmm

That....

[safe oriole]

i got idea, i got no programming 😄

[stable hawk]

I might actually be able to do that

Like I made my concactenator to extract and concatenate images from and to video.

Made my editing life and batch processing life so much easier lol.

I can extra beats and onsets and etc.

I don't see why I can't figure out how to do motion conversion.

something like a motion tracking built into the auto alpha systems.

[safe oriole]

oh what do you use to concactenate? i use this program 'bulk rename' bit of manual work to prep the images , once i have them all bulk renamed i get the video for controlnet

[stable hawk]

use that and have it generate the equations.

[stable hawk]

https://github.com/GeekyGhost/Concactenator

I mean, Online code with an MIT license so like, if you don't know code it might not be best to use it lol.

But that's what I made for myself to do my digital content creation stuff.

[safe oriole]

oh nice 🙂

[stable hawk]

I called it the Concactenator because of Dr. Doofenshmirtz lol. So the Concactenator makes the video and then I have the Extractenator to extract it lol.

Concatenate was so close I had to lol.

[daring ginkgo]

Keys were off so it has alot of blurring

Sad!

Tried going for some kinda futuristic looking city

[safe oriole]

thank you so much! that translation turned out like a tapestry! it looks like a tonne of depth too from the motion

got to interpolate and upscale before i put on yt thou

[stable hawk]

WOot! I'm glad. These equations are mad fun. They are like art by themselves lol.

[safe oriole]

i thought it was going to be rekt due to so much edge pulling 😄

[stable hawk]

Yeah, it's like because it goes in on the 3D plane and slightly up, it doesn't tear. It's kind of cool. Its like Stokes with the water. It just has the right motion.

[safe oriole]

yea, your one doesnt tear, you used your formula correctly 😄 i applied them incorrectly to translation xyz... 😄

[stable hawk]

Lol! I do that to sometimes lll

Accidents have lead to some cool stuff though.

I accidentally made a deforum video with my riffusion model. Decided to convert the audio anyways and pair it up so I can see the effects of the distortion and hear it at the same time.

Terrible music, great research lol.

Now I can make 1:21 long clips in a single embro cycle lol.

[safe oriole]

was going to make some car controlnet video presets from different POV for cars, then stich the lot into one CN video to use

[stable hawk]

Embryo cycle* <--- Prompt travel setting.

[stable hawk]

That could be awesome. Can't wait to see it!

[safe oriole]

fingers crossed 😄

most of my CN tends to be too much going on in the video

need to improve my workflow

[stable hawk]

Never! Hope for the best, but embrace the worst! Lol. My chess coach worked hard to instill in me that the best games are the ones we lose. Those are the games that teach us the most.

Well, something like that. I butcher quotes because I usually remember the intent more than the actual words lol.

[worthy coral]

hello

[stable hawk]

How's Rose? Do you two still talk? How's she feel about you and Sally? <--- Sorry 😅 I saw the titanic and nightmare before Christmas joke combo and had to lol. Crossover humor! 👀😅

[ripe plaza]

Hybrid Video + Tile CN with depth preprocessor + p2p CN
https://www.instagram.com/reel/CsdDLxugMMQ/?utm_source=ig_web_copy_link&igshid=MzRlODBiNWFlZA==
https://youtube.com/shorts/54AVTvmt_44?feature=share

[timber carbon]

Dude this was dope

[stable hawk]

Thanks!

[timber carbon]

Do you have any tutorial videos for movement? I could do the math myself, just not sure how to start. I previously I tried to make sort of, moving to the side, but camera turns so that the movement is counter-acted, but wasnt able to test it out completely to get the effect

[stable hawk]

Let me see if I got those somewhere. One moment

[timber carbon]

You are a treasure , man🙏

[stable hawk]

"angle": "0:(5 * sin(2 * 3.14 * t / 120))",
"zoom": "0:(1.0025+0.002sin(1.253.14*t/30))",
"translation_x": "0:(0)",
"translation_y": "0:(0)",
"translation_z": "0:(1.75)",

These should be the only ones that matter for the rotation.

You can adjust as needed.

[timber carbon]

Why is there 1.253.14?

What does the last dot do?

[stable hawk]

Hmm

I'm supposed to remember something.

[timber carbon]

Oh nvm discord ruined format

[stable hawk]

There's a reason why it looks weird. It's missing the

[timber carbon]

It was * between 1.25*3.14

[stable hawk]

Oh Oh the * gets taken out lol

Yeah sorry lol

[timber carbon]

Nah, you good, I had this issue before too😂

Thanks thought, Imma test it out

[timber carbon]

I only now realized that zooming into a picture for intro was not supposed to be with z translations, but with angle settings

😄

[stable hawk]

lol... there is a strain called Zoom Pie lol. I keep thinking about it looking at the equation lol.

[timber carbon]

Im probs not familiar with it 🤔

[stable hawk]

I know way to many random things that connect in weird ways lol.

don't mind me lol

[timber carbon]

Or zoom, shiet

[stable hawk]

I wonder how complex of a shape I can get in the motions. Like could I trace the outline of a horse in the air with the motion? How complex would that equation be?

[timber carbon]

Hmm, would you be able to keep the horse in the pictures lined up correctly with generator?

I mean

Hm, I wonder if I said that correctly

[stable hawk]

no lol, it would serve no purpose and no one would even know it was tracing the shape besides me lol.

[timber carbon]

Not sure of complex that would be, but im gonna try in the following days more on the movement settings

[stable hawk]

Movement settings are fun. Perspective flip is a great one. I'm working on a comic book animation with them but, it's rough. The math has to be a lot more precise.

[timber carbon]

Do you just draw the movements you want to express and then calculate by sin and translation values?

[stable hawk]

Sometimes, sometimes I think about it for a few days. Sometimes I use GPT to help me figure it out. Sometimes I just throw what I think seems logical at the moment and work my way there by doing short clips. Sometimes I'll do it in 3D and cheat lol! Oh! I have a horse 3D model already lol. I can cheat! lol

[timber carbon]

Yeah, thats good

Wait does blender work for trying this out as well?

Sorry for redudancy of questions, I just finally found a guy that works and knows movements😂

[stable hawk]

Should be, I use 3D printing software for measurements though lol.

I think a lot of my 3D printing knowledge and experience is translating over to the latent space. That's kind of neat.

[timber carbon]

Of course

I hope my math skills from high school and a bit of physics degree will translate as well ;D

I think getting x, y translations and transforming them into one function could help guide a bit with how they should be written 🤔 Or may be I am wrong on this one

[stable hawk]

Oh that's awesome!! Mad props to you man. You definitely got a leg up on me lol. I only study physics for fun lol. Definitely not something I could ever get a degree in lol.

[timber carbon]

Well, I kind of drop out after 1st year, but getting back to it next sept

To be sure, studying for fun usually leads to more expertize, than a degree, so don't worry

[stable hawk]

Yeah, typically I have to think about it in a 3D space. In the realms of x, y, and z. Sometimes I can do it in my head if it's a simple shape. Harder ones I need to visualize because the math gets jumbled otherwise. Blenders good, you can use GPT to write scripts for it if it doesn't help with the measurements.

It's really good at blender scripts.

[timber carbon]

I saw dudes who just walked in and walked out of uni, after which they only played video games, so you usually don't really learn anything, expect pass tests

I am gonna have to try test movements on blender and then convert them to SD format

Because rendering the animation every time takes up quite a bit of time

[stable hawk]

Yeah, if you go to the github for deforum there's a pretty in depth guide for math there. I have to keep it handy at all times lol. I forget a lot of the rules. It's almost python but not really lol. It throws me off a bit lol. Am I coding or doing math?! Who knows at the moment lol.

for Deforum math to be specific.

[timber carbon]

That's true

Okay, I gotta for now, but if you wanna keep in touch, I can send you a friend request, I'd like to keep in touch

[stable hawk]

Sent and have a good night!

[timber carbon]

Thanks, have a good time too

[stable hawk]

lol... had a weird 20 second audio clip I generated that didn't sound to bad so I made a 20 second video for it lol. Mobius strip equations ftw! I love this movement lol.

[stable hawk]

Extended version of the song with a different video. Don't use Decimals with the mobius strip motion equations. Took me 5 videos to find out this was problematic for some reason lol.

[misty trail]

Can i make animation from the hugging face stable diffusion ui i am not sure if this the right channel

[alpine acorn]

How do I create stable diffusion animations by Discord?

[stable hawk]

Uberduck for the text to rap, used one I wrote. Deforum kind of broke in the middle of making the video lol. Gonna redo it once I fix it lol.

[daring ginkgo]

Highly doubt it. Animation rendering tends to be hardware expensive

[stable hawk]

Alright, I'm calling this the final lol. If it gets much better I'm not gonna be able to stand it lol.

[stable hawk]

Tonight the equations shall be an attempt to simulate a Penrose Triangle lol. It's not really a good translation but it should give me a triangle in and out motion with a bit of cylindrical motion to go with it to try at least seem like it lol. "translation_3D_x": "0:cos(t33.14/42), 42:cos(t33.14/42)",

"translation_3D_y": "0:sin(t33.14/42), 42:sin(t33.14/42)",

"translation_3D_z": "0:0.5sin(t23.14/42), 42:0.5sin(t23.14/42)"

"translation_x": "0:cos(t33.14/42), 42:cos(t33.14/42)",

"translation_y": "0:sin(t33.14/42), 42:sin(t33.14/42)"

[stable hawk]

The 42 isn't really important 😅 I just use it a lot lol.

[stable hawk]

One last Mobius strip with a twist before the Penrose triangle attempts lol.

[dense crater]

https://www.youtube.com/watch?v=COJT6dxWZCc

[stable hawk]

The quasi Penrose triangle equation kind of gives that drunken kind of motion. Mixing it with Film gives it a wobbly type of hallucination type of aesthetic. It's pretty neat lol.

[summer dove]

My game using SD videos as background. https://www.youtube.com/watch?v=2HMeXVprohI

[stable hawk]

Playing with 2D equations in Deforum.

[timber carbon]

Any idea why model generates image like this ?

[stable hawk]

Looks like the results of an embryo base image for prompt shifting?

I don't save them, but it looks like that and it's used for the noise.

[timber carbon]

Hmm, weird, the other models worked fine, and now I got LoRa and this happens

[vivid nacelle]

if you want i can make it smoother

[stable hawk]

Ah nah lol. It was a test of some equations. I've been hitting and breaking through this wall in latent space and I'm trying to push that a bit more and more while I figure out how to utilize it lol. Predicting where the wall pops up isn't easy lol. It seems to be different for each seed.

[stable hawk]

Wait, did you mean with equation help or video editing help. The former is always yes, the latter just seems like it's needed because I'm lazy and random lol.

[vivid nacelle]

frame interpolation

https://tenor.com/view/kek-kekw-distorted-gif-19883544

this is what i meant

do you feel any difference

https://tenor.com/view/bruh-bro-really-omg-wow-gif-21835931

[dark sage]

Say no to phone videos!! Enjoy! https://youtu.be/6zZW1JPKU7U

[stable hawk]

Deforum just doing some 2D equations today.

[dense crater]

https://www.youtube.com/watch?v=ZuAE2Bnf2TA

[stable hawk]

Nice, kind of got glitch thing going on.

[safe oriole]

i like it but it stresses me out 😄

[wide flicker]

wow, awesome. Which tool do you use for this?

[rich hamlet]

I like art the most when it's uncomfortable and it hurts :)

[misty dagger]

Queen of the deep

[daring ginkgo]

I used Ebsynth Utility as workflow

Involving Controlnet, Ebsynth, FFMPEG

Basically generating key frames using masking and Canny, generating art of the key frames, color correcting them, using Ebsynth to interpolate between frames

[wide flicker]

Nice work

[daring ginkgo]

Embarking on a larger project

4050 frames, 42fps

[vast vale]

I'll be using deforum sd, i've shot a video in that i'm just seated and started meditating then close my eyes

so which model should i get for universe kind of cosmic cooool thing?

[vast vale]

yo

[daring ginkgo]

I don't use Deforum so

[grizzled lagoon]

/https://cdn.discordapp.com/attachments/1025266140445933648/1110134625843552387/ae95dc00-f7ed-11ed-b7e1-472e79a8e450.jpg :: dog

[safe oriole]

testing out cyberpunk car theme

[rough pagoda]

If you are into kpop and ai:
https://vm.tiktok.com/ZM2LeMCLv/

[fallen girder]

Aggressor (ModelScope text2video / Animov 0.1) [4K] https://youtu.be/baButXmqyV8

[faint rivet]

https://www.youtube.com/watch?v=dkkw6QfgF6k

[faint rivet]

I haven't been posting enough, so I just threw a bunch of my latest vids together!
https://www.youtube.com/watch?v=YrEw8idzcCI

[stable hawk]

Music made with Riffusion video with Deforum.

[faint rivet]

you probably have cfg scale on like 0.7 instead of 7 or something.

[timber carbon]

Nah, it was on 7

It is as if my deforum stable diffusion was not able to load the model

or somehow settings are incompatible with the model

🤷‍♂️

[faint rivet]

it would be really great if latent space was in millimeters. lol

[timber carbon]

Btw guys, does anybody know a great and easy way to test out movement prompts without having to render whole animation?

[faint rivet]

I just reworked the 3D animation depth warping functions in Deforum to allow us to use different depth models, and fixed some bugs that were always in there. But, I just had to use the existing matrix stuff. It converts keyframes into a 2x2 rotation and 2x2 translation matrix and those are sent to a torch function to do the 3D warp of the scene with depth map.

Yeah, matrix maths are tricky for me. I'm trying to figure out a way, with ChadGPT, to get 2-way conversions that are 1 to 1 between keyframes for 3D translation and rotation and the matrices. See, I developed RANSAC tracking in Deforum, which extracts a transformation matrix from video. So, I can copy the movement from video. However, the warping functions I'm using right now are separate from the warping functions that do the 3D animation. If I can get the 2-way conversion working with keys and matrices, I can import a path from video and you can fly through the scene in 3D mode (with or without compositing the video in).

[timber carbon]

I think I just read it, and understood nothing, will have to re-read it tommorow 😄

[faint rivet]

um yeah, I've never used it, but pharmapsychotic made a colab tool like that.
https://colab.research.google.com/github/pharmapsychotic/ai-notebooks/blob/main/pharmapsychotic_AnimationPreview.ipynb

[timber carbon]

Oh shit

Imma have to try

[faint rivet]

if you don't know about it, pharma maintains a great list of AI resources, including that.
https://pharmapsychotic.com/tools.html

[timber carbon]

Thanks a lot, I have whole lot do now tommorow

I was always looking for a tool like that

[faint rivet]

yeah, I've been thinking about trying to make a preview mode in deforum that would just skip all the rendering and make boxes like that

pharma made that back in DISCO Diffusion days!

in the before-time

[timber carbon]

I just today learned how to generate the same image over and over again (yeah, it feels kinda stupid that I wasn't able to figure it out it was seed)

But this, combined with testing out movement prompts

Hell, everything will be so much easier now ;D

I wasn't sure how to interpret movement settings

To make motion

But with this I might be able to teach myself to do the math on the thing

[stable hawk]

One day I'm going to try and tweak deforum so that it adds audio based on the Spectrogram data well the conversion of it anyways. I'm working that part out 😅. Light and sound both use the same data in different ways. I'm working on an appealing way of translating light and colors into frequency and amplitude and etc lol. Once I work it out, I'm going to make music videos fully inside of Deforum 👀.

[faint rivet]

I'd like to see riffusion and deforum combined in a meaningful way

[stable hawk]

Yeah, converting regular images wasn't an issue, making them sound good, that's the catch lol. I'm trying to apply some constraints with wheel of fifths and the infinite jukebox theory well the 4 chords anyways lol.

[stable hawk]

Making it conform to a tune that sounds decent based on the first translated sound basically lol.

I've gotten in pretty deep with the Spectros.

[faint rivet]

Hey, everything gets done one step at a time. I'm the creator of hybrid video compositing in deforum, and a bunch of other things. At the beginning of the year, I was still modifying the notebook and I had started programming in Python for real around like August or September. I didn't have auto1111 or anything, and I developed it all one step at a time. Soon after that I made hybrid video work in webui, and now I'm just developing for webui, and have done a bunch of stuff.

basically, I learned python so that I could accomplish my goals, not the other way around. I have a history of programming, but had never even touched python until last year using colabs with DISCO and Illustrip. Now, I'm running local with a 4090 24GB. LOL... things change

[stable hawk]

Same 😅 I was doing Java and studying software engineering for a completely different path. Now I'm all in on Python. I love python now though. It's what I hoped Java was going to be lol.

[faint rivet]

yeah, I wasn't sure I'd like it, but I find a bunch of the syntax really cool actually... it was hard for me to break myself of ending lines with semicolons though. lol! (a lot of javascript in my past)

I'm 49 and I've seen languages change so much. I try not to get attached to any one language, and just retain the concepts that apply to most languages.

my first programs were in BASIC on an Atari 400, and I stored them on a cassette tape! So, I'm just stoked every day...

[faint rivet]

oh, if you try to render too small with a 768 model, you can get stuff like that...

[timber carbon]

Oh so you guys are also programmers, its kinda neat, looking from my perspective. You have internal knowledge of how this actually works😅

[timber carbon]

Hmm render too small with a 768? What would be solution to that? (Sorry to say this, but explain like I am 5)

[faint rivet]

are you using a 768 model? If so, at least one of your dimensions should be at least 768... Same if using a 512 model - it should be at least 512

[timber carbon]

Ohhhhh

[faint rivet]

also, it depends on how smart you were at 5

[timber carbon]

Prolly smarter than I feel now😂

[faint rivet]

there's lots of minutia to wade through - don't feel bad

[timber carbon]

Ill have to check it and try again, its night time right now, but that might be what caused the issue

[faint rivet]

but, that one really looked to me like a setting was off... like not enough steps, strength at 7 instead of 0.7.. cfg_scale at 0.7 instead of 7... something like that

I do it accidentally all the time

[timber carbon]

I wasnt really planning on going this deep the rabbit hole, but I want to make content and post it, and I have pretty high standards for my posts, I want story behind, somewhat good motion settings, to make it sort of an art piece

So here we are 😂

[timber carbon]

Nah, I always had CFG on 7 by default

And steps were at 65

[faint rivet]

prompt was "messy noisy image"

lol

[timber carbon]

Probably☠️ I will try testing with different LoRa model

I wonder if it changes, because checkpoints never had this issue

[rich hamlet]

some more shpongling

[stable hawk]

Discord didn't like the upscaled version lol. Second version of Prismic Astronaut lol.

[ripe plaza]

Made this yesterday in deforum
https://instagram.com/stableswirls?igshid=MzNlNGNkZWQ4Mg==

[stable hawk]

Test on a script.

[lyric niche]

Anyone know a good open source motion blur post processor?

[lyric niche]

I wrote one

[timber carbon]

Has anyone ever tried to write motion for moving?

As if someone is holding the camera and it's shaking a bit

[timber carbon]

It seems that pharmapshychotic is not able to take in equations :c, that's for anyone wanting to try it

Maybe there is an updated version which does

[faint rivet]

I bet he'd update it if someone showed interest.

[timber carbon]

but that other link you sent me has god amount of sources for AI tools

[faint rivet]

He posted on twitter that his clip interrogator was now in auto1111 with all features. I responded and asked whether that included batch processing like the notebook... he added it like the next day.

[timber carbon]

I'll have to reach out to him, maybe he'll be able to find solution. I would try myself, but there is a couple of issues I ran into

As the program sends the specified movement to functions that I cannot access

[faint rivet]

yeah, I've never looked at the hunk of code that processes the equations... but I bet it wouldn't be too hard to incorporate it

[timber carbon]

Yeah, it just has to take for example 0:( equation)

and instead of formatting into a float

[faint rivet]

I think you may be able to preprocess the equations into values and then run them through.

[timber carbon]

evaluate it before

But it seems that the function sends the whole line 0:(), 50:(), 100()

Into the function

Althought, yeah, now that you say, I probably would be able to do it

[faint rivet]

yeah, just need the equation to value converter

[timber carbon]

I am so glad I learned a bit of programming a year back xD

Hmm, thought, even if I do preprocess it, the equation contains t variable

[faint rivet]

Deforum and DISCO both have equation code in em... but I dunno about modifying pharma's colab without looking at it.

[timber carbon]

Yeah, understandable, dw, I'll figure this out, thanks for sharing on it anyway

[faint rivet]

I'm actually surprised it doesn't support equations. Maybe pharma uses a different letter for time?

or maybe it's just that old....

I still want to make a preview mode in deforum... but, it's one of a bunch of things I'd like to get around to... I keep knocking em out, but I have a lot of goals 😉

[timber carbon]

It ain't easy to wrap your ahead around so many things

[spiral wagon]

prompt: like

[timber carbon]

I think the best thing is to have things to work on, no matter if you can realize them all

[faint rivet]

I can keep lots of ideas in my head at once... but yeah, to work on any particular, gotta go step by step and fill your brain to the gills for that dev time.

except for the occasional lucky thought which can just be executed... I created Optical flow redo and Redo generation features the same day. All I had to do was realize the idea and the code was easy.

[timber carbon]

I would guess that without realizing the idea first you are like walking through the maze

[faint rivet]

I also fixed the nasty depth warping stretching problem you used to see in all non 1:1 deforum animations in a day... actually, in a few minutes. If it was landscape, things would stretch vertically as you approached them, and in portrait, it would stretch horizontal... and the bigger aspect difference, the more stretching there was.

I felt guilty for not looking at the code sooner. It was a problem I had been planning to "tackle" eventually... I just hadn't looked at the actual code. I opened up the function, and found the problem within two steps.... It was one bad line.
aspect_ratio = float(W)/float(H)

This would make some sense IF we were talking about the aspect ratio of an image... but the depth stretching aspect ratio isn't that! It's the aspect ratio of how much the depth is stretched for 3D perspective... So, do you know what I changed it to?

aspect_ratio = 1

and everything has worked beautifully ever since, even if you go to 4:1 or whatever... so funny...

[timber carbon]

Ohhhhh

Now that I've read it again it made sense

At first I was like what could be wrong with that ;D

[faint rivet]

actually, I made it into a schedule after that, so you can get the stretching back if you want... or change it as it goes or whatever

We even have like a Legacy bug mode...

[timber carbon]

I mean it could be useful in some cases

[faint rivet]

well, we don't call it that, but...

[timber carbon]

Like I wanted to make sort of an intro

[faint rivet]

it's not really useful as much as artistic

[timber carbon]

And streching with movement format would create sort of a travel effect

[faint rivet]

it also causes an accelerated stretching the more you move towards things

[timber carbon]

forward*

[faint rivet]

people were always cutting off their forward motion before things would get too bad... you literally couldn't do a continuous forward motion unless the image was 1:1 aspect ratio

[timber carbon]

Oh

That my head hardly wraps arounds

[faint rivet]

So yeah, I fixed that... in like 2 mins... and felt bad for not looking sooner. I didn't know how all the functions worked exactly, but I could tell that whatever the bug was, it was something involving the width being divided by the height or vice-versa... when I found the line, I tested it backwards and verified that was wrong the other way (as I suspected), and then changed it to 1... knowing that would totally work, but in complete disbelief for the rest of the night as I rendered 4:1 continuous forward animations!

[timber carbon]

Because I imagine, that with the stretch effect and forward motion it would just create colorful lines which go from the middle as it progresses

Damn, props to you man

Good job

[faint rivet]

yeah, it would just turn into vertical taffy on a 16:9 video within not too long

I don't feel like a genius for doing it... more just right place right time, and I think all the other devs were thinking as I was - that it was gonna be hard, or that it was deeper than just those functions.

[timber carbon]

Understandably so

[faint rivet]

But, depth animation functions were still messed up... had random glitches where you would jump forward and the bits near the camera would get corrupted...

until I fixed all that last month and allowed for us to use multiple depth models too!

[timber carbon]

Btw, do you have any idea why for example, if I set z_transition to 200 and try to quickly deaccelarate to 10 or 20 quickly, like within 10 frames, the engine just seems to be stuck at 200 and does not slow

[faint rivet]

200! That's a lot of z, yo!

[timber carbon]

Thats how I tried to make the quick travel effect xDD

[faint rivet]

Well, if you're using cadence and the actual rendered frame doesn't hit on your actual keyframe, it won't do anything

well, I should add caveats to that, cuz the motion still goes during cadence

but renders don't happen, so it depends on how your stuff is lining up/

although, I've never tried a value that high, so I can't say if there's a bug there or not!

I mean, it should just spread the values out over that many frames.

[timber carbon]

It's okay tho, I am eventually going to work around it. It usually comes to, I try to make something, It doesn't work, so I try to approach the problem from different angle

I try to adapt to tools that I have xD

[timber carbon]

Yeaah, exactly

[faint rivet]

although, I have to say... depending on FOV, I'm not sure if I could even perceive the difference between z transition of 100 or 200... I mean, everything is just a blurry mess at high speed like that

[timber carbon]

Yeah, that's true

It is not usable

[faint rivet]

but you can cool brief effects like that and stuff

[timber carbon]

Or even if it is usable, the animations or frames created usually don't bind together with the idea

As, even if I tried to make quick travel effect, it would just move quickly and generate images based on the prompt

[faint rivet]

probably better to do a quick slow down transition like that in post 😉

[timber carbon]

So if I modified the prompt, I could achieve it with lower speeds for sure

[faint rivet]

or, you can slow it way down and render more frames

[timber carbon]

Yeah, that would also work

I just looked at it from reality perspective

That if I moved quickly with my camera

as if A.I. would understand that the lines of the object have to be strected

[faint rivet]

I've done that with fast video where I want optical flow to track it better... just slow it down 2x or 4x with RIFE and then run that through, then decimate it in post.

[timber carbon]

Damn, imma need a dictionary if I want to fully learn from you, which I do 😄

[faint rivet]

I'm so high, lol

you just need to get your cardinal grammeter aligned with the upend of the girdle spring

[timber carbon]

https://tenor.com/view/confused-mad-whatever-done-steve-carell-gif-7281251

[faint rivet]

anyway, I don't tell you about my deforum successes to show how cool I am. I do it to say that I was in the right place at the right time with the right skills. I never even learned any real python until like August or September of last year. I certainly didn't think I would be the guy to bring optical flow to deforum.

[timber carbon]

That's cool bro, I understand

Thanks again for the chat

[faint rivet]

many people have thought I was some expert or went to school for these subjects or something... nope. I just wanted to make cool stuff and I learned python to do that! This is all a whirlwind for me... just the way the industry is right now I guess! Take care, and happy animating!

[clever schooner]

First run ever using deforum. I didn't pay attention to the sine function used by default in the zoom parameter. Either way, I like this

the prompt was along the lines of brutalist architecture and a hive mind society

[summer wagon]

/

[neon coral]

https://www.youtube.com/watch?v=7p7_AWyATGI New Animation beatsynced with Parseq

[clever schooner]

this one kinda shit. wont try that again

[timber carbon]

Thanks, good luck to you

[wicked matrix]

prompt

[eager trail]

Made this for fun...hope this is right channel for this:

Could do higher resolution but point was to generate in minutes not hours.

[stable hawk]

Audio made with Riffusion and tweaked slightly in audacity to bring out some of the sounds. Video made with Deforum. Simple 2D equations.

[lethal ridge]

Probably took like 5 hours

[crimson canyon]

Sky

[robust oracle]

Don't ask me, but yes, i was bored as hell

[ripe plaza]

It took about 7 to 8 hours for me

[lethal ridge]

dang

[hallow crane]

So cool!

[eager trail]

looks great. what was your denoising strength?

[lyric niche]

I think it was like 0.15. First pass was pipeline with openpose, p2p, tile, reference-only I think with strengths 1.0, 1.0, and 0.5. and 0.5 for the reference only style. I also have a custom cross-frame attention processor I use. Then I did a second pass through img2img

Oh, I'm using ECCV2022-RIFE to generate extra frames then I average them to get the motion blur (average in linear space because I saw someone else do it idk)

[hazy parrot]

is it possible to make anime style ?

[ripe plaza]

For sure

Using lora's will make it easier

[hazy parrot]

which lora should I use?

I would like to make high quality anime

And I think I might need your help.

but I think I may have to pay you

since you do commisions for people

[marsh scaffold]

Super saiyan AI

[vast vale]

my first attempt 🥲

[serene veldt]

YouTube 's compression wasn't particular kind on all the small changing color gradients, so best viewed at high quality, but other than that I really like how this one turned out. https://youtu.be/mIhISLY96_w

[sharp depot]

https://www.youtube.com/watch?v=162aRqPG1fI

[whole pelican]

is anyone doing videos with sdxl? It seems like it is a bit more consistently better than 1.5 with images at least

[timber carbon]

After working with a lot of settings I have encountered an issue where my movement is no longer working xD Any ideas?

I have very simple movement settings

And most settings I have changed were seed, CFG, but as I imagine they shouldn't have to correlate between motion

One thing I can add, is that zoom settings work

Btw in the screenshot there was an error where I forgot to close brackets, but correcting that didn't seem to solve it

As it appears 0:(0) was the issue

Once it made it 0:(1) it had no issue to further progress with movement, I fixed it

[fallen girder]

360º VR Deforum Animation (Drag to move the camera, also using WASD on the keyboard)
https://youtu.be/uxh5OUPxokU

[hollow mason]

I also want to do this, can someone tell me how to start?

[lethal ridge]

Kid: Mom, can we have Goku at home?
Mom: No sweetheart, we already have Goku at home
Goku at home:

[rotund snow]

Started experimenting with the SD CN extension. Color me impressed.

[lyric niche]

A little test. I'm redoing atm to get rid of the extra hand.

[rotund snow]

She needs tan shoes, pink shoelaces, and a pink Panama with a purple hat band.

[vast vale]

hey i want to make this too but need one info

[fallen girder]

Im using a 360 equirectangular LORA from Civitai in the animation prompts for the VR effect

https://civitai.com/models/10753/latentlabs360

[vast vale]

can i make it look like a city where i live in reality? like the places we have here, the buildings, Kolkata,India is where i live, is there any model that will help?

[fallen girder]

Photorelistic?

[young flower]

Does anyone have resources on Stable Animation UI? I'm trying to leverage it instead of automatic1111, but haven't seen hardly any videos/tweets/etc

[fallen girder]

you can mix this LORA whit a photorealistic model like "Realistic Vision V2.0" for example

[vast vale]

actually i skipped the "learn about lora" chapter since i started using sd 😛

[fallen girder]

It's something like trained styles that you can invoke using any model by typing its tokken in the prompt

[vast vale]

ohkaay, so i will have to find a lora image of my city?

[fallen girder]

you can train your own LORA with real photos of your city, it's hard to explain but you have a lot of tutorials on youtube

[vast vale]

okay

one last help, can hyou please provide a simple to understand video link?

[fallen girder]

https://youtu.be/j-So4VYTL98

[vast vale]

THANK YOU!

[fallen girder]

you're welcome😉

[sinful nebula]

how can i ulpod a video and ask for animtions for the video?

[round jay]

playing around with depth and 3d maps

[fallen girder]

"Magical Forest" | 360º VR Deforum Animation
https://youtu.be/g7bj8Vl26zY

[void girder]

Stable Diffusion Claymation

https://www.youtube.com/watch?v=pW3F35-aaP4

[lusty star]

https://www.youtube.com/@DrakenZA_/shorts

call me if you need assist @blazing matrix with your stable animation stuff :)

[daring warren]

https://www.instagram.com/p/CstF2KyIsJJ/

veiled.vision.studio

Anastasia 👑

Guided imgs it's always a fun tool to use on🔥

Instagram

[dark sage]

Enjoy Deforum / Deforumation anination by LAInol https://youtu.be/SwXz1k-P_uc

[faint rivet]

I made a lyric video for one of my old songs!
https://www.youtube.com/watch?v=Jjq_Lvb7b9E

[faint rivet]

Is this deforum and my hybrid flow? Nice job, even if not.

[faint rivet]

zoom and angle don't do anything in 3D mode

in webui version of deforum they hide depending on the mode

[mossy ridge]

https://www.youtube.com/watch?v=uOAt5s_z9rM

[misty dagger]

Thanks I don't use deforum

[faint rivet]

warpfusion?

[misty dagger]

No it's SD-CN

[faint rivet]

oh! it looked so much like optical flow, but I get it now.

[misty dagger]

It's been my fav

[faint rivet]

CN offers so many different ways to do things. In deforum, we've been tripping out on how much tile can do for animation consistency, all by itself... add some other stuff, and it's amazing.

[misty dagger]

I like flowy surreal inconsistent

[faint rivet]

me too... I was doing stuff like this with my hybrid video before controlnet even came out. https://www.youtube.com/watch?v=IoH_4bFUpSo

[lusty star]

https://www.youtube.com/shorts/zyo_99ZBAm8

[pulsar sage]

That's awesome - I need to try that sort of datamosh thing

[faint rivet]

it's an old fractal animation of mine I used to feed it.

[pulsar sage]

the datamosh thing has never been easy to do without specific versions of certain apps

even touchdesigner couldn't do it right

i never thought of trying it with SD/deforum - but that looked exactly like the effect

i'm still trying to understand how to get continuity with deforum and your hybrid video

[faint rivet]

well, nowadays you can use controlnet alone to get some pretty good stuff, esp using tile. But, many people still use hybrid motion optical flow without actually using hybrid compositing. Or, you can mix a little compositing in too.

[pulsar sage]

haven't explored controlnet with animations yet - looking forward to it. it's a crazy deep rabbithole! so exciting

[stable sky]

https://youtu.be/oImzHMTWzBo

[stoic frigate]

https://www.youtube.com/watch?v=LejSJGmtEvE

[cunning hatch]

after reading up the AnimAnon-Deforum guide, i can't find specifically what i need to smoothen this animation and i wonder if it's possible with a few tweaks in the webui settings
my goal is to have a clean loop of this moving background (this short snippet is when the end loops back to beginning),
originally it's 3840x2160 resolution, 54:29 seconds long, 1649 frames, 29.97 fps,
is there some sort of predictive function that cleans up the stagger/jump so i can infinitely playback the video?

[ionic zenith]

I made a little comparison video to show what my latest music video looked like straight out of Blender, versus what it looks like after running it through Stable. Might be of interest 😊
https://youtu.be/HqAgj4KfHh4

The original for those who hadn't seen it: https://youtu.be/EV7pifgfAWc

[dense crater]

https://www.youtube.com/watch?v=ZrCOxndwREI

[stable moon]

Perfect!im you make this video where?

[misty dagger]

Thanks! It's done locally in a1111 or vladdiffusion using SD-CN and the Ascension model.

[pulsar marsh]

I just created a blender add-on for making the camera movements for Deforum. Its not fully ready yet but please let me know what you all think! https://youtu.be/h1JD4qMNpuk

[cobalt thunder]

https://www.youtube.com/watch?v=ueWFNekxlgs

[ripe plaza]

Deforum Hybrid Video + Tile + Openpose CN

[vast vale]

using deforum sd, i want to create something for a dance

this has the girl (can't see) in the background i want the animation or style anything like that, any suggestion or tutorial? just the sky

@ripe plaza can you please help

[vast vale]

bapre baap XD

[ripe plaza]

creating alpha matte and using it as mask in deforum should do it ig

I'm looking at ways to create alpha matte rn

you can do it in runway but not for free

[vast vale]

alpha matte?

model/

?*

[ripe plaza]

no

this

if you mask the girl and invert mask in deforum, it will affect the sky only

[vast vale]

where to find alpha matte option?

[fallen girder]

"Peter Gabriel - Panopticom" | #DiffuseTogether Animation Challenge [4K]
https://youtu.be/ATesCJmaLXM

[stable hawk]

Yeah, this is the best method I've found. You can actually extract the frames of your video and run it through one of the batch bg removes then batch alpha scripts and get the same effect.

Once you have you alpha series of images you can concactenate it back together and use it as a mask in actually I think Deforum has an option for an alpha video. Doesn't it?

concatenate lol, sorry my script that does this I called a concactenator lol.

[ripe plaza]

It does have a human mask option that generates alpha matte as video or png imgs

But what I want to try is, isolate one guy in a video where two guys are fighting.

I'm sure it can be done in some editing software, need to see how to do it.

[stable hawk]

hmmm, like ddetailer could probably be used to do that.

after detailer seems similar but I haven't played with it as much.

[ripe plaza]

Yeah I haven't tried them either

[faint rivet]

yeah, you can enable a video init mask on video init tab. And, you can invert the mask from mask init tab if you need.
The humanmask option isn't fully integrated yet, but yeah, it will make humanmasks. Just set it to video and cancel when the actual rendering begins (after the masks are generated). Then, go find your humanmask in a folder and use that for video init mask.

To isolate individual items you're going to have to use whatever other software you need to. RunwayML does crazy masking, but there's a lot of options out there.

[daring warren]

https://www.instagram.com/p/Cs1Yduvochj/

veiled.vision.studio

In my culture and in many others, encountering a white owl flying overhead means a synonym of significant change in life, and it happened that three times I was lucky enough to encounter this majestic and wise animal during night walks, before unfortunately visiting the hospital for a serious health incident, happening a fourth and last time onc...

Instagram

[cobalt thunder]

https://www.youtube.com/watch?v=H0jRRvf3cT0

[azure hearth]

https://www.instagram.com/p/Csx36TQsOX0/

eduefe.artworks

Oniric Vampires AIfeat.

Playing around again with Deforum this time using the Hybrid method and some video edits with AfterEffects

Instagram

[gusty wedge]

I'm trying to reduce the flickering on the masks. I've trained Dreambooth 6 times and still can't get it. This is my closest attempt

[mossy ridge]

Future is loading

[cobalt thunder]

what is the best sampler for anime look?

[glass grove]

Helllooo can someone tell me which loRA is really good for beautiful symmetrical faces ? thanks

[glass grove]

which lora is this ? 🙂

just learning about LoRAs and also not sure if the Hash is the same as Trigger word ?

[safe oriole]

no loras, just text deforum cadence 5

[wraith flame]

first attempt at transforming a video into a stylized look

[ripe plaza]

it's really good for first attempt

Deforum + MultiControlNet
https://www.instagram.com/reel/Cs2glP8g040/

[random dirge]

good job bro

hi all. Do you know any tutorials for animation? (video to animaton) thanks.

[ember grove]

https://www.youtube.com/watch?v=beNpmT5MGGE

Stable Diffusion batch processing, control net, masking faces, and original dance choreography

[fallen girder]

"Omegagon - Attunement" | Deforum Video Input (ControlNet) [4K]
https://youtu.be/_wqCFiUcArM

[mental pecan]

https://vimeo.com/831329933

[ruby cradle]

https://youtube.com/shorts/Kxpu-GO3Yjo

[faint rivet]

https://www.youtube.com/watch?v=aUndGg18CcE.

[haughty isle]

Hi all, Parseq (https://sd-parseq.web.app/) now has Deforum motion preview, inspired by pharmapsychotic AnimationPreview notebook. It won't be as accurate as the notebook, but should help give you a decent sense of your movement as you tweak parameters in realtime. Currently it only supports 3D motion parameters (so zoom and angle won't do anything, use translation and rotation). Hope it's useful!

[cobalt thunder]

https://www.youtube.com/watch?v=EbXdvrvYsnE

[cloud smelt]

Warpfusion https://youtube.com/shorts/Kbie73RMPDU?feature=share

[dense crater]

https://www.youtube.com/watch?v=PVAcX4DcBHE

[vast vale]

Hey

Is it possible to make a 20-30secs animated video but I can’t shoot a video but click selfies/photos

[mellow kettle]

Bonfire. Burning Heart. Music video.
https://youtu.be/C2sfczn0RmY

[dark sage]

https://youtu.be/HWwnt4tHGx8

https://youtu.be/WQMAp7_V8eI

[sage yacht]

My second try with animation

[faint rivet]

https://www.youtube.com/watch?v=ilGBg8iwNfA

[sage yacht]

smoother version

[tender cedar]

child

[lean tundra]

Cirith Ungol, great band

[misty dagger]

Cool you know them ya they are awesome

[lean tundra]

Yeah, they are great

[barren pike]

this was very inspiring. Would love to know any details about how you made this!

[cobalt thunder]

Was controlnet used for this?

[sage yacht]

Yeah openpose and another one i cant remember from my head right now

[cobalt thunder]

When I try to create videos with people they seem to start falling apart further into the frames and have less and less detail as it goes on. It's only text to video here no init or video input. Is there a way to fix this or keep the level of detail throughout the animation? https://www.youtube.com/watch?v=JiP4RG0mmf0

[barren pike]

This is incredible! Can I DM you about how you made this?

[fallen girder]

I have included the workflow in the video description

[ruby cradle]

https://youtube.com/shorts/e02KgCIZhRs

[safe oriole]

thought i would try this, but omg how do you run depth_zoe, my it/s went to is/t :\ this was just a 0.5 depth softedge , tileresample lineart 1.0 test run

[lyric niche]

That's really good. What's your method?

[near badger]

https://github.com/s9roll7/sd_loopback_music_sync_wave
Denoising Strength: 0.4
controlnet
[1] lineart_realistic / control_v11p_sd15_lineart / My prompt is more important
[2] none / control_v11e_sd15_ip2p / My prompt is more important
[3] openpose_full / control_v11p_sd15_openpose / Balanced
[4] reference_only / Balanced

[dusk island]

What's your peak VRAM usage using 4 cnets?

[near badger]

I wasn't concerned about it, but I think it probably won't reach 10 GB.

[dusk island]

Huh, not as bad as I thought. I'mma give it a try. Thanks.

[fallen girder]

Yes it's quite slow, I let it run all night, for this video it took me around 12 hours, I also reduced the frame rate of the original video to 10fps to avoid flickering and speed up the process, I also think in anime style it is easier to get good results

[fallen girder]

"Acid Rain" (ModelScope text2video / Zeroscope 320x) [4K] https://youtu.be/rKSoIDhMqlw

[faint rivet]

https://www.youtube.com/watch?v=Uc50FVpsjIk

[bold swan]

https://vt.tiktok.com/ZSLFF9nhN/

[stoic frigate]

https://www.youtube.com/watch?v=LejSJGmtEvE

[trail willow]

This is very cool! Which model did you use?

[crystal gust]

did you use mask?

[azure hearth]

have a nice sunday https://www.instagram.com/eduefe.artworks

[wraith flame]

Thank you!. I used Inkpunk Diffusion.

[spare goblet]

Aged-up Arnold.

And, turned into Chris Pine. (edit: Chris Evans 🙂 )

[lyric niche]

yea

[cobalt thunder]

https://www.youtube.com/watch?v=3EaFAI8azBQ

[viral sand]

hope you'll like my first music video made with deforum 😉 thanks to all the community for all the tips 😄
https://www.youtube.com/watch?v=PsHZusgWozA

[bold swan]

How to make this? It's the consistent rendering!

[rustic snow]

Damn, the animation is great!

[rustic snow]

How did you make it?

[rich gale]

I did this for our local theater’ s Pride performances. The logo will fit at the top

[pulsar marsh]

https://civitai.com/articles/248

[dusk island]

It nice to see people posting some actually useful tools on civit. That's dope.

[dark sage]

https://youtu.be/8Z0A4EnvQno

[cobalt thunder]

https://www.youtube.com/watch?v=9xPOpuk3Oy0

[velvet crag]

https://www.tiktok.com/t/ZTRoK9sqr/

[silver parrot]

Steampunk Pro, Stable Diffusion Deforum, Model https://civitai.com/models/7371/rev-animated

[plucky plover]

Our Colorful World 🌈 https://www.youtube.com/watch?v=tYLhesf_tnA

[haughty isle]

Example of audio-reactive prompts with Deforum+Parseq – this uses a fixed seed with low Deforum strength (equivalent to high a1111 strength); amplitude influences facial expression, pitch influences the presence of the PasteVectorAI Lora: https://www.youtube.com/watch?v=fgiev3A93RU

[twilit shale]

My first test with SD+Deforum - https://www.youtube.com/shorts/zZwOonsBiRA

[vast mural]

Trying to rotate around a subject with deforum

[sharp depot]

https://www.youtube.com/watch?v=S75wBnk7lho

[bold swan]

https://www.tiktok.com/t/ZSLYCYU6Q/

[sage yacht]

Heyo, im playing around with grids for more consistent animations, but I ran into the issue where after img2img it does not keep the borders between the pictures. Looks something like this. Anyone got an idea for a workaround maybe or some tips?

[dark sage]

Good job, is this using a lora or just Prompting? What model?

[vast mural]

Thanks! Just prompting. Think the model was dreamshaper.

Life…

[dark sage]

Trying to achive the same result with RevAnimated122. And using our tool Deforumation. Have not been able to do a full rotation.

[vast mural]

can post the deforum settings when I am back home. This clip is part of a continous 10 minute long animation where ”girl” was supposed to be part of the negative prompt but something went wrong, and right 🤪

[safe oriole]

you really perfected this clean background, do we have any automasking for deforum as a tool being worked on that you know of? eg specify subject with text to mask, feed it video, it disappears masked subject, thats now the controlnet/feeder video for these other tools

[somber geode]

Would anyone love to have a video like this ?

[oak urchin]

Nice, some anti-blur between 0.5 and 1 would help here. Or less steps, nah ?

I named this one: "A normal woman's life" 👀

or "Regan MacNeil's life", can work too

Deforum Generation :

I made a basic Deforum Prompter for ChatGPT, some weeks ago. Ideally gpt4, way less dumb and more context.
If some wanna try, for a good base to edit. Can write some pretty cool stuff, and some surprising motions, sometimes.
Regenerate several times and select or mix the responses.
Also, edit your first request to change the theme, or it may forget a part of the prompter.
Feed ChatGPT, or ideally gpt4 with this :
https://civitai.com/models/59785/deforum-chatgpt-prompter?modelVersionId=64236

Hope it can easily bring some peeps in

Made another private version, way better prompting, but for gpt4 only, can't manage to teach ChatGPT with more features..

[serene veldt]

https://youtu.be/tSveT8Y2ZUU

[vast mural]

Great advice. Will try it

[oak urchin]

For me it's simple (in 2D tho), if zoom is not at 0, then anti-blur = 0.5 to 1
Kind of the same when just 'moving'
If still blurry, when moving slowly, then less steps <20

[vast mural]

so just less steps or antiblur during the faster movements?

[oak urchin]

antiblur when motion/fast motion
less steps when it stops, slow down a lot

[vast mural]

ok

[oak urchin]

It's my experience in 2d motions

Don't like 3d

so weird

[vast mural]

in my prompt I have ”depth of field”, bloom and bokeh, so some blueryness is intentional, but too much

[oak urchin]

You can barely achieve the same motions in 2d

Some instructions can be tricky in deforum, like writing 'ray tracing' in your prompt. You'll see a growing aura around the points of interest.

Kind of doing it over and over and over, each frame

you see

[vast mural]

so much to discover. 😅

[oak urchin]

Dang yeah, can go so far. Using things like Deforumation, to 'target' where you wanna go in the pic. Not zooming into walls in loop.

I stay at the basis, no controlnet, no shit. Fully generated videos are way more interesting

That's like cheating to me to take one video, or poses and process it ..

I made a 10 000 frames video from the chatgpt prompter

Let me find it

11min lol

This one. "The Parallel Worlds". No sound

Enjoy the trip

The angle motion was kind of inversed, sometimes, tho. And no stop to edit the motion, writed at start and launched.

Settings file

And it was before the 'Continue generating' button in chatgpt, huh. So easy now.

[vast mural]

Wow! Mindblowing! you have a version with music or sound?

[oak urchin]

Nop, unfortunately i didn't spend time yet to pack some musics. I will at some point

I'm playing with the prompter, i love it.

Here is the 'private' version. It's cause it's only working with gpt4, chatgpt is too dumb for this, and not enough tokens allowed.. Can run on a local model, like using gpt4all, tho.

[vast mural]

Really nice timing with new prompts and zoom/motions flexing speed. Great work

[oak urchin]

better prompter for gpt4

[vast mural]

i have gpt4 access. Will try it

[oak urchin]

https://tenor.com/view/um-hm-thinking-no-words-gif-24149930

You can see that he understand the motions, the camera movements, can use equations. It's not just basic motions in loop, it vary

[vast mural]

super cool. There goes the evening! 😎

[oak urchin]

NB: If a motion value is repeated, it's not a mistake, it's to start the next one.

[vast mural]

@dark sage @oak urchin here is my deforum settings for the rotation videos: https://github.com/RobertSvebeck/StableDiffusionThings/tree/main/deforum_settings/object_rotation

More rotations

[dark sage]

Is robots getting popular? Panic at the robot factory! https://youtu.be/DqZdGODTXt4

[dark sage]

Now make a 360 rotation :)

[oak urchin]

Nice, I never tried 3D yet, never

Let's see if i can rotate all around, 360°

Just read the comment above lol

ho god, can't post the prompt here, blocked, dunno which word

hum, was just too long

[vast mural]

My guess is that it will go 360 if I just render more frames. 480 frames. cant test now, GPU is busy…. but should work?

This should generate 360 rotation (30fps, one prompt starting at 0):

"animation_mode": "3D",
"max_frames": 480,
"border": "replicate",
"angle": "0: (0)",
"zoom": "0: (0)",
"translation_x": "0:(2.5)",
"translation_y": "0: ((0.06 * cos((106 / 120 * 3.141 * (t + 13) / 30))**30 + 0.01))",
"translation_z": "0: ((0.06 * cos((106 / 240 * 3.141 * (t + 13) / 30))**3 + 0.01))",
"transform_center_x": "0: (0)",
"transform_center_y": "0: (0)",
"rotation_3d_x": "0: (0)",
"rotation_3d_y": "0:(-0.5)",
"rotation_3d_z": "0: (0)",

i need more computers….

[oak urchin]

900 frames at 15 fps, so 112 frame per scene below with your rotation speed, revised prompt :
{
"0": "Cinematic movie still, highly detailed, fine details, intricate, lens flare:0.5, bloom:0.5, raytracing, specular lighting, shallow depth of field, 200mm lens, hard focus, smooth, cinematic film still, cinematic, 8K UHD, bokeh:0.4, a girl seen directly from the front in an endless expanse of an azure ocean --neg side-view, back-view",
"112": "Cinematic movie still, highly detailed, fine details, intricate, lens flare:0.5, bloom:0.5, raytracing, specular lighting, shallow depth of field, 200mm lens, hard focus, smooth, cinematic film still, cinematic, 8K UHD, bokeh:0.4, a girl seen from her right quarter in an endless expanse of an azure ocean --neg front-view, back-view",
"224": "Cinematic movie still, highly detailed, fine details, intricate, lens flare:0.5, bloom:0.5, raytracing, specular lighting, shallow depth of field, 200mm lens, hard focus, smooth, cinematic film still, cinematic, 8K UHD, bokeh:0.4, a girl seen from her right side in an endless expanse of an azure ocean --neg front-view, back-view",
"336": "Cinematic movie still, highly detailed, fine details, intricate, lens flare:0.5, bloom:0.5, raytracing, specular lighting, shallow depth of field, 200mm lens, hard focus, smooth, cinematic film still, cinematic, 8K UHD, bokeh:0.4, a girl seen from her right rear quarter in an endless expanse of an azure ocean --neg front-view, side-view",

"448": "Cinematic movie still, highly detailed, fine details, intricate, lens flare:0.5, bloom:0.5, raytracing, specular lighting, shallow depth of field, 200mm lens, hard focus, smooth, cinematic film still, cinematic, 8K UHD, bokeh:0.4, a girl seen directly from the back in an endless expanse of an azure ocean --neg side-view, front-view",
"560": "Cinematic movie still, highly detailed, fine details, intricate, lens flare:0.5, bloom:0.5, raytracing, specular lighting, shallow depth of field, 200mm lens, hard focus, smooth, cinematic film still, cinematic, 8K UHD, bokeh:0.4, a girl seen from her left rear quarter in an endless expanse of an azure ocean --neg front-view, side-view",
"672": "Cinematic movie still, highly detailed, fine details, intricate, lens flare:0.5, bloom:0.5, raytracing, specular lighting, shallow depth of field, 200mm lens, hard focus, smooth, cinematic film still, cinematic, 8K UHD, bokeh:0.4, a girl seen from her left side in an endless expanse of an azure ocean --neg front-view, back-view",
"784": "Cinematic movie still, highly detailed, fine details, intricate, lens flare:0.5, bloom:0.5, raytracing, specular lighting, shallow depth of field, 200mm lens, hard focus, smooth, cinematic film still, cinematic, 8K UHD, bokeh:0.4, a girl seen from her left quarter in an endless expanse of an azure ocean --neg front-view, back-view"
}

[oak urchin]

The description doesn't fit the rotation. Will try to face the cam at any time. That's my point, to be sure to be clear*

Here comes the prompter

huh, ffmpeg not found

again

I'll take time to prepare a starting pic and a solid prompt, i'll be back

[vast mural]

The idea is that you just have one prompt starting at zero and no other prompts. Then the 3d rotation rotates around that object. It works. No need to have more than one prompt.

[dark sage]

same model? you tried it with revanimated122?

[oak urchin]

Okay, i got it, you seems to mix motion and prompt.
The motion is unique, especially using an equation. So yes, you run it and it's going.
But this is only the camera movement.
The prompt is here to describe what is printed on the camera 'lens', in some way.
If your prompt is 'front-view of a girl', the model don't care at all about your motion or movement, it will roughly create a front-view at any time, ruining your rotation motion.

Am i false here peeps !? Seems legit to me, still preparing the prompt for the best quality, without "multi-interpretation words", lol

[vast mural]

yes. And since each frame is influenced by the previous, consistency is kept

complete deforum example: https://raw.githubusercontent.com/RobertSvebeck/StableDiffusionThings/main/deforum_settings/object_rotation/20230607203915_settings.txt

[oak urchin]

Define consistency, regarding the model

It can't be that way

What when you're in the rear ? The model draw hairs ? no eyes growing here and there ?

[vast mural]

cosistency as in ”luck”

😅

[oak urchin]

I'm maybe wrong, still never tested, but playing with SD long enough to think this is strange as well

[vast mural]

i was also surprised

[oak urchin]

If i wanna blink eyes, i describe each step, and adapt the scene duration

Always did like this

[vast mural]

rotating around a house is less consistent, more weird artefacts

[oak urchin]

same for any 'in the picture' motion, regarding the camera movement

[oak urchin]

I'll fix that for you, wait for it

Same vibes, if you describe 'an house', it's pretty random

If you describe the right side of Michael Jackson house, here is a reference point, the house won't change

[vast mural]

yeah

[oak urchin]

same for people, clothes and cie

Here, i went for a forest, for example. Not to have the sun above ocean, in the pic

Embarrassing to describe every pic feature and element in each scene

[vast mural]

looking at the Viking helmet above, the ”horns” are not rotating correct but the rest of the head is rather good

[oak urchin]

I'll start here, pretty lambda, no body or dress for now, to not care if it change

[oak urchin]

Until what rotation angle ?

70°-90° ? probably not much more

let's try with only 1 scene in the prompt

[oak urchin]

Okay, i'm not bad. Sketchy, but the motion and the view are the point

But i need to fix ffmpeg

[oak urchin]

Bad motion, retry

The eyes leave the camera at frame 30 only

Frame 26, 30 and 40

[vast mural]

there are ai’s to fix eye focus 😅

[oak urchin]

Starting body position clearly isn't the best

[oak urchin]

cheat. You just need to tell him, this is the way.

process a video, it's even shorter

[vast mural]

fair point

[oak urchin]

Fully text generated is interesting

Subjective pov, of course

Frame 100 and 300, look the shoulders. Prompt at this point is " seen from her left side", until frame 336, starting the rear side

I didn't fix the context, may vary. Can be fixed as well

Frame 400, "seen from her left rear quarter". And frame 500, loosing the face, don't give attention, seems pretty normal for my prompt current state. The light direction is still good.

The motion is still bad. Need to be improved, i have a bad translation, the camera go away. I think i should use standard values, and maybe only one (or none) equation to start with.
I'll disallow any element in the foreground too and work on this prompt. I'll ping you later this week, it's a first for me, need some time.
Not tonight tho

[vast mural]

same here. will look more at this tomorrow

[twilit shale]

https://www.youtube.com/watch?v=bbBkaouPEAA

[cobalt thunder]

https://www.youtube.com/watch?v=Nkci1PXBR4A

[digital osprey]

what's the trick to frame-to-frame consistency? Each row top-to-bottom has a static rng (1024, 2048, 4096). The columns are additional img2img passes on the previous column (left to right).

[polar pivot]

hey, dose anyone know if there is the copyright for a deforum movie i created?

[oak urchin]

Consistency = high strenght (0.75+ in deforum), low strenght otherwise (0.25-). Still wondering why it's inversed..
Not sure about the other words in your sentence

[oak urchin]

You are the creator, you choose

Copyright is a pain in the ass of (almost) everyone, tho

To be clear, the video doesn't have any copyright when generated, you'll apply a copyright when releasing the video, in the way you release it, and where you release it.

[polar pivot]

yes... i aske for if i need to mention anything

[oak urchin]

Saw this yesterday. Barely related to this conversation, good example. Viva Japan !
https://petapixel.com/2023/06/05/japan-declares-ai-training-data-fair-game-and-will-not-enforce-copyright/

Each juridiction can decide for themself, ofc

So, last meeting for today and i'll look again at the rotation video.
Meanwhile, if anyone have motion settings somewhere, would be nice, lol
The video is :

• A character, in front-view, ~1-2m distance between the cam and the char.
• The camera must achieve a complete rotation around the character, same distance, linear speed.
  3d motion, would save me an hour or two, don't wanna loose time on this motion

[digital osprey]

I'm using a strength of .20 with diffusers in my own application; The rest was describing my static seeds and the grid layout. To get a stronger effect with consistency I apply the .2 img2img recursively.

Maybe my prompt needs work if all others are doing is running with a low strength pipeline, since I'm doing that already?

[digital osprey]

... I think the issue is just bad datasets. I hate that these models just load in photos of objects from the front and not the backs. I may just need to build a dataset of furniture images with 360 rotations and make a dreambooth lora

[fallen girder]

Driving Simulation (Deforum 3D mode)
https://youtu.be/g_jLC2wMUO8

[oak urchin]

Hum, interesting idea. Nice result

[twilit shale]

https://www.youtube.com/shorts/m2xfquWrYJ0

[vast mural]

Full rotation will require cos/sin formulas on both x and z translation

[oak urchin]

Hey ((:

I didn't worked on it again

true

I have a good (far) contact for this job

an equation master

let me find him lol

yeah, GeekyGhost

This guy rocks. Learned a lot from him, at the time.

This part of the channel

i won't tag him

[vast mural]

I also think we need to have another prompt when the camera starts to be behind the object

Look here. This is one prompt, rotation go behind the object and suddenly the object change

[oak urchin]

wait, my last one

But you need to adjust the frames

[vast mural]

aaaaalmost there...

[oak urchin]

"prompts": {
"0": " masterpiece, best quality, A portrait of a 20 years-old girl seen directly from the front in an endless expanse of an epic forest, beautiful dress, half body, cinematic movie style, highly detailed, fine details, intricate, specular lighting, shallow depth of field, 200mm lens, hard focus, smooth, cinematic, 8K, UHD --neg side-view, back-view, profile, quarter-view, diagonally-back-view, diagonally-front-view, top-view, bottom-view ",
"112": " masterpiece, best quality, A portrait of a 20 years-old girl seen from her left quarter in an endless expanse of an epic forest, beautiful dress, half body, cinematic movie style, highly detailed, fine details, intricate, specular lighting, shallow depth of field, 200mm lens, hard focus, smooth, cinematic, 8K, UHD --neg front-view, back-view, side-view, diagonally-back-view, diagonally-front-view, top-view, bottom-view ",
"224": " masterpiece, best quality, A portrait of a 20 years-old girl seen from her left side in an endless expanse of an epic forest, beautiful dress, half body, cinematic movie style, highly detailed, fine details, intricate, specular lighting, shallow depth of field, 200mm lens, hard focus, smooth cinematic, 8K, UHD --neg front-view, back-view, quarter-view, diagonally-back-view, diagonally-front-view, top-view, bottom-view ",

"336": " masterpiece, best quality, A portrait of a 20 years-old girl seen from her left rear quarter in an endless expanse of an epic forest, beautiful dress, half body, cinematic movie style, highly detailed, fine details, intricate, specular lighting, shallow depth of field, 200mm lens, hard focus, smooth, cinematic, 8K, UHD --neg front-view, side-view, quarter-view, diagonally-front-view, top-view, bottom-view ",
"448": " masterpiece, best quality, A portrait of a 20 years-old girl seen directly from the back in an endless expanse of an epic forest, beautiful dress, half body, cinematic movie style, highly detailed, fine details, intricate, specular lighting, shallow depth of field, 200mm lens, hard focus, smooth, cinematic, 8K, UHD --neg side-view, front-view, quarter-view, diagonally-front-view, diagonally-back-view, top-view, bottom-view ",
"560": " masterpiece, best quality, A portrait of a 20 years-old girl seen from her right rear quarter in an endless expanse of an epic forest, beautiful dress, half body, cinematic movie style, highly detailed, fine details, intricate, specular lighting, shallow depth of field, 200mm lens, hard focus, smooth, cinematic, 8K, UHD --neg front-view, side-view, quarter-view, diagonally-front-view, top-view, bottom-view ",

"672": " masterpiece, best quality, A portrait of a 20 years-old girl seen from her right side in an endless expanse of an epic forest, beautiful dress, half body, cinematic movie style, highly detailed, fine details, intricate, specular lighting, shallow depth of field, 200mm lens, hard focus, smooth, cinematic, 8K, UHD --neg front-view, back-view, quarter-view, diagonally-back-view, top-view, bottom-view ",
"784": " masterpiece, best quality, A portrait of a 20 years-old girl seen from her right quarter in an endless expanse of an epic forest, beautiful dress, half body, cinematic movie style, highly detailed, fine details, intricate, specular lighting, shallow depth of field, 200mm lens, hard focus, smooth, cinematic, 8K, UHD, --neg front-view, back-view, side-view, diagonally-back-view, top-view, bottom-view "
},

using lyriel, to mention it

the negative do the job here

My rotation was pretty nice, until the camera was gone

still no ffmpeg, i'll do it, i'll do it

I don't wanna break things on the dev branch

I had to remove many words from the prompt, to not get artefacts

you can't repeat 'raytracing', or things like that here, with this kind of prompt in deforum. It just leads to artefacts

[oak urchin]

I have this exact render. Not close at all lol

The equation just doesn't fit

need to shorten it

[vast mural]

I don't think I need so many prompts. Just front and back

[oak urchin]

Aww, it's just with one prompt ?

hum, amazing

But i need an explanation lol

still weirdly flattened

I can see the light and trees moving behind

with the original depth of view, until the middle of the rotation

I'll spend some time on it, i jsut need a good motion, that's it. Give it a try, you have ffmpeg

[oak urchin]

This, init 0.75

[vast mural]

i think the settings in depthWarp/FOV is also relevant

[oak urchin]

Not used to 3D. As i understand it, it's a part of the '3D zoom' aka translation_z, realted to the speed effect/motion.

I'll make a prompter to teach 3D to gpt4, to understand these motions.
My personal problem here is that i'm not familiar with these equations, i don't/can't see the motion in my mind.

I can't take enough time until this week-end.. to test and learn as i want to.

[oak urchin]

I'll post this here too.
(Pretty big gif, may need to open in the browser)
"The Forest Traveller" from Deforum

And,

https://media.discordapp.net/attachments/1079140203563859978/1104466876140101733/20230506173915.gif

Both from the prompter.
Nice and Smooth motion on the forest one, good camera movements.

Rerun of the forest, same model i think. Not much luck with the camera zooming into things, still the motion is the interesting part here.
Also from the prompter.
Ask for settings. 2min30 travel

Another interesting try, same generation data. Default music, 39 sec only, stopped

The "Mythics Animals Sanctuary", very short version. Still default music, 43sec

[oak urchin]

man

only one prompt ?

This is impressive, for sure

I want to know how now, seems so simple

Did you give a low res try to my prompt ?

I can't see my render, meh

[vast mural]

one prompt yes, but ….

two films, mounted after

so its cheating

[oak urchin]

reversed ? or different ?

[vast mural]

but rotation one direction, one film, then reversed direction another film

film 1: rotating left, film 2: rotating right

then mounted together

in a video editor

at least a start

[oak urchin]

https://tenor.com/view/cat-gif-21997232

Where are you regarding the equation, can you manipulate and edit them ? I mean you understand them ?

I mean, if not much, i can show you other equations, to play with

This one won't fit the full rotation, i feel it

I wanna dig now, but i need to sleep

[vast mural]

yes i understand the general concept, but I am struggeling with plus 90 degrees events

so i can go from front to left side, and front to right side, but not more