#Midwinter - All Purpose Anime Screencap Model / LoRA

386 messages · Page 1 of 1 (latest)

terse bone
#

@violet fulcrum

#

I've collected the Dataset, Tagged the Images etc.

#

Total Dataset Images are : 1795

#

Total Repeats : 10 - 20

#

4 Different Datasets are Included

#

which are :

#
  1. Ufotable Dataset
#
  1. Kyoto Dataset
#
  1. Jamex Hentai Stash
#
  1. Eufonuiz Style ( Similar to Screencap Style )
#

Settings I will use for the First Train are :

#

Batch Size : 6
Repeats : 10
Optimizer : DAdaptation
Learning Rate : 1

#

Scheduler : constant_with_warmup

#

warmup steps : 130

#

Max Train Steps : 3000

#

Base Model : Anime_Full_Pruned

zealous thistle
#

Idk if its needed but I have an entire list of screencap artists if u need it

terse bone
#

This is the first test run for it so I'm just testing the waters.

zealous thistle
#

Just hit me up if u need material, im loaded 👍

terse bone
#

@odd jetty would it really be a good idea to train a LoCon with like Thousands of Images ?

odd jetty
#

hrmm Im not too sure on that

#

Most I've gotten up to is 9

#

But same style. but with 9 characters

terse bone
#

oh well

#

let's see how this goes

#

@odd jetty Also what would be better in this case

#

10 Epochs or Batch Size 6 ?

odd jetty
#

you asking locon, lora, or pur dreambooth

#

oh

#

Uhh depends on size

#

For my 9 charas (50 images each, aesthetic filtered and tagged with 5 repeats) I believe at epoch 4 it is OK.

#

Another locon I had used around 230 images for 1 style. that took 5 repeats and by the end of epoch 5 it is good enough.

#

TBH its really a toss up

#

I think with 200 images on 1 style with 5 repeats per epoch should fit the style in 4/5 epoches

terse bone
#

each style has

odd jetty
#

gotta add Im using dadapt so YMMV.

terse bone
#

400 images

odd jetty
#

hrmm maybe 3 repeats? save ckpt at each epoch for 5 total epoches.

terse bone
#

Midwinter - All Purpose Anime Screencap Model / LoRA

terse bone
odd jetty
#

Hrmm maybe bump it to 2 repeats then.

#

or 1 would be okay lol

terse bone
#

Okay, so let's see

odd jetty
#

I'd prefer more epiches since you can test more frequently and select which best epoch. but bigger batch size probably fits faster (? Untested)

terse bone
#

1795 Images * 2 Repeats = 3590.

#

Then Multiplied by 5 epochs

#

17950 total steps

#

then on Batch Size 2

#

9000 steps

odd jetty
#

🤔 seems about right

terse bone
#

Will save an epoch at around 1800 steps

#

and since it's batch size 2 it should be fast

#

alright I guess I'll go with this for the first train.

odd jetty
#

I dont have a powerful gpu so I cant test higher batch sizes

terse bone
odd jetty
#

I dont colab either lol

#

I try to keep my files local

#

Gonna send a few OOC Screenshots for your viewing pleasure.

image.png image.png image.png image.png image.png image.png image.png image.png image.png image.png
#

That's All I think... @terse bone pick whicn one your like and I'll say the anime name

terse bone
odd jetty
#

Mainly to add to the dataset

#

Idk what to actually add lol

terse bone
#

you have Datasets for all of these ?

odd jetty
#

So I just listed whichever ones look intresting

terse bone
odd jetty
#

I have the anime as mkv's so I can wand my way to a dataset

terse bone
#

but Thank you

odd jetty
#

Ah ok

terse bone
#

will defo want these later down the line

odd jetty
#

np

terse bone
#

Alright, training started

#

let's see how this goes

#

🍿 popo

odd jetty
#

popcorn time

forest yacht
#

oh

#

ive got 11576 anime screenshots all upscaled to 4k if you want

#

subtitle free

#

pm if interested

odd jetty
#

yikes

#

IMO keep anime at native res

#

or at least when screenshotted

#

like 1080p

#

Since upscaling introduces additional artifacts

forest yacht
odd jetty
#

your 3rd screenshot shows some rining issues coming from upscaling

#

I preseume it has been sharpened as well

forest yacht
#

oh i probably took at that at an interpolation frame

#

whoops

odd jetty
#

did you also resample to like 60fps

forest yacht
#

only for panning scenes. looks like shit otherwise

odd jetty
#

Yikes

#

the lines are too thin and might not survive conversion back to 1080p or 1024px sooo

forest yacht
#

oh well. there if you want it. use them in my wallpaper rotation but they are just sitting there otherwise

odd jetty
#

Anyway I have way too many anime scrernshots

#

ah ok

forest yacht
#

although i do have 322 from this season that hve yet to be upscaled and are still at 1080p

#

buuuuuuuuuuuut then again i use a lot of custom filters for mpv

odd jetty
#

warpsharp, esrgan, etc

forest yacht
#

realcugan for upscaling, esrgan is... not a favorite

#

and i use glsl-shaders in mpv, not anything gan related for playback

#

also debanding, scale, cscale

#

the normal shit

clear sluice
odd jetty
#

We need more screencap loras!

terse bone
#

Alright

#

I've gotten the first iteration to test

#

Ufotable Style.

#

3rd Epoch.

terse bone
clear sluice
#

oh I see

terse bone
#

Final Test version done @odd jetty

odd jetty
#

...doesn't feel much diff

terse bone
#

@odd jetty

odd jetty
#

ooh

#

intresting?

terse bone
#

I'm gonna try with Anything V4.5

terse bone
#

@odd jetty

#

It's having trouble doing the part of dataset that looks like actual screencaps even though that part has the most images.

#

Should I increase the DIM or the repeats in the next version to make it better ?

violet fulcrum
terse bone
terse bone
#

Okay, I increased repeats and am now training a Model instead of a LoRA.

#

since I need a good Base Model for this.

#

I increase the Repeats from 2 repeats to 10 repeats

violet fulcrum
#

or AnyLoRA

zealous thistle
#

Its not enough to have one popular one that works generally well we gotta have 5 other people make their own

terse bone
clear sluice
#

except with more styles

zealous thistle
#

in this case yeah, im excluding this one from what im talking about

odd jetty
#

You should add a tag inside the caption telling what style it is

#

Got told by derrian that the folder name doesn't matter (except the repeat count)

#

@terse bone

#

Sorry I have Touhou AI server mited. Just peeked over.

terse bone
#

For Ufotable I added "ufotable style, ufocoloring"

odd jetty
#

probably also do keep tokens

terse bone
#

For Kyoani I did "kyoani style, kyoani coloring"

#

for the Hentai Stash I did "jamexx style, coomer style"

terse bone
#

keep tokens is at "2"

#

which is the amount of custom tokens I added.

#

but it wasn't getting the style in the Hentai / Eufonuiz Folder at all.

#

even though it got Kyoani and Ufotable almost perfectly.

#

I'm thinking I increase the repeat by 1 on each of those folders.

odd jetty
#

Balance the no of repeats with the no of file Ig

violet fulcrum
#

i have 3.1k images from takt op destiny lol

#

@terse bone btw

#

Recommendation for Tle is 0

#

Since it's an artstyle

terse bone
violet fulcrum
#

Loha is better by the way

terse bone
#

I don't think I can add more than double of what I already have.

violet fulcrum
#

For Artstyles in general

#

try maybe 3e-4 lr
tle 0
and dims 8/4 for network

#

for conv i need more knowledge

#

maybe 0.3/16?

#

3 batch size

odd jetty
clear sluice
#

or X loras and supermerge them into a ckpt

terse bone
#

but the Optimizer Args were not working for the ckpt Colab and I didn't know how to fix it so I got frustrated and quit.

#

I will do it eventually.

near cloak
#

You could also ask the Myne Factory guys. They're training a their own model. They also have high quality datasets.

terse bone
#

Okay I took a break yesterday so my bad.

#

I just tested the checkpoint I had trained the day before yesterday.

#

it seems to work just fine.

#

Ufotable Style. ( No Anime Screencap Tags )

#

Gens are with Aurora

#

Will be more screencap like with other models.

#

Conclusion, This needs a CKPT to be at it's full potential.

clear sluice
#

a ckpt would also be very useful for mixes

zealous thistle
#

I dont see much difference with jamexx and eufoniuz

#

Besides, eufoniuz just straight mimicks how the anime looks, at least facially speaking. So it wouldnt settle on any particular look

terse bone
#

I want to make a base model that can capture that look

#

without having Influence from other models

#

Because right now, it barely looks like the images in the dataset.

#

I could also try retraining at 128 dim Idk

#

let me know your thoughts @clear sluice

clear sluice
#

lora latent space is simply too small to accomodate too many styles at dims lower than 128

violet fulcrum
terse bone
wicked nest
#

so are you going to do checkpoint or a lora

terse bone
#

going to start training now.

terse bone
#

Started Training

#

Total Steps = 14150

#

Reconfigured for Batch Size 1

#

Now total steps are :

#

28 300 steps

storm shore
#

nicceee

terse bone
#

Hopefully CKPT can capture Style Better.

#

and work as a base model for further training.

storm shore
#

add makoto shinkai

terse bone
#

bruh

#

How do I fix this ?

#

I'm literally on Batch Size 1

violet fulcrum
#

Is it 768x768?

terse bone
#

no

#

it's 512

violet fulcrum
#

rip

terse bone
#

oh wait

#

it's probably because of the sample prompt option

#

let me try again

violet fulcrum
#

I summon thee @hasty jackal

hasty jackal
#

which optimizer

#

can you show me the full training options

terse bone
hasty jackal
#

for finetuning? ehehe

#

(my bad if I misread the conversation)

terse bone
hasty jackal
#

oh honestly I don't know if dadapation works with finetuning either uhhhhh I was just curious since I've only seen people use it with loras so I wasn't sure if I was misunderstanding or not

#

hmm

terse bone
hasty jackal
#

adamw8bit might work

#

ah, good to hear

terse bone
hasty jackal
#

mefr yeah that's true

#

if you end up having more issues you can turn on gradient checkpointing I suppose

#

(if that's an option)

#

not sure if colab has enough ram for gradient checkpointing or not think_marisa

terse bone
#

that's what colab has

hasty jackal
#

hmm..

#

might be enough but I'm not entirely sure

terse bone
#

since it's a finetune, it shouldn't need much tweaking with the settings

hasty jackal
#

naruhodo if you need any more help let me know

terse bone
zealous thistle
terse bone
#

it was using DAdaptation.

terse bone
#

it's training pretty well judging by the samples

terse bone
#

update : it did not turn out well

#

reconsidering the LoRA / LoCON Option.

storm shore
#

bruh

storm shore
#

idk maybe @violet fulcrum could train

terse bone
#

nah

#

it was going well

#

at around 5000 - 8000 steps

#

will just make it less.

#

I'm just going to start training a 256 dim LoCon and while it trains I go to bed.

#

I do not have enough energy for this rn.

storm shore
#

I cant wait for one day

#

need release

violet fulcrum
terse bone
#

honestly, this could take me a month

#

or less to finish completely.

#

with my current computing power

storm shore
#

if only andite were still here

violet fulcrum
#

I realized

#

Unet kinda matters

storm shore
#

Breh

gleaming nymph
terse bone
#

2nd Locon Retry Results :

#

LoCon was trained at 256 DIM using DAdaptation.

#
  1. Ufotable Style : Is good, represents Dataset well and is flexible,
#
  1. Kyoani Style : Is good, represents Dataset although anatomy may vary slightly.
#
  1. Eufonuiz Style : Is better compared to previous trains, represents coloring well and has somewhat flat shading.
#
  1. Jamex Style ( Hentai Dataset ) : Despite having the most data and repeats ( more than 500 images and 5 repeats ), results stilll do not match dataset and Models fail to represent the style.
#

Conclusion : Will train the style seperately from the other dataset to see if the results presist and if there is any improvement in representation of the dataset.

#

Previously trained CKPT represented Dataset much better than the LoRA / LoCon Method, will investigate further down the line and experiment more with finetuning in the future.

#

Just gonna drop this here in case anyone want's to test.

#

@odd jetty

odd jetty
#

SD 1.5 and any other model?

terse bone
#

KyoAni and Ufotable are prominent when using.

odd jetty
#

what's the trigger word/words?

terse bone
#

while Eufonuiz and Jamex are not

#

there are alot

#

wait

#

@odd jetty

  1. For Jamex Style : "jamex style, jamexx style, coomer style"
  2. For Eufonuiz : "eufonuiz style, coomerv2 style"
  3. For Kyoani : "kyoani style, kyoani coloring, kyoani screencap"
  4. For Ufotable : "ufotable style, ufocoloring, ufoscreencap"
storm shore
#

lora

#

or model

odd jetty
#

@terse bone probably Silicon29 doesn't like it

01068-3859587702-waifu_anime_exceptional_best_aesthetic_new_newest_best_quality_masterpiece_extremely_detailed_1.2_1girl_ascot_black.png
#

Works better on AOM3 Though

01069-3757147892-waifu20anime20exceptional20best20aesthetic20new20newest20best20quality20masterpiece20extremely20detailed_1.png
#

I didn't request for glasses but got one anyway lol

01070-3306296499-waifu20anime20exceptional20best20aesthetic20new20newest20best20quality20masterpiece20extremely20detailed_1.png
#

oh

#

that's ptobably my fuckup tho

storm shore
#

got this

#

UFOTABLE

#

its kinda ehhh

zealous thistle
#

Wheres
The screenshottiness

storm shore
#

its called mid for a reason

violet fulcrum
#

hires does difference

#

try AuroraONE

#

It's most flexible

violet fulcrum
#

doesn't look like ufo at all

terse bone
#

With Ufotable ?

#

You probably did something wrong.

#

Ufotable is the one of the style it does good

#

I'll get get to it tmrw.

#

Rn I sleep

storm shore
#

Idk

storm shore
#

I messed up

#

I switched it with a file with a very similar name

#

its still a mid style

zealous thistle
storm shore
#

need v2

#

mid

terse bone
#

Any further progress is delayed as I focus on those.

storm shore
#

Relatable

terse bone
#

Okay I'm semi-somewhat back to working on this.

#

I've found the LoRAs trained by my method using Dadaptation don't translate well to other models when not using my own merges.

#

So I'm going to start testing again but with AdamW8bit and Lion settings this time

#

Also, going to reduce the amount of images in each dataset so to reduce the training times.

#

Because right now it's like 450 images per dataset.

#

Which is already too many.

terse bone
#

Also, I'm going to be deleting the ( almost 1250 images ) of just straight up porn from the dataset.

#

So anyone who has some anime Datasets they would like to share instead would be appreciated.

#

salt 💗

#

I will put you in credits if this ever gets finished.

violet fulcrum
#

You remind me

#

Let me get it

#

Wait

#

My trigun stampede dataset+700 images

#

My trigger studio dataset+1700 images

#

@terse bone

odd jetty
#

Actually let me pass it to you like later in the evening

terse bone
#

Thanks ya'll 💗 , for now I'm only going to be trying out 200-250 images per dataset.

#

will cap at 150 ( quality picked ) images if the steps get too high.

zealous thistle
#

Id contribute but all I have are names and trained loras of screencap artists

#

Well actually i might have one dataset lying around but others are already putting forth their stuff so its probably not needed

clear sluice
#

Style bleeding might be unavoidable when mixing too many styles. Not sure if text encoder conditioning is enough to bring one clean style out of the latent space

terse bone
#

but then it isn't

#

an AIO ( All in one ) LoRA.

clear sluice
zealous thistle
gaunt anvil
#

@sonic elbow

image.png
terse bone
#

I never made any of the 15 Midwinter models public

gaunt anvil
#

Yeah but like I just guided him here that’s all

#

Also gib model

woeful iris
#

OP left?

gaunt anvil
#

He should be back in a few months

#

Silvelter is another such person

woeful iris
gaunt anvil
#

@terse bone

terse bone
gaunt anvil
#

it's for the Wiz guy

#

he needed your help but saw that you left