EasyFluff v11.2 8217 Artist Comparison | 東方Project AI | Page 1

frozen swift Dec 5, 2023, 4:08 PM

#

Wanted to share comparisons of this very good local model, as it can handle anime styles and humans even though it is trained on e621. Using automatic1111's APIs, for EasyFluff v11.2, I generated 8,217 artist samples based on the top artists by post count from e621 for artist tags with greater than ~100 posts. Includes SFW human and anthro comparisons. If any NSFW slipped through (I didn't observe any during my review), let me know and I'll address it.

SFW only comparisons
https://mega.nz/folder/YQMhHDqa#czASBKMNvoaPqJgH3sLo-w

Model link - https://huggingface.co/zatochu/EasyFluff/blob/main/EasyFluffV11.2.safetensors

Required config, place in same folder as the model - https://huggingface.co/zatochu/EasyFluff/blob/main/EasyFluffV11.2.yaml

Required Extension to rescale CFG - https://github.com/Seshelle/CFG_Rescale_webui

#

Hmm, something is up with mega, the link isn't working, looking into it

#

Fixed

runic jasper Dec 6, 2023, 4:05 PM

#

It does have some anime in it. If I remember right, this model is a merge of Fluffyrock, Polyfur, E6laion, and 526mix. E6laion is a branch of fluffyrock that is e621+laion+danbooru. Polyfur has the same images as e6laion, but captioned by gpt4mini. E6laion has a strong anime influence, but I wouldn't expect danbooru artists to be that strong if you use them since it's more weighted towards fluffyrock and polyfur lacks artist tags I think.

#

526mix isn't related to Fluffyrock, that's just a general model that was used in the merge. Heard it's good and doesn't have as much bias compared to popular ones.

#

(imo, the laion dataset kinda ruins e6laion. Furry+anime model would be nice)

frozen swift Dec 6, 2023, 9:34 PM

#

Didn't know all that, thanks for the info!

frozen arrow Jan 9, 2024, 11:35 PM

#

Holy shit the level to which this model understands tags is crazy

frozen swift Jan 9, 2024, 11:44 PM

#

frozen arrow Holy shit the level to which this model understands tags is crazy

Yes, it is a very impressive model, I'd say FluffyRock (the base model) is the most technically impressive model trained by a hobbyist. Someone has been training a large anime lyco on top of FluffyRock (same epoch base model as EasyFluff v11.2). Latest lyco - https://huggingface.co/CluelessC/hll-test/blob/main/lyco/hll6.3-fluff-a7.safetensors. The lyco is trained the same as EasyFluff (v-pred, znsr, high res), but doesn't seem to need rescale. It is trained on almost 1 million images, booru tags, and clip skip 1. Might be worth trying out. It can handle a mix of booru and e621 tags as a result (though some e621 tags lose their meaning due to the lyco).

#

Really would probably be better to train an anime focused model on top of 1.5 using the techniques the person behind FluffyRock did.

frozen arrow Jan 10, 2024, 12:00 AM

#

frozen swift Yes, it is a very impressive model, I'd say FluffyRock (the base model) is the m...

It's really crazy though, like, I prompted artist style of artist who has literally 18 images on e621 and it nailed the artstyle pretty much

#

Very clearly knows it

#

ok maybe not nails

#

But yea really understands the general idea

frozen swift Jan 10, 2024, 12:05 AM

#

Ah, probably still some seed dependency due to not having enough data, impressive it can do well at all with only 18 though!

frozen arrow Jan 10, 2024, 12:10 AM

#

frozen swift Ah, probably still some seed dependency due to not having enough data, impressiv...

Yup

frozen arrow Jan 10, 2024, 12:10 AM

#

frozen swift Really would probably be better to train an anime focused model on top of 1.5 us...

Or SDXL

#

Man

#

Coming weeks/months will be big

frozen arrow Jan 10, 2024, 12:12 AM

#

frozen swift Yes, it is a very impressive model, I'd say FluffyRock (the base model) is the m...

Isn't working for me? Missing yaml maybe?

#

Also none seemed to need rescale for me so far

#

Tho maybe I'm just not having high enough expectations idk

frozen swift Jan 10, 2024, 12:13 AM

#

Hmm, if you are on the latest auto1111, it supports lycos out of the box, not sure how long ago that was implemented.

#

Should be able to just use it like a lora

frozen arrow Jan 10, 2024, 12:13 AM

#

frozen swift Hmm, if you are on the latest auto1111, it supports lycos out of the box, not su...

I'm not on auto1111

frozen arrow Jan 10, 2024, 12:14 AM

#

frozen swift Should be able to just use it like a lora

OH

#

It's a lora?

#

So I apply it to fluff?

#

I mean I know not lora lora

#

But like

#

Same type

#

Of model?

frozen swift Jan 10, 2024, 12:14 AM

#

lycos are a different type, but similar concept, if whatever you have supports them, you'd apply them the same way most likely

frozen arrow Jan 10, 2024, 12:14 AM

#

Yeah it should

#

Alr

#

Btw

#

Is it using danbooru tags?

#

I'm assuming

frozen swift Jan 10, 2024, 12:18 AM

#

Yeah, it's booru + quality tags (masterpiece, best quality, etc)

frozen arrow Jan 10, 2024, 12:21 AM

#

Damn

#

It really sucks at artists

#

Sad

frozen swift Jan 10, 2024, 12:22 AM

#

Unfortunate, probably didn't tag them if I had to guess, everyone used to NAIv1 not doing it and people creating thousands of loras to cover all of them

#

But, NAIv3 knows artist tags, as does FluffyRock, that should be the standard now, include artist tags.

frozen arrow Jan 10, 2024, 12:24 AM

#

frozen swift But, NAIv3 knows artist tags, as does FluffyRock, that should be the standard no...

NAIv3 isn't open source right?

#

Or can it be downloaded

frozen swift Jan 10, 2024, 12:25 AM

#

It's behind a service

frozen arrow Jan 10, 2024, 12:25 AM

#

Lame then

frozen swift Jan 10, 2024, 12:26 AM

#

Yeah, it's unfortunate, given how good it is, being a large scale anime finetune on SDXL

frozen arrow Jan 10, 2024, 12:29 AM

#

frozen swift But, NAIv3 knows artist tags, as does FluffyRock, that should be the standard no...

Link to fluffyrock btw? Can't find it

#

Is it better than EasyFluff or is EasyFluff better right now?

frozen swift Jan 10, 2024, 12:31 AM

#

EasyFluff is to FluffyRock, like Anything is to NAI, so would train on it, but typically not use it directly (though I haven't used it enough to say). For EasyFluff v11.2, this is the version of it that is it's base

https://huggingface.co/lodestones/furryrock-model-safetensors/blob/ed56de42df23a393fadda38e17e36d234ab70685/fluffyrock-1088-megares-terminal-snr-vpred/fluffyrock-576-704-832-960-1088-lion-low-lr-e159-terminal-snr-vpred-e132.safetensors

https://huggingface.co/lodestones/furryrock-model-safetensors/raw/ed56de42df23a393fadda38e17e36d234ab70685/fluffyrock-1088-megares-terminal-snr-vpred/fluffyrock-576-704-832-960-1088-lion-low-lr-e159-terminal-snr-vpred-e132.yaml

frozen arrow Jan 10, 2024, 12:32 AM

#

Thank u

frozen swift Jan 10, 2024, 12:32 AM

#

They create so many epochs, they deleted these due to age (so had to find an older commit), but you want to use the ones that align with EasyFluff 11.2 for training or the lyco (as it uses the same one)

frozen arrow Jan 10, 2024, 12:32 AM

#

Will compare

#

Hm

#

I'll try to find most recent

#

In that case

frozen swift Jan 10, 2024, 12:32 AM

#

Nah, you can download these from the old commit that these links are

frozen arrow Jan 10, 2024, 12:33 AM

#

Ah ok

frozen swift Jan 10, 2024, 12:33 AM

#

The newer ones are trained further or differently, so you can use them if you like, just might not align with EasyFluff 11.2 or the lyco

frozen arrow Jan 10, 2024, 12:34 AM

#

frozen swift The newer ones are trained further or differently, so you can use them if you li...

I'm usually not really training. Anyways, guessing most recent is megares-terminal-snr-bpred? or minsnr-zsnr-vpred?

#

Also what's the difference ema vs no ema

frozen swift Jan 10, 2024, 12:34 AM

#

ema are weights for continued finetune training, don't need them

frozen arrow Jan 10, 2024, 12:34 AM

#

Ah kk

frozen swift Jan 10, 2024, 12:35 AM

#

I think you would want to stick to the fluffyrock-1088-megares-terminal-snr-vpred line

frozen arrow Jan 10, 2024, 12:35 AM

#

Thx

#

Ah dang

#

4GB, that means it's not half precision or whatever?

#

I guess I can probably somehow convert it?

frozen swift Jan 10, 2024, 12:37 AM

#

There are ways, not sure about standalone tools, I know you can in auto1111

#

It only takes up more disk space, it will load at fp16

frozen arrow Jan 10, 2024, 12:44 AM

#

Ah nice then

runic jasper Jan 11, 2024, 3:22 AM

#

frozen swift Really would probably be better to train an anime focused model on top of 1.5 us...

There's an anime model training with similar settings in terms of zsnr vpred. It doesn't quite have the same settings as Fluffyrock as it's trained on 704 resolution instead of the multi resolution that Fluffyrock does.

#

It's by the Fluffusion guy, but it incorporates most of the improvements that Fluffyrock vpred uses.

#

Also, unlike Fluffyrock, it will be clipskip 2

#

Same with the new version of their furry model that's currently training

frozen arrow Jan 11, 2024, 4:26 PM

#

runic jasper There's an anime model training with similar settings in terms of zsnr vpred. It...

Can it be downloaded yet?

runic jasper Jan 11, 2024, 6:48 PM

#

it's only on the 2nd epoch(soon to be 3rd today)

#

out of 21(maybe a bit more) epochs planned for training

#

but it is training fast, so should be done in a little over a month

#

I could DM you links to the early epochs or the server where they're posted

#

but I wouldn't use it unless you want to compare progress between epochs

#

I wouldn't expect it to be that good until it's through 70% of training

frozen arrow Jan 12, 2024, 9:41 PM

#

runic jasper I wouldn't expect it to be that good until it's through 70% of training

Makes sense, thx

frozen arrow Jan 22, 2024, 11:45 PM

#

@runic jasper what was that model in the end?

runic jasper Jan 22, 2024, 11:45 PM

#

the anime one? Waifusion

#

training is on pause for now as multi gpu training is figured out

worthy forge Jan 23, 2024, 4:54 AM

#

PeepoEnter

frozen arrow Jan 24, 2024, 4:00 PM

#

runic jasper the anime one? Waifusion

All results I'm getting are some shitty NFTs

#

https://cdn.discordapp.com/emojis/1013515285300510801.webp?size=48

runic jasper Jan 24, 2024, 4:46 PM

#

frozen arrow All results I'm getting are some shitty NFTs

it's not public yet and there isn't info about it outside of the maker's server. Would you like me to dm you the server?

#EasyFluff v11.2 8217 Artist Comparison