Nanashi v2 beta (RMVPE) (RVC v2, 800 Epoch)

[dawn sleet]

Well, the v2 beta version of this pretrain is a complete failure, I don't recommend using it. Especially the base model.
Below are the samples and problems that this pretrain has and their possible causes
This pretrain was designed for Brazilian Portuguese

Epoch: 800 (base) finetune (270)
Steps: 1005600 (Base) 143910 (finetune)
Batch Size: 32
Dataset Length: 31 hours (Base) 13 hours (finetune)
F0 pitch extraction: RMVPE
Sample Rate: Only 32k
FP16
Made from scratch
Link: https://huggingface.co/shiromiya/nanashi-pretrain/tree/main/v2
Trained on a 4090 for 4 days

If you want to help me recover financially from the loss of this pretrain, consider donating to me at ko-fi

Samples

• One of the problems this pretrain has is lines in the spectrum when using the base model. The cause of this MAY BE because it was made using contentvec transformers instead of contentvec fairseq

[slate flume]

distorted harmonics?

probably hifigan / contentvec limitations. idk how razer did to avoid these harmonic issues on dmr pretrain lol

[dawn sleet]

I think so

[slate flume]

i wont suggest doing custom pretrains unless you want to, at cost of a bunch of issues

[dawn sleet]

Yes, I wanted to do it

[slate flume]

I want to do custom pretrains using lightning.ai but a problem is the issues i mentioned before

it's impossible to do a pretrain with no issues (dmr did not have distroted harmonics but it got noise)

@dense acorn is a good idea to do a custom pretrain on a old rvc version? like, before they added k/loss?

[dawn sleet]

Yes, I made this pretrain using the latest version of Applio, which changed fairseq to transformers contentvec, which may have caused the lines

[slate flume]

you can try training on a old version of Applio, like before contentvec was added

[dawn sleet]

That's what I'm going to do in the future when I have the money

[slate flume]

i hate hifigan and contentvec limitations

[dawn sleet]

Me too...

[dense acorn]

What are the limitations? Could adding a loss function to punish those lines help?

[slate flume]

probably yes, and a loss function to reduce distorted harmonics or remove them completely, probably another loss function to prevent any type of noise of the custom pretrain

we wait until rvc v3 is released

[dawn sleet]

I don't expect that to happen anymore...

[dense acorn]

If you know more about audio processing, do you mind reviewing my loss functions to see if I made any silly mistakes?

[dawn sleet]

Sure, I can take a look, but I don't know if I'll be of any help

[dense acorn]

I've added a loss balancer with a bunch of auxiliary losses: https://github.com/SayanoAI/Comfy-RVC/blob/main/lib/train/losses.py

[dawn sleet]

Hmmmm I've never worked with this part of the RVC, so I know basically nothing about it

[quiet ginkgo]

I wish

[lunar crystal]

a pronuncia ficou melhor achei promissor

[broken sorrel]

half-life 3

[broken sorrel]

https://tenor.com/view/why-half-life-three-will-gif-18150439

[dawn sleet]

Hmmmmmm, usou o finetune, né?

[dawn sleet]

lol

[lunar crystal]

Ss eu fiquei em dúvida em qual escolher

Boa sorte com o projeto.

[dawn sleet]

Fez certo, o base é só para fazer finetunes

[dawn sleet]

Vlw

[quiet ginkgo]

me watching valve struggle with counting past 2

[copper pagoda]

"is This what You've Been Waiting For Say 1, 2, and 4!"