Is There Any 40K Pretrain Model for High Pitch Singing Inferring?

[burnt moss]

My GPU does not work with Python 3.10 or above, therefore, I am using Mainline RVC, not Applio. I can only use standard HifiGAN, not RefineGAN.

My training dataset has low, mid, and high pitch range in singing, however, the high pitch part is not a lot.

I'm hoping to use a pretrain model already covers the high pitch range, so after training, when I infer high pitch vocals, the high pitch part is covered, there will be no electric-like artifacts in the result.

So far, I have found KLM 4.2 made by Seoul, and SingerPretrain by Sztef, are covering high pitch. However, both models are in 32K only.

If my inference target is 1/3 English, 1/3 Chinese, 1/3 Japanese, in singing, standard 40K, in Mainline RVC, what would be the best pretrain model to use, to achieve this high pitch coverage goal?

[buoyant knot]

• KLM 4.3 is the best so far, though it may still struggle on notes higher than A5 but better than other ordinary ones. the refinegan pretrain may do better tho.
• I dont recall even GTX 10-series gpus don't work on Python 3.10. please check or reinstall CUDA toolkit and install some other dependencies that the error messages might say.

[burnt moss]

Actually our music studio's computer is on Windows 7 running a pre-built PyTorch wheel, and therefore highest Python we can go is 3.8, because the studio can't change computer easily, we have to use RVC mainline, not Applio. The original RVC is very solid, except for the new RefineGAN part.

[burnt moss]

Looks like Souel deleted their post on the introduction of KLM 4.3. Which one is the best? X4 means expetiment 4, is the latest, I assume?

[buoyant knot]

RVC could only run properly on 3.9 or 3.10 for Applio, consider upgrading to win 10/11 or switch to a Linux distro

[burnt moss]

Our studio has hundreds of legacy music software, and many of them are not compatible with Windows 10, that's why we keep our Windows 7.

[buoyant knot]

yea X4 is the latest, there are actually the x2 and x3 in their huggingface repo but it might have a bit more glitches due to being undertrained

[buoyant knot]

https://tenor.com/view/penguin-linux-tux-tux-linux-gif-341950137175367051

[burnt moss]

So, you are saying: X4 is better than x1, x2, and x3, for KLM 4.3?

[burnt moss]

lol!

Yes, we installed a Linux Mint in a Virtual Machine, it is pretty good!

However, RVC mainline is so solid on python 3.8, we simply don't bother building a new Torch of 3.10 for Applio and Mangio.

[burnt moss]

Can I assume, starting from KLM 4.4 and 5.0, new RefineGAN is being used, therefore, the last KLM compatible with HifiGAN, will be KLM 4.3?

[buoyant knot]

https://discord.com/channels/1159260121998827560/1327283027776245810
for refinegan, I'd recommend the new "base" pretrain, currently only the 32k version ready to use, the others are coming soon

[burnt moss]

No, no, RefineGAN is too new and too expetimental, and I won't use it for now. I will just go with the good old HifiGAN, with KLM 4.3 40K on RVC mainline.

https://discord.com/channels/1159260121998827560/1315222227922452491

In this most, you said you tried all x2, x3, x4, which one is the least glitchy one?

I don't have much high pitch singing in my dataset for training, but I want to infer high pitch singing for inference, to avoid artifacts, should I go with KLM 4.3 X4?

The KLM 4.2 description marked in yellow, is exactly what I want. I wonder, does KLM 4.3 have the same high pitch feature as 4.2, or is 4.3 a completely different model?

[buoyant knot]

you mentioned 5.0 which is the refinegan one, also hifigan ones have some quality limitations, including how the models handle the very high pitches

[buoyant knot]

then why not try it, as well as some others having mentioned above

also there are cloud alternatives to train, i.e. some colab and kaggle notebooks

[burnt moss]

I should have told you, the highest I need to go, is C5. No need to worry anything above C5, so old HifiGAN is more than enough.

[burnt moss]

I hope to try it on the 40K, not the 32K one, that's why I want to skip KLM 4.2.

Most of our source are recorded in 44.1K, so 40K model is most suitable.

[buoyant knot]

again you should try the said, hifigan stuffs or whatever instead of the mere "I need it do shit" mantra

I mean try whatever available so far, as there might be nothing satisfy 100% of your needs

[burnt moss]

Does KLM 4.3 has a description webpage like 4.2? I hope to read it before take a jump.

[burnt moss]

You mentioned Cloud, and I have tried it. For Colab, with 16 GB VRAM, I can set the batch size as high as 30, but it would result in overfitting. Can I ask you, for an 60 minutes long (after truncating) dataset, on Colab, should I go with batch size 8, or 16, per your experience?

[buoyant knot]

yea this is what you want
https://huggingface.co/SeoulStreamingStation/KLM43/resolve/main/D_KLM43_x4_40k.pth?download=true
https://huggingface.co/SeoulStreamingStation/KLM43/resolve/main/G_KLM43_x4_40k.pth?download=true

[buoyant knot]

you can try batch size 8 for most cases

[burnt moss]

My other dataset is 2 hours long (120 minutes singing), in this case, is batch size 8 still the golden number on Colab?

Also, Colab recently updated to Python 3.11 and CUDA 12.4.
I can run training on native python 3.11.
Also, I can install a Conda enviroment and run Colab on python 3.8 or 3.10.
Both are working.
Which python version would you choose for training, if you were me?

[buoyant knot]

the results may vary for unknown reason and it only differs on the speed & vram usage
also consistency and variations on pitch and dynamic range matter more than the dataset quantity

[burnt moss]

Totally agreed! Pitch range is so important! Especially on the high pitch end. So, if you were me, to train on a 2 hour wide range dataset, you would still choose 8 for batch size, right?

[buoyant knot]

I'd say 8 is the default choice for most cases

regarding the dynamic range, the compressor & limiter effect on vocals may reduce it

[burnt moss]

Got it!
Eight is gold!
I will go eight for all my trainings, regardless the dataset length!

[burnt moss]

You mean, the bigger the better, or the smaller the better?

My understanding is, the narrower the dynamic range, the better, am I right?

[buoyant knot]

ideally without such the effect, keep the dynamic range natural as possible

[burnt moss]

You want to apply compression and normalization, to keep the volume consistent as possible, right?

You want to normalize to -23.0 LUFS, right?

[buoyant knot]

it is normalization, but not the compressor

[burnt moss]

On Audacity, normalization and compression do the same thing, essential lower the high part, and raise the low part, on volumes.

[buoyant knot]

there's a little misconception,

• the normalization changes the overall volume by difference of its loudness level to the target loudness, however the dynamic range should stay same (i.e. loudness difference of quiet and loud parts)
• compressor reduces volume of signals above arbritary threshold according to arbitrary ratio, which could reduce the dynamic range

[burnt moss]

Got you hater!
You mean: Normalization does not break internal relative waveform relationship, only overall.
Compression does!

[burnt moss]

Hater, I wonder, have you recently run training on Colab's new py311?
Can you give your opinion, on whether to use py311 or py310 for training? Is py310 safer than py311?

#1336863207213498410 message

[buoyant knot]

python 3.11 breaks the rvc compatibility, so try use the latest applio colab that actually downgrades it to 3.10 within the environment and does some other workarounds that resolve some issues that might happen on the old colabs

[burnt moss]

OK! As I run mainline with both py311 and py38 on Colab, the terminal shows totally different outputs. I can tell, the new py311 is a totally different animal! Spooky!