heady hornet
So I have been trying to train a model using RMVPE as I hear it is superior for musical vocals. But I end up with this really disgusting and distorted mess that just makes no sense whatsoever. I have attached an audio clip of this for obvious reasons. Anyone got any ideas on how to resolve this?
(EDIT): This is solved, thank you to all those that have helped. Check this message #1227554503633666159 messageto see the fix I had
This is the model
And this is the input for the voice swap
But I'm sure you all could probably tell which is which
(Skipping to 1:30 will get you actual audio, there's a large gap at the start for instrumentals)
Nevermind discord doesn't like .flac for some reason, so here's the input audio in .wav form instead
long hound
This is the model
This is most likely feature extraction screwing up
Hm...
Check the logs folder of that model, see the 0_gt_wavs
If there's only a few of em something screwed up
heady hornet
If there's only a few of em something screwed up
I believe I have them all
Ignore my drive name...
And the name of the model is confusing too, It's called that because it's mostly singing data, but I decided to have a small amount of normal conversation in there to see if it could clean up the edges on pronunciations
Oh that's cool, just listened to some of the audio and it's literally just small chunks of my voice
wanton wolfBOT
Ayo? @heady hornet level 7 !!! 
heady hornet
wheeyyy level 7
long hound
Check the extract-f0-feature (or something similar) log file in the logs folder... hmmm
heady hornet
These are all the folders here
I do not seem to posses the extract-f0-feature folder
long hound
it's a file
heady hornet
foundit
long hound
should've explained it better 
heady hornet
['extract_feature_print.py', 'cuda:0', '1', '0', '0', 'D:\\AI\\Mangio-RVC-Fork/logs/HarrySing2', 'v2']
D:\AI\Mangio-RVC-Fork/logs/HarrySing2
load model(s) from hubert_base.pt
move model to cuda
all-feature-271
now-0,all-271,0_1.wav,(184, 768)
now-27,all-271,0_44.wav,(184, 768)
now-54,all-271,1_11.wav,(184, 768)
now-81,all-271,1_139.wav,(184, 768)
now-108,all-271,1_173.wav,(184, 768)
now-135,all-271,1_204.wav,(184, 768)
now-162,all-271,1_233.wav,(184, 768)
now-189,all-271,1_27.wav,(184, 768)
now-216,all-271,1_35.wav,(184, 768)
now-243,all-271,1_68.wav,(184, 768)
now-270,all-271,1_98.wav,(184, 768)
all-feature-done
nothing wrong here... hm
maybe the number of CPU processes you picked screwed it all up? it says there it is using 11
long hound
I have cores >:(
u should always use 2
heady hornet
well lemme re run allat
nope did not affect the output
Imma restart the webui first tho just to make surezies
or do I have to fully retrain the whole model instead of just the features index?
heady hornet
yep not working, gonna just retrain it all, yolo right?
here's all of dis if it helps at all
['extract_feature_print.py', 'cuda:0', '1', '0', '0', 'D:\\AI\\Mangio-RVC-Fork/logs/HarrySingV4', 'v2']
D:\AI\Mangio-RVC-Fork/logs/HarrySingV4
load model(s) from hubert_base.pt
move model to cuda
all-feature-144
now-0,all-144,0_1.wav,(44, 768)
now-14,all-144,0_26.wav,(184, 768)
now-28,all-144,0_7.wav,(184, 768)
now-42,all-144,1_22.wav,(184, 768)
now-56,all-144,1_40.wav,(93, 768)
now-70,all-144,1_59.wav,(184, 768)
now-84,all-144,1_76.wav,(184, 768)
now-98,all-144,2_14.wav,(184, 768)
now-112,all-144,2_31.wav,(184, 768)
now-126,all-144,3_14.wav,(184, 768)
now-140,all-144,3_5.wav,(33, 768)
all-feature-done```
Also there's this log if it might help either, just the terminal output when I try to train the model
certified pyenv user moment
chilly light
no pretrained Generator
no pretrained Discriminator
chilly light
isnt it suppose to load the pretrain unless you have like 20 hours or more data
heady hornet
I mean these are empty, I have never had to touch them before though...
maybe I don't have the base models installed?
wanton wolfBOT
Ayo? @heady hornet level 8 !!!
heady hornet
Or is that an entirely different thing?
chilly light
you'd need a pretrain if its like less than 5 hours of data
and if you dont wanna train for like 10 days
chilly light
usually it'll sound weird if you train like 1 hour without a pretrain for a not long time
heady hornet
huh
well
that would explain that
idk why the fucc I'm missing everything, I even had to download hubert manually the other day too
But I had already been through that
chilly light
you dont need the pretrains if your using existing models but you'd need it for training
heady hornet
Yeah well the result was from weird training issues
But yeah I am trying to train, this is very helpful tho
thankies c:
I will check back to see if it worky
loud turret
This is the model
I have experienced this before, it's a pitch extraction error. No idea what's the cause, either the training pitch extraction or the input audio inference extraction. It was from ilaria rvc for inference I used btw.
But the way I fixed it is that I just restart the space and it works back
can you link your model here tho?
sleek wave
i had this exact same error yesterday
its definitely an error in the extraction process
but the fix for it was super weird
i reinstalled every rvc version, reinstalled gpu drivers, reinstalled cuda, adding to path, pretty much everything you could reinstall
nothing worked really
or you get this Could not load library cudnn_ops_infer64_8.dll even though i have them 💀
idk how i managed to fix it but if you were using some sort of modified rvc version or ilaria rvc then that might have been broken
also make sure your folder paths dont include any spaces or weird letters
loud turret
well the pitch works just fine tho, but the noise handling is what gives that robotic effect
sleek wave
this is how it sounded for me
and after the fix
is your tensorboard really messed up too?
and you used mangio rvc so something broke ig, unless you didnt miss a step
loud turret
is your tensorboard really messed up too?
not really, cuz i remember the model works just fine then all of a sudden it breaks, it's the ilaria rvc is the problem with mine
sleek wave
i thought that too but it looks like he used mangio
i wanted to train this same model with ilaria again to see if thats the issue or not but im training something else rn
loud turret
from what I remember, rvc has this 2 generator thing that are separate from each other, one is the thing that handles harmonics and the other one that handles the noise, then rvc will merge them in the final
the problem here is the noise handling I believe
sleek wave
sounds like it 🗿
loud turret
ahh I think it's a GAN's problem, it can't reconstruct audio fully. It's being talked about for years
