#Help with mv to thousands of files

1 messages · Page 1 of 1 (latest)

shut dew
#

Trying to figure out what is causing my computer to hard reset while doing large mv.

I mapped some directories(and datasets) incorrectly in a debian installation of truenas and need to move thousands of movies and tv shows to an appropriate directory to support hardlinks.

This isnt a huge problem, just time consuming, I started to mv /mnt/NAS/Data/Media/movies/* /mnt/NAS/Data/plex/movies and it starts fine. However after an hour or two if i check on it the ssh connection will be closed, it has hard reset twice in the process. I didnt think the process would be so resource hungry that it would crash my machine, but it seems to be the case.

is there a better way to move ~35TB of data( i dont have the space to directly cp the entirety of the data)

or what resource would i be draining so much that it would be causing the crash? i was thinking my PSU might be a bit low on wattage for the amount of drives i have (700W for 13 drives, 12hdd one m.2) or maybe it was hogging too much RAM I only have 16GB of Ram in there but i feel like even if RAM was capped ive never heard of a pc crashing due to not enough RAM.

blazing estuary
#

The problem here lies to do with copying over the network. If the said source and destination is on the same machine, this wouldn't have been an issue with it.

shut dew
#

the destination and source are on the same machine, i only ssh because the machine runs headless.

blazing estuary
#

Then the task should have been backgrounded or made non-interactive.

#

System journal/log may tell the tale, and if you do believe it is a PSU issue, then either limit the amount of drives that are connected to it, or at least the more power hungry components, and/or consider upgrading the PSU or even buying another PSU.

fiery lantern
#

for the move itself I'd recommend opening up a screen or tmux session so you can reconnect to it if need be, also I prefer rsync when dealing with massive amounts of data, even when all on local drives

steel ferry
#

mv itself shouldn't an issue. You can try it in a tmux session if the move itself is just taking so long that the ssh session gets timed out. Otherwise you can also do it incrementally by moving them in batches

shut dew
#

i appreciate all the advice guys. i will follow it and see how it goes. Thank you