sturdy cape
Evening, it's my first time using Immich and it's awesome however before i realised quite what i was doing, i hadn't turned off transcoding..
I now have but obviously there's a large amount of already transcoded videos in my filesystem.. Whats the best way to get rid of these? I'm using external libraries in read only mode.
sturdy cape
if i have to just start again.. i'd rather know now lol
tribal raptor
Why would u want to turn off transcoding? Based on what i understand the purpose of the transcoded videos is to help with the performance of immich, because if it wasnt on the videos will be playing at full resolution and u might have issues with file format not being supported depending on how you access immich. Some videos wont be able to play in the web interface/app.
Immich is not overwriting your original files. It just creates a lower quality file in the right format for preview purposes.
sturdy cape
Because i don't have the storage available to store the transcoded files.
I'm aware of why it does it, it's just that right now, whilst i'm evaluating, i cannot afford the extra storage
odd panther
I'm in the same boat. i disabled transcoding after accumulating quite a large library. Newer photos/videos play fine on all devices so i assume transcoding is not required for my devices.
It would be great to have a way of cleaning up transcodes.
sand pilot
I'm in the same boat. i disabled transcoding after accumulating quite a large li...
see this answer
#immich message
odd panther
so optimistically that means "not yet" ?
sand pilot
so optimistically that means "not yet" ?
Right.
It is in progress
odd panther
lovely! i'm not running out of space in my server yet so i can wait 🙂
sand pilot
lovely! i'm not running out of space in my server yet so i can wait 🙂
If you ever run out of space consider switching to an encoder that can save space with a low impact on the file
It looks like it will [overwrite the old encoded file](#immich message) and save you some space.
odd panther
nice. i will let it run over night and see if it manages to transcode everything (~780 "videos", most of them live photos)
odd panther
i changed profile to "veryslow" and now i'm getting a lot of these errors:
[Nest] 8 - 12/26/2023, 10:51:17 PM ERROR [JobService] Error: ffprobe exited with code 1
ffprobe version 6.0-Jellyfin Copyright (c) 2007-2023 the FFmpeg developers
built with gcc 12 (Debian 12.2.0-14)
configuration: --prefix=/usr/lib/jellyfin-ffmpeg --target-os=linux --extra-version=Jellyfin --disable-doc --disable-ffplay --disable-ptx-compression --disable-static --disable-libxcb --disable-sdl2 --disable-xlib --enable-lto --enable-gpl --enable-version3 --enable-shared --enable-gmp --enable-gnutls --enable-chromaprint --enable-libdrm --enable-libass --enable-libfreetype --enable-libfribidi --enable-libfontconfig --enable-libbluray --enable-libmp3lame --enable-libopus --enable-libtheora --enable-libvorbis --enable-libopenmpt --enable-libdav1d --enable-libwebp --enable-libvpx --enable-libx264 --enable-libx265 --enable-libzvbi --enable-libzimg --enable-libfdk-aac --toolchain=hardened --enable-cross-compile --arch=arm64 --cross-prefix=/usr/bin/aarch64-linux-gnu-
libavutil 58. 2.100 / 58. 2.100
libavcodec 60. 3.100 / 60. 3.100
libavformat 60. 3.100 / 60. 3.100
libavdevice 60. 1.100 / 60. 1.100
libavfilter 9. 3.100 / 9. 3.100
libswscale 7. 1.100 / 7. 1.100
libswresample 4. 10.100 / 4. 10.100
libpostproc 57. 1.100 / 57. 1.100
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x5c252180180] moov atom not found
upload/library/chris/2023/2023-01-22/IMG_0277.MOV: Invalid data found when processing input
at ChildProcess.<anonymous> (/usr/src/app/node_modules/fluent-ffmpeg/lib/ffprobe.js:233:22)
at ChildProcess.emit (node:events:514:28)
at ChildProcess._handle.onexit (node:internal/child_process:294:12)
[Nest] 8 - 12/26/2023, 10:51:17 PM ERROR [JobService] Object:
{
"id": "68c1e2fc-c032-4d3e-ac92-4f9e6771901c"
}
I'm not sure but it looks like a bug
Maybe @frosty apex will know the answer
frosty apex
Ugh
We ran into this "moov atom not found" issue a couple of times before but couldn't really pinpoint the issue
What's interesting here is that it only happens for one transcoding setting, correct?
sand pilot
What's interesting here is that it only happens for one transcoding setting, cor...
It seems the team already knows about the bug
Maybe the OP would like to look here
#1175117734435098624 message
frosty apex
It seems the team already knows about the bug
Maybe the OP would like to look he...
You literally linked me a comment of myself lol
And no, afaik we haven't figured out why it's broken
sand pilot
You literally linked me a comment of myself lol
Oops, I meant Mertalev omment lol
frosty apex
Yeah this one seems to be different though
At least a little bit. Because in all the other cases it failed for whatever transcoding setting whereas here most of them work
@odd panther Would you be able to send an example video?
Looks like we may need to dive deeper into this next year
odd panther
it is currently transcoding my library and i will try to provide a sample tomorrow.
I don’t know if changing back to „medium“ made a difference or if it just happened to a frew files, no matter the quality. Anyways, it is eating CPU now and i will not interrupt it till tomorrow
odd panther
If you ever run out of space consider switching to an encoder that can save spac...
This worked quite well. My library is 20GB. Before, transcoded videos took additional 19GB - now after encoding everything in x265 "medium" transcode take ~6GB
odd panther
<@643095357890494494> Would you be able to send an example video?
@frosty apex I checked the logs and picked one video where it failed (IMG_0151.MOV). I think the moov atom error is just a symptom of a previos data corruption.
[Nest] 8 - 12/28/2023, 4:00:00 AM ERROR [JobService] Error: ffprobe exited with code 1
ffprobe version 6.0-Jellyfin Copyright (c) 2007-2023 the FFmpeg developers
built with gcc 12 (Debian 12.2.0-14)
configuration: --prefix=/usr/lib/jellyfin-ffmpeg --target-os=linux --extra-version=Jellyfin --disable-doc --disable-ffplay --disable-ptx-compression --disable-static --disable-libxcb --disable-sdl2 --disable-xlib --enable-lto --enable-gpl --enable-version3 --enable-shared --enable-gmp --enable-gnutls --enable-chromaprint --enable-libdrm --enable-libass --enable-libfreetype --enable-libfribidi --enable-libfontconfig --enable-libbluray --enable-libmp3lame --enable-libopus --enable-libtheora --enable-libvorbis --enable-libopenmpt --enable-libdav1d --enable-libwebp --enable-libvpx --enable-libx264 --enable-libx265 --enable-libzvbi --enable-libzimg --enable-libfdk-aac --toolchain=hardened --enable-cross-compile --arch=arm64 --cross-prefix=/usr/bin/aarch64-linux-gnu-
libavutil 58. 2.100 / 58. 2.100
libavcodec 60. 3.100 / 60. 3.100
libavformat 60. 3.100 / 60. 3.100
libavdevice 60. 1.100 / 60. 1.100
libavfilter 9. 3.100 / 9. 3.100
libswscale 7. 1.100 / 7. 1.100
libswresample 4. 10.100 / 4. 10.100
libpostproc 57. 1.100 / 57. 1.100
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x45a7e180180] moov atom not found
upload/library/chris/2022/2022-12-21/IMG_0151.MOV: Invalid data found when processing input
at ChildProcess.<anonymous> (/usr/src/app/node_modules/fluent-ffmpeg/lib/ffprobe.js:233:22)
at ChildProcess.emit (node:events:514:28)
at ChildProcess._handle.onexit (node:internal/child_process:294:12)
[Nest] 8 - 12/28/2023, 4:00:00 AM ERROR [JobService] Object:
{
"id": "56095721-d86a-4ce4-9c99-aedc9c13b3dc"
}
I downloaded it from the path in the error message and compared it to the original file i manually backed up from my phone to PC. The original is ~243 MB and the one i donwloaded from immich library is only ~29MB.
It is hard to tell for me when/how the initial corruption happened but i do remember that bigger videos needed many attempts to upload due to being stopped by iOS.
The bad thing about this is now I don't know how many uploads were corrupted.
i just recently updated immich so the container logs are just from the last 25 hours. if the transcode attempt caught all the corrupted video files, i have 39 of them
~ » docker logs immich-microservices 2>&1 | grep "Invalid data found when processing input" | uniq | wc -l
39
When you converted the videos to the new encoding, did it delete the old converted videos?
According to what I see, this is a new option that was added only two days ago and it is not included in v1.91.4
https://github.com/immich-app/immich/pull/6027
Be sure to check if the previous files still exist on the disk
odd panther
@sand pilot size went down drastically so i assume it worked. i was already on 1.91.4
trying to encode or generating thumbnails seems to be a good way of finding these "moov" errors / corrupted uploads. i just wish there was a more clear warning in the UI about these kind of failures because i could only catch them by watching the logs.