#everything doubled!

1 messages · Page 1 of 1 (latest)

faint swallow
#

So I have imported around 400GB of photos from Google Takeout using immich-go into my Immich. They were up to 29.3.2025. All looks fine and I was doing some more testing. Yesterday I started backup of my phone photos in the Immich mobile app. Today I noticed the upload folder is around 80GB bigger! I wonder why such a big increase? The photos and videos I created since march are only about 9,5GB total!
Unfortunately my suspicion was right and I see that everything is now doubled! Photos from immich-go import are alongside backup which was done via immich mobile app and they just differ by size (because I have google photos set to save quota mode of course)!

Why does this happen and why didn't backup process just replace the same photos with the bigger files for example? That would make sense! It's really stupid to simply leave everthing doubled. Is this normal and immich-go is just incompatible with Immich? What can I do?

fickle pastureBOT
#

:wave: Hey @faint swallow,

Thanks for reaching out to us. Please carefully read this message and follow the recommended actions. This will help us be more effective in our support effort and leave more time for building Immich immich.

References

#

Checklist

I have...

  1. :blue_square: verified I'm on the latest release(note that mobile app releases may take some time).
  2. :blue_square: read applicable release notes.
  3. :blue_square: reviewed the FAQs for known issues.
  4. :blue_square: reviewed Github for known issues.
  5. :blue_square: tried accessing Immich via local ip (without a custom reverse proxy).
  6. :blue_square: uploaded the relevant information (see below).
  7. :blue_square: tried an incognito window, disabled extensions, cleared mobile app cache, logged out and back in, different browsers, etc. as applicable

(an item can be marked as "complete" by reacting with the appropriate number)

Information

In order to be able to effectively help you, we need you to provide clear information to show what the problem is. The exact details needed vary per case, but here is a list of things to consider:

  • Your docker-compose.yml and .env files.
  • Logs from all the containers and their status (see above).
  • All the troubleshooting steps you've tried so far.
  • Any recent changes you've made to Immich or your system.
  • Details about your system (both software/OS and hardware).
  • Details about your storage (filesystems, type of disks, output of commands like fdisk -l and df -h).
  • The version of the Immich server, mobile app, and other relevant pieces.
  • Any other information that you think might be relevant.

Please paste files and logs with proper code formatting, and especially avoid blurry screenshots.
Without the right information we can't work out what the problem is. Help us help you ;)

If this ticket can be closed you can use the /close command, and re-open it later if needed.

dull herald
#

Well, the pictures are not the "same" so this is intended behavior. I believe there is a duplicate detection tool within immich that may help you sort through this

faint swallow
#

well sorry, but I doubt this is how backup should work, that I need to use some tools to fix some mess! This is a problem with the software, not the user

dull herald
#

That's certainly your opinion. I'd rather photo backup software not start deleting my precious photos at random because it thinks that they are similar enough, but that's why people are allowed to have different opinions

faint swallow
faint swallow
#

now I see some photos are even tripled! Like the one in the error log I attached

lusty radish
#

@faint swallow some feedback: We're all here to help but the angry tone you tend to come in with puts people off of that

faint swallow
#

so I wonder how people using immich-go do that to prevent this situation. If I should have deleted all media from my phone prior the date when the takeout was done? Seems a bit difficult

lusty radish
#

Did you come from google photos?

#

I'm guessing you probably had the storage saver setting on, where they compress your files

faint swallow
#

I have written it all in OP

#

so no need to guess 😉

lusty radish
#

Well no, you didn't specifically mention what system you came from there

faint swallow
#

...they just differ by size (because I have google photos set to save quota mode of course)

lusty radish
#

It's not just size, google remove the exif data and such (and immich-go does its best to add it back)

faint swallow
#

yes I used the storage saver in google photos of course. The quality is still really good, so really there is no need to keep it on "original" because that would eat up quota really fast

lusty radish
#

Working all this out automatically like you want is fairly challenging. That's why we offer the similarity deduplication tool, it makes it pretty easy to clear this sort of thing up after the fact.

faint swallow
# lusty radish It's not just size, google remove the exif data and such (and immich-go does its...

some photos even have the same size (maybe because I switched it from saving to original quality).
Google Takeout keeps the metadata in JSON files and immich-go successfuly imported all this information. I see in Immich web ui that all EXIF is still there, so filename, timestamp, camera parameters, GPS is still there and matching - Immich backup should consider this as the same photo or at least ask what to do

lusty radish
#

If you take two photos at the same moment should it also consider those as the same photo then? What about files that have no meaningful exif data and so end up all seeming the same this way?

#

Like I said, this stuff is nontrivial

faint swallow
#

what do these numbers mean in the corner?

image.png
lusty radish
#

Those are stacked assets

faint swallow
# lusty radish Like I said, this stuff is nontrivial

like I said - I wonder what should people do in such case as this looks to me like pretty common scenario when people migrate from GPhotos to Immich. So if it happens to everyone that photos are doubled, then I don't understand what all of them do then 😮

faint swallow
#

seems like all the doubled photos in my case are stacked then

#

actually no - not all of them are stacked this way...but most of them it seems

#

examples. These two have the same details, but different size (as most of them) and they are stacked from some reason (similarity detection?)

image.png image.png image.png
dull herald
#

This link may be of interest to you

image0.jpg
faint swallow
#

yes I will hope that this tool will fix that somehow, but first I try to understand it

#

I didn't find any documentation about this stacking, so I wonder how it works and what exactly it means and how can it be used in the UI

dark zenith
#

probably immich-go stacked them based on Google metadata.
the truth is, as it was explained, the files stored in Google are not the same binary files as your originals, even if you have original size enabled in G, they are modified.
Immich relay on binary comparisons to prevent duplication, a minor difference in the file (like stripping metadata) will make them different.
so the only way is to use the deduplication tool to compare by similarity, and that requires human intervention

faint swallow
#

immich-go couldn't stack them because there was only ONE of each in the Google Takeout folder (I just checked few example files). I don't keep doubles of the photos with the same filename (not sure how I would even be able to do it). The doubling and stacking happened last night during the mobile app backup to my Immich server.

#

yes I can imagine if I would have Google Photos set to save everything in original quality, then this would not have happened probably, but I would be paying probably double the money to Google for bigger quota 😆

faint swallow
atomic halo
#

If you haven't done that yourself it must be coming from Immich-go/gphotos

atomic halo
faint swallow
#

well this would make sense but as you can see this is not what happened and those assets got doubled :-]

atomic halo
#

That exact asset you posted the logs of cannot have been duplicated

dark zenith
faint swallow
atomic halo
#

That is not what I said

#

As has already been mentioned, those aren't exactly the same assets

faint swallow
#

if I search for it via filename I find it three times!

image.png
atomic halo
#

Sure

faint swallow
#

seems like 2 of those three are stacked

#

so I see, many of the photos are not doubled but tripled. On the timeline I see one copy and then two other copies stacked 😮

faint swallow
atomic halo
faint swallow
#

again - if you don't want to actually understand what I tried to do and read my OP, then no need to comment. Thank you. I still wonder how other people migrating from Google Photos deal with this...

#

about the duplicity finder - there is no documentation explaining what should be the result of this task. Does it somehow choose the best photo and deletes the rest or what should I expect?

dark zenith
faint swallow
#

the task finished and NOTHING has changed.
Size of the upload folder is exactly the same and all the doubled photos are still there. What is the purpose of this task? I also checked the container logs and can't find any string dupl or dupe, so I wonder how it's meant to work 😮

image.png
lusty radish
#

This task just finds the potential duplicates

faint swallow
#

OK thx. It might be good to update this documentation page to say that result of this task is in menu under tools.
https://immich.app/docs/administration/system-settings#duplicate-detection
now I only need to click 20830 times 😂 Good think I already have an AutoIt script for such situations... bad thing that I will not be able to use my computer during that procedure

#

I have a suggestion for this tool: add there some general rules for processing the results. For example

  • keep all higher quality images and delete lower
  • keep all lower quality images and delete higher
  • keep all images from external library and delete others
    That would make it really good for such situations
atomic halo
#

keep all images from external library and delete others
This it doesn't support (currently) though

faint swallow
#

OH you know what - at first glance I did not even see the buttons on top! They are not colored and it really seemed to me I will have to decide each pair.

atomic halo
#

Yeah that is fair. It's the typical design language of the app (many pages have buttons there), but if you're (relatively) new it's easy to miss :)

faint swallow
#

I might rather like if the default behavior was the opposite then what is is now - delete higher file size 🙂 Because I kind of like how Google shrinks the file sizes and it really makes things faster in case I am presenting photos to someone and not have the greatest internet or have capped data, when the photo is only 2,5MB instead of 5MB 🙂
Unfortunately mobile camera apps usually don't allow to change JPEG save quality, so it is this ridiculous 95% or something like that which is creating stupidly big files for no real reason. When I was downloading photos to my PC I usually bulk converted it to JPEG 68% wich is totally fine and you won't see a difference even at 100% zoom...

atomic halo
#

and it really makes things faster in case I am presenting photos to someone and not have the greatest internet
We usually don't show you the original anyways

#

We generate both low- and high-res previews, both of which are smaller

faint swallow
#

if I am browsing the photos on my phone in full screen then what size is being sent to the device? If it's not local file

atomic halo
#

There's a setting in the mobile app to force originals, if that's off the high resolution preview

faint swallow
#

well this section about previews says this is used when viewing one item and the default resolution is 1440p. But what if I am using a PC with a 4K monitor? Seems to me this settings is just for mobile app?

atomic halo
#

There are also phones with >1440p resolution

#

This applies to all platforms

#

If you want 4k, you should change it

faint swallow
#

so there is no differentiation for viewing photos via app or web?

#

I mean it's a shame I only get 1440p versions of the photos when server is just in the other room connected via gigabit LAN :-]

atomic halo
atomic halo
#

Live transcoding based on your network would definitely be slower here though

neat peak
faint swallow
#

well OK, but I am viewing photos on my phone and my PC as well, so it's kind of not practical to keep changing the settings. On the phone 1440pp is really better as it's faster and also saves battery, but on a PC the opposite.
I think it would be great to add some connectivity / platform dependent dynamic settings to this

atomic halo
atomic halo
#

Adding another one just takes up more extra space

faint swallow
#

I think some checkbox like "preffer original photos when using web interface viewer" would be great

atomic halo
#

And that toggles the user setting Aviv linked?

neat peak
atomic halo
#

Or only for a specific asset?

faint swallow
#

definitelly it would be silly to create 8Mpix preview versions of 16Mpix photos... waste of disk space

spare rain
#

How I went about it:

  • created a Google Takeout archive
  • at that point I disabled Google Photos from automatically uploading new assets
  • in Google Photos I used "Free up space on this device" which deletes all local photos that are backed up into Google Photos
  • enabled automatic sync in Immich
  • after the archive got created, I imported it using immich-go
faint swallow
#

finally some real world example! Thanks a lot! I think this should go to the official documentation as a preferred procedure for migration from Google Photos!

#

the only problem is that Google Takeout creation takes a long time so it might be difficult to not be able to manage your photos via Google Photos web UI on a PC during that time when backup is disabled, if you are used to it 🙂