#After update to 3.2.2, I have broken links and HTTP 500

1 messages · Page 1 of 1 (latest)

sonic compass
#

Many things don't show up ... popup saying "Failed to load assets" ... then suddenly some things work.

gilded voidBOT
#

:wave: Hey @sonic compass,

Thanks for reaching out to us. Please carefully read this message and follow the recommended actions. This will help us be more effective in our support effort and leave more time for building Immich immich.

References

#

Checklist

I have...

  1. :blue_square: verified I'm on the latest release(note that mobile app releases may take some time).
  2. :blue_square: read applicable release notes.
  3. :blue_square: reviewed the FAQs for known issues.
  4. :blue_square: reviewed Github for known issues.
  5. :blue_square: tried accessing Immich via local ip (without a custom reverse proxy).
  6. :blue_square: uploaded the relevant information (see below).
  7. :blue_square: tried an incognito window, disabled extensions, cleared mobile app cache, logged out and back in, different browsers, etc. as applicable

(an item can be marked as "complete" by reacting with the appropriate number)

Information

In order to be able to effectively help you, we need you to provide clear information to show what the problem is. The exact details needed vary per case, but here is a list of things to consider:

  • Your docker-compose.yml and .env files.
  • Logs from all the containers and their status (see above).
  • All the troubleshooting steps you've tried so far.
  • Any recent changes you've made to Immich or your system.
  • Details about your system (both software/OS and hardware).
  • Details about your storage (filesystems, type of disks, output of commands like fdisk -l and df -h).
  • The version of the Immich server, mobile app, and other relevant pieces.
  • Any other information that you think might be relevant.

Please paste files and logs with proper code formatting, and especially avoid blurry screenshots.
Without the right information we can't work out what the problem is. Help us help you ;)

If this ticket can be closed you can use the /close command, and re-open it later if needed.

sonic compass
#

Looks like memory usage climbs over a few minutes to all 32 GiB I gave the instance ...

agile karma
#

Its a known Bug #immich message

sonic compass
#

So what is my best bet?
Roll back to 3.1? Or is there a fix?
I mean ... it looks like a ton of the images are gone/corrupted 🙁

dusky scroll
#

It's just failed thumbnails

#

Rerun thumbnail generation and it should be fine

sonic compass
#

Half the time when I click on images, I get:
NetworkError when attempting to fetch resource. - 500
undefined

#

So I can't really re-run thumbnails ... I can barely even get to the administration interface without getting network errors 🙁

#

Is it safe to downgrade to 3.1?

dusky scroll
#

That sounds to me like it's still doing work and your machine is just overloaded

sonic compass
#

There are no active jobs listed though...

#

CPU isn't particularly busy, so it also doesn't look like it's doing much.

dusky scroll
#

Then we'll really need the info the bot requested

sonic compass
#

Yep hang on ...

#

Interestingly, the IOS app on my phone doesn't seem to have any issues.

#

Oh wow ... and it works fine in Edge too. But Firefox is a shitshow!

agile karma
#

Maybe cuz everything is cached? clear the cache and try again

sonic compass
#

Wow yes that made a difference.
What is that all about?

agile karma
#

the browser creates cache files so its faster when you open the website again

#

and you had older cache files, thats why it was working

#

so the problem still exist

sonic compass
#

But isn't the cached files supposed to be updated when they change on the server?

And do you have it reversed?
I had older cache files, that's why it was NOT working.

sonic compass
agile karma
#

something which @tight raven can tell

tight raven
#

I have been summoned 👀

tight raven
sonic compass
#

It looks like I'm badly affected by the node.js leak. After upgrading to 3.2.2, my server climbs to 32 GiB RAM usage in minutes.

tight raven
#

Are you currently uploading a lot of assets?

sonic compass
#

No server is idle

#

All job queues show nothing is going on.
But within a couple minutes of starting the server, RAM is maxxed.

#

For a while I thought my install was badly hosed. HTTP 500's and broken thumbnails over the place.
But then I noticed the IOS app and another browser worked fine.
After clearing the Firefox cache, it's usable again (not sure why that would be needed since I would assume the cache should expire when the files changed).

dusky scroll
#

I'm pretty sure the known leak does depend on ML processing happening

tight raven
#

Are you sure it is actually 32GB memory usage? How are you checking that?

tight raven
sonic compass
#

Hmm maybe it's not then?
Proxmox does show it using all memory allowed.
But '"free" within the VM looks sane.
So maybe it's just all cached in the VM?
Don't remember other VMs doing that...

tight raven
#

What proxmox shows is "wrong"

#

Linux likes to heavily use the RAM as a cache

sonic compass
#

But anyways ... if I was to start uploading a lot of new images, it would then truly leak - right>?

tight raven
#

Which, to external inspectors, is indistinguishable to regular RAM usage

tight raven
#

Which is why I didn't think it would be such a pressing issue

#

So I am glad you don't actually have a 32GB memory leak lol

sonic compass
#

Hmm ok ... maybe it's not then.
I got scared like hell due to all the network errors thrown all over the place.
But that seems to have gone away after clearing the FFox cache.
That seems bad too ... why was that needed?

tight raven
#

I have no idea, on my phone so can't look at the logs

#

500s are generally server issues though

#

A possible issue could be a bad setup where Immich lost access to its files

#

Regardless, I am 98% certain this is not an Immich issue

sonic compass
#

Alright ... I guess I'll leave it running and see if it starts acting up again.

#

Thanks for chiming in.

tight raven
#

You should also monitor the server logs and see if anything is happening there