Possible memory leak in Appwrite container

[outer vault]

Recently I've set logging in to my server and noticed that Appwrite container takes a lot of memory and it's growing over time. Looks like leaks. Are you aware of it on version 1.4.4? Maybe somehow I can diagnose it.

It is for the last 24 hours

[dire bane]

Can you share more info about your config, is your app in production or in development environment? do you have any function? how many collections? do you see any weird entry in your logs?

[deep haven]

It could grow due to Redis cache and other operations

[outer vault]

• It should be production env, I used a compose file from an install script.
• Yep. I have a few functions, but they run by a schedule (every 24 hours), they are not supposed triggered by a user
• Two collections. about 350-400 documents
• How can I provide these logs? I have a lot of these requests:

[Error] Timestamp: 2023-10-17T12:43:28+00:00
[Error] Method: GET
[Error] URL: /v1/storage/buckets/:bucketId/files/:fileId/view
[Error] Type: Utopia\Exception
[Error] Message: Invalid `fileId` param: UID must contain at most 36 chars. Valid chars are a-z, A-Z, 0-9, and underscore. Can't start with a leading underscore
[Error] File: /usr/src/code/vendor/utopia-php/framework/src/App.php
[Error] Line: 742

[outer vault]

Redis is in another container

[deep haven]

Oh, yes, that's true

Weird 👀

[dire bane]

those logs are from docker compose logs appwrite?

[outer vault]

Yep

[dire bane]

Does your appwrite instance available on public? I mean, are you getting real traffic? or is just you at this moment?

Did you migrate from a previous version?

[outer vault]

Yes. This is a real traffic. About 15-20 requests per minute according traefik stats.
Yes. I've done a few migration on it

[deep haven]

And according dashboard stats?

Are you sure you ran migration command after the corresponding upgrades?

[outer vault]

Neh. It shows 565k request for the past 24 hours.

[outer vault]

I'm sure

[outer vault]

Hello again. I performed a clean install on a separate DO Droplet yesterday and copied all the data I had from the previous server. It appears that there is still a leak.

New version is 1.4.5

[deep haven]

Are you using realtime feature?

[outer vault]

No as far as I know but I'll check it for sure

[deep haven]

Also, the requests contains images?

[outer vault]

Frequently. And videos as well. Request them from a storage bucket

[deep haven]

That makes sense

How are you getting them?

With the preview endpoint?

[outer vault]

Download/View requests. Mostly View

[deep haven]

Could you please share a fragment of the code you're using to get the images?

I think this is a caching issue 🤔

[outer vault]

My iOS client is not performing requests anymore, but guys who made Android version of my app is still in business. I'll be back with snippets later

[outer vault]

It has been used for images and videos as well

[outer vault]

I've turned on access.log from traefik and see GET requests on /view. I'll try to find out who does it

[deep haven]

That should not use cache from what I understand 👀

[slim rock]

So this is the appwrite container itself right?

[outer vault]

Yes

[amber rain]

Is this still happening? Do you know if it was happening before 1.4.4?

[outer vault]

Sorry for the long answer. I don't update the server anymore. It always causes problems and it isn't worth it. But I finally convinced an Android developer to get rid of view methods in his app and don't see any significant leaks anymore. I've made Golang server to prepare pre-signed links to data by file ID. Hope it'll help me to fix the response time of Appwrite which slows down to 4 seconds when a lot of people come onto the server

[deep haven]

Respect this:

1. There has been a lot of fixes since this issue and I could say that latest version (1.4.13) is very stable and can be used safely in any production environment, and is a way better than previous versions. It's not the same upgrading from a path version to a major than path upgrades. In other words, upgrading to latest patch should have only positive impact and improve stability.

2. This issue should not happen as said, something is wrong with your machine or appwrite/docker installation that causes the issue

3. The slowing down issue could be caused due to several factors. It could be due to the main issue in this post, indexes not set properly, a machine with insufficient resources and a lot more reasons. So in other words, this is a generic issue you could face with any backend in any language

[outer vault]

Thank you. I'll check out the change log and maybe we need to update the server. I don't blame the implementation for slowdowns. I think it is because mobile clients sync their catalogs each start and it contains pretty big json attachments to some rows. We need to update it with "update date" filtration. Also, clients download mp4 files from the storage, which should affect response time.

[deep haven]

I would instead of syncing each time, implementing some kind of caching client side so it gets updated only when it has been modified.

Also I recommend creating indexes in the database if you have a very significant amount of things, since it will speed up queries

I recommend creating another post explaining your specific use case and I could try to help you make it more optimized and faster if possible

[outer vault]

Ok. Thank you. Anyway, I think we solved the issue with leaks. I don't see any leaks for 3 days straight now.

[deep haven]

And what was the solution?

Just curious 😅

[outer vault]

Don't use "view" method. And make a cron task to check free memory every 5 minutes and restart the Appwirte container 😄

[deep haven]

I think it should not be needed to restart the container 🤔

[outer vault]

It saved me a lot of time past 30-40 days

[deep haven]

Yes, I suppose. I would try upgrading the instance. Also make a backup for in case something goes wrong during upgrade

And run the migration command after upgrading