#Is this normal RAM usage for a FastAPI app?

51 messages · Page 1 of 1 (latest)

verbal bluff
#

First off, in the spirit of gratitude for Railway, I wanna say thanks because this company just saved my ass for my undergrad thesis.

So here's my problem:
I basically deployed a 225mb machine learning model as a RESTful API service (FastAPI).

I've been on the Developer Plan for more than a month now. So the 512mb cap is off, but I just wanna know if this memory usage is normal for an app like this. Even when it's idle, it's consuming 1.8GB RAM constantly.

This usage has bumped me up to $4.51 in just 9 days of December and is estimated to be $18.03 by the end of the month. I don't know if either Railway is scamming me at this point just because the 512mb cap on memory is off or if I'm just unnecessarily using too much memory in my app.

My temporary solution
I did find a way to lower it down just now which is basically just loading the model within a function instead of globally within main.py. (you can see the drop in the graph at the end there).

The result is a somewhat slower request (because the model has to load again for every request), but lower usage when the API is not being used.

I monitored this on Task Manager so it does save around 200mb. What doesn't make sense to me how it uses more memory on Railway when deployed compared to when it's running on my computer. It has lower usage on my computer. Also, the model is only 220mb, why does it have to use 1.8GB memory constantly, even when idle.

Another thing, after every request, the idle memory usage sort of increases. Is this normal for FastAPI apps? I didn't know RAM usage grows over time. Does this mean I have to restart the app every now and then so it goes back to the lower initial memory usage again?

image.png
dusky boneBOT
#

Project ID: d1329dc6-0d59-4946-bd19-62c2e54a03eb

#

It is normal for an app like this to use a large amount of memory, especially when the 512mb cap is off.

⚠️ experimental feature
verbal bluff
#

d1329dc6-0d59-4946-bd19-62c2e54a03eb

honest merlin
#

hey there @verbal bluff - this is something we are aware of. We are working on adding configurable limits so that applications don't just hover up memory

#

RE: Python runtime, Python assumes that your model calls (or anything in mem) is going to be used again so I would configure a https://docs.python.org/3/library/gc.html in your applicaiton if you need finer control over running memory.

verbal bluff
#

Thanks for the quick reply @honest merlin ! That's great to hear. I'll also read up on the garbage collector reference that you sent. Seems promising. Thanks! ✨

honest merlin
#

Anything that costs you, it should be done with good reason. I never want you to feel like you are being hosed on this platform.

#

Let me know if I can help with anything else!

sage prawn
#

Or has the solution you mentioned, already been implemented.

spring sand
sage prawn
#

Oh ok, any idea how i can control such things in the project itself?

spring sand
#

Check out the link that they sent

sage prawn
#

Would it make sense if i did a "gc.collect()" in every function? would that help my case?

spring sand
#

Not in every function, only when your app is finished processing and doesn’t need any of the data collected anymore

#

I’m not familiar with the library so I can’t give recommendations on how it can be used, but I’m sure google has some decent results

sage prawn
#

I tried doing gc.collect() just before returning a request. It still dosent help

#

Is there any way I can make a system where my project just redeploys every hour or so? I know this is not a proper solution but just a make fix.

spring sand
#

Unfortunately no, not directly with an API

#

you can use the CLI in a hacky fashion on your pc to run railway up every hour

#

not sure if you can get the railway CLI to work on railway, that would be interesting

sage prawn
spring sand
#

Yeah

sage prawn
#

Hmmmm.

#

Would it be possible to change some configuration in Uwsgi , ive seen such things on stackoverflow

spring sand
#

what sort of configuration would you be looking to change?

sage prawn
#

In railway.toml, configure it in such a way that it redeploys every hour. Possible ?

spring sand
#

No

#

When a program crashes on Railway you can set it to restart

#

You could run a cron/scheduler job that raises an unhandled exception and causes your app to crash, which would trigger a restart

sage prawn
spring sand
#

it should be set to restart 10 times by default

#

Though you should probably increase that

sage prawn
#

I think that's only for if the deployment crashes and not for the app itself crashing. Correct me if I'm wrong.

spring sand
#

Those are the same thing

#

The deployment is an instance of the app, if the deployment crashes then the app is considered crashed

sage prawn
#

Oh ok let me try that out

#

What is the difference between 'Always' and 'On failure' in restart policy. ? Can u elaborate on "Always".

spring sand
#

On failure restarts the service whenever it crashes, always restarts the service if it stops for any reason

#

eg. if it's a script that runs once and stops

sage prawn
#

Oh right, so I should set it to on failure right ?

#

And also, an idea just popped into my mind. To limit the ram usage, I thought of putting the project in a docker and limiting the ram of the docker. I don't know how exactly, but I'll figure it out.
Can you let me know if my idea makes sense?

spring sand
#

Not sure if that’s possible, many many people ask for the ability to limit memory on Railway. I’m sure that would’ve already been suggested if it was possible

sage prawn
#

Oh sad

#

Is railway developing a setting where we can limit ram usage in a project ?

spring sand
#

Yes

sage prawn
sage prawn
#

And happy new year!!

spring sand
#

No problem, glad it worked