Pro Plan not being exausted

[cold topaz]

I'm using a pro plan but it seems like my service only uses up to 16vcpu and 7gb ram instead of the promised 32vcpu and 32gb ram

[pine latch]

Project ID: N/A

[cold topaz]

N/A

[zenith anchor]

To be clear, the pro plan can use up to the max amount. But if your service doesn't require it, it doesn't go tht high. It only uses what it needs. That said, it is also techincally possible you set the service at a max cap in the settings. But more likely, your service just doesn't need all cores and ram (very few really do ... )

[cold topaz]

My server is a video generating service , so using more resourses should increase the speed right ? right now its slow> Any reason why it cant use up more resourses so the process is faster ?

[zenith anchor]

video gen? As in AI?
Before I give my opinion; it is really hard (alsmot impossible) for me to comment perfectly on what is holding back your program. Especially if I don't see the code / architecture. But in short, there is some bottleneck besides just "cpu" or "ram". There are many parts of a service and few programs are properly designed to actually use the full computer.
Ok, with that out of the way, if this is AI, you are going to want a GPU. And as of right now, railway doesn't have a gpu option. Not sure what the current status is on that (IE if they are leaning more on the "yes lets get GPUs" or the "no lets not" side)

[cold topaz]

Not neccessarily AI. The server takes in a bunch of images and uses ffmpeg to convert the images into videos and also add captions. All done with ffmpeg

[zenith anchor]

Makes sense! ffmpeg is an amazing program. But that doesn't mean it automatically will be able to work around physical limitations. I/O over many files is a slow operation and you are likely being limited by the speed of the disk! IE you are bottlenecked by I/O and not CPU nor ram. There isn't all too much that can be done about this. Not at least through railway. Solutions would either require faster disks or more disks (basically 1 per worker so that individuals max out the IO). The only way to get faster disks is through your own hardware ... (the railway hardware is actually pretty good compared to other cloud solutions. But technically some providers do offer faster solutions ... at a massive up-charge. Like so massive that you can buy your own server in about 3 months!!)

So what to do ... well that depends where you want to go and how you want to optimize. There are things that you likely can do to make your system faster. But just remember, that there are more boundaries than just the ones that show up on the metrics. Maybe instal other monitoring services so you can better debugg (don't have any good reconodations off the top of my head. But ask in general or another help thread and you will get some good ones)

[cold topaz]

Thank you so much for all the info. I'll do some more research on this. One more thing , Is it possible for each request i send to the railway server to be it's own instance. Instead of all the requests sharing the 32vcpu and 32gb ram resourse , i want each request to use its own 32vcpu and 32gb ram individual resourse.

I want to do this because when multiple users are using the server at the same time, it crashes.

[zenith anchor]

It’s sort of possible. You can replicate your service and have requests balanced across your replicas. This is sorta what I was getting at with the “more disks” solution. However, you might have to engineer a clever solution to be able to dynamically spin up more replicas as requests grow.

What I mean, is imagine you had 4 replicas always on. You can handle requests as they come but are paying for the idle time too. However, if your requests are growing you might want to preemptively spin up 4 more replicas to handle more requests but once they die down, take the extras down too. It costs money to have an idle replica (depends how efficient your program is on idle on if it is actually expensive here.) but It also takes time to spin up a replica. So you want to balance how many you have running vs threshold for when to make more. Or … the cost is negligible since it’s efficient at idle and you get charged for usage here … and so you just have a ton of replicas.

All to say,,, yes, it’s 100% possible. And sounds like what you need.

[cold topaz]

Seems reasonable. I guess my questions would be how much on average would i need to pay for an idle replica on the pro plan and also how long on average does it take to spin up a replica

[zenith anchor]

You will have to experiment to figure that out. It’s 100% dependent on your services.

For idle, creat a copy (not replica) and see what it idles at and then do the usage calculator to figure out monthly cost per box.
For spin up time; make a replica and just … time it. A few times to get an average.
These are the naive but good enough ways.