Monitoring CPU and memory usage in actors | Apify & Crawlee | Page 1

umbral whale Jun 3, 2024, 3:05 PM

#

Hi, I'm develping an actor in rust and I'm trying to access resources utilisation (CPU and memory). I've seen that Crawlee uses os.cpus() from node, but I'm looking for a rust equivalent.
I can make it work locally by mounting the docker socket on the container (docker run -v /var/run/docker.sock:/var/run/docker.sock <image_tag>), but it does not work on the Apify platform.
Are there any resources/pointers I could check on how Apify runs an actor's container? And how I could read these resources utilisation?
Any help would be appreciated.

twilit flameBOT Jun 3, 2024, 3:05 PM

#

@umbral whale just advanced to level 1! Thanks for your contributions! 🎉

umbral whale Jun 5, 2024, 9:24 AM

#

Turns out I had missed this page from the docs: https://docs.apify.com/platform/actors/development/programming-interface/system-events.

The websocket sends 2 events (the second one is not documented though):

{
   "name":"cpuInfo",
   "data":{
      "isCpuOverloaded":false,
      "createdAt":"2024-06-05T09:16:37.916Z"
   }
}

{
    "name": "systemInfo",
    "data": {
        "memAvgBytes": 127389235.71674345,
        "memCurrentBytes": 168525824,
        "memMaxBytes": 257417216,
        "cpuAvgUsage": 1.655466795311761,
        "cpuMaxUsage": 115.72125786163522,
        "cpuCurrentUsage": 7.616518557239937,
        "isCpuOverloaded": false,
        "createdAt": "2024-06-05T09:16:39.092Z"
    }
}

celest frost Jun 5, 2024, 10:18 AM

#

Hello @umbral whale, thanks for pointing out the missing documentation. I will pass it on the team so we can fix it.

umbral whale Jun 6, 2024, 1:33 AM

#

Hi @celest frost , thanks that'd be great.
What seems to be missing from these stats is the total amount of available CPU/memory, is there a way to get that?
The goal would be to compute a usage % to be able to throttle the system.
Also I have no idea what "cpuCurrentUsage": 7.616518557239937 means, what unit is that?
Thanks for your help.

umbral whale Jun 7, 2024, 5:41 AM

#

Ok total amount of memory is given by env variable ACTOR_MEMORY_MBYTES, and number of CPU cores depends on the memory: "1 GB gives you 0.25 CPU cores".

So the only question remaining is what unit is the cpu usage in?

regal socket Aug 19, 2024, 2:22 AM

#

how do you find total CPU/RAM usuage?

#Monitoring CPU and memory usage in actors