#Effective PDF Download and Distribution with Cloudflare Workers

23 messages · Page 1 of 1 (latest)

astral ore
#

Hey,

I am trying to figure out the best way to download a PDF from an external API, save it to an R2 instance, and then send it to all of my users. I am currently using Cloudflare Workers to handle this process, but I'm running into issues with the worker running out of processing time.

Would switching to an unbound worker mode help with this issue? Are there any other suggestions or tips for handling this process effectively and efficiently?

Any input would be greatly appreciated. Thank you!

junior sail
#

Unbound workers will give you more CPU time, yes

#

But there's probably some optimisation that could be done in your code if you're willing to share it

astral ore
#

Sure, give me a sec

#

its a pretty rough code that used to work with a little amont ouf users, but now its running out of cpu time

#

let me get a snippet of my code

#

This is my hanlder function:

 const accounts = await fetchAccounts();
    const accounts_ids = accounts.map((account: { id: string; }) => account.id);
    
    
    // for all account id send email
    for (let i = 0; i < accounts_ids.length; i++) {
      await sendAllClientsAccountStatement(accounts_ids[i]);
    }```
#

and this is my sendAllClientsAccountStatement function

# 
async function sendAllClientsAccountStatement(account_id: string){
  
  const statement = await getAccountStatementAll(account_id);
     
  const accountData = await getAccountDataAll(account_id);
  const email = accountData.contact.email_address;
  const name = accountData.identity.given_name + ' ' + accountData.identity.family_name
  const account_number = accountData.account_number;


  
  if(statement != null){
    const message = '';
  
    await sendEmail(email, 'test@mail.com', 'Test', message)
    console.log("Account statement sent correctly to " + email);
  }

}```
#

It now it's pretty unefficient 🤣

junior sail
#

Oh this sounds like something #queues would be perfect for to do operations in smaller batches

#

Because yeah as you said, there's a lot going on there

astral ore
#

yep, I need to fetch a lot of stuff

#

Could you roughly explain a basic pipeline with queues?

junior sail
#

Your worker(s) will send items to a queue of items. On the other side, your worker has an async queue() handler (like fetch), which receives an array of messages to process. You can configure the maximum number of messages per batch, and the maximum wait time that the queue can fill up with items before sending off a batch. When the job is done processing, the next batch comes in

astral ore
#

I see, sounds like a prefect use case for my problem

#

thanks~

junior sail
#

yeah I'd definitely recommend using it for that kind of case

astral ore
#

I've been a little hesitant to use queues since they are still in beta

junior sail
astral ore
#

but I;ll give them a try

junior sail
#

I've not seen any availability issues in all the time I've had access to it

#

(which is longer than the beta)