#Deferred callback not updating

399 messages · Page 1 of 1 (latest)

high heath

I wanna do this:
Defer > Do something > edit original callback to a message

However, it only defers, doesn't edit later, no error somehow...

I already tried awaiting stuff and checking the docs again.

Code:
Note, that I wrote my own kind of "wrapper". However, it is very simple.

is it, because I don't return a response directly? I'm using CF Workers and I can't send a response and continue afterwards.

crystal tinselBOT
wheat prairie

try placing some logs in between just in case

also why not use /core?

high heath

Because I got no client.

high heath
wheat prairie
high heath

This an http interactions bot.

So why would I need /core?

wheat prairie

better DX, that's the only reason behind it

since you can still use /rest (and /ws but you're not using it) directly, similar to what you're doing

/core is pretty much just for better DX

high heath

Ah, okay. I will give it a shot later 👍

high heath

welp, it errors because The "path" argument must be of type string or an instance of URL. Received undefined

The error derives from the /ws package.

wheat prairie

that's the error of your original issue or your attempt of using /core?

high heath
wheat prairie

yes to which?

oh, the latter

high heath

uhm, what?

wheat prairie

but from your code looks like the latter

high heath

oh, well this was my first attempt using /core.

I'm not sure what you mean by "latter" 😅

wheat prairie

I wanted to ask whether you found out that was the error of your original issue (callback not updating), or it was an error you encountered when trying to add /core

high heath

oh, the last one. It occured when I was trying to add /core.

wheat prairie

yeah that seems kinda weird, specially the stacktrace

high heath

specifically, after I implemented it and tried to deploy.

wheat prairie

I'm not sure how cloudflare workers works, are you running ts or js? since it mentions tsup there

high heath

ts

wheat prairie

oh, actually it looks like it was raised internally already

I'll ask whether there's a known solution

high heath

sure ^^ thx

wheat prairie

and it's specifically an issue when deploying through cf workers

violet quartz

certainly not, considering you're never hitting that code path

wheat prairie

there's bundled code that tries to replicate __dirname, but it errors since import.meta.url is undefined in cf workers

the reason was known, just not a fix

high heath

yikes

violet quartz

no actually, I did not know that's what it was

im not really convinced that's it (i.e. that its something we can fix in regards to our tsup setup)

wheat prairie

that's what NyR mentioned on his report

violet quartz

considering I have a worker bot using core/http-only that works

the difference is that I don't bundle

high heath

update: When I use API from /core/http-only then it shows a different error:

Uncaught Error: Disallowed operation called within global scope. Asynchronous I/O (ex: fetch() or
connect()), setting a timeout, and generating random values are not allowed within global scope.
To fix this error, perform this operation within a handler.
https://developers.cloudflare.com/workers/runtime-apis/handlers/
  at null.<anonymous>
(file:///Users/lucahent/Private/projects/vote-handler/node_modules/.pnpm/@discordjs+rest@2.6.0/node_modules/@discordjs/rest/src/lib/REST.ts:97:21)
in setupSweepers
  at null.<anonymous>
(file:///Users/lucahent/Private/projects/vote-handler/node_modules/.pnpm/@discordjs+rest@2.6.0/node_modules/@discordjs/rest/src/lib/REST.ts:84:8)
in REST
  at null.<anonymous> (file:///Users/lucahent/Private/projects/vote-handler/src/index.ts:13:14)
  [code: 10021]
violet quartz

oh you weren't even importing from http-only lol

is that local?

high heath

nope

violet quartz

yeah its because you're creating rest at the top-level

high heath

oh, my bad

violet quartz

though thinking about that

violet quartz
high heath

well, that did it.

violet quartz

yeah, if you import from /http-only there really shouldn't be issues unless you have your own weird bundle step that breaks our builds

wheat prairie

doesn't look like there's mention of http-only anyways kek

violet quartz

where is that

high heath

that domain is funny

wheat prairie

that's on npm

violet quartz

thanks, its not mine

high heath
violet quartz

tough one

high heath

funny one thinkLul

violet quartz

is it deferring in-client?

high heath

yep

violet quartz

sounds like our promise isn't resolving then

let me try to glance at the source code for that

high heath

sure ^^

should I provide a signal btw?

violet quartz

i mean you could but this seems like a weird bug of some sort

so wait, does this happen locally too?

or just when deployed

high heath

didn't try locally yet because I didn't set up ngrok yet.

that would take some time

wheat prairie

cf tunnel maybe?

high heath

let me have a look at that

violet quartz

now, there's a few ways to tell what's happening

really the main async op here that I suspect is possibly hanging is makeNetworkRequest

high heath

cloudflare tunnel seems like it's harder to set up than ngrok. I will do that, but not right now.

violet quartz

fortunately that emits information in its body that can tell us if its really that

try listening to the Response event on your REST instance and see if it emits

high heath

will do 👍

violet quartz

if it doesn't then our culprit is manager.options.makeRequest

which in your case should be undici.fetch after tracking this down 🙃

wheat prairie
violet quartz

its fine

they don't need to

high heath

I added this listener:

rest.addListener("response", (request, response) => {
  console.log(`[REST] ${request.method} ${request.path} -> ${response.status} ${response.statusText}`);
});

It doesn't log anything. not even listener type "restDebug" logs something.

glacial turret

cf has native fetch

oh we don't special case cf

so yes undici request

high heath

so did I find a bug or is that intended? 😅

glacial turret

actually...can you help me with that

high heath

with what exactly?

glacial turret

if you bundle with wrangler and look at the output, does it pickup the web bundle of @discordjs/rest or the main export

violet quartz

but web is main as far as i can tell

    "exports": {
        ".": {
            "node": {
                "require": {
                    "types": "./dist/index.d.ts",
                    "default": "./dist/index.js"
                },
                "import": {
                    "types": "./dist/index.d.mts",
                    "default": "./dist/index.mjs"
                }
            },
            "default": {
                "require": {
                    "types": "./dist/web.d.ts",
                    "default": "./dist/web.js"
                },
                "import": {
                    "types": "./dist/web.d.mts",
                    "default": "./dist/web.mjs"
                }
            }
        },
        "./*": {
            "require": {
                "types": "./dist/strategies/*.d.ts",
                "default": "./dist/strategies/*.js"
            },
            "import": {
                "types": "./dist/strategies/*.d.mts",
                "default": "./dist/strategies/*.mjs"
            }
        }
    },```
high heath
violet quartz

god knows why

glacial turret

we never use undici.fetch

high heath

😂

violet quartz

what do you mean

what is this then

glacial turret

it's either global fetch or undici.request

violet quartz

oh you're right

that's a cast

jesus

k well

that's not good

high heath

Should I add setDefaultStrategy(fetch) somewhere to test it?

glacial turret

not a bad idea

high heath

top-level or inside the fetch handler?

actually, I can't - it's not exported

violet quartz

so, like, why would the global workers fetch hang is my question

glacial turret

that's why I want to know if it picked up the right bundle

but also that's funny

we should probably export that

violet quartz

well we can still figure it out

just uh, do this top level

high heath
violet quartz
const hehe = globalThis.fetch;
globalThis.fetch = (...args) => {
  console.log('hi!');
  return hehe(...args);
};
glacial turret

can't you wrangler build and it writes it to disk but doesn't deploy?

high heath

I will try

glacial turret

It's been a while so I can't remember

high heath

yep, got it

glacial turret
high heath

I now got like 30k lines of code 👀

violet quartz

since if hi is logged we know it's invoking the global fetch

i.e. tells us what import path it resolved from rest

without you having to inspect the bundle

high heath

ah, okay.

give me minute

silk sleet

I eventually ended up not using /core but even with direct api calls, it was hanging after defer for me too (no idea why, as I didn't have time to investigate). So i ended up returning a Response (as a defer) early in the code and wrapping rest of the logic im waitUntil. And it worked

violet quartz

i wonder if it has something to do with the empty http response

high heath

it doesn't log "hi"

violet quartz

lol uh ok

glacial turret

oh interesting

violet quartz

so undici.request?

glacial turret

that explains a lot

violet quartz

I def. don't expect that to play well with cf... yeah

okay, so try importing from @discordjs/rest/web now

high heath

that doesn't exist

well, it does it just doesn't find the module

glacial turret

yeah, we don't actually have an exported web strategy (where the pattern matching looks for it)

violet quartz

dead

losing my mind

glacial turret

who wrote this code...it's missing so many obvious things

oh wait

I'm really suprised we haven't seen any of this come up

high heath

however @discordjs/rest/undiciRequest is exported lmao

glacial turret

yeah, it's the only file in the strategies folder

looks like wrangler looks for the workerd export key

high heath

Thonkang

I guess it's nothing I can do at the moment, except running a fetch() manually?

glacial turret

I can get a PR up fixing like....all of this in a few minutes. We can either publish that pr or push a full release pretty quick

high heath

that would be great! 😄 but only if you got the time

glacial turret

would you be okay updating to a dev version of rest 3 or should I do rest 2.6 first?

violet quartz

im def curious as to why undici.request hangs particularly on a defer call

high heath

I'm fine with everything as long as it works

violet quartz

since it seems like its otherwise working

but it's ill-advised to use it in workers anyway soo

high heath

the monorepo works btw

vivid lintel

surely not node...

glacial turret

probably main

violet quartz

yeah but that's weird ckohen

because again

glacial turret

wait

violet quartz
        ".": {
            "node": {
                "require": {
                    "types": "./dist/index.d.ts",
                    "default": "./dist/index.js"
                },
                "import": {
                    "types": "./dist/index.d.mts",
                    "default": "./dist/index.mjs"
                }
            },
            "default": {
                "require": {
                    "types": "./dist/web.d.ts",
                    "default": "./dist/web.js"
                },
                "import": {
                    "types": "./dist/web.d.mts",
                    "default": "./dist/web.mjs"
                }```
glacial turret

right

violet quartz

like, default is web

so it def seems to be going /node

high heath
violet quartz

which is

quite Insane

vivid lintel

but shouldn't it still use globalThis.fetch? We have that check in node for shouldUseGlobalblahblah

glacial turret

only if process is not defined or if we're in deno or bun

vivid lintel

i thought that also checked runtime

oh

violet quartz

OMEGALUL

vivid lintel

right, i only did proper checks in getUserAgentAppendix

meguFace

glacial turret

anyways, should I just do this in main and leave 2.6 in the dust or....

violet quartz

bump on when I pinged you guys in this thread btw, should we write docs on how to disable sweeprs for worker-like environments or are we winding up working on actual platform detection to do it automatically?

vivid lintel

we can backport it no?

glacial turret

we can

violet quartz

id say do it on main first

glacial turret

I feel like the web bundle should maybe not sweep by default?

that sounds reasonable

cause realistically the web bundle is for anything non-node

violet quartz

ah yeah. we can just decide at the bundle level

sounds good, I can do that later

or you can cover it in your PR

high heath

but even if sweepers are disabled, the issue with the globalThis.fetch still persists. right?

violet quartz

yeye, that's a diff topic

high heath

ah

violet quartz

the sweepers are a waste of compute for one in this context, and for two prevented instantiating REST as a global, which is arguably less of a big deal

glacial turret

@high heath for funsies can you search that bundled output for setDefaultStrategy and see which call it was making, just to make sure we are fixing this right

high heath

well... That ain't helpful is it? 😅

whoups, missed one

glacial turret

ok now i'm confused

@violet quartz this weird

high heath

I could also upload the full bundled file if you guys like

violet quartz

right. I guess that runs before updating globalThis.fetch

glacial turret

the only thing I can think of is that it's not getting to fetch

OH

high heath

nvm it does I was just blind

violet quartz

lol

glacial turret

that makes sense

violet quartz

yeah uh

if you do it before the imports?

well whatever

this poses a bigger problem now anyways

now that we know its not undici

glacial turret

the other thing is....defered resposne doesn't hit burst

only initial response does

violet quartz

oh

well yes sure but

we already established where it hangs

because no Respone event is emitted

high heath
violet quartz

yea its fine

high heath
glacial turret
violet quartz

yes but

consider the following

they said that the bot does show "[...] is thinking"

so we def do invoke fetch

we just fail to resolve the response

glacial turret

the defer is fine

the more stuff is fine

it just doesn't edit right?

or does the more stuff never happen?

violet quartz

no, they said the log right after defer doesn't show

the call to .defer hangs

glacial turret

ohhh

violet quartz

yeah

high heath

exactly

violet quartz

because if it was something later you'd see Response

glacial turret

I misunderstood, thought it was after

violet quartz

and if it was something earlier you wouldn't see feedback in discord

so the question is like

what the hell

high heath

discord issue?

violet quartz

nah

high heath

well, I didn't explicitly state "with_response" - is that an issue?

violet quartz

well, that seems to be the issue but it shouldn't be an issue, if that makes sense

like, that was my theory from the start that whatever network request mechanism we were invoking, was hanging due to empty response

but it's quite important to figure out why that happens

glacial turret

if you try it with that and it works then we're onto a discord bug that's pretty high priority

violet quartz

why would it be a discord bug though ckohen

because like

if they were failing to respond to the request

it should eventually hang, no?

violet quartz

you'd see a promise rejection in like

30s or whatever it is

glacial turret

how long can a worker wait for responses

I guess async stuff is longer

violet quartz

right, I guess that could miss-lead us

glacial turret

so either cf bug or discord bug, discord does send 204 back though right?

high heath
high heath
violet quartz

ye its 204 im p sure

glacial turret

then not discord bug

violet quartz

we can try a few things w/o with_response

silk sleet

This was happening to me with workers native fetch too. I'm not sure this is d.js bug specifically

violet quartz

to better understand why it happens

high heath

it currently does not respond at all, give me a moment

glacial turret
violet quartz

okay yeah

this is either like

a bug or like

i dont even know

something to do with the params we pass

high heath

could it be because I user itty-router and that thing has cors? (even though I don't think I'm using that)

violet quartz

this is looking like a cloudflare moment for sure

glacial turret

can you send that bundled file, the only thing left to check is if something else is overriding fetch

otherwise it's cloudflare

high heath

seems like it.

glacial turret

or it's the good old classic global ratelimit that hits cloudflare workers all the time

but defer somehow processes before the ratelimit code?

violet quartz

no, its not that

they have a raw fetch call

high heath

the bundled file (very big)

violet quartz

that's not waiting on a ratelimit

the promise would resolve and status would be 429

glacial turret

true

violet quartz

and also they see feedback in-discord

high heath

I do

violet quartz

so like, out of curiosity

do other commands where you don't defer work?

just as a sanity check

because im sort of losing my mind at the idea that even with_response=true leads to this

high heath
violet quartz

and to be clear, that just uh

channel message, right?

standard reply

high heath

Yep

violet quartz

lol

back to the theory that discord is responding with some weird stuff

high heath

I did native fetch() - that also hangs...

glacial turret

I just confirmed that native fetch is in fact native

violet quartz

the next logical step here is like

run a defer call locally, with cURL or something

glacial turret

it's not being overriden by anything

violet quartz

set up your own API that returns something similar

call it in workers and see if it hangs

so we at least identify what sort of response payload causes it

wait so ckohen

in node we use undici.request by default right

glacial turret

yea

violet quartz

global node fetch presumably doesn't have this bug anyway

mmh

high heath

Uuuuh how could I do that? I mean, I could easily add an endpoint to an existing app of mine which returns something similar.

violet quartz

i think its just llike, either the CF implementation isn't respecting the spec in regards to something

or Discord isn't respecting the spec with their response

and CF isn't handling it as nice as other fetch impls

its extremely unclear to me what this is about though and why its specifically only defer regardless of with_response=true/false

to me that suggests its maybe some header or something like that causing something weird

cf has their own logs right? which includes your outgoing requests?

can you try looking for that in the dashboard

high heath

But in d.js (gateway event stuff) it works.

I did but I didn't see anything there. I might also be blind. Give me a moment to send you a screenshot.

glacial turret

The best thing I can think of is to run this in workerd somehow and see if it works there

then it would use undici's fetch impl

because they don't rebake their own (other than to put a little sugar on top)

if it happens there then....

but i'm suspecting it won't

high heath
high heath
glacial turret

yeah it's intense

I wonder if cloudflare is killing the worker before it finishes processing the fetch response

like it's somehow not actually awaiting it?

high heath

that could be an issue, I had to use ctx.waitUntil

glacial turret

Is this all inside the wait until?

high heath

not that I'm aware of. I'm just checking something.

glacial turret

Alright well I’m gonna go to sleep, I will end up Pring some of the things we talked about, but since it is picking the right entry point I’m okay leaving it for tonight as they wont fix this

high heath

Because I use itty-router, I don't have the option for waitUntil.
However, I implemented it here and it finally got me some logs!

I wanna throw something out of the window

glacial turret

In guessing the errors are because it returns the response first and discord doesn’t like it

high heath

mind the timestamps, it's is

  • second
  • first
glacial turret

Could you try this same thing but without wait until

high heath

sure

glacial turret
high heath

yup, same issue persists - doesn't log anything

glacial turret

Oh wait, did you await the promise when you moved it out of wait until

high heath

I removed the promise because it now lives directly in fetch (it's just that waitUntil wants a Promise and a simple async function didn't work)

glacial turret

Gotcha

high heath
glacial turret

For shits and giggles, try adding the promise back in? I doubt it’ll work but who knows

high heath

will do later, work is calling

high heath

Update: that doesn't change anything

high heath

I'm wondering, what do I have to

  • return to the http interaction request from discord?
  • send to the callback api route?

Is there a difference?

Actually, I found the answer:

If you send this request for an interaction received over HTTP, respond to the original HTTP request with a 202 and no body.

However, this throws me off. The fetch() is still not resolving - I have to send a callback first and then respond to the http-interaction with a 202 but the callback is not resolving...

Update: even with the normal reply it doesn't work.
It is, as if it is a cf workers issue and not djs related...

I'm now using hono, because I really like hono, and even with that it seems that it's not resolving...
The code below shows my new logic (reply -> editReply)

high heath

Last update for now: fetch() with the method POST seems to be not not resolving at all.
I updated the code and now it creates a channel message and logs the response data - except it doesn't.
A fetch call before the whole thing with method GET resolves, but with method POST it doesn't.

I guess the only way now is to set up cf tunnels or ngrok so I can test it on my machine. If it works on my PC, then it's a CF Worker issue. If not, must be discord or node related (what I don't think though).

high heath
violet quartz

k, so its just inherent

case closed i guess, though it raises some questions on what we should be doing

high heath

I think Discord should change their docs and remove the cf worker example for interactions then.

I think I'm gonna make an issue in the discord api docs repo.

my god I even had a typo in there. now it works when I use ngrok as a tunnel...

_ _
_ _
do you guys got any other idea on how I should deploy my app then?

violet quartz

how exactly that is going to work is a bit unclear to me right now, we might need to redesign our API a bit in terms of how the networkRequest function is defined, since we always need to be given the context

high heath
violet quartz

yes, that's what I'm saying

but I think we can make that work

high heath
violet quartz

in fact, you have a temporary fix immediately, because the function we use internally can actually be provided into the REST constructor

the stuff with the environments we were looking into is just for defaulting

you probably just need to pass in ```ts
async makeRequest(...args): Promise<ResponseLike> {
// use ctx.waitUntil with fetch here
}

and that should fix it

im just saying we can probably provide a nicer way for us to do this in the future, since we're deliberately trying to support those serveless environments

high heath

I will try this, but I'm not convinced that'll do

actually, waitUntil is for something different. waitUntil is for extending the lifetime of your Worker beyond the initial response.
Unlike regular async operations that must complete before returning a response, waitUntil allows you to perform background tasks after sending the response back to the client.

I doubt that suits the purpose then

I now did this and it doesn't work:

async makeRequest(_url, _init): Promise<ResponseLike> {
  // Execute the fetch and let waitUntil handle the background logging
  const fetchPromise = fetch(_url, _init as RequestInit).then((response) => {
    console.log("Background fetch response:", response.status, response.statusText);
    return response;
  });

  // Use waitUntil to ensure the logging completes (c.executionCtx because hono)
  c.executionCtx.waitUntil(fetchPromise.catch(() => {}));

  // Return the actual fetch promise
  return fetchPromise;
}

Note that waitUntil can't actually return anything because it's considered a background task.

I FOUND THE SOLUTION

ctx.waitUntil(
  new Promise(async function (resolve) {
    // Do deferr or reply and it's non-blocking
    return resolve(undefined);
  })
);

return Response.json({}, { status: 202 });

This works now because we accept the interaction first, but then in the background we do something in that promise.

vivid lintel

That is so nasty wtf