How to upload large files(~1 gb) through next js(maybe it's a more generic web question)?

[crisp birch]

So I am using Next Js 15(latest version) and useActionState
to submit a file with a file attachment. The file is of the aforementioned size. How do I upload it to my server. I am running aws and everything is inhouse, no AWS/GCP etc.
I was able to do it in earlier version of nextjs but somehow this upgrade added some limitations.

 GET /upload 200 in 3086ms
 ⨯ TypeError: fetch failed
    at async fileUploadAction (src\app\(loggedIn)\upload\actions.tsx:16:19)
  14 |   }
  15 |
> 16 |   const response = await fetch(benposFileUploadLink, {
     |                   ^
  17 |     method: "POST",
  18 |     headers: {
  19 |       Authorization: `Token ${accessToken}`, {
  digest: '1827344237',
  [cause]: [Error [HeadersTimeoutError]: Headers Timeout Error] {
    code: 'UND_ERR_HEADERS_TIMEOUT'
  }
}
 POST /upload 500 in 339566ms

[burnt quartz]

Post created!

🔎 This post has been indexed in our web forum and will be seen by search engines so other users can find it outside Discord

🕵️ Your user profile is private by default and won't be visible to users outside Discord, if you want to be visible in the web forum you can add the "Public Profile" role in id:customize

✅ You can mark a message as the answer for your post with Right click -> Apps -> Mark Solution
(if you don't see the option, try refreshing Discord with Ctrl + R)

[crisp birch]

So, since the file is big, in production scenario, uploading a file like bare form data will be impractical. So I looked at Streaming api in browser and also the websocket way. Now according to what I understand, the streaming API approach fails since first file is uploaded to next js backend and then my django rest server through streams since the use of streams would be in "action".

So what should be the correct way to handle it? Any examples?

[crisp birch]

https://claude.ai/public/artifacts/e3b673ad-46b5-4675-bd32-4dff9a21e3f7

is this the right approach? it's just too much code .

[dapper moss]

Why don't you send directly from frontend to a different backend

[crisp birch]

using webworkers only?

Also then why is even all this abstraction made in nextjs? After doing it a few times it's just looks like I am just creating more code

like I can submit data directly to my backend, why even use this useFormAction thing

[dapper moss]

If you've a different backend, you don't need to use backend in Next.js

I m talking about server actions

[crisp birch]

understood

my backend is django

[dapper moss]

You can do everything in django itself

[crisp birch]

i will need nextjs for ui

i know that I can now use react using cdn in templates wherever required but don't know why it feels awkward

[dapper moss]

You can like use Next.js for fetching items in server, like if you're displaying products it can be fetched

[crisp birch]

ok, I get your point

wherever it's rendering, I will use next js, but wherever it 's client wants to send data to backend, i will straight away send to django instead of nextjs node(or whatever) server

[dapper moss]

Yes, you can do that, as long as you do the necessary stuff like authentication in the backend

[crisp birch]

I do have that

the django backend is what actually mutates data so it gotta be protected

also whats the strategy for uploading a big file?

websockets or the worker pool?

is worker pool overkill?

[dapper moss]

websockets would be better

[crisp birch]

are you saying it from experience or guessing?

ok, will use websockets, thank you @dapper moss

I think the websocket will need to run in a worker

whatever async await cannot do it without that

[dapper moss]

I'm not really saying from experience in this case, but I beleive websocket would be better for uploading file

[crisp birch]

how can I join the discussion group?

maybe someone has done it before in plenty there?

[marble cypress]

Access to Discussions Channel

We have limited write access to #discussions. You need to be active in the #1007476603422527558 channel to gain write access. Learn more.

Requested by Anay-208 | Ping in replies

[dapper moss]

You can also request access for it in #discord-feedback, I'm not sure if they'll give the roles, but you can try

[crisp birch]

Thanks, I tried messaging a moderator

[dapper moss]

I don't recommend message, just post it in #discord-feedback without pinging.

[crisp birch]

Will do this then, thanks

[dapper moss]

Summary:

• You can just directly send request to backend for uploading file, since its not in Next.js
• I recommended you to use streaming for it, since its for large file

Can you mark above message as a solution

[crisp birch]

How to?

[dapper moss]

Wait @crisp birch, What do you exactly want to discuss, like just the methods, difference between webworkers and websocket, and which one is better, right?
anything else except for that in #discussions ?

[crisp birch]

Yes for now

[dapper moss]

State that in #discord-feedback

Just write something like I wanted to discuss different methods of uploading large file, like web workers or Websockets

[crisp birch]

I wrote that only

[dapper moss]

@crisp birch Leave it, I don't think they'll give access for this type of discussion

[crisp birch]

How to mark it resolved?

[marble cypress]

Mark an answer as the solution

To mark the message as solution:

1. Hover over the message you want to mark as the solution.
2. Right click the message Click the three dots that appear on the right side.
3. You'll see a menu which should have the option of Apps. Hover Over it.
4. Click on the Mark as Answer option.
   Note: If you don't see the Mark as Answer option or Apps option, restart/update your discord app!

Requested by Anay-208 | Ping in replies