#Help processing multipart/form-data

46 messages · Page 1 of 1 (latest)

tropic scaffold
#

Im trying to solve this issue i have for couple of days now. I tried everything but it just not working with http based webserver (node module)

#

So when for example i upload a file larger then 100mb the request goes trough but stops on the OPTIONS request and not going trough the POST at all
indicating something wrong with the webserver itself
i did the same thing with golang, setting up a simple webserver and processing multipart -> works fine
no nginx, no apache, no reverse proxy

# 
if (req.method === "OPTIONS") {
    res.writeHead(200, {
      "Access-Control-Allow-Origin": "*",
      "Access-Control-Allow-Methods": "POST",
      "Access-Control-Allow-Headers": "Content-Type",
    });
    console.log("OPTIONS request");
    res.end();
  }
  if (req.method === "POST") {
    if (req.url === "/evidence/add") {
      console.log("POST request");
      const bb = busboy({
        headers: req.headers,
      });
      bb.on("file", (name, file, info) => {
        console.log(`File [${name}]: start`);
        const saveTo = path.join(__dirname, "uploads", info.filename);
        file.pipe(fs.createWriteStream(saveTo));
      });
      bb.on("close", () => {
        res.writeHead(200, {Connection: "close"});
      });
      req.pipe(bb);
      return;
    }
  }
#

small files -> goes trough OPTIONS -> large files(over 100mb) stops at OPTIONS
there has to be something im missing out

#

ive tried everything
using 5 frameworks -> express, koa, nest, fastify -> tried multiple parsers multer, formidable, multiparty
ive tried it with reverse proxy module for nginx and apache

#

can anyone try recreate this issue

#

and if it works pls tell me node version and typescript version

autumn elbow
#

show your express setup

tropic scaffold
autumn elbow
#

Ok, so what is it

tropic scaffold
#

just http and createServer

autumn elbow
#

Ok, show your configuration for the server

tropic scaffold
# autumn elbow Ok, show your configuration for the server 
createServer((req, res) => {
  req.on("error", (err) => {
    console.error(err);
  });

  if (req.method === "OPTIONS") {
    res.writeHead(200, {
      "Access-Control-Allow-Origin": "*",
      "Access-Control-Allow-Methods": "POST",
      "Access-Control-Allow-Headers": "Content-Type",
    });
    console.log("OPTIONS request");
    console.log(req.headers);
    res.end();
  }
  if (req.method === "POST") {
    if (req.url === "/evidence/add") {
      console.log("POST request");
      const bb = busboy({
        headers: req.headers,
      });
      bb.on("file", (name, file, info) => {
        console.log(`File [${name}]: start`);
        const saveTo = path.join(__dirname, "uploads", info.filename);
        file.pipe(fs.createWriteStream(saveTo));
      });
      bb.on("close", () => {
        res.writeHead(200, {Connection: "close"});
      });
      req.pipe(bb);
      return;
    }
  }
  if (req.method === "GET") {
    console.log("GET request");
  }
}).listen(httpPort, () => {
  console.log(`vanilla server listening on${httpPort}`);
});
#

this just something i made for testing

# 
const upload = multer({dest: "uploads/"});

router.use("/evidence", express.static("evidence"));

router.get("/", (req: Request, res: Response) => {
  res.status(200).json({
    status: "success",
    message: "ready",
  });
});

router.post("/evidence/add", upload.single("file"), (req: Request, res: Response, next: NextFunction) => {
  console.log("adding evidence");

  evidence
    .insertEvidence(req, res, next)
    .then(() => {
      res.status(200).json({
        status: "success",
        message: "evidence added",
      });
    })
    .catch((error) => {
      res.status(500).json({error: error.message});
    });
});
#

second one is my standart express setup

#

but with multer

#

first one using busboy with http

#

both throw the same issue

#

since express is just a middleware its the http module

#

mad nodejs

autumn elbow
#

With express you normally configure the file upload limit using the limit key in both the express JSON and express.urlEndocded middleware

tropic scaffold
#

thats what they say in their docs

#

tricky right?

#

i try different node version

#

lets see

autumn elbow
#

Of course they do, they never used to , it used to be down to body parser but that got integrated into express

tropic scaffold
autumn elbow
#

2017

tropic scaffold
#

send me

# 
In Express 4, req.files is no longer available on the req object by default. To access uploaded files on the req.files object, use multipart-handling middleware like busboy, multer, formidable, multiparty, connect-multiparty, or pez.
``` @autumn elbow
autumn elbow
#

I am talking about configuring the allowed file body size by express.

tropic scaffold
#

look, the request doesn tgo trough

#

it doesnt even reach express

#

it has to go trough the http module

#

first

#

and there the error is thrown

tropic scaffold
tropic scaffold
#

i cant even catch the error

tropic scaffold
#

FIXED!!!!

#

@autumn elbow it was because of bun

#

fuck bun