unborn wigeon
I get the following errors when trying to run a function with Puppeteer library using the Bun 1.0 runtime.
An internal curl error has occurred within the executor! Error Number: 6. Error Msg: Could not resolve host: someId\nError Code: 500
An internal curl error has occurred within the executor! Error Number: 104. Error Msg: Connection reset by peer\nError Code: 500
Full code below:
Full code:
import axios from "axios";
import { execSync } from "node:child_process";
import puppeteer from "puppeteer";
let installed = false;
// This is your Appwrite function
// It's executed each time we get a request
export default async ({ req, res, log, error }: any) => {
// Why not try the Appwrite SDK?
//
// const client = new Client()
// .setEndpoint('https://cloud.appwrite.io/v1')
// .setProject(Bun.env["APPWRITE_FUNCTION_PROJECT_ID"])
// .setKey(Bun.env["APPWRITE_API_KEY"]);
// Init!
log("Function Invocation Started");
// You can log messages to the console
log("Hello, Logs!");
// If something goes wrong, log an error
error("Hello, Errors!");
// The `req` object contains the request data
if (req.method === "POST") {
try {
// Assuming the body is already parsed as JSON
const { url } = JSON.parse(req.body);
// Validate the URL
if (!url || typeof url !== "string") {
return res.json(
{ error: "Invalid or missing url ", data: url },
400
);
}
log(`Fetching HTML content for: ${url}`);
// Use Axios to fetch the HTML content
const response = await axios.get(url, {
// Axios config: Receive response as a string
responseType: "text",
});
if (response.data) {
// Return the HTML content
return res.send(response.data);
} else {
if (!installed) {
try {
log("Installing Chromium...");
execSync(
"apk update && apk add chromium nss freetype harfbuzz ca-certificates ttf-freefont",
{ stdio: "inherit" }
);
log("Chromium installed successfully...");
installed = true;
} catch (installError) {
log(`Error installing``````Chromium: ${installError}); return res.send( Error installing Chromium: ${installError.message}`
);
}
} else {
log("Chromium already installed.");
}
try {
const browser = await puppeteer.launch({
headless: true,
executablePath:
process.env.PUPPETEER_EXECUTABLE_PATH || "chromium-browser",
args: ["--no-sandbox", "--disable-gpu", "--disable-dev-shm-usage"],
});
log("Puppeteer launched successfully.");
const page = await browser.newPage();
log("Browser page opened.");
const response = await page.goto(url, {
waitUntil: "networkidle0",
});
log(`Page loaded with status code: ${response.status()}`);
const content = await page.content();
log("Content retrieved.");
await browser.close();
log("Browser session closed.");```
if (content) {
return res.send(content);
} else {
return res.send("No HTML document was found for the given URL");
}
} catch (error) {
error(`Error during Puppeteer operations: ${error}`);
return res.send(`Error: ${error.message}`);
}
}
} catch (err) {
error(`Error fetching document: ${err.message}`);
return res.json({ error: `Could not fetch document: ${err.message}` }, 500);
}
} else {
// Method not allowed
return res.json({ error: "Method Not Allowed" }, 405);
}
};```Image of executions
unborn wigeon
For anyone encountering this issue on Bun, try Node. It' could just be that Bun isn't as reliable as its touted to be.
Full code that actually works a lot better than first having to install the browser and you only need the core functionality this is the full code in parts:
import axios from "axios";
import chromium from "@sparticuz/chromium-min";
import puppeteer from "puppeteer-core";
async function getBrowser() {
return puppeteer.launch({
args: [...chromium.args, "--hide-scrollbars", "--disable-web-security"],
defaultViewport: chromium.defaultViewport,
executablePath: await chromium.executablePath(
`https://github.com/Sparticuz/chromium/releases/download/v116.0.0/chromium-v116.0.0-pack.tar`
),
headless: chromium.headless,
ignoreHTTPSErrors: true,
});
}```// This is your Appwrite function
// It's executed each time we get a request
export default async ({ req, res, log, error }) => {
// Why not try the Appwrite SDK?
//
// const client = new Client()
// .setEndpoint('https://cloud.appwrite.io/v1')
// .setProject(Bun.env["APPWRITE_FUNCTION_PROJECT_ID"])
// .setKey(Bun.env["APPWRITE_API_KEY"]);
// Init!
log("Function Invocation Started");
// You can log messages to the console
log("Hello, Logs!");
// If something goes wrong, log an error
error("Hello, Errors!");
// The `req` object contains the request data
if (req.method === "POST") {
try {
// Assuming the body is already parsed as JSON
const { url } = JSON.parse(req.body);
// Validate the URL
if (!url || typeof url !== "string") {
return res.json(
{ error: "Invalid or missing url", data: url },
400
);
}
log(`Fetching HTML content for: ${url}`);
// Attempt to fetch using Axios
const axiosResponse = await axios.get(url, {
responseType: "text",
});``` // Check if the Axios response is not empty
if (axiosResponse.data) {
// If Axios fetch was successful and not empty, return the data
return res.send(axiosResponse.data);
} else {
// Axios response is empty, use Puppeteer as a fallback
const browser = await getBrowser();
const page = await browser.newPage();
await page.goto(url, { waitUntil: "networkidle0" });
const puppeteerContent = await page.content(); // Get HTML content with Puppeteer
await browser.close();
// Return the HTML content fetched by Puppeteer
return res.send(puppeteerContent);
}
} catch (err) {
error(`Error fetching document: ${err.message}`);
return res.json({ error: `Could not fetch document: ${err.message}` }, 500);
}
} else {
// Method not allowed
return res.json({ error: "Method Not Allowed" }, 405);
}
};```