How do I use npm's puppeteer with new npm syntax (also Steatlh plugin) | Deno | Page 1

chrome surge Nov 24, 2022, 10:54 PM

#

I've tried this but nothing works:

import * as p1 from 'npm:puppeteer@^16.2.0';
import puppeteer from "npm:puppeteer-extra@^3.3.4";
import Stealth from "npm:puppeteer-extra-plugin-stealth@^2.11.1";

#

I'm supposed to install puppeteer alongside puppeteer extra, but how do I install npm modules with deno?

#

Error: Could not find expected browser (chrome) locally. Run `npm install` to download the correct Chromium revision (1022525).
    at ChromeLauncher.launch (file:///home/ubuntu/.cache/deno/npm/registry.npmjs.org/puppeteer/16.2.0/lib/cjs/puppeteer/node/ChromeLauncher.js:70:23)
    at PuppeteerNode.launch (file:///home/ubuntu/.cache/deno/npm/registry.npmjs.org/puppeteer/16.2.0/lib/cjs/puppeteer/node/Puppeteer.js:145:31)
    at PuppeteerExtra.launch (file:///home/ubuntu/.cache/deno/npm/registry.npmjs.org/puppeteer-extra/3.3.4/dist/index.cjs.js:128:41)
    at async file:///home/ubuntu/www/briskreader.com/app/app.ts:298:22
    at async dispatch (https://deno.land/x/oak@v11.1.0/middleware.ts:41:7)
    at async dispatch (https://deno.land/x/oak@v11.1.0/middleware.ts:41:7)
    at async dispatch (https://deno.land/x/oak@v11.1.0/middleware.ts:41:7)
    at async dispatch (https://deno.land/x/oak@v11.1.0/middleware.ts:41:7)
    at async file:///home/ubuntu/www/briskreader.com/app/app.ts:526:2

vestal osprey Nov 24, 2022, 11:20 PM

#

chrome surge I'm supposed to install puppeteer alongside puppeteer extra, but how do I instal...

I believe deno doesn't run post-install hooks by design (that decision may have changed)

#

In puppeteer's case that hook downloads an arch/os specific binary

chrome surge Nov 24, 2022, 11:22 PM

#

The deno version of puppeteer works fine but the stealth plug-in doesn’t work on deno

vestal osprey Nov 24, 2022, 11:22 PM

#

const browser = await puppeteer.launch({executablePath: '/path/to/Chrome'});

#

You can explicitly provide a path to your own chrome binary

chrome surge Nov 24, 2022, 11:23 PM

#

Yeah I have chromium for that

#

It’s the stealth plug-in which I need

#

I tried esm.sh but that didn’t work

vestal osprey Nov 24, 2022, 11:24 PM

#

The stacktrace your shared above doesn't fail on the stealth plugin

chrome surge Nov 24, 2022, 11:24 PM

#

It gripes about a kind-of plug-in not able to load

#

If someone can get stealth plug-in running with deno that would be appreciated

#

And share how

vestal osprey Nov 24, 2022, 11:25 PM

#

Could you share the stacktrace/error-message you're seeing ?

#

(the one you shared above appears unrelated to the stealth plugin you're talking about)

chrome surge Nov 24, 2022, 11:49 PM

#

error: Uncaught Error: Dynamic require of "kind-of" is not supported
    at https://cdn.esm.sh/v55/clone-deep@0.2.4/es2021/clone-deep.js:2:583
    at Function.o [as typeOf] (https://cdn.esm.sh/v55/lazy-cache@1.0.4/es2021/lazy-cache.js:2:968)
    at s (https://cdn.esm.sh/v55/clone-deep@0.2.4/es2021/clone-deep.js:2:1251)
    at m.exports (https://cdn.esm.sh/v55/merge-deep@3.0.3/es2021/merge-deep.js:2:1229)
    at new h (https://cdn.esm.sh/v55/puppeteer-extra-plugin@3.2.0/es2021/puppeteer-extra-plugin.js:2:635)
    at new r (https://cdn.esm.sh/v55/puppeteer-extra-plugin-stealth@2.9.0/es2021/puppeteer-extra-plugin-stealth.js:2:1115)
    at w (https://cdn.esm.sh/v55/puppeteer-extra-plugin-stealth@2.9.0/es2021/puppeteer-extra-plugin-stealth.js:2:1893)
    at file:///home/ettinger/www/briskreader.com/app/app.ts:32:15

#

I'm loading with import StealthPlugin from "https://cdn.esm.sh/v55/puppeteer-extra-plugin-stealth@2.9.0/es2021/puppeteer-extra-plugin-stealth.js";

vestal osprey Nov 24, 2022, 11:58 PM

#

With npm:... instead of esm ?

chrome surge Nov 24, 2022, 11:58 PM

#

if i do that it requires i use puppeteer-extra and puppeteer

vestal osprey Nov 24, 2022, 11:59 PM

#

It depends on them anyway, no ?

chrome surge Nov 25, 2022, 12:00 AM

#

npm install puppeteer puppeteer-extra puppeteer-extra-plugin-stealth

vestal osprey Nov 25, 2022, 12:00 AM

#

Or just use npm:... imports

chrome surge Nov 25, 2022, 12:02 AM

#

if i do that i get this:

#

http://pastie.org/p/0nFEAcGdD3FeVCjKfSzXnp

#

import puppeteer from 'npm:puppeteer-extra';
import StealthPlugin from 'npm:puppeteer-extra-plugin-stealth';

#

i still need to do npm:puppeteer somewhere

#

or at least load it somehow

#

it needs that base module loaded

vestal osprey Nov 25, 2022, 12:06 AM

#

You can import it too

chrome surge Nov 25, 2022, 12:06 AM

#

but its the same name as the extra plugin

#

like this?

#

import puppeteer from 'npm:puppeteer';
import puppeteer from 'npm:puppeteer-extra';
import StealthPlugin from 'npm:puppeteer-extra-plugin-stealth';

vestal osprey Nov 25, 2022, 12:08 AM

#

Or with a different identifier

#

import 'npm:puppeteer'; might also work

chrome surge Nov 25, 2022, 12:10 AM

#

i get a 101 http response error

vestal osprey Nov 25, 2022, 12:10 AM

#

Where ?

#

Keep in mind that it's hard for others to assist without the context of stacktraces, error-messages or code (without running and implementing it themselves)

chrome surge Nov 25, 2022, 12:12 AM

#

yeah i know

#

http://pastie.org/p/6DU5pe0KlzGTDLJc8e21TB

#

import 'npm:puppeteer';
import puppeteer from 'npm:puppeteer-extra';
import StealthPlugin from 'npm:puppeteer-extra-plugin-stealth';

#

it can't do the websocket upgrade i guess

vestal osprey Nov 25, 2022, 12:19 AM

#

Yeah, looks like the node-compat layer isn't fully correct/compatible wrt to ws clients

#

Should be fixable

#

But I don't personally have time to dive into that

chrome surge Nov 25, 2022, 12:21 AM

#

fixable where in deno or in puppeteer

#

i'm going to rewrite this backend in node anyway.

#

so don't worry about it

#

unless you can do it in the next few months

vestal osprey Nov 25, 2022, 12:23 AM

#

From a quick glance at the code (https://github.com/websockets/ws/blob/master/lib/websocket.js#L894)

#

It seems like the bug would originate from deno's node-compat not emitting/supporting the upgrade event on client requests

#

I think there's all the fundamental infra to support it

#

So it might be a quick fix, taking a few minutes or hours

chrome surge Nov 25, 2022, 12:26 AM

#

should i create a bug ticket

#

?

vestal osprey Nov 25, 2022, 12:29 AM

#

Sure, or take a look yourself

#

The relevant code is roughly here: https://github.com/denoland/deno_std/blob/main/node/http.ts#L130

#

From a quick glance we aren't emitting the 'upgrade' event and the underlying client might be h2 whereas the logic here assumes h1

chrome surge Nov 25, 2022, 12:30 AM

#

probably above my paygrade. i don't know much about internal modules

#

i'm just a user

vestal osprey Nov 25, 2022, 12:30 AM

#

Yeah, open a ticket and maybe someone from the core team might get around to it

chrome surge Nov 25, 2022, 12:30 AM

#

is h2 related to the sesrver?

vestal osprey Nov 25, 2022, 12:31 AM

#

chrome surge is h2 related to the sesrver?

h2 => HTTP/2

chrome surge Nov 25, 2022, 12:31 AM

#

i see

vestal osprey Nov 25, 2022, 12:31 AM

#

Node's http/https is http/1

#

Websockets over http/1 and http/2 broadly follow the same core ideas but have slightly different mechanisms and semantics

chrome surge Nov 25, 2022, 12:31 AM

#

file it here? https://github.com/denoland/deno

GitHub

GitHub - denoland/deno: A modern runtime for JavaScript and TypeScr...

A modern runtime for JavaScript and TypeScript. Contribute to denoland/deno development by creating an account on GitHub.

vestal osprey Nov 25, 2022, 12:32 AM

#

Or deno_std since that's where the node-compat code is

#

Either works, I believe the admit team can transfer the issue over if necessary

#

It's probably best to recap the context rather than just pointing to a discord thread

#

The issue is that npm's ws lib (and thus all consumers) doesn't work client-side, because deno's node-compat impl of http/https clients don't handle upgrades / emit upgrade-events

#

With the sub-issue that http/https uses fetch, which might use h2 for https whereas node's https module only supports/assumes h1

#

Though in your specific case, you probably only need http/h1 so that h2 mixup isn't an issue per se, since puppeteer should handshake over http locally

chrome surge Nov 25, 2022, 12:48 AM

#

updated

#

thanks

#

do you work at deno?

#

i used to work with Bert at StrongLoop

#

I heard he co-founded deno

#How do I use npm's puppeteer with new npm syntax (also Steatlh plugin)