#piping for async sub process

17 messages · Page 1 of 1 (latest)

limpid holly
#

In Deno.process, is there a way to set its STDOUT to something I can read line by line while that primary process is still running?

undone merlin
#

Set stdout to piped and use the readable stream, I think?

https://doc.deno.land/deno/stable/~/Deno.run

https://doc.deno.land/deno/stable/~/Deno.Process

Deno CLI APIs – Deno.run | Deno Doc

Spawns new subprocess. RunOptions must contain at a minimum the opt.cmd, an array of program arguments, the first of which is the binary.

const p = Deno.run({ cmd: ["curl", "https://example.com"], }); const status = await p.status();

Subprocess uses same working directory as parent process unless opt.cwd is specified.

Environmental variable...

Deno CLI APIs – Deno.Process | Deno Doc

Represents an instance of a sub process that is returned from {@linkcode Deno.run} which can be used to manage the sub-process.

#

Or stdin if you meant reading line by line inside the spawned process.

limpid holly
#

Thank you!

This works, but is VERY ugly

  • Is there a better way to terminate the reading loop?

  • Is there a better way to deal with the reads not filling the buffer?

const cmd = ["sh", "long-running-script"]

const proc = Deno.run({stdout: "piped" , cmd: cmd});

async function handleAll(){
    const decoder = new TextDecoder()
    const buf =new Uint8Array(10000);
    while (true){
    const n = await proc.stdout.read(buf)
    if (n === null){ break;}
    const outout = decoder.decode(buf)
    console.log(outout.slice(0,n));
    }
}

const promises = [handleAll() , proc.status()]

await Promise.allSettled(promises)
undone merlin
limpid holly
#

I read through the doc's and the sources (to the extent I can) and I just don't get it.

Clearly what I am doig is not right.

error: TS2339 [ERROR]: Property 'pipeThrough' does not exist on type 'Reader & Closer & { readable: ReadableStream<Uint8Array>; }'.

async function handleAll(){
    const output = await proc.stdout.pipeThrough((new TextDecoderStream()))
    console.dir(output)
}

Can you point me at a simple example?

undone merlin
#

stdout.readable.pipeThrough

limpid holly
#

Thank you -- i will try it

limpid holly
# undone merlin stdout.readable.pipeThrough

Thank you!

This is much less ugly!

Is there also a way to make the control of the while loop less ugly?

async function handleAll() {
  const outputStream = await proc.stdout.readable.pipeThrough(
    new TextDecoderStream(),
  );
  const r = outputStream.getReader();
  let firstOutput = { done: false };
  while (!firstOutput.done) {
    firstOutput = await r.read();
    console.log(firstOutput);
  }
  return r;
}
undone merlin
#

I presume your end goal is not to just log the output?

You might consider implementing your eventual use case as a writable stream. But if you just need to get the output then yeah, a while loop doing awaits is pretty much the way. You can improve it a bit by doing:

let firstOut;
do {
  firstOut = await r.read();
} while (!firstOut.done);
limpid holly
#

thank you a, a "do while" was exactly what I was hoping for

#

indeed there is more work to do with the results, but I am trying to understands.

#

the basics.

#

when I tried

async function handleAll() {
  const outputStream = await proc.stdout.readable.pipeThrough(
    new TextDecoderStream(),
  );
  for await (const firstOutput of outputStream) {
    console.log(firstOutput);
  }
}

it waited for the process to finish and then gives me all the reads rapidly rather tthan giving to me as they become available

undone merlin
#

Humm...

limpid holly