#Using the stream option in Javascript

9 messages · Page 1 of 1 (latest)

hybrid sun
#

Looking for any tips for making use of the completions API stream option in JavaScript. I'm very new to JS, but it doesn't seem too straight forward. I can't get it working with the standard Fetch

hollow pewter
#

Something like this

// server will just proxy the req to the openAi 
const response = await fetch('/api/stream', {
  method: 'POST',
  headers: {
    accept: 'text/event-stream',
    'content-type': 'application/json'
  },
  body: JSON.stringify(body)
});

async function readAllChunks(readableStream) {
  const reader = readableStream.getReader();

  let done, value;
  while (!done) {
    ({ value, done } = await reader.read());
    if (done) {
      return 'Done';
    }

    const regex = /data: .*?(\{.*\})/g;
    const dataString = new TextDecoder().decode(value);
    const match = [...dataString.matchAll(regex)];
    streamedAnswer.value += match.reduce((aac, match) => {
      const dataObject = JSON.parse(match[1]);

      return dataObject.choices[0].text;
    }, '');

    // streamedAnswer is a reactive store that will update the UI
    console.log('streamedAnswer.value', streamedAnswer.value);
  }
}

await readAllChunks(response.body);
#

the only issue is that the response omits the usage

#

trying to figure that out

coarse shell
#

did you ever figure out the usage part yet?

solemn hull
real meadow
#

here is an example of using stream with JS

solemn hull
#

@real meadow What am I missing here ? I added openai key, and throw an error!

GPT-3-5-stream-example-forked-CodeSandbox.png