JS SDK

[small seal]

I dropped those settings into JS file, added a prompt and they seem to be honoured, I got 376 chars, terminated by the sequence red.

$ node settings.js |tee >(wc -c)
Once upon a time, in a small town nestled amidst rolling hills, lived a young girl named Lily. Lily was an inquisitive and adventurous child, always seeking new experiences and knowledge. One day, while exploring the attic of her grandmother's house, she stumbled upon a dusty old backpack hidden in a forgotten corner.

Curiosity piqued, Lily opened the backpack and discove
376

you do have some issues though - you're trying to set the UNSPECIFIED harm category, which isn't a thing, and you spelled temperature wrong. these should be picked up when you call generateContent though.

[dawn bough]

i didnt notice that most of this was made at like 4 am lol

[small seal]

ha ok you figured it out then?

[dawn bough]

well it still goes over the max

[small seal]

I can't repro - taking that same code and setting maxOutputTokens changes the output (and putting the output back through countTokens confirms it's always under the max token count

here's the exact script I'm using: https://gist.github.com/markmcd/6304c2c7d14dc56f4f98059d2dcda43d

Run that with TOKEN_COUNT=900 node gemini_output_tokens.js |tee >(wc -c |sed -e 's/^/wc: /g') and again with TOKEN_COUNT=1500 and the output I got was 2380 and 3092 chars (respective) which, when tokenised, came to 731 and 963 tokens rsp.

here's the script I used to count tokens: https://gist.github.com/markmcd/da378d52ffaaeef427a051760700ad59

[dawn bough]

yeah its sending back over 500 tokens when the max i gave it is 200

it might be how im asking the model?

i think bun might be the issue? not exactly sure but its possible

[small seal]

ok, I can't replicate that behaviour, but I also haven't seen your full code or prompt. are you able to replicate using my script? if so, can you share the diff?

[dawn bough]

trying to run it im using the model.countTokens thingy, for some reason running it on bun is bugging out

$ node prompt.js
node:internal/process/promises:288
            triggerUncaughtException(err, true /* fromPromise */);
            ^

GoogleGenerativeAIError: [400 Bad Request] API key not valid. Please pass a valid API key. [{"@type":"type.googleapis.com/google.rpc.ErrorInfo","reason":"API_KEY_INVALID","domain":"googleapis.com","metadata":{"service":"generativelanguage.googleapis.com"}}]
   

Node.js v18.19.0
$ bun prompt.js

{
  totalTokens: 1,
}```

it should be the right api key tho?

its weird because the api token only works when im not using js

i dont think it should matter tho?

[small seal]

are you passing an api key to GoogleGenerativeAI(...)? don't paste it here, but make sure it starts with AIza and doesn't have any extra spaces or characters around it

[dawn bough]

it was cause i didnt import dotenv correctly lol, but for some reason i think if the response is over 200 tokens it just wont say anything

if i do "write me the philosphy of computers" or "Lorem ipsum" it outputs nothing, but if i do something small like "hello!" or "who are you?" itll work

[small seal]

right - yeah I saw that too. I'm unsure if that's WAI or if that's a bug, but I'll bring it up with the team anyway

ah wait, you should get finishReason: 'MAX_TOKENS' in the response

[dawn bough]

i tried to get the finishReason using

response.candidates.forEach(s => {
        console.log(s)
      })

and i got finishReason: "OTHER"

[small seal]

hmm ok I am seeing both

though the response isn't coming out so it's hard to tell what the trigger is

[small seal]

I've found a couple of bugs here that I've filed & we'll look at. One is that we're getting finishReason: OTHER but only sometimes, even with temperature of 0. Another is that it's not returning output when we hit MAX_TOKENS

[dawn bough]

shouldn't it make an output less than the MAX_TOKENS

[small seal]

yeah, that's the second thing I said