#Adjust Z.AI Provider Max Token Values?

2 messages · Page 1 of 1 (latest)

spark island
#

I noticed in packages/types/src/providers/zai.ts that the maxToken values are set lower than the ones listed in their docs.

Is this intentional to preserve functionality or could I put a PR to adjust them?

IMG_7018.jpg
delicate raven
#

If you crank them up, you get less context window to work with (the inference engine will end up reserving maxTokens in the context window).

So a 200K context - 131K max tokens leaves you with 70K context. You don’t want that.