NodeJS library for tokenization

[deep gust]

Hello!!

Could somebody point me out to some tokenization library that actually works for GPT-4 chatCompletion API that works in NodeJS?

I've used already "gpt-3-encoder" and also this one: "https://github.com/niieani/gpt-tokenizer", same result with both (big amount of tokens count, ie. in this image you see a token count of 13500, and the API accepts it!! (of course, because it is not that number, it is really lower).

Thanks in advance!

[deep gust]

Solved already, sharing here in case somebody needs it:

https://www.npmjs.com/package/@dqbd/tiktoken

[fleet hemlock]

why not use tiktoken? Works like a charm for python too 🙂