Large JSON inputs for GPT3.5 | OpenAI | Page 1

raw shore Mar 23, 2023, 2:43 AM

#

🧵

prisma island Mar 23, 2023, 2:44 AM

#

maybe just use the json as your prompt?

raw shore Mar 23, 2023, 2:44 AM

#

It's larger than the token limit, roughly 2 years of transaction data.

prisma island Mar 23, 2023, 2:44 AM

#

is it like an Q&A?

raw shore Mar 23, 2023, 2:45 AM

#

Yeah, mostly

prisma island Mar 23, 2023, 2:45 AM

#

or you want to summarize it at once

raw shore Mar 23, 2023, 2:45 AM

#

The exact use case: take 2 years of transactions as input, and answer questions about them

prisma island Mar 23, 2023, 2:45 AM

#

oh a Q&A

#

you can use embeddings

#

#dev-chat message

#

take a look at this^

raw shore Mar 23, 2023, 2:48 AM

#

Super useful, thanks!

#

so in this case, would our transactions JSON be a single embedding?

prisma island Mar 23, 2023, 2:51 AM

#

no, you should cut it into as many embeddings as possible

#

but dont cut it into a few words lol, like around 50 words per embedding

#

also you should label the small pieces with pre-set questions and embed the questions instead, so the hit rate would be higher

raw shore Mar 23, 2023, 2:52 AM

#

okay, so I can take our transactions JSON, cut it into CSV-like data (one row per transaction), create an embedding per transaction

prisma island Mar 23, 2023, 2:52 AM

#

yes

raw shore Mar 23, 2023, 2:52 AM

#

gotcha

#

and I have to store these embeddings myself

#

or use some hosted version of OpenAI APIs

prisma island Mar 23, 2023, 2:54 AM

#

openai api doesn't store any data for you

#

you need to store it by yourself

#

or use a vector DB

#

theres a cookbook for that in the link i gave you

raw shore Mar 23, 2023, 3:39 AM

#

what if they're all roughly the same cosine similarity to the prompt?

#Large JSON inputs for GPT3.5