#Reuse Piece of Text as Context

26 messages · Page 1 of 1 (latest)

flat orchid
#

I have a long piece of text (~500 tokens) I want to extract multiple information from. Since the extraction process is quite complex and the different kinds of information I want to extract differ semantically, I do not believe I can use just one prompt to extract it all.
So I was wondering if there was any way I could encode the long piece of text only once and use it in multiple API calls so I would not have to pay the cost of encoding the context multiple times.

Something like this:

  • API call 1: encode "very long piece of text" -> context
  • API call 2: completion of ({context} + "What is A?")
  • API call 3: completion of ({context} + "What is B?")

Again, the idea is that I do not have to encode the context multiple times. I actually asked ChatGPT and it said there was a context property in each choice in the response - which does not exist according to the API documentation.

Any tips?

exotic harness
#

So for each of the requests are you sending the same context?

#

Why can’t you send the same context once and then ask multiple questions for the completion, instead of a separate question with each completion?

#

Did you run into any problems with that?

flat orchid
flat orchid
#

Anyhow, computing the embedding once would be the most efficient solution in the long run. So I wondered if it would work somehow.

#

If you have any pointers on prompt engineering for more than one task at once, please do link them. Maybe the issue is that I’m not really good at doing it yet.

exotic harness
#

Ah I see, and I think I agree that computing the embedding once would be the most efficient solution for this use case

#

unfortunately even if you get a pretty good prompt setup, there is no guarantee that it'll return the answers in the format that you want, so using templated embeddings is best I guess

exotic harness
flat orchid
# 
Context:
Woman, 35 years old, comes into the ED with a chief complaint of palpitations. She reports a sudden onset of palpitations one hour ago associated with weakness and dizziness. She also reports that two weeks ago she felt a stabbing chest pain rated 6/10. No syncope was reported. She has a history of DM-I and is on metformin 500 mg/day. Family history is unremarkable. 

Vitals were HR 41, BP 100 / 62, RR of 21, saturating 97% on R.A. Heart sounds were unremarkable, lung was clear to auscultation.

EKG revealed AV node block (Mobitz II).
#

That would be a context.

#

I want to extract the symptoms (along with their characteristics and temporality), their history (both personal and family), the physical examination findings (name and description) and additional findings (name, description).

#

To extract the symptoms, for example, I would use this as a prompt:

Command: Extract each symptom name, description and temporality in CSV format. If no description or temporality is found, write "N/A". The first line should be "Symptom Name, Description, Temporality".
flat orchid
#

Experimenting a little bit, I found that a JSON output is better a this sort of thing. It works, but I wonder about how stable it is...

lethal valve
#

You might want to tinker with ending your prompt with a header for the data like this:

name age personality physical-description general-description greeting
#

Start with something like "A table about <whatever>:" and end with something like that.

#

I'm not sure if it matters but I skipped a line between the opening line and the data, and then skipped a line after the data and before the table header, and made sure to end the line at the end of the second line of the table header.

#

So far that sort of setup has been pretty solid for me, but I'm also not extracting information, I'm having it make stuff up, so I'm not sure how that will affect it.

#

I'm not sure if it would intuit "temporality" properly with just the table header, but it's worth a shot.

flat orchid
lethal valve
#

CSV is problematic if the data might have commas in it so I haven't really used it. I tried json but it used a lot of tokens.

flat orchid
#

I’m wondering about the stability of generation

flat orchid