#Mixtral 8x7b: Extracting data from long documents

6 messages · Page 1 of 1 (latest)

frank bronze
#

Hello!

I want to use Mixtral 8x7b for extracting data from long (up to 100 pages) documents.

In the best case I could just include all the content in the same request but I feel like it’s too big for it’s context window (I‘ve read different things about Mixtral‘s context window 🤔).

If that does not work I won‘t get around splitting the document and augmenting the extracted data gradually, right?

Would be great to hear your opinions or experiences 😄

Regards from Vienna,
Lukas

idle jetty
#

Use LLM to summarize each page or list out the insights succintly from each page. Do a final request on outputs from all of those other requests to answer questions.

frank bronze
#

Hi @idle jetty , thanks! I think I will split the text into several chunks that still fit into the context window and then do a final request as you proposed 🙂

median heart
#

What information have you found on the context window? Don't see anything mentioning it in the model card

frank bronze
#

@median heart In the API documentation it says 32K (https://docs.mistral.ai/platform/endpoints/). @idle vault mentioned that it only worked well for him up to 16K and I‘ve also read conflicting information about Mistral‘s context window. I have to make sure that my results are accurate so I think I will just try to keep my context below 16K for now.

median heart