#WroteScan - OpenAI without the size limits: upload any pdf and apply any prompt to it

27 messages · Page 1 of 1 (latest)

regal cliff
#

hi all!

I created a simple free tool where you can summarize and query documents of any size and estimate the cost to do so: https://www.wrotescan.com

You can edit the prompts as well as automatically chunk and combine documents.

Let me know if you want me to run some examples for you! Send me a pdf and tell me what you'd like summarized or extracted.

Tips

Please be sure to keep {text} in both prompts or the program will not input your document's text into the map reduce summarizer.

{text} can only appear once in each prompt. It is where the text from each chunk to be summarized is input into the prompts.

Learnings

Some interesting learnings I had while creating the tool:

  • Minimizing the number of steps through the AI improved summarization, so map reduce was often better than a more advanced refine workflow which passes the output through the model many more times.
quaint blade
#

I'll check it out soon

edgy path
#

I like it! Is there a way to use the text-davinci-001 in stead of 003 to reduce the processing costs per pdf?

raw drum
#

would it be possible to run an example for me?

raw drum
dry ermine
#

Ill do it

timid urchin
#

Sourcecode please

regal cliff
#

Now updated with chatgpt-turbo-3.5 option!

regal cliff
edgy path
#

awesome

#

Source code would indeed be appreciated my friend$

tardy trail
#

@regal cliff do you have a version that works with text aswell or is this project solely pdfs?

regal cliff
tardy trail
#

big brain

regal cliff
#

So many years of clerical work haha

tardy trail
#

and is step 2 necesary?

#

as in, if I give it a huge chunk of text and ask it to make bullet point notes, can it just supply the bullet point notes?

regal cliff
#

This directly implements that notebook

edgy path
#

Would it be possible to add the merging step chuncks to the downloadable word file at the end of the summarization. Alternativly a button with 'download output csv' would be equally usefull.

regal cliff
#

@edgy path : check out the "intermediate steps" document from map reduce. does that do what you need?

timid urchin
regal cliff
#

sample code for that: if completions_model == "gpt-3.5-turbo":
llm = ChatOpenAI(
temperature=configs_map_reduce[completions_model]["temperature"],
model_name=configs_map_reduce[completions_model]["model_name"],
openai_api_key=OpenAI_api_key,
max_tokens=configs_map_reduce[completions_model]["token_sizing_output"],
)

    llm_combine = ChatOpenAI(
        temperature=configs_map_reduce[completions_model]["temperature"], 
        model_name=configs_map_reduce[completions_model]["model_name"], 
        openai_api_key=OpenAI_api_key,
        max_tokens=configs_map_reduce[completions_model]["combine_sizing_output"],
    )
#

openai_api_key=OpenAI_api_key, is how to do it

#

@timid urchin

timid urchin