how do i increase output response size? | OpenAI | Page 1

sleek thistle Jun 26, 2023, 6:12 PM

#

are you using gpt-index?

#

that looks like gpt-index code

winged hull Jun 26, 2023, 7:56 PM

#

sleek thistle are you using gpt-index?

yes ```python

def construct_index(directory_path):
# set maximum input size
max_input_size = 4096
# set number of output tokens
num_outputs = 2000
# set maximum chunk overlap
max_chunk_overlap = 20
# set chunk size limit
chunk_size_limit = 600

# define prompt helper
prompt_helper = PromptHelper(max_input_size, num_outputs, max_chunk_overlap, chunk_size_limit=chunk_size_limit)

# define LLM
llm_predictor = LLMPredictor(llm=OpenAI(temperature=0.5, model_name="gpt-3.5-turbo-0613", max_tokens=num_outputs))

documents = SimpleDirectoryReader(directory_path).load_data()

service_context = ServiceContext.from_defaults(llm_predictor=llm_predictor, prompt_helper=prompt_helper)
index = GPTSimpleVectorIndex.from_documents(documents, service_context=service_context)

index.save_to_disk('index.json')

return index

def ask_ai():
index = GPTSimpleVectorIndex.load_from_disk('index.json')
while True:
query = input("What do you want to ask? ")
response = index.query(query)
display(Markdown(f"Response: {response.response}"))

#

I am, what does that have to do with the resposne length?

#

using gpt-3.5-turbo-0613 model

sleek thistle Jun 27, 2023, 5:55 AM

#

service_context = ServiceContext.from_defaults(
llm_predictor=llm_predictor,
chunk_size=1024
)

Output is related with service context configuration.
Please try to remove prompt_helper, and just configure service_context like this.

#

Let me know the update.

#

@winged hull

winged hull Jun 27, 2023, 12:55 PM

#

Thank you I’ll test it

winged hull Jun 27, 2023, 1:25 PM

#

@sleek thistle

#

sleek thistle Jun 27, 2023, 1:27 PM

#

@winged hull

#

that is because of llama-index version

#

could you please remove current version and install latest version?

winged hull Jun 27, 2023, 1:29 PM

#

sleek thistle that is because of llama-index version

I did, still same error

#

'

#

from llama_index import SimpleDirectoryReader, GPTListIndex, readers, GPTSimpleVectorIndex, LLMPredictor, PromptHelper, ServiceContext
from langchain import OpenAI
import sys
import os
from IPython.display import Markdown, display

def construct_index(directory_path):
    # set maximum input size
    max_input_size = 4096
    # set number of output tokens
    num_outputs = 10000
    # set maximum chunk overlap
    max_chunk_overlap = 20
    # set chunk size limit
    chunk_size_limit = 600 

    # define prompt helper
    prompt_helper = PromptHelper(max_input_size, num_outputs, max_chunk_overlap, chunk_size_limit=chunk_size_limit)

    # define LLM
    llm_predictor = LLMPredictor(llm=OpenAI(temperature=0.5, model_name="gpt-3.5-turbo-0613", max_tokens=num_outputs))
 
    documents = SimpleDirectoryReader(directory_path).load_data()
    
    # service_context = ServiceContext.from_defaults(llm_predictor=llm_predictor, prompt_helper=prompt_helper)
    service_context = ServiceContext.from_defaults(
            llm_predictor=llm_predictor,
            chunk_size=1024
        )
    index = GPTSimpleVectorIndex.from_documents(documents, service_context=service_context)

    index.save_to_disk('index.json')

    return index

def ask_ai():
    index = GPTSimpleVectorIndex.load_from_disk('index.json')
    while True: 
        query = input("What do you want to ask? ")
        response = index.query(query)
        display(Markdown(f"Response: <b>{response.response}</b>"))

sleek thistle Jun 27, 2023, 1:32 PM

#

Please check llama-index version.

#

before I get that issue using low version

winged hull Jun 27, 2023, 1:34 PM

#

i removed the old version number

#

maybe need to restart kernel sec

winged hull Jun 27, 2023, 1:37 PM

#

sleek thistle Please check llama-index version.

Note: you may need to restart the kernel to use updated packages.
Note: you may need to restart the kernel to use updated packages.
Name: llama-index
Version: 0.5.6
Summary: Interface between LLMs and your data.
Home-page: https://github.com/jerryjliu/gpt_index
Author: 
Author-email: 
License: MIT
Location: /opt/homebrew/lib/python3.11/site-packages
Requires: dataclasses-json, langchain, numpy, openai, pandas, tenacity, tiktoken
Required-by: ``` this ur version?

#

still getting the errror

sleek thistle Jun 27, 2023, 1:39 PM

#

my llama-index version is 0.6.34

winged hull Jun 27, 2023, 1:40 PM

#

ok sec

sleek thistle Jun 27, 2023, 1:40 PM

#

@winged hull , are you using venv, right?

#

I am normally using venv to detect issue of current project, global packages.

#

please reference this. 🙂

winged hull Jun 27, 2023, 1:41 PM

#

winged hull Jun 27, 2023, 1:41 PM

#

sleek thistle <@609989218625781761> , are you using venv, right?

I'm in a notebook

#

i got this error last time

#

this is why we had to downgrade versions

sleek thistle Jun 27, 2023, 1:42 PM

#

GPTVectorStoreIndex
Please use this

#

GPTSimpleVectorIndex is not working in latest version

winged hull Jun 27, 2023, 1:42 PM

#

#

do I replace that with the new library, need to change the construction of it?

sleek thistle Jun 27, 2023, 1:43 PM

#

just replace to GPTVectorStoreIndex from GPTSimpleVectorIndex

#

GPTSimpleVectorIndex is not working in latest version

winged hull Jun 27, 2023, 1:43 PM

#

winged hull Jun 27, 2023, 1:43 PM

#

sleek thistle GPTSimpleVectorIndex is not working in latest version

ok now this is giving the error we had last time

sleek thistle Jun 27, 2023, 1:44 PM

#

please share me your full code

#

let me update them and return to you.

winged hull Jun 27, 2023, 1:46 PM

#

sleek thistle let me update them and return to you.

%pip -q install langchain==0.0.148 openai
%pip -q install llama_index==0.6.34
!pip show llama_index
from llama_index import SimpleDirectoryReader, GPTListIndex, readers, GPTVectorStoreIndex, LLMPredictor, PromptHelper, ServiceContext
from langchain import OpenAI
import sys
import os
from IPython.display import Markdown, display

def construct_index(directory_path):
    # set maximum input size
    max_input_size = 4096
    # set number of output tokens
    num_outputs = 10000
    # set maximum chunk overlap
    max_chunk_overlap = 20
    # set chunk size limit
    chunk_size_limit = 600 

    # define prompt helper
    prompt_helper = PromptHelper(max_input_size, num_outputs, max_chunk_overlap, chunk_size_limit=chunk_size_limit)

    # define LLM
    llm_predictor = LLMPredictor(llm=OpenAI(temperature=0.5, model_name="gpt-3.5-turbo-0613", max_tokens=num_outputs))
 
    documents = SimpleDirectoryReader(directory_path).load_data()
    
    # service_context = ServiceContext.from_defaults(llm_predictor=llm_predictor, prompt_helper=prompt_helper)
    service_context = ServiceContext.from_defaults(
            llm_predictor=llm_predictor,
            chunk_size=1024
        )
    index = GPTVectorStoreIndex.from_documents(documents, service_context=service_context)

    index.save_to_disk('index.json')

    return index

def ask_ai():
    index = GPTSimpleVectorIndex.load_from_disk('index.json')
    while True: 
        query = input("What do you want to ask? ")
        response = index.query(query)
        display(Markdown(f"Response: <b>{response.response}</b>"))
# os.environ["OPENAI_API_KEY"] = input("Paste your OpenAI key here and hit enter:")
os.environ["OPENAI_API_KEY"] = ""
construct_index("data/")
ask_ai()```

#

this is in a notebook

sleek thistle Jun 27, 2023, 1:47 PM

#

prompt_helper = PromptHelper(max_input_size, num_outputs, max_chunk_overlap, chunk_size_limit=chunk_size_limit)

#

remove this code

#

also remove PromptHelper from this code.
from llama_index import SimpleDirectoryReader, GPTListIndex, readers, GPTVectorStoreIndex, LLMPredictor, PromptHelper, ServiceContext

#

@winged hull

#

can you try?

winged hull Jun 27, 2023, 1:50 PM

#

ok sec

#

Cell In[9], line 32, in construct_index(directory_path)
     26 service_context = ServiceContext.from_defaults(
     27         llm_predictor=llm_predictor,
     28         chunk_size=1024
     29     )
     30 index = GPTVectorStoreIndex.from_documents(documents, service_context=service_context)
---> 32 index.save_to_disk('index.json')
     34 return index

AttributeError: 'VectorStoreIndex' object has no attribute 'save_to_disk'```

winged hull Jun 27, 2023, 1:51 PM

#

sleek thistle can you try?

getting this error now

sleek thistle Jun 27, 2023, 1:52 PM

#

yes, save_to_disk function is not working in latest version

#

storage_context = StorageContext.from_defaults()
storage_context.persist(persist_dir="Directory to save")

#

so, please add StorageContext.

from llama_index import SimpleDirectoryReader, GPTListIndex, readers, GPTVectorStoreIndex, LLMPredictor, PromptHelper, ServiceContext

#

storage_context = StorageContext.from_defaults()
storage_context.persist(persist_dir="Directory to save")

and define storage_context like this

winged hull Jun 27, 2023, 1:57 PM

#

Storagecontext not defined

winged hull Jun 27, 2023, 1:57 PM

#

sleek thistle storage_context = StorageContext.from_defaults() storage_context.persist(persist...

sleek thistle Jun 27, 2023, 1:58 PM

#

from llama_index import SimpleDirectoryReader, GPTListIndex, readers, GPTVectorStoreIndex, LLMPredictor, PromptHelper, ServiceContext, StorageContext

#

you still didn't add StorageContext

sleek thistle Jun 27, 2023, 1:59 PM

#

sleek thistle from llama_index import SimpleDirectoryReader, GPTListIndex, readers, GPTVectorS...

use this code

winged hull Jun 27, 2023, 1:59 PM

#

sleek thistle use this code

Cell In[21], line 36, in construct_index(directory_path)
     30 service_context = ServiceContext.from_defaults(
     31         llm_predictor=llm_predictor,
     32         chunk_size=1024
     33     )
     34 index = GPTVectorStoreIndex.from_documents(documents, service_context=service_context)
---> 36 index.save_to_disk('index.json')
     38 return index

AttributeError: 'VectorStoreIndex' object has no attribute 'save_to_disk'```

#

that fixed that, now this error

sleek thistle Jun 27, 2023, 2:00 PM

#

Please remove index.save_to_disk('index.json')

#

storage_context = StorageContext.from_defaults()
storage_context.persist(persist_dir="Directory to save")

And add this below service_context

winged hull Jun 27, 2023, 2:01 PM

#

---------------------------------------------------------------------------
NameError                                 Traceback (most recent call last)
Cell In[32], line 1
----> 1 ask_ai()

Cell In[29], line 41, in ask_ai()
     40 def ask_ai():
---> 41     index = GPTSimpleVectorIndex.load_from_disk('index.json')
     42     while True: 
     43         query = input("What do you want to ask? ")

NameError: name 'GPTSimpleVectorIndex' is not defined```

sleek thistle Jun 27, 2023, 2:02 PM

#

yes, need to update those part as well

winged hull Jun 27, 2023, 2:02 PM

#

ok sec

#

def ask_ai():
index = GPTVectorStoreIndex.load_from_disk('index.json')
while True:
query = input("What do you want to ask? ")
response = index.query(query)
display(Markdown(f"Response: {response.response}"))

sleek thistle Jun 27, 2023, 2:02 PM

#

storage_context = StorageContext.from_defaults(persist_dir="Directory of JSON")
index = load_index_from_storage(storage_context,service_context=service_context)

#

storage_context = StorageContext.from_defaults(persist_dir="Directory of JSON")
index = load_index_from_storage(storage_context,service_context=service_context)

#

update using this

winged hull Jun 27, 2023, 2:03 PM

#

#

like this?

sleek thistle Jun 27, 2023, 2:03 PM

#

def ask_ai():
storage_context = StorageContext.from_defaults(persist_dir="Directory of JSON")
index = load_index_from_storage(storage_context,service_context=service_context)
while True:
query = input("What do you want to ask? ")
response = index.query(query)
display(Markdown(f"Response: {response.response}"))

#

like this

#

yes, but need to do following things

#

import load_index_from_storage

#

and define service_context in ask_ai function

winged hull Jun 27, 2023, 2:04 PM

#

from llama_index import SimpleDirectoryReader, GPTListIndex, readers, GPTVectorStoreIndex, LLMPredictor, PromptHelper, ServiceContext, StorageContext
from langchain import OpenAI
import sys
import os
from IPython.display import Markdown, display

def construct_index(directory_path):
    # set maximum input size
    max_input_size = 4096
    # set number of output tokens
    num_outputs = 10000
    # set maximum chunk overlap
    max_chunk_overlap = 20
    # set chunk size limit
    chunk_size_limit = 600 

    # define prompt helper
    # prompt_helper = PromptHelper(max_input_size, num_outputs, max_chunk_overlap, chunk_size_limit=chunk_size_limit)

    # define LLM
    llm_predictor = LLMPredictor(llm=OpenAI(temperature=0.5, model_name="gpt-3.5-turbo-0613", max_tokens=num_outputs))
 
    documents = SimpleDirectoryReader(directory_path).load_data()
    

    # storage_context = StorageContext.from_defaults()
    # storage_context.persist(persist_dir="/Users/og/Downloads/RozGPT-main")

    # service_context = ServiceContext.from_defaults(llm_predictor=llm_predictor, prompt_helper=prompt_helper)
    service_context = ServiceContext.from_defaults(
            llm_predictor=llm_predictor,
            chunk_size=1024
        )
    index = GPTVectorStoreIndex.from_documents(documents, service_context=service_context)

    storage_context = StorageContext.from_defaults()
    storage_context.persist(persist_dir="/Users/og/Downloads/GPT-main")
    # index.save_to_disk('index.json')

    return index

def ask_ai():
    storage_context = StorageContext.from_defaults(persist_dir="index.json")
    index = load_index_from_storage(storage_context,service_context=service_context)
    while True: 
        query = input("What do you want to ask? ")
        response = index.query(query)
        display(Markdown(f"Response: <b>{response.response}</b>"))```

#

is this right? It giving errors

sleek thistle Jun 27, 2023, 2:05 PM

#

sleek thistle import load_index_from_storage

you can import that from llama-index

#

from llama_index import SimpleDirectoryReader, GPTListIndex, readers, GPTVectorStoreIndex, LLMPredictor, PromptHelper, ServiceContext, StorageContext, load_index_from_storage

winged hull Jun 27, 2023, 2:05 PM

#

Cell In[52], line 1
----> 1 ask_ai()

Cell In[49], line 43, in ask_ai()
     42 def ask_ai():
---> 43     storage_context = StorageContext.from_defaults(persist_dir="index.json")
     44     index = load_index_from_storage(storage_context,service_context=service_context)
     45     while True: 

File /opt/homebrew/lib/python3.11/site-packages/llama_index/storage/storage_context.py:75, in StorageContext.from_defaults(cls, docstore, index_store, vector_store, graph_store, persist_dir, fs)
     73     graph_store = graph_store or SimpleGraphStore()
     74 else:
---> 75     docstore = docstore or SimpleDocumentStore.from_persist_dir(
     76         persist_dir, fs=fs
     77     )
     78     index_store = index_store or SimpleIndexStore.from_persist_dir(
     79         persist_dir, fs=fs
     80     )
     81     vector_store = vector_store or SimpleVectorStore.from_persist_dir(
     82         persist_dir, fs=fs
     83     )

File /opt/homebrew/lib/python3.11/site-packages/llama_index/storage/docstore/simple_docstore.py:57, in SimpleDocumentStore.from_persist_dir(cls, persist_dir, namespace, fs)
...
--> 319         self.f = open(self.path, mode=self.mode)
    320         if self.compression:
    321             compress = compr[self.compression]

NotADirectoryError: [Errno 20] Not a directory: '/Users/og/Downloads/GPT-main/index.json/docstore.json'

sleek thistle Jun 27, 2023, 2:05 PM

#

and define service_context in ask_ai function

#

you are using that without define

#

@winged hull

winged hull Jun 27, 2023, 2:06 PM

#

Cell In[57], line 1
----> 1 ask_ai()

Cell In[54], line 44, in ask_ai()
     42 def ask_ai():
     43     service_context = ServiceContext.from_defaults(
---> 44             llm_predictor=llm_predictor,
     45             chunk_size=1024
     46         )
     47     storage_context = StorageContext.from_defaults(persist_dir="index.json")
     48     index = load_index_from_storage(storage_context,service_context=service_context)

NameError: name 'llm_predictor' is not defined```

sleek thistle Jun 27, 2023, 2:06 PM

#

persist_dir is where json file exist

#

also need to define llm_predictor

#

you are using that without define

#

hehe

#

hehe

winged hull Jun 27, 2023, 2:07 PM

#

so all this needs to go in ask_ai?

sleek thistle Jun 27, 2023, 2:07 PM

#

it seems that you are not familiar with python, friend

winged hull Jun 27, 2023, 2:08 PM

#

I am lol

sleek thistle Jun 27, 2023, 2:08 PM

#

winged hull so all this needs to go in ask_ai?

Yes

#

lol, I guess you are not familiar with that.

#

I can fix that script today for you.

#

just need to set up env for you.

winged hull Jun 27, 2023, 2:14 PM

#

hmm I defined it outside the scope to try and rig it, still doesn't like it

#how do i increase output response size?