Davinci 3 Context & Memory & Best Fine Tuning Model?

[stray roost]

Has anyone working with the API managed to give "text_davinci_003" limited recall or context sensitivity by linking it to a local database that it can query for relevant information and is there a specific set of logic that you used to make it contextually do so?

I'm trying to see if I can get the API to show the context sensitivity of ChatGPT, but it seems whatever I do it just doesn't work short of forcing a hidden version of all previous prompts and responses to be relayed to it (Which quickly accumulates into an ungodly number of tokens that hits the maximum).

Also, is curie the best model you can fine tune right now?

[wicked goblet]

For semantic search text embedding is a good idea

I'm just learning it

    const input = "Your message";  //the message sent
    const response = await openai.createEmbedding({
      model: "text-embedding-ada-002",
      input: input,
    });
    const array = response.data.data[0].embedding;
    const passage = "Avocado, also known as Persea americana, is a fruit that is highly valued for its nutritional and health benefits. Scientifically, it belongs to the Lauraceae family and is native to Mexico and Central America. The avocado tree can grow up to 20 meters tall, and its fruit is rich in monounsaturated fats, fiber, vitamins, and minerals.\n\nStudies have shown that consuming avocados can have positive effects on cardiovascular health by reducing cholesterol levels and inflammation. The fruit also contains antioxidants that may help prevent cancer and other chronic diseases.\n\nAvocado is also used in the cosmetic industry, as it is rich in essential fatty acids and vitamins that are beneficial for skin and hair. Additionally, avocado oil is used in cooking and as a salad dressing due to its high smoke point and mild flavor.\n\nOverall, the avocado is a versatile and nutritious fruit that has gained increasing popularity in recent years due to its health benefits and culinary uses.";
    const sentenceTokenizer = new natural.SentenceTokenizer();
    const sentences = sentenceTokenizer.tokenize(passage);
    let sentenceInfo;
    let sentenceArray = [];
    let success = false;
    for (let i = 0; i <= sentences.length - 1; i++){
      while (!success){
        try{
          sentenceInfo = await openai.createEmbedding({
            model: "text-embedding-ada-002",
            input: sentences[i],
          });
          sentenceArray[i]= sentenceInfo.data.data[0].embedding;
          success = true;
        }catch(error){console.log(error);}
      }
      success = false;
    }
    let output = sentences[0];
    let sim = similarity(array,sentenceArray[0]);
    for (let i = 1; i <= sentences.length - 1; i++){
      if (similarity(array,sentenceArray[i]) > sim){
        sim = similarity(array,sentenceArray[i]);
        output = sentences[i];
      }
    }
    message.reply(`${output}`);
    busy = false;
  })();

This is just an example

First let's see the variable "passage"

This is a passage of avocado

I use natural.SentenceTokenizer() to break the passage into sentences then store them into an array

sentences[]

then I use the textembedding api to convert each sentence into an embedding array

stored in the sentenceArray[]

Now assume that you want to search the height of the avocado tree

your input is "height"

Your input will also be turned into an embedding array: "array"

View this block

for (let i = 1; i <= sentences.length - 1; i++){
      if (similarity(array,sentenceArray[i]) > sim){
        sim = similarity(array,sentenceArray[i]);
        output = sentences[i];
      }
    }

this will compare the cos similarity of array with each sentences's embedding array ( the more the similarity is, the more relevant a sentenc are)

Then it will return the most relevant sentence to your input "height"

Output is The avocado tree can grow up to 20 meters tall, and its fruit is rich in monounsaturated fats, fiber, vitamins, and minerals.

You can use this to get the most relevant information in your database.

This is just what I've learnt, feel free to ask me if you need any help.

[final pawn]

wait, so you guys are fine-tuning text-davinci-003 here?

[wicked goblet]

nono, not fine-tuning

just talk about text embedding