Oh nice we should connect | OpenAI | Page 1

flint raven Apr 6, 2023, 1:35 AM

#

Threading this to be kind to the channel. Reading up now 👀

late wolf Apr 6, 2023, 1:36 AM

#

Yes! I’m use to slack

#

This would trigger a similar service but hyper focused for example a credit app which is really long!

flint raven Apr 6, 2023, 1:40 AM

#

\o/ cheers then, this is a great prompt and you've clearly got your head in the right place with this so far. I have not seen the approach of including the message contents in the JSON before. My only concern there would be that it may cause GPT to hallucinate.

Adding a little bit of a guard rail can help prevent this:

Please analyze the following customer message and identify if the contents contain specific customer information that can help us populate the MESSAGE ANALYSIS REPORT for [Customer First Name] [Customer Last Name].

Report Requirements: Please respond using the exact same JSON format as the MESSAGE_ANALYSIS_REPORT so our server can reliably parse it. Do not add any additional insight or comments. Use ONLY the contents of the message to complete the report. Assume only information in the message content is valid.

{
  "MESSAGE_ANALYSIS_REPORT": {
    "Vehicle_Updating": {
      "[Vehicle_1_Year_Make_Model_Trim]": "Y_or_N",
      "[Vehicle_2_Year_Make_Model_Trim]": "Y_or_N"
    },
    "Vehicle_of_Interest": {
      "Active_Deals": {
        "Deal_1": "[Vehicle_1_Year_Make_Model_Trim]",
        "Deal_2": "[Vehicle_2_Year_Make_Model_Trim]"
      },
      "Relevance": "Y_or_N"
    },
    "Trade_Ins": "Y_or_N",
    "Payment_Method": "Y_or_N",
    "Credit_Application": "Y_or_N",
    "Documents": {
      "Drivers_License": "Y_or_N",
      "Auto_Insurance": "Y_or_N",
      "Proof_of_Residence": "Y_or_N",
      "Proof_of_Income": "Y_or_N",
      "Reference": "Y_or_N"
    }
  }
}
### Message for analysis
{
  "MESSAGE_CONTENTS_FOR_ANALYSIS": {
    "type": "[EMAIL_or_SMS]",
    "role": "CUSTOMER",
    "actor_name": "[Customer First Name] [Customer Last Name]",
    "time": "[Time]",
    "date": "[Date]",
    "subject": "[Subject]",
    "body": "[Customer_Message]"
  }
}

If using the Chat endpoint instead of Completions, I would split the ### Message for analysis section out of the system instructions and instead place it as the first user message, allowing the assistant to respond to it.

#

tl;dr on the above is that it helps loads to just add some sort of natural signifier of where the output stops and where the input starts.

#

You might be able to simplify a bunch of the prompt as well, but I don't want to just monologue at you or tell you how you should best build it.
Happy to answer any questions you have though if I can!

late wolf Apr 6, 2023, 1:45 AM

#

Oh ya that’s a great idea to split it. Is that the main lever that would help it not hallucinate?no please would love to hear your feedback! I’m entertaining a toddler while trying to do this 😂

#

Ya I’d love make it shorter. A lot of this is informed by our current API. btw what company do you work for?

flint raven Apr 6, 2023, 1:51 AM

#

late wolf Oh ya that’s a great idea to split it. Is that the main lever that would help it...

There's other techniques but you have some of them baked in already, like being explicit about how you want the output to be (no additional insight). You can take this a step further by providing it default options, like saying "If you can't make an accurate determination, default to N".

#

I am not sure there's any de-facto resource out there on it but "anti-hallucination techniques for LLMs" on a search will get you far.

flint raven Apr 6, 2023, 1:52 AM

#

late wolf Ya I’d love make it shorter. A lot of this is informed by our current API. btw w...

Right now I'm in a side industry supplying data to the CRMs that supply dealerships, but I used to work for DealerOn! Your JSON struct reminds me of their work, or maybe someone from Dealer.COM 😆

#

I miss it tbh, even though car dealers are a PITA to deal with sometimes.

#

(and manufacturers...)

#

Who are you with if you don't mind sharing?

late wolf Apr 6, 2023, 1:54 AM

#

Ya the industry has been super neglected by tech which is fun for me.

#

I work at space.auto

flint raven Apr 6, 2023, 1:54 AM

#

You got that right. There's so much demand for development in that space from many angles.

late wolf Apr 6, 2023, 1:55 AM

#

I know all those companies well- where do you work I’m interested! Btw I’m hiring this month 😬

flint raven Apr 6, 2023, 1:56 AM

#

Right now I'm on a short contract with SambaSafety 🙂 doing driver risk assessment and analysis for insurance purposes, safety monitoring and that kind of thing.

#

Hiring eh, need any senior engineers / architects?

late wolf Apr 6, 2023, 1:56 AM

#

I do!

flint raven Apr 6, 2023, 1:56 AM

#

Well shoot, let me hit you up in DMs real quick!

#Oh nice we should connect