surreal snow
Attached is a short sample of a longer (and excellent) conversation. I asked Chat to apply a simple framework for evaluating materials based on desired characteristics. 1 is good or better, 0 is average, -1 is poor or worse. It understood the request immediately. I observed two problems:
-
From response to response, it's evaluations changed (see Corrosion Resistance for Gold as one example). I never got two consecutive identical assessments for the same material in the entire conversation.
-
New calculated values (such as Score) are very unreliable, and often take two or three requests to fix (during which time the underlying data may shift). In this example it's easy to spot, but there are many cases, such as simple average, where the user may not be able to tell on sight that there's a problem.
Asking ChatGPT for a descriptive assessment of, say, the Corrosion Resistance of Gold provided more consistent results, so it "knows" the correct answer. It also clearly understands the framework I developed. But it appears to assess the data each time, and isn't sticking to it's original assessment.
