TJ Bot | Creating Tests for AI Portion | Together Java | Page 1

fresh creek Jun 14, 2023, 2:53 AM

#

I want to get some work done on integrating better tests into the AI/ChatGPT portion of the bot. Thing is I am a bit new to mocking/unit/testing in general (I get the general concepts but no practical experience). My main question is:
Do I need to mock the service itself or just the config? In regards to Mockito, what does mocking mean?

loud heraldBOT Jun 14, 2023, 2:53 AM

#

<@&987246924425994290> please have a look, thanks.

loud heraldBOT Jun 14, 2023, 2:53 AM

#

loud herald <@&987246924425994290> please have a look, thanks.

#

While you are waiting for getting help, here are some tips to improve your experience:

Code is much easier to read if posted with syntax highlighting and proper formatting.

If nobody is calling back, that usually means that your question was not well asked and hence nobody feels confident enough answering. Try to use your time to elaborate, provide details, context, more code, examples and maybe some screenshots. With enough info, someone knows the answer for sure.

Don't forget to close your thread using the command </help-thread close:1027500463647621170> when your question has been answered, thanks.

shy pelican Jun 14, 2023, 4:22 AM

#

fresh creek I want to get some work done on integrating better tests into the AI/ChatGPT por...

Mocking means to fake data.

For example, if your service relies on a clock, you wouldn't want to have to wait for the clock to reach a specific time in order to test what you're trying to test.

For example, maybe you're testing a service that sends an email at a specific time. You shouldn't have to wait for the clock to reach the time needed for the email to send.

So you'd use a fake/mock clock. The mock allows you to specify the time for the clock.

#

You wouldn't mock the service you are testing, that wouldn't make sense. You would use the real service, and mock it's dependencies

fresh creek Jun 14, 2023, 4:26 AM

#

shy pelican You wouldn't mock the service you are testing, that wouldn't make sense. You wou...

Service as in the API I am communicating with (ChatGPT)?

pallid fog Jun 14, 2023, 5:25 AM

#

u can have a look at some of the existing tests for details on mocking. but essentially, mockito let's u freely define what a method should return, instead of calling it's true code

#

Foo foo = mock(Foo.class);
when(foo.bar()).thenReturn(5);

...
int result = foo.bar(); // 5

#

u can do that with any class and any of its methods

#

for the ai thing, what u have to avoid is that u actually talk to the real openai api. u shouldn't produce any traffic

#

what u mock depends on what u test

#

when u test code that would use maybe some OpenApiService class that it gets, then u mock that service

#

for example maybe it has an askQuestion method, returning a string response, which the code ur testing is using

#

OpenAiService service = mock(OpenAiService.class);

when(service.askQuestion(any()).then("Test response");

myThingUnderTest.foo(service,...);

...

#

if u want to test the service itself, u have to mock a level deeper. it depends on what u test and what its inputs are

#

u mock the inputs that u don't want to create true objects for

#

the should be plenty of examples in the existing tests. we make heavy use of mocking for anything discord related

#

for example when u want to test a class that sends a message and then want to check that it send a specific text, u have to do that via our mocked discord api

#

every second test in the code base will be full with mockito things

shy pelican Jun 14, 2023, 2:58 PM

#

Yeah. For example, if the service was to display a message obtained from ChatGPT, you wouldn't want ChatGPT going down affecting your ability to test your service @fresh creek

#

So you would mock the responses you'd get from the API

pallid fog Jun 14, 2023, 3:21 PM

#

to make it concrete

#

if ur writing tests for ChatGptCommand, it wants ChatGptService as param

#

so u mock that

#

if u write tests for ChatGptService, u would mock its OpenAiService class that it uses

fresh creek Jun 15, 2023, 3:41 AM

#

Ok - That helped to clear up a bit of what I need to do. Thanks everyone.

Just to clarify what I am doing (and to ensure that I am doing the right thing), I am to simulate a raw response back from the AI API. The ChatGPTService needs to break up the message into smaller bits if the response is greater than 2000 characters (even though we asked it very nicely not to do that). So I want to test the response on a known answer and hopefully get consistent results.

shy pelican Jun 15, 2023, 4:58 AM

#

fresh creek Ok - That helped to clear up a bit of what I need to do. Thanks everyone. Just ...

You want to test if ChatGPT gives a correct answer?

#

Not sure what you mean by "Test a response on a known answer and get consistent results"

fresh creek Jun 15, 2023, 5:00 AM

#

shy pelican You want to test if ChatGPT gives a correct answer?

No.
Based on how the discord API works, we can only send messages of 2000 chars at a time. If we get a larger response, we have to stream them in as 2000 char blocks. So I need to split it. I am really testing the splitting and ensuring I am still below 2000 characters, and that the formatting isn't bad.

#

Consistent in this case means "for a known response (an old ChatGPT answer), the splitting works and doesn't break the response over too messages poorly, and the length is below 2000 for each block passed back"

#

An example of a bad split: because of where the split was made, the code sections get spliced and now all the normal text comes off as code.

shy pelican Jun 15, 2023, 5:04 AM

#

Oh, so yeah, you would mock the response

fresh creek Jun 15, 2023, 5:05 AM

#

Trying too - still hitting the API despite mocking so not sure what I am doing wrong. Also, need to pass in rather large test cases so that is taking me a while to setup.

shy pelican Jun 15, 2023, 5:06 AM

#

You must not be mocking properly. Would have to see some code to be able to help

#

Although your service should just take in a message object of some sort, yeah? If it's meant to receive responses, it doesn't need to reference the actual API

#

It could be the responsibility of a separate object to actually communicate with the API & pass the response to the service that parses it

fresh creek Jun 15, 2023, 5:07 AM

#

so splitting is (dumbly) within the same method that generates responses (as a private method call).

pallid fog Jun 15, 2023, 7:06 AM

#

again

#

if ur writing tests for ChatGptCommand, it wants ChatGptService as param
if u write tests for ChatGptService, u would mock its OpenAiService class that it uses

#

this is as concrete as it can be

#

looking at ur PR, uve put the splitting in the wrong method

#

the service should not deal with splitting

#

the service must not assume how the result is used

#

ur assuming that the service is used to post discord messages

#

but the servivce must be general purpose

#

its only supposed to ask a question to chatgpt and return the result as string (or empty optional, ...)

#

its not supposed to assume intention

#

the splitting thing has to happen a level later

#

that said, with the way u have it set up right now, u would write a test for ChatGptService and mock OpenAiService

#

when(openAiService.createChatCompletion(any()).thenReturn(loooooongText);

#

var result = chatgptService.ask("foo");

assertTrue(result.isPresent());
assertEquals(result.orElseThrow(), 3); // split into 3 messages

#

and so on

#TJ Bot | Creating Tests for AI Portion