#mock unit-integration testing

99 messages · Page 1 of 1 (latest)

jagged shale

Thread about developing a mock gateway/REST server for integration testing -- might be useful for other discord libraries as well, but we all know djs is the best. I've already got a working PoC with a gateway and REST server, and discord-api-types would make development much quicker.

mock unit-integration testing

loud saffron

What I always wondered, is whether we need this at all.

If you use a proper testing framework, you'll have access to not only mocks, but also spies that are able to intercept calls and allow you to assert both input and output, something more powerful and more convenient than reverse-engineering Discord (which is against Discord's ToS), as you can also test if you're passing the right instances and not just clones.

And of course, you can also use this exact same system, not only for discord.js, but also for your bot's code.

An example of spies in Jest: https://jestjs.io/docs/jest-object#jestspyonobject-methodname.

What's more handy for me is an easy way of creating mocks without needing to specify all the data, for which I just create a template object and then use object spread to assign custom values for tests. What could be improved in discord.js's side is that we could type the payloads each structure's constructor accepts, as opposed to just unknown.

severe mesa

Which I am going to do probably later today actually, been planning on it for a bit

jagged shale

So is a REST/gateway server useful for this?

loud saffron

Not at all

You can completely avoid those by mocking your d.js calls

const channelSendSpy = jest.spyOn(channel, 'send'); and then you run operations on it. When you do channel.send(), it'll call whichever implementation you passed on channelSendSpy.mockImplementation(...), then you can assert whether the parameters and the returned data was right.

severe mesa

alrighty, here's the pr

grizzled spokeBOT
buoyant parrot

How are people making mock structures?

buoyant parrot
loud saffron

discord-api-types to type the raw data as the one Discord sends us

buoyant parrot

So you have to create the entire hierarchy just to make a mock message structure 🤔

loud saffron

Not necessarily

You can test a partial DM message, works fine for most cases

Unless you're testing stuff that requires a guild, or the channel, then in that case... yeah, you gotta mock more structures

buoyant parrot

Yeah, I'm migrating a ton of existing tests from Eris, for which we never really established the best structure for

currently they're all breaking because the logging facilities log the channel snowflake

loud saffron

I'm unsure what you're referring to with logging facility

buoyant parrot

I think I can work around it with channelId instead of channel.id

loud saffron

Ah, yeah

buoyant parrot
loud saffron

As a side note, our plans for v14 will probably make mocking enormously easier, since we'll store a good chunk of the raw data

buoyant parrot

Aha, that's exciting 🙂

loud saffron

And use getters to decouple structures from state

buoyant parrot

Unrelated, but can you tell me what that GitHub bot is with the slash command? I think it might be useful in some of our servers

loud saffron

Lemme check if I can find it

loud saffron
buoyant parrot

Oh damn, even CloudFlare has a Lambda competitor

So it's self-hosted

loud saffron

Yeah, a free one and pretty good

Souji is staff here

It's this cutie

buoyant parrot

Neat

It's hard to get word in here because the channels are always flooded with mostly questions about programming fundamentals... the woes of popularity XD

loud saffron

XD

buoyant parrot

I'll ask something else less on topic for this thread: is it a design choice of Discord.js for almost all actions to be invoked as methods on object instances? Even when they might correspond directly to a single REST API call?

loud saffron

And we might accidentally kill all competition once we finish our raw components

wdym

buoyant parrot

For example, to send a message, I need to fetch the channel object first. To leave a server, I need to fetch the guild object first.

loud saffron

It's called OOP, and it's part of SOLID

buoyant parrot

Under Eris, the Client object already has createMessage and leaveGuild methods, mapping one-to-one to REST APIs, since you don't actually need to make a pre-request to fetch all the data to construct the channel and guild objects to perform the action.

loud saffron

It was a lot worse in the past, you had to fetch the message in order to edit or delete it, we now have those methods in channel.messages so we can do those operations without cache (since v13)

Yeah, I know

buoyant parrot

So it is a design choice

loud saffron

I don't think there are plans at all to one-up that hierarchy, buuuuuuuuuut

buoyant parrot

I did hear from the channel in the API server about @discordjs/rest being spun off, so that might help for some one-offs

severe mesa

GuildManager#leave could exist, that's not too hard to do

loud saffron

Once we implement the full new stack (discord.js in TS), we might need to expose the raw REST handler

buoyant parrot

Oh that's sick

loud saffron

And by then we'll also have pretty convenient builders to create the data it accepts

buoyant parrot

That makes it more like Detritus doesn't it

loud saffron

It's what I meant with us possibly killing all competition in the future, our plans for d.js's future are pretty bright

buoyant parrot

but then you'll have people asking about TypeScript fundamentals too

loud saffron

We plan on making @discordjs/rest very complete, supporting custom ratelimit cache strategies (hint: Redis)

buoyant parrot

👀

~~why are reactions disabled in this server 😭 ~~

loud saffron

At the same time we'll soon also work on @discordjs/ws, with opt-in internal worker sharding

So if you want to go full raw, you'd ideally use /rest + /ws

discord.js will just leverage all the components into a single, power-user oriented, package

loud saffron
buoyant parrot

I'm not actually into "full raw" (evidenced by my struggles with unit testing), just annoyed when an API that's meant to be a convenience is cumbersome or less efficient

loud saffron

If you ask me, KODER

The inefficiency of discord.js for mocking is enormously outweighed by the slowness of Jest

buoyant parrot

Okay, in that context I meant inefficiency in terms of Discord API calls made (this was referring to leaving guilds and sending messages)

Inefficiency in terms of mocking is just how long the tests take to write and how complex they are to set up

loud saffron

You... don't do Discord API calls in test suites

buoyant parrot

For testing, it's more about my developer time, not how long they take to run on CI, unless we start wasting tens of minutes or worse

loud saffron
buoyant parrot

We're talking two things at once and it's confusing 😅

buoyant parrot
loud saffron

I mean... for the test suite, I can just keep adding one-line helpers to construct data of one type

You can see how I can construct a User instance with just createUser(), a role with createRole(), and so on

buoyant parrot

Yeah, I'm going to try it out, this helps

loud saffron
buoyant parrot

So embeds and attachments are instances of classes in Discord.js and not just plain old objects. In a test where Message::reply has been stubbed out, how do you make assertions on them?

loud saffron

Embeds and attachments are instances of classes that users can construct easily, since they're what we call, constructor classes

As for stubbing Message#reply, you have 2 ways for that:

1.- Using spies: jest.spyOn(message, 'reply').mockImplementation(...), very powerful, but can be repetitive and skips a lot of discord.js's logic, not ideal if you want to do e2e integration testing, also dependent of a testing suite, which may not be ideal

2.- Using HTTP mocks: A lot lower level, can be tricky in some specific cases, but overall can be a lot easier (as easy as creating mock data for most cases, just like I'm doing in my tests), using the library I linked before (https://mswjs.io/)

(You can use nock too if you want)

buoyant parrot

Situation: Message#reply is already stubbed. The command in question is going to reply with an embed or an attachment. Under Eris, I either had an assertion on a specific nested property or a deep comparison since it was a plain object. Here I don't think a deep comparison works since it's not a plain object. The property assertion is rather messy with the typing of the reply arguments I think. I'm trying something like

const replySpy = this.stub(sampleMessage, "reply").resolves();
await command(sampleMessage):
const reply = replySpy.args[0][0];
expect(reply).to.have.property("embeds");
if (typeof reply === "object" && "embeds" in reply) {
    expect(reply.embeds?.[0]?.fields?.[1]?.value).to.equal("Some specific thing");
}
loud saffron

You can mix the two things I shared btw, although the first makes the second less useful since you can't check for the arguments that were passed

loud saffron

If you're passing a MessageEmbed, you can use another to compare

buoyant parrot

mocha, chai, sinon ecosystem, both jest and mocha have Jasmine syntax

loud saffron

Otherwise, if what you're doing is comparing MessageEmbed vs raw data, do either me.toJSON() or new MessageEmbed(raw)

buoyant parrot

And the same is true for attachments?

loud saffron

Yes

Attachments are a bit more complicated

The data you send is different from the data you receive, the file contents are defined in the former (for obvious reasons) while missing in the latter

I don't think you can really do that for attachments, now that I think of it, unless you have the raw data

empty merlin

Well I will try here now... Is there a channel I can ask about unit tests? Of a bot, not the library. It is proving quite a bit more difficult with slash commands as I pretty much need CommandInteraction objects. Is there a way to create one through the library with particular values? Can I "new" one somehow?

loud saffron

There's literally a ton of information about that in this channel, please read the previous messages, @empty merlin

empty merlin

I surely will. I didn't know that this was a "thread" and/or how it worked.

empty merlin

The messages seem to indicate (at least) two layers of unit testing. I wasn't so very interested in testing parts of discord.js as I assume any errors there would be found by testers of the library. Not entirely unlike our relying on JS or Node or the .Net libraries to work according to spec. My testing system for message commands worked very well precisely because the values needed for testing were parsed out of a string and passed along to functions as primitives. Simple to generate dozens of them for tests. One thought I had (but I am very reluctant to write my functions this way) is to separate the "function" into 2 parts. If there are "interaction aware" and "primitive value aware" functions then the unit tests could test the primitive value ones basically trusting that something like interaction.options.getString('msg') would return a string (or discord.js is broken).

I'd be interested how other expose private functions for testing but this system I read about works well for me. It obviously doesn't make the function private if someone is intent on calling it but it makes it very well known that calling it isn't intended. So for example: