#Merging namespace members across modules

130 messages · Page 1 of 1 (latest)

misty burrow
#

I'm trying to merge namespace members across modules. I used to have all of them in a single file, but they're getting pretty big, since their main purpose is to correctly type all of my game packets.

I tried the code below, but when I hover over the Event type, it says never. TypeScript isn't properly extending anything and I have no idea what's the proper way to do this.

index.ts

export * from './Actions.js';
export * from './Items.js';
export * from './Chat.js';
export * from './Flock.js';
export * from './Io.js';
export * from './Items.js';
export * from './Join.js';
export * from './Prompt.js';

export namespace Client {
    interface Packets {};

    export type Event = keyof Packets;
    export type Payload<E extends Event> = Packets[E];

    export namespace Events {};
    export namespace Payloads {};
}

Chat.ts

export namespace Client {
    export interface Packets {
        [Events.Chat.MESSAGE]: Payloads.Chat.Message;
        [Events.Chat.MESSAGE_FAIL]: Payloads.Chat.MessageFail;
    }

    export namespace Events.Chat {
        export const MESSAGE = 'chat:message';
        export const MESSAGE_FAIL = 'chat:message-fail';
    }

    export namespace Payloads.Chat {
        export interface Message {
            id: number;
            message: string;
        }

        export interface MessageFail {
            message: string;
        }
    }
}
real crest
#

what's your motivation to wrap the entire file in export namespace Client { … } rather than just exporting things from the top level of the modules?

misty burrow
#

Not sure, so they're still accessible through Client.blablabla in other modules?

#

I'm just trying to merge the Client namespace

#

there will be another top namespace Called Packets, which will include both Client and Server

#

so in order to access a packet event, it would become like
Packets.Client.Events.Chat.MESSAGE, and its payload,
Packets.Client.Payloads.Chat.Message

real crest
#

if you care at all about bundling/tree-shaking you should avoid the style you had since it means Client is either all or nothing

#

anyway, to tie this back to your original question: does declaration merging work the way you want it to if you put things at the top level? like just as a test try moving Packets up to the top

misty burrow
misty burrow
#

So you're suggesting this right?
Chat.ts

export interface Packets {
    [Events.Chat.MESSAGE]: Payloads.Chat.SendMessage;
    [Events.Chat.MESSAGE_FAIL]: Payloads.Chat.MessageFail;
}

export namespace Events.Chat {
    export const MESSAGE = 'chat:message';
    export const MESSAGE_FAIL = 'chat:message-fail';
}

export namespace Payloads.Chat {
    export interface SendMessage {
        id: number;
        message: string;
    }

    export interface MessageFail {
        message: string;
    }
}
real crest
#

yeah, something like that

real crest
misty burrow
#

oh god

real crest
#

typical dramatic clickbait headline. read about them and come to your own conclusions (also after skimming that article it seems like the author is concerned about bundle sizes, which definitely aren't going to get any worse than they are with your current approach. also also i'm pretty sure modern bundlers can generally tree-shake things through barrel files just fine, though you may need to tell the bundler that your modules don't have top-level side effects (but if this is something you care about you should validate it yourself))

real crest
opaque spokeBOT
#
patofrango#0
Playground Link

Preview:```ts
// @filename: index.ts
export * from './Chat.js';

export namespace Client {
interface Packets {};

export type Event = keyof Packets;
export type Payload<E extends Event> = Packets[E];

export namespace Events {};
export namespace Payloads {};

...```

You can choose specific lines to embed by selecting them before copying the link.
misty burrow
#

First time using the playground

#

But here it is

real crest
#

oh, so first of all i meant to ditch export namespace Client from both files, but also the other Packets definition isn't even in scope when you do the keyof

uneven pendant
#

I vaguely recall some difficulty when trying to merge an interface cross-esmodule

#

oh and it's runtime merging of namespaces across esm, I don't think that's actually possible

#

will do some experimenting

real crest
#

if it turns out it's not possible to get implicit declaration merging to behave the way you want, keep in mind that you can always explicitly construct the type yourself:

import * as Chat from './Chat.js';
import * as Actions from './Actions.js';

type Packets = Chat.Packets & Actions.Packets
uneven pendant
#

do you know if

export namespace Events.Chat {
    export const MESSAGE = 'chat:message';
    export const MESSAGE_FAIL = 'chat:message-fail';
}

needs to be runtime available? or is this supposed to all be type-only

#

because they could use declare module to merge the types, but not via import/export

misty burrow
misty burrow
misty burrow
misty burrow
uneven pendant
#

then I can tell you the current approach isn't possible: typescript doesn't let you merge namespaces with imported namespaces even though they're very similar

real crest
#

looks like you could use declare global blocks even within modules:

opaque spokeBOT
#
mkantor#0
Playground Link

Preview:```ts
// @filename: index.ts
export * from './Chat.js';

declare global {
interface Packets {};
}

export namespace Client {
export type Event = keyof Packets;
export type Payload<E extends Event> = Packets[E];

export namespace Events {};
export namespace Payloads {};

...```

You can choose specific lines to embed by selecting them before copying the link.
misty burrow
misty burrow
uneven pendant
#

perhaps something like this?

opaque spokeBOT
#
webstrand#0
Playground Link

Preview:```ts
// @filename: index.ts
import * as C from "./Chat.js";

export namespace Client {
interface Packets {};
interface Packets extends C.Packets {}

export type Event = keyof Packets;
export type Payload<E extends Event> = Packets[E];


export namespace Events {

...```

You can choose specific lines to embed by selecting them before copying the link.
uneven pendant
#

since you're explicitly importing Chat.js

misty burrow
#

wow that looks promising, upon testing with a simple

const event = Client.Events.Chat.MESSAGE;
const payload: Client.Payloads.Chat.SendMessage;

it was able find those

#

the only downside is needing to keep manually merging them in index.ts by both importing and adding a namespace member

#

I actually used export import somewhere else in the codebase for something similar

#

what about the Packets interface though?

uneven pendant
misty burrow
#

the Packets interface would also need a new line for each module?

uneven pendant
#

interface Foo extends String, Number {} is apparently legal syntax

#

so you could do it as a list or one line each, the result is the same

#

declaration merging across modules that only modifies types is okay, since typescript doesn't actually execute sequentially

#

but if it's visible at runtime, i.e. exposes some variables that can be accessed, it can be really painful as you're at the mercy of ESM load-order

misty burrow
#

index.ts

import * as actions from './actions.js';
import * as items from './items.js';
import * as chat from './chat.js';
import * as flock from './flock.js';
import * as join from './join.js';
import * as prompt from './prompt.js';

export type Event = keyof Packets;
export type Payload<E extends Event> = Packets[E];

type Packets = actions.Packets &
        chat.Packets &
        flock.Packets &
        items.Packets &
        join.Packets &
        prompt.Packets;

export namespace Events {
    export import Chat = chat.Events;
}

export namespace Payloads {
    export import Chat = chat.Payloads;
}```

`chat.ts`
```ts
export interface Packets {
    [Events.MESSAGE]: Payloads.SendMessage;
    [Events.MESSAGE_FAIL]: Payloads.MessageFail;
}

export namespace Events {
    export const MESSAGE = 'chat:message';
    export const MESSAGE_FAIL = 'chat:message-fail';
}

export namespace Payloads {
    export interface SendMessage {
        id: number;
        message: string;
    }

    export interface MessageFail {
        message: string;
    }
}```

This is what I'm currently trying to work with
#

while this works, there's 2 things that suck about it:

  • I'm required to import each thing and add it to the list of Packets, Events, and Payloads
  • Each concrete module (like chat.ts) is required to follow this specific structure, where if I don't, I get no warnings whatsoever. Is there a way to at least define the structure a module should follow or something?
uneven pendant
#

there isn't a way to declare a schema for a module

#

this approach isn't one that typescript makes convenient

#

you can use

declare module "./index" {
    namespace Client {
       interface Packets {
           // merge your stuff here
       }
    }
}

which is how most projects do plugins which need to inject types

#

but it only works for types, not runtime values

real crest
#

beauty is in the eye of the beholder, but personally i find what i suggested [before](#1335979493776687155 message):

type Packets =
  & actions.Packets
  & chat.Packets
  & flock.Packets
  & items.Packets
  & join.Packets
  & prompt.Packets

to be stylistically nicer than this:

interface Packets
  extends actions.Packets,
    chat.Packets,
    flock.Packets,
    items.Packets,
    join.Packets,
    prompt.Packets {}
#

(type info may display differently though)

uneven pendant
#

yeah either way I don't think it matters. I used interface because that's what was already there

#

but if you don't need ad-hoc extension, type is probably better

misty burrow
misty burrow
real crest
#

oh heh i guess i didn't scroll back up after you edited

misty burrow
#

it does act a bit different:

image.png
#

it allows me to directly use the correct type of event. BUT. it writes it as a string, which I NEVER use. I always refer to them from the consts namespace, so like

const packets: Packets = {
    [Events.Actions.ANIMATION]: myTypeHere
}```
#

either way, it's not important, because Packets is not an exported type, if you didn't notice: it's encapsulated and abstracted by the types Event and Payload

#

these:

export type Event = keyof Packets;
export type Payload<E extends Event> = Packets[E];```
#

hmmm... but I guess since its an alias it might have the same problem

#

maybe if I use an enum instead of a namespace of consts when defining events

#

which I was actually using before

#

then it will force me to use the enum

real crest
#

sorry i didn't mean to derail the discussion to focus on that specific detail, it's not that important

misty burrow
#

nono you're not derailing at all, it's very ontopic since its part of the major refactor I'm doing to packets. I was just rambling alone at this point

misty burrow
real crest
#

so stepping back, big-picture-wise it seems like you have two dimensions:

  • actions/chat/flock/items/etc
  • packets/events/payloads/etc
    (i dunno what to refer to those categories as, if you have suggestions it'd make discussion easier)
#

it seems like some of your pain is coming from the fact that you want to group things into files along the first dimension, but then in index.ts you want to slice it the other way and group things together by the second dimension

#

instead could you split things up in a consistent way? either by changing how things are grouped in the exports of index.ts (so you'd just have re-exports in that file) or rearranging your files to have like packets.ts instead of chat.ts

misty burrow
#

So the first dimension (actions/chat/flock/items/etc) refer to what are internally called "network plugins" or just "plugins". they're plugins that register the event listeners for a group of events (listeners for player actions, listeners for the chat, etc.).

The second dimension is like the "building blocks" of each plugin. Each plugin has its event names, the payload that event carries across the network, and the "packets" is just a map that binds an event to a payload. The packets map is useful for when I need to type-out methods that send packets.

For example, in my code I have:

public send<E extends Packets.Client.Event>(
    event: E,
    payload: Packets.Client.Payload<E>
) {
    this.socket.emit(event, payload);
}

This method assures that I can only send the correct payload structure associated with a given event.

An example of good usage is:

this.send(Packets.Client.Event.Chat.MESSAGE, { id: 1, message: 'My message' };

And a bad example is:

this.send(Packets.Client.Event.Chat.MESSAGE, { blabla: true };

TypeScript will alert me that the MESSAGE event doesn't accept that payload format.

real crest
#

ah, your use of the term "plugins" already tells me a lot. do they need to be pluggable by third parties? like someone using your game engine can decide to add their own plugin without your codebase knowing anything about it?

#

because if so i think none of the suggestions that involve explicitly enumerating the packet types will work

misty burrow
#

So, while flexibility isn't as crucial as "anyone is gonna add plugins", I wanted to make it flexible so that me and eventual team members don't need to fret too much when adding more

real crest
#

yeah, so you want to be able to add/remove plugins without touching 10 different files

#

makes sense

misty burrow
#

as few as possible, yeah

# 
export type Event = keyof Packets;
export type Payload<E extends Event> = Packets[E];

import * as actions from './actions.js';
import * as items from './items.js';
import * as chat from './chat.js';
import * as flock from './flock.js';
import * as join from './join.js';
import * as prompt from './prompt.js';

type Packets = actions.Packets &
    chat.Packets &
    flock.Packets &
    items.Packets &
    join.Packets &
    prompt.Packets;

export namespace Events {
    export import Actions = actions.Events;
    export import Items = items.Events;
    export import Chat = chat.Events;
    export import Flock = flock.Events;
    export import Join = join.Events;
    export import Prompt = prompt.Events;
}

export namespace Payloads {
    export import Actions = actions.Payloads;
    export import Items = items.Payloads;
    export import Chat = chat.Payloads;
    export import Flock = flock.Payloads;
    export import Join = join.Payloads;
    export import Prompt = prompt.Payloads;
}```
#

this is the current client/index.ts

#

there's also going to be a server/index.ts with the same structure, but meant for packets sending to the server

#
  • client -> packets sending to client
  • server -> packets sending to server
#

so it will be yet another module with listings

real crest
#

one idea to possibly be able to get both declaration merging and still allow plugins to provide value-level code is to split each plugin into two files. one would be ambient (not a module) and the other one would just contain the values

#

former could even be a .d.ts file

misty burrow
misty burrow
#

I'm confused at the ambient part

real crest
#

Sorry had to step away, but I can when I get back to my computer later

misty burrow
#

no problem, please do let me know once you can :)

real crest
#

actually seems like you don't even need to use separate files; using declare global like i mentioned [earlier](#1335979493776687155 message) lets you do what i was suggesting from within a module:

opaque spokeBOT
#
mkantor#0
Playground Link

Preview:```ts
// @filename: index.ts
export type Event = keyof Packets
export type Payload<E extends Event> = Packets[E]

import * as actions from './actions.js'
import * as chat from './chat.js'

export namespace Events {
export import Actions = actions.Events
export import Chat = chat.Events
...```

You can choose specific lines to embed by selecting them before copying the link.
real crest
#

☝️ with that setup the Packets interface will be declaration-merged together like you originally wanted (Payloads too, though you could change that if it's not desirable)

misty burrow
#

I fail to understand what the point/advantage of that would be

#

that I dont have to list out the packet interfaces and payloads, but still need to list the events?

#

also, declare global would probably make those things global, when they should be nested inside Packets.Client

#

src/index.ts

import * as client from './client/index.js';
import * as server from './server/index.js';

namespace Packets {
    export import Client = client;
    export import Server = server;
}

export default Packets;
#

src/client/index.ts

export type Event = keyof Packets;
export type Payload<E extends Event> = Packets[E];

import * as actions from './actions.js';
import * as items from './items.js';
import * as chat from './chat.js';
import * as flock from './flock.js';
import * as io from './io.js';
import * as join from './join.js';
import * as prompt from './prompt.js';

type Packets = actions.Packets &
    chat.Packets &
    flock.Packets &
    items.Packets &
    join.Packets &
    prompt.Packets;

export namespace Events {
    export import Actions = actions.Events;
    export import Io = io.Events;
    export import Items = items.Events;
    export import Chat = chat.Events;
    export import Flock = flock.Events;
    export import Join = join.Events;
    export import Prompt = prompt.Events;
}

export namespace Payloads {
    export import Actions = actions.Payloads;
    export import Items = items.Payloads;
    export import Chat = chat.Payloads;
    export import Flock = flock.Payloads;
    export import Join = join.Payloads;
    export import Prompt = prompt.Payloads;
}```
real crest
#

and you can nest things in namespaces with this approach:

opaque spokeBOT
#
mkantor#0
Playground Link

Preview:```ts
// @filename: index.ts
export type Event = keyof Client.Packets
export type Payload<E extends Event> = Client.Packets[E]

import * as actions from './actions.js'
import * as chat from './chat.js'

export namespace Events {
export import Actions = actions.Events
...```

You can choose specific lines to embed by selecting them before copying the link.
uneven pendant
#

This is a common pattern:

opaque spokeBOT
#
webstrand#0
Playground Link

Preview:```ts
// @filename: index.ts
import type { Packets, Payloads } from "./plugins";
export type Event = keyof Packets
export type Payload<E extends Event> = Packets[E]

import * as actions from './actions.js'
import * as chat from './chat.js'

export namespace Events {
...```

You can choose specific lines to embed by selecting them before copying the link.
uneven pendant
#

instead of using global, it uses a dedicated module which plugins declare types into

misty burrow
#

I modified it a bit by doing:

src/client.ts

interface Packets {}

export type Event = keyof Packets;
export type Payload<E extends Event> = Packets[E];

export namespace Events {}
export namespace Payloads {}

src/client/actions.ts

import type { AvatarAnimationKey } from '@quacker/consts';

declare module '../client.js' {
    interface Packets {
        [Events.Actions.POSITION]: Payloads.Actions.UpdatePosition;
        [Events.Actions.ANIMATION]: Payloads.Actions.UpdateAnimation;
    }

    namespace Events {
        export enum Actions {
            POSITION = 'actions:position',
            ANIMATION = 'actions:animation'
        }
    }

    namespace Payloads.Actions {
        export interface UpdatePosition {
            x: number;
            y: number;
        }
    
        export interface UpdateAnimation {
            key: AvatarAnimationKey;
            save: boolean;
        }
    }
}
#

this preserves exactly the same structure I initially had, and is the true way of splitting it into multiple modules. Since the declared members become officially part of the client.ts module, VS Code refactoring (like renaming) also works.

#

declare module essentially "brings in" the scope of the module you declare

#

and since declaration merging only works in the same module, this is the way to do it across modules

#

you need to expand the parent module's scope by using declare module

#

so yeah, I think I'm going with this. no weird lists required, and the structure is preserved

uneven pendant
#

users of your library ought not to be surprised by it

misty burrow
#

sorry, i realized i sent the old version. there, just edited the message

#

now it uses declare module

uneven pendant
#

they still haven't removed it, for want of a better solution

misty burrow
#

I bumped into an issue

#

let me prepare an example