#Intersection with a modified generic function in a higher-order function

102 messages · Page 1 of 1 (latest)

sudden mason
#

Hi yall, I'm trying to do some advanced type wrangling. I need a higher-order function that takes in a generic function as parameter. It should return a modified version of that function with a key attached to it. Here's what works and where it starts to break:
(playground link)

any clue on how to get scenario #4 to work? some trick to make typescript not bail out? all help greatly appreciated!

sudden mason
#

here's the code from the playground:

const identity = </* const */ T>(x: T) => x

// #1 - just returning the function.
// this works, the return value is still generic
declare function scenario1<TFn extends (...args: any[]) => any>(fn: TFn): TFn;
const type1 = scenario1(identity)(123)
//    ^? 123

// #2 - returning the function with a key attached to it
// this works, the return value is still generic and the key is there
declare function scenario2<TFn extends (...args: any[]) => any>(fn: TFn): TFn & {foo: 'bar'}
const type2_1 = scenario2(identity).foo
//    ^? 'bar'
const type2_2 = scenario2(identity)(123)
//    ^? 123

// #3 - returning a modified generic function
// this works, though two generic params are needed as opposed to just TFn,
// because using Parameters<TFn> and ReturnType<TFn> removes the genericness
declare function scenario3<TParams extends any[], TReturn>(
  fn: (...args: TParams) => TReturn
): (...args: TParams) => {call: () => TReturn}
const type3 = scenario3(identity)(123).call()
//    ^? 123

// #4 - returning a modified generic function with a key attached to it
// this DOESN'T work. it seems like Typescript bails out and doesn't consider
// the function to be generic anymore
declare function scenario4<TParams extends any[], TReturn>(
  fn: (...args: TParams) => TReturn
): ((...args: TParams) => {call: () => TReturn}) & {foo: 'bar'}

const type4_1 = scenario4(identity).foo
//    ^? 'bar'
const type4_2 = scenario4(identity)(123).call()
//    ^? any
// the above should be 123 instead of any
worthy void
#

are you having a problem with #2? that is IMO the "right" way to do it

sudden mason
# worthy void are you having a problem with #2? that is IMO the "right" way to do it

hi, in #2 the function is unaltered. but if you look at scenarios 3 and 4, I alter the function. In the case of the identity function, it changes from <T>(x: T) => T to <T>(x: T) => {call: () => T} (I need it in this form precisely). scenario #3 shows that working, but as soon as I add that intersection with that additional key (scenario #4), it stops working.

#

does that make sense?

worthy void
#

ah, i didn't read closely enough. i saw .call but assumed it was this

sudden mason
#

yup, sorry, just placeholder names anyway

worthy void
#

there may not be a way to do what you want. in a generic function the genericness is associated with the function type itself, and you can't easily pick it apart while keeping it generic

#

does the function in question always have a single type parameter?

#

should i be allowed to pass a non-generic function in?

sudden mason
#

the function would be user supplied - any function at all should work. then my function does some stuff and spits out the modified function with the call() and foo: 'bar'

#

yeah it seems hard to get typescript to keep the genericness. but maybe there is some kind of trick... I tried doing stuff like interfaces, putting stuff behind more types, etc. but none worked so far

worthy void
#

how complicated is your real use case? if it's not too wild, can you share it? it may not be possible to solve this in a general/abstract way, but i might have ideas about how to pull off the specific thing you are trying to pull off

sudden mason
#

it's curious though because scenario #3 works, so it seems like maybe typescript has some code to detect that type of type signature and it preserves the genericness. but if the signature is more complex, i.e. with an intersection, it bails. just a guess

#

well the implementation is a little wild I guess. but it really boils down to the code snippet above, since at the end of the implementation I do a type assertion anyway... I can try to give you a gist of it in a sec, it's a state management library. what's your idea?

worthy void
#

check out how it infers:

meager pewterBOT
# 
const identity = <const T>(x: T) => x

declare function scenario3<TParams extends any[], TReturn>(
  fn: (...args: TParams) => TReturn
): (...args: TParams) => {call: () => TReturn}
scenario3(identity)
//      ^? - function scenario3<[x: T], T>(fn: (x: T) => T): <const T>(x: T) => {
//          call: () => T;
//      }
worthy void
#

TS is apparently smart enough to realize it can lift the type parameter from identity up to take the place of the second type parameter for scenario3

#

which is kind of cool, actually. i'm not sure i knew that was possible

worthy void
sudden mason
#

yeah just checked, it works with two params

#

there must be some way to have that behavior kick in even when the type signature is not trivial...

worthy void
#

is there a problem caused by having two type parameters?

sudden mason
#

I think you misunderstood. I'm saying this works:

const test = <const A, const B>(a: A, b: B) => ({a, b})

declare function scenario3<TParams extends any[], TReturn>(
  fn: (...args: TParams) => TReturn
): (...args: TParams) => {call: () => TReturn}
const type1 = scenario3(test)(123, 456).call()

but still it's impossible to attach that key, the genericness is lost

#

maybe there could be a way for typescript to trigger that behavior early somehow, like calling another function to modify the function and then attach the key... checking now

worthy void
#

does the key belong at the top level of the returned function or in the return value of that function?

#

i think you're just getting mixed up about type operator precedence:

sudden mason
#

at the top level, on the function itself

meager pewterBOT
# 
const identity = <const T>(x: T) => x

declare function scenario3<TParams extends any[], TReturn>(
  fn: (...args: TParams) => TReturn
): ((...args: TParams) => {call: () => TReturn}) & {foo: 'bar'}
const type3_1 = scenario3(identity).foo
//    ^? - const type3_1: "bar"
const type3_2 = scenario3(identity)(123).call()
//    ^? - const type3_2: any
worthy void
#

erm wait

#

sorry i thought that worked in the playground, but had the wrong variable name. too many scenarios 😅

sudden mason
#

yeah this type signature is what I want
((...args: TParams) => {call: () => TReturn}) & {foo: 'bar'}
i.e. foo sits on the function itself, before it's called

#

just tried this also:

const test = <A>(a: A) => a

const alter =<TParams extends any[], TReturn>(fn: (...args: TParams) => TReturn) => {
  return null as any as (...args: TParams) => {call: () => TReturn}
}

const attach = <TParams extends any[], TReturn>(
  fn: (...args: TParams) => TReturn
) => {
  const altered = alter(fn)
  return Object.assign(altered, {foo: 'bar'}) as typeof altered & {foo: 'bar'}
}

const type = attach(test)(123).call()
//    ^? any

so it didn't work, but maybe it's closer to the solution?

#

putting return altered in attach does maintain genericness, but yeah, I need to add that key...

rare cradle
#

I'm surprised #3 even worked, didn't know TS could do that.

worthy void
worthy void
sudden mason
#

hmm I don't think it's really possible to modify the design - I need to put some metadata on the function so that I can read from it without having to call it (important), and the function needs to be slightly altered
I'll try to show you a reduced snippet in just a second, but I do think it boils down to the above

worthy void
rare cradle
#

Two things I tried:

  • { (...args): ..., foo: 'bar' } instead of ((...args) => ...) & { foo: 'bar' }, doesn't work.
  • scenario3 + addFoo, and it works if you do addFoo(scenario3(identity)), but it doesn't work if you try to abstract that into a function.
sudden mason
rare cradle
#

Here:

meager pewterBOT
# 
const identity = <T>(x: T) => x

declare function scenario3<TParams extends any[], TReturn>(
  fn: (...args: TParams) => TReturn
): (...args: TParams) => {call: () => TReturn}

declare function addFoo<T>(fn: T): T & { foo: 'bar' }

const result = addFoo(scenario3(identity))
//    ^? - const result: (<T>(x: T) => {
//        call: () => T;
//    }) & {
//        foo: "bar";
//    }
sudden mason
worthy void
#

i was more curious about why/how scenario3 works at all. figured there must be some special case for it in the checker

sudden mason
#

false alarm... got confused with all the variables lol 😐

#

this (equivalent of scenario #3, without key) also doesn't work - it seems that both functions must use TParams and TReturn , and it breaks if the outer one uses TFn. I was hoping having that separation could help solve it but nope

declare function alter<TParams extends any[], TReturn>(
  fn: (...args: TParams) => TReturn
): (...args: TParams) => {call: () => TReturn}

const attach = <TFn extends (...args: any[]) => any>(
  fn: TFn
) => {
  const altered = alter(fn)
  return altered
}

const result = attach(identity)(123).call()
//    ^? any
#

the actual implementation is basically this... but yeah I don't think it's of any use here

export const makeSelect = <TSelector extends (...args: any[]) => any>(
  selector: TSelector,
) => {
  const target = Object.assign(
    // NOTE: types here are irrelevant because we do a type assertion below anyway
    (...args: any[]) => {
      return {
        get: (...pass: Parameters<ReturnType<typeof makeGet>>) =>
          makeGet(target, args, selector)(...pass),

        use: (...pass: Parameters<ReturnType<typeof makeUse>>) =>
          makeUse(target, args)(...pass),

        sub: (...pass: Parameters<ReturnType<typeof makeSub>>) =>
          makeSub(target, args, selector)(...pass),
      }
    },
    {
      _meta: {
        type: 'select',
        /* more stuff */
      },
    },
  )

  return target as TheTypeAssertionHere...
}
#

this is when I noticed that it doesn't work with generic functions at all (so it doesn't use TParams, TReturn, etc.), and is what prompted this post

worthy void
#

i might get more than i really want here, but what do makeGet/makeUse/makeSub do?

sudden mason
#

makeGet is the simplest:

const makeGet =
  (target: any, args: any[], selector: (...args: any[]) => any) =>
  (customSelector?: (...args: any[]) => any) => {
    const { value: selectedValue, depsList } = callAndTrackDeps(
      { trackDeps: !target._meta.depsList },
      () => selector(...args),
    )
    target._meta.depsList ??= depsList

    const value = (customSelector ?? identity)(selectedValue)

    return value
  }

basically, here's what the user would do with the library/what the api looks like:

const myStore = store(() => {
  const counter = atom(1)
  const multiply = select((factor: number) => counter.get() * factor)

  return {counter, double}
})

myStore.multiply(2).get()
myStore.multiply(2).use()

then store reads the _meta field from counter and double, and does conditional logic based on that (and also the field is used in type scenarios where I want to restrict what object/function can be returned, which is why I can't just omit it from the type)
and you can imagine that the function passed to select could be generic (the above works as long as it's not generic)

worthy void
#

this is mostly just my own curiosity, but i'm also wondering what exactly a "selector" is in this context. from the name i'd expect a type like (query: Query) => Set<Result> or whatever, not an arbitrary function

#

(thanks for sharing details, btw)

sudden mason
#

in this case it can actually take anything and return anything. it's basically just a function, but I do some processing, namely track dependencies (atoms) that this function calls

#

when select.get() is first called, it will track those dependencies and set them on _meta.depsList. this is then used in select.use() to subscribe to changes in the store

worthy void
#

i see. is this for something akin to a vue-like reactivity system? e.g. you need to recompute stuff when dependencies change?

sudden mason
#

it doesn't seem to me like a different pattern could be used since a tthe end of the day I need a way to attach metadata to the function

sudden mason
worthy void
#

you could for example not try to reflect _meta in the type, and do runtime checks for it when needed. or don't attach it to the function at all, instead have some global Map<Func, Metadata> registry

#

(both of those are kinda lame though)

sudden mason
# sudden mason yes basically

if you're familiar with react and zustand, it's like zustand but the selectors are only recalculated when the atoms they depend on change, for optimization reasons

worthy void
#

i'm not much of a frontend guy, but have worked a bit with Vue and i think that's how its system works also

sudden mason
#

hmm, a global Map/WeakMap is interesting, though not sure if it's possible fully in my case...

sudden mason
worthy void
#

another idea is to return something like { function, metadata } (an object containing a function rather than the function value itself), though i'm not sure if that helps

sudden mason
#

hmm it wouldn't help because then it changes the api for the user

worthy void
#

ah, is the API set in stone already?

rare cradle
#

Yeah I was about to say, the API could take some inspiration from various reactivity systems, like Vue's or S.js.

sudden mason
#

yeah I basically have a huge chunk of it ready and only now noticed it breaks with generics. and a nice api is a part of the reason i'm writing it. it's mostly for internal use at our company

#

it's hard to articulate without more examples, but you can have nested selects, atoms, etc. and it kinda only works nicely with this api

#

circling back, omiting the meta field on the function as a whole and using a global Map or something actually wouldn't be great, because I want to be able to limit where each type of function can be used (e.g. use _meta as a type restriction)

worthy void
#

it might be possible to support a few specific generic patterns via a giant conditional type with infers, but i'm not sure if you'll find that acceptable

sudden mason
#

hmm that irks me the wrong way a little 😸

worthy void
#

yep, it's not great

sudden mason
#

it might be the last solution i reach for if everything else fails. but really want it to work in the normal way

#

also not sure if it would work everywhere

worthy void
#

out of get/use/sub, is one the "main" function? assuming it's get and that that has the same signature as the original selector, another idea is to not try to alter the call signature and just attach use and sub as properties (basically trying to steer towards your original scenario #2)

#

(though that falls over too if use and sub also need to preserve the genericness)

rare cradle
#

it kinda only works nicely with this api
How so? In Vue for example, you would just write a composable that takes in a ref (the equivalent of your atom), and returns another ref:

function useMultiplied(source: Ref<number>, factor: number) {
    return computed(() => source.value * factor)
}

const doubledCounter = useMultiplied(myStore.counter, 2)
console.log(doubledCounter.value) // will be in sync with counter
sudden mason
# worthy void (though that falls over too if `use` and `sub` also need to preserve the generic...

yeah all of them need to preserve it. and the signature needs to be basically (...args) => {get: (irrelevantStuff) => TReturn, use: (irrelevantStuff) => TReturn} etc.
this is because I have a more complex system of nesting select, deriving from them, etc. it's a little hard to articulate (cc @rare cradle), but the functions/fields that the user uses must be this way.
the problem is also that I can't just put the meta on the returned object {get: ..., use:..., _meta:...} because that would require calling the actual selector function (with some dummy args?), which just doesn't work...

#

hmm but here's an idea - put the meta on the function, but don't include it in the type, and then put in the type of the returned object

rare cradle
#

I'm just curious about the API design, but if it's already set in stone then well, not much can be done about that.

#

I'm mentioning because it looks very similar to Vue's reactivity system.

sudden mason
#

yeah I'll be happy to send it over to you once I have it all documented and published to github, but yeah I don't think the userfacing api can change because of some constraints

sudden mason
#

testing that now..

#

seems to work fine!

declare function select<TParams extends any[], TReturn>(
  fn: (...args: TParams) => TReturn
): (...args: TParams) => {get: () => TReturn, _meta: {type: 'select'}}
const test = select(identity) // correct type

console.log(test._meta) // {type: 'select'}

declare function onlySelectAllowed(fn: (...args: any[]) => {_meta: {type: 'select'}}): void

onlySelectAllowed(test) // ok!
onlySelectAllowed(() => {}) // err!

I don't see any issues with this so far...

#

it's actually an obvious solution I guess. I got too stuck on the idea that typescript and javascript had to match up

#

does this look right to yall?
and thanks so much @worthy void and @rare cradle !

rare cradle
#

I would personally get rid of the anys, but otherwise if it works then sure.

#

I got too stuck on the idea that typescript and javascript had to match up
Does this mean _meta doesn't actually exist at runtime?

worthy void
#

i think it does, they'd just do a type assertion or whatnot when they look it up too

#

i'd suggest not even using the name _meta if you're intentionally mis-aligning things. that's just going to lead to confusion later. you're essentially using it as a brand

#

so instead consider a private unique symbol for the key (then users can't accidentally access it), or at least something more explanatory like _phantomDataToIndicateSelectishness: …

sudden mason
sudden mason
rare cradle
#

Well it doesn't exist at runtime then, in which case I was going to suggest branded type to attach metadata purely on the TS level with no impact at runtime.

sudden mason
#

yeah that makes sense. well, I guess the user could access it for some advanced use cases, plus it makes it easier when everything is typed correctly ootb for things like tests, etc.
but that can all be solved with a type assertion, or even a type that uncovers the field, like ShowMeta<T>

#

thanks again yall!