Circular dependencies in validators

[high kelp]

I'm defining my tables using Table from convex helpers, and I want to define some enriched versions of these documents using a Convex validator in the same schema.ts file such that I can pass it around to other validators and Convex functions. I'm running into circular dependency issues, however.

See the attached screenshot for the specific example. I can seemingly fix this by putting enrichment validators into a enrich.ts file in each table folder, but is this really the only way? I would love to have both table validator and enriched validators and types in the same schema.ts file.

[upper sky]

There's a couple ways off the top of my head I can think of:

1. have a file that both the schema.ts and individual table files imports from (I think this is your enrich.ts example)
2. Instead of doing validators that are composed into tables, extract validators from the table schema

In both instances, I think you want the dependency chain to be one of these:
A:

1. schema depends on individual schema files
2. individual validators depend on schema / individual schema files, but aren't in the individual schema files

B:

1. individual schema files depend on smaller validators (in a different file)
2. schema / individual schema files import those files

C:

1. all tables are defined in schema.ts (not using Table)
2. all individual table schema files extract the validators from the schema

For (C) you can access table validators like schema.tables.channel.validator.fields.videoCollection - you'll get type-safe validators out.
To get other variants, you could use withSystemFields("channel", schema.tables.channel.validator.fields) or for the doc, just schema.tables.channel.validator.

Sorry I haven't updated that stack post with this info (yet). Table existed before you could introspect validators

[high kelp]

Thanks @upper sky this is great! Can't believe I didn't stumble upon the withSystemFields helper, that looks like it will solve my issue! I've been very happy with the Table helper due to the very ergonomic .doc, but in my case it's been a problem. Using withSystemFields I can ensure that all my validators in all my lower-level schema.ts files will only ever import from the root schema.ts and not from each other. I could probably even set up a linting rule for this, and safely still use Table.doc and all the other ergonomic helpers for functions. Cool! Thanks man

[upper sky]

btw ChannelTable.doc should be equivalent to schema.tables.channel.validator

[high kelp]

schema.tables.channel.validator doesn't have system fields while Table.doc does - or am I trippin?

[upper sky]

oh shoot, sorry yeah that's right

[upper sky]

but I did make a doc validator helper that I never merged in the doc-validator branch:
https://github.com/get-convex/convex-helpers/blob/doc-validator/packages/convex-helpers/validators.ts#L161-L190

The trickiest thing is if you have a table that is a v.union at the top level (instead of an object/ Record<string, Validator>), since then you need to add system fields to all of its children

and a typedV (working title) helper. I just published it to convex-helpers@0.1.68-alpha.1 - if it works well for you I can merge https://github.com/get-convex/convex-helpers/pull/427 and publish it

[high kelp]

dude.. you're in my mind, I was literally trying to write a light helper/wrapper function that would allow me to input a tableName and return withSystemFields 😄

AND I was then going to ask you for advice regarding the problems we have had with discriminated v.union schemas...!!

I'm trying it out now

[upper sky]

the typedV is slick too. gives you v.doc("channels") and v.id("channels") that is typesafe against your schema, but idk what to call it. I think I suggest export const vv = typedV(..

[high kelp]

Hmm this is strange... Now my dependency "graph" just looks like this, but the problem is back (I combined everything into the nested schema.ts files again). Doesnt seem like there should be any circular references left. I'll go look in other schema.ts files, but the problem seemed located to these

Is there now some circular reference to the root schema.ts? both of those three nested schema.ts files now import root schema.ts, and root schema.ts imports all nested schema.ts files..

[upper sky]

yeah the validator helpers depend on schema.ts, so that can't depend on any files that use the validator helpers.. I'm guessing you really don't want to add all your table definitions to schema.ts / have dedicated "table schema only" files that don't also have the validator utilities?

[high kelp]

yeah makes good sense.. I'm seeing that I'll have to concede and go for probably the second option! It's so nice to be able to ctrl + p and type "video schema" and go directly to the table definition for video. We have >40 tables already and more coming, so I think everything in the root schema.ts file will make it cumbersome to make changes

Thanks so much for your help Ian! I understand a lot more now. I'm running into some specific pain points with v.union schema definitions that you also alluded to that I think you'd enjoy thinking about as well, but I'll leave that for another day, it's past midnight here already!!

[upper sky]

yeah sometimes I think it's easier to work with a schema like defineTable({ data: v.union(... than defineTable(v.union(.. directly. certainly a lot of the helpers struggle with top-level unions. but yeah let's take that offline

lmk if you think the validators are good to go / review the code in the PR at your leisure

[high kelp]

oh and your doc helper worked super-well btw, so go ahead and merge that as far as I'm concerned.

Your typedV: do you mean a version of v.id("...") that, in e.g. args: to a mutation or query will have type safety/intellisense for table names? that would be amazing. I've often wondered why that wasn't already the behavior??
I guess our whole conversation here is kind of the answer to that question though: it's the same v that's used in schema definition

[upper sky]

Yeah, the typedV can't be used in schema.ts b/c it takes schema as a parameter. like export const vv = typedV(schema); then use vv.id(...) (or call it v and remember to import that v instead)