convex-service

[mellow mist]

Lets quickly define some services with this new system

we do this with zod, I think ill add the default convex validator in eventually but ill need to do some thorough refactoring

We will define the schema, triggers, rls, for a basic user/identity system (with some other things thrown in to show off the api)

//schema.ts
export const IdentityTypes = z.enum([
  'password',
  'oauth',
  'magic_link',
  'otp',
  'saml',
])

const IdentitySchema = z.object({
  type: IdentityTypes,
  isActive: z.boolean(),
  providerId: z.string().uuid(),
  userId: zid('users'),

  // Refine Test
  password: z.string(),
  confirmPassword: z.string(),

  email: z.string().email(),
  passwordHash: z.string().optional(),
  tokenHash: z.string().optional(),

  metadata: z.record(z.any()).optional(),
  lastUsedAt: z.number().optional(),
})

export const IdentitiesService = defineService({
  name: 'identities',
  schema: IdentitySchema,
})
  .default('isActive', true)
  .default('providerId', () => crypto.randomUUID())
  .default('type', 'password')
  .unique(['providerId', 'email'])
  .rls({
    read: async (ctx, doc) => {
      const identity = await ctx.auth.getUserIdentity()
      if (!identity) return false
      return identity.subject === doc.userId
    },
    insert: async (ctx, doc) => {
      const identity = await ctx.auth.getUserIdentity()
      if (!identity) return false
      return identity.subject === doc.userId
    },
    modify: async (ctx, doc) => {
      const identity = await ctx.auth.getUserIdentity()
      if (!identity) return false
      return identity.subject === doc.userId
    },
  })
  .index('by_user', ['userId'])
  .searchIndex('email', {
    searchField: 'email',
    filterFields: ['type'],
  })
  .validate(
    IdentitySchema.refine((data) => data.password === data.confirmPassword, {
      message: 'Passwords do not match',
      path: ['confirmPassword'],
    })
  )

Lets piece this together real quick. First off, we have some extra builders, more than the index, searchIndex, and vectorIndexes builtin to convex

There is
default
unique
rls
validate

Not shown
trigger

Already have added and am working on
relation

lets break each of these builders down

.default will allow you to pass in a typesafe field path and also a value or function to run to create a value

.unique lets you pass a list of field paths to validate uniquness on the table

.rls lets you configure the rls for that table, you pass an object of functions each containing ctx (queryCtx) and doc (the document)

.validate This function will validate each of the fields based on their zod schema on every insert/update, you can leave it empty or you can pass a schema to it. When defining your schema you have to define a ZodSchema and just a ZodSchema, if you want to add any refinements or super refines you have to do it in the way i showed above. But in most cases you can just create your schema in the schema field and then add the .validate() builder to it and it will validate

.trigger Lets you add a convex-helpers trigger to the table.

.relation is in the works and it will act as a helper function to make your db relational through automatically adding triggers to your db

lets now define our users service this service is what our identities will relate to

but we can see a trigger is added to this service that on every update we will sync the updatedAt time

//schema.ts
export const UsersService = defineService({
  name: 'users',
  schema: z.object({
    name: z.string().min(3),
    email: z.string().email(),
    avatarUrl: z.string().url().optional(),

    // Defaults
    onboarded: z.boolean(),
    isAdmin: z.boolean(),
    updatedAt: z.number(),
  }),
})
  .default('onboarded', false)
  .default('isAdmin', false)
  .default('updatedAt', () => Date.now())
  .unique(['email']) // if email is not always set, make sure it's sparse or conditionally enforced

  .trigger(async (ctx, { id, operation, newDoc, oldDoc }) => {
    if (operation === 'insert' || (operation === 'update' && newDoc)) {
      const oldUpdatedAt = newDoc.updatedAt
      const newUpdatedAt = Date.now()

      if (oldUpdatedAt && newUpdatedAt - 100 < oldUpdatedAt) {
        return
      }

      const updatedDoc = {
        ...newDoc,
        updatedAt: newUpdatedAt,
      }
      await ctx.db.patch(id, updatedDoc)
    }
  })
  .index('by_name', ['name'])
  .searchIndex('name', {
    searchField: 'name',
    filterFields: ['email'],
  })
  .validate()

You can also see in our UsersService we are validating the table without passing a schema to the builder. This way we use the schema that was passed in to the defineService function.

Lets add it to our convex schema:

//schema.ts
export default defineSchema({
  users: UsersService.getConvexTable(),
  identities: IdentitiesService.getConvexTable(),
})

Now we can use our services

Lets create some functions for our Users

the ConvexService class builds off of a lot of pieces from convex and convex-helpers, as such we can use Service.crud and pass our defined schema in to create basic crud functions, you cannot pass your own builders in, but you can pass in visiblity, incase you want all the mutation functions of the crud to be internal you can do that or vise versa with the query functions

//users.ts
import schema, { UsersService } from '@/convex/schema'

export const { create, read, paginate, update, destroy } =
  UsersService.crud(schema)

We can do the same to create basic crud for our identities

//identities.ts
import schema, { IdentitiesService } from '@/convex/schema'
import { CreateRLSWrapper } from 'convex-service-package-not-made-yet' // this is also where the defineService comes from but i havent made the package yet :P

export const { create, read, paginate, update, destroy } =
  IdentitiesService.crud(schema)

const { queryWithRLS } = CreateRLSWrapper(schema)

export const GetEmailsFromIdentites = queryWithRLS({
  handler: async (ctx) => {
    const identities= await ctx.db.query('identities').collect()
    return identities.map((identity) => identity.email)
  },
})

Lets create a user

Notice how in our create functions validator we dont need to pass the
required fields for isAdmin, onboarded, or updatedAt, the crud functions validators removes the need for that by creating them on default.

The .default builder will only work when using a function from the ConvexSerivce.crud api though, so be aware of that. I am looking into solutions for that but it might be a while

Lets create another user, but lets leave everything blank

we get this clean looking error.

Say we are creating a user on a form with this, we can actually catch the catch the error and recieve the zod error itself.

import { ConvexServiceValidationError } from "package"

const { mutateAsync: createUser} = useMutation(api.users.create); // this might be the correct syntax for the convex react hooks idk :shrug:

const onSubmit = (formData) => {
  try {
    const user = await createUser(formData);
  } catch (err) {
    if (err instanceof ConvexSerivceValidationError) {
      const zodParseError = err.zodError;
      // do something with the zod error
    }
  }
}

Ill probably look into making a function/system that will automattical give you back either the error or the data when you run it so you dont need to do so much wrapping

something like

const { error, user } = await  TryCatchServiceError(createUser(formData)); // idk this isnt that clean but wtv```

If we were to update that user, we would see it also validates the update field.
If the validation passes then we run our triggers, and our UsersService has a trigger to update the updatedAt field everytime it updates so thats what would happen.

lets now create a UserIdentity which also has
password and confirm password on it to show off the func of the validation normally youd only want to store like the hashed password or some

We tried to create an identity with not matching passwords the the validators refine worked and returned an error!

Okay, i fixed the passwords not matching, and now i want to make another identity:

So this error isnt a great example of how i designed it, but this shows that uniqueness works. But in actuality youd want to make the email unique on the type of identity (like saml, magic link, etc) since you could have multiple idenities for that one email if the identitiy types are different.

I plan to add a way to make a field unique on another field like
email uniqueness based on the type
but thats a future release sort of thing, i need to first actually finish what ive made so far lol

Okay, i successfully made another identity, lets now query all the emails in the identity table with our
GetEmailsFromIdentities function that we made, this function will query with our RLS rules that we defined in the service def since we are using the CreateRLSWrapper function

so we can see if we act as no user then we see nothing (not logged in) but if we are logged in we can see all the documents. You can configure your RLS rules to be SUPER powerful so make sure to make use of them well

Thats mostly what ive made so far. The services are very powerful and should be used in your function defs and also in your frontends since the services define validators and types that are useful for both

for example
we could create this function

we can also use the omit function from the convex-helpers/validators
to remove the updatedAt field and set have that get set in the function def

const { internalMutationWithRLS } = CreateRLSWrapper(schema)

export const CreateUserWithoutDefaultsAndTriggers = internalMutationWithRLS({
  args: UsersService.withoutSystemFields,
  handler: async (ctx, args) => {
    return await ctx.db.insert('users', args)
  },
})
``` and we would get this on our convex dashboard

something like this would work better

export const CreateUserWithoutDefaultsAndTriggers = internalMutationWithRLS({
  args: omit(UsersService.withoutSystemFields, ['updatedAt']),
  handler: async (ctx, args) => {
    const user = {
      ...args,
      updatedAt: Date.now(),
    }
    return await ctx.db.insert('users', user)
  },
})

So we can use the ConvexService class to create functions with validators easier, but how can it help on the frontend.

well since we have the zod schema stored in the service you can use that to create inferred types.

something like

type User = z.infer<UserService.schema> // and this will include system fields aswell 
{
_id: Id<"users";
_creationTime: number;
//... rest of the def
}```

you can also use it for form validation too!

const { data: validated, error } = await 
UserService.schema.safeParse(data);```

sadly this will validate with the system fields in mind

ill be looking into a good way and good api to make a better validator to help you validate, something that would build off the .validate builder.

That is all i have for now. Ill be cleaning up the proto type and creating an actual package for it and then ill release it on github. If yall think its cool or have any ideas let me know in here or in DMs