Future-proof: `/api` resource routes vs API express server

[wet spoke]

Hi everyone,

We're in the process of restructuring our project: we have a mandate to do so in 1 month 🙈

The ultimate goal is to:
– simplify our product,
– correct the rookie mistakes we made (not using Remix as a BFF, reduce our dependencies footprint, etc.),
– prepare for the future (we would like to develop a native application in mid-term).

In practice, we are going to have an intermediary API layer:
– authenticated cookie would be reduced to a sessionId/token (kind of the same, right?),
– this sessionId will be used in every API call to identify the user, assess if he has the correct role, and return data accordingly.
– loaders/actions in Remix will not query the DB directly, but rather fetch the API endpoints with the sessionId (BFF !!)
– this way, the native application will easily plug to the same API.

Now the question is: how will the overall architecture looks like?

I'm thinking about two options (see the nice schema attached 😅 )

1. We use a separate Node.js Express server,
2. We use a /api resource routes directly in Remix.

The pros of 2 is the simplicity of the rework: everything stay in Remix, we should be able to work fast.
The pros of 1 is the state-of-the-art solution: separate the API from the frontend.

Option 2 is a quick win from my point of view.

But I'm wondering:
a. Is option 2 scaleable ? Anybody with some experience with decent traffic. We have <100 users but we hope to onboard billions of users when we promote our product, obviously 🤞
b. If we would need to transit to option 1 later, how easy will it be? Copy/paste easy? Fairly easy? Not so straightforward easy?
c. Is there a way to use express middlewares in /api? (I believe the answer is no for the moment: https://github.com/remix-run/remix/discussions/1432)

Thank you for your input 🙏

[ocean steeple]

If you're reworking things, I'd try my best to consolidate everything to a single service and remove the "API" layer all together for now.

I'd personally do something like this:

• Keep existing API server around to make migration easy
• Move code from API server into Remix project and remove calls to the API server 1 by 1.
• As you start working on the mobile app, you'll most likely find that your remix routes already contain all the logic / info you need to expose for the mobile version

At this point you have two paths you could take:

• 1. start creating remix "/api/..." routes that just call the existing functions defined for the remix web app routes, or even simpler, re-export loaders from the existing routes
• 2. expose express routes on top of your remix app server (i.e, serve the mobile API and remix app from the same express server)

1. allows you to just work within the Remix WebFetch API conventions
2. allows you to copy and paste existing express API handlers over and remove a separate service you have to manage.

Either way, if it was me I would use the oportunity to reduce operating costs both from a hardware / resource perspective, as well as an engineering maitenence perspective.

I think my main point here is this though: You don't need two separate services even if you decide to write the API layer in express. Your remix app and API layer can be served from a single Node.js process as remix is just an express middleware.

[outer marlin]

Interesting i also need to finalize api

I@build foundation based on loopback 4 bare bones its just some resource routes with loopback 4 inspired features fully remix. Not behind desk can share some examples tomorrow

[wet spoke]

Thank you for your feedback 🙏

We have currently everything in Remix. The reason why we were thinking about adding an API layer is to prepare for the future (that is, a native application).

I used React Native in the past, and I remembered that cookies were a nightmare to use. I also recall a discussion with @glossy lantern (sorry for the ping but your opinion matters to me; I can't find back the Discord discussion but it was triggered by this talk https://youtu.be/eh6FfHFpYMM?t=575) where he explained how he's doing things @Daffy and how they use Remix as a BFF.

I really like the BFF paradigm: Remix loaders/actions act between the user and the backend, aggregating from multiple sources if necessary, and prepare the data in a way that it can be used as-it-comes by the route component.

To give you an example of what I'm currently scaffolding:

// /routes/api+/users+/index.ts

import { json, type DataFunctionArgs } from "@remix-run/node";
import { type User } from "~/db/schema";
import { db } from "~/drizzle.server";

export async function loader({}: DataFunctionArgs) {
  const result = await db.query.users.findMany();

  return json(result);
}

export async function getUsers(): Promise<User[]> {
  return fetch(process.env.API_URL + "/users").then((res) => res.json());
}

// /routes/users

import { json } from "@remix-run/node";
import { useLoaderData } from "@remix-run/react";
import { getUsers } from "./api+/users+";

export async function loader() {
  const users = await getUsers();

  return json({ users });
}

export default function UsersList() {
  const { users } = useLoaderData<typeof loader>();

  return (
    <div>
      <pre className="font-mono">{JSON.stringify(users, null, 2)}</pre>
    </div>
  );
}

I'm fully aware that it is fare from ideal: the loader could directly hit the database, instead, it makes a fetch on /api/.... Some sort of double-call one could say.

But it allows us to build some sort of REST API (95% REST principles).

[glossy lantern]

if you're using Express I would go with Remix as API for your mobile app

if you're using some other backend (like I do with Rails as my API) then it makes sense to fetch the API from Remix

between Express and Remix there's really not a huge difference since Express by itself doesn't do much

so moving the API to Express doesn't give you a lot of benefits, and it can be even more work

[wet spoke]

What is the UI needs are different between the Web and the native app?

[glossy lantern]

in that case, creating a folder for your data-layer that you can call from the web app routes (the ones exporting components) and api routes makes more sense

[wet spoke]

It already feels like additional work, but perhaps for the best (<-- that's what I would like to assess before doing the wrong choice ^^)

[glossy lantern]

export an API, but do it from Remix

unless you plan to eventually move the API to a separate deployment it's simpler to use Remix for the API and web app

that is, if you use Express

if you where to use a backend framework in another language (like Rails, Django, Laravel, etc.) it makes sense to fetch the API from Remix and your mobile app, and use Remix loaders to aggregate data from multiple endpoints

or if you plan to use a more complete backend framework in Node like Nest.js, then again use that for your API and fetch it from Remix

but that is because those frameworks gives you way more things than Express does

Express is just a router, and Remix already has one

[wet spoke]

It would mean to use routes such as http://localhost:3000/en/auth/sign-in?_data=routes%2F%24lang%2B%2Fauth%2B%2Fsign-in instead of http://localhost:3000/api/auth/sign-in for the native app ?

[glossy lantern]

no no

use a resource route in Remix to expose an API

so you have route files like app/routes/api.users.ts which has a loader and an action

then you fetch it on https://company.tld/api/users

[wet spoke]

Oh I see, so keep them separated like in my snippets above?

But extracting in a separation function (the data-layer you mentioned?) so I can reuse the same logic in the Remix page route?

[glossy lantern]

kinda, instead of doing this in the api route

import { json, type DataFunctionArgs } from "@remix-run/node";
import { type User } from "~/db/schema";
import { db } from "~/drizzle.server";

export async function loader({}: DataFunctionArgs) {
  const result = await db.query.users.findMany();

  return json(result);
}

export async function getUsers(): Promise<User[]> {
  return fetch(process.env.API_URL + "/users").then((res) => res.json());
}

I would create a file like app/models/users.ts and export getUsers which does the db query, then call getUsers on the api route and UI route

[wet spoke]

Ok

Yeah I see

[glossy lantern]

so don't fetch yourself from ui routes, just call getUsers which does the DB

[wet spoke]

It sounds better

[glossy lantern]

eventually if you move that to a separate app for your API you can keep the getUsers call in your UI routes but replace the DB query with a fetch if you need

[wet spoke]

I like that

Thank you for taking the time to brainstorm on that 😉

@glossy lantern going further in my reflexion, it would mean the authention (is the user authenticated, does he have the correct role, etc.) will be replicated on both places: ui routes & api routes.

Long time ago I used React Native and the cookie handling was a nightware. I even believe someone invited me to use an authorization token instead.

This would mean in the API route, I would have to check the Authorization token from the header, match the token to an existing user in the sessions sql table, perform my checks, and act accordingly.

Whereas in the UI route, I would have to retrieve the session token from the cookie, match the token to an existing user in the same sessions sql table, etc.

Does that sound right, or do you think of a different approach?

[glossy lantern]

you should use Remix sessions for your UI routes (your web app) and Authorization header for your API route for the mobile app

you don't need to check if the token from the mobile app is on the sessions table, if you use an JWT access token everything can be already on the token

but at the end you will need to have two auth checks, one cookie based for the web app and one headers based for the api

I would just wrap everything in a function authenticate that receives the request and check if there's an Authorization or Cookie header, and does each validation and returns the user

then the role validation is the same in both cases

[wet spoke]

Unless you want to be able to invalidate an access, correct? Like you can deactivate an API key in some external services.

[glossy lantern]

yes, but then you should keep the access token in the DB, not the session

[wet spoke]

What I mean is that for the web app, I currently store a sessionId returned on a successful login in a cookie using createCookieSessionStorage. In a native app, I would store the same in a local storage.

The UI route will check the cookie, the API route will check the Authorization header.

Either case, the sessionId would be challenged against the existence in the sessions table.

[glossy lantern]

yep, that should work

[wet spoke]

You sound like you're not convinced it is the right way 🙈

[glossy lantern]

I'm mostly used to use services like Auth0 where you get an access token and they handle the rest, so you just integrate with them and either store it on the mobile app or in a session and then use that

[wet spoke]

Ok 😉

[outer marlin]

just to chime in perhaps this is of use https://gist.github.com/Jarrodsz/2d434df323b972d7e17a0cd2f886a1a6

what do you think of that Sergio? im still need to implement this havent got to it, but i plan to use this mechanism to have some resourceless routes in remix routes folder to faciilitate the api feature

[glossy lantern]

It looks like too much work, if it was automatic that the controller worked as the route loader and actions it would be similar to Rails which I do like

[outer marlin]

Forgot to poet back i ended up creating resourceless routes