What's the best way to secure API endpoints and allow it only from the front-end?

[true jolt]

I have a website (next.js) and its getting the content from payload api. Currently, I have access control made so that anyone can read it, meaning anyone with the link to the api can see the json output.

The problem is that some of the content should be secret (for example a document that's available only after the user fills out a form (email address)). But with current access controls anyone that has the link can just go and see the json output straight from the API.

What's the best way of handling this in payload (+ nextjs)? Do I create a "api token" global and make it public, and then use it in all my fetch requests within the server components of my front end? Or is there a better way?

[scenic ocean]

When you create a collection, you'll want to explicitly list permissions

For example, if you have a form submission collection, you'll want to the user to be able to "create" but maybe not read/delete/update

Or maybe you create a field on your Users collection that defines some roles, you could also differentiate by role

Documentation on Access Control: https://payloadcms.com/docs/access-control/collections

Regarding authentication, I would recommend reading through https://payloadcms.com/docs/authentication/overview to understand the available methods

I personally use the cookie sessions, but it all depends on your setup

[true jolt]

So even if I don't have users in my front end (it's a simple marketing site), I would still need to create a user for this purpose and then in my server-side components log in with a user specifically created for the front-end to get the functionality I want, right?

[scenic ocean]

Hmm

I think I'm misunderstanding @true jolt

Do you mind explaining again, I'm sorry

[true jolt]

No worries, thanks for trying to help me.

Let me explain it with a different example.

[scenic ocean]

Sure thing

[true jolt]

I have a backend, that has an API (payload). And I have a website, a simple next.js front-end, which gets the content through the backend (payload) api. What I want to do is make the api return the content only if I fetch the content with some kind of an API key. If I don't have an API key (either through headers, or the request body), I should not be able to fetch the data.

[scenic ocean]

Ahhh okay I see

[true jolt]

Now I'm used to working with different api's and they usually require to use an API_KEY within the request headers for it to actually return the data.

So without the front-end having to actually "log in".

[scenic ocean]

Right, in this case you have a few options

Is the content created by users?

Are they the owners of the documents

Generally a user would make a fetch request with their credentials, which would then provide a token or set a cookie that authorizes them

In your scenario

You want a global api key?

[true jolt]

The content is created by the payload admin

[scenic ocean]

Ok so not by users

[true jolt]

yup

[scenic ocean]

How do you want users to see the content

You dont want it to be public

So how will the users know the api key

Will you give it to them manually?

[true jolt]

I don't want anyone having the api endpoint url being able to see the data. I want only my front-end site (using server-components) be able to fetch the data.

[scenic ocean]

Ah

[true jolt]

Ideally, I would create an API key by hand in payload, and then use that API key within my fetch requests of the front-end site.

[scenic ocean]

Well

If you use the api key in fetch requests

Then the key is not secured

You could just inspect the network requests

[true jolt]

hmm, true

oh, but wait, I'm using server components

so the requests would be server-side, the client wouldn't be able to inspect them

[scenic ocean]

Oh if that is the case, why not use the Local API

https://payloadcms.com/docs/local-api/overview

and manually check if a provided key matches the one on your server

[true jolt]

Prob is that back-end lives in another server/service 😦

Front-end is on vercel, back-end is on google cloud run.

[scenic ocean]

I see, we have a similar setup at work

We have a frontend (we do manage login/reg)

an API that talks to our db and payload

and payload

Our process is

The user logs in on the frontend, payload sends an http-only cookie to the user

The user then makes a request to the API, with credentials included (the cookie storing a jwt token)

I then decrypt the JWT on our API, because the API and Payload both have the same secret key

If I can decypt the JWT I know the request was made from the site

Because they cant be tampered with

[true jolt]

RIght, but this requires user login, which my front-end doesn't have. The content is fetched (accessed) only by the front-end code through server-side components.

[scenic ocean]

I think you can also lock down requests with CSFR config

@dusty peak Any idea on how they would do this?

I'm guessing its going to be strict CSFR or something so only requests from a specific IP are valid

https://payloadcms.com/docs/authentication/overview#csrf-protection

[true jolt]

Oh man, last year we worked with BMW and we had to take care of a lot of security stuff, even though the stuff (CSFR stuff mostly) we were developing for them was a marketing website without user registrations or user data or anything and it still gives me nightmares 😄

[scenic ocean]

Ah well payload makes it easy

import { buildConfig } from 'payload/config';

const config = buildConfig({
  collections: [
    // collections here
  ],
  csrf: [ // whitelist of domains to allow cookie auth from
    'https://your-frontend-app.com',
    'https://your-other-frontend-app.com',
  ],
});

export default config;

So if a request doesn't come from the frontend IP

It will be blocked

That's for "cookie auth" though as it states

So

You may need to go into server.ts

and look at where the express logic is

and add cors / csrf to the express handler

[true jolt]

Yeah, really hoping I can avoid cookie auth for this simple sitch

[scenic ocean]

which is a similar process

[true jolt]

Unless I absolutely can't

[scenic ocean]

Well can you try something now

I would say, add a cors config

for starter

allowing only your frontend IP

then try to request data locally

[true jolt]

tried it, still lets me see the output json if I'm fetching the api

[scenic ocean]

can i see your payload config?

In addition, in server.ts

import cors

import cors from "cors";

then before payload init, try

app.use(
  cors({
    origin: [
      "http://localhost:4200",
    ],
    credentials: true,
  })
);

maybe without the credentials part

that way you know cors is set on both payloads config and express

(not sure if payloads auto sets cors on express)

[true jolt]

import cors from "cors";
import { buildConfig } from "payload/config";
import path from "path";
import { cloudStorage } from "@payloadcms/plugin-cloud-storage";
import { gcsAdapter } from "@payloadcms/plugin-cloud-storage/gcs";
// import Examples from './collections/Examples';
import Users from "./collections/Users";

// Globals

import Homepage from "./globals/Homepage";
import About from "./globals/About";

// Assets Collections

import Logos from "./collections/Logos";
import Icons from "./collections/Icons";

const gcAdapter = gcsAdapter({
  bucket: process.env.GCS_BUCKET,
  options: {
    credentials: JSON.parse(process.env.GCS_CREDENTIALS || "{}"),
  },
});

export default buildConfig({
  serverURL: process.env.PAYLOAD_URL,
  // cors: [process.env.FRONTEND_URL],
  cors: ["lol"],
  admin: {
    user: Users.slug,
  },
  globals: [
    Homepage,
    About,
    // Add Globals here
  ],
  collections: [
    Logos,
    Icons,
    Users,
    // Add Collections here
    // Examples,
  ],
  plugins: [
    cloudStorage({
      collections: {
        logos: {
          adapter: gcAdapter,
          disableLocalStorage: true,
          disablePayloadAccessControl: true,
          prefix: "logos",
        },
        icons: {
          adapter: gcAdapter,
          disableLocalStorage: true,
          disablePayloadAccessControl: true,
          prefix: "icons",
        },
      },
    }),
  ],
  localization: {
    locales: ["en", "lt", "de"],
    defaultLocale: "en",
    fallback: true,
  },
  typescript: {
    outputFile: path.resolve(__dirname, "payload-types.ts"),
  },
  graphQL: {
    schemaOutputFile: path.resolve(__dirname, "generated-schema.graphql"),
  },
});

[scenic ocean]

cors: "lol" ?

[true jolt]

lol

😄

[scenic ocean]

oh you removed it for example

ok

[true jolt]

yeah, tried having something random and see if I still see the output

[scenic ocean]

ok and your server.ts

[true jolt]

but shouldn't this setting work already if I set it within my payload config?

[scenic ocean]

its simple enough where it would be silly not to try

also add the csrf array with the same url as your frontend

export default buildConfig({
  serverURL:
    process.env.NODE_ENV === "production"
      ? process.env.PAYLOAD_PUBLIC_SERVER_PROD
      : process.env.PAYLOAD_PUBLIC_SERVER_DEV,
  admin: {
    user: Admins.slug,
  },
  cors: [
    "http://localhost:4200",
  ],
  csrf: [
    "http://localhost:4200",
  ],

[true jolt]

Ok, are we trying to see whether setting up cors and csrf will prevent anyone with the api url seeing the output json?

[scenic ocean]

yes

We want to set cors / csrf on BOTH payload

and the server.ts

to make double sure

[true jolt]

kk, one min

ok, so my config is:

cors: [process.env.FRONTEND_URL],

And my server.ts looks like:

import express from "express";
import payload from "payload";
import cors from "cors";

require("dotenv").config();
const app = express();

app.use(
  cors({
    origin: [process.env.FRONTEND_URL],
    credentials: true,
  })
);

// Redirect root to Admin panel
app.get("/", (_, res) => {
  res.redirect("/admin");
});

const start = async () => {
  // Initialize Payload
  await payload.init({
    secret: process.env.PAYLOAD_SECRET,
    mongoURL: process.env.MONGODB_URI,
    express: app,
    onInit: async () => {
      payload.logger.info(`Payload Admin URL: ${payload.getAdminURL()}`);
    },
  });

  // Add your own express routes here

  app.listen(process.env.PAYLOAD_PORT);
};

start();

Going to http://localhost:8000/api/globals/about still lets me see all the content

localhost:8000 being my payload url

[scenic ocean]

hmmmm

what about csrf on payload config?

did you add that too

cors: [
"http://localhost:4200",
],
csrf: [
"http://localhost:4200",
],

[true jolt]

oh sorry yeah

export default buildConfig({
  serverURL: process.env.PAYLOAD_URL,
  cors: [process.env.FRONTEND_URL],
  csrf: [process.env.FRONTEND_URL],
  admin: {
    user: Users.slug,
  },
...

[scenic ocean]

cool and server reloaded?

[true jolt]

yup

[scenic ocean]

weird

ok lets just try a basic middleware example

// Define a list of allowed IP addresses
const allowedIPs = ['127.0.0.1'];

function ipFilter(req, res, next) {
  const clientIP = req.ip;
  if (allowedIPs.includes(clientIP)) {
    next();
  } else {
    res.status(403).send('Access denied');
  }
}
app.use(ipFilter);

That would technically restrict it to only a certain IP

(express)

not to say someone couldnt try ip spoofing

but lets see if at least that works

[true jolt]

yes, getting access denied

[scenic ocean]

Niceee

😄

[true jolt]

when going straight to the api url

😄

[scenic ocean]

Niceeee

You can further harden things

and make spoofing harder to do, or prevent it

I'd read a bit about https://expressjs.com/en/guide/behind-proxies.html

But for a basic use case, that middleware does work

[true jolt]

But what does this mean for payload.config that had both csrf and cors setup to only allow the localhost and it didn't work.

[scenic ocean]

Well

It's not for every request IIRC

It's for requests with Auth enabled

Is what payload handles

[true jolt]

got it

[scenic ocean]

also https://www.npmjs.com/package/helmet

is good for security

maybe you can harden the ip approach

[true jolt]

well, another prob is that without setting up an additional service in google cloud and doing a bunch of configs I won't be able to have a static ip for the front-end, so I'll have to search for a better solution not using IP's 😄

[scenic ocean]

@trim lichen @quasi coral do you guys have any input on this scenario?

Hopefuly I'm not missing anything important that makes this kind of thing easier

[true jolt]

Or worst case resort to trying out the programmatic login route if I won't manage to find a solution for a simple API key that would be set both on payload instance env vars and in the front-end server-side code

[scenic ocean]

Yeah I guess I'm just not familiar enough with your setup. Generally, anything on the frontend is visisble to the public

[true jolt]

Yup, I just thought of this situation when testing payload and wondered if there's a graceful/easy solution for that

Ideally, this would suffice to get the content from the API

  const res = await fetch('https://backend.com/api/about', {
    headers: {
      'api-key': 'dawoiawiodoaiwjdo',
    },
  })

while still preventing anyone visiting https://backend.com/api/about in their browser and seeing all the content publicly

[scenic ocean]

oh

I guess you could make a new user

Get their JWT token

then https://payloadcms.com/docs/authentication/overview#identifying-users-via-the-authorization-header

make your website requests use the token

obvs though anyone could see the token on the frontend in the network tab

[true jolt]

not if you fetch on the server, not the client

[scenic ocean]

Then yes, that should work nciely

nicely*

[true jolt]

oh wait

https://payloadcms.com/docs/authentication/config#api-keys

[scenic ocean]

Right but wasn't sure if it fit your reqs

Create a user for the third-party app, and log in each time to receive a token before you attempt to access any protected actions
Enable API key support for the Collection, where you can generate a non-expiring API key per user in the collection```

per-user

But in this case, its one user

and only the server will see it

so that can work

[true jolt]

Oh, it's for collections only, no globals 😦

Oh man this would solve all my problems if it was available for globals too 😄

[scenic ocean]

hmm

its not for globals?

that seems odd

[true jolt]

yeah, it’s specifically stated that it’s for collections. Globals don’t have an ‘auth’ option

[scenic ocean]

I think auth for Globals is a sound feature req to make

#community-help-archive message

Looks like people have discussed / built workaroudns

[digital narwhal]

How about just limiting access by the API token using Payload's access control mechanism?
https://payloadcms.com/docs/access-control/overview

You could create a plugin that attaches that access control to all collections and globals.

The access control function could be just coded in a way that it checks the headers of the request if it matches one of the registered API clients

[true jolt]

that's exactly what I'm trying out right now

[digital narwhal]

Yeah, sorry if I it already came up somewhere in the conversation. I couldn't read it all 😅

[true jolt]

Yeah no, it just came up when @scenic ocean found a community help archive thread (thank you @scenic ocean ❤️) but thanks for your input too, @digital narwhal !

[digital narwhal]

Remember to mark this thread as answered

[true jolt]

Yep, once I'll have this working I will!

YES, it works perfectly!!

Thank you @scenic ocean & @digital narwhal !

For anyone finding this thread, here's my solution (pitch in anyone if you would do it differently):

I have an access control function isFrontEndOrAdmin which looks like this:

import { Access } from "payload/types";

export const isFrontEnd: Access = ({ req }) => {
  // Check if the request has an api-key header and compare it to the preset variable
  // OR if a user is logged in (for them to be able to see it in the admin panel
  if (req?.headers["api-key"] === process.env.PAYLOAD_API_KEY || req?.user) {
    return true;
  }
  // Reject everyone else
  return false;
};

And in my global I use the access option like this:

  access: {
    read: isFrontEnd,
  },

This will allow to fetch the data through the front-end using an api-key header, but will prevent anyone without the api key to see the data (including anyone going directly to the api endpoint url). For it to be safer, you should only fetch with this api key header from the server (server components), because fetching on the client will expose the api key to anyone able to use dev tools 😄

[scenic ocean]

Hey all back from lunch!

So glad you figured it out 😄

[trim lichen]

super late to the party

good answers, api key sounded like what you were needing

[scenic ocean]

@true jolt Just for good measure

Is your comparison logic safe enough?

Please someone else correct me but I can imagine this scenario (logs true)

  
  const req = {
      headers: {
        "api-key": false
    }
  }
  
  const process = {
      env: {
        PAYLOAD_API_KEY: false
    }
  }
  
   if (req?.headers["api-key"] === process.env.PAYLOAD_API_KEY) {
       console.log(true)
 } else {
console.log(false)
 }

your comparison logic is

 if (req?.headers["api-key"] === process.env.PAYLOAD_API_KEY || req?.user) {
    return true;
  }

if the api-key is false and env doesnt get picked up for some reason

Isn't it possible for it to return true?

Just a thought

[true jolt]

hmm, good catch, I’ll have to look into this when I’m on my mac tomorrow

thanks @scenic ocean

[violet crown]

Hi, sorry to bump up an old thread but since you've managed to make this work, can you kindly pass me the code please? I can't seem to find out how to enforce an api key globally and not per-collection

[true jolt]

Hey, you have to still use access control (read: isApieKeyAuthorized or something similar) for each collection that you want secured with the API key. Just like my example above, where I name the function to check the key isFrontEnd. I don't think there's an easy way to make it global, sorry 😦