Trending posts / articles etc

[brazen dust]

How can I implement a trending posts or articles feature?

I'd like to be able to autoIncrement a post's views every time the record is accessed. However this wouldn't be enough on its own.
For trending posts we need to compare access over a short time period, 2-3 days for example, otherwise the displayed posts would create a positive feedback cycle.

I've considered doing this with an API route in NextJS (to read and prune data within a record based on custom date + views fields), however I was wondering whether I could use any of the built-in Payload hooks to achieve this?

Any thoughts or advice would be greatly appreciated!

[blazing moth]

New Community-Help Thread Created!

Help is on the way! To mark it as solved, use the /solve command. In the meantime, here are some existing threads that may help you:

Documentation:

• Stripe Plugin - Examples
• Form Builder Plugin - Examples
• Blocks Field - Block configs
• Querying your Documents - And / Or Logic
• Lexical Rich Text - Extending the lexical editor with Features

[whole sun]

@brazen dust There is an afterRead hook for collections, you could check if the user is not a specific role in the hook and update a field on the record

https://payloadcms.com/docs/hooks/collections#afterread

[south osprey]

I feel like afterRead isn't a good case for it. Why?

1. afterRead is called every time the doc is retrieved, including population (for example if you add a link to a post), visiting it in the admin panel, or even updating / deleting / creation (because Payload calls afterRead hooks in this case too)
2. Your response from CMS can be cached for the actual website, so then it won't be called at all

I would do something like this

type Visit = {
  ip: string;
  visitedAt: Date;
};

const minute = 60 * 1000;

const hour = minute * 60;

const day = 24 * hour;

class PostsViewsController {
  map: Map<string, Visit[]>;
  staleTime: number = day * 3;

  constructor() {
    this.map = new Map();

    setInterval(this.purgeOld, hour);
  }

  private purgeOld() {
    for (const [key, visits] of this.map) {
      const filteredVistis = visits.filter((visit) => {
        return Date.now() - this.staleTime > visit.visitedAt.getTime();
      });

      this.map.set(key, filteredVistis);
    }
  }

  checkIpAndIncrement(postId: string) {
    const postVisits = this.map.get(postId) ?? [];

    const headersStorage = headers();

    const ip = headersStorage.get('x-forwarded-for');

    if (!ip) return;

    if (postVisits.some((visit) => visit.ip === ip)) return;

    postVisits.push({ ip, visitedAt: new Date() });

    if (!this.map.get(postId)) {
      this.map.set(postId, postVisits);
    }
  }

  getPostViews(postId: string) {
    return this.map.get(postId) ?? [];
  }
}

export const postViewsController = new PostsViewsController();

const getPost = async (slug: string) => {
  const {
    docs: [post],
  } = await fetch(`https://payload-api.com/posts?where[slug][equals]=${slug}`).then((data) =>
    data.json(),
  );

  postViewsController.checkIpAndIncrement(post.id);
};

If you want it to be preserved on restarts, you can sync it with some external storage. including just using global from Payload

[shrewd yacht]

personally I fetch top / trending posts from GA every two hours and cache the results in redis. instead of redis, you could add something like a trending: number field to your collection and fetch trending posts by querying on that field.

[brazen dust]

Thanks all for your replies. Very helpful.

@south osprey thanks for your thorough example. I like that the data resides outside of the collection records.
Would you create a new global collection in Payload, “trendingPosts” for example? Or create a generic collection that could store future similar data?

@shrewd yacht Nice idea. I have no experience with redis, and wouldn’t know where to start. Do you have simple example you can share, or a link?

[shrewd yacht]

i just use vercel kv, super simple to set up: https://vercel.com/docs/storage/vercel-kv/quickstart

this is basically how I use it:

import * as redis from 'vercel/kv'
const popular_slugs = []

try {
  const cache = await redis.get('ga:popularArticles')
  popular_slugs.push(cache.slugs)
} catch (e) {
  const slugs = await getPopularArticles()
  popular_slugs.push(slugs)
  await redis.set('ga:popularArticles', popular_slugs)
}

const popular_articles = await payload.find({
  collection: 'articles',
  where: {
    slug: {
      in: [popular_slugs]
    }
  }
})

[brazen dust]

Thanks @shrewd yacht — I'll check it out!

How are you fetching the GA data every two hours?

[shrewd yacht]

with this package https://www.npmjs.com/package/@google-analytics/data

  const [response] = await analyticsDataClient.runReport({
    property: `properties/${propertyId}`,
    dateRanges: [
      {
        startDate: '7daysAgo',
        endDate: 'today',
      },
    ],
    dimensions: [
      {
        name: 'pagePath',
      },
    ],
    orderBys: [
      {
        metric: {
          metricName: 'screenPageViews',
        },
        desc: true,
      },
    ],
    metrics: [
      {
        name: 'screenPageViews',
      },
    ],
    dimensionFilter: {
      filter: {
        fieldName: 'pagePath',
        stringFilter: {
          matchType: 'BEGINS_WITH',
          value: '/article/',
        },
      },
    },
    limit: 16,
  })