I'm building an asset tracking app with IoT integration. A typical use case involves a tracking ~500 assets, each equipped with beacons that report their presence every 5 seconds.
I want to show a table in the frontend listing the assets with their real-time presence status. My concern is the query result being re-sent to the client every 5 seconds, leading to high egress and other inefficiencies.
How can I design my queries (or data model) to handle this scenario efficiently? What's the right way to think about structuring data and subscriptions for high-frequency updates like this?
#Optimizing queries for high-frequency updates
faint pier
flint mirageBOT
Hi @faint pier. I'm kapa.ai, your friendly support bot. I will try to answer your question by searching through the knowledge base. :wave:
You're right to be concerned: with 500 assets each reporting every 5 seconds, a naive Convex query that reads the presence/heartbeat data directly will be invalidated and re-sent to all subscribed clients every time any asset reports in. This can quickly lead to high bandwidth usage and inefficiency, as every small update triggers a full re-send of the query result to all clients.
Best Practices for High-Frequency Updates
1. Isolate Frequently Updated Data
Instead of storing the frequently changing presence/heartbeat data directly on the asset document, use a separate table (e.g., heartbeats) to store the last-seen timestamp for each asset. This way, queries that don't care about presence won't be invalidated by heartbeat updates, and only queries that explicitly read the heartbeat data will be affected.
// schema.ts
assets: defineTable({
name: v.string(),
heartbeatId: v.id("heartbeats"),
}),
heartbeats: defineTable({
lastSeen: v.number(),
}),
2. Batch and Coarsen Presence Status
Instead of having your real-time query depend on the raw heartbeat data (which changes every 5 seconds per asset), introduce a "presence" table that stores a coarser-grained online/offline status for each asset. Update this status only when it actually changes (e.g., from offline to online, or vice versa), not on every heartbeat.
- When a heartbeat is received, if the asset was previously offline, mark it as online.
- Use a scheduled cron job to periodically check for assets that haven't reported in recently and mark them as offline in batch.
This way, your frontend query can subscribe to the presence status, which only changes when an asset actually goes online or offline, not on every heartbeat.
// schema.ts
presence: defineTable({
isOnline: v.boolean(),
heartbeatId: v.id("heartbeats"),
}).index("by_isOnline", ["isOnline"]),
// When heartbeat received
if (!presence.isOnline) await ctx.db.patch(presence._id, { isOnline: true });
await ctx.db.patch(heartbeat._id, { lastSeen: Date.now() });
// Cron job to mark offline
if (heartbeat.lastSeen < tooOld) await ctx.db.patch(presence._id, { isOnline: false });
3. Query Only What Matters
Your frontend should subscribe to the presence table, not the heartbeat table. This ensures that the UI only updates when an asset's online/offline status changes, not on every heartbeat.
// Query for presence status
export const getPresenceStatus = query({
args: { assetIds: v.array(v.id("assets")) },
handler: async (ctx, args) => {
return await Promise.all(args.assetIds.map(async (assetId) => {
const asset = await ctx.db.get(assetId);
const presence = asset && await ctx.db.get(asset.presenceId);
return presence?.isOnline ?? false;
}));
},
});
Summary
- Separate high-frequency data (heartbeats) from user-facing status (presence).
- Batch and coarsen updates to only trigger query invalidation when the status meaningfully changes.
- Subscribe to the coarser-grained status in your frontend, not the raw heartbeat data.
This approach will dramatically reduce unnecessary query invalidations and bandwidth usage, making your real-time asset tracking app scalable and efficient with Convex.