What's the best way to implement table-level or document-level TTL? e.g. 4 hour ttl for documents
#What's the best way to implement table-level or document-level TTL?
cyan kettle
tacit vigilBOT
Hi @cyan kettle. I'm kapa.ai, your friendly support bot. I will try to answer your question by searching through the knowledge base. :wave:
Convex does not have built-in, automatic table-level or document-level TTL (Time-To-Live) features like some databases (e.g., MongoDB's TTL indexes). However, you can implement TTL behavior yourself using a combination of fields and scheduled cleanup.
Recommended approach:
-
Add an expiration field:
Add a field to your documents, such asexpiresAt, which stores the timestamp when the document should expire. -
Scheduled cleanup:
Use a scheduled function (cron job) to periodically scan the table and delete documents whoseexpiresAtis in the past. Convex supports scheduled functions, and you can use them to implement this cleanup logic. -
Query filtering:
When querying, always filter out expired documents by checking theexpiresAtfield.
Example:
// When inserting a document, set expiresAt to 4 hours from now
await ctx.db.insert("myTable", {
...data,
expiresAt: Date.now() + 4 * 60 * 60 * 1000, // 4 hours in ms
});
// Scheduled function to delete expired documents
export const cleanupExpiredDocs = mutation({
handler: async (ctx) => {
const now = Date.now();
const expiredDocs = await ctx.db
.query("myTable")
.filter((q) => q.lt("expiresAt", now))
.collect();
for (const doc of expiredDocs) {
await ctx.db.delete(doc._id);
}
},
});
You can schedule this cleanup mutation to run periodically using Convex's scheduling features.
Reference:
- Convex supports scheduled functions and you can use them for periodic cleanup tasks. See the
_scheduled_functionssystem table for more details on how scheduled jobs are tracked and executed in Convex SystemDataModel: _scheduled_functions. - For more on writing and deleting data, see Writing Data.
Note:
If you want a more plug-and-play solution for caching with TTL, the Action Cache component provides TTL-based caching for expensive function results, but it's not a general-purpose document TTL for arbitrary tables.
If you need more details on scheduling or want code for the scheduled function setup, let me know!
Note: you can ask me a follow up question by @ mentioning me again :speech_balloon:
cyan kettle
@tacit vigil will this work for deleting thousands of rows
tacit vigilBOT
Let me check the knowledge base for you again. One moment! :books:
Deleting thousands of rows in a single Convex mutation or scheduled function is not recommended and will likely hit system limits. Convex enforces limits such as:
- A maximum of 8,192 documents written per mutation
- A maximum of 4,096 reads per function execution
- A 1-second execution time limit for mutations
- 8 MiB data scanned or written per function
If you attempt to delete thousands of documents at once, you may encounter errors like "Too many reads in a single function execution" or timeouts. The recommended approach is to process deletions in batches, using pagination and scheduling to avoid hitting these limits. For example, you can:
- Query and delete a batch (e.g., 500) of expired documents in one mutation.
- Schedule the next batch using a scheduled function or recursively schedule the cleanup until all expired documents are deleted.
This pattern is discussed in Convex documentation and community support threads, where it's suggested to keep each function "light" and use recursive scheduling or pagination to handle large numbers of documents safely and reliably Read/write limit errors Discord support thread.
Summary:
No, deleting thousands of rows in a single function will not work reliably. Use batching and scheduled or recursive functions to process large deletions safely.
Note: you can ask me a follow up question by @ mentioning me again :speech_balloon:
cyan kettle
@tacit vigil so is convex the right tool for a TTL table or not
tacit vigilBOT
Let me check the knowledge base for you again. One moment! :books:
Convex does not have a built-in, automatic TTL (Time-To-Live) feature for tables or documents like some other databases. However, you can implement TTL behavior yourself by adding an expiration field (such as expiresAt) and using scheduled functions to periodically delete expired documents. This approach is commonly recommended in the Convex community and documentation.
That said, there are important limitations: Convex mutations and scheduled functions have strict limits on execution time and the number of documents that can be read or written in a single operation. If you need to delete thousands of documents at once, you must implement batching and possibly recursive or scheduled cleanup to avoid hitting these limits. This means that while Convex can support TTL-like behavior, it requires more manual work and careful handling for large-scale deletions, and may not be as seamless as in databases with native TTL support Production Limits Discord support thread.