#Deferring complex DB operations by storing and passing around closures?

4 messages · Page 1 of 1 (latest)

hybrid falcon
#

This is a somewhat vague question.

In a database application, I do something like the following:

  • Get a row of work
  • Mark the work as started
  • Very slowly modify a bunch of rows (problem - APIs in the outside world and sometimes the database itself is slow for operations that cannot be simplified/broken down any more than they are)
  • Mark the work as finished

However, if an error occurs in the middle, and the program exits, I can leave records in an invalid state.

The "right" way to do it is to wrap the entire long running operation in a transaction. I'd like to avoid this route based on the unsubstantiated hunch that leaving tons of (>100k) transactions open for very long periods of time would cause performance issues in my application. (PostgreSQL)

Instead, I'd like to run the slow part first, and slowly build a list of write operations (maybe closures? maybe dyn objects?) and then quickly write them at at the end in a dedicated transaction (or fail, in which case the important bits of state are not corrupted).

I'm looking for advice on how to do that. I fear modeling DB operations as traits/dyn objects may be too inflexible (I've often encountered "pain" trying to work with heterogeneous dyns). On the other hand passing around closures (or some equivalent) also seems complicated.

I suspect I probably would have to use closures/functions pointers/etc. just to handle complex sequences of operations (create a record, use that ID to update a second record).

This is a 100% sync library/app/etc. that does one thing at a time.

On the other hand, maybe the concept as a whole is flawed. (Possibly solving the wrong problem)

#

Deferring complex DB operations by storing and passing around closures?

fathom geyser
#

If all you are doing is deferring known work until later, then Vec<Box<dyn FnOnce(...) -> Result>> is a perfectly good way to do that.

fast nest