When designing payment systems, one of the less obvious challenges is that you are effectively dealing with a distributed system. You cannot run a database transaction across an external HTTP boundary. The moment you call a payment processor, your local ACID guarantees stop at that boundary.
The happy path in non-production environments can make the flow seem straightforward, but things become very different when you have an active database serving thousands of requests, experiencing contention, retries, timeouts, and partial failures. The real question becomes: how do you ensure that your internal system state remains consistent and recoverable before and after the external transaction is processed?
After spending some time building these systems and encountering bugs that were not immediately obvious, I think the most durable pattern is to persist intent locally before performing the external side effect. Create a durable local entity in a transitional state such as Pending or PaymentPending, commit that state, process the payment, then transition the entity based on the result.
You might ask, why not take the payment first and create the entity afterwards? The problem is that this creates a failure window where the payment succeeds but the local write does not. You have now taken the customer's money without necessarily having enough durable application state to reliably recreate or reconcile what the payment was for.
This is essentially a state-machine approach, often combined with patterns such as the Transactional Outbox and idempotent processing. The principle is simple: persist intent first, perform the external side effect second, then make every subsequent state transition recoverable and idempotent.