Unit 07 · Chapter 2 · 9 min read

Decision engines, rules, and reliable execution

Turn evidence into one controlled action under time pressure.

Two services both approve the same withdrawal. Each saw enough available balance. Together, they sent too much. A risk decision must be correct in the presence of concurrency, retries, and partial failure, not only in a clean unit test.

Separate constraints scores and actions

A decision engine should distinguish mandatory constraints, risk estimates, and product actions. A confirmed prohibition cannot be outweighed by a favorable model score. Missing required evidence is also a state that needs an explicit response.

Use a clear precedence order and record every relevant rule result. The following pseudocode illustrates structure, not a deployable legal policy. The actual checks and responses need approved definitions.

if required_control_unavailable(context):
    return approved_contingency(context)
if prohibited_under_applicable_policy(context):
    return required_disposition(context)
return choose_product_action(score(context), context)
Separate constraints scores and actions — the flow
Separate constraints scores and actions Separate constraints scores and actions — the flow Follow the sequence. Choose the approved product response. Constraints Apply required boundaries Estimate Compute permitted risk signals Action Choose the approved product response
  1. ConstraintsApply required boundaries
  2. EstimateCompute permitted risk signals
  3. ActionChoose the approved product response
Follow the sequence. Choose the approved product response. Chapter sources · Open image
Separate constraints scores and actions — the distinction
Separate constraints scores and actions Separate constraints scores and actions — the distinction These concepts answer different questions. Read each definition in the context of the section. Mandatory constraint Cannot be offset by expected profit Risk estimate Informs a choice within permitted activity
Mandatory constraint
  • Cannot be offset by expected profit
Risk estimate
  • Informs a choice within permitted activity
These concepts answer different questions. Read each definition in the context of the section. Chapter sources · Open image
Decision precedence
Separate constraints scores and actions Decision precedence Fictional teaching record. No score override. Decision precedence Illustrative data; not a real customer record or a prescribed policy. Sanctions disposition required hold Binding policy result Fraud score low Separate estimate Final action required hold No score override Different decision types carry different authority
Fictional educational excerpt / Not for execution

Decision precedence

Illustrative data; not a real customer record or a prescribed policy.

  1. Sanctions dispositionrequired hold

    Binding policy result

  2. Fraud scorelow

    Separate estimate

  3. Final actionrequired hold

    No score override

Different decision types carry different authority

Fictional teaching record. No score override. Chapter sources · Open image
Separate constraints scores and actions — control and failure modes
Separate constraints scores and actions Separate constraints scores and actions — control and failure modes Different decision types carry different authority. The branches show why alternative designs fail. Control design Make precedence explicit and auditable. Different decision types carry different authority. Failure mode 1 Average all rules into one score. Mandatory constraints can disappear. avoid Failure mode 2 Treat missing screening as low risk. Unavailable evidence is not clearance. avoid Failure mode 3 Return only a number. The product still needs an action. avoid
Control design

Make precedence explicit and auditable. Different decision types carry different authority.

Failure mode 1avoid
Average all rules into one score. Mandatory constraints can disappear.
Failure mode 2avoid
Treat missing screening as low risk. Unavailable evidence is not clearance.
Failure mode 3avoid
Return only a number. The product still needs an action.
Different decision types carry different authority. The branches show why alternative designs fail. Chapter sources · Open image

Reserve capacity atomically

A limit or balance check must remain valid when multiple requests arrive together. Use database transactions, unique constraints, or another reviewed concurrency design to reserve the resource once. A separate read followed by an unprotected write can overspend capacity.

The teaching SQL below performs a conditional decrement and returns a row only when capacity was reserved. A real system also needs a unique operation key, an immutable journal, expiry handling, and reconciliation.

UPDATE risk_capacity
SET available_minor = available_minor - :amount_minor
WHERE account_id = :account_id
  AND available_minor >= :amount_minor
RETURNING available_minor;

Suppose an account has $100 of remaining capacity and two $80 requests arrive together. Each request can read the same available amount and independently pass a correct-looking limit check. The combined result exceeds the limit. The safety property belongs to the shared state transition, not to the arithmetic in either request.

Use a concurrency strategy appropriate to the datastore and service boundary, such as a transactional conditional update with defined conflict handling. The reservation needs an identifier, amount, currency, status, and expiry or release rule. A failed downstream action must not leave capacity reserved forever. A late success must not consume capacity that was released and used elsewhere without a defined resolution path. These are lifecycle cases that deserve explicit engineering treatment.

Reserve capacity atomically — the flow
Reserve capacity atomically Reserve capacity atomically — the flow Follow the sequence. Return capacity if the action does not proceed. Check and reserve Use one atomic operation Commit Link the reservation to the action Release Return capacity if the action does not proceed
  1. Check and reserveUse one atomic operation
  2. CommitLink the reservation to the action
  3. ReleaseReturn capacity if the action does not proceed
Follow the sequence. Return capacity if the action does not proceed. Chapter sources · Open image
Reserve capacity atomically — the distinction
Reserve capacity atomically Reserve capacity atomically — the distinction These concepts answer different questions. Read each definition in the context of the section. Unprotected read Can become stale before the write Atomic reservation Enforces the condition at the update boundary
Unprotected read
  • Can become stale before the write
Atomic reservation
  • Enforces the condition at the update boundary
These concepts answer different questions. Read each definition in the context of the section. Chapter sources · Open image
Concurrency example
Reserve capacity atomically Concurrency example Fictional teaching record. Second lacks remaining capacity. Concurrency example Illustrative data; not a real customer record or a prescribed policy. Available 100000 minor units Shared capacity Two requests 80000 each Compete for the same funds Allowed one request Second lacks remaining capacity Concurrent reads can both look safe
Fictional educational excerpt / Not for execution

Concurrency example

Illustrative data; not a real customer record or a prescribed policy.

  1. Available100000 minor units

    Shared capacity

  2. Two requests80000 each

    Compete for the same funds

  3. Allowedone request

    Second lacks remaining capacity

Concurrent reads can both look safe

Fictional teaching record. Second lacks remaining capacity. Chapter sources · Open image
Reserve capacity atomically — control and failure modes
Reserve capacity atomically Reserve capacity atomically — control and failure modes Concurrent reads can both look safe. The branches show why alternative designs fail. Control design Enforce shared limits at the transactional boundary. Concurrent reads can both look safe. Failure mode 1 Trust a cached balance indefinitely. It may not reflect other reservations. avoid Failure mode 2 Retry with a new operation key. That can reserve twice. avoid Failure mode 3 Forget failed reservations. Capacity can remain unavailable without cause. avoid
Control design

Enforce shared limits at the transactional boundary. Concurrent reads can both look safe.

Failure mode 1avoid
Trust a cached balance indefinitely. It may not reflect other reservations.
Failure mode 2avoid
Retry with a new operation key. That can reserve twice.
Failure mode 3avoid
Forget failed reservations. Capacity can remain unavailable without cause.
Concurrent reads can both look safe. The branches show why alternative designs fail. Chapter sources · Open image

Budget latency and failure modes

The decision path has a time budget. Identify required dependencies, optional enrichment, timeout behavior, and the point at which an action can no longer wait. A slow optional feature should not silently become a mandatory outage for every payment.

Use bounded timeouts and explicit fallback reasons. Mandatory legal controls require an approved contingency rather than an automatic fail-open. Optional fraud enrichment may have a different risk-based fallback. Test the actual combination of failures. Several dependencies that each meet a latency target can still exceed the total budget when called sequentially.

Failure behavior should follow the action’s consequence and the applicable constraint. A missing optional signal may justify a reduced limit or another evidence request. An unavailable mandatory control may require the affected action to wait. One global fail-open switch cannot express these differences. Document dependency-specific behavior, latency budgets, and the owner who can authorize a change. Test the customer-visible outcome as well as the service response: an HTTP success is not evidence that the intended protection occurred.

Budget latency and failure modes — the flow
Budget latency and failure modes Budget latency and failure modes — the flow Follow the sequence. Record the approved degraded action. Budget Allocate time across the decision path Bound Use dependency-specific timeouts Fallback Record the approved degraded action
  1. BudgetAllocate time across the decision path
  2. BoundUse dependency-specific timeouts
  3. FallbackRecord the approved degraded action
Follow the sequence. Record the approved degraded action. Chapter sources · Open image
Budget latency and failure modes — the distinction
Budget latency and failure modes Budget latency and failure modes — the distinction These concepts answer different questions. Read each definition in the context of the section. Required dependency Its absence changes permitted processing Optional enrichment May have a bounded alternative treatment
Required dependency
  • Its absence changes permitted processing
Optional enrichment
  • May have a bounded alternative treatment
These concepts answer different questions. Read each definition in the context of the section. Chapter sources · Open image
Latency budget
Budget latency and failure modes Latency budget Fictional teaching record. Bounded timeout. Latency budget Illustrative data; not a real customer record or a prescribed policy. Total 200 milliseconds Illustrative service target Required check 80 milliseconds Allocated portion Optional enrichment 40 milliseconds Bounded timeout Not every failure permits the same response
Fictional educational excerpt / Not for execution

Latency budget

Illustrative data; not a real customer record or a prescribed policy.

  1. Total200 milliseconds

    Illustrative service target

  2. Required check80 milliseconds

    Allocated portion

  3. Optional enrichment40 milliseconds

    Bounded timeout

Not every failure permits the same response

Fictional teaching record. Bounded timeout. Chapter sources · Open image
Budget latency and failure modes — control and failure modes
Budget latency and failure modes Budget latency and failure modes — control and failure modes Not every failure permits the same response. The branches show why alternative designs fail. Control design Define dependency-specific contingencies. Not every failure permits the same response. Failure mode 1 Fail open for all timeouts. Required controls may be bypassed. avoid Failure mode 2 Wait without a bound. The product can stall indefinitely. avoid Failure mode 3 Add sequential targets without measuring total. Combined latency may exceed the promise. avoid
Control design

Define dependency-specific contingencies. Not every failure permits the same response.

Failure mode 1avoid
Fail open for all timeouts. Required controls may be bypassed.
Failure mode 2avoid
Wait without a bound. The product can stall indefinitely.
Failure mode 3avoid
Add sequential targets without measuring total. Combined latency may exceed the promise.
Not every failure permits the same response. The branches show why alternative designs fail. Chapter sources · Open image

Make decisions replayable

A replay reconstructs the decision from the evidence and versions available at the time. Record policy version, model version, feature definition, input references, action, reasons, and relevant dependency states. A current model score does not explain a historical action.

Replay should not execute financial side effects. Separate pure evaluation from release, posting, and notification. Use dry-run outputs in a restricted analysis environment. Compare historical and candidate policies to understand changes, while recognizing that counterfactual outcomes for previously rejected transactions may remain unknown.

Make decisions replayable — the flow
Make decisions replayable Make decisions replayable — the flow Follow the sequence. Explain differences and unknown outcomes. Capture Preserve evidence and versions Evaluate Recompute without side effects Compare Explain differences and unknown outcomes
  1. CapturePreserve evidence and versions
  2. EvaluateRecompute without side effects
  3. CompareExplain differences and unknown outcomes
Follow the sequence. Explain differences and unknown outcomes. Chapter sources · Open image
Make decisions replayable — the distinction
Make decisions replayable Make decisions replayable — the distinction These concepts answer different questions. Read each definition in the context of the section. Decision replay Reconstructs a historical evaluation Payment replay Repeats an external action and can move value
Decision replay
  • Reconstructs a historical evaluation
Payment replay
  • Repeats an external action and can move value
These concepts answer different questions. Read each definition in the context of the section. Chapter sources · Open image
Replay contract
Make decisions replayable Replay contract Fictional teaching record. Analysis artifact. Replay contract Illustrative data; not a real customer record or a prescribed policy. Mode evaluation only No payout calls Policy historical v9 Original version Output action comparison Analysis artifact Replay must not repeat money movement
Fictional educational excerpt / Not for execution

Replay contract

Illustrative data; not a real customer record or a prescribed policy.

  1. Modeevaluation only

    No payout calls

  2. Policyhistorical v9

    Original version

  3. Outputaction comparison

    Analysis artifact

Replay must not repeat money movement

Fictional teaching record. Analysis artifact. Chapter sources · Open image
Make decisions replayable — control and failure modes
Make decisions replayable Make decisions replayable — control and failure modes Replay must not repeat money movement. The branches show why alternative designs fail. Control design Separate evaluation from side effects. Replay must not repeat money movement. Failure mode 1 Use the current policy for historical explanation. That can change the result. avoid Failure mode 2 Call payment APIs during analysis. It can create real duplicate actions. avoid Failure mode 3 Treat counterfactual approval as observed repayment. The outcome may never have occurred. avoid
Control design

Separate evaluation from side effects. Replay must not repeat money movement.

Failure mode 1avoid
Use the current policy for historical explanation. That can change the result.
Failure mode 2avoid
Call payment APIs during analysis. It can create real duplicate actions.
Failure mode 3avoid
Treat counterfactual approval as observed repayment. The outcome may never have occurred.
Replay must not repeat money movement. The branches show why alternative designs fail. Chapter sources · Open image

Release rules with rollback evidence

A policy release can affect many customers immediately. Use versioned configuration, review, controlled rollout, monitoring, and an executable rollback. Shadow evaluation can compare decisions without applying the candidate action, but it does not reveal all behavioral outcomes.

Define stop conditions for error, customer harm, coverage, and latency. Reconcile which decisions used each version. If a rollback occurs, identify any actions already taken under the candidate policy and determine whether remediation is needed. Restoring the old configuration does not undo past declines or transfers.

Release rules with rollback evidence — the flow
Release rules with rollback evidence Release rules with rollback evidence — the flow Follow the sequence. Restore and assess affected actions. Preview Compare candidate decisions safely Roll out Limit exposure and observe guardrails Rollback Restore and assess affected actions
  1. PreviewCompare candidate decisions safely
  2. Roll outLimit exposure and observe guardrails
  3. RollbackRestore and assess affected actions
Follow the sequence. Restore and assess affected actions. Chapter sources · Open image
Release rules with rollback evidence — the distinction
Release rules with rollback evidence Release rules with rollback evidence — the distinction These concepts answer different questions. Read each definition in the context of the section. Configuration rollback Future evaluations use the old version Customer remediation Addresses actions already taken
Configuration rollback
  • Future evaluations use the old version
Customer remediation
  • Addresses actions already taken
These concepts answer different questions. Read each definition in the context of the section. Chapter sources · Open image
Rule release
Release rules with rollback evidence Rule release Fictional teaching record. Past actions still need impact review. Rule release Illustrative data; not a real customer record or a prescribed policy. Candidate v14 New policy Affected decisions 600 Observed rollout population Rollback complete Past actions still need impact review Changing configuration does not undo prior effects
Fictional educational excerpt / Not for execution

Rule release

Illustrative data; not a real customer record or a prescribed policy.

  1. Candidatev14

    New policy

  2. Affected decisions600

    Observed rollout population

  3. Rollbackcomplete

    Past actions still need impact review

Changing configuration does not undo prior effects

Fictional teaching record. Past actions still need impact review. Chapter sources · Open image
Release rules with rollback evidence — control and failure modes
Release rules with rollback evidence Release rules with rollback evidence — control and failure modes Changing configuration does not undo prior effects. The branches show why alternative designs fail. Control design Pair rollback with impact assessment. Changing configuration does not undo prior effects. Failure mode 1 Release globally without version tracking. The affected population becomes hard to find. avoid Failure mode 2 Treat shadow results as full outcome proof. Customers did not experience candidate actions. avoid Failure mode 3 Stop monitoring after deployment. Behavior and data can change. avoid
Control design

Pair rollback with impact assessment. Changing configuration does not undo prior effects.

Failure mode 1avoid
Release globally without version tracking. The affected population becomes hard to find.
Failure mode 2avoid
Treat shadow results as full outcome proof. Customers did not experience candidate actions.
Failure mode 3avoid
Stop monitoring after deployment. Behavior and data can change.
Changing configuration does not undo prior effects. The branches show why alternative designs fail. Chapter sources · Open image

Chapter connections

This chapter builds on Risk data contracts and event time. Continue with Risk models, calibration, and delayed outcomes to follow the next part of the system. Use the glossary for terminology and risk mathematics for formulas and worked calculations.

Sources

Reviewed 2026-09-17
  1. PostgreSQL: transaction isolation
  2. Stripe: idempotent requests (provider example)
  3. Google SRE: handling overload