Rails · Active Job · database transactions

The Rails job that ran before checkout committed

A background worker was fast enough to read an order while checkout still held the new state inside a transaction. The job behaved correctly for the data it could see, which is exactly what made the bug confusing.

The checkout request succeeded. The order ended up marked as paid. The background job also reported success. Yet the customer never received a receipt.

Nothing had crashed. The job had simply started a few milliseconds too early. It loaded the order through another database connection, saw the last committed status, decided there was nothing to do, and returned. A moment later the checkout transaction committed, but no work remained to send the receipt.

The short version

Putting a job on a queue is an external side effect. If it happens before the database transaction commits, a worker may run before the data it needs becomes visible.

The harmless-looking symptom

This class of bug rarely looks like a transaction problem at first. The visible clues are usually mundane:

A fast successful job is easy to dismiss as good news. Here it is the strongest clue. The worker reached a guard clause, saw an order that was not yet paid, and stopped exactly as its code instructed.

A shop example

Consider a checkout service that changes an existing order from pending to paid. Stock reservation is delegated to another object, and that method schedules the receipt job.

class Checkout
  def initialize(order)
    @order = order
  end

  def complete!
    Order.transaction do
      order.update!(status: "paid", paid_at: Time.current)
      reserve_stock_and_schedule_receipt
    end
  end

  private

  attr_reader :order

  def reserve_stock_and_schedule_receipt
    InventoryReservation.transaction do
      order.line_items.each do |item|
        InventoryReservation.create!(
          order: order,
          product: item.product,
          quantity: item.quantity
        )
      end
    end

    SendReceiptJob.perform_later(order.id)
  end
end

The job is defensive. It refuses to send a receipt for an unpaid order:

class SendReceiptJob < ApplicationJob
  def perform(order_id)
    order = Order.find(order_id)

    return unless order.status == "paid"

    ReceiptMailer.with(order: order).receipt.deliver_now
  end
end

Read one method at a time and the code looks plausible. The reservation method has its own transaction, that block finishes, and only then does it enqueue the job. The hidden fact is that the whole method is still inside the transaction opened by Checkout#complete!.

The race, step by step

1
Checkout opens a transactionThe web process uses database connection A.
2
The order becomes paid on connection AThe new status exists inside the transaction but has not committed.
3
The reservation method finishes its transaction blockBecause it is nested, this does not make the outer changes visible.
4
The receipt job reaches the queueA worker can pick it up immediately.
5
The worker reads through connection BPostgreSQL returns the last committed order status: pending.
6
The job exits successfullyThe guard clause treats the old status as a legitimate reason to stop.
7
Connection A commitsThe order is now visibly paid, but the receipt job has already gone.

Sometimes connection B will start after the commit and everything works. Sometimes it wins the race and sees the previous state. Queue load, database latency, network timing, and unrelated work all change the window, which is why the failure can disappear during debugging.

Why the worker saw old data

PostgreSQL uses READ COMMITTED as its default isolation level. A normal query sees rows committed before that query began. It does not see uncommitted changes made by another transaction.1

Observer Connection Visible order status before commit
Checkout code A, inside the transaction paid, including its own uncommitted write
Background worker B, separate transaction pending, the last committed value

This is not a stale Active Record object or a cache invalidation bug. It is the database preserving transaction isolation. The web process and worker are asking valid questions from different visibility boundaries.

The nested transaction illusion

The inner InventoryReservation.transaction block is the trap. By default, nested Active Record transactions join the existing transaction. Finishing the inner block does not issue an independent durable commit. Even requires_new: true normally introduces a savepoint, not a change that other database connections can see before the outer commit.2

Order.transaction do                 # real outer boundary
  order.update!(status: "paid")

  InventoryReservation.transaction do # joins the outer transaction
    InventoryReservation.create!(order: order)
  end                                 # no externally visible commit here

  SendReceiptJob.perform_later(order.id)
end                                   # visibility changes here

The dangerous method does not need to contain the outer transaction itself. A controller, service object, callback, import routine, or test fixture can call it from inside one. Transaction safety is therefore a property of the complete call path, not just the method where perform_later appears.

What modern Rails does

Rails 7.2 added automatic Active Job deferral. When the configured queue adapter supports it, a job enqueued inside an Active Record transaction is held until commit and discarded if the transaction rolls back.3 This removes the most common version of the race, but it is not a reason to stop thinking about the boundary.

Check the actual application rather than assuming:

# Make the intended boundary explicit for jobs that require committed data.
class ApplicationJob < ActiveJob::Base
  self.enqueue_after_transaction_commit = true
end

The current Active Job guide also warns against accidentally depending on a same-database queue's transactional behavior, because moving that queue to another database changes the guarantee.4

Choosing the right fix

When a service may run inside or outside a transaction

ActiveRecord.after_all_transactions_commit expresses the requirement directly. It runs immediately when no transaction is open, waits for the outermost commit when transactions are nested, and does not run if one of those transactions rolls back.5

def schedule_receipt(order)
  order_id = order.id

  ActiveRecord.after_all_transactions_commit do
    SendReceiptJob.perform_later(order_id)
  end
end

Capture stable identifiers, not a mutable model instance. The job should reload the order from committed database state.

When the code owns the transaction

Rails can yield a transaction object, making the commit hook local to the operation:

Order.transaction do |transaction|
  order.update!(status: "paid", paid_at: Time.current)
  reserve_stock(order)

  transaction.after_commit do
    SendReceiptJob.perform_later(order.id)
  end
end

If this transaction is nested, Rails transfers the callback toward the parent and runs it only after the outer boundary commits.6

When the event belongs to a model transition

An after_update_commit callback can be appropriate when sending the job is inseparable from a specific persisted transition:

class Order < ApplicationRecord
  after_update_commit :schedule_receipt, if: :just_became_paid?

  private

  def just_became_paid?
    saved_change_to_status? && status == "paid"
  end

  def schedule_receipt
    SendReceiptJob.perform_later(id)
  end
end

This is concise, but callbacks can hide control flow and may fire from more update paths than intended. A service-level commit hook is often clearer when checkout owns the business event.

When losing the job is unacceptable

An after-commit callback orders two actions, but it does not make the database commit and external queue write atomic. The database can commit and the queue can still be unavailable a millisecond later. Rails also notes that an exception in an after-commit callback cannot roll back data that has already committed.6

For a durable handoff, write an outbox row in the same transaction as the order change. A separate dispatcher reads committed outbox rows and publishes them with retries and idempotency.

Order.transaction do
  order.update!(status: "paid", paid_at: Time.current)

  OutboxEvent.create!(
    topic: "order.paid",
    aggregate_id: order.id,
    payload: { order_id: order.id }
  )
end
Need Useful mechanism
A normal Active Job must see committed records Rails 7.2+ enqueue-after-commit behavior, verified for the adapter
A reusable method may be called under nested transactions ActiveRecord.after_all_transactions_commit
The operation owns a clear transaction boundary Transaction object's after_commit
The job must survive a queue outage after commit Transactional outbox with retry and idempotency

What does not fix it

Adding another transaction

A nested block does not create an early visibility boundary. It usually joins the outer transaction, and a savepoint still belongs to that outer transaction.

Using with_lock

A row lock solves a different problem: competing writers. with_lock runs its block inside a transaction, so enqueuing from that block still needs commit-aware behavior. It also cannot protect against an unknown outer transaction.

Adding a delay to the job

Scheduling the job five seconds later makes the race less likely, not impossible. A slow transaction, queue clock difference, retry, or deployment pause can reopen the window. Time is not a commit signal.

Retrying every early return

A retry can be useful defensive behavior, but it should not replace a correct publication boundary. The worker cannot reliably distinguish "not committed yet" from "this order should never become paid" without additional state.

Testing without timing guesses

Do not write a test that sleeps for 100 milliseconds and hopes the worker loses the race. Test the ordering contract directly: nothing should reach the queue inside the transaction, and the job should appear after commit.

test "receipt is enqueued only after checkout commits" do
  order = orders(:pending)
  clear_enqueued_jobs

  Order.transaction do
    order.update!(status: "paid")
    schedule_receipt(order)

    assert_no_enqueued_jobs
  end

  assert_enqueued_with job: SendReceiptJob, args: [order.id]
end

Add a rollback case as well:

test "receipt is not enqueued when checkout rolls back" do
  order = orders(:pending)

  assert_no_enqueued_jobs do
    Order.transaction do
      order.update!(status: "paid")
      schedule_receipt(order)
      raise ActiveRecord::Rollback
    end
  end
end

Then keep one integration test with the production-like adapter and separate worker processes. Inline execution changes the topology: it runs on the caller's thread and can hide the separate-connection behavior that caused the production bug.

How to recognize it in production

The most useful log is a small causal trace, not a larger pile of unrelated SQL:

checkout order=4821 status=paid transaction=open
receipt_job enqueued order=4821
receipt_job started order=4821
receipt_job observed order=4821 status=pending
receipt_job skipped order=4821 reason=not_paid
checkout committed order=4821

Record these signals together:

Jobs that finish in one or two milliseconds deserve attention when their useful path normally sends mail, calls an API, renders a document, or processes several records. A success metric without an outcome metric can turn a skipped job into a healthy-looking dashboard.

The design lesson

The bug was not that the queue was too fast or PostgreSQL was stale. Each component honored its contract. The application published work before the state required by that work was durably visible.

The reliable question is not "am I inside the method's transaction?" It is "what is the outermost commit boundary, and can this side effect escape before it?" Once phrased that way, nested transactions stop being mysterious and job scheduling becomes part of the data-consistency design.

Practical rule

If a job depends on a database change, publish it only after the outermost transaction commits. If publishing the job must itself be guaranteed, persist an outbox event in that transaction and dispatch it separately.

Sources

  1. PostgreSQL documentation: transaction isolationpostgresql.org
  2. Rails API: Active Record transactionsapi.rubyonrails.org
  3. Rails 7.2 release notes: prevent jobs from being scheduled within transactionsguides.rubyonrails.org
  4. Rails guide: Active Job transactional integrityguides.rubyonrails.org
  5. Rails API: after_all_transactions_commitapi.rubyonrails.org
  6. Rails API: transaction commit and rollback callbacksapi.rubyonrails.org