Enqueue is one transaction, two tables

Free preview 5 min to read

The junior course "Solid Queue: background jobs" listed this app's eleven solid_queue_* tables and read config/queue.yml line by line. It stopped at the names. This course reads the code that moves rows between those tables, using solid_queue 1.3.2. It captures the SQL each move produces against this app's development queue database, and runs a real worker against it.

Start with a snapshot from when this lesson was written. The development queue database held 45 jobs, 26 ActiveStorage::AnalyzeJob and 19 ActiveStorage::PurgeJob. All were enqueued on 4 March 2026, all still sat in solid_queue_ready_executions, and none had finished. SolidQueue::Queue.find_by_name("default").human_latency reported "5 months, 4 weeks, 1 day".

Nothing was wrong with them. On 30 August, commit 5cfccfa switched config/environments/development.rb from queue_adapter = :solid_queue to :async. No process polled this database after that, because bin/dev starts no job process and SOLID_QUEUE_IN_PUMA is unset in development. They were an ideal specimen: enqueues frozen where the enqueue transaction left them. That database has since been cleared, but the rows below are from that snapshot.

What perform_later writes

ActiveJob::QueueAdapters::SolidQueueAdapter#enqueue is one line, SolidQueue::Job.enqueue(active_job), and Job.enqueue is a create! with these attributes:

def attributes_from_active_job(active_job)
  {
    queue_name: active_job.queue_name || DEFAULT_QUEUE_NAME,
    active_job_id: active_job.job_id,
    priority: active_job.priority || DEFAULT_PRIORITY,
    scheduled_at: active_job.scheduled_at,
    class_name: active_job.class.name,
    arguments: active_job.serialize,
    concurrency_key: active_job.concurrency_key
  }
end

arguments isn't your arguments. It's active_job.serialize, the whole Active Job envelope as JSON: job_class, job_id, queue_name, arguments, executions, exception_executions, locale, timezone, enqueued_at and scheduled_at. The oldest row carried "arguments" => [{"_aj_globalid" => "gid://newbie-on-rails/ActiveStorage::Blob/30"}] inside that envelope, plus "executions" => 0 and "locale" => "en". Lesson 4 shows why it matters for retries that executions lives inside the JSON.

A callback writes the second table. SolidQueue::Job::Executable declares after_create :prepare_for_execution:

def prepare_for_execution
  if due? then dispatch
  else
    schedule
  end
end

def dispatch
  if acquire_concurrency_lock then ready
  else
    handle_concurrency_conflict
  end
end

private
  def ready
    ReadyExecution.create_or_find_by!(job_id: id)
  end

due? is scheduled_at.nil? || scheduled_at <= Time.current. A plain perform_later sets scheduled_at to now, so the job is due. With no concurrency key, acquire_concurrency_lock returns true without touching the database. Here's the result, captured from this app's query log for a job on a throwaway probe queue, with the concurrency lines that belong to lesson 3 left out:

BEGIN
INSERT INTO "solid_queue_jobs" ("queue_name", "class_name", "arguments", "priority",
  "active_job_id", "scheduled_at", "finished_at", "concurrency_key", "created_at", "updated_at")
  VALUES ('probe', 'ProbeJob', '{"job_class":"ProbeJob", ...}', 0, '213942c3-...', ...)
SAVEPOINT active_record_1
INSERT INTO "solid_queue_ready_executions" ("job_id", "queue_name", "priority", "created_at")
  VALUES (72, 'probe', 0, '2026-09-01 20:00:31.479980') RETURNING "id"
RELEASE SAVEPOINT active_record_1
COMMIT

One transaction, two inserts. The savepoint comes from create_or_find_by!, which guards against a duplicate job_id that the unique index on solid_queue_ready_executions.job_id would reject. A job row without an execution row is a job Solid Queue never looks at again. The two inserts committing together is what makes the state machine trustworthy.

Why the execution row repeats queue_name and priority

Look at the ready row for the oldest job in that snapshot:

{"id" => 45, "job_id" => 45, "queue_name" => "default", "priority" => 0,
 "created_at" => 2026-03-04 17:26:22.354953000 UTC +00:00}

queue_name and priority are already on solid_queue_jobs. SolidQueue::Execution::JobAttributes copies them here:

class_attribute :assumable_attributes_from_job, instance_accessor: false, default: %i[ queue_name priority ]

def assume_attributes_from_job
  self.class.assumable_attributes_from_job.each do |attribute|
    send("#{attribute}=", job.send(attribute))
  end
end

ReadyExecution calls assumes_attributes_from_job with no extra names, so it copies those two. ScheduledExecution adds scheduled_at, and BlockedExecution adds concurrency_key.

This is deliberate denormalization. The query a worker runs every tenth of a second, lesson 2's ORDER BY priority, job_id LIMIT n, filters and sorts on the execution table alone, through an index that holds those columns, and never joins solid_queue_jobs. The job row is read once, after the claim, when the work starts.

Bulk enqueue is the same shape, wider

ActiveJob.perform_all_later goes through Job.enqueue_all, which trades the callback for two multi-row statements inside one transaction:

INSERT INTO "solid_queue_jobs" (...) VALUES (...), (...), (...)
SELECT "solid_queue_jobs".* FROM "solid_queue_jobs"
  WHERE "solid_queue_jobs"."active_job_id" IN ('9f7a667e-...', '43e0b7ef-...', '465f0ac7-...')
  ORDER BY "solid_queue_jobs"."id" ASC
INSERT INTO "solid_queue_ready_executions" ("queue_name","priority","job_id","created_at")
  VALUES ('probe', 0, 49, CURRENT_TIMESTAMP), ('probe', 0, 50, CURRENT_TIMESTAMP), ('probe', 0, 51, CURRENT_TIMESTAMP)
  ON CONFLICT DO NOTHING RETURNING "id"
COMMIT

Then come two plucks, one against solid_queue_ready_executions and one against solid_queue_blocked_executions, to learn where each job landed. dispatch_all sends jobs without a concurrency key through the bulk insert, and jobs with one through dispatch, one at a time. successfully_enqueued is set only for jobs found in one of the two tables.

The transaction you aren't in

Solid Queue's adapter still defines enqueue_after_transaction_commit? returning true, but Rails 8.1 doesn't consult it. The per-job enqueue_after_transaction_commit setting decides, it defaults to false, and this app doesn't turn it on.

It wouldn't make the enqueue atomic anyway. The mid-level course "Pattern: outbox pattern" worked through why: the queue lives in a second database, so the two inserts above can never share a transaction with your Partners::Partner row. This course doesn't repeat that argument. It takes the job row as given and follows it.

What this course covers

  1. This lesson: the enqueue transaction, and why execution rows repeat job columns.
  2. The claim: FOR UPDATE SKIP LOCKED, the two poll indexes, and what a worker does ten times a second.
  3. Concurrency controls as rows: a semaphore with a unique key, and a blocked execution with an expiry.
  4. Scheduled, failed and retried: the dispatcher's batch, the failure record, and why retry_on creates a second job row.
  5. Processes, heartbeats, and what happens to a claimed job when its worker dies.
  6. Recurring tasks, pauses, and the validation that refuses to start with this app's deploy values.
  7. One real job traced through every table, and the design choices the tables encode.

Lesson 2 is where rows like those 45 finally move, once something polls them.

Lesson 1 of 7 · free

Keep going with lesson 2

The claim: FOR UPDATE SKIP LOCKED and two poll indexes

How a worker claims rows with FOR UPDATE SKIP LOCKED, orders them by priority, and uses two poll indexes.

A free account also gives you the quizzes, your progress and notes, and the discussion.

Type to start searching...