Tag: Laravel

In enterprise Laravel systems, the queue is where the business actually runs: imports, billing calculations, notifications, webhooks, reports, AI ingestion, and scheduled workflows. The fastest way to create outages is to treat queues like “background tasks” instead of a production-critical system.

This guide shows how to run Laravel queues + Horizon safely at scale: queue separation, correct retry configuration, idempotency, long-running jobs, batch pipelines, and monitoring—without duplicate processing or random worker meltdowns.

For the full enterprise build guide, start here: Laravel Development (2026): The Complete Guide to Building & Scaling Enterprise Applications. If you need help hardening your production queues under SLA: Laravel Maintenance.

Quick navigation

• 1) Why queues break in production
• 2) Enterprise queue design (separation + priorities)
• 3) timeout vs retry_after (the #1 duplicate-processing cause)
• 4) Idempotency patterns (do this or pay later)
• 5) Long-running jobs: imports, exports, heavy rating
• 6) Batches & pipelines: how to structure multi-stage processing
• 7) Monitoring & alerting (what to track)
• 8) Incident playbook: backlog spikes, deadlocks, failures
• 9) Copy/paste checklist
• Next steps

1) Why queues break in production

• Mixed workloads: imports + notifications + CPU-heavy rating all share the same workers.
• Wrong timeouts: workers kill jobs early; jobs restart; duplicates happen.
• No idempotency: retries create double charges, double emails, double exports.
• Unbounded jobs: one job tries to do “everything” and eats memory.
• No queue monitoring: you discover problems only after customers complain.

Enterprise rule: A job must be safe to retry. If it isn’t, it’s not production-ready.

2) Enterprise queue design (separation + priorities)

Start by separating workloads by behavior. A clean enterprise baseline looks like:

• critical (payments, billing actions, contract state changes)
• notifications (emails/SMS/webhooks)
• imports (file ingestion + normalization)
• compute (rating engines, analytics, heavy transforms)
• exports (CSV/XLS/PDF generation)

Why this matters: if exports spike, your payments must still run. If imports spike, your notifications must still send. Queue separation prevents “cross-contamination.”

Horizon example: dedicated supervisors per workload

Here’s a clean starting point (adapt to your infra). The key is: separate queues + different timeouts + right worker counts.

// config/horizon.php (example)
'environments' => [
  'production' => [
    'critical' => [
      'connection' => 'redis',
      'queue' => ['critical'],
      'balance' => 'auto',
      'minProcesses' => 2,
      'maxProcesses' => 10,
      'tries' => 3,
      'timeout' => 120,
    ],

    'notifications' => [
      'connection' => 'redis',
      'queue' => ['notifications'],
      'balance' => 'auto',
      'minProcesses' => 1,
      'maxProcesses' => 8,
      'tries' => 5,
      'timeout' => 120,
    ],

    'imports' => [
      'connection' => 'redis',
      'queue' => ['imports'],
      'balance' => 'auto',
      'minProcesses' => 1,
      'maxProcesses' => 4,
      'tries' => 1,
      'timeout' => 3600,
    ],

    'compute' => [
      'connection' => 'redis',
      'queue' => ['compute'],
      'balance' => 'auto',
      'minProcesses' => 2,
      'maxProcesses' => 20,
      'tries' => 2,
      'timeout' => 1800,
    ],

    'exports' => [
      'connection' => 'redis',
      'queue' => ['exports'],
      'balance' => 'auto',
      'minProcesses' => 1,
      'maxProcesses' => 6,
      'tries' => 2,
      'timeout' => 900,
    ],
  ],
];

3) timeout vs retry_after (the #1 duplicate-processing cause)

This mistake causes “phantom duplicates” in billing systems: the worker kills a job at timeout, but Redis thinks it’s still running, then re-queues it after retry_after and the job runs again.

Rule of thumb: retry_after must be greater than your longest real job runtime (plus buffer), and worker timeout must align to that reality.

// config/queue.php (Redis)
'redis' => [
  'driver' => 'redis',
  'connection' => 'default',
  'queue' => env('REDIS_QUEUE', 'default'),
  // Must be > longest job runtime + buffer
  'retry_after' => 5400,
  'block_for' => null,
  'after_commit' => true,
];

Enterprise reality: If you don’t measure job runtime distribution (p95/p99), you’ll guess timeouts—and guessing creates duplicates.

4) Idempotency patterns (do this or pay later)

Idempotency means: if a job runs twice (retry, redeploy, crash), the business outcome remains correct. Here are enterprise-grade patterns that work.

A) Idempotency key + DB unique constraint

Best for payments, invoices, webhooks, and “create once” operations.

// Example: write once using a unique key
// Table has UNIQUE(idempotency_key)

DB::transaction(function () use ($key, $payload) {
  $already = DB::table('job_outcomes')
    ->where('idempotency_key', $key)
    ->lockForUpdate()
    ->first();

  if ($already) {
    return; // safe retry
  }

  // Do the business action
  // ...

  DB::table('job_outcomes')->insert([
    'idempotency_key' => $key,
    'status' => 'done',
    'created_at' => now(),
    'updated_at' => now(),
  ]);
});

B) Distributed lock (Redis) for “one-at-a-time” processing

Best for imports, per-customer processing, per-file pipelines, and allowance tracking (bundles).

use Illuminate\Support\Facades\Cache;

$lock = Cache::lock("import:{$supplierId}:{$period}", 7200);

if (! $lock->get()) {
  return; // another worker is processing this already
}

try {
  // do work safely
} finally {
  optional($lock)->release();
}

C) “Exactly-once” isn’t real—design for safe “at-least-once”

Queues are designed for at-least-once delivery. Your architecture must assume duplicates are possible and handle them safely.

5) Long-running jobs: imports, exports, heavy rating

Enterprise strategy is simple: no single job should do unbounded work. Use chunking + batches.

• Parse files using streams (don’t load entire CSV into memory).
• Process in chunks (1k–5k rows) based on your workload.
• Commit progress frequently (and log progress events).
• Use “replace vs patch” write modes depending on the pipeline.

CTO signal: Chunking + idempotency is how you make “reprocessing” safe—critical in billing and finance systems.

6) Batches & pipelines: structure multi-stage processing

Most enterprise workflows are pipelines, not single jobs. Example:

Import (chunked) →
Normalize + dedupe →
Rate (compute queue) →
Apply bundles →
Update summary →
Export / notify

Use batch IDs to tie all stages together so ops can answer: “What happened for period X?”

• Store job_id / batch_id on records created/updated
• Write processing history per stage (start/end + counts + errors)
• Make each stage rerunnable independently (with idempotency)

7) Monitoring & alerting (what to track)

Enterprise queue monitoring should trigger alerts before customer impact.

• Backlog depth per queue (critical/imports/compute)
• Job failure rate (spikes = release regression or upstream data shift)
• Runtime p95/p99 for heavy jobs (timeout tuning and capacity planning)
• Deadlocks/lock wait timeouts (DB-level contention)
• Worker restarts/OOM (memory leaks or oversized jobs)

This is exactly what an enterprise maintenance plan should include: monitoring + tuning + incident response. See: Laravel Maintenance.

8) Incident playbook: backlog spikes, deadlocks, failures

Scenario A: backlog spike on imports

• Confirm upstream change (new file format/volume).
• Scale imports supervisor temporarily.
• Reduce chunk size if memory is spiking.
• Enable strict validation + issue logging so pipeline doesn’t stall.

Scenario B: deadlocks/lock wait timeouts

• Identify hot queries and add the right composite indexes.
• Reduce transaction scope (shorter transactions).
• Batch updates (e.g., 200–500 rows) instead of huge updates.
• Implement deadlock retry around critical updates.

Scenario C: job failures after release

• Pause only the affected queue (not all queues).
• Rollback the release or hotfix the specific job.
• Replay failed jobs safely only if idempotency is confirmed.

9) Copy/paste enterprise checklist

1. Separate queues by workload (critical/notifications/imports/compute/exports).
2. Align worker timeout and Redis retry_after to real p95/p99 runtimes.
3. Make every job safe to retry (idempotency keys + locks + unique constraints).
4. Chunk long tasks (stream files, small batches, frequent commits).
5. Use batch IDs + processing histories per stage.
6. Monitor backlog depth, failure rates, runtime distribution, and worker memory.
7. Create an incident playbook for spikes, deadlocks, and regressions.

Next steps (internal links)

Upgrading to Laravel 12 safely? Laravel Upgrade Service. Building AI ingestion pipelines (RAG indexing/agents) on queues? Laravel AI Development.

FAQ

What is the most common cause of duplicate processing in Laravel queues?

Misconfigured retry_after vs worker timeout, combined with jobs that aren’t idempotent. If a job is killed and re-queued, it will run twice unless your design prevents duplicates.

How do enterprises scale Horizon safely?

By separating workloads into dedicated queues and supervisors, measuring job runtimes (p95/p99), and applying idempotency patterns so retries are safe.

Should we use one queue or multiple queues?

Multiple queues. Mixing imports, CPU-heavy compute, and critical billing actions causes unpredictable latency and incidents. Queue separation makes performance predictable and safer under load.

Enterprise Laravel success isn’t about “writing more code.” It’s about building a system that stays stable as teams grow, features multiply, and traffic becomes unpredictable. This blueprint gives you a practical reference architecture for DDD-inspired structure, a clean Service Layer, event-driven workflows, and queue design that scales—without overengineering.

If you want the full enterprise guide first, start here: Laravel Development (2026): The Complete Guide to Building & Scaling Enterprise Applications.

What you’ll get in this blueprint

• When DDD is worth it (and when it isn’t)
• Reference architecture: layers + responsibilities
• Service Layer pattern: the enterprise “center of gravity”
• Domain events vs integration events
• Queues at scale: Horizon, retries, idempotency, outbox
• Testing strategy that survives growth
• Copy/paste checklist for your next Laravel project

When DDD is worth it (and when it isn’t)

DDD is not a requirement. It’s a scaling strategy. Use it when the problem domain is complex and the business rules change frequently (pricing, billing, compliance, multi-tenant workflows, approvals, auditability). Avoid heavy DDD when your app is mostly CRUD with light rules—because complexity becomes cost.

Enterprise shortcut: You don’t need “full DDD.” In Laravel, a DDD-inspired modular structure + a strong Service Layer gives you 80% of the value with far less ceremony.

Reference architecture: layers + responsibilities

This architecture keeps your domain rules stable while allowing UI, API, jobs, and integrations to evolve independently.

How this looks in a real Laravel codebase

app/
  Domains/
    Billing/
      Actions/
      DTO/
      Events/
      Jobs/
      Models/
      Policies/
      Repositories/
      Services/
      ValueObjects/
  Http/
    Controllers/
  Console/
  Support/
    Idempotency/
    Outbox/
    Tenancy/
    Observability/

Important: You don’t have to rename everything. The point is separation: controllers stay thin, services orchestrate, domain rules are centralized, and infrastructure details don’t leak everywhere.

Service Layer pattern: the enterprise “center of gravity”

The Service Layer is where most enterprise Laravel teams win or lose. It’s the layer that:

• Coordinates use cases (create order, approve invoice, renew subscription, apply bundle, etc.)
• Wraps operations in transactions
• Enforces authorization + policies
• Emits events after state changes
• Schedules jobs for async work (emails, exports, webhooks, heavy computation)

Thin Controllers rule (copy/paste standard)

Controllers validate input, call a single service method, and return a response. If a controller starts accumulating “ifs” and complex logic, it’s leaking business rules.

Service Layer example (credible Laravel shape)

<?php
// app/Domains/Billing/Services/InvoiceService.php

namespace App\Domains\Billing\Services;

use App\Domains\Billing\Events\InvoiceIssued;
use App\Domains\Billing\Models\Invoice;
use App\Domains\Billing\Repositories\InvoiceRepository;
use Illuminate\Support\Facades\DB;

class InvoiceService
{
    public function __construct(
        private InvoiceRepository $invoices,
    ) {}

    public function issue(array $data): Invoice
    {
        return DB::transaction(function () use ($data) {
            // 1) validate invariants (domain rules should live in domain models/value objects too)
            // 2) persist state changes via repository
            $invoice = $this->invoices->createIssued($data);

            // 3) emit domain event (listeners/jobs can react)
            event(new InvoiceIssued($invoice->id));

            return $invoice;
        });
    }
}

This pattern keeps your workflows testable and your controllers clean. Most importantly, it creates a single place to trace “what happened” in production.

Events: Domain events vs Integration events (don’t mix them)

Enterprise event-driven systems fail when events become “random notifications.” The fix is classification:

• Domain events: something meaningful happened in your domain (InvoiceIssued, PaymentCaptured, SubscriptionRenewed).
• Integration events: messages that leave your system or cross boundaries (InvoiceIssuedToERP, CustomerSyncedToCRM).

Domain event example

<?php
// app/Domains/Billing/Events/InvoiceIssued.php

namespace App\Domains\Billing\Events;

class InvoiceIssued
{
    public function __construct(public string $invoiceId) {}
}

Listener that schedules async work (email/webhook/export)

<?php
// app/Domains/Billing/Listeners/QueueInvoiceNotifications.php

namespace App\Domains\Billing\Listeners;

use App\Domains\Billing\Events\InvoiceIssued;
use App\Domains\Billing\Jobs\SendInvoiceEmailJob;

class QueueInvoiceNotifications
{
    public function handle(InvoiceIssued $event): void
    {
        SendInvoiceEmailJob::dispatch($event->invoiceId)
            ->onQueue('notifications');
    }
}

Enterprise rule: Your domain event is not “send an email.” Your domain event is the business fact. Emails, webhooks, and exports are reactions.

Queues at scale: how enterprise Laravel stays fast

Queues aren’t just “background processing.” In enterprise apps they are the safety valve that protects your API from slow tasks and unpredictable workloads.

Queue design: separate workloads (don’t dump everything into “default”)

• default: small, quick jobs only
• imports: heavy CSV parsing, normalization, batching
• rating/compute: CPU-heavy calculations
• notifications: email/SMS/webhooks
• reports: exports, PDFs, scheduled reports

Reliability essentials: retries, timeouts, and backoff

• Timeout should match real job runtime (plus headroom).
• Backoff prevents retry storms during outages.
• Max attempts should match business risk (don’t retry payments 20 times).

Idempotency: the #1 enterprise queue rule

Every job should be safe to run more than once. This protects you from retries, worker restarts, and duplicate delivery. Use an idempotency key strategy per use-case.

<?php
// app/Support/Idempotency/IdempotencyGuard.php

namespace App\Support\Idempotency;

use Illuminate\Support\Facades\Cache;

class IdempotencyGuard
{
    public static function once(string $key, int $ttlSeconds, callable $fn)
    {
        $lock = Cache::lock("idem:{$key}", $ttlSeconds);

        if (! $lock->get()) {
            return null; // already processing or already processed recently
        }

        try {
            return $fn();
        } finally {
            optional($lock)->release();
        }
    }
}

In jobs: build a key like invoice-email:{invoiceId} or webhook:{eventId}:{endpoint}.

The Outbox Pattern (enterprise-grade event delivery)

If you publish messages to other systems (webhooks, Kafka, CRM, ERP), you will eventually face a failure window: DB commits, but message publish fails. The Outbox Pattern solves this by storing outbound events in your DB inside the same transaction, then delivering them asynchronously.

// Outbox table fields (conceptual):
id (uuid)
aggregate_type (e.g., Invoice)
aggregate_id
event_type (e.g., InvoiceIssued)
payload (json)
status (pending|sent|failed)
attempts
next_attempt_at
created_at

Enterprise win: your core data and “messages to the outside world” are consistent and recoverable.

Queues + Events diagram (paste into a diagram tool if needed)

HTTP Request
  ↓
Controller (thin)
  ↓
Service Layer (transaction)
  ↓
DB Commit  →  Domain Event emitted
                  ↓
            Listener schedules Jobs
                  ↓
             Queue Workers
                  ↓
     Notifications / Webhooks / Exports

Testing strategy that survives enterprise growth

Enterprise Laravel teams stop trusting tests when tests are slow, brittle, or poorly scoped. The fix is layering your tests the same way you layered your architecture.

• Unit tests (Domain): fast tests for business rules and invariants.
• Service tests (Application): test use-case orchestration with DB transactions.
• Contract tests (Integrations): external APIs mocked + verified.
• Smoke tests (Production): critical endpoints and queue health checks.

Enterprise testing rule

If a test fails, it should clearly answer: “Which business rule broke?” If you can’t tell, the test is noise.

Enterprise Laravel architecture checklist (copy/paste)

1. Controllers are thin (validate, call service, return response).
2. Service Layer owns transactions and orchestrates use cases.
3. Domain rules are centralized (entities/value objects, not scattered).
4. Events are classified: domain vs integration events.
5. Queues are separated by workload type (imports/compute/notifications).
6. Jobs are idempotent (safe retries).
7. Outbox used for reliable outbound integrations.
8. Observability exists: structured logs, error tracking, queue metrics.
9. Regression suite exists around revenue/auth/data boundaries.
10. Upgrade cadence defined (quarterly dependencies, yearly Laravel baseline).

Recommended Next Steps (Internal Links)

FAQ

Do we need full DDD to build enterprise apps in Laravel?

No. Most teams succeed with a DDD-inspired modular structure + a strict service layer. Use “full DDD” only when business rules are truly complex and constantly changing.

Where should business rules live?

In the domain (entities/value objects/domain services) and orchestrated by the service layer. Avoid scattering core rules inside controllers, jobs, or random model observers.

When should we use events and queues?

Use events for meaningful domain facts, and queues for slow or failure-prone work: notifications, exports, webhook delivery, third-party APIs, and heavy computations.

What’s the biggest queue mistake in enterprise Laravel?

Non-idempotent jobs. Retries happen. Workers restart. Networks fail. If your jobs aren’t safe to run twice, your system will eventually corrupt state or double-send actions.