Data & Analytics

Redis Queue (BullMQ)

Build reliable job queues and background workers with BullMQ, powered by Redis for distributed processing.

Claude Code Cursor Copilot Windsurf

Overview

BullMQ is a robust, feature-rich job queue library for Node.js built on Redis. It provides reliable message processing with at-least-once delivery semantics, ensuring that jobs are never lost even if workers crash. BullMQ supports delayed jobs, repeatable jobs (cron), job priorities, rate limiting, and complex workflow orchestration with parent-child job relationships.

The library uses Redis Streams and Lua scripts for atomic operations, providing strong guarantees about job processing order and completion. Workers can process jobs concurrently with configurable concurrency limits, and failed jobs are automatically retried with exponential backoff. The event system provides real-time visibility into queue state changes, job progress, and worker health.

BullMQ is the successor to Bull and provides significant improvements including support for flows (multi-step job pipelines), sandboxed processors for isolated job execution, and global events. The library includes a dashboard UI (Bull Board) for monitoring queues, and integrates with OpenTelemetry for distributed tracing. It is widely used for email sending, image processing, data pipelines, and webhook delivery.

Who Is This For?

Process email sending jobs in the background
Build image/video processing pipelines with worker pools
Schedule recurring tasks like report generation
Implement webhook delivery with retry logic

Installation

Setup for Claude Code

npm install bullmq

Configuration

import { Queue, Worker } from "bullmq"

const emailQueue = new Queue("emails", {
  connection: { host: "localhost", port: 6379 },
})

// Add a job
await emailQueue.add("welcome-email", {
  to: "user@example.com",
  subject: "Welcome!",
})

// Process jobs
const worker = new Worker("emails", async (job) => {
  await sendEmail(job.data)
}, { connection: { host: "localhost", port: 6379 }, concurrency: 5 })