Core Concepts

To make the most out of JoobQ, it's essential to understand its building blocks and how they work together to efficiently process tasks. This section introduces the core concepts that power JoobQ and some additional advanced features to unlock its full potential.

---

Jobs

Jobs are the fundamental units of work that JoobQ processes asynchronously. A job could involve tasks like sending emails, processing images, or making API requests.

Defining a Job

In JoobQ, jobs are defined as classes that include the JoobQ::Job module. Each job implements a perform method where the task's logic resides. Jobs can also include parameters to customize their behavior.

class EmailJob
  include JoobQ::Job

  property recipient : String
  property message : String

  def perform
    # Logic to send an email
    puts "Sending email to #{recipient}: #{message}"
  end
end

Key Attributes of a Job

• Job ID: A unique identifier assigned to each job.

• Queue: The queue where the job is placed.

• Retries: Number of retry attempts allowed for failed jobs.

• Status: Possible statuses include Enqueued, Running, Failed, and Completed.

---

Queues

A queue organizes jobs awaiting processing. Queues can group jobs by categories or priorities, making it easier to manage and control job processing. For example, you can use separate queues for "emails" and "image_processing."

Queue Prioritization

JoobQ allows prioritization by assigning workers to specific queues. Time-sensitive tasks can be assigned to high-priority queues, ensuring they are processed promptly.

Defining a Queue

Queues are defined in the configuration:

JoobQ.configure do |config|
  config.queue "email_queue", 5, EmailJob, { limit: 10, period: 1.minute }
end

This example defines a queue named email_queue with 5 workers and throttling limits of 10 jobs per minute.

---

Workers

Workers are processes responsible for executing jobs. They continuously listen to queues and execute jobs as they become available. JoobQ supports multiple workers for parallel job processing, allowing you to scale your system as needed.

Worker Lifecycle

1. Start: Workers connect to queues and wait for jobs.

2. Execute: A job's perform method is called.

3. Retry or Complete: Jobs that fail are retried based on the retry policy. Successful jobs are marked as Completed.

---

Scheduling Jobs

JoobQ allows scheduling jobs to run at a specific time in the future or at regular intervals. Scheduling can be done using delays or cron-based recurring schedules.

Delayed Jobs

Delay job execution until a specified time:

JoobQ.scheduler.delay(job, till: Time.utc + 10.minutes)

This enqueues the job to run 10 minutes later.

Recurring Jobs

Schedule recurring jobs for tasks like daily reports or regular cleanups:

JoobQ.scheduler.cron("0 9 * * *", job)

The above example schedules the job to run daily at 9:00 AM, using cron syntax.

---

Middlewares

Middlewares provide a mechanism to customize and enhance the job processing pipeline. They allow you to intercept job execution to add cross-cutting concerns like logging, retries, timeouts, and throttling.

Using Middlewares

Middlewares are globally configured through the use method:

JoobQ.configure do |config|
  config.use do |middlewares|
    middlewares << JoobQ::Middleware::Retry.new(max_retries: 3)
  end
end

Key Built-in Middlewares

• RetryMiddleware: Retries failed jobs.

• TimeoutMiddleware: Enforces execution time limits.

• ThrottleMiddleware: Limits the rate of job execution.

• AsyncLoggingMiddleware: Logs job details asynchronously.

---

Schedulers

Schedulers manage the timing of job execution. JoobQ supports several types of schedulers:

1. Delayed Jobs: Schedule jobs to run at a specific future time.

2. Cron Scheduler: Use cron syntax to define recurring jobs.

3. Recurring Scheduler: Schedule jobs at fixed intervals.

Defining a Scheduler

Schedulers are configured like so:

JoobQ.configure do |config|
  config.scheduler do |scheduler|
    cron("0 12 * * *", CleanupJob.new)
    delay(ReminderJob.new, till: Time.utc + 1.hour)
  end
end

---

Advanced Concepts

• Dead Letter Queues: Store jobs that fail after exhausting all retries for post-mortem analysis.

• Metrics and Monitoring: JoobQ provides built-in support to track queue performance, job execution times, and failure rates.

• Custom Middlewares: Create your own middlewares to add custom logic like notifications or analytics.

---

Summary

JoobQ’s powerful concepts—Jobs, Queues, Workers, Middlewares, Schedulers—allow you to build efficient, scalable, and maintainable background processing systems. Ready to dive deeper? Explore the Configuration section to tailor JoobQ to your application’s needs!