Getting Started

This guide will help you set up JoobQ, a high-performance job queue system for the Crystal programming language. Follow these steps to integrate JoobQ into your project and start managing background jobs efficiently.


Prerequisites

Before installing JoobQ, ensure your system meets the following requirements:

  1. Crystal Language: Version 1.0 or later. Install Crystal.

  2. Redis: JoobQ uses Redis as its backend for queue management. Ensure Redis is installed and running on your system.


Installation Styles

JoobQ is a robust job queue and scheduler library for the Crystal programming language, designed to handle background tasks efficiently. Depending on your application's architecture, you can set up JoobQ in three distinct configurations:

  1. Client Only: For applications that enqueue jobs without processing them locally.

  2. Client with Embedded Server: For applications that both enqueue and process jobs within the same environment.

  3. Server Only: For dedicated job processing servers that handle jobs from multiple clients.

Below are detailed installation and configuration guides for each setup.


1. Client Only Setup

This configuration is ideal for applications that need to enqueue jobs to be processed by external workers.

Prerequisites

  • Crystal Language: Ensure Crystal is installed on your system. Install Crystal

  • Redis Server: JoobQ utilizes Redis for job storage. Install Redis

Installation Steps

  1. Add JoobQ to Your Project

    Include JoobQ in your shard.yml dependencies:

    dependencies:
      joobq:
        github: azutoolkit/joobq
        version: "~> 0.4.1"
  2. Install Dependencies

    Run the following command to install the dependencies:

    shards install
  3. Configure JoobQ

    Set up JoobQ to connect to the Redis server:

    require "joobq"
    
    JoobQ.configure do |config|
      config.store = JoobQ::RedisStore.new(
        host: "localhost",
        port: 6379,
        password: nil,
        pool_size: 5
      )
    end
  4. Enqueue Jobs

    Define your job classes and enqueue jobs as needed:

    class MyJob 
      include JoobQ::Job
    
      def perform
        # Job logic here
      end
    end
    
    # Enqueue a job
    MyJob.enqueue

2. Client with Embedded Server Setup

This setup is suitable for applications that enqueue and process jobs within the same environment.

Prerequisites

  • Crystal Language: Ensure Crystal is installed on your system. Install Crystal

  • Redis Server: JoobQ utilizes Redis for job storage. Install Redis

Installation Steps

  1. Add JoobQ to Your Project Include JoobQ in your shard.yml dependencies:

    dependencies:
      joobq:
        github: azutoolkit/joobq
        version: "~> 0.4.1"
  2. Install Dependencies Run the following command to install the dependencies:

    shards install
  3. Configure JoobQ Set up JoobQ with queue definitions and scheduling:

    require "joobq"
    require "./jobs/*"
    
    JoobQ.configure do |config|
      config.store = JoobQ::RedisStore.new(
        host: "localhost",
        port: 6379,
        password: nil,
        pool_size: 5
      )
    
      # Define queues
      queue name: "default", workers: 2, job: MyJob
    
      # Schedule jobs
      scheduler do
        every(1.minute, MyJob)
      end
    end
  4. Start the JoobQ Server Run the following command to start processing jobs:

    JoobQ.forge

3. Server Only Setup

This configuration is intended for dedicated servers that process jobs from multiple clients.

Prerequisites

  • Crystal Language: Ensure Crystal is installed on your system. Install Crystal

  • Redis Server: JoobQ utilizes Redis for job storage. Install Redis

Installation Steps

  1. Add JoobQ to Your Project

    Include JoobQ in your shard.yml dependencies:

    dependencies:
      joobq:
        github: azutoolkit/joobq
        version: "~> 0.3.1"
  2. Install Dependencies

    Run the following command to install the dependencies:

    shards install
  3. Configure JoobQ

    Set up JoobQ with queue definitions:

    require "joobq"
    require "./jobs/*"
    
    JoobQ.configure do |config|
      config.store = JoobQ::RedisStore.new(
        host: "localhost",
        port: 6379,
        password: nil,
        pool_size: 5
      )
    
      # Define queues
      queue name: "default", workers: 5, job: MyJob
    end
  4. Start the JoobQ Server

    Run the following command to start processing jobs:

    JoobQ.forge

By following the appropriate setup guide, you can configure JoobQ to match your application's architecture, ensuring efficient job queuing and processing.


Additional Features

  • REST API: Enable the REST API for monitoring queues and jobs:

    config.rest_api_enabled = true
  • Cron Schedules: Use cron-like expressions for job scheduling:

    scheduler do
      cron(pattern: "*/5 * * * * *") { puts "Runs every 5 seconds!" }
    end
  • Middlewares: Customize job execution with middleware:

    config.use do |middlewares|
      middlewares << MyCustomMiddleware.new
    end

Troubleshooting

  1. Redis Connection Issues: Ensure Redis is running and accessible at the default host (localhost) and port (6379).

  2. Missing Jobs: Confirm that all job classes are required in your application.

  3. Worker Not Processing Jobs: Verify that the worker process is running and that jobs are being enqueued.


Congratulations! JoobQ is now installed and ready to handle your background tasks. Explore the JoobQ Documentation for advanced configurations and features.

Last updated