Rate Limiting

Rate limiting is an important feature in job processing systems like JoobQ to ensure that external resources are not overwhelmed by excessive requests. The Queue class in JoobQ provides an optional throttling mechanism that allows you to control the rate of job processing for each queue.

This section covers how to use rate limiting to manage job execution rates, including setting up throttling parameters and understanding how it affects queue behavior.

Overview of Rate Limiting

Rate limiting helps you control the frequency of job execution in a given queue. This is particularly useful when interacting with APIs or services that enforce rate limits or when jobs require access to shared resources that must be protected from overload.

In JoobQ, rate limiting is configured through the use of throttle limits, which define the number of jobs that can be processed within a specific time period.

Configuring Rate Limiting for a Queue

To configure rate limiting for a queue, you can set a throttle limit when initializing the Queue class. This is done by providing a NamedTuple with the limit and period.

Example Setup

Here is an example of how to create a queue with rate limiting enabled:

throttle_limit = {limit: 10, period: 1.minute}
queue = JoobQ::Queue(ExampleJob).new("rate_limited_queue", 5, throttle_limit)

In this example:

• limit: Defines the maximum number of jobs that can be processed within the specified time period.

• period: Defines the time window within which the rate limit is applied.

In the above code, a queue named rate_limited_queue is created with a limit of 10 jobs per minute.

How Rate Limiting Works

When a rate limit is applied to a queue, the worker processes jobs while respecting the specified throttling limit. This ensures that jobs are executed at a controlled rate, helping prevent rate-limit violations from external services and avoiding overloading shared resources.

Internals of Throttling

• Throttler Component: The Throttler class is responsible for enforcing the rate limit. When a worker fetches a job from a rate-limited queue, it uses the Throttler to determine whether the job can be processed or if it should be delayed.

• Job Delays: If the rate limit has been reached, the worker will delay processing the job until the throttling period resets. This mechanism prevents more jobs from being processed than allowed within the time frame.

Example Usage

Here is a complete example that demonstrates how to use rate limiting in a queue:

require "joobq"

# Define a job
struct ExampleJob
  include JoobQ::Job
  property x : Int32

  def initialize(@x : Int32)
  end

  def perform
    puts "Performing job with x = #{x}"
  end
end

# Configure the queue with rate limiting
throttle_limit = {limit: 5, period: 1.minute}
queue = JoobQ::Queue(ExampleJob).new("rate_limited_queue", 3, throttle_limit)

# Add jobs to the queue
10.times do |i|
  queue.add(ExampleJob.new(x: i).to_json)
end

# Start the queue
queue.start

In this example:

• limit: 5 means that at most 5 jobs will be processed every minute.

• total_workers: 3 indicates that there are 3 workers fetching jobs from the queue.

With rate limiting in place, even if there are multiple workers available, only 5 jobs will be processed every minute, as specified by the throttle limit.

Monitoring Rate Limited Queues

When rate limiting is applied to a queue, it's important to monitor the performance and ensure that jobs are being processed as expected. The following metrics can be helpful:

• Jobs Processed Per Period: Indicates how many jobs have been processed within the rate limit period.

• Throttled Jobs: Shows how many jobs were delayed due to hitting the rate limit.

• Queue Size: Tracks the number of pending jobs in the queue. If the queue size is consistently high, you may need to adjust the throttle limit or add more workers.

You can retrieve these metrics using the JoobQ metrics API or by calling the queue.info method:

queue_info = queue.info
puts queue_info.to_json

The output will include information such as the number of jobs processed, throttled jobs, and other relevant metrics.

Best Practices for Rate Limiting

• Align Rate Limits with External APIs: If you are interacting with an API that enforces rate limits, set the throttle limit to match or be slightly below the API's rate limit to avoid errors.

• Monitor Performance: Use metrics to monitor how rate limiting is affecting job processing and adjust the throttle limit accordingly.

• Use Rate Limits to Protect Resources: Rate limiting can be useful for managing access to shared resources, such as databases or external services, preventing them from being overwhelmed.

• Increase Workers for Higher Throughput: If you need to increase the throughput, consider adding more workers while keeping an appropriate throttle limit in place.

---

Rate limiting in JoobQ helps you maintain control over the job execution rate, protecting external services and shared resources from being overwhelmed. By configuring throttle limits for each queue, you can ensure that your job processing is both efficient and compliant with any rate restrictions that may apply.