Repair Background Job Starvation

I have analyzed the current queue configuration. The root cause is a lack of concurrency limits on the imports queue, which allows a single tenant to saturate the default worker pool.

Sidekiq Fairness Implementation

The following implementation introduces a sidekiq-limit_fetch approach to ensure transactional emails (critical) are not starved by bulk imports.

# config/initializers/sidekiq.rb
Sidekiq.configure_server do |config|

  # Define strict limits for the imports queue to prevent starvation
  config.caps = {
    'imports' => 5 # Maximum 5 concurrent import jobs across the cluster
  }
  
  # Ensure the critical queue is always checked first, 
  # but high-volume queues are weighted to prevent total starvation
  config.queues = [
    ['critical', 10],
    ['default', 5],
    ['imports', 1]
  ]
end

# lib/workers/bulk_import_worker.rb
class BulkImportWorker
  include Sidekiq::Worker

  # Apply a client-side limit to ensure no single tenant takes more than 2 slots
  sidekiq_options queue: :imports, lock: :until_executed, 
                  lock_args: ->(args) { args.first } # client_id

  def perform(client_id, data_url)
    # Job logic here
  end
end

Deterministic Test Plan

To verify this invariant without waiting for real jobs:
1.

Concurrency Test: Use Sidekiq::Testing.fake! to enqueue 20 jobs for client_A and 5 for client_B. Verify that the scheduler only permits the defined cap for the imports queue.
2.

Priority Test: Enqueue 100 imports followed by 1 critical. Verify the critical job is popped within the next 2 worker cycles regardless of the import backlog.

Next steps

• Install the sidekiq-limit_fetch gem to enable the caps functionality.
• Update the BulkImportWorker to include the client_id as the primary lock argument.
• Monitor the sidekiq_queue_latency_seconds metric specifically for the critical queue.