Single-Worker Job Claim Queue with Crash-Safe Requeue

    1

    When one process produces work and separate workers do it, the queue in the middle has to survive a worker dying mid-job.

    $29

    Secure checkout via Stripe

    Money-back guarantee

    Converts to your local currency at checkout

    Security scanned

    Works with the AI Agents you already use

    Claude CodeClaude CodeCursorCursorCodex CLICodex CLIDeepSeekDeepSeekMuseMuse+22 more

    Single-Worker Job Claim Queue with Crash-Safe Requeue

    Example session with this skill installed

    I need a new work queue instance for my media processing server. It should persist to ./queue-data, have a 5-minute timeout, and allow 3 retries. Provide the initialization and a claim example.

    • Read your context and instructions
    • Compiled the single-worker job claim

    The work queue is initialized with a local data directory for persistence. The following code demonstrates setting up the queue and the logic for a worker to claim and complete a job.

    Work Queue Implementation

    import { WorkQueue } from './work-queue.mjs';
    
    // 1. Initialize the queue with persistence and retry logic
    const queue = new WorkQueue({
      dataDir: './queue-data',
      workerKey: 'settled-secret-key-123',
      claimTimeoutSeconds: 300,
      maxAttempts: 3,
      retentionHours: 24
    });
    
    // 2. Worker authentication and job claim
    const workerId = 'worker-node-01';
    try {
      // Validate worker access
      queue.requireWorker('settled-secret-key-123');
      
      // Claim the oldest available job
      const job = queue.claim(workerId);
      
      if (job) {
        console.log(`Claimed job ${job.id}. Attempt: ${job.attempts}`);
        
        // Report progress during processing
        queue.progress(job.id, { percent: 50 }, workerId);
        
        // Mark as finished with result descriptors
        queue.complete(job.id, { name: 'processed_video.mp4', bytes: 1048576 }, workerId);
      }
    } catch (err) {
      console.error('Worker action failed:', err.message);
    }
    
    // 3. Maintenance (should run on a timer)
    queue.sweepClaims();
    queue.sweepRetention();
    

    Next steps

    • Ensure the ./queue-data directory exists and has write permissions.
    • Implement an external result store (like S3 or a local folder) for the actual output bytes.
    • Set up a setInterval to trigger sweepClaims() every 60 seconds to recover timed-out jobs.

    Connects securely to your tools. The creator never sees your data.

    About this skill

    When one process produces work and separate workers do it, the queue in the middle has to survive a worker dying mid-job. A naive queue either loses the job or runs it forever. This is a small, file-backed queue that claims a job to exactly one worker, requeues it if that worker goes silent, and caps retries so a poison job cannot loop. It is independent of what the work is; the payload is opaque.

    work-queue.mjs: enqueue(payload) adds a unit of work with a payload-size cap. claim(workerId) moves the oldest queued job to running, stamps the worker that holds it, and counts the attempt; a second worker claiming at the same moment gets a different job or nothing. progress, complete and fail reject any worker that is not the holder (409), so a worker that lost its job to a timeout cannot overwrite the result of the worker that re-claimed it. complete records result descriptors ({ name, bytes }), never the result bytes. fail returns a job to the queue while it has retries left and makes it terminal once it does not. sweepClaims() returns a running job whose worker has been silent past the claim timeout to the queue, or fails it if out of retries, so a crashed worker never strands a job; sweepRetention() removes finished and aged-out jobs. The worker surface is gated by a shared key compared in constant time.

    Jobs persist as one JSON file each, written to a temp file and renamed into place, so a reader never sees a half-written job and a crash mid-write leaves at most an ignored temp file. On restart the queue reloads every job and skips a single unreadable file rather than failing the load.

    Reference: the full lifecycle diagram, why a claim stamps the worker and counts attempts at claim time, why the claim timeout is the liveness contract that must run on a timer, how atomic persistence works, and how to turn at-least-once delivery into something safe with idempotent work or a completion marker. 12 tests. Node 20+, standard library, no network. For one host with multiple worker processes; it is not a distributed queue.

    Find more skills like this on the Claude Skills Marketplace, the Codex Skills Marketplace and the Gemini Skills Marketplace, or browse Claude Skills for APIs & Backend.

    Single-host only; does not support distributed queue instances sharing one directory. Does not store result bytes, only descriptors. Requires external result storage logic.

    How to use

    1. 1

      Connect your agent to the Agensi MCP

      Copy the prompt and paste it into Claude, ChatGPT, Cursor or any agent that supports MCP. It sets up the connection for you. Manual setup

    2. 2

      Sign in to Agensi

      Your agent shows a sign-in link. Sign in and click Allow.

    3. 3

      Ask for what you need

      Your agent loads this skill and uses it on your task. It always asks before buying a skill.

    Reviews

    No reviews yet

    Be one of the first to try it. Every listed skill goes through the trust checks below.

    Security scanned

    Checked by our 8-point scan before listing

    Fresh listing

    Recently published to Agensi

    Money-back guarantee

    Doesn't do what its page promises? You get your money back.

    Trust & safety

    Security scan

    Security scan passed 18 days ago

    Software
    Node.js 20+

    Permissions

    Read Files
    Write Files
    Listed18 days ago

    What's inside

    Frequently Asked Questions

    This skill provides a file-backed work queue for Node.js environments that guarantees job persistence without the overhead of external databases like Redis, specifically solving the problem of job loss during worker crashes or system reboots.

    The skill requires Node.js 20 or higher and is designed for any AI agent or backend application running on a single host that can access the local file system.

    The package includes the core queue logic, a worker authentication system, cleanup scripts for automated maintenance, and a configuration guide to help you integrate it into your producer-worker workflow.

    While it doesn't support network-based distribution across different machines, you can run multiple worker processes on a single host as long as they all point to the same local data directory and use unique worker IDs.

    It uses atomic file operations (temp-file renames) and 'worker gating' to ensure that if a process crashes mid-action, the job remains in a recoverable state and cannot be corrupted by overlapping worker claims.

    New in APIs & Backend