log analyzer kafka

    1

    Expert diagnostic tool for Apache Kafka logs, identifying broker failures, consumer lag, and KRaft/ZooKeeper issues.

    $1

    Secure checkout via Stripe

    Money-back guarantee

    Converts to your local currency at checkout

    Security scanned

    Works with the AI Agents you already use

    Claude CodeClaude CodeCursorCursorCodex CLICodex CLIMuseMuseDotsDots+22 more

    log analyzer kafka

    Example session with this skill installed

    Analiza este log de un broker en PROD: [2023-10-27 10:15:01,234] ERROR [ReplicaFetcher replicaId=1, leaderId=2, fetcherId=0] Error in response with solr-logs-0 (kafka.server.ReplicaFetcherThread)

    • Read your context and instructions
    • Compiled the log analyzer kafka

    Analicé los logs del broker proporcionados. Se identifica un problema de conectividad entre réplicas en un entorno de producción.

    REPORTE DE ANÁLISIS DE LOG — APACHE KAFKA

    ╔══════════════════════════════════════════════════════════════╗
    ║ REPORTE DE ANÁLISIS DE LOG — APACHE KAFKA ║
    ╚══════════════════════════════════════════════════════════════╝

    📋 RESUMEN EJECUTIVO
    Tecnología: Apache Kafka [3.4.0] / [ZooKeeper]
    Entorno: PROD
    Componente(s): Broker (ReplicaFetcherThread)
    Período log: 2023-10-27 10:15:01
    Total eventos: 1 crítico | 0 warnings | 0 notables
    Diagnóstico breve: El seguidor (Broker 1) no puede buscar datos del líder (Broker 2) para el topic 'solr-logs'.

    🔴 HALLAZGOS CRÍTICOS

    ReplicaFetcher Error: El hilo de recuperación de réplicas falló al intentar sincronizar la partición solr-logs-0. Esto resultará en under-replicated partitions si persiste.

    Network/Auth: El error en ReplicaFetcherThread suele indicar problemas de red entre brokers o fallos en la autenticación SASL interna.

    ⚠️ DATOS SENSIBLES

    • Endpoint interno detectado: <span style="color:red;font-weight:bold">leaderId=2</span> (Requiere mapear a hostname interno).

    ❓ PREGUNTAS ABIERTAS / 🛠️ PRÓXIMOS PASOS

    1. ¿Cuál es el replication.factor del topic solr-logs?
    2. Revisar logs del Broker 2 para verificar si el líder está rechazando conexiones o sufriendo pausas de GC.
    3. Verificar conectividad en el puerto 9092/9093 entre el Broker 1 y el Broker 2.

    Next steps

    • Verifique si el Broker 2 está bajo carga extrema o si el proceso ha muerto.
    • Ejecute kafka-topics --describe para confirmar el estado de ISR de la partición afectada.
    • ¿Deseas el reporte como documento entregable?

    Connects securely to your tools. The creator never sees your data.

    About this skill

    Diagnosing distributed systems like Apache Kafka through raw logs is a manual, error-prone grind. This skill automates the identification of cluster health issues, consumer lag, and connectivity failures across the entire Kafka ecosystem.

    What it does

    • Multi-component parsing processes logs from brokers, controllers, Connect, Streams, and clients (producer/consumer).
    • Architecture detection identifies whether the cluster uses ZooKeeper or KRaft to contextualize leader election and quorum issues.
    • Error pattern matching detects ISR shrinkage, offset ranges, rebalancing loops, and SASL/SSL handshake failures.
    • PII/Security auditing flags sensitive data like JAAS configs, keystore passwords, and internal endpoints in red.
    • Resource monitoring spots Disk I/O bottlenecks, GC pauses, and OOM errors affecting broker performance.

    How it works

    1. Log Ingestion involves pasting raw logs containing markers like kafka.server or org.apache.kafka.clients.
    2. Context Elicitation triggers specific questions about environment (PROD/DEV), replication factors, and coordination modes.
    3. Structured Analysis maps log events to known patterns like CommitFailedException or under-replicated partitions.
    4. Report Generation outputs a formal diagnostic report including critical findings and open questions.

    Frameworks & tools

    Apache Kafka (2.x/3.x), ZooKeeper, KRaft, Kafka Connect, Kafka Streams, Confluent Platform, Strimzi, and Java-based Kafka clients.

    Why this beats prompting it yourself

    Standard LLM prompts often hallucinate the severity of under-replicated partitions without knowing the replication factor. This skill follows a strict diagnostic framework that checks for environment-specific nuances and specific Kafka configuration limits.

    Use cases

    • Troubleshooting sudden consumer group rebalancing loops in production.
    • Validating KRaft migration leader election issues.
    • Auditing Kafka Connect task failures and Schema Registry incompatibilities.
    • Identifying PII leakage in DEBUG logs before they hit storage.

    Known limitations

    Requires specific log markers to trigger; cannot analyze binary data or direct JMX metrics without log-based representation.

    How to use

    Skills on Agensi run inside your AI agent. Copy the prompt, paste it into your agent, and it connects to Agensi and installs this skill for you. If a payment is needed, your agent shows it to you first.

    Reviews

    No reviews yet

    Be one of the first to try it. Every listed skill passes our trust checks below.

    Security scanned

    Passed our 8-point scan before listing

    Fresh listing

    Recently published to Agensi

    Money-back guarantee

    Doesn't do what its page promises? You get your money back.

    Trust & safety

    Security scanned

    Verified clean today

    • Passed all security checks, Safe to install

    Listedtoday

    What's inside

    Frequently Asked Questions