Automation Incident Recovery Doctor

INCIDENT SUMMARY

Confirmed: One Stripe event for order #A-104 was accepted. A HubSpot create request then timed out. A later manual run returned 201, and two HubSpot deals now exist for the same customer.

Confirmed: This is a duplicate-record and ambiguous-timeout incident. The timeout does not prove that the first HubSpot create failed; it may have succeeded before Make stopped waiting.

EVIDENCE LEDGER

User-supplied: evt_71 was accepted at 10:02. This identifies the original event and incident window.
User-supplied: The HubSpot create timed out after 30 seconds. The first create has an unknown outcome.
User-supplied: The manual run returned 201 at 10:06. A later create succeeded.
User-supplied: Two deals now exist. A duplicate side effect is confirmed.
Unknown: Deal IDs, timestamps, associations, and downstream activity. These are required before merging, deleting, or replaying.
Unknown: Whether a stable order or event key is stored on the deal. This is required for an idempotent recovery decision.

IMPACT AND CONTAINMENT

Do not run the scenario again for evt_71 or order #A-104. Preserve the Make execution history and original payload. Reconcile both HubSpot deals by record ID, creation time, order or payment association, owner, stage, and any downstream activity before changing either record.

ROOT-CAUSE HYPOTHESES

Likely: An ambiguous timeout was followed by a non-idempotent retry. The first request may have created a deal before timing out; the manual run then created another. Confirm this by comparing both deal timestamps and inspecting the original Make execution details.

Possible: Duplicate event delivery or a second execution path. Search the execution history for evt_71 and order #A-104 before assuming the timeout is the only cause.

Possible: No lookup-before-create step. Inspect whether the scenario searches for an existing deal by a stable order or payment key before creating a new record.

REPLAY DECISION

Manual reconciliation first. A record-creation side effect may already have completed, and a duplicate is observed. Do not replay until the existing deals are reconciled and a duplicate check can establish whether a record already exists.

REPAIR PLAN

1. Select a canonical deal only after reviewing both records and their downstream activity.
2. Store an immutable order or event key on every HubSpot deal.
3. Add lookup-before-create: if that key already exists, stop the create path and log the event as already processed.
4. For timeouts, query by the immutable key before any retry. Route unresolved cases to a review queue instead of creating blindly.
5. Log the event key, execution ID, lookup result, confirmed HubSpot deal ID, and final processing state.

VERIFICATION MATRIX

Duplicate delivery of the same event: An existing deal is found and no new deal is created.
Ambiguous timeout: The workflow checks the immutable key before retrying and queues unresolved cases.
Missing order or event key: The create path is blocked or sent for review; no uncorrelated deal is created.
Normal clean run: Exactly one deal is created, and the stable key plus deal ID are logged.

RELAUNCH CHECKLIST

The two existing deals have been reconciled and downstream effects reviewed.
A stable event or order key is available in the workflow and target record.
Lookup-before-create is verified for an existing key.
Ambiguous timeouts use reconciliation before retry.
A responsible operator has reviewed the evidence and approved relaunch.