Repair Object Storage Integration Failures

I have analyzed the multipart upload lifecycle and identified a risk where the AbortMultipartUpload call is bypassed during specific context cancellations, leading to orphaned parts and increased storage costs.

Multipart Upload Reliability Repair

The following implementation ensures a canonical invariant: every initiated upload must either reach a Complete state or be explicitly Aborted regardless of the failure mode.

func UploadWithReliableCleanup(ctx context.Context, svc *s3.S3, input *s3.CreateMultipartUploadInput, data io.Reader) error {
    // 1. Initiate upload and capture UploadID immediately
    out, err := svc.CreateMultipartUploadWithContext(ctx, input)
    if err != nil {
        return fmt.Errorf("initiation failed: %w", err)
    }
    uploadID := out.UploadId
    
    // 2. Define cleanup closure to handle all exit paths
    var completed bool
    defer func() {
        if !completed {
            abortInput := &s3.AbortMultipartUploadInput{
                Bucket:   input.Bucket,
                Key:      input.Key,
                UploadId: uploadId,
            }
            // Use a background context for cleanup to ensure it runs even if ctx is cancelled
            _, _ = svc.AbortMultipartUploadWithContext(context.Background(), abortInput)
        }
    }()

    // 3. Part processing logic (omitted for brevity)
    // ... logic to upload parts and populate completedParts slice ...

    // 4. Finalization
    completeInput := &s3.CompleteMultipartUploadInput{
        Bucket:          input.Bucket,
        Key:             input.Key,
        UploadId:        uploadId,
        MultipartUpload: &s3.CompletedMultipartUpload{Parts: completedParts},
    }
    
    _, err = svc.CompleteMultipartUploadWithContext(ctx, completeInput)
    if err == nil {
        completed = true
    }
    return err
}

Deterministic Test Plan

Next steps

• Verify if your S3 bucket has a LifecycleConfiguration to automatically clean up failed multipart uploads after 7 days as a secondary safety net.
• Audit the MaxRetries setting in your SDK Session configuration to ensure it matches your downstream timeout requirements.
• Apply the background context pattern to the Abort call to prevent cleanup truncation during SIGTERM.