Acknowledge webhooks after trustworthy admission
How to study this lesson
Read one section, trace the worked example, and try the knowledge check. Bookmark saves a shortcut; notes appear in My notebook. Mark lesson complete is your own assessment and does not mark its lab passed.
By the end, you can
- Verify signed requests using original bytes.
- Separate prompt acknowledgment from durable work admission.
- Deduplicate and retry without losing event meaning.
The webhook endpoint is an admission boundary
A webhook receiver should do a small amount of bounded work: authenticate the request, validate its basic shape, identify the event, and commit it to a durable inbox or queue. Then it can acknowledge delivery while a worker performs the slower agent task. Slack documents an HTTP 2xx acknowledgment within three seconds and retries failed deliveries; this is a reason to avoid running a long model call inside the request handler. [Slack Events API documentation](https://docs.slack.dev/apis/events-api/).
Acknowledging before any durable admission can lose an event if the process crashes immediately afterward. Waiting for the entire agent run can miss the platform deadline and trigger duplicate deliveries. The design target is a short, reliable admission transaction, followed by asynchronous processing. An in-memory queue is useful for a demonstration but cannot fulfill the durability claim after process restart.
Authenticate the bytes that were signed
Slack request verification uses an HMAC-SHA256 signature over a base string containing the version, request timestamp, and raw request body. The documented recipe checks timestamp freshness and compares signatures securely. Deserializing and reserializing JSON before verification can change whitespace or key order, so verify the original body bytes first. [Slack request verification documentation](https://docs.slack.dev/authentication/verifying-requests-from-slack/).
The code implements this small mechanism with a synthetic secret and timestamp. It is not a complete HTTP endpoint and should not be reused as an all-platform verifier. Discord's HTTP interaction signatures follow a different mechanism; a channel adapter must use the platform's specified scheme. Transport authentication establishes the request's origin and integrity. It does not by itself prove that every requested business action is authorized for the human sender represented inside the event.
Deduplicate the event and the effect separately
An event identifier prevents duplicate delivery from creating repeated work. Use a namespaced key containing platform and tenant along with the native event ID. Store its admission state atomically with the queued job when possible. If two receiver processes check a set and then both insert, a race can create two jobs; a database uniqueness constraint or transactional insert handles this boundary more reliably than a process-local check.
A separate operation key protects outbound effects. One admitted user message can legitimately produce several response parts or actions, each with its own logical identifier. Conversely, a retry of one effect should reuse its original key. Deduplicating inbound events alone does not prevent duplicate responses after a worker crashes between sending and checkpointing. Record the delivery result and reconcile uncertain outcomes before blindly sending again.
Make retry categories explicit
Not every error benefits from retry. A temporary upstream timeout may be retried with bounded exponential backoff and jitter. An invalid signature should be rejected. A malformed supported event needs a diagnostic record, while an unsupported but authentic event may be acknowledged and ignored according to the integration contract. A revoked authorization requires a different recovery path from a rate limit.
Suppose an event is admitted at 12:00 and its response credential expires at 12:15. A worker starting at 12:16 cannot assume the original response path still works. Persist relevant deadlines with the job and choose a supported fallback only if it is authorized. A dead-letter queue should contain enough context to inspect and deliberately replay a job, but should not become a place where secrets and entire private conversations accumulate without a retention policy.
Observe the full delivery lifecycle
Track receive time, authentication result, admission time, dequeue time, execution outcome, and outbound delivery status. These timestamps separate a slow agent from a slow queue or a failing channel API. Measure acknowledgment latency and end-to-end response latency independently. A receiver can satisfy its three second requirement while users wait minutes because a worker backlog has grown unnoticed.
Test the crash boundaries with fixtures: before inbox commit, after inbox commit before acknowledgment, after processing before outbound send, and after send before completion recording. The desired result depends on which durable fact exists at each boundary. Document this state table in the project, then use fault injection to verify it. The main lesson is that trustworthy admission, event deduplication, and effect reconciliation solve different problems. Treating them as one generic retry feature leaves important gaps hidden.
Work through the code
This deterministic demonstration follows the documented Slack signature base-string mechanism with synthetic inputs. It verifies bytes before parsing and rejects timestamps beyond a five-minute window. It is not a server, secret-management system, durable queue, or verifier for other platforms.
import hashlib
import hmac
def signature(secret, timestamp, raw_body):
base = b'v0:' + timestamp.encode('ascii') + b':' + raw_body
return 'v0=' + hmac.new(secret, base, hashlib.sha256).hexdigest()
def verify(secret, timestamp, raw_body, supplied, now):
try:
age = abs(now - int(timestamp))
except ValueError:
return False
if age > 300:
return False
return hmac.compare_digest(signature(secret, timestamp, raw_body), supplied)
secret = b'synthetic-teaching-secret'
raw = b'{"event_id":"e7","text":"status"}'
timestamp = '1000'
supplied = signature(secret, timestamp, raw)
print('valid:', verify(secret, timestamp, raw, supplied, 1100))
print('tampered:', verify(secret, timestamp, raw + b' ', supplied, 1100))
print('expired:', verify(secret, timestamp, raw, supplied, 1400))
valid: True tampered: False expired: False
Run Python snippets locally with the prerequisites named above. The module coding lab runs directly in your browser.
Pause and reason
Your endpoint commits an event to the inbox, then crashes before returning HTTP 2xx. The platform retries. What should happen?
Check your understanding
Why verify the raw body before parsing JSON?
Your notes
Explain the mechanism in your own words. Add a failure you want to test.