Protocol Clinic
An interoperability simulator that localizes schema and lifecycle drift across MCP, A2A, and AG-UI
The problem worth solving
A tool server, delegated agent, and user-interface stream can each pass a simple isolated test while the combined workflow misroutes a result, revives a completed task, or displays completion for a failed operation. Protocol evolution creates another failure source: adapters silently assume an older wire shape or lifecycle. Build a local clinic that generates protocol traces, applies controlled drift, checks version-specific contracts, and identifies the earliest incompatible event. The three protocols retain separate responsibilities: tools and data, agent collaboration, and agent-to-interface interaction.
What could make it stand out
The contribution is a version-aware interoperability test harness with cross-layer correlation checks and minimal reproductions. It goes beyond validating individual JSON messages by testing whether a synthetic tool result, delegated task, and interface run remain correctly associated under retries, delayed delivery, cancellation, and schema change. The project explicitly separates normative protocol failures from application-profile violations. It does not present one universal state machine for all protocols or claim conformance from a small fixture suite. Compatibility claims always name the exact specification snapshots and adapter revisions tested.
On a held-out corpus of valid and drifted workflows, version-specific schema validation plus lifecycle and correlation checks will detect at least 95% of injected incompatibilities, with less than 5% false alarms on allowed traces, and localize the first faulty boundary more accurately than schema-only checks. Reject the hypothesis if the benchmark labels legitimate protocol variation as an error or if the checker succeeds by relying on fixture names rather than protocol content.
System architecture
| Component | Responsibility |
|---|---|
| Pinned specification registry | Record official document URLs, retrieved dates, schema hashes, and selected versions. Use the MCP 2026-07-28 stateless core as the current primary profile and keep older stateful examples explicitly labeled as legacy compatibility fixtures. |
| Protocol-specific adapters | Parse real wire messages into a loss-accounted internal event model while preserving original payloads. Keep MCP calls, A2A tasks and messages, and AG-UI runs and UI events distinct instead of collapsing their identifiers or states. |
| Trace generator and transport simulator | Generate valid multi-step workflows and apply delayed delivery, duplication, dropped events, version mismatches, changed fields, and cancellation races. Every mutation records the intended fault and its permitted scope. |
| Contract and correlation engine | Validate message schemas, version-specific lifecycle rules, negotiated capabilities, and application-level mappings between call, task, and run identifiers. Assign each finding a rule source and severity. |
| Clinic replay and minimizer | Show the earliest failing event with the original payload, normalized representation, governing rule, and downstream symptom. Reduce a failing trace to a smaller reproduction without changing the detected failure. |
Data and reproducibility
Create 80 valid workflows and at least 160 single-fault variants, balanced across schema, version, lifecycle, correlation, and transport categories. Freeze source snapshots as of the project build date. The starting research reviewed official documentation on 2026-09-07: MCP's 2026-07-28 core is stateless, while current A2A documentation uses TASK_STATE_* task-state values. The seed checks only a normalized educational subset.
Start with a synthetic benchmark
Generate complete valid traces from explicit protocol profiles before mutating them. Include direct-message A2A responses as well as task workflows, and avoid treating every UI run as a unique long-lived task. Hold out combinations of drift operators and adapter versions. Add legitimate duplicate delivery and allowed ordering variations as negative controls, with an application-defined deduplication layer where needed. Store each expected finding separately from the tool-visible payload and distinguish a protocol rule from a stricter local policy.
Baselines you must beat
- A JSON parser and required-field checker with no selected protocol version or temporal state.
- Protocol-specific schema validation that examines each message independently and cannot detect validly shaped lifecycle or correlation failures.
- A single unversioned normalized state machine, demonstrating false alarms and missed changes caused by assuming every protocol has the same lifecycle.
Measure the claim
Targets below are proposed success criteria. No results have been achieved on your behalf.
| Metric | Definition and target |
|---|---|
| Injected incompatibility detection | Fraction of hidden mutated traces flagged for their intended failure category, separating normative schema or protocol violations from application-profile violations. Target, not achieved result: at least 95% recall on held-out drift combinations, with per-protocol and per-category counts. |
| Valid-trace false alarm rate | Fraction of independently reviewed valid traces flagged as incompatible, including legitimate duplicate delivery, direct-message responses, and allowed event-order variants. Target, not achieved result: below 5%; publish every false alarm with its specification or profile interpretation. |
| Boundary localization and reproduction size | Distance in events between the known injected fault and the checker's first relevant finding, plus the size of a minimized replay that preserves that finding. Target, not achieved result: correct boundary on at least 90% of detected single-fault traces and median minimized reproduction of at most six events. |
Experiments and ablations
- Compare schema-only and lifecycle-aware checking on traces where every message is individually well formed but a completed task receives a conflicting new state or a UI run is associated with the wrong delegated task.
- Replay a legacy MCP initialization exchange against the explicitly selected 2026-07-28 stateless-core profile. Then select the corresponding legacy profile and show why the compatibility judgment changes.
- Inject transport duplicates and reordering within documented or application-permitted limits. Measure false alarms before and after deduplication and correlate identifiers across retries without assuming that retries create new logical work.
Your execution plan
0 of 12 deliverables checked. Tick a deliverable after recording evidence in your project repository.
Select and pin protocol profiles
Implement trace generation and adapters
Build baseline validation and drift operators
Add lifecycle and correlation checking
Challenge compatibility judgments
Release a focused protocol clinic
Working seed
Run this small, deterministic core first. It demonstrates the central mechanism. The complete system, experiments, and deployment are your capstone work.
"""Protocol Clinic: a small normalized trace checker, not full conformance."""
import json
TERMINAL = {"TASK_STATE_COMPLETED", "TASK_STATE_FAILED",
"TASK_STATE_CANCELED", "TASK_STATE_REJECTED"}
KNOWN = TERMINAL | {"TASK_STATE_SUBMITTED", "TASK_STATE_WORKING",
"TASK_STATE_INPUT_REQUIRED", "TASK_STATE_AUTH_REQUIRED"}
def check_trace(events):
issues, tasks, runs = [], {}, set()
for index, event in enumerate(events):
protocol = event.get("protocol")
reason = None
if protocol == "MCP":
# Educational profile pins the stateless 2026-07-28 core.
if event.get("version") != "2026-07-28":
reason = "version_mismatch"
elif event.get("method") == "initialize":
reason = "legacy_handshake_in_stateless_profile"
elif event.get("method") == "tools/call":
params = event.get("params", {})
if not isinstance(params.get("name"), str) or not isinstance(params.get("arguments"), dict):
reason = "tool_call_shape_drift"
elif protocol == "A2A":
task_id, state = event.get("task_id"), event.get("state")
if not isinstance(task_id, str) or state not in KNOWN:
reason = "task_schema_drift"
elif tasks.get(task_id) in TERMINAL and state != tasks[task_id]:
reason = "update_after_terminal_state"
else:
tasks[task_id] = state
elif protocol == "AG-UI":
run_id, kind = event.get("run_id"), event.get("type")
if kind == "RUN_STARTED":
if run_id in runs:
reason = "duplicate_run_start"
else:
runs.add(run_id)
elif kind in {"RUN_FINISHED", "RUN_ERROR"}:
if run_id not in runs:
reason = "terminal_event_without_start"
else:
runs.remove(run_id)
elif run_id not in runs:
reason = "event_outside_active_run"
else:
reason = "unknown_protocol"
if reason:
issues.append({"index": index, "protocol": protocol, "reason": reason})
return issues
def main():
clean = [
{"protocol": "MCP", "version": "2026-07-28", "method": "tools/call",
"params": {"name": "read_fixture", "arguments": {}}},
{"protocol": "A2A", "task_id": "t1", "state": "TASK_STATE_WORKING"},
{"protocol": "AG-UI", "run_id": "r1", "type": "RUN_STARTED"},
{"protocol": "A2A", "task_id": "t1", "state": "TASK_STATE_COMPLETED"},
{"protocol": "AG-UI", "run_id": "r1", "type": "RUN_FINISHED"},
]
drifted = clean + [
{"protocol": "A2A", "task_id": "t1", "state": "TASK_STATE_WORKING"},
{"protocol": "MCP", "version": "2026-07-28", "method": "initialize"},
{"protocol": "AG-UI", "run_id": "r2", "type": "RUN_FINISHED"},
]
assert check_trace(clean) == []
assert check_trace(clean[:4] + [clean[3]] + clean[4:]) == []
issues = check_trace(drifted)
assert len(issues) == 3 and [x["index"] for x in issues] == [5, 6, 7]
print(json.dumps({"clean_issues": [], "drift_issues": issues}, sort_keys=True, indent=2))
if __name__ == "__main__":
main()
Failure modes to investigate
- Protocol documents evolve, especially unversioned latest URLs. Persist retrieved schemas and hashes and display the selected profile in every compatibility report.
- Overly strict state machines can reject legitimate behavior. Ground each rule in the selected specification or label it as application policy, and include valid variation in the test corpus.
- Normalization can erase the very field that reveals a bug. Preserve original messages, document any lossy mappings, and test correlation with protocol-specific identifiers intact.
Your demo, moment by moment
- Run a valid fictional workflow that reads a tool result, completes a delegated task, and finishes the corresponding interface run.
- Inject a field-shape change and show its schema error, then inject a well-formed task-state revival and show why message-level validation alone misses it.
- Switch between a legacy MCP profile and the selected stateless profile to demonstrate that compatibility depends on declared version rather than a universal handshake assumption.
- Minimize a failing mixed-protocol trace, inspect the exact rule and original message, and compare detection with valid-trace false alarms on the held-out suite.
Write the resume bullet after the experiment
Replace every placeholder with your actual measurements. Keep the dataset size, baseline, and evaluation conditions available for interview questions.
A research extension
Attach the same fixture suite to two independently implemented local SDK adapters and compare interoperability results across pinned releases. A focused distributed-systems extension studies cancellation races: the user cancels an interface run while a delegated task is completing, and the system must preserve an honest final status without duplicating work. Treat authentication and deployment conformance as separate studies unless explicitly implemented and tested.
Related work to challenge your idea
Documents the stateless core and version changes. The clinic uses those changes to construct explicit current-versus-legacy compatibility profiles; the seed is not a full MCP implementation.
Defines agent collaboration, messages, task states, and protocol bindings. The clinic checks a pinned subset and preserves A2A task semantics separately from UI-run state.
Describes an event-based interface protocol. The proposed work tests stream and cross-layer correlation behavior alongside tool and delegation protocols without treating their event models as interchangeable.