Workspace/Lesson workspace
Loading progress
Training & research45 min

Turn the capstone into evidence a reviewer can trust

Lesson 3 of 3
How to study this lesson

Read one section, trace the worked example, and try the knowledge check. Bookmark saves a shortcut; notes appear in My notebook. Mark lesson complete is your own assessment and does not mark its lab passed.

By the end, you can

  • Create a repository that supports quick inspection and reproduction.
  • Report quality and operational metrics with truthful scope.
  • Write resume bullets that describe actual contributions and measured evidence.

A portfolio repository should answer practical questions quickly

The first page should explain the user problem, what the system does, and how to run a small example. Then show the architecture, supported workflow, evaluation approach, and current limitations. GitHub's README documentation describes how a repository README serves as an entry point. Use that entry point to guide a reviewer toward executable evidence rather than making them infer the project from a long list of frameworks.

Provide a minimal dependency setup, example configuration without secrets, a synthetic fixture, and an expected output. Separate the local demonstration from commands that need external credentials or incur usage costs. Include screenshots or a short recording only when they help explain a completed workflow. A screenshot can demonstrate interface behavior, but it cannot substitute for reproducible evaluation or establish that the backend enforces the permissions shown on screen.

Build an evidence table with explicit scope

For each claim, point to a run or artifact that supports it. A quality result needs dataset description, sample size, metric definition, baseline, and configuration. A latency result needs workload, concurrency, hardware or service, timing boundaries, and failure handling. An isolation result needs tested attacker capabilities and the invariant checked. State which numbers are measured, which are arithmetic estimates, and which come from synthetic fixtures.

Suppose the seed completes three invented tasks and passes six local assertions. That supports the statement that the seed passed those checks, not that a real model achieved 100 percent accuracy. If no external model or protocol server was exercised, say so. An honest blank cell labeled not measured is more useful than a guessed throughput value. Keep raw results and a script that regenerates the summary so reviewers can inspect denominators and excluded cases.

Show the difference your work made

An evaluation report should compare a baseline and the final system under stated constraints. Include failure categories and an ablation that addresses the project's main architectural choice, such as whether specialist delegation improves conflict resolution relative to a single workflow with the same call budget. If delegation adds latency without improving the measured task, record that outcome and explain whether interoperability or ownership still justifies it.

Separate your original work from dependencies and generated scaffolding. Describe the contracts you designed, the failure you diagnosed, the policy you implemented, and the experiment you ran. A reviewer can assess engineering judgment from those concrete decisions. Do not claim that using a protocol means you designed the protocol, or that adapting an example means you trained a foundation model. Attribution and precise scope make the contribution easier to understand.

Write resume bullets from verified facts

A useful resume bullet names an action, a system or problem, and supported evidence of its result. Before a live measurement exists, a truthful statement could be: "Built an incident-review prototype with scoped retrieval, task-state replay, and synthetic regression tests." After an actual evaluation, replace general claims with the measured quantity and its setting, such as a held-out case count, a latency percentile, or a reproducible failure reduction. Include the baseline and workload when they are needed to interpret a percentage.

Never transform fixture success into customer impact, estimated memory into measured savings, or a personal prototype into deployed production experience. Avoid unqualified claims of state-of-the-art performance or guaranteed employment. If a team built the system, describe your own responsibility accurately. Keep a private evidence note for every number so you can explain how it was calculated and what it does not establish during an interview.

Deliver a maintained artifact with a bounded next step

The final capstone package should contain the runnable system, a small demonstration dataset, evaluation code and results, architecture and decision notes, an operations guide, and known limitations. Include pinned protocol and SDK versions, an example of a recoverable failure, and a tested cancellation or rollback path. Review the repository for secrets, private records, and claims that exceed the available evidence before making it public.

End the project report with the most informative next experiment rather than a feature wishlist. Perhaps the missing question is whether results transfer to real incident tickets, whether a different specialist reduces latency, or whether a more demanding adversarial fixture exposes a new boundary failure. A strong capstone demonstrates that you can formulate a problem, build a coherent system, measure its behavior, and communicate its limits. That combination provides useful portfolio evidence even when the best conclusion is conditional rather than spectacular.

Work through the code

The formatter forces a scope and denominator into the claim. The six checks are an illustrative input record, not newly executed tests inside this snippet; its only assertion checks formatting. Replace the record with actual evaluation output before presenting it as evidence about a project.

truthful_metric.py
python
def describe_result(result):
    successes, total = result["successes"], result["total"]
    if not 0 <= successes <= total or total <= 0:
        raise ValueError("invalid outcome counts")
    scope = result["scope"]
    allowed = {"synthetic fixture", "held-out evaluation"}
    if scope not in allowed:
        raise ValueError("unknown measurement scope")
    rate = successes / total
    return (f"{result['name']}: {successes}/{total} ({rate:.1%}) "
            f"on {scope}; run {result['run_id']}")

fixture = {"name": "Local contract checks", "successes": 6, "total": 6,
           "scope": "synthetic fixture", "run_id": "fixture-001"}
print(describe_result(fixture))
print("Evidence type: deterministic local tests")
print("Model quality, user impact, and production latency: not measured")
assert "synthetic fixture" in describe_result(fixture)
EXPECTED / ILLUSTRATIVE OUTPUT
Local contract checks: 6/6 (100.0%) on synthetic fixture; run fixture-001
Evidence type: deterministic local tests
Model quality, user impact, and production latency: not measured

Run Python snippets locally with the prerequisites named above. The module coding lab runs directly in your browser.

Pause and reason

You have built a working local protocol-inspired simulation and passed 25 synthetic authorization tests, but have not deployed a live MCP or A2A service. Write one accurate resume bullet and identify two claims to avoid.

Check your understanding

Which portfolio statement is best supported by a deterministic seed that passes all six synthetic checks?

Your notes

Explain the mechanism in your own words. Add a failure you want to test.

Saved notes appear in your notebook

Go deeper with primary sources

Practice this module