Make multitenancy and governance executable
How to study this lesson
Read one section, trace the worked example, and try the knowledge check. Bookmark saves a shortcut; notes appear in My notebook. Mark lesson complete is your own assessment and does not mark its lab passed.
By the end, you can
- Propagate trusted tenant identity through data, tools, caches, and background tasks.
- Define accountable release gates and operational ownership.
- Connect failure budgets and constraints to admission and rollback behavior.
Identity must survive every hop
A multitenant agent service shares infrastructure while serving users with different data and permissions. The authenticated principal must be established by a trusted boundary and propagated through retrieval, tool calls, background jobs, storage, and cache access. A tenant name found in a prompt or model-generated JSON is a requested value, not proof of identity. Recheck permissions where protected data or actions are actually accessed.
Consider a task queued by tenant red and executed later by a worker. If the queue message contains only a document identifier, the worker may lose the context needed to enforce tenant isolation. Store the trusted subject and tenant references, bind them to the task, and resolve current permissions when execution occurs. Long-running work also needs a decision about revoked access: authorization at submission time may not remain valid at execution or artifact download time.
Shared caches and quotas are part of isolation
A cache key should include the dimensions that determine whether reuse is valid, such as tenant scope, resource revision, model revision, and relevant configuration. Omitting a tenant dimension can create cross-tenant response reuse even when the underlying retrieval database is correctly scoped. Cache invalidation should respond to resource changes and permission changes where the cached artifact could otherwise disclose newly inaccessible data.
Resource isolation also matters. One tenant sending very long prompts or many parallel tool calls can exhaust shared capacity and degrade other tenants. Apply request-size limits, concurrent-task limits, and resource budgets using trusted tenant identity. A global limit alone does not guarantee fairness. Test the noisy-neighbor case: one tenant saturates its budget while another submits a small legitimate request. Decide whether reserved capacity, weighted scheduling, or separate workers are needed for the promised service behavior.
Governance defines who decides and what evidence they need
Governance is not satisfied by a document that says someone should monitor the model. Name an accountable owner for model changes, data access, incident response, evaluation acceptance, and rollback. The NIST AI Risk Management Framework organizes voluntary risk-management outcomes through govern, map, measure, and manage. Use that framing to connect organizational decisions to concrete evidence, without treating it as a certification or a substitute for the organization's applicable requirements.
A release proposal should state intended use, unsupported use, changed components, evaluation results, resource impact, and unresolved limitations. The approver needs the exact artifact and configuration being proposed, not just a model family name. Define when human review is required and where its decision is enforced. If an action requires approval, bind that approval to the specific action, resource, scope, and expiry so a different proposed operation cannot reuse it.
Operational constraints belong in the architecture
Latency, cost, availability, privacy, and staffing limits can conflict. A system that depends on a human approval within 30 seconds cannot meet that promise when no reviewer is on call. A fallback that sends a confidential prompt to a different provider may violate the configured data boundary even if it improves uptime. A retry policy that ignores a task budget can convert one temporary failure into excessive cost or duplicate actions.
Write degradation behavior explicitly. When a provider fails, the service may return a retriable error, switch to an allowed alternative, reduce functionality, or require manual handling. Choose based on the task contract. For an invented monthly task budget of 10,000 units, reserving a maximum of 25 units before admission prevents 500 simultaneous tasks from each spending unbounded amounts. Reconcile reservation with actual use and release unused budget when a task finishes or is canceled.
Operate a versioned decision system
Keep a release record linking the model, prompts, tools, data snapshot, policy configuration, evaluation manifest, and deployment revision. A rollback must restore a compatible set, not just the model weights. If a new tool schema was introduced alongside a model change, rolling back only one component can produce a broken combination. Test rollback and cancellation while the system is healthy, when diagnosis is easier.
Monitor violations and near misses with minimal necessary data. Alert on unauthorized access attempts, unusual export volume, repeated budget exhaustion, and stale background work using defined thresholds and owners. Review whether a failure indicates a code defect, a changed workload, or an outdated policy assumption. These lessons address engineering and operational governance rather than jurisdiction-specific legal advice. The goal is a service whose decisions can be inspected, constrained, and corrected by the people responsible for its use.
Work through the code
Canonical JSON makes the key deterministic while tenant and revision fields distinguish reuse domains. Hashing is not an authorization check or encryption. Real retrieval must still validate the authenticated principal and current resource permissions.
import hashlib
import json
def cache_key(tenant, resource, revision, model_revision):
identity = {"tenant": tenant, "resource": resource,
"revision": revision, "model": model_revision}
encoded = json.dumps(identity, sort_keys=True, separators=(",", ":"))
return hashlib.sha256(encoded.encode()).hexdigest()
cache = {}
red_key = cache_key("red", "guide", 3, "model-r1")
blue_key = cache_key("blue", "guide", 3, "model-r1")
new_key = cache_key("red", "guide", 4, "model-r1")
cache[red_key] = "Red-specific answer"
print("same tenant hit:", red_key in cache)
print("other tenant hit:", blue_key in cache)
print("new revision hit:", new_key in cache)
assert len({red_key, blue_key, new_key}) == 3
print("scope: key separation; authorization still required")
same tenant hit: True other tenant hit: False new revision hit: False scope: key separation; authorization still required
Run Python snippets locally with the prerequisites named above. The module coding lab runs directly in your browser.
Pause and reason
A queued task was authorized yesterday, but the user lost access today before the worker runs. Describe a policy and test that avoid relying on stale permissions.
Check your understanding
A cache includes the resource identifier and model revision but omits tenant identity. The database itself enforces tenants correctly. What risk remains?
Your notes
Explain the mechanism in your own words. Add a failure you want to test.