Follow sensitive information through the whole system

An agent system can copy information into uploaded files, retrieval indexes, prompts, tool arguments, provider requests, traces, caches, feedback records, and training datasets. A privacy review that inspects only the user-facing response misses much of this path. Draw an inventory with the purpose of each copy, the actor that can read it, its storage location, its retention rule, and the mechanism that removes it. Include backups and exported evaluation artifacts when they are part of the system.

Start with one concrete task. A support assistant might need an order identifier and delivery status but not the customer's full payment history. Retrieve only fields needed for that operation, and pass only the required subset to the model. Minimization lowers the amount of sensitive material that can leak through a bug, a compromised component, or an unexpectedly verbose trace. It also makes later deletion and inspection easier.

Encryption protects content under specified storage or transport conditions, but an authorized application still sees plaintext when using it. Access control decides which authenticated actor may perform a particular operation on a resource. Pseudonymization replaces direct identifiers while retaining some linkage; it does not automatically make records anonymous. A stable hash of an email address can still be guessed or linked, especially when the possible inputs are easy to enumerate.

Suppose an evaluation trace replaces a customer's name with customer_17 but retains their full address and order details. The replacement reduces one obvious identifier while leaving substantial identifying information. Consider the entire record and the likely recipient's background information. Use synthetic examples for portfolio artifacts whenever possible. When real data is needed, work within the organization's documented handling rules and obtain the relevant internal review for the actual use, rather than inferring permission from technical access alone.

The training-data extraction research linked below demonstrates that language models can emit memorized training material under certain conditions. This motivates treating adaptation datasets and resulting model artifacts as part of the information lifecycle. It does not establish that every model will reveal every training record, and a failed extraction attempt does not prove that a model contains no sensitive information.

A retrieval design and a fine-tuned model have different deletion properties. Removing a document from a retrieval index can stop future authorized retrieval when all relevant copies and caches are handled, but deleting the source file does not automatically remove an influence already encoded in model parameters. Record which datasets were used in each training run and retain artifact lineage. Decide before training how correction, revocation, or data withdrawal would be handled operationally. Avoid collecting secrets merely because a later filter might remove them.

A synthetic canary is a unique marker placed in an authorized test record to reveal unintended movement. For example, insert CANARY_TENANT_BLUE_7Q into a blue-tenant fixture, then ask a red-tenant account to retrieve a similarly named document. The expected result is an authorization denial with no canary in the returned content or ordinary user-visible error. Test caches, search snippets, tool outputs, and traces because leaks can occur outside the final answer.

Define the observer precisely. A trace restricted to the security test team may legitimately contain test markers, while a cross-tenant response must not. Match against the surfaces that observer can access. A canary test establishes evidence about the tested route and configuration only. It cannot prove privacy for every prompt, tenant, model revision, or side channel. Preserve the test setup and repeat relevant cases when data paths or permissions change.

A retention policy becomes useful when each storage system has an owner, a time basis, and a deletion or expiration operation. Distinguish event time from ingestion time and clarify how retries affect expiry. A cache that refreshes expiry on every read may retain an old record indefinitely under frequent access unless its design includes an absolute maximum age. Deletion should also invalidate derivative indexes or cache entries where applicable.

For an invented seven-day trace rule, a record created at day zero should be unavailable to normal trace queries at day seven if the boundary is exclusive. Write that convention as a test instead of arguing about it after an incident. Retain only the audit metadata necessary to establish that deletion occurred, according to the system's requirements. These are engineering practices for implementing organizational constraints; legal obligations depend on the actual setting and should be supplied by qualified organizational owners.

The allowlist is a small purpose-based data inventory, not an automatic legal or privacy classifier. It demonstrates how a trace can contain an unnecessary field even when the prompt is minimized. Extend the fixture with caches, exports, and training copies.