Select a strategy by utility, cost, and admissible risk
How to study this lesson
Read one section, trace the worked example, and try the knowledge check. Bookmark saves a shortcut; notes appear in My notebook. Mark lesson complete is your own assessment and does not mark its lab passed.
By the end, you can
- Compare direct answering, retrieval, search, and escalation as strategies.
- Compute expected utility under a belief state.
- Separate exploration value from permission to experiment.
Strategy selection is a decision of its own
An agent can answer immediately, retrieve evidence, execute a deterministic calculation, search multiple candidates, or ask for a required missing fact. Choosing among these strategies is a metalevel decision: the action changes how the system will solve the task. A fixed expensive strategy wastes resources on easy cases, while a fixed cheap strategy can fail when evidence or verification is necessary.
Start with a small menu whose costs and benefits can be measured. For example, direct response may be appropriate for a stable fact already supplied by the user; retrieval may resolve a current policy question; execution may verify a calculation. Strategy selection should inspect the task and evidence state, not infer difficulty solely from prompt length. A short question can require a crucial external fact.
Expected utility exposes the tradeoff
Suppose there are two states, simple and ambiguous, with probabilities 0.6 and 0.4. Direct answering yields value 8 in the simple state and negative 6 in the ambiguous state, with cost 1. Its expected utility is 0.6 times 8 plus 0.4 times negative 6 minus 1, or 1.4. Retrieval yields values 7 and 6 with cost 2, giving 4.6. Under these assumptions, retrieval is preferable.
All quantities must share a meaningful utility scale before subtraction. Dollars, milliseconds, and accuracy points cannot be combined without an explicit conversion or multi-objective policy. The example uses synthetic utility units. A production policy may instead constrain latency and error risk, then maximize expected task value within those limits. The arithmetic clarifies assumptions; it does not determine the organization's preferences automatically.
Hard boundaries precede scoring
Filter strategies for capability, authorization, data availability, and budget before ranking their utility. An unauthorized action does not become eligible because its expected benefit is large. A model-based score also cannot substitute for an explicit required verification gate. Keep the admissibility check separate so reviewers can inspect whether a prohibited option ever entered the candidate set.
Abstention can be a legitimate outcome when every available strategy has nonpositive utility or insufficient evidence. In the lab, the function returns no strategy in that case. This is a teaching convention that makes the option explicit. Applications should define what abstention means operationally: preserve the partial artifact, state the unresolved fact, or route to an authorized reviewer. It should not disappear into an unexplained error message.
Exploration learns which strategies work
Strategy quality is initially uncertain. Bandit research studies the tension between exploiting apparently good choices and exploring alternatives to learn their value. Confidence-bound approaches increase the priority of options whose estimates remain uncertain. This supplies a useful conceptual lens for choosing among evaluated routing policies, but it does not grant permission to experiment with arbitrary high-impact actions.
Explore using offline replay, shadow decisions, or bounded low-impact tasks when possible. A strategy that performs well on easy examples may be selected frequently and receive even more easy examples, creating biased feedback. Log the selection context and consider how the data was collected before comparing raw success rates. Otherwise the router may conclude that a cheap strategy is universally best when it simply receives the easiest workload.
Measure the selector as part of the system
Evaluate task quality, latency, resource cost, and unresolved outcomes by scenario class. Compare the adaptive selector with simple baselines such as always retrieve and always use the cheapest valid strategy. A complex router earns its place only if its measured tradeoff improves the intended objective. Revisit assumptions when tools, models, or task distributions change.
The code computes expected utility for two strategies using a fixed belief and a budget. There is no learned policy or online exploration. Its transparent calculation gives a baseline against which a learned router can be checked. Extend it by adding a deterministic calculator with high value only for arithmetic tasks, then inspect how the chosen strategy changes with the belief distribution rather than assuming one architecture should handle every task identically.
Work through the code
A fixed belief weights synthetic state-dependent utilities, then subtracts a cost in the same utility units. The budget is a hard eligibility constraint. Change ambiguous-state probability to explore when retrieval becomes preferable.
belief = {"simple": 0.6, "ambiguous": 0.4}
strategies = [
{"name": "direct", "cost": 1, "values": {"simple": 8, "ambiguous": -6}},
{"name": "retrieve", "cost": 2, "values": {"simple": 7, "ambiguous": 6}},
]
def evaluate(strategy, belief):
value = sum(probability * strategy["values"][state]
for state, probability in belief.items())
return value - strategy["cost"]
budget = 2
eligible = [strategy for strategy in strategies if strategy["cost"] <= budget]
scores = {strategy["name"]: evaluate(strategy, belief) for strategy in eligible}
selected = sorted(scores, key=lambda name: (-scores[name], name))[0]
for name, score in scores.items():
print(name, f"{score:.1f}")
print("selected:", selected)
direct 1.4 retrieve 4.6 selected: retrieve
Run Python snippets locally with the prerequisites named above. The module coding lab runs directly in your browser.
Pause and reason
A strategy has excellent success rate but is mostly assigned easy tasks. Another handles ambiguous tasks and has lower success rate. Why is ranking them by raw success rate misleading?
Check your understanding
An unauthorized strategy has the highest expected utility. What should the selector do?
Your notes
Explain the mechanism in your own words. Add a failure you want to test.