Evidence Twin

The problem worth solving

What could make it stand out

System architecture

Data and reproducibility

Start with a synthetic benchmark

• W3C Semantic Sensor Network Ontology
• LBL Buildings Library: Chiller Examples

Baselines you must beat

1. Threshold rules over temperature and flow, with the same observation budget and an explicit unknown state.
2. Text RAG over flattened telemetry summaries and notes, using the same language model and token budget as the proposed planner.
3. Typed evidence retrieval without the physical residual, isolating whether the simulator adds value beyond cleaner metadata.

Measure the claim

Targets below are proposed success criteria. No results have been achieved on your behalf.

Experiments and ablations

1. Run paired evaluations on identical episodes while removing the ontology gate, physical residual, and abstention policy one at a time. Keep retrieval size and model configuration fixed so gains cannot be attributed to larger contexts.
2. Sweep missingness from 0% to 40% and observation delays across the freshness threshold. Plot coverage and unsupported explanations together; inspect whether the planner becomes appropriately uncertain as evidence degrades.
3. Hold out simultaneous load increase and sensor bias until final testing. Compare the next-measurement policy with a fixed checklist and a random read policy, measuring evidence gained per tool call.

Your execution plan

0 of 12 deliverables checked. Tick a deliverable after recording evidence in your project repository.

Working seed

Run this small, deterministic core first. It demonstrates the central mechanism. The complete system, experiments, and deployment are your capstone work.

"""Evidence Twin: a synthetic cooling-loop evidence gate, not a device model."""
from dataclasses import dataclass, replace
import json


@dataclass(frozen=True)
class Observation:
    id: str
    component: str
    property: str
    value: float
    unit: str
    minute: int


ONTOLOGY = {
    "flow": ("cooling_loop", "kg/s"),
    "inlet": ("cooling_loop", "degC"),
    "outlet": ("cooling_loop", "degC"),
    "heat": ("cooling_loop", "kW"),
}


def simulate(flow, heat=10.0, minute=100):
    # Water heat capacity is an illustrative constant; no equipment-specific calibration.
    values = {"flow": flow, "inlet": 18.0,
              "outlet": 18.0 + heat / (4.18 * flow), "heat": heat}
    return [Observation(k, "cooling_loop", k, v, ONTOLOGY[k][1], minute)
            for k, v in values.items()]


def investigate(observations, now=100):
    grouped = {}
    for o in observations:
        if (o.component, o.unit) != ONTOLOGY.get(o.property):
            return {"status": "abstain", "reason": "ontology_or_unit_error"}
        if not 0 <= now - o.minute <= 5:
            return {"status": "abstain", "reason": "stale_or_future_evidence"}
        grouped.setdefault(o.property, []).append(o)
    if set(grouped) != set(ONTOLOGY):
        return {"status": "abstain", "reason": "missing_property"}
    if any(max(o.value for o in group) - min(o.value for o in group) > 0.1
           for group in grouped.values()):
        return {"status": "abstain", "reason": "contradictory_evidence"}
    v = {k: group[0].value for k, group in grouped.items()}
    if v["flow"] <= 0:
        return {"status": "abstain", "reason": "invalid_flow"}
    residual = abs(v["heat"] - 4.18 * v["flow"] * (v["outlet"] - v["inlet"]))
    if residual > 0.5:
        return {"status": "abstain", "reason": "energy_balance_mismatch"}
    candidate = "low_flow_investigation" if v["flow"] < 0.5 else "no_simulated_alert"
    return {"status": candidate, "residual_kW": round(residual, 6),
            "evidence": sorted(o.id for o in observations)}


def main():
    normal = simulate(1.0)
    restricted = simulate(0.25)
    stale = [replace(o, minute=90) for o in restricted]
    conflicting = restricted + [replace(restricted[0], id="flow_backup", value=0.8)]
    outputs = [investigate(x) for x in (normal, restricted, stale, conflicting)]
    assert [x["status"] for x in outputs] == [
        "no_simulated_alert", "low_flow_investigation", "abstain", "abstain"]
    assert investigate([replace(normal[0], unit="litres/min")] + normal[1:])["status"] == "abstain"
    print(json.dumps(outputs, sort_keys=True, indent=2))


if __name__ == "__main__":
    main()

Failure modes to investigate

• Toy physics may make the task artificially easy. Hold out parameter regimes, include unmodeled disturbances, and describe the residual as a consistency signal rather than proof of a physical cause.
• Latent fault labels can leak through note templates or file names. Randomize surface wording and audit every tool-visible field before evaluating explanation quality.
• A convincing simulator can imply more validity than its evidence supports. Label all assets fictional and keep the assistant limited to simulated engineering investigation with no equipment control or servicing instructions.

Your demo, moment by moment

1. Open a fictional cooling-loop episode with a rising outlet temperature. Ask what the evidence supports and show the component and observation identifiers behind the first candidate.
2. Introduce a stale flow sensor and a contradictory fresh backup reading. The assistant retracts its confident answer and names the missing evidence instead of inventing a cause.
3. Request one permitted fresh observation. Reveal how the candidate set changes and show the energy-balance residual alongside its simplifying assumptions.
4. Finish with the frozen risk-coverage comparison and one failure case, explaining what the measured improvement can and cannot establish.

Write the resume bullet after the experiment

Replace every placeholder with your actual measurements. Keep the dataset size, baseline, and evaluation conditions available for interview questions.

A research extension

Related work to challenge your idea