Repair: decision.consistent_with_truth on port_alert

Repair: decision.consistent_with_truth@1.0

Originating issue: #1

Witness before

Scenario port_alert, seed 20261026, minimal schedule [{op: delay, target: {event_id: correction-001}, params: {by_ns: 300000000000}}]. Witness: correction-001, ais-006, patrol-001. duty_officer chose escalate but truth as of 1500s gives monitor. Before digest (delayed branch): sha256:6ede77e3a634ba2faf8c0ee8f15b551b68a1640b27445d2c6167934fca5990d4 (root: sha256:d16ac4392733589d5dbc12c893be62d64a28e73184c8eb8ae0a2bbb18354dd17).

Mechanism

The duty_officer rule counted a credible sighting (sighting-003, a radar ghost) that was still open to correction, so a correction arriving after the 1500s decision made the decision stale.

Change

  • New scenarios/regressions/decision.consistent_with_truth_late_correction.yaml: identical to port_alert.yaml (injects, roles, delivery, transitions, invariants, decision time and window) except the escalate_or_monitor rule, which now counts only credible sightings settled for a grace window: count(kind='sighting', payload.confidence>=0.6, event_time_ns<=1100s, not_corrected) >= 2. 1100s = decision time 1500s minus a 400s grace window, which exceeds the 300s delay of the minimal schedule. The fault is kept, not removed.
  • New tests/test_regression_decision_late_correction.py (10 tests after review; 5 original plus 5 for the variant, see Review follow-up): PASS on the minimal schedule, PASS on the no-fault baseline, determinism, the same schedule still FAILs on the unrepaired port_alert.yaml (so the repair is not vacuous), and the repaired scenario differs from the original only in that rule.
  • No code under worldline/ was changed. No test was deleted, skipped or xfailed.

Regression scenario

scenarios/regressions/decision.consistent_with_truth_late_correction.yaml (scenario id port_alert_regression_late_correction).

After

  • Minimal schedule on the regression scenario: decision.consistent_with_truth PASS, witness []. Digest: sha256:b23fe2ec7c547919eadec1a95ec07c07857407bc79845911b202cbd6ff618653.
  • No-fault baseline on the regression scenario: PASS. Digest: sha256:0a019836d13d827ef1df727bc52ff9f43c8a91ce3b5dea3b91d5aed1a2db0709.
  • uv run pytest -q: 168 passed (158 before + 5 original + 5 added for the review).
  • worldline explore --budget 200 --seed 42 re-run on the regression scenario: the same 146 branches (guided + random) as on port_alert; no branch has a new FAIL against the port_alert run; 8 branch-invariant FAILs of decision.consistent_with_truth disappear (guided 12 to 4, random 3 to 3). Some decision FAILs remain on other schedules (for example duplicate sighting-002 combined with other faults); this MR repairs the schedule in issue #1 only.

Limitations, for the reviewer

  • scenarios/port_alert.yaml is deliberately NOT edited. Its header documents the seeded latent defect, and 11 existing tests (test_explorer, test_genome, test_worlds, test_cli) plus the genome/ddmin pipeline depend on it failing under this schedule. Editing it makes them fail, and deleting or skipping them is forbidden. As a result the CI explore job, which uses port_alert.yaml, still finds the FAIL. Promoting the rule to the main scenario needs a maintainer decision about those tests.
  • In the original inject set the repaired rule CANNOT escalate (see Review follow-up). That is a behaviour trade-off and a modelling choice for the maintainer to accept; it is acceptable only together with the reachability variant.

Review follow-up: the repaired rule cannot escalate in the original inject set

Stated plainly: with the 1100s cutoff and the original injects, the credible sightings (confidence>=0.6) are sighting-002 (1000s), sighting-003 (1200s) and sighting-005 (2600s). Only sighting-002 is at or before 1100s, so the count is at most 1 and the >= 2 threshold is unreachable. escalate_or_monitor always yields monitor there, with or without correction-001. The cutoff works by excluding exactly the sighting the correction targets. PASS on decision.consistent_with_truth_late_correction.yaml therefore only shows the rule is constant on that inject set; it does not show the rule is robust to late corrections in general.

This is acceptable only together with a reachability variant, added in this update:

  • scenarios/regressions/decision.consistent_with_truth_late_correction_escalate.yaml (id port_alert_regression_late_correction_escalate): identical rule, roles, delivery, decision time and window, same incident (sighting-003 is still the ghost that correction-001 downgrades), plus one inject, sighting-006 (1050s, confidence 0.75). Now two credible, uncorrected, settled sightings (sighting-002, sighting-006) exist, and the decision IS escalate.
  • New tests in tests/test_regression_decision_late_correction.py: the original inject set cannot escalate (documents the limitation, both baseline and minimal schedule); the variant escalates and PASSes with no fault; the variant escalates and PASSes under the minimal schedule (correction-001 verifiably absent from known_event_ids); the variant is deterministic; the variant differs from the regression scenario only by sighting-006 and has identical actors/rules.
  • Kept unchanged: the minimal-schedule PASS test, the baseline PASS test, and the test that the unrepaired port_alert.yaml still FAILs with witness correction-001, ais-006, patrol-001.
  • Variant digests: no fault sha256:df1a70942e1e0764d7be502a7f3ae09eee647c8ec345fca41f8a1e8685f24836 (escalate, PASS); minimal schedule sha256:e5cf52473c2baf474e01939cf24afdf0fa2b976b72cc91a45d6f81331bcb8fa9 (escalate, PASS). Regression scenario digests are unchanged (b23fe2ec... minimal, 0a019836... baseline).
  • Remaining limitation: the repaired rule depends on sightings settled by 1100s, so on the variant a schedule that drops sighting-006 makes the actor choose monitor where truth says escalate (FAIL, witness sighting-006, ais-006, patrol-001). This repair addresses only the late-correction schedule of issue #1. Explore (budget 200, seed 42) on the variant shows decision.consistent_with_truth branch FAILs guided 10 / random 8 (port_alert: 12 / 3); these are other fault schedules, not the repaired one.
  • uv run pytest -q: 168 passed. Only scenarios/regressions/ and tests/ changed in this update.

Evidence kind: EMPIRICALLY_VALIDATED (claim ceiling EMPIRICALLY_VALIDATED; bounded horizon 1h, synthetic scenario, declared delivery policies; not ATTESTED, the kernel decides).

Untouched by contract: invariants/, policies/, worldline/evidence/, kernel CI jobs.

Edited by WORLDLINE Repair

Merge request reports

Loading
Loading