← All results
GO1 open decision
Curated scenario

Three GOs — and one decision the panel refused to make for you.

Every seat voted GO; the panel converged on path-aware routing via crossed-path concessions — and deliberately left one sequencing decision priced on both sides for the operator.

Review cost$1.27
Session3 rounds · ~5 min model time
Result1 measurement trap the fix would have hidden
Panel
GPTGrokMuse
01

The plan under review

A 15-engineer-hour Friday sprint to restore confidence in a Rust build-graph tool's 12-job GitHub Actions matrix. Forty-one percent of PRs had at least one flaky red and authors waited 18 minutes per run. Five forks were on the table: quarantine flaky suites, deterministically fix the top two, move most testing to a fast PR gate and full merge queue, add process-level nextest retries, or automate triage and reruns. The request required one shippable choice, a concrete watcher-race trace, and a measurable failure invariant.

Scenario context20 PRs/day; 18-minute full matrix; 41% of PRs with a flaky red; watcher FS race 32%, Tokio timeout 24%, nondeterministic graph order 18%, macOS disk pressure 14%; ~1 in 20 flaky failures exposed a real bug on retry.Figures as stated in the reviewed plan — no inferred savings.
02

What changed before ship

Proposed
Landed
Fast-gate-for-all AND full-matrix-for-all
Path-aware routing via crossed paths: fast gate default; full matrix pre-merge for watcher/cache/platform/CI and classifier-uncertain diffs; full matrix always on merge-group
Fsync barrier + deterministic rename order
Sentinel-observed watcher readiness + semantic assertions
Retry loops inside tests
Process-level nextest retries with attempt identity preserved
Retried green counted as success
The ratified rule, signed verbatim; allowlist expiry auto-files the owned issue
Aggregate PR-red rate as scorecard
Per-test first-attempt rate on merge-group + scheduled stress + fast-pass-to-queue-fail as the routing-specific metric
Routing forever
Reversion trigger: classifier false-negative rate >5% over ten days flips back to full matrix, no new debate needed
Five competing options
One 15-hour hybrid budget with a documented degraded mode for either side of the open decision
03

Findings

Ship blockerDefect
Round 1

Durability was mistaken for readiness in the watcher fix

TempDir and fsync do not prove the watcher observed a rename. The test must wait for sentinel-observed readiness and assert the semantic event, not sleep longer or retry inside the assertion loop.

Retries belonged at the process boundary

In-test loops hide which attempt failed and normalize nondeterminism. Nextest retries preserve clean process state and must report PASS_AFTER_RETRY separately.

MaterialDefect
Round 2

Blanket fast gates and full matrices were both dominated by path-aware routing

The 80% of ordinary library changes use the fast gate; watcher, cache, platform, CI, and classifier-uncertain changes run the full matrix pre-merge; every merge-group runs everything.

MaterialProcess gap
Round 2 · Reactions

Invented parameters got policed

A 45-minute re-engagement cost was downgraded from arithmetic input to a bounded sensitivity estimate, alongside three more precision challenges in one round. The routing decision survives on stipulated wait times and coverage, not invented precision.

MaterialMeasurement validity
Round 3

The fix would bias its own success metric

Once 80% of PRs stop running flaky suites pre-merge, aggregate PR-red improves mechanically; '41% became 10%' is invalid across different execution stages and denominators. The routing stratum is observed, so the bias is recoverable by stratification.

Remediation: Report per-test first-attempt rates on merge-group + scheduled stress runs + stratified PR-red; never compare aggregates across execution stages.

MaterialDefect
Round 3

Path classifiers miss semantic dependencies

Transitive Cargo changes, build.rs, proc macros, generated code, feature flags, renames, synthetic merge commits, and base-branch movement can route a sensitive change to the fast path. False negatives need a dedicated metric and automatic fallback.

04

Decision

Ratified ruleUnanimous
A PASS_AFTER_RETRY is always YELLOW and never improves first-attempt metrics; every test must maintain >=99% first-attempt passes over rolling 200; the allowlist expires after 14 days with an owned issue.

The ratified rule was signed verbatim with "no further edits." The open cut-line is recorded as a decision boundary, not chosen — each side carries its own degraded-mode invariant.

05

The open decision

Open decision2-1

If the watcher rewrite overruns Thursday, does routing still ship?

Defer routing — root-cause first2 votes

Exporting the largest known flake class into the merge queue makes innocent PRs pay later (~2.1 innocent late queue failures/day at the stipulated volumes) and corrupts the latency win. Held by Sol and Muse — Muse switched sides with its own arithmetic.

Ship routing first1 vote

The ratified 99% rule forces the watcher race into owned remediation regardless; the reversion trigger flips the system back automatically if the classifier degrades. Held by Grok.

Decision owner · principal
06

Evidence trail