> ## Documentation Index > Fetch the complete documentation index at: https://www.plungeai.com/docs/llms.txt > Use this file to discover all available pages before exploring further. # Testing and gates > Everything in oceancode is verified by one script — ./check.sh — and a handful of opt-in lanes on it. This page covers what those lanes are, and then the one test surface a terminal UI needs that unit tests cannot give it: the interface probe, which drives the real TUI over a pseudo-terminal … Everything in `oceancode` is verified by one script — `./check.sh` — and a handful of opt-in lanes on it. This page covers what those lanes are, and then the one test surface a terminal UI needs that unit tests cannot give it: the **interface probe**, which drives the real TUI over a pseudo-terminal and proves the interface never freezes. ## The lanes ```bash export PATH="$HOME/.cargo/bin:$PATH" ./check.sh # fmt · clippy -D warnings · build · nextest · goldens · doc + gate checks E2E=1 ./check.sh # + real-engine PTY tests, chaos, crash-recovery drills, the interface probe PERF=1 ./check.sh # + divan benches and the budget gates COVERAGE=1 ./check.sh # + llvm-cov with the zero-covered-handler tripwire MUTANTS=1 ./check.sh # + cargo-mutants over the risk-ranked modules ``` The default lane dials no model provider and touches no network. Every checker prints a sentinel as its **last line** and exits non-zero on failure; the full roster is in `scripts/README.md` (`apps/ocean-coding-agent-rust/scripts/README.md` in the repository). No lane spends money — the only live-provider scripts (`scripts/probes/live-*.sh`) are run by hand, never by `check.sh`. ## The interface probe A TUI can pass every unit test and still lock up the instant a slash command awaits the whole turn on the event loop — the exact freeze that shipped and was fixed on 2026-09-15. Unit tests run the update function in-process; they cannot catch a real event loop that has stopped turning keystrokes into frames. The interface probe can, because it is black-box: it spawns the real `oceancode` binary behind a pseudo-terminal, renders its screen with `pyte`, feeds it real keys, and asserts on what a person would see. It costs nothing to run. The binary is wired to a **fake Anthropic provider** with a fake key, an isolated `HOME`, and a loopback `--listen` port; the owner's real provider keys never enter a probe run. ```bash bash scripts/probe/run-probe.sh # every scenario bash scripts/probe/run-probe.sh -k slash # only scenarios whose id (or file) contains "slash" bash scripts/probe/run-probe.sh -k baseline # one file's scenarios (-k matches the module name too) bash scripts/probe/run-probe.sh -j 4 # run 4 at a time, each on its own ports bash scripts/probe/run-probe.sh --list # module:id of every discovered scenario bash scripts/probe/run-probe.sh --keep # keep each scenario's temp HOME to debug a failure ``` The last line is the sentinel: ``` INTERFACE_PROBE_OK scenarios=10 passed=10 xfail=0 # exit 0 INTERFACE_PROBE_FAIL scenarios=10 failed=1 mcp_roundtrip # exit 1 ``` `run-probe.sh` bootstraps its own Python venv at `target/probe-venv` (`pexpect` + `pyte`, idempotent and offline-tolerant), resolves the binary from `$OCEANCODE_BIN` — building `target/debug/oceancode` with `cargo build -p ocean-cli` when it is unset — and runs the harness. It is wired into the `E2E=1` lane after the crash-recovery drills, so a freeze is a red build. ### The liveness oracle Every scenario ends on the same universal check, `t.assert_live()`, which the harness calls for you after the scenario body returns: > type a unique nonce into the composer, assert the composer echoes it within 1.5 s, then erase > it. A frozen event loop cannot echo a keystroke, so that one assertion is the proof the interface is still alive — no matter what the scenario did before it. Because of this, a scenario only has to assert its own behavior and leave the composer on screen; the liveness check is free. ### Driving the TUI interactively (for agents) The same probe also runs as a **persistent session**, so an agent can drive the app step by step from separate Bash calls — open it, look, type, look again — instead of only inside a batch scenario. `scripts/probe/session.py` holds one PTY and `pyte` screen in a daemon; a client speaks to it over a unix socket: ``` session.py open --app oceancode|pager|cli|cmd [--cols 120 --rows 40] [--keep] [-- cmd args…] session.py screen | styled | history | send | key [k…] | paste session.py wait [timeout] | wait_re [timeout] | resize | alive | requests | evidence | close ``` `oceancode` / `pager` boot the real binary on the fake provider in a fresh `HOME` (the scenarios' isolation, but colours kept so `styled` can read them); `cli` runs the `cli/` REPL against deployed Studio. Wait for the app's **ready marker** before driving — `❯` for the pager (`oceancode` boots Ocean pager too since the native TUI was removed on 2026-10-05), `Ask anything` for the REPL — and **wait for a marker, never sleep** (`wait` polls the screen every 150 ms). ```bash S="target/probe-venv/bin/python scripts/probe/session.py" $S open pager --app pager # fresh HOME, fake provider, PTY 120×40 $S pager wait '❯' 40 # block on the ready marker $S pager send 'hello'; $S pager key enter $S pager wait 'ok: hello' 40 $S pager styled 38 # one row as (fg, bg, bold) runs — colour/bold asserts $S pager close # kill app + fixture, remove the temp HOME ``` End every interactive check the way a scenario does: type a unique nonce and confirm it echoes in the composer — the liveness oracle, by hand. `bash scripts/probe/session-selftest.sh` drives all three surfaces from separate processes and prints `PROBE_SESSION_OK apps=` as its last line; `--no-cli` runs the hermetic surfaces only (the pager), which is how the `E2E=1` lane wires it in. ### Adding a scenario Drop a function named `scenario_(t, fx)` into any file under `scripts/probe/scenarios/`. New files and functions are discovered automatically — you never register anything. `t` is a live `Tui`, `fx` a `Fixture` on the fake provider. Raise `AssertionError` (or call `t.fail(msg)`) to fail; a screen excerpt is attached automatically. ```python # scripts/probe/scenarios/my_feature.py def scenario_models_open_and_esc(t, fx): """/models opens the model picker and Esc returns to a live composer.""" t.send("/models") t.key("enter", 0.6) if "MODEL" not in t.text(): t.fail("/models did not open the model picker") t.key("esc", 0.4) if t.PLACEHOLDER not in t.text(): t.fail("Esc did not return to the composer") # the harness calls t.assert_live() after this returns ``` **Driving the screen (`t`):** | Call | What it does | |---|---| | `t.send(text)` | type text into whatever has focus | | `t.key(name)` | one key: `enter`, `esc`, `tab`, `backtab`, `backspace`, `up`/`down`/`left`/`right`, `home`/`end`, `pgup`/`pgdn`, `ctrl-c`, `ctrl-d`, `ctrl-u`, `shift-enter`, `f1`–`f12` | | `t.paste(text)` | bracketed paste | | `t.resize(cols, rows)` | resize the terminal (delivers `SIGWINCH`) | | `t.wait_for(needle, timeout)` | wait until `needle` (a substring or a compiled regex) is on screen; returns `True`/`False` | | `t.text()` / `t.line(n)` | the whole screen / one row, as text | | `t.alive()` / `t.pid` | is the process still running / its pid | | `t.snapshot(label)` | record a labelled screen for later | | `t.fail(msg)` | raise with a screen excerpt attached | | `t.assert_live()` | the liveness oracle (also usable mid-scenario) | | `t.assert_version_bar()` | assert a `vX.Y.Z` build stamp is in the status bar | | `t.PLACEHOLDER` | the composer's empty-prompt placeholder text | **Steering the provider (`fx`):** the fake provider picks its reply from MAGIC WORDS in the last user message — put one in the prompt text your scenario sends: | Word | Reply | |---|---| | *(none)* | one text block `ok: ` | | `SLOW` | the same text paced over 30 deltas (a visibly streaming turn) | | `ERR500` | HTTP 500 with an Anthropic error body | | `HANG` | an open stream that never answers (abort it from the TUI) | | `TOOLUSE` | a `tool_use` for the `bash` tool on the first turn (the permission dialog appears), text on the follow-up | | `WORKFLOW` | a `tool_use` for the `workflow` tool launching the built-in two-step `pipeline` (its sub-agents get default replies and the run completes) — the pager's Workflows-dock regression guard | | `GOAL` | a `tool_use` launching the built-in `goal` supervisor; its planner/skeptic `output_schema` requests are answered with valid JSON so the run converges (Plan → Work → Verify) | | `MD` | a markdown-heavy reply (heading, table, fenced code, wide CJK + emoji) | | `TASKAGENT` | a `tool_use` for the `task` tool on the first turn: one sub-agent whose prompt carries `DOCKCHILD`; text on the follow-up | | `DOCKCHILD` | the sub-agent's reply, 30 words paced over ~12 s, so a probe reads the dock while it runs (`pager_dock_agents`) | Read what the binary actually sent with `fx.requests()`, `fx.last_request()`, `fx.user_texts()` (the last-user text of every request), and `fx.base_url` / `fx.port`. Each scenario is fully isolated: its own `HOME`, its own fixture project directory carrying its own `.claude/commands/{long,short}.md`, its own two free ports, and its own fresh fixture process. Scenarios never share state, so `-j N` runs them in parallel safely. ### xfail policy A scenario for a **known, tracked** defect is decorated `@xfail("REASON-ID")`: ```python from scenarios.xfail import xfail @xfail("OCA-1234-slash-refusal-toast") def scenario_unknown_slash_is_refused_visibly(t, fx): ... ``` It is reported as `XFAIL ` and does **not** fail the run; a scenario that is xfail-marked but unexpectedly passes is reported `XPASS` and counts as a pass. Use `@xfail` only for a defect that has a tracking id — never to silence a flaky or half-finished scenario. The moment the defect is fixed, remove the decorator so a regression turns the scenario red again.