FiWiManager/docs/fiwi-test-authoring.md

Fi-Wi framework — authoring test cases and automation

This document is for people who write repeatable checks, calibration procedures, or scripts around Fi-Wi—not for end users who only run panel calibrate interactively. For shell commands, see fiwi-cli.md. For code structure, see fiwi-design.md.

The repository does not yet ship a pytest suite for fiwi/; the patterns below apply whether you drive subprocess against fiwi.py or import fiwi as a library.

Choose a control style

1. Shell / subprocess — Easiest to run exactly what operators run. Parse JSON from stdout for machine-readable commands (calibrate-ports-json, lsusb-lines-json, wlan-info-json). Stable for CI-style wrappers.
2. Python imports — Use FiWiHarness, SshNode, FiberRadioPort, and fiwi.diag_log for tighter integration, error handling, and async (ainvoke_capture, alog_dmesg, and so on).

Mix both: for example import SshNode to run remote JSON probes while keeping local steps in the shell.

Stable machine-readable commands

These are intended for automation (same argv on local or remote via --ssh):

Command	Typical use
calibrate-ports-json	Ordered [[hub, port], …] for calibration or power walks.
lsusb-lines-json	USB topology snapshot on the host where the command runs.
wlan-info-json	Wireless NIC metadata for map fill-in over SSH.
status	Human tables; less ideal for parsing than JSON helpers.

When testing remote behavior, run the same subcommand with fiwi.py --ssh user@host … so the environment matches production.

Fiber map as test fixture

• Keep a golden fiber_map.json (or merge from fiber_map.example.json) under version control for non-interactive tests.
• For SSH-mapped fibers, ensure the map on the workstation points at the DUT, and the map on the DUT marks those ports as local to avoid forwarding loops.
• Use power fiber-port <id> on|off in scripts when the goal is “same as operator” routing through the map.

Deferred remote work (overlap)

If your automation fires many SSH captures in parallel (similar to panel calibrate):

• Set FIWI_REMOTE_DEFER=1 or pass --async so deferred calls return RemoteCallHandle or asyncio.Task instead of blocking immediately inside each call.
• Join with .result() on handles or await on tasks before asserting outcomes.

Avoid assuming synchronous blocking remote calls if defer is enabled.

Async library example (sketch)

import asyncio
from fiwi import SshNode, alog_hardware_snapshot, get_diag_log

async def remote_baseline(host: str):
    log = get_diag_log()
    ssh = SshNode.parse(host)
    await alog_hardware_snapshot(log, ssh=ssh, caption=f"baseline {host}")
    code, out, err = await ssh.ainvoke_capture(["wlan-info-json"], timeout=60)
    assert code == 0, (code, err)
    # parse JSON from `out` …

asyncio.run(remote_baseline("pi@192.168.1.50"))

Use ainvoke_capture / araw_ssh when you are already inside an async test harness; use sync invoke_capture / raw_ssh for straight-line scripts.

Diagnostic log in tests

• get_diag_log() returns a process-wide DiagLog; call log.clear() at the start of a test case if you need isolation without subprocess boundaries.
• alog_hardware_snapshot appends note + dmesg + lspci in a fixed order (captures may run concurrently, but log order is stable).
• KernelDumpEvent is reserved for future kdump/vmcore steps; today you can append kernel_dump_event(...) manually to exercise JSONL export.

dump_jsonl(path) writes one JSON object per line suitable for attaching to bug reports.

Good practices

• Timeouts: Always pass explicit timeouts on remote calls that your test framework allows to fail slowly.
• Idempotence: Prefer power-off at the end of destructive sequences; document required starting hub state.
• Secrets: Do not commit remote_ssh.env with passwords; prefer keys and FIWI_SSH_OPTS.
• Assertions: When checking JSON from invoke_capture, assert exit code and parse errors separately so failures show stderr from the remote fiwi.py.

Where to add real unit tests later

A future tests/ package can import fiwi modules with BrainStem mocked or skipped, and subprocess tests can point fiwi.paths.configure at a temporary directory with a throwaway fiber_map.json. This document stays valid: same JSON commands and public API surface.