79 lines
4.5 KiB
Markdown
79 lines
4.5 KiB
Markdown
# Fi-Wi framework — authoring test cases and automation
|
|
|
|
This document is for people who write **repeatable checks**, **calibration procedures**, or **scripts** around Fi-Wi—not for end users who only run `panel calibrate` interactively. For shell commands, see [fiwi-cli.md](fiwi-cli.md). For code structure, see [fiwi-design.md](fiwi-design.md).
|
|
|
|
The repository does not yet ship a pytest suite for `fiwi/`; the patterns below apply whether you drive **`subprocess`** against `fiwi.py` or import **`fiwi`** as a library.
|
|
|
|
## Choose a control style
|
|
|
|
1. **Shell / subprocess** — Easiest to run exactly what operators run. Parse JSON from stdout for machine-readable commands (`calibrate-ports-json`, `lsusb-lines-json`, `wlan-info-json`). Stable for CI-style wrappers.
|
|
2. **Python imports** — Use `FiWiHarness`, `SshNode`, `FiberRadioPort`, and `fiwi.diag_log` for tighter integration, error handling, and async (`ainvoke_capture`, `alog_dmesg`, and so on).
|
|
|
|
Mix both: for example import `SshNode` to run remote JSON probes while keeping local steps in the shell.
|
|
|
|
## Stable machine-readable commands
|
|
|
|
These are intended for automation (same argv on local or remote via `--ssh`):
|
|
|
|
| Command | Typical use |
|
|
|---------|-------------|
|
|
| `calibrate-ports-json` | Ordered `[[hub, port], …]` for calibration or power walks. |
|
|
| `lsusb-lines-json` | USB topology snapshot on the host where the command runs. |
|
|
| `wlan-info-json` | Wireless NIC metadata for map fill-in over SSH. |
|
|
| `status` | Human tables; less ideal for parsing than JSON helpers. |
|
|
|
|
When testing **remote** behavior, run the same subcommand with `fiwi.py --ssh user@host …` so the environment matches production.
|
|
|
|
## Fiber map as test fixture
|
|
|
|
- Keep a **golden** `fiber_map.json` (or merge from `fiber_map.example.json`) under version control for non-interactive tests.
|
|
- For SSH-mapped fibers, ensure the map on the **workstation** points at the DUT, and the map on the **DUT** marks those ports as local to avoid forwarding loops.
|
|
- Use **`power fiber-port <id> on|off`** in scripts when the goal is “same as operator” routing through the map.
|
|
|
|
## Deferred remote work (overlap)
|
|
|
|
If your automation fires **many SSH captures** in parallel (similar to `panel calibrate`):
|
|
|
|
- Set **`FIWI_REMOTE_DEFER=1`** or pass **`--async`** so deferred calls return **`RemoteCallHandle`** or **`asyncio.Task`** instead of blocking immediately inside each call.
|
|
- Join with **`.result()`** on handles or **`await`** on tasks before asserting outcomes.
|
|
|
|
Avoid assuming synchronous blocking remote calls if defer is enabled.
|
|
|
|
## Async library example (sketch)
|
|
|
|
```python
|
|
import asyncio
|
|
from fiwi import SshNode, alog_hardware_snapshot, get_diag_log
|
|
|
|
async def remote_baseline(host: str):
|
|
log = get_diag_log()
|
|
ssh = SshNode.parse(host)
|
|
await alog_hardware_snapshot(log, ssh=ssh, caption=f"baseline {host}")
|
|
code, out, err = await ssh.ainvoke_capture(["wlan-info-json"], timeout=60)
|
|
assert code == 0, (code, err)
|
|
# parse JSON from `out` …
|
|
|
|
asyncio.run(remote_baseline("pi@192.168.1.50"))
|
|
```
|
|
|
|
Use **`ainvoke_capture`** / **`araw_ssh`** when you are already inside an async test harness; use sync **`invoke_capture`** / **`raw_ssh`** for straight-line scripts.
|
|
|
|
## Diagnostic log in tests
|
|
|
|
- **`get_diag_log()`** returns a process-wide **`DiagLog`**; call **`log.clear()`** at the start of a test case if you need isolation without subprocess boundaries.
|
|
- **`alog_hardware_snapshot`** appends note + **dmesg** + **lspci** in a fixed order (captures may run concurrently, but log order is stable).
|
|
- **`KernelDumpEvent`** is reserved for future kdump/vmcore steps; today you can append **`kernel_dump_event(...)`** manually to exercise JSONL export.
|
|
|
|
**`dump_jsonl(path)`** writes one JSON object per line suitable for attaching to bug reports.
|
|
|
|
## Good practices
|
|
|
|
- **Timeouts**: Always pass explicit timeouts on remote calls that your test framework allows to fail slowly.
|
|
- **Idempotence**: Prefer power-off at the end of destructive sequences; document required starting hub state.
|
|
- **Secrets**: Do not commit `remote_ssh.env` with passwords; prefer keys and `FIWI_SSH_OPTS`.
|
|
- **Assertions**: When checking JSON from `invoke_capture`, assert **exit code** and parse errors separately so failures show stderr from the remote `fiwi.py`.
|
|
|
|
## Where to add real unit tests later
|
|
|
|
A future **`tests/`** package can import **`fiwi`** modules with BrainStem mocked or skipped, and subprocess tests can point `fiwi.paths.configure` at a temporary directory with a throwaway `fiber_map.json`. This document stays valid: same JSON commands and public API surface.
|