FiWiControl/README.md

FiWiControl

Tools, libraries and statistical software for automating, managing, monitoring and testing Umber Fi‑Wi networks.

Where it runs: FiWiControl is built to run on the Umber concentrator — the Fi‑Wi control plane described in the architecture spec (**html/Fi-Wi-L4S.html** in this repo; open that file in a browser for local viewing) — for lab and customer automation. Day-to-day development still uses workstation installs and lab rigs (e.g. Raspberry Pi) as in **docs/install.md**.

Naming: The Git repository and checkout directory are FiWiControl (mixed case). The Python distribution and import package are **fiwicontrol** (all lowercase, PEP 8) — same project, different casing rules for Git vs Python. Use **fiwicontrol** for pip install / import, not FiWiControl.

Intended use — not for life support or safety-critical systems

FiWiControl is network and lab automation software for Fi‑Wi / concentrator bring-up, inventory, power/USB control, and related measurement workflows. It is not a medical device, not certified or validated for life support, patient monitoring, clinical decision-making, or other safety-critical functions (e.g. ISO 14971 / IEC 62304 / IEC 61508 style assurance). Do not rely on it where failure could cause death or serious injury. If you need that class of system, use products and processes designed and regulated for that purpose. See also docs/install.md → Security and trust boundaries.

Architecture & FDIR: system design and mapping to the Fi‑Wi spec → docs/architecture.md; fault detection, isolation, recovery, and fabric_realize exit codes → docs/fdir.md. The product architecture narrative remains in html/Fi-Wi-L4S.html (read-only reference in-repo).

This repository ships that distribution (**fiwicontrol** on PyPI / pip) with import root **fiwicontrol**:

1. **fiwicontrol.commands** — run commands on remote rigs via OpenSSH or **ush** (Command, CommandManager, ssh_node). Remote Pi bootstrap: **python3 -m fiwicontrol.commands <ip> --remote-repo …** (see **docs/install.md**). Must not import **fiwicontrol.power**.
2. **fiwicontrol.lab** — USB discovery (Acroname / Monsoon) and **configs/*.ini** inventory load + verify. Imports **fiwicontrol.commands** for SSH discovery only.
3. **fiwicontrol.fronthaul** — concentrator→RRH fronthaul link identity (**FrontHaul**: medium, PCIe-style ids, link state); used in lab and production.
4. **fiwicontrol.telemetry** — telemetry schemas split by domain; today **telemetry.fronthaul** exposes **FrontHaulTelemetry** (retimer board / PCIe slot / cable / SFP / rails, e.g. Adnacom Monitor fields). Future: Wi‑Fi / Fi‑Wi–wide modules alongside **common** for shared envelopes when needed.
5. **fiwicontrol.radio** — logical RRH / AP aggregate (**RadioHead**: Wi‑Fi, hostapd, clients, **FrontHaul**, optional **Power**); lab and production.
6. **fiwicontrol.concentrator** — local concentrator workstation snapshot (**ConcentratorPlatform**, ConcentratorPlatformSnapshot: CPU from /proc/cpuinfo, optional Linux PCIe sysfs / lspci / DMI). scripts/system/dump_concentrator.py prints a short human report by default — docs/system-test-scripts.md. Same “outside the fabric JSON graph” role as fiwicontrol.radio for RRH facts.
7. **fiwicontrol.fabric** — **Fabric**: tuple of **RadioHead** (**rrhs**), optional concentrator **ssh_node**, shared **Power**, **power_lock**, and **rrh_power_ports** (from JSON). **FabricDefinition** JSON: discovery fingerprint + RRH bindings (on-disk list key rrhs); optional lab INI [fabric] / [fabric.rrh.*] merged over JSON (**ini_merge**). Builder: python3 -m fiwicontrol.fabric build / bind / status — docs/fabric-builder.md. Imports **fiwicontrol.commands**, **fiwicontrol.radio**, **fiwicontrol.lab**; **Power** is type-only for a light **import fiwicontrol.fabric**.
8. **fiwicontrol.power** — **Power** (.on() / .off() / .voltage() / …) via **AcronamePower** and **MonsoonPower**, plus a small CLI (**--discovery-json**, **--list-power-devices**, **--verify-inventory** with **-c**). Re-exports **fiwicontrol.lab** discovery/inventory for the power extra. May import **fiwicontrol.commands** and **fiwicontrol.lab**; the reverse is forbidden.
9. **fiwicontrol.flows** — async iperf-driven traffic flows over **ssh** (remote server/client processes, sampling, histogram / KS tooling). Lives in **flows/flows.py**, structured for Python 3.11 (**asyncio.timeout, **asyncio.gather**, **await iperf_flow.run_traffic** and related coroutines; sync **run** / **commence** / … call **asyncio.run** only when no loop is already running). Depends on SciPy, NumPy, Matplotlib, and a compatible iperf binary on endpoints—use **pip install -e ".[flows]" when you need it; **import fiwicontrol.flows** alone stays lightweight (lazy load). Deeper integration with **ssh_node** and the Pi 5 iperf 2 lab story in this README is planned separately.
10. **fiwicontrol.spc** — SPC primitives: **ShewhartControlChart** (individuals / MR limits) and **HotellingT2** (Phase I mean–covariance, T² and UCL). Under src/fiwicontrol/spc/ (not flows/). Optional **pip install -e ".[spc]"** (NumPy, SciPy); **import fiwicontrol.spc** is lazy until you reference a class.

Relationship to the Fi‑Wi architecture spec

The spec (**html/Fi-Wi-L4S.html**) describes the Fi‑Wi system: the Umber concentrator as the centralized control, queueing, and time plane, RRHs on the PCIe / fronthaul fabric, and the L4S-oriented latency model that depends on that split. FiWiControl does not implement the datapath or MAC; it is the concentrator-resident (and dev-workstation) Python layer that manages and validates the deployed system around that architecture.

Concretely, it is how we keep what we think is connected aligned with what is actually cabled and powered: INI inventory and verification (**fiwicontrol.lab, **fiwicontrol.power**), scripted remote work over the same SSH / ush paths we use in production (**fiwicontrol.commands**), programmable power and USB so bring-up and fronthaul hot-plug sequences are repeatable (PCIe is the fronthaul medium today), fronthaul telemetry (**fiwicontrol.telemetry) for host retimer readouts, async iperf flows for load and measurement against RRHs or lab stand-ins (**fiwicontrol.flows), and SPC when we need statistical discipline across long campaigns (**fiwicontrol.spc). Read the spec for why the topology and timing model look the way they do; read this repo and **docs/** for how we install, verify, and operate it day to day.

Near-term focus: fronthaul hot-swap testing (PCIe today)

A first concrete goal for this stack is to support fronthaul hot-swap (hot-plug) testing in the lab—PCIe for now: controlled remove / restore of the link, predictable enumeration and driver behavior, and repeatable runs across builds and rigs.

That work fails if the bench is informal—wrong port, wrong power path, or no shared picture of what was connected when. FiWiControl targets that gap by combining documented inventory (INI + **--verify-inventory), remote automation (**ssh_node, **python3 -m fiwicontrol.commands** to bring up rigs), and programmable power / USB paths (**fiwicontrol.power) so sequences are scripted and checkable before the fronthaul harness runs. Example async harness skeleton: **scripts/system/pcie_hotswap_harness.py (setup: **docs/pcie-hotswap-setup.md); how to write similar scripts: **docs/system-test-scripts.md**. **fiwicontrol.spc** is aimed at many hot-plug cycles—spotting drift in failure rates or side metrics—not only general Fi‑Wi bring-up (see **docs/spc.md).

Lab fleet: Raspberry Pi 5 and iperf 2 (planned)

We expect a fleet of Raspberry Pi 5 boards to act as controllers and actuators in the lab: remote **fiwicontrol** install, discovery over SSH, PCIe / Fi‑Wi harness hosts (docs already assume Pi-class rigs). Those Pi 5s will run iperf 2 as synthetic traffic endpoints—controlled load and measurement during Fi‑Wi and bring-up experiments, not one-off manual runs. None of this is automated in-repo yet beyond what you can already drive by hand over **ssh_node**; first-class orchestration and reporting are roadmap.

ESP32, IEEE 802.11, and advanced telemetry (planned)

Separately from the Pi 5 **iperf** plane, we plan ESP32-based nodes for advanced telemetry, including rich use of IEEE 802.11 (Wi‑Fi): airlink statistics, channel / PHY-adjacent metrics, retries, timing, and correlation with harness and inventory state. The goal is lab- and field-grade 802.11 telemetry, not only GPIO or serial counters. This stack is not implemented yet in **fiwicontrol**; today’s packages are commands, lab, fronthaul, telemetry (fronthaul slice), radio, concentrator, fabric, power, flows, and spc above.

ush expansion (planned)

Expanded ush orchestration alongside OpenSSH—not only the **ush** transport already wired through **ssh_node**, but first-class tooling for benches that standardize on ush. Roadmap only beyond today’s transport path.

Product differentiation

We expect advanced telemetry (including 802.11-centric, e.g. ESP32, data) and advanced use of iperf 2 on Raspberry Pi 5 hosts—patterns tied to rig state, inventory, and SPC (**fiwicontrol.spc**), not ad hoc throughput screenshots—to help differentiate Umber Fi‑Wi products in the market.

Statistical process control

In tree: **fiwicontrol.spc** — **ShewhartControlChart** and **HotellingT2** (Phase I fit, T², Phase II UCL). Optional **pip install -e ".[spc]"**; overview **docs/spc.md**.

Planned: tighter coupling with inventory / harness automation, campaign-scale reporting, and richer charting beyond these primitives.

Layout

FiWiControl/
├── LICENSE
├── README.md
├── pyproject.toml
├── scripts/
│   ├── setup_pi_power.sh
│   └── system/
├── html/
│   └── Fi-Wi-L4S.html
├── docs/
│   ├── install.md
│   ├── architecture.md
│   ├── fdir.md
│   ├── node-control-asyncio-design.md
│   ├── power-control-and-inventory.md
│   ├── flows.md
│   ├── spc.md
│   ├── system-test-scripts.md
│   ├── fabric-builder.md
│   └── pcie-hotswap-setup.md
├── src/
│   └── fiwicontrol/
│       ├── commands/
│       │   ├── __init__.py
│       │   ├── __main__.py
│       │   ├── node_control.py
│       │   └── remote_setup.py
│       ├── fronthaul/
│       │   ├── __init__.py
│       │   └── link.py
│       ├── lab/
│       │   ├── __init__.py
│       │   ├── discovery.py
│       │   ├── inventory_config.py
│       │   └── inventory_verify.py
│       ├── radio/
│       │   ├── __init__.py
│       │   └── radiohead.py
│       ├── concentrator/
│       │   ├── __init__.py
│       │   └── host.py
│       ├── fabric/
│       │   ├── __init__.py
│       │   ├── __main__.py
│       │   ├── builder.py
│       │   ├── fabric.py
│       │   ├── fingerprint.py
│       │   ├── ini_merge.py
│       │   └── definition_from_ini.py
│       ├── telemetry/
│       │   ├── __init__.py
│       │   ├── common.py
│       │   └── fronthaul.py
│       ├── power/
│       │   ├── __init__.py
│       │   ├── __main__.py
│       │   ├── acroname.py
│       │   ├── control.py
│       │   └── monsoon.py
│       ├── flows/
│       │   ├── __init__.py
│       │   └── flows.py
│       └── spc/
│           ├── __init__.py
│           ├── shewhart.py
│           └── hotelling.py
└── tests/

Requirements

• Python 3.11+
• For **sshtype="ssh": passwordless SSH to **root@<host> (non-interactive ssh; see docs/node-control-asyncio-design.md).
• Security / trust model (SSH targets, lab JSON/INI, CI): docs/install.md → Security and trust boundaries.

Install

First-time / new machine: follow **docs/install.md** → “Bring up systems (new workstation + lab rig)”, then the Workstation and Remote host setup sections.

Full guide (workstation, remote host setup for the Pi, **pip install -e**, **PYTHONPATH=src**, **scripts/setup_pi_power.sh**): **docs/install.md**.

Minimal editable install from the repo root:

cd ~/Code/FiWiControl
python3 -m pip install -e ".[dev]"
# optional: power / USB discovery (Acroname, Monsoon)
python3 -m pip install -e ".[power]"
# optional: async iperf flows / SPC (see docs/flows.md, docs/spc.md)
python3 -m pip install -e ".[flows]"
python3 -m pip install -e ".[spc]"

Imports:

from fiwicontrol.commands import ssh_node, Command, CommandManager

Tests

**tests/** — **pytest** package tests (CI-oriented, import smoke, gated remote tests, etc.).

**scripts/system/** — manual or long-running harness scripts that exercise real hardware (e.g. fronthaul PCIe hot-swap campaigns). Not part of the default **pytest** tree; run explicitly when the bench is wired. Fabric INI / JSON: **docs/fabric-builder.md, scripts/system/fabric_realize.py (uses fiwicontrol.fabric.definition_from_ini). Local concentrator snapshot: **scripts/system/dump_concentrator.py — docs/system-test-scripts.md. PCIe hot-swap harness walkthrough: **docs/pcie-hotswap-setup.md** (example: **scripts/system/pcie_hotswap_harness.py**).

After install, use docs/power-control-and-inventory.md (Lab INI layout, Verification checklist).

Commands, example **pytest** output, and **unittest** entry points for **node_control**: **docs/node-control-asyncio-design.md** → section “Running tests” (top-level ## heading near the top of the file).

Whole suite (from repo root; **pyproject.toml** adds **src** to **PYTHONPATH** for pytest):

cd ~/Code/FiWiControl
python3 -m pytest tests/ -q

Example summary when opt-in tests are not enabled (FIWI_VERIFY_POWER_INI unset, FIWI_RUN_REMOTE_TESTS unset — several tests skipped; exact counts depend on the tree):

.........................................sssssss...............ss..      [100%]
58 passed, 9 skipped in 0.38s

Run a single test by id (replace with a name from **python3 -m pytest tests/<file>.py --collect-only -q**):

python3 -m pytest tests/test_package_layout.py::test_import_subpackages -v

Layout / import smoke (no network):

cd ~/Code/FiWiControl
python3 -m pytest tests/test_package_layout.py -q

Remote integration (opt-in: FIWI_RUN_REMOTE_TESTS=1, plus FIWI_REMOTE_IP and key auth):

cd ~/Code/FiWiControl
FIWI_RUN_REMOTE_TESTS=1 FIWI_REMOTE_IP=192.168.1.39 python3 -m pytest tests/test_node_control.py -q

Power / USB inventory (needs **brainstem** and **pyserial**, e.g. **pip install -e ".[power]"**): see **docs/power-control-and-inventory.md**. Lab layout defaults to **configs/default.ini** (override with **-c** or **FIWI_LAB_INI**).

cd ~/Code/FiWiControl
python3 -m pip install -e ".[power]"
python3 -m fiwicontrol.power --verify-inventory
# or: python3 -m fiwicontrol.power -c configs/default.ini --verify-inventory

The same check from pytest (set **FIWI_VERIFY_POWER_INI=1** or pytest skips the live INI test):

FIWI_VERIFY_POWER_INI=1 python3 -m pytest -q tests/test_inventory_verify_live.py

Remote package sanity (imports + **python3 -m fiwicontrol.power --discovery-json**) for the host in **FIWI_REMOTE_IP** (default **192.168.1.39**). Opt-in under pytest (**FIWI_RUN_REMOTE_TESTS=1**); running **python3 tests/test_remote_power_dependencies.py** directly enables that automatically, same pattern as **tests/test_node_control.py**.

FIWI_RUN_REMOTE_TESTS=1 python3 -m pytest -q tests/test_remote_power_dependencies.py

Example when SSH and the rig are configured (**-v**; line spacing may vary):

tests/test_remote_power_dependencies.py::TestRemotePowerDependencies::test_remote_imports_fiwicontrol_power_brainstem_serial PASSED [ 50%]
tests/test_remote_power_dependencies.py::TestRemotePowerDependencies::test_remote_power_module_discovery_json PASSED [100%]

============================== 2 passed in 0.65s ==============================

Remote (Gitea)

git remote add origin "https://git.umbernetworks.com/rjmcmahon/FiWiControl.git"
git push -u origin main

Use a Gitea personal access token as the HTTPS password if prompted.