245 lines
17 KiB
Markdown
245 lines
17 KiB
Markdown
# FiWiControl
|
||
|
||
Tools, libraries and statistical software for automating, managing, monitoring and testing Umber Fi‑Wi networks.
|
||
|
||
**Where it runs:** FiWiControl is built to run **on the Umber concentrator** — the Fi‑Wi control plane described in the architecture spec (`**html/Fi-Wi-L4S.html`** in this repo; open that file in a browser for local viewing) — for **lab and customer** automation. Day-to-day development still uses **workstation** installs and **lab rigs** (e.g. Raspberry Pi) as in `**docs/install.md`**.
|
||
|
||
**Naming:** The **Git repository** and checkout directory are **FiWiControl** (mixed case). The **Python distribution** and **import package** are `**fiwicontrol`** (all lowercase, PEP 8) — same project, different casing rules for Git vs Python. Use `**fiwicontrol**` for `pip install` / `import`, not `FiWiControl`.
|
||
|
||
## Intended use — not for life support or safety-critical systems
|
||
|
||
FiWiControl is **network and lab automation software** for Fi‑Wi / concentrator bring-up, inventory, power/USB control, and related measurement workflows. It is **not** a medical device, **not** certified or validated for **life support**, **patient monitoring**, **clinical decision-making**, or other **safety-critical** functions (e.g. ISO 14971 / IEC 62304 / IEC 61508 style assurance). **Do not** rely on it where failure could cause death or serious injury. If you need that class of system, use products and processes designed and regulated for that purpose. See also **`docs/install.md`** → **Security and trust boundaries**.
|
||
|
||
**Architecture & FDIR:** system design and mapping to the Fi‑Wi spec → **`docs/architecture.md`**; fault detection, isolation, recovery, and `fabric_realize` exit codes → **`docs/fdir.md`**. The product architecture narrative remains in **`html/Fi-Wi-L4S.html`** (read-only reference in-repo).
|
||
|
||
This repository ships that distribution (`**fiwicontrol**` on PyPI / `pip`) with import root `**fiwicontrol**`:
|
||
|
||
1. `**fiwicontrol.commands**` — run commands on remote rigs via OpenSSH or `**ush**` (`Command`, `CommandManager`, `ssh_node`). Remote Pi bootstrap: `**python3 -m fiwicontrol.commands <ip> --remote-repo …**` (see `**docs/install.md**`). Must not import `**fiwicontrol.power**`.
|
||
2. `**fiwicontrol.lab**` — USB discovery (Acroname / Monsoon) and `**configs/*.ini**` inventory load + verify. Imports `**fiwicontrol.commands**` for SSH discovery only.
|
||
3. `**fiwicontrol.fronthaul**` — concentrator→RRH **fronthaul link** identity (`**FrontHaul`**: medium, PCIe-style ids, link state); used in **lab and production**.
|
||
4. `**fiwicontrol.telemetry`** — telemetry **schemas split by domain**; today `**telemetry.fronthaul`** exposes `**FrontHaulTelemetry**` (retimer board / PCIe slot / cable / SFP / rails, e.g. **Adnacom Monitor** fields). Future: Wi‑Fi / Fi‑Wi–wide modules alongside `**common`** for shared envelopes when needed.
|
||
5. `**fiwicontrol.radio**` — logical **RRH / AP** aggregate (`**RadioHead`**: Wi‑Fi, hostapd, clients, `**FrontHaul**`, optional `**Power**`); **lab and production**.
|
||
6. `**fiwicontrol.concentrator**` — local **concentrator workstation** snapshot (`**ConcentratorPlatform**`, **`ConcentratorPlatformSnapshot`**: CPU from **`/proc/cpuinfo`**, optional Linux PCIe sysfs / **`lspci`** / DMI). **`scripts/system/dump_concentrator.py`** prints a short human report by default — **`docs/system-test-scripts.md`**. Same “outside the fabric JSON graph” role as **`fiwicontrol.radio`** for RRH facts.
|
||
7. `**fiwicontrol.fabric**` — `**Fabric`**: tuple of `**RadioHead**` (`**rrhs**`), optional concentrator `**ssh_node**`, shared `**Power**`, `**power_lock**`, and `**rrh_power_ports**` (from JSON). `**FabricDefinition**` JSON: discovery fingerprint + RRH bindings (on-disk list key **`rrhs`**); optional lab INI **`[fabric]`** / **`[fabric.rrh.*]`** merged over JSON (`**ini_merge**`). **Builder:** **`python3 -m fiwicontrol.fabric build`** / **`bind`** / **`status`** — **`docs/fabric-builder.md`**. Imports `**fiwicontrol.commands**`, `**fiwicontrol.radio**`, `**fiwicontrol.lab**`; `**Power**` is type-only for a light `**import fiwicontrol.fabric**`.
|
||
8. `**fiwicontrol.power`** — `**Power**` (`.on()` / `.off()` / `.voltage()` / …) via `**AcronamePower**` and `**MonsoonPower**`, plus a small CLI (`**--discovery-json**`, `**--list-power-devices**`, `**--verify-inventory**` with `**-c**`). Re-exports `**fiwicontrol.lab**` discovery/inventory for the power extra. May import `**fiwicontrol.commands**` and `**fiwicontrol.lab**`; the reverse is forbidden.
|
||
9. `**fiwicontrol.flows**` — async **iperf**-driven traffic **flows** over `**ssh`** (remote server/client processes, sampling, histogram / KS tooling). Lives in `**flows/flows.py**`, structured for **Python 3.11** (`**asyncio.timeout`**, `**asyncio.gather**`, `**await iperf_flow.run_traffic**` and related coroutines; sync `**run**` / `**commence**` / … call `**asyncio.run**` only when no loop is already running). Depends on **SciPy**, **NumPy**, **Matplotlib**, and a compatible **iperf** binary on endpoints—use `**pip install -e ".[flows]"`** when you need it; `**import fiwicontrol.flows**` alone stays lightweight (lazy load). Deeper integration with `**ssh_node**` and the Pi 5 **iperf 2** lab story in this README is planned separately.
|
||
10. `**fiwicontrol.spc`** — **SPC** primitives: `**ShewhartControlChart`** (individuals / MR limits) and `**HotellingT2**` (Phase I mean–covariance, *T*² and UCL). Under **`src/fiwicontrol/spc/`** (not **`flows/`**). Optional `**pip install -e ".[spc]"**` (**NumPy**, **SciPy**); `**import fiwicontrol.spc`** is lazy until you reference a class.
|
||
|
||
## Relationship to the Fi‑Wi architecture spec
|
||
|
||
The spec (`**html/Fi-Wi-L4S.html**`) describes the **Fi‑Wi system**: the **Umber concentrator** as the centralized **control, queueing, and time** plane, **RRHs** on the **PCIe / fronthaul** fabric, and the **L4S-oriented** latency model that depends on that split. FiWiControl does not implement the datapath or MAC; it is the **concentrator-resident (and dev-workstation) Python layer** that **manages and validates** the deployed system around that architecture.
|
||
|
||
Concretely, it is how we keep **what we think is connected** aligned with **what is actually cabled and powered**: **INI inventory and verification** (`**fiwicontrol.lab`**, `**fiwicontrol.power**`), **scripted remote work** over the same **SSH / `ush`** paths we use in production (`**fiwicontrol.commands**`), **programmable power and USB** so bring-up and **fronthaul hot-plug** sequences are repeatable (PCIe is the fronthaul medium today), **fronthaul telemetry** (`**fiwicontrol.telemetry`**) for host retimer readouts, **async iperf flows** for load and measurement against RRHs or lab stand-ins (`**fiwicontrol.flows`**), and **SPC** when we need statistical discipline across long campaigns (`**fiwicontrol.spc`**). Read the spec for **why** the topology and timing model look the way they do; read this repo and `**docs/`** for **how** we install, verify, and operate it day to day.
|
||
|
||
## Near-term focus: fronthaul hot-swap testing (PCIe today)
|
||
|
||
A **first concrete goal** for this stack is to support **fronthaul hot-swap (hot-plug) testing** in the lab—**PCIe** for now: controlled **remove / restore** of the link, predictable **enumeration** and driver behavior, and **repeatable** runs across builds and rigs.
|
||
|
||
That work fails if the bench is informal—wrong port, wrong power path, or no shared picture of what was connected when. FiWiControl targets that gap by combining **documented inventory** (INI + `**--verify-inventory`**), **remote automation** (`**ssh_node`**, `**python3 -m fiwicontrol.commands**` to bring up rigs), and **programmable power / USB paths** (`**fiwicontrol.power`**) so sequences are **scripted and checkable** before the fronthaul harness runs. Example async harness skeleton: `**scripts/system/pcie_hotswap_harness.py`** (setup: `**docs/pcie-hotswap-setup.md`**); how to write similar scripts: `**docs/system-test-scripts.md**`. `**fiwicontrol.spc**` is aimed at **many hot-plug cycles**—spotting drift in failure rates or side metrics—not only general Fi‑Wi bring-up (see `**docs/spc.md`**).
|
||
|
||
## Lab fleet: Raspberry Pi 5 and iperf 2 (planned)
|
||
|
||
We expect a **fleet of Raspberry Pi 5** boards to act as **controllers and actuators** in the lab: remote `**fiwicontrol`** install, discovery over SSH, PCIe / Fi‑Wi harness hosts (docs already assume Pi-class rigs). Those Pi 5s will run **iperf 2** as **synthetic traffic** endpoints—controlled load and measurement during Fi‑Wi and bring-up experiments, not one-off manual runs. **None of this is automated in-repo yet** beyond what you can already drive by hand over `**ssh_node`**; first-class orchestration and reporting are roadmap.
|
||
|
||
## ESP32, IEEE 802.11, and advanced telemetry (planned)
|
||
|
||
Separately from the Pi 5 `**iperf**` plane, we plan **ESP32**-based nodes for **advanced telemetry**, including rich use of **IEEE 802.11 (Wi‑Fi)**: airlink statistics, channel / PHY-adjacent metrics, retries, timing, and correlation with harness and inventory state. The goal is lab- and field-grade **802.11 telemetry**, not only GPIO or serial counters. **This stack is not implemented yet** in `**fiwicontrol`**; today’s packages are **commands**, **lab**, **fronthaul**, **telemetry** (fronthaul slice), **radio**, **concentrator**, **fabric**, **power**, **flows**, and **spc** above.
|
||
|
||
## `ush` expansion (planned)
|
||
|
||
Expanded **ush** orchestration alongside OpenSSH—not only the `**ush`** transport already wired through `**ssh_node**`, but first-class tooling for benches that standardize on **ush**. Roadmap only beyond today’s transport path.
|
||
|
||
## Product differentiation
|
||
|
||
We expect **advanced telemetry** (including **802.11-centric**, e.g. **ESP32**, data) and **advanced use of iperf 2 on Raspberry Pi 5** hosts—patterns **tied to rig state, inventory, and SPC** (`**fiwicontrol.spc`**), not ad hoc throughput screenshots—to help **differentiate** Umber Fi‑Wi products in the market.
|
||
|
||
## Statistical process control
|
||
|
||
**In tree:** `**fiwicontrol.spc`** — `**ShewhartControlChart**` and `**HotellingT2**` (Phase I fit, *T*², Phase II UCL). Optional `**pip install -e ".[spc]"**`; overview `**docs/spc.md**`.
|
||
|
||
**Planned:** tighter coupling with inventory / harness automation, campaign-scale reporting, and richer charting beyond these primitives.
|
||
|
||
**Layout**
|
||
|
||
```
|
||
FiWiControl/
|
||
├── LICENSE
|
||
├── README.md
|
||
├── pyproject.toml
|
||
├── scripts/
|
||
│ ├── setup_pi_power.sh
|
||
│ └── system/
|
||
├── html/
|
||
│ └── Fi-Wi-L4S.html
|
||
├── docs/
|
||
│ ├── install.md
|
||
│ ├── architecture.md
|
||
│ ├── fdir.md
|
||
│ ├── node-control-asyncio-design.md
|
||
│ ├── power-control-and-inventory.md
|
||
│ ├── flows.md
|
||
│ ├── spc.md
|
||
│ ├── system-test-scripts.md
|
||
│ ├── fabric-builder.md
|
||
│ └── pcie-hotswap-setup.md
|
||
├── src/
|
||
│ └── fiwicontrol/
|
||
│ ├── commands/
|
||
│ │ ├── __init__.py
|
||
│ │ ├── __main__.py
|
||
│ │ ├── node_control.py
|
||
│ │ └── remote_setup.py
|
||
│ ├── fronthaul/
|
||
│ │ ├── __init__.py
|
||
│ │ └── link.py
|
||
│ ├── lab/
|
||
│ │ ├── __init__.py
|
||
│ │ ├── discovery.py
|
||
│ │ ├── inventory_config.py
|
||
│ │ └── inventory_verify.py
|
||
│ ├── radio/
|
||
│ │ ├── __init__.py
|
||
│ │ └── radiohead.py
|
||
│ ├── concentrator/
|
||
│ │ ├── __init__.py
|
||
│ │ └── host.py
|
||
│ ├── fabric/
|
||
│ │ ├── __init__.py
|
||
│ │ ├── __main__.py
|
||
│ │ ├── builder.py
|
||
│ │ ├── fabric.py
|
||
│ │ ├── fingerprint.py
|
||
│ │ ├── ini_merge.py
|
||
│ │ └── definition_from_ini.py
|
||
│ ├── telemetry/
|
||
│ │ ├── __init__.py
|
||
│ │ ├── common.py
|
||
│ │ └── fronthaul.py
|
||
│ ├── power/
|
||
│ │ ├── __init__.py
|
||
│ │ ├── __main__.py
|
||
│ │ ├── acroname.py
|
||
│ │ ├── control.py
|
||
│ │ └── monsoon.py
|
||
│ ├── flows/
|
||
│ │ ├── __init__.py
|
||
│ │ └── flows.py
|
||
│ └── spc/
|
||
│ ├── __init__.py
|
||
│ ├── shewhart.py
|
||
│ └── hotelling.py
|
||
└── tests/
|
||
```
|
||
|
||
## Requirements
|
||
|
||
- Python **3.11+**
|
||
- For `**sshtype="ssh"`**: **passwordless** SSH to `**root@<host>`** (non-interactive `ssh`; see `docs/node-control-asyncio-design.md`).
|
||
- **Security / trust model** (SSH targets, lab JSON/INI, CI): **`docs/install.md`** → **Security and trust boundaries**.
|
||
|
||
## Install
|
||
|
||
**First-time / new machine:** follow `**docs/install.md`** → **“Bring up systems (new workstation + lab rig)”**, then the **Workstation** and **Remote host setup** sections.
|
||
|
||
Full guide (workstation, **remote host setup** for the Pi, `**pip install -e`**, `**PYTHONPATH=src**`, `**scripts/setup_pi_power.sh**`): `**docs/install.md**`.
|
||
|
||
Minimal editable install from the repo root:
|
||
|
||
```bash
|
||
cd ~/Code/FiWiControl
|
||
python3 -m pip install -e ".[dev]"
|
||
# optional: power / USB discovery (Acroname, Monsoon)
|
||
python3 -m pip install -e ".[power]"
|
||
# optional: async iperf flows / SPC (see docs/flows.md, docs/spc.md)
|
||
python3 -m pip install -e ".[flows]"
|
||
python3 -m pip install -e ".[spc]"
|
||
```
|
||
|
||
Imports:
|
||
|
||
```python
|
||
from fiwicontrol.commands import ssh_node, Command, CommandManager
|
||
```
|
||
|
||
## Tests
|
||
|
||
`**tests/**` — `**pytest**` package tests (CI-oriented, import smoke, gated remote tests, etc.).
|
||
|
||
`**scripts/system/**` — **manual or long-running harness scripts** that exercise real hardware (e.g. fronthaul **PCIe hot-swap** campaigns). Not part of the default `**pytest`** tree; run explicitly when the bench is wired. **Fabric INI / JSON:** `**docs/fabric-builder.md`**, **`scripts/system/fabric_realize.py`** (uses **`fiwicontrol.fabric.definition_from_ini`**). **Local concentrator snapshot:** `**scripts/system/dump_concentrator.py`** — **`docs/system-test-scripts.md`**. **PCIe hot-swap harness walkthrough:** `**docs/pcie-hotswap-setup.md`** (example: `**scripts/system/pcie_hotswap_harness.py**`).
|
||
|
||
After install, use **`docs/power-control-and-inventory.md`** (**Lab INI layout**, **Verification checklist**).
|
||
|
||
Commands, example `**pytest`** output, and `**unittest**` entry points for `**node_control**`: `**docs/node-control-asyncio-design.md**` → section **“Running tests”** (top-level `##` heading near the top of the file).
|
||
|
||
**Whole suite** (from repo root; `**pyproject.toml`** adds `**src**` to `**PYTHONPATH**` for pytest):
|
||
|
||
```bash
|
||
cd ~/Code/FiWiControl
|
||
python3 -m pytest tests/ -q
|
||
```
|
||
|
||
Example summary when opt-in tests are **not** enabled (**`FIWI_VERIFY_POWER_INI`** unset, **`FIWI_RUN_REMOTE_TESTS`** unset — several tests **skipped**; exact counts depend on the tree):
|
||
|
||
```text
|
||
.........................................sssssss...............ss.. [100%]
|
||
58 passed, 9 skipped in 0.38s
|
||
```
|
||
|
||
Run a **single** test by id (replace with a name from `**python3 -m pytest tests/<file>.py --collect-only -q`**):
|
||
|
||
```bash
|
||
python3 -m pytest tests/test_package_layout.py::test_import_subpackages -v
|
||
```
|
||
|
||
Layout / import smoke (no network):
|
||
|
||
```bash
|
||
cd ~/Code/FiWiControl
|
||
python3 -m pytest tests/test_package_layout.py -q
|
||
```
|
||
|
||
Remote integration (opt-in: **`FIWI_RUN_REMOTE_TESTS=1`**, plus **`FIWI_REMOTE_IP`** and key auth):
|
||
|
||
```bash
|
||
cd ~/Code/FiWiControl
|
||
FIWI_RUN_REMOTE_TESTS=1 FIWI_REMOTE_IP=192.168.1.39 python3 -m pytest tests/test_node_control.py -q
|
||
```
|
||
|
||
Power / USB inventory (needs `**brainstem**` and `**pyserial**`, e.g. `**pip install -e ".[power]"**`): see `**docs/power-control-and-inventory.md**`. Lab layout defaults to `**configs/default.ini**` (override with `**-c**` or `**FIWI_LAB_INI**`).
|
||
|
||
```bash
|
||
cd ~/Code/FiWiControl
|
||
python3 -m pip install -e ".[power]"
|
||
python3 -m fiwicontrol.power --verify-inventory
|
||
# or: python3 -m fiwicontrol.power -c configs/default.ini --verify-inventory
|
||
```
|
||
|
||
The same check from **pytest** (set `**FIWI_VERIFY_POWER_INI=1`** or pytest **skips** the live INI test):
|
||
|
||
```bash
|
||
FIWI_VERIFY_POWER_INI=1 python3 -m pytest -q tests/test_inventory_verify_live.py
|
||
```
|
||
|
||
Remote **package** sanity (imports + `**python3 -m fiwicontrol.power --discovery-json`**) for the host in `**FIWI_REMOTE_IP**` (default `**192.168.1.39**`). **Opt-in** under pytest (`**FIWI_RUN_REMOTE_TESTS=1**`); running `**python3 tests/test_remote_power_dependencies.py**` directly enables that automatically, same pattern as `**tests/test_node_control.py**`.
|
||
|
||
```bash
|
||
FIWI_RUN_REMOTE_TESTS=1 python3 -m pytest -q tests/test_remote_power_dependencies.py
|
||
```
|
||
|
||
Example when SSH and the rig are configured (`**-v**`; line spacing may vary):
|
||
|
||
```text
|
||
tests/test_remote_power_dependencies.py::TestRemotePowerDependencies::test_remote_imports_fiwicontrol_power_brainstem_serial PASSED [ 50%]
|
||
tests/test_remote_power_dependencies.py::TestRemotePowerDependencies::test_remote_power_module_discovery_json PASSED [100%]
|
||
|
||
============================== 2 passed in 0.65s ==============================
|
||
```
|
||
|
||
## Remote (Gitea)
|
||
|
||
```bash
|
||
git remote add origin "https://git.umbernetworks.com/rjmcmahon/FiWiControl.git"
|
||
git push -u origin main
|
||
```
|
||
|
||
Use a Gitea **personal access token** as the HTTPS password if prompted. |