diff --git a/doc/AI_strategy_detector.html b/doc/AI_strategy_detector.html new file mode 100644 index 0000000..de1a6e1 --- /dev/null +++ b/doc/AI_strategy_detector.html @@ -0,0 +1,141 @@ + + + + + + AI Strategy: Wi-Fi Collapse Detection + + + + +
+ +

AI Strategy: Wi-Fi Collapse Detection

+

Technical Brief for Development Team

+

This document outlines the machine learning pipeline for detecting Wi-Fi "Collapse" events (Hidden Node, Saturation, Interference) using ESP32 sensors. We utilize an LLM-Assisted Weak Supervision approach to overcome the lack of labeled training data.

+ +
+

🚀 The Core Concept

+

We cannot hard-code thresholds because RF environments vary wildly. Instead, we use Gemini (LLM) as a "Senior Network Engineer" to label raw logs, and then train a fast, lightweight model (Random Forest/XGBoost) to mimic that decision logic in real-time.

+
+ +

Phase 1: Firmware Data Engineering

+

The ESP32 firmware is the data generator. It must output time-series features suitable for ML, not just human-readable logs.

+ +

1. Feature Vector (The CSV)

+

Every 1 second, the firmware writes a CSV line to the internal storage partition. This is our feature set:

+ + +

2. Storage Strategy

+

We use a custom partition table to allocate ~5-13MB for LittleFS/SPIFFS. NVS is strictly for config. This allows 24-72 hours of continuous logging.

+ +
+ +

Phase 2: The "Weak Supervision" Pipeline

+

We solve the "Cold Start Problem" (having data but no labels) by using Generative AI.

+ +

Step A: Context-Aware Capture

+

Technicians capture logs in known scenarios. The filename conveys the context:

+ + +

Step B: LLM Labeling (Gemini)

+

We feed the raw CSV chunks to Gemini via API with a prompt that injects domain knowledge:

+
+ "Act as a Wi-Fi expert. Analyze this CSV log. This data represents a Hidden Node scenario. Look for periods where AvgNAV is high (>10ms) but AvgPHY remains normal, yet Retries spike. Label each timestamp row as 'Normal', 'Congestion', or 'Collapse'." +
+

Result: A "Silver Standard" labeled dataset ready for supervised learning.

+ +
+ +

Phase 3: Training & Inference

+

Gemini is too slow for real-time packet analysis. We train a classical model for the actual work.

+ +

1. Model Selection

+

Algorithm: Random Forest Classifier or XGBoost.

+ + +

2. Runtime Inference Architecture

+

Deployment Target: Linux Gateway / Edge Server.

+
    +
  1. 1 ESP32 streams CSV lines via UDP (Broadcast/Unicast) to Linux.
    2. +
  2. 2 Python script listens on UDP port.
    3. +
  3. 3 Script loads pre-trained model.pkl.
    4. +
  4. 4 Incoming CSV -> Feature Vector -> model.predict().
    5. +
  5. 5 If Collapse_Prob > 0.8 for 3 consecutive seconds -> TRIGGER ALERT.
    6. +
+ +
+ + + diff --git a/doc/AI_strategy_detector.md b/doc/AI_strategy_detector.md new file mode 100644 index 0000000..b33df4c --- /dev/null +++ b/doc/AI_strategy_detector.md @@ -0,0 +1,94 @@ +# AI Strategy: Wi-Fi Collapse Detection + +**Target Audience:** Development Team +**Objective:** Build a real-time detection engine using Weak Supervision. + +## 🏗️ Architecture Overview + +The system uses a two-stage AI approach: +1. **Teacher (Offline):** Gemini (LLM) analyzes historical logs to create "Ground Truth" labels. +2. **Student (Real-Time):** A lightweight Random Forest model runs on the Linux gateway for sub-second inference. + + + +[Image of AI Pipeline Diagram] + + +--- + +## Phase 1: Firmware Data Engineering + +The ESP32 firmware is responsible for **Feature Extraction**. It must aggregate raw packet events into 1-second statistical snapshots. + +### The Feature Vector +The firmware writes the following struct to flash/UDP every 1000ms: + +| Feature | Type | Description | +| :--- | :--- | :--- | +| `timestamp` | `uint32` | Epoch or Uptime. | +| `retry_rate` | `float` | % of frames with Retry bit set. | +| `avg_nav` | `uint16` | Average Network Allocation Vector (microseconds). | +| `max_nav` | `uint16` | Maximum contention window observed. | +| `collisions` | `uint8` | Count of inferred collisions (High NAV + Retry). | +| `avg_phy` | `uint16` | Average PHY Rate (Mbps). | +| `mismatches` | `uint8` | Count of duration anomalies (Spoofing/Bugs). | + +**Storage:** * Do **NOT** use NVS. Use a custom partition table with **LittleFS** or **SPIFFS**. +* Capacity: ~1.1 days (8MB chip) to ~3 days (16MB chip) at 1Hz sampling. + +--- + +## Phase 2: The "Weak Supervision" Pipeline + +We lack labeled data. We cannot manually look at 100,000 rows of logs and say "That's a collapse." We use Gemini to do this. + +### 1. Data Collection (Contextual) +Technicians run `async_mass_deploy` and collect logs in specific, controlled environments: +* **Clean:** Basement, Faraday cage. +* **Noisy:** Microwave running, Baby monitor active. +* **Hostile:** Hidden Node simulation (2 ESPs blasting UDP, hidden from each other). + +### 2. The Labeling Loop (Python + Gemini API) +We will write a script (`label_data.py`) that: +1. Reads the raw CSVs. +2. Injects a **System Prompt** based on the filename (Context). +3. Asks Gemini to output a classification column: `0` (Normal), `1` (Interference), `2` (Collapse). + +> **Prompt Logic:** "In a hidden node scenario, we expect high Retries and low Throughput, but standard NAV values might look normal because the nodes can't hear each other. Label rows matching this pattern as 'Collapse'." + +--- + +## Phase 3: Runtime Inference (Linux) + +We do not run the LLM live. We run a compiled Scikit-Learn model. + +### Training +* **Input:** The Gemini-labeled CSVs. +* **Model:** Random Forest Classifier (Robust, interpretable feature importance). +* **Artifact:** `wifi_collapse_model.pkl` + +### The Real-Time Loop +The Linux monitoring service performs the following loop: + +```python +import socket +import joblib +import pandas as pd + +# Load Model +model = joblib.load('wifi_collapse_model.pkl') + +# Listen for ESP32 Data +sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM) +sock.bind(('0.0.0.0', 5000)) + +while True: + data, addr = sock.recvfrom(1024) + # Parse CSV -> DataFrame + features = parse_packet(data) + + # Inference (< 1ms) + prediction = model.predict(features) + + if prediction == "COLLAPSE": + trigger_alert(addr, "Network Collapse Detected") \ No newline at end of file