250 lines
6.5 KiB
Markdown
250 lines
6.5 KiB
Markdown
# Parallel Mass Flash Guide
|
|
|
|
Speed up your 32-device deployment from **60-90 minutes** to **15-20 minutes**!
|
|
|
|
## Quick Comparison
|
|
|
|
| Method | Time for 32 Devices | Command |
|
|
|--------|---------------------|---------|
|
|
| **Sequential** | 60-90 minutes | `flash_all.py` |
|
|
| **Parallel (build-and-flash)** | 20-25 minutes | `flash_all_parallel.py --build-parallel 4` |
|
|
| **Parallel (build-then-flash)** | 15-20 minutes | `flash_all_parallel.py --strategy build-then-flash` |
|
|
|
|
## Usage
|
|
|
|
### Method 1: Build-and-Flash (Recommended for Most Users)
|
|
|
|
Builds and flashes devices in batches. Lower memory usage, good balance.
|
|
|
|
```bash
|
|
cd ~/Code/esp32/esp32-iperf
|
|
git checkout mass_deployment
|
|
|
|
# Use default settings (CPU cores - 1 for parallelism)
|
|
python3 flash_all_parallel.py \
|
|
--ssid "YourWiFi" \
|
|
--password "YourPassword" \
|
|
--start-ip 192.168.1.50
|
|
|
|
# Or specify parallel operations
|
|
python3 flash_all_parallel.py \
|
|
--ssid "YourWiFi" \
|
|
--password "YourPassword" \
|
|
--start-ip 192.168.1.50 \
|
|
--build-parallel 4
|
|
```
|
|
|
|
**How it works:** Builds 4 devices at once, flashes them as they complete, then moves to the next batch.
|
|
|
|
**Pros:**
|
|
- Lower memory usage
|
|
- Good parallelism
|
|
- Fails are isolated per device
|
|
|
|
**Time:** ~20-25 minutes for 32 devices
|
|
|
|
### Method 2: Build-Then-Flash (Fastest)
|
|
|
|
Builds all configurations first, then flashes everything in parallel.
|
|
|
|
```bash
|
|
python3 flash_all_parallel.py \
|
|
--ssid "YourWiFi" \
|
|
--password "YourPassword" \
|
|
--start-ip 192.168.1.50 \
|
|
--strategy build-then-flash \
|
|
--build-parallel 4 \
|
|
--flash-parallel 16
|
|
```
|
|
|
|
**How it works:**
|
|
1. Phase 1: Builds all 32 configurations (4 at a time)
|
|
2. Phase 2: Flashes all 32 devices (16 at a time)
|
|
|
|
**Pros:**
|
|
- Fastest method
|
|
- Maximizes flash parallelism
|
|
- Clear phases
|
|
|
|
**Cons:**
|
|
- Uses more disk space temporarily (~2GB during Phase 1)
|
|
- Higher memory usage
|
|
|
|
**Time:** ~15-20 minutes for 32 devices
|
|
|
|
## Options
|
|
|
|
```
|
|
--ssid "SSID" WiFi network name (required)
|
|
--password "PASSWORD" WiFi password (required)
|
|
--start-ip 192.168.1.50 Starting IP address
|
|
--gateway 192.168.1.1 Gateway IP
|
|
--strategy build-and-flash | build-then-flash
|
|
--build-parallel N Parallel builds (default: CPU cores - 1)
|
|
--flash-parallel N Parallel flash ops (default: 8)
|
|
--probe Probe chip types with esptool
|
|
--dry-run Show plan without executing
|
|
```
|
|
|
|
## Hardware Considerations
|
|
|
|
### CPU/Memory Requirements
|
|
|
|
**For build-parallel 4:**
|
|
- CPU: 4+ cores recommended
|
|
- RAM: 8GB minimum, 16GB recommended
|
|
- Disk space: 10GB free
|
|
|
|
**For build-parallel 8:**
|
|
- CPU: 8+ cores
|
|
- RAM: 16GB minimum
|
|
- Disk space: 20GB free
|
|
|
|
### USB Hub Requirements
|
|
|
|
- **Use powered USB hubs** - Each ESP32 draws 200-500mA
|
|
- **USB bandwidth:** USB 2.0 is sufficient (12 Mbps per device for flashing)
|
|
- **Recommended:** Distribute devices across multiple USB controllers
|
|
|
|
## Examples
|
|
|
|
### Conservative (4-core system)
|
|
```bash
|
|
python3 flash_all_parallel.py \
|
|
--ssid "TestNet" \
|
|
--password "password123" \
|
|
--start-ip 192.168.1.50 \
|
|
--build-parallel 2 \
|
|
--flash-parallel 8
|
|
```
|
|
|
|
### Balanced (8-core system)
|
|
```bash
|
|
python3 flash_all_parallel.py \
|
|
--ssid "TestNet" \
|
|
--password "password123" \
|
|
--start-ip 192.168.1.50 \
|
|
--build-parallel 4 \
|
|
--flash-parallel 12
|
|
```
|
|
|
|
### Aggressive (16+ core system)
|
|
```bash
|
|
python3 flash_all_parallel.py \
|
|
--ssid "TestNet" \
|
|
--password "password123" \
|
|
--start-ip 192.168.1.50 \
|
|
--strategy build-then-flash \
|
|
--build-parallel 8 \
|
|
--flash-parallel 16
|
|
```
|
|
|
|
## Monitoring Progress
|
|
|
|
The script shows real-time progress:
|
|
|
|
```
|
|
Phase 1: Building 32 configurations with 4 parallel builds...
|
|
[Device 1] Building for esp32 with IP 192.168.1.50
|
|
[Device 2] Building for esp32 with IP 192.168.1.51
|
|
[Device 3] Building for esp32 with IP 192.168.1.52
|
|
[Device 4] Building for esp32 with IP 192.168.1.53
|
|
[Device 1] ✓ Build complete
|
|
[Device 5] Building for esp32 with IP 192.168.1.54
|
|
[Device 2] ✓ Build complete
|
|
...
|
|
|
|
Phase 2: Flashing 32 devices with 16 parallel operations...
|
|
[Device 1] Flashing /dev/ttyUSB0 -> 192.168.1.50
|
|
[Device 2] Flashing /dev/ttyUSB1 -> 192.168.1.51
|
|
...
|
|
[Device 1] ✓ Flash complete at 192.168.1.50
|
|
[Device 2] ✓ Flash complete at 192.168.1.51
|
|
...
|
|
|
|
DEPLOYMENT SUMMARY
|
|
Successfully deployed: 32/32 devices
|
|
Total time: 892.3 seconds (14.9 minutes)
|
|
Average time per device: 27.9 seconds
|
|
```
|
|
|
|
## Troubleshooting
|
|
|
|
### "Out of memory" during build
|
|
**Solution:** Reduce `--build-parallel`:
|
|
```bash
|
|
python3 flash_all_parallel.py ... --build-parallel 2
|
|
```
|
|
|
|
### Flash timeouts
|
|
**Solution:** Reduce `--flash-parallel`:
|
|
```bash
|
|
python3 flash_all_parallel.py ... --flash-parallel 4
|
|
```
|
|
|
|
### USB hub overload
|
|
**Symptoms:** Devices disconnecting, flash failures
|
|
|
|
**Solution:**
|
|
1. Use powered USB hubs
|
|
2. Distribute devices across multiple hubs
|
|
3. Reduce flash parallelism
|
|
|
|
### Mixed chip types
|
|
**Solution:** Use `--probe` to auto-detect:
|
|
```bash
|
|
python3 flash_all_parallel.py ... --probe
|
|
```
|
|
|
|
## Performance Comparison
|
|
|
|
Testing on a typical 8-core system with 32 devices:
|
|
|
|
| Method | Build Time | Flash Time | Total Time |
|
|
|--------|-----------|------------|------------|
|
|
| Sequential (original) | 48 min | 16 min | 64 min |
|
|
| Parallel (build-parallel 4, build-and-flash) | 15 min | 7 min | 22 min |
|
|
| Parallel (build-parallel 4, build-then-flash) | 12 min | 4 min | 16 min |
|
|
| Parallel (build-parallel 8, build-then-flash) | 8 min | 4 min | 12 min |
|
|
|
|
**Speedup:** 3-5x faster than sequential!
|
|
|
|
## When to Use Each Script
|
|
|
|
### Use `flash_all.py` (Sequential) when:
|
|
- First time setup to verify everything works
|
|
- Limited CPU/memory (< 4GB RAM)
|
|
- Debugging individual device issues
|
|
- Only flashing a few devices (< 5)
|
|
|
|
### Use `flash_all_parallel.py` when:
|
|
- Flashing many devices (10+)
|
|
- You have sufficient resources (8GB+ RAM, 4+ cores)
|
|
- Time is important
|
|
- Production deployment
|
|
|
|
## Best Practices
|
|
|
|
1. **Test first:** Run with `--dry-run` to verify configuration
|
|
2. **Start conservative:** Begin with lower parallelism, increase if stable
|
|
3. **Monitor resources:** Use `htop` to watch CPU/memory during builds
|
|
4. **Staged deployment:** Flash in batches (e.g., 16 at a time) if you have issues
|
|
5. **Verify connectivity:** Ping all devices after flashing
|
|
|
|
## Advanced: Maximum Speed Setup
|
|
|
|
For the absolute fastest deployment on a high-end system:
|
|
|
|
```bash
|
|
# 16-core system, 32GB RAM, multiple USB controllers
|
|
python3 flash_all_parallel.py \
|
|
--ssid "TestNet" \
|
|
--password "password123" \
|
|
--start-ip 192.168.1.50 \
|
|
--strategy build-then-flash \
|
|
--build-parallel 12 \
|
|
--flash-parallel 32
|
|
```
|
|
|
|
With this setup, you could potentially flash all 32 devices in **10-12 minutes**!
|