Compare commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
aee1d3e0ef | ||
|
|
75b6a21f3c | ||
|
|
259eda6663 | ||
|
|
f42ede5fa4 |
@@ -5,6 +5,44 @@ All notable changes to this project will be documented in this file.
|
||||
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
|
||||
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
|
||||
|
||||
## [1.9.10] - 2026-09-20
|
||||
|
||||
### Fixed
|
||||
|
||||
- **WiFi password lost after captive-portal setup, no reconnect after reboot** (#12):
|
||||
after a successful connect via SDK-cached credentials (Try 1) or the WiFiManager
|
||||
portal, only the SSID was written to EEPROM. On the next boot `config.password`
|
||||
was empty, so `setupWiFi()` and the reconnect loop called `WiFi.begin(ssid)` as
|
||||
if the network were open and failed with `WL_WRONG_PASSWORD`. v1.9.6 masked this
|
||||
through the bare `WiFi.begin()` fallback in the retry loop, which 02834ba (v1.9.7)
|
||||
replaced with `WiFi.begin(config.ssid)`. Now the password is read back with
|
||||
`WiFi.psk()` and saved next to the SSID in all three places that sync the SSID.
|
||||
Devices already stuck recover by entering the password once in the web UI.
|
||||
|
||||
## [1.9.9] - 2026-05-19
|
||||
|
||||
### Fixed
|
||||
|
||||
- **Factory reset atomicity**: cleared WiFi credentials are now committed BEFORE
|
||||
the reset counter is zeroed. If power fails between the two writes, the device
|
||||
still boots into WiFiManager AP on next start (cleared creds win over stale counter)
|
||||
instead of being stuck in an inconsistent state
|
||||
- **`isValidSSID` rejected single-char SSIDs**: length-1 networks like "A" were
|
||||
incorrectly flagged as all-same garbage. Now allowed (per 802.11 spec)
|
||||
- **`ntp_interval` config changes ignored until reboot**: `NTPClient` is constructed
|
||||
in `setup()` with this value and never re-initialized. Now triggers `needsRestart`
|
||||
when the interval actually changes
|
||||
- **Open WiFi (no password) failed to connect at setup** (M2): Try-2 block in
|
||||
`setupWiFi()` required both ssid and password; now falls back to `WiFi.begin(ssid)`
|
||||
when password is empty
|
||||
- **Unseeded PRNG, identical dissolve pattern every boot** (M4): `randomSeed()`
|
||||
now called with `ESP.getChipId() ^ micros()` — varies between devices and boots
|
||||
|
||||
### Removed
|
||||
|
||||
- **Dead `NTPState` enum values** (M3): `NTP_WAITING`, `NTP_SUCCESS`, `NTP_FAILED`
|
||||
were defined but never assigned. Removed to reduce code surface area
|
||||
|
||||
## [1.9.8] - 2026-05-19
|
||||
|
||||
### Fixed
|
||||
|
||||
+33
-5
@@ -73,14 +73,42 @@ Feature requests are welcome! Please include:
|
||||
arduino-cli compile --fqbn esp8266:esp8266:generic firmware/weather_clock
|
||||
```
|
||||
|
||||
### Testing
|
||||
### Flashing
|
||||
|
||||
```bash
|
||||
# Upload via FTDI (first time)
|
||||
arduino-cli upload -p /dev/cu.usbserial* --fqbn esp8266:esp8266:generic
|
||||
|
||||
# Upload via OTA (subsequent)
|
||||
# OTA upload (preferred, when device is on the network)
|
||||
curl -u admin:admin -F "file=@build/*.bin" http://192.168.x.x/update
|
||||
|
||||
# Initial flash via FTDI (3.3V! ESP-01S in socket — no soldering)
|
||||
# 1. Pull ESP-01S from socket on TJ-56-654 PCB
|
||||
# 2. Connect: FTDI 3V3→3V3, GND→GND, TX↔RX crossed, GND→GPIO0
|
||||
# 3. Power on with GPIO0 grounded → bootloader mode
|
||||
esptool.py --port /dev/cu.usbserial-0001 --baud 115200 write_flash \
|
||||
--flash_size 1MB --flash_mode dout 0x0 firmware.bin
|
||||
```
|
||||
|
||||
### Running the test suite
|
||||
|
||||
Hardware-in-the-loop tests verify functionality and resilience:
|
||||
|
||||
```bash
|
||||
python3 tests/test_device.py <device-ip>
|
||||
```
|
||||
|
||||
73 test cases cover REST API, config validation, fuzz testing, and heap stability.
|
||||
Safe to run repeatedly — validation rejects garbage, only WiFi changes reboot the device.
|
||||
|
||||
### Recovery (bricked device)
|
||||
|
||||
Triple power-cycle (≤10s apart, 3 times) triggers factory reset — clears WiFi
|
||||
credentials and shows AP info on the OLED. Connect to `TJ56654-Setup` /
|
||||
`12345678` and reconfigure via `http://192.168.4.1/config`.
|
||||
|
||||
If that fails, full reset via FTDI:
|
||||
|
||||
```bash
|
||||
esptool.py --port /dev/cu.usbserial-0001 erase_flash
|
||||
esptool.py --port /dev/cu.usbserial-0001 write_flash 0x0 firmware.bin
|
||||
```
|
||||
|
||||
## Project Structure
|
||||
|
||||
@@ -22,19 +22,25 @@ I bought a cute weather clock kit from AliExpress ([TJ-56-654](https://pt.aliexp
|
||||
|
||||
## Table of Contents
|
||||
|
||||
**Story**
|
||||
|
||||
- [The Discovery: When "Smart" Means "Insecure"](#the-discovery-when-smart-means-insecure)
|
||||
- [The Device](#the-device)
|
||||
- [The Investigation](#the-investigation)
|
||||
- [The Solution: Custom Firmware](#the-solution-custom-firmware)
|
||||
- [Technical Deep Dive](#technical-deep-dive)
|
||||
- [The Journey: v1.7 → v1.9.4](#the-journey-v17--v194)
|
||||
- [What's Next: Home Assistant Integration](#whats-next-home-assistant-integration)
|
||||
- [Hardware Quirk: Swapped I2C Pins](#hardware-quirk-swapped-i2c-pins)
|
||||
- [Version History](#version-history)
|
||||
|
||||
**Use it**
|
||||
|
||||
- [How to Flash This Firmware](#how-to-flash-this-firmware)
|
||||
- [Web Interface](#web-interface)
|
||||
- [API Documentation](#api-documentation)
|
||||
- [Testing](#testing)
|
||||
- [Security Improvements](#security-improvements)
|
||||
- [Lessons Learned](#lessons-learned)
|
||||
- [Credits](#credits)
|
||||
- [Project Structure](#project-structure)
|
||||
|
||||
For internal architecture notes (state machines, memory, EEPROM layout), see [docs/ARCHITECTURE.md](docs/ARCHITECTURE.md).
|
||||
|
||||
---
|
||||
|
||||
@@ -246,367 +252,23 @@ All endpoints return JSON:
|
||||
|
||||
---
|
||||
|
||||
## Technical Deep Dive
|
||||
## Hardware Quirk: Swapped I2C Pins
|
||||
|
||||
### Architecture: Fully Async State Machines
|
||||
|
||||
The firmware uses **zero blocking operations** in the main loop. Everything is state-machine-based:
|
||||
|
||||
#### Weather State Machine
|
||||
ESP-01S exposes only GPIO0 and GPIO2. The TJ-56-654 board designer used them as I2C — but **swapped from typical breakouts**:
|
||||
|
||||
```cpp
|
||||
enum WeatherState { IDLE, REQUESTING, SUCCESS, FAILED };
|
||||
Wire.begin(0, 2); // SDA=GPIO0, SCL=GPIO2 (non-standard!)
|
||||
```
|
||||
|
||||
Uses `AsyncHTTPRequest` library:
|
||||
Standard ESP8266 boards use GPIO4=SDA, GPIO5=SCL. Easy to miss if you're porting code from a different ESP8266 setup. The display is a GM009605v4.3 OLED (SSD1306-compatible) at I2C address 0x3C.
|
||||
|
||||
- Non-blocking HTTP requests
|
||||
- Callback-based response handling
|
||||
- Exponential backoff on failures (1s → 2s → 4s)
|
||||
- Maximum 3 retries before giving up
|
||||
For architecture details (async state machines, memory budget, EEPROM layout, factory reset), see [docs/ARCHITECTURE.md](docs/ARCHITECTURE.md).
|
||||
|
||||
#### NTP State Machine
|
||||
## Version History
|
||||
|
||||
```cpp
|
||||
enum NTPState { IDLE, REQUEST_SENT, WAITING, SUCCESS, FAILED };
|
||||
```
|
||||
The clock has gone through many iterations — display hardware discovery (v1.5–v1.7), stability and security fixes (v1.8), full async refactor (v1.9.0), and a long series of bug fixes informed by community reports and AI-assisted code review (v1.9.1–v1.9.10).
|
||||
|
||||
Custom manual NTP implementation:
|
||||
|
||||
- Builds raw UDP packets (48 bytes)
|
||||
- Non-blocking `parsePacket()` checks
|
||||
- 5-second timeout
|
||||
- Independent epoch tracking for accuracy between syncs
|
||||
|
||||
#### WiFi State Machine
|
||||
|
||||
```cpp
|
||||
enum WiFiConnectionState { IDLE, CONNECTING, CONNECTED, FAILED };
|
||||
```
|
||||
|
||||
**Hybrid model** (this was critical!):
|
||||
|
||||
- **Setup phase**: Synchronous connection (waits up to 10 seconds)
|
||||
- Why? OTA, web server, NTP all need WiFi ready
|
||||
- Without this, device shows blank display for 10+ seconds
|
||||
- **Loop phase**: Async reconnection (checks every 5 seconds)
|
||||
- Why? Don't freeze the entire system if WiFi drops
|
||||
|
||||
### Memory Optimization
|
||||
|
||||
ESP8266 has strict memory limits:
|
||||
|
||||
| Memory Type | Total | Used | Usage | Status |
|
||||
| ----------- | --------- | ------- | ------- | ----------- |
|
||||
| **Flash** | 1,048,576 | 408,844 | 38% | ✅ Plenty |
|
||||
| **RAM** | 80,192 | 37,644 | 46% | ✅ Safe |
|
||||
| **IRAM** | 65,536 | 61,987 | **94%** | ⚠️ Critical |
|
||||
|
||||
**IRAM Crisis Solution:**
|
||||
|
||||
IRAM (Instruction RAM) is limited and fills fast. The solution: `ICACHE_FLASH_ATTR` macro.
|
||||
|
||||
```cpp
|
||||
void ICACHE_FLASH_ATTR handleConfig() {
|
||||
// This function's code lives in Flash, not IRAM
|
||||
// Saves precious IRAM at cost of slightly slower execution
|
||||
}
|
||||
```
|
||||
|
||||
Applied to 26 functions (web handlers, display, config utilities), reducing IRAM pressure from **overflow risk** to **sustainable 94%**.
|
||||
|
||||
**String Safety:**
|
||||
|
||||
Avoid String concatenation in loops (causes heap fragmentation):
|
||||
|
||||
```cpp
|
||||
// ❌ BAD - 140+ concatenations
|
||||
String html = "";
|
||||
html += F("<!DOCTYPE html>");
|
||||
html += F("<head>..."); // x138 more times
|
||||
|
||||
// ✅ GOOD - Chunked responses
|
||||
server.setContentLength(CONTENT_LENGTH_UNKNOWN);
|
||||
server.send(200, "text/html", "");
|
||||
server.sendContent_P(HTML_HEADER);
|
||||
server.sendContent_P(HTML_FOOTER);
|
||||
server.sendContent(""); // End
|
||||
```
|
||||
|
||||
### Configuration Storage
|
||||
|
||||
26-field struct stored in EEPROM (512 bytes):
|
||||
|
||||
```cpp
|
||||
struct Config {
|
||||
char ssid[32];
|
||||
char password[64];
|
||||
int timezone_offset;
|
||||
bool dst_enabled;
|
||||
uint8_t brightness;
|
||||
char ntp_server[64];
|
||||
unsigned long ntp_interval;
|
||||
bool hour_format_24;
|
||||
char hostname[32];
|
||||
float latitude;
|
||||
float longitude;
|
||||
char city_name[32];
|
||||
bool weather_enabled;
|
||||
unsigned long weather_interval;
|
||||
unsigned long display_rotation_sec;
|
||||
bool show_weather;
|
||||
bool show_sunrise_sunset;
|
||||
uint8_t display_orientation;
|
||||
uint32_t magic; // 0xC10CC10C - validation
|
||||
};
|
||||
```
|
||||
|
||||
**EEPROM validation**: Magic number check prevents loading corrupted data. On validation failure, gracefully defaults to safe config.
|
||||
|
||||
### Display Hardware Discovery
|
||||
|
||||
This took **3 firmware iterations** to get right:
|
||||
|
||||
**v1.5**: Assumed TM1637 (7-segment LED driver)
|
||||
|
||||
- ❌ Wrong - device has OLED, not 7-segment LEDs
|
||||
|
||||
**v1.6**: Tried TM1650 (another LED driver)
|
||||
|
||||
- ❌ Wrong - I2C addresses didn't match
|
||||
|
||||
**v1.7**: Identified GM009605v4.3 (SSD1306-compatible OLED)
|
||||
|
||||
- ✅ Correct! Used Adafruit_SSD1306 library
|
||||
- ✅ Discovered swapped pins: SDA on GPIO0, SCL on GPIO2
|
||||
|
||||
**Pin mapping quirk:**
|
||||
|
||||
Standard ESP8266 I2C uses GPIO4 (SDA) and GPIO5 (SCL), but ESP-01S only exposes GPIO0 and GPIO2. The board designer mapped:
|
||||
|
||||
- GPIO0 → SDA (unusual)
|
||||
- GPIO2 → SCL (unusual)
|
||||
|
||||
This is **backwards** from typical breakout boards, but works perfectly once configured:
|
||||
|
||||
```cpp
|
||||
Wire.begin(0, 2); // SDA=GPIO0, SCL=GPIO2
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## The Journey: v1.7 → v1.9.4
|
||||
|
||||
### v1.7: Display Discovery ✅
|
||||
|
||||
- Identified correct display hardware
|
||||
- Basic time display working
|
||||
- WiFiManager integration
|
||||
- First OTA deployment
|
||||
|
||||
### v1.8: Stability & Security 🔒
|
||||
|
||||
**Goals**: Fix memory issues, eliminate security holes
|
||||
|
||||
**Changes:**
|
||||
|
||||
- IRAM optimization (added `ICACHE_FLASH_ATTR` to 26 functions)
|
||||
- Removed hardcoded WiFi credentials
|
||||
- Fixed NTP interval bug (config value was ignored)
|
||||
- Fixed boolean parsing in JSON import
|
||||
- Added input validation (buffer overflow protection)
|
||||
- Chunked HTTP responses (eliminated 140+ String concatenations)
|
||||
|
||||
**Result**: IRAM usage 94% → 70%, no memory leaks, secure config storage
|
||||
|
||||
### v1.9.0: Full Async Refactoring ⚡
|
||||
|
||||
**Goals**: Eliminate all blocking operations
|
||||
|
||||
**Changes:**
|
||||
|
||||
- Async HTTP weather fetch (AsyncHTTPRequest library)
|
||||
- Custom async NTP implementation (manual UDP packets)
|
||||
- Async WiFi connection (state machine)
|
||||
- Removed all `delay()` calls from `loop()`
|
||||
- Exponential backoff retry logic
|
||||
|
||||
**Performance:**
|
||||
|
||||
| Operation | Before (v1.8) | After (v1.9.0) | Improvement |
|
||||
| -------------- | -------------- | -------------- | ------------------- |
|
||||
| Weather fetch | 1-10s blocking | 0ms | ✅ Async callback |
|
||||
| NTP sync | 5-20s blocking | 0ms | ✅ Non-blocking UDP |
|
||||
| WiFi reconnect | 15s blocking | 0ms | ✅ State machine |
|
||||
| Loop time | 10ms minimum | <1ms | ✅ 10x faster |
|
||||
|
||||
**Result**: Device stays responsive during OTA updates while weather is fetching!
|
||||
|
||||
### v1.9.1: Hybrid Fix 🎯
|
||||
|
||||
**Problem Discovered:**
|
||||
|
||||
After deploying v1.9.0, the display showed **blank screen for 10 seconds** after boot, with `DNS resolution failed` errors in logs.
|
||||
|
||||
**Root Cause:**
|
||||
|
||||
Making WiFi fully async broke the **initialization order**:
|
||||
|
||||
```cpp
|
||||
void setup() {
|
||||
setupWiFi(); // Returns immediately (async)
|
||||
setupOTA(); // WiFi NOT ready! ❌
|
||||
setupWebServer(); // WiFi NOT ready! ❌
|
||||
testInternetConnectivity(); // WiFi NOT ready! → DNS error
|
||||
}
|
||||
```
|
||||
|
||||
**Solution: Hybrid Model**
|
||||
|
||||
| Phase | WiFi Mode | Blocking? | Why? |
|
||||
| --------- | ------------ | --------- | --------------------------- |
|
||||
| `setup()` | Synchronous | 10s max | OTA/web/NTP need WiFi ready |
|
||||
| `loop()` | Asynchronous | 0s | Don't freeze on reconnect |
|
||||
|
||||
**Results:**
|
||||
|
||||
- ✅ Display shows time immediately after WiFi connects (~15 sec boot)
|
||||
- ✅ No "DNS resolution failed" errors
|
||||
- ✅ Proper initialization order guaranteed
|
||||
- ✅ Device never freezes on WiFi loss during operation
|
||||
|
||||
### v1.9.2: WiFi Resilience 🛡️
|
||||
|
||||
**Problem Discovered:**
|
||||
|
||||
After WiFi outages, the device would **clear stored credentials** and enter AP mode, requiring manual reconfiguration every time the router restarted.
|
||||
|
||||
**Root Cause:**
|
||||
|
||||
Aggressive credential clearing on connection failure:
|
||||
|
||||
```cpp
|
||||
if (wifiRetry.currentRetry >= wifiRetry.maxRetries) {
|
||||
memset(config.ssid, 0, sizeof(config.ssid)); // ❌ Clears credentials!
|
||||
memset(config.password, 0, sizeof(config.password));
|
||||
saveConfig();
|
||||
// Enter AP mode...
|
||||
}
|
||||
```
|
||||
|
||||
**Solution: Resilient WiFi**
|
||||
|
||||
| Feature | Before (v1.9.1) | After (v1.9.2) |
|
||||
| ------------------- | -------------------------- | ------------------------------- |
|
||||
| Credential clearing | After 5 failed attempts | Never |
|
||||
| Retry strategy | Give up after 5 tries | Infinite with backoff |
|
||||
| Max retry interval | N/A | 5 minutes |
|
||||
| Fallback AP | After clearing credentials | After ~5 min (dual STA+AP mode) |
|
||||
| Clock during outage | Blank display | Shows last synced time |
|
||||
|
||||
**Key Changes:**
|
||||
|
||||
- **Never clear credentials** on connection failure
|
||||
- **Exponential backoff**: 5s → 10s → 20s → ... → 5min max
|
||||
- **Fallback AP** ("TJ56654-Setup") enabled after ~5 min, while still retrying
|
||||
- **Dual STA+AP mode**: Device continues reconnect attempts while AP is active
|
||||
- **SDK credentials**: Used only on first boot (no saved SSID); subsequent boots go straight to saved credentials
|
||||
- **"No WiFi" display**: Shows retry countdown instead of cryptic numbers
|
||||
- **"!" indicator**: Shown in date line when WiFi disconnected
|
||||
|
||||
**Network Activity Summary:**
|
||||
|
||||
| Service | Interval | Endpoint | Protocol |
|
||||
| ------- | ---------- | ------------------ | ------------- |
|
||||
| NTP | 1 hour | pool.ntp.org:123 | UDP |
|
||||
| Weather | 30 min | api.open-meteo.com | HTTP |
|
||||
| mDNS | continuous | 224.0.0.251 | UDP multicast |
|
||||
|
||||
~50 requests/day total.
|
||||
|
||||
**Results:**
|
||||
|
||||
- ✅ Credentials persist through WiFi outages
|
||||
- ✅ Device automatically reconnects when WiFi returns
|
||||
- ✅ Clock continues running with last synced time
|
||||
- ✅ User can reconfigure via fallback AP if needed
|
||||
|
||||
**Startup Timeline:**
|
||||
|
||||
```
|
||||
[0-5s] Display init, startup animation
|
||||
[5-15s] WiFi connection (SYNCHRONOUS in setup())
|
||||
✅ WiFi connected! IP assigned
|
||||
[15-20s] OTA init, web server start, NTP client ready
|
||||
✅ Internet test: PASSED
|
||||
[20-30s] First async NTP sync
|
||||
✅ Time synced and displayed
|
||||
```
|
||||
|
||||
### v1.9.3: Modular Architecture 🗂️
|
||||
|
||||
Split monolithic 2,100-line `.ino` into focused modules:
|
||||
|
||||
| File | Responsibility |
|
||||
| ------------------- | --------------------------------------- |
|
||||
| `weather_clock.ino` | Entry point: `setup()` and `loop()` |
|
||||
| `config.h` | Config struct, EEPROM layout, constants |
|
||||
| `globals.h` | Shared state and extern declarations |
|
||||
| `display.cpp` | OLED rendering |
|
||||
| `ntp_client.cpp` | Async NTP sync |
|
||||
| `weather.cpp` | Open-Meteo API fetch |
|
||||
| `web_server.cpp` | Web UI and REST API |
|
||||
| `wifi_manager.cpp` | WiFi connection and resilience |
|
||||
|
||||
### v1.9.4: Bug Fixes & Cleanup ✅ (Current)
|
||||
|
||||
Community-reported bugs fixed:
|
||||
|
||||
- **Date timezone** (#5): Date now changes at local midnight, not UTC midnight
|
||||
- **Weather refresh** (#7): Periodic weather updates no longer blocked after first fetch
|
||||
- **WiFi hotspot** (#3): Device no longer connects to open SDK-cached networks (e.g. public hotspots) when a saved SSID exists, preventing config corruption
|
||||
- **ArduinoJson v7**: Updated `StaticJsonDocument` → `JsonDocument` for library compatibility
|
||||
- **Compiler warnings**: Removed unused variable, fixed sprintf buffer size
|
||||
|
||||
### Memory Evolution
|
||||
|
||||
| Version | RAM Usage | IRAM Usage | Flash Usage | Notes |
|
||||
| ------- | ------------ | ---------------- | ------------- | --------------------- |
|
||||
| v1.7 | 34,980 (43%) | **61,987 (94%)** | 407,500 (38%) | IRAM crisis |
|
||||
| v1.8 | 36,980 (46%) | **45,120 (68%)** | 407,800 (38%) | ICACHE_FLASH_ATTR fix |
|
||||
| v1.9.0 | 37,516 (46%) | **61,987 (94%)** | 408,540 (38%) | Async libs added |
|
||||
| v1.9.1 | 37,644 (46%) | **61,987 (94%)** | 408,844 (38%) | Hybrid WiFi fix |
|
||||
| v1.9.2 | 37,800 (47%) | **61,987 (94%)** | 409,100 (39%) | WiFi resilience |
|
||||
|
||||
**Verdict**: Stable memory usage, no leaks detected after 24h+ uptime tests.
|
||||
|
||||
---
|
||||
|
||||
## What's Next: Home Assistant Integration
|
||||
|
||||
The firmware is designed to be extensible. Next planned features:
|
||||
|
||||
### Custom Display Screens
|
||||
|
||||
Pull data from Home Assistant via REST API:
|
||||
|
||||
- **Smart home stats**: Energy usage, room temperatures
|
||||
- **Sensor data**: Air quality, CO2 levels
|
||||
- **Automation states**: Alarm status, door locks
|
||||
|
||||
### MQTT Integration
|
||||
|
||||
- Publish time/weather data to MQTT broker
|
||||
- Subscribe to topics for display content
|
||||
- Enable automation triggers (e.g., display alert when door opens)
|
||||
|
||||
### WebSocket Live Updates
|
||||
|
||||
Replace polling with WebSocket for:
|
||||
|
||||
- Real-time config changes without page refresh
|
||||
- Live display preview in web UI
|
||||
- Push notifications for firmware updates
|
||||
See [CHANGELOG.md](CHANGELOG.md) for the full version history with technical details.
|
||||
|
||||
---
|
||||
|
||||
@@ -932,39 +594,6 @@ Device reboots immediately.
|
||||
|
||||
---
|
||||
|
||||
## Lessons Learned
|
||||
|
||||
### Hardware
|
||||
|
||||
1. **Always check pinouts**: Don't assume standard pin mappings - this device swaps SDA/SCL
|
||||
2. **FTDI is your friend**: A $2 adapter unlocks any ESP8266 device
|
||||
3. **Transparent cases are great**: Made debugging and identification trivial
|
||||
4. **Read the PCB silk screen**: Model numbers and pin labels save hours of guessing
|
||||
|
||||
### Software
|
||||
|
||||
1. **Async is hard but worth it**: Fully non-blocking architecture eliminates user-facing freezes
|
||||
2. **IRAM is precious**: On ESP8266, use `ICACHE_FLASH_ATTR` liberally
|
||||
3. **Hybrid approaches work**: Don't be dogmatic - synchronous WiFi in setup() solved a critical UX issue
|
||||
4. **State machines scale**: Better than callback hell for complex async operations
|
||||
5. **Test on real hardware**: Emulators can't catch pin mapping errors or memory constraints
|
||||
|
||||
### Security
|
||||
|
||||
1. **IoT security is often terrible**: Always audit devices before trusting them on your network
|
||||
2. **Open source is safer**: Closed firmware is a black box - you have no idea what it's doing
|
||||
3. **Defaults matter**: Insecure defaults (open AP, plaintext passwords) lead to real vulnerabilities
|
||||
4. **Defense in depth**: Multiple layers (WiFiManager timeout, password protection, validation) catch mistakes
|
||||
|
||||
### Development
|
||||
|
||||
1. **OTA from day 1**: Flashing via FTDI gets old fast - build OTA support early
|
||||
2. **Version your work**: Backup files (.bak, .bak2) saved me multiple times
|
||||
3. **Document as you go**: Release notes and architecture docs prevent "what was I thinking?" moments
|
||||
4. **Incremental improvements**: v1.7 → v1.8 → v1.9.x made debugging manageable
|
||||
|
||||
---
|
||||
|
||||
## Credits
|
||||
|
||||
**Hardware**: TJ-56-654 Weather Clock Kit ([AliExpress](https://pt.aliexpress.com/item/1005008333782531.html))
|
||||
@@ -996,8 +625,17 @@ Device reboots immediately.
|
||||
```
|
||||
esp8266-weather-clock/
|
||||
├── firmware/
|
||||
│ └── weather_clock/
|
||||
│ └── weather_clock.ino # Main firmware (~2,100 lines)
|
||||
│ └── weather_clock/ # Modular firmware
|
||||
│ ├── weather_clock.ino # setup() and loop()
|
||||
│ ├── config.h # Config struct, EEPROM layout
|
||||
│ ├── globals.h # Shared state
|
||||
│ ├── display.cpp # OLED rendering
|
||||
│ ├── ntp_client.cpp # Async NTP + DST
|
||||
│ ├── weather.cpp # Open-Meteo API
|
||||
│ ├── web_server.cpp # HTTP UI + REST API
|
||||
│ └── wifi_manager.cpp # WiFi resilience
|
||||
├── tests/
|
||||
│ └── test_device.py # HW-in-the-loop test suite (73 cases)
|
||||
├── docs/
|
||||
│ ├── HARDWARE.md # Hardware specifications
|
||||
│ └── INSTALLATION.md # Flashing guide
|
||||
@@ -1005,6 +643,21 @@ esp8266-weather-clock/
|
||||
└── README.md # This file
|
||||
```
|
||||
|
||||
## Testing
|
||||
|
||||
```bash
|
||||
# Hardware-in-the-loop test suite — verifies API, fuzzes config endpoint,
|
||||
# checks heap stability over 5 samples
|
||||
python3 tests/test_device.py 192.168.x.x
|
||||
```
|
||||
|
||||
The test suite covers:
|
||||
|
||||
- All REST API endpoints (functional validation)
|
||||
- Boundary fuzz against `/config` (oversized SSID, invalid intervals, etc.)
|
||||
- Malformed JSON fuzz against `/api/config` import
|
||||
- Heap stability check (must stay >20KB, drift <2KB across runs)
|
||||
|
||||
---
|
||||
|
||||
## License
|
||||
@@ -1013,29 +666,6 @@ This project is released into the public domain. Do whatever you want with it. I
|
||||
|
||||
---
|
||||
|
||||
## Final Thoughts
|
||||
|
||||
This project started as "I don't trust this device" and ended as "I built something better."
|
||||
|
||||
The original firmware had security holes you could drive a truck through. The custom replacement:
|
||||
|
||||
- ✅ Doesn't leak WiFi passwords
|
||||
- ✅ Uses free, open APIs
|
||||
- ✅ Updates over WiFi
|
||||
- ✅ Runs fully async (no freezing)
|
||||
- ✅ Integrates with Home Assistant (coming soon)
|
||||
- ✅ Is completely auditable (you're reading the source)
|
||||
|
||||
Total cost: €5 hardware + a weekend of tinkering.
|
||||
|
||||
If you have one of these devices, **flash this firmware**. If you're buying IoT gadgets, **always audit them first**. And if something seems insecure, **fix it yourself** - that's the hacker spirit.
|
||||
|
||||
Now go make something cool. 🚀
|
||||
|
||||
---
|
||||
|
||||
**P.S.**: If you found this useful, consider starring the repo. If you found a bug, open an issue. If you want to add Home Assistant screens, let's collaborate - I'm planning that next!
|
||||
|
||||
**Author**: apetrochenko
|
||||
**Date**: 2026-01-06
|
||||
**Firmware Version**: v1.9.4 (Production Ready)
|
||||
**Author**: apetrochenko · **License**: MIT · **Firmware**: v1.9.10
|
||||
|
||||
@@ -0,0 +1,118 @@
|
||||
# Architecture Notes
|
||||
|
||||
Internal notes for contributors modifying the firmware. End users don't need this.
|
||||
|
||||
## Async model
|
||||
|
||||
The main loop performs zero blocking operations. All network I/O runs through callback-based state machines:
|
||||
|
||||
- **Weather**: `AsyncHTTPRequest` callback updates `weatherState` (IDLE → REQUESTING → IDLE on success/failure), with a 15s watchdog in the loop to recover from TCP hangs
|
||||
- **NTP**: Manual UDP packet build + non-blocking `parsePacket()` with 5s timeout
|
||||
- **WiFi**: Synchronous in `setup()` (so OTA / web / NTP have a working network at start), async reconnect in `loop()` with exponential backoff
|
||||
|
||||
State variables shared between callbacks and the main loop are declared `volatile` (`weatherState`, `ntpState`) since ESPAsyncTCP callbacks fire from within `yield()`/TCP poll and can interrupt the loop at any point.
|
||||
|
||||
## Timer correctness
|
||||
|
||||
`millis()` is `uint32_t` and rolls over every ~49.7 days. All deadline comparisons use the subtraction-safe pattern:
|
||||
|
||||
```cpp
|
||||
// Unsafe: fails at rollover
|
||||
if (millis() >= nextRetryTime) { ... }
|
||||
|
||||
// Safe: works across rollover
|
||||
if ((millis() - nextRetryTime) < 0x80000000UL) { ... }
|
||||
```
|
||||
|
||||
See `RetryConfig::isRetryTime()` in `config.h` for the canonical implementation.
|
||||
|
||||
## Memory budget (ESP-01S, 1MB flash)
|
||||
|
||||
| Pool | Total | Used | % | Notes |
|
||||
| --------- | --------- | -------- | --- | -------------------------------------------------------- |
|
||||
| **Flash** | 1,048,576 | ~411,000 | 39% | Headroom for features |
|
||||
| **RAM** | 80,192 | ~37,700 | 46% | Stable across all 1.9.x releases |
|
||||
| **IRAM** | 65,536 | 61,987 | 94% | **Critical** — `ICACHE_FLASH_ATTR` required on new funcs |
|
||||
|
||||
### IRAM discipline
|
||||
|
||||
IRAM is the tightest constraint. Apply `ICACHE_FLASH_ATTR` to any non-critical function so its code lives in flash instead of IRAM:
|
||||
|
||||
```cpp
|
||||
void ICACHE_FLASH_ATTR handleConfig() {
|
||||
// code in flash, slightly slower but saves IRAM
|
||||
}
|
||||
```
|
||||
|
||||
Currently 26 functions use this attribute (web handlers, display routines, config helpers). Functions called from interrupt context or hot loops should stay in IRAM.
|
||||
|
||||
### Heap fragmentation
|
||||
|
||||
ESP8266 heap is never compacted. Repeated `String` concatenation in long-running handlers (especially `/api/*` endpoints polled by the web UI) permanently fragments the heap until reboot.
|
||||
|
||||
Pattern to follow for HTTP responses:
|
||||
|
||||
```cpp
|
||||
char buf[256];
|
||||
server.setContentLength(CONTENT_LENGTH_UNKNOWN);
|
||||
server.send(200, "application/json", "");
|
||||
snprintf_P(buf, sizeof(buf), PSTR("{\"...\":%d}"), value);
|
||||
server.sendContent(buf);
|
||||
server.sendContent(""); // close chunked transfer
|
||||
```
|
||||
|
||||
Never:
|
||||
|
||||
```cpp
|
||||
String html = "";
|
||||
html += "..."; // <- heap fragmentation
|
||||
```
|
||||
|
||||
## EEPROM layout
|
||||
|
||||
512 bytes mapped via `EEPROM.begin(512)`:
|
||||
|
||||
| Offset | Size | Content |
|
||||
| ------ | ------ | ------------------------------------------------- |
|
||||
| 0 | ~260 B | `Config` struct (validated by magic `0xC10CC10C`) |
|
||||
| 480 | 2 B | `ResetCounter` for triple power-cycle reset |
|
||||
|
||||
Writes are flash-sector-erase operations (4KB), so the entire 512-byte image is rewritten on each `EEPROM.commit()`. Expect ~100k cycles before wear matters.
|
||||
|
||||
The `Config` struct includes a magic number — on validation failure, the firmware falls back to defaults rather than loading garbage.
|
||||
|
||||
## Factory reset
|
||||
|
||||
Three quick power cycles within 10 seconds (controlled by `RESET_COUNTER_WINDOW`) triggers a factory reset:
|
||||
|
||||
1. Each boot increments the counter at EEPROM offset 480
|
||||
2. If the device runs for 10s, the counter clears back to 0
|
||||
3. If count reaches 3 first, WiFi credentials are wiped and the device reboots into WiFiManager AP mode
|
||||
|
||||
For atomicity, credentials are committed BEFORE the counter is zeroed. Power loss between the two writes still results in a recoverable state (cleared creds → AP boot).
|
||||
|
||||
## Display
|
||||
|
||||
128×64 OLED, SSD1306-compatible (GM009605v4.3 panel). The ESP-01S exposes only GPIO0 and GPIO2, and the board designer mapped them as I2C:
|
||||
|
||||
```cpp
|
||||
Wire.begin(0, 2); // SDA=GPIO0, SCL=GPIO2 (non-standard!)
|
||||
```
|
||||
|
||||
This is the inverse of typical breakout boards — easy to get wrong if you're porting code.
|
||||
|
||||
## Web server
|
||||
|
||||
`ESP8266WebServer` (not async, but fast enough for low-frequency requests). All API responses use chunked transfer with `snprintf` + `sendContent`. Form save (`/config`) only triggers a reboot when WiFi/network fields actually change; other settings apply live.
|
||||
|
||||
Input validation in `handleConfigSave` clamps numeric fields and rejects garbage SSIDs (HTTP 400) — see `isValidSSID()` in `web_server.cpp`.
|
||||
|
||||
## Adding new features
|
||||
|
||||
1. Check IRAM usage after build — if growing, add `ICACHE_FLASH_ATTR` to your new functions
|
||||
2. Don't use `String` in handlers or callbacks — use `char buf[]` + `snprintf`
|
||||
3. New shared state between callbacks and loop → declare `volatile`
|
||||
4. New timer logic → use the subtraction-safe `millis()` pattern
|
||||
5. New API endpoints → run `tests/test_device.py` against the device to verify heap stability
|
||||
|
||||
For more context on past design decisions, see `CHANGELOG.md`.
|
||||
@@ -499,6 +499,34 @@ To save memory, disable unused features:
|
||||
|
||||
---
|
||||
|
||||
## Factory Reset (no tools needed)
|
||||
|
||||
Since v1.9.6, three quick power cycles within 10 seconds trigger a factory reset:
|
||||
|
||||
1. Power off the device
|
||||
2. Power on (briefly, < 10 sec)
|
||||
3. Power off, power on
|
||||
4. Power off, power on
|
||||
|
||||
On the third boot the OLED shows `FACTORY RESET! WiFi: TJ56654-Setup Pass: 12345678`.
|
||||
Connect your phone to that AP and reconfigure WiFi at `http://192.168.4.1/config`.
|
||||
|
||||
Use this if:
|
||||
|
||||
- Forgot configured WiFi credentials
|
||||
- Device stuck in "No WiFi" loop after router change
|
||||
- Tests/fuzzing corrupted config
|
||||
|
||||
## Verifying Installation
|
||||
|
||||
Run the test suite against your device:
|
||||
|
||||
```bash
|
||||
python3 tests/test_device.py 192.168.x.x
|
||||
```
|
||||
|
||||
A healthy device passes all 73 tests, with heap drift under 1 KB across the run.
|
||||
|
||||
## Getting Help
|
||||
|
||||
If you're still stuck:
|
||||
|
||||
@@ -9,7 +9,7 @@
|
||||
#include <Arduino.h>
|
||||
|
||||
// Firmware version
|
||||
#define FIRMWARE_VERSION "1.9.8"
|
||||
#define FIRMWARE_VERSION "1.9.10"
|
||||
|
||||
// OLED I2C Configuration
|
||||
#define I2C_SDA 0 // GPIO0 (I2C Data) - SWAPPED!
|
||||
@@ -120,13 +120,11 @@ enum WeatherState {
|
||||
WEATHER_FAILED
|
||||
};
|
||||
|
||||
// Async NTP state machine
|
||||
// Async NTP state machine — only IDLE and REQUEST_SENT are used
|
||||
// (response handler transitions back to IDLE directly on success or timeout)
|
||||
enum NTPState {
|
||||
NTP_IDLE,
|
||||
NTP_REQUEST_SENT,
|
||||
NTP_WAITING,
|
||||
NTP_SUCCESS,
|
||||
NTP_FAILED
|
||||
NTP_REQUEST_SENT
|
||||
};
|
||||
|
||||
// Async WiFi state machine
|
||||
|
||||
@@ -133,16 +133,22 @@ void ICACHE_FLASH_ATTR checkFactoryReset() {
|
||||
|
||||
if (rc.count >= RESET_COUNTER_TRIPS) {
|
||||
Serial.println("!!! FACTORY RESET triggered !!!");
|
||||
|
||||
// Atomicity: clear credentials FIRST (saveConfig commits), then zero counter.
|
||||
// If power fails between the two commits, the device boots into AP mode next
|
||||
// time (creds are already cleared) — instead of being in a "counter zeroed but
|
||||
// creds still valid" inconsistent state.
|
||||
memset(config.ssid, 0, sizeof(config.ssid));
|
||||
memset(config.password, 0, sizeof(config.password));
|
||||
EEPROM.end(); // close current handle before saveConfig opens its own
|
||||
saveConfig(); // commits cleared credentials to flash
|
||||
|
||||
EEPROM.begin(512);
|
||||
rc.count = 0;
|
||||
EEPROM.put(RESET_COUNTER_ADDR, rc);
|
||||
EEPROM.commit();
|
||||
EEPROM.end();
|
||||
|
||||
// Clear WiFi credentials only — keep other settings
|
||||
memset(config.ssid, 0, sizeof(config.ssid));
|
||||
memset(config.password, 0, sizeof(config.password));
|
||||
saveConfig();
|
||||
|
||||
// Show reset screen
|
||||
display.clearDisplay();
|
||||
display.setTextColor(SSD1306_WHITE);
|
||||
@@ -244,6 +250,11 @@ void ICACHE_FLASH_ATTR setupOTA() {
|
||||
void setup() {
|
||||
Serial.begin(115200);
|
||||
delay(100);
|
||||
|
||||
// Seed PRNG with hardware entropy: chip ID is unique per device,
|
||||
// micros() varies on each boot due to power-on timing jitter
|
||||
randomSeed(ESP.getChipId() ^ micros());
|
||||
|
||||
Serial.println("\n\nTJ-56-654 NTP Clock with OTA v" FIRMWARE_VERSION);
|
||||
Serial.println("==========================================");
|
||||
Serial.println("Display: GM009605v4.3 OLED 128x64 (SSD1306 I2C)");
|
||||
|
||||
@@ -294,7 +294,8 @@ static bool isValidSSID(const String& s) {
|
||||
if (c < 0x20 || c > 0x7E) return false; // non-printable
|
||||
if (c != first) allSame = false;
|
||||
}
|
||||
return !allSame; // reject "AAAAA...", "BBBBB...", etc.
|
||||
// Single-char SSIDs ("A") are valid per 802.11. Only reject all-same for length>1.
|
||||
return n == 1 || !allSame;
|
||||
}
|
||||
|
||||
void ICACHE_FLASH_ATTR handleConfigSave() {
|
||||
@@ -358,7 +359,9 @@ void ICACHE_FLASH_ATTR handleConfigSave() {
|
||||
}
|
||||
if (server.hasArg("ntp_interval")) {
|
||||
long ni = server.arg("ntp_interval").toInt();
|
||||
config.ntp_interval = constrain(ni, 60L, 86400L);
|
||||
unsigned long newInterval = constrain(ni, 60L, 86400L);
|
||||
if (newInterval != config.ntp_interval) needsRestart = true; // NTPClient constructed at boot with this
|
||||
config.ntp_interval = newInterval;
|
||||
}
|
||||
if (server.hasArg("ntp_server")) {
|
||||
String n = server.arg("ntp_server");
|
||||
|
||||
@@ -28,9 +28,10 @@ void ICACHE_FLASH_ATTR processWiFiConnection() {
|
||||
WiFi.mode(WIFI_STA);
|
||||
}
|
||||
|
||||
// Sync connected SSID to config
|
||||
// Sync connected SSID + password to config
|
||||
if (strlen(config.ssid) == 0) {
|
||||
safeStringCopy(WiFi.SSID(), config.ssid, sizeof(config.ssid));
|
||||
safeStringCopy(WiFi.psk(), config.password, sizeof(config.password));
|
||||
saveConfig();
|
||||
}
|
||||
|
||||
@@ -96,7 +97,12 @@ void ICACHE_FLASH_ATTR setupWiFi() {
|
||||
Serial.print("DNS: ");
|
||||
Serial.println(WiFi.dnsIP());
|
||||
|
||||
// Persist the password too, not just the SSID. The SDK keeps the
|
||||
// password in its own flash area, but every later boot reads
|
||||
// config.password: an empty one is treated as an open network
|
||||
// (WiFi.begin(ssid)) and fails with WL_WRONG_PASSWORD (issue #12).
|
||||
safeStringCopy(WiFi.SSID(), config.ssid, sizeof(config.ssid));
|
||||
safeStringCopy(WiFi.psk(), config.password, sizeof(config.password));
|
||||
saveConfig();
|
||||
|
||||
showIP();
|
||||
@@ -105,10 +111,14 @@ void ICACHE_FLASH_ATTR setupWiFi() {
|
||||
}
|
||||
}
|
||||
|
||||
// Try 2: If we have EEPROM credentials, try those
|
||||
if (strlen(config.ssid) > 0 && strlen(config.password) > 0) {
|
||||
// Try 2: If we have EEPROM credentials, try those (M2: support open networks)
|
||||
if (strlen(config.ssid) > 0) {
|
||||
Serial.println("\nTrying EEPROM credentials...");
|
||||
WiFi.begin(config.ssid, config.password);
|
||||
if (strlen(config.password) > 0) {
|
||||
WiFi.begin(config.ssid, config.password);
|
||||
} else {
|
||||
WiFi.begin(config.ssid); // open network — no password
|
||||
}
|
||||
|
||||
int attempts = 0;
|
||||
while (WiFi.status() != WL_CONNECTED && attempts < 20) {
|
||||
@@ -162,7 +172,10 @@ void ICACHE_FLASH_ATTR setupWiFi() {
|
||||
Serial.print("IP: ");
|
||||
Serial.println(WiFi.localIP());
|
||||
|
||||
// Save both SSID and password (issue #12): WiFiManager stores them in the
|
||||
// SDK flash, but the next boot connects from config.* only.
|
||||
safeStringCopy(WiFi.SSID(), config.ssid, sizeof(config.ssid));
|
||||
safeStringCopy(WiFi.psk(), config.password, sizeof(config.password));
|
||||
saveConfig();
|
||||
|
||||
showIP();
|
||||
|
||||
Reference in New Issue
Block a user