diff --git a/PROJECT_CONTEXT.md b/PROJECT_CONTEXT.md deleted file mode 100644 index ae143f9..0000000 --- a/PROJECT_CONTEXT.md +++ /dev/null @@ -1,800 +0,0 @@ -# ESP8266 Weather Clock - Full Project Context - -**Last Updated**: 2026-01-03 -**Current Version**: v1.9.1 (Production Ready) -**GitHub**: https://github.com/petrochen/esp8266-weather-clock-opensource - ---- - -## Table of Contents - -1. [Project Overview](#project-overview) -2. [Hardware Details](#hardware-details) -3. [Development History](#development-history) -4. [Current Status (v1.9.1)](#current-status-v191) -5. [Technical Architecture](#technical-architecture) -6. [Security Issues Fixed](#security-issues-fixed) -7. [Files & Structure](#files--structure) -8. [Git Repository](#git-repository) -9. [Next Steps](#next-steps) -10. [Key Learnings](#key-learnings) - ---- - -## Project Overview - -### What Is This? - -Complete reverse engineering and firmware replacement for an ESP8266-based weather clock purchased from AliExpress (model TJ-56-654, €5). - -### Why? - -**Original firmware had critical security vulnerabilities:** -- WiFi password displayed in plaintext -- Persistent open access point running in parallel to home WiFi -- Dependency on Chinese cloud service (QWeather) requiring API key registration -- No OTA updates (required physical FTDI access for updates) - -**Solution:** Complete custom firmware with security, performance, and features. - -### Key Features - -- ✅ **Security**: No password leaks, WiFiManager captive portal, no hardcoded credentials -- ✅ **Performance**: Fully async architecture, <1ms loop time (was 10ms+) -- ✅ **Weather**: Open-Meteo API (free, no registration, no API key) -- ✅ **Updates**: OTA via web interface + ArduinoOTA -- ✅ **Interface**: Web UI, REST API, full configuration -- ✅ **Display**: 3 rotating modes (time, weather, sunrise/sunset with daylight duration) -- ✅ **Time**: NTP sync with timezone + automatic European DST - ---- - -## Hardware Details - -### Device Purchased - -- **Product**: ESP8266 Mini Weather Clock Kit -- **Model**: TJ-56-654 -- **Source**: [AliExpress Link](https://pt.aliexpress.com/item/1005008333782531.html) -- **Price**: €5 EUR (~$5.50 USD) -- **Size**: 40mm x 40mm x 43mm (transparent acrylic case) - -### Components - -#### ESP-01S WiFi Module -- **Chip**: ESP8266EX -- **Flash**: 1MB (8Mbit) -- **RAM**: 80KB total -- **CPU**: 80MHz -- **WiFi**: 802.11 b/g/n (2.4GHz only) -- **GPIO**: Only GPIO0 and GPIO2 available -- **Voltage**: 3.3V ⚠️ NOT 5V tolerant - -#### Display Module -- **Model**: GM009605v4.3 -- **Type**: OLED (128x64 pixels, 0.96 inches) -- **Controller**: SSD1306/SH1106 compatible -- **Interface**: I2C -- **I2C Address**: 0x3C (default), 0x3D (fallback) -- **Colors**: Monochrome (white on black) - -#### Pin Mapping (CRITICAL!) - -**Non-standard I2C mapping:** -- SDA: GPIO0 (not GPIO4 as typical) -- SCL: GPIO2 (not GPIO5 as typical) - -This mapping is **backwards** from standard ESP8266 breakout boards! - -```cpp -Wire.begin(0, 2); // SDA=GPIO0, SCL=GPIO2 -``` - -### Power Supply -- Input: 5V via Micro-USB -- Current: 80-120mA typical -- Regulator: Onboard 3.3V LDO - ---- - -## Development History - -### Timeline - -**2025-12-29**: v1.5 (unreleased) -- Attempted TM1637 7-segment display support -- ❌ Wrong - device has OLED, not 7-segment LEDs - -**2025-12-30**: v1.6 (unreleased) -- Attempted TM1650 LED driver support -- ❌ Wrong - I2C addresses didn't match - -**2025-12-31**: v1.7 ✅ -- ✅ Identified correct display: GM009605v4.3 (SSD1306-compatible OLED) -- ✅ Discovered swapped pins: SDA=GPIO0, SCL=GPIO2 -- ✅ Switched to Adafruit_SSD1306 library -- ✅ Basic time display working -- ✅ WiFiManager integration -- ✅ First OTA deployment - -**2026-01-01**: v1.8 🔒 -- 🔒 **Security fixes**: - - Removed hardcoded WiFi credentials - - Added config validation (magic number check) - - Input sanitization (buffer overflow protection) -- 🛠️ **Stability fixes**: - - IRAM crisis fix (94% → 70% via ICACHE_FLASH_ATTR on 26 functions) - - NTP interval bug (config value was ignored) - - Boolean parsing errors in JSON import/export - - Infinite loop protection in display rotation -- ⚡ **Performance**: - - Chunked HTTP responses (eliminated 140+ String concatenations) - - Peak heap usage reduced by ~8KB - - Memory leaks fixed - -**2026-01-02**: v1.9.0 ⚡ -- ⚡ **Full async refactoring**: - - Async HTTP weather fetch (AsyncHTTPRequest library) - - Custom async NTP implementation (manual UDP packets) - - Async WiFi connection (state machine) - - Removed all delay() calls from loop() - - Exponential backoff retry logic -- 📊 **Performance results**: - - Loop time: 10ms → <1ms (10x improvement) - - Weather fetch: 1-10s blocking → 0ms - - NTP sync: 5-20s blocking → 0ms - - WiFi reconnect: 15s blocking → 0ms -- ❌ **Problem discovered**: - - Display blank for 10+ seconds on boot - - "DNS resolution failed" errors - -**2026-01-03**: v1.9.1 (Current) 🎯 -- 🔧 **Critical fix**: Hybrid WiFi model - - Synchronous WiFi in setup() (waits up to 10 seconds) - - Async WiFi reconnect in loop() (non-blocking) - - Ensures proper initialization order: WiFi → OTA → web → NTP -- 📺 **Display improvements**: - - Fixed sunrise/sunset labels cutoff (removed labels, kept arrows) - - Superscript degree symbol (°c) - - Daylight duration display instead of static "Sun Times" -- ✅ **Result**: Production ready, tested 24/7 - ---- - -## Current Status (v1.9.1) - -### Version Information - -**Firmware**: v1.9.1 (Production Ready) -**Released**: 2026-01-03 -**Status**: Actively running 24/7, stable - -### Memory Usage - -| Resource | Used | Total | Usage | Status | -|----------|------|-------|-------|--------| -| Flash | 408,844 | 1,048,576 | 38% | ✅ Plenty | -| RAM | 37,644 | 80,192 | 46% | ✅ Safe | -| IRAM | 61,987 | 65,536 | 94% | ⚠️ Critical but stable | - -**IRAM Note**: 94% is acceptable because: -- ICACHE_FLASH_ATTR applied to 26 functions -- Stable across v1.8 → v1.9.1 -- No growth observed in testing - -### Performance Metrics - -- **Loop time**: <1ms (was 10ms+ before async) -- **Boot to time display**: ~15 seconds -- **WiFi connection**: 5-10 seconds (synchronous in setup) -- **NTP sync interval**: Configurable (default 1 hour) -- **Weather update interval**: Configurable (default 30 minutes) - -### Uptime - -- ✅ 24+ hours stable -- ✅ No memory leaks -- ✅ No crashes or reboots -- ✅ OTA updates work during operation - ---- - -## Technical Architecture - -### Async State Machines - -#### Weather State Machine -```cpp -enum WeatherState { IDLE, REQUESTING, SUCCESS, FAILED }; -``` - -- Library: AsyncHTTPRequest_Generic v1.13.0 -- API: Open-Meteo (free, no API key) -- Callback: `onWeatherResponse()` -- Retry: Exponential backoff (1s → 2s → 4s, max 3 retries) - -#### NTP State Machine -```cpp -enum NTPState { IDLE, REQUEST_SENT, WAITING, SUCCESS, FAILED }; -``` - -- Custom manual UDP packet building/parsing -- Independent epoch tracking: `syncedEpoch`, `syncedMillis`, `timeIsSynced` -- Non-blocking UDP checks via `parsePacket()` -- Timeout: 5 seconds -- Why manual? NTPClient library is inherently blocking - -#### WiFi State Machine -```cpp -enum WiFiConnectionState { IDLE, CONNECTING, CONNECTED, FAILED }; -``` - -**Hybrid model** (critical for v1.9.1): -- **Setup phase**: Synchronous (waits up to 10 seconds) - - Why? OTA, web server, NTP all need WiFi ready - - Prevents "DNS resolution failed" errors - - Ensures time appears on display immediately after WiFi connects -- **Loop phase**: Asynchronous (checks every 5 seconds) - - Why? Don't freeze device if WiFi drops during operation - - Graceful reconnection without user impact - -### Configuration Storage - -**EEPROM struct (512 bytes, 26 fields):** - -```cpp -struct Config { - char ssid[32]; // WiFi network name - char password[64]; // WiFi password - int timezone_offset; // Seconds from UTC - bool dst_enabled; // Auto DST (European rules) - uint8_t brightness; // Display brightness (0-7) - char ntp_server[64]; // NTP server address - unsigned long ntp_interval; // NTP sync interval (seconds) - bool hour_format_24; // 24h vs 12h display - char hostname[32]; // mDNS hostname - float latitude; // Weather location - float longitude; // Weather location - char city_name[32]; // Display in weather mode - bool weather_enabled; // Feature toggle - unsigned long weather_interval; // Update interval (seconds) - unsigned long display_rotation_sec; // Mode switch interval - bool show_weather; // Enable weather mode - bool show_sunrise_sunset; // Enable sunrise/sunset mode - uint8_t display_orientation; // Screen rotation (0-3) - uint32_t magic; // 0xC10CC10C - validation -}; -``` - -**Validation**: Magic number check prevents loading corrupted EEPROM data. - -### Display Modes - -**Mode 1: Time Mode** -- Large HH:MM display -- Blinking colon (500ms interval) -- Day of week + date -- 24/12 hour format support - -**Mode 2: Weather Mode** -- Temperature with superscript °c -- City name at bottom -- Example: "15.4°c" + "Portimao" - -**Mode 3: Sunrise/Sunset Mode** -- Sunrise time with ↑ arrow -- Sunset time with ↓ arrow -- Daylight duration (e.g., "Day 9h 41m") -- Calculation: sunset - sunrise = total daylight minutes - -**Rotation**: Configurable interval (default 5 seconds), gracefully skips disabled modes. - -### Memory Optimization Techniques - -**1. ICACHE_FLASH_ATTR** - -Applied to 26 functions to move code from IRAM to Flash: -- All web handlers (15 functions) -- Setup functions (5 functions) -- Utilities (6 functions) - -Result: IRAM usage manageable at 94% - -**2. Chunked HTTP Responses** - -```cpp -// ❌ BAD - 140+ concatenations -String html = ""; -html += F(""); -html += F("..."); - -// ✅ GOOD - Chunked transfer -server.setContentLength(CONTENT_LENGTH_UNKNOWN); -server.send(200, "text/html", ""); -server.sendContent_P(HTML_HEADER); -server.sendContent_P(HTML_FOOTER); -server.sendContent(""); -``` - -Result: ~8KB peak heap reduction - -**3. Fixed-Size Buffers** - -No dynamic String allocations in loops: -```cpp -char buf[150]; -snprintf_P(buf, sizeof(buf), PSTR("
IP: %s
"), - WiFi.localIP().toString().c_str()); -``` - ---- - -## Security Issues Fixed - -### Original Firmware Vulnerabilities - -**1. WiFi Password Leak (CRITICAL)** -- Open access point remained active after setup -- Web interface displayed WiFi password in plaintext -- Anyone within range could connect and read password -- **Impact**: Full network compromise - -**2. Cloud Dependency** -- Required QWeather API (Chinese service) -- Needed account registration + API key -- Unknown data collection practices -- **Impact**: Privacy concerns, vendor lock-in - -**3. No OTA Updates** -- Required physical FTDI connection for updates -- Difficult for non-technical users -- **Impact**: Security vulnerabilities can't be patched remotely - -**4. Hardcoded Credentials** -- Default WiFi credentials in source code -- No secure setup flow -- **Impact**: Easy attack vector - -### Custom Firmware Solutions - -**1. WiFiManager Integration ✅** -- Captive portal for secure first-time setup -- AP automatically closes after 180 seconds -- Fallback AP only on connection failure -- Password-protected fallback (customizable) - -**2. Open-Meteo API ✅** -- Free weather service -- No registration required -- No API key needed -- European service (GDPR compliant) - -**3. OTA Updates ✅** -- Web-based upload at `/update` -- ArduinoOTA for IDE uploads -- Password-protected (admin/admin - customizable) -- Non-blocking during operation - -**4. No Hardcoded Secrets ✅** -- All credentials stored in EEPROM -- Magic number validation -- Factory reset capability -- Configuration import/export - ---- - -## Files & Structure - -### Project Directory - -**Location**: `/Users/apetrochenko/Library/Mobile Documents/com~apple~CloudDocs/src/arduino/clock/esp8266-weather-clock-opensource` - -### Key Files - -**Root Level:** -- `README.md` (26KB) - Main documentation (blog-style) -- `LICENSE` - MIT License -- `CHANGELOG.md` - Version history -- `CONTRIBUTING.md` - Contribution guidelines -- `PROJECT_STRUCTURE.md` - Directory layout -- `PROJECT_CONTEXT.md` - This file -- `PUBLISH_TO_GITHUB.md` - GitHub publishing guide -- `.gitignore` - Git exclusions - -**Source Code:** -- `src/clock_ntp_ota_v1.9.ino` (65KB, 2,096 lines) - -**Documentation:** -- `docs/INSTALLATION.md` (18KB) - Complete installation guide -- `docs/HARDWARE.md` (8KB) - Hardware specs + pinout -- `docs/v1.9_RELEASE_NOTES.md` (7KB) - v1.9.0 changelog -- `docs/v1.9.1_HYBRID_FIX.md` (6KB, Russian) - v1.9.1 fix explanation - -**Images:** -- `images/product/` - 6 AliExpress product photos - - 01-main-product.webp - - 02-components.webp - - 03-weather-forecast.webp - - 04-temperature-display.webp - - 05-details.webp - - 06-size.webp -- `images/build/` - 3 display screenshots - - display-time.png - - display-temperature.png - - display-sunrise-sunset.png - -**GitHub Config:** -- `.github/workflows/build.yml` - CI/CD pipeline -- `.github/ISSUE_TEMPLATE/bug_report.md` -- `.github/ISSUE_TEMPLATE/feature_request.md` - -### Compiled Artifacts (not in git) - -``` -build/ -├── clock_ntp_ota_v1.9.ino.bin (409KB) - Flash this via OTA -├── clock_ntp_ota_v1.9.ino.elf - Debug symbols -└── clock_ntp_ota_v1.9.ino.map - Memory map -``` - ---- - -## Git Repository - -### GitHub Details - -- **Repository**: https://github.com/petrochen/esp8266-weather-clock-opensource -- **Owner**: petrochen (Andrey Petrochenko) -- **Visibility**: Public -- **License**: MIT -- **Created**: 2026-01-03 - -### Repository Configuration - -**Enabled Features:** -- ✅ Issues -- ✅ Discussions -- ✅ GitHub Actions (CI/CD) -- ✅ Releases - -**Topics (Tags):** -- `esp8266`, `arduino`, `iot`, `weather-station` -- `reverse-engineering`, `security`, `oled-display` -- `ntp`, `ota-updates`, `open-meteo` - -**Badges:** -- Release version (auto-updates) -- License (MIT) -- Build status (CI/CD) -- Open issues count -- Hardware (ESP-01S) -- Status (Production Ready) - -### Current Release - -**Tag**: v1.9.1 -**URL**: https://github.com/petrochen/esp8266-weather-clock-opensource/releases/tag/v1.9.1 -**Status**: Production Ready -**Created**: 2026-01-03 - -### Git Commits - -**Total**: 3 commits -1. `8f0df02` - Initial commit: v1.9.1 production firmware -2. `f2dc2fb` - Add dynamic GitHub badges to README -3. `2205445` - Update price: $12 → €5 - -### Web Interface URLs - -- **Main**: https://github.com/petrochen/esp8266-weather-clock-opensource -- **Issues**: https://github.com/petrochen/esp8266-weather-clock-opensource/issues -- **Discussions**: https://github.com/petrochen/esp8266-weather-clock-opensource/discussions -- **Actions**: https://github.com/petrochen/esp8266-weather-clock-opensource/actions -- **Releases**: https://github.com/petrochen/esp8266-weather-clock-opensource/releases - ---- - -## Next Steps - -### Planned Features (v1.10 or v2.0) - -**1. Home Assistant Integration (High Priority)** - -User specifically wants custom display screens pulling data from Home Assistant. - -**Implementation ideas:** -- Add REST API client to fetch HA sensor data -- New display modes: - - Energy usage dashboard - - Room temperatures (multiple sensors) - - Air quality / CO2 levels - - Automation states (alarm, doors, lights) -- Configuration: HA server URL, access token, entity IDs -- Update interval: configurable (default 30 seconds) - -**Technical approach:** -- Use AsyncHTTPRequest for non-blocking HA API calls -- JSON parsing with ArduinoJson library -- Store HA config in EEPROM (new fields) -- New web UI section for HA configuration - -**2. MQTT Support** - -- Publish time/weather data to MQTT broker -- Subscribe to topics for display content -- Enable automation triggers -- Library: PubSubClient (async wrapper needed) - -**3. WebSocket Live Updates** - -- Replace polling with WebSocket -- Real-time config changes -- Live display preview in web UI -- Push notifications for updates - -**4. Multiple Weather Locations** - -- Store 2-3 favorite locations -- Rotate between them -- Useful for travelers or multiple homes - -**5. Display Animations** - -- Smooth transitions between modes -- Weather icons (sunny, cloudy, rainy) -- Sunrise/sunset animations - -### Code Quality Improvements - -**1. Modular Architecture (v2.0)** - -Split monolith (2,096 lines) into modules: -``` -src/ -├── main.ino -├── config.h -├── display.cpp/h -├── network.cpp/h -├── weather.cpp/h -├── webserver.cpp/h -└── home_assistant.cpp/h (new) -``` - -**2. ArduinoJson Integration** - -Replace manual JSON parsing (173 lines) with library: -- Cleaner code -- Better error handling -- Type safety - -**3. Constants Organization** - -Eliminate magic numbers: -```cpp -namespace Hardware { - constexpr uint8_t I2C_SDA_PIN = 0; - constexpr uint8_t I2C_SCL_PIN = 2; -} -``` - -**4. Unit Tests** - -- Test state machines -- Test JSON parsing -- Test time calculations -- Mock network calls - -### Documentation Improvements - -**1. Video Tutorial** - -- YouTube walkthrough -- FTDI connection demo -- OTA update demo -- Web configuration walkthrough - -**2. Troubleshooting Flow Chart** - -Visual guide for common issues: -- Display not working -- WiFi not connecting -- Time not syncing -- Weather not updating - -**3. Localization** - -Translate docs to: -- Russian (v1.9.1_HYBRID_FIX.md already in Russian) -- Spanish -- Portuguese - -**4. Home Assistant Integration Guide** - -Complete guide when HA support is added. - -### Community Engagement - -**1. Reddit Posts** - -Subreddits to share on: -- r/esp8266 -- r/arduino -- r/selfhosted -- r/homeassistant (after HA integration) -- r/ReverseEngineering - -**2. Hackaday** - -Submit project tip: https://hackaday.com/submit-a-tip/ - -**3. Hackster.io** - -Create full project page with build guide. - -**4. Awesome Lists** - -Add to "awesome ESP8266" and "awesome IoT" lists. - ---- - -## Key Learnings - -### Hardware - -1. **Always verify pinouts** - Don't assume standard mappings -2. **FTDI is essential** - $2 adapter unlocks any ESP8266 device -3. **Read PCB markings** - Model numbers save hours of guessing -4. **Test voltage** - ESP8266 is NOT 5V tolerant -5. **Document discoveries** - Pin mappings, I2C addresses, display models - -### Software - -1. **Async is hard but worth it** - Fully non-blocking eliminates freezes -2. **IRAM is precious** - Use ICACHE_FLASH_ATTR liberally on ESP8266 -3. **Hybrid approaches work** - Don't be dogmatic (sync WiFi in setup() was correct) -4. **State machines scale** - Better than callback hell for complex async -5. **Test on real hardware** - Emulators miss pin issues and memory constraints - -### Security - -1. **IoT security is often terrible** - Always audit before trusting -2. **Open source is safer** - Closed firmware is a black box -3. **Defaults matter** - Insecure defaults (open AP, plaintext passwords) are vulnerabilities -4. **Defense in depth** - Multiple layers catch mistakes -5. **Update mechanism is critical** - OTA enables security patches - -### Development - -1. **OTA from day 1** - FTDI flashing gets old fast -2. **Version control** - Backups (.bak, .bak2) saved the project multiple times -3. **Document as you go** - Release notes prevent "what was I thinking?" moments -4. **Incremental improvements** - v1.7 → v1.8 → v1.9.x made debugging manageable -5. **User testing** - Photos from user revealed display cutoff issues - -### Project Management - -1. **Understand user needs** - User wanted Home Assistant integration (plan for it) -2. **Security first** - Privacy/security was main motivation -3. **Performance matters** - 10ms loop → <1ms dramatically improves UX -4. **Documentation is product** - Good docs = more users = more contributors -5. **Publish early** - GitHub repo enables community contributions - ---- - -## Quick Reference - -### Current Device Configuration - -**Hardware:** -- Device: TJ-56-654 Weather Clock -- Location: User's home network -- IP: 192.168.2.47 -- Hostname: tj56654-clock.local - -**Software:** -- Firmware: v1.9.1 -- WiFi: SibWings -- Weather: Portimao, Portugal (37.19°N, 8.54°W) -- Timezone: UTC+0 (Lisbon) with auto DST -- NTP: pool.ntp.org (1 hour interval) - -**Access:** -- Web UI: http://192.168.2.47 or http://tj56654-clock.local -- OTA Update: http://192.168.2.47/update (admin/admin) -- API Base: http://192.168.2.47/api/ - -### Important Commands - -**Compile:** -```bash -arduino-cli compile --fqbn esp8266:esp8266:generic \ - src/clock_ntp_ota_v1.9.ino -``` - -**Upload OTA:** -```bash -curl -u admin:admin \ - -F "file=@build/clock_ntp_ota_v1.9.ino.bin" \ - http://192.168.2.47/update -``` - -**Check status:** -```bash -curl http://192.168.2.47/api/status | jq -``` - -**View logs:** -```bash -# Connect via serial (if FTDI attached) -screen /dev/cu.usbserial* 115200 -``` - -### Library Dependencies - -Required libraries (install via Arduino Library Manager): - -| Library | Version | Purpose | -|---------|---------|---------| -| Adafruit GFX Library | 1.11.0+ | Graphics primitives | -| Adafruit SSD1306 | 2.5.0+ | OLED display driver | -| NTPClient | 3.2.0+ | NTP time sync (base) | -| WiFiManager | 2.0.0+ | Captive portal | -| AsyncHTTPRequest_Generic | 1.13.0+ | Async weather fetch | -| ESPAsyncTCP | 1.2.2+ | Async TCP layer | - -### External APIs - -**Open-Meteo:** -- URL: https://api.open-meteo.com/v1/forecast -- Authentication: None (free, no API key) -- Rate limit: None (reasonable use) -- Documentation: https://open-meteo.com/en/docs - -**NTP:** -- Default: pool.ntp.org -- Protocol: UDP port 123 -- Fallbacks: time.google.com, time.cloudflare.com - ---- - -## Contact & Collaboration - -**GitHub**: https://github.com/petrochen/esp8266-weather-clock-opensource -**Issues**: https://github.com/petrochen/esp8266-weather-clock-opensource/issues -**Discussions**: https://github.com/petrochen/esp8266-weather-clock-opensource/discussions - -**Contributions welcome!** See [CONTRIBUTING.md](CONTRIBUTING.md) - ---- - -## Summary - -This project successfully transformed a €5 AliExpress IoT device with critical security vulnerabilities into a fully secure, high-performance, feature-rich smart clock with open-source firmware. - -**Key achievements:** -- ✅ Eliminated WiFi password leak vulnerability -- ✅ Achieved <1ms loop time (10x performance improvement) -- ✅ Enabled OTA updates (no more FTDI wiring) -- ✅ Free weather API (no registration) -- ✅ Full async architecture (zero blocking) -- ✅ Production-ready stability (24/7 uptime) -- ✅ Open-source on GitHub (MIT license) -- ✅ Comprehensive documentation (100KB+ docs) - -**Next milestone**: Home Assistant integration for custom display screens. - -**Status**: Project complete and ready for community contributions! 🚀 - ---- - -**Last session work (2026-01-03):** -1. ✅ Compiled v1.9.1 with daylight duration feature -2. ✅ Uploaded via OTA to device (192.168.2.47) -3. ✅ Created complete GitHub repository structure -4. ✅ Published to https://github.com/petrochen/esp8266-weather-clock-opensource -5. ✅ Created release v1.9.1 -6. ✅ Added badges, topics, documentation -7. ✅ Fixed price ($12 → €5) -8. ✅ Saved full project context - -**Repository ready for sharing on Reddit, Hackaday, and other communities!** diff --git a/PROJECT_STRUCTURE.md b/PROJECT_STRUCTURE.md deleted file mode 100644 index 1650d64..0000000 --- a/PROJECT_STRUCTURE.md +++ /dev/null @@ -1,249 +0,0 @@ -# Project Structure - -This document describes the organization of the ESP8266 Weather Clock firmware repository. - -## Directory Layout - -``` -esp8266-weather-clock-opensource/ -│ -├── README.md # Main documentation (start here!) -├── LICENSE # MIT License -├── CHANGELOG.md # Version history -├── CONTRIBUTING.md # Contribution guidelines -├── PROJECT_STRUCTURE.md # This file -├── .gitignore # Git exclusions -│ -├── src/ # Source code -│ └── clock_ntp_ota_v1.9.ino # Main firmware (2,096 lines) -│ -├── docs/ # Documentation -│ ├── INSTALLATION.md # Complete installation guide -│ ├── HARDWARE.md # Hardware specs and pinout -│ ├── v1.9_RELEASE_NOTES.md # v1.9.0 release notes -│ └── v1.9.1_HYBRID_FIX.md # v1.9.1 WiFi startup fix -│ -├── images/ # Photos and screenshots -│ ├── product/ # AliExpress product photos -│ │ ├── 01-main-product.webp # Main product shot -│ │ ├── 02-components.webp # Kit components -│ │ ├── 03-weather-forecast.webp -│ │ ├── 04-temperature-display.webp -│ │ ├── 05-details.webp # Transparent case details -│ │ └── 06-size.webp # Dimensions (40x40x43mm) -│ │ -│ └── build/ # Custom firmware screenshots -│ ├── display-time.png # Time display mode -│ ├── display-temperature.png # Weather display mode -│ └── display-sunrise-sunset.png # Solar display mode -│ -└── .github/ # GitHub-specific files - ├── workflows/ - │ └── build.yml # CI: Auto-build on push - │ - └── ISSUE_TEMPLATE/ - ├── bug_report.md # Bug report template - └── feature_request.md # Feature request template -``` - -## Key Files - -### Root Level - -**README.md** (11KB) -- Main project documentation -- Blog-style narrative about reverse engineering -- Security issues discovered -- Complete feature list -- Installation quickstart -- API documentation - -**LICENSE** (MIT) -- Permissive open source license -- Use freely, modify, distribute - -**CHANGELOG.md** -- Version history: v1.5 → v1.9.1 -- Features, fixes, breaking changes -- Migration notes - -**CONTRIBUTING.md** -- How to contribute -- Code style guidelines -- Testing requirements - -### Source Code (`/src`) - -**clock_ntp_ota_v1.9.ino** -- Main firmware file (2,096 lines) -- ESP8266 Arduino sketch -- Requires libraries: - - Adafruit GFX & SSD1306 - - NTPClient - - WiFiManager - - AsyncHTTPRequest_Generic - - ESPAsyncTCP - -**Architecture:** -- Fully async (zero blocking in loop) -- State machines: WiFi, NTP, Weather -- Hybrid model: sync WiFi in setup(), async in loop() -- Memory-optimized: ICACHE_FLASH_ATTR on 26 functions - -**Configuration:** -- 26-field struct stored in EEPROM -- Magic number validation -- Web-based config UI - -### Documentation (`/docs`) - -**INSTALLATION.md** (18KB) -- Complete step-by-step installation guide -- Arduino IDE setup -- FTDI wiring diagrams -- OTA update instructions -- Comprehensive troubleshooting - -**HARDWARE.md** (8KB) -- ESP-01S specifications -- Pin mapping (SDA=GPIO0, SCL=GPIO2) -- Display module details (GM009605v4.3) -- Power requirements -- Memory layout -- Safety warnings - -**v1.9_RELEASE_NOTES.md** (7KB) -- Detailed v1.9.0 changelog -- Performance improvements -- Async architecture explanation -- Memory usage comparison -- Testing checklist - -**v1.9.1_HYBRID_FIX.md** (Russian, 6KB) -- Critical startup fix documentation -- WiFi synchronous vs async tradeoffs -- Timeline diagrams -- Before/after comparison - -### Images (`/images`) - -**Product Photos** (`/product`) -- Original AliExpress product images -- DIY kit components -- Transparent acrylic case -- Size reference (40mm cube) - -**Build Photos** (`/build`) -- Custom firmware screenshots -- Three display modes: - 1. Time mode (10:34 + date) - 2. Weather mode (15.4°c + city) - 3. Sunrise/sunset mode (times + daylight duration) - -### GitHub Config (`/.github`) - -**Workflows** -- `build.yml`: CI pipeline - - Auto-compile on push - - Check firmware size < 470KB - - Upload build artifacts - - Attach binaries to releases - -**Issue Templates** -- `bug_report.md`: Structured bug reports -- `feature_request.md`: Feature suggestions - -## Build Artifacts (ignored by git) - -When you compile locally, these are created: - -``` -build/ -├── clock_ntp_ota_v1.9.ino.bin # Flash this via OTA -├── clock_ntp_ota_v1.9.ino.elf # Debug symbols -└── clock_ntp_ota_v1.9.ino.map # Memory map -``` - -**Note**: `build/` is in `.gitignore` - artifacts not committed to repo. - -## File Sizes - -| File | Size | Description | -|------|------|-------------| -| `src/*.ino` | 65KB | Main source code | -| `build/*.bin` | 409KB | Compiled firmware | -| `README.md` | 45KB | Main docs | -| `docs/INSTALLATION.md` | 18KB | Install guide | -| `docs/HARDWARE.md` | 8KB | Hardware specs | - -## Memory Usage - -**Compiled firmware (v1.9.1):** -- Flash: 408,844 / 1,048,576 bytes (38%) -- RAM: 37,644 / 80,192 bytes (46%) -- IRAM: 61,987 / 65,536 bytes (94%) ⚠️ - -**Why 94% IRAM is acceptable:** -- ICACHE_FLASH_ATTR applied to all web handlers -- Stable across versions v1.8-v1.9.1 -- No IRAM growth observed in testing - -## Version Control - -**Branches:** -- `main`: Stable releases (v1.9.1) -- `develop`: Work-in-progress features -- `feature/*`: New feature branches - -**Tags:** -- `v1.9.1`: Current production release -- `v1.9.0`: Async refactoring -- `v1.8.0`: Security + stability fixes -- `v1.7.0`: Initial working firmware - -## Not Included (Why) - -**What's NOT in this repo:** -- Build artifacts (`.bin`, `.elf`, `.map`) - generated locally -- Backup files (`.bak`, `.bak2`) - development artifacts -- IDE configs (`.vscode/`, `.idea/`) - personal preferences -- macOS metadata (`.DS_Store`) - system files -- Secrets (`config_local.h`) - would leak credentials - -These are excluded via `.gitignore`. - -## How to Navigate - -**For users:** -1. Start with `README.md` (overview + quickstart) -2. Follow `docs/INSTALLATION.md` (step-by-step setup) -3. Check `CHANGELOG.md` (version history) - -**For developers:** -1. Read `CONTRIBUTING.md` (guidelines) -2. Study `src/clock_ntp_ota_v1.9.ino` (source code) -3. Review `docs/HARDWARE.md` (hardware constraints) -4. Check `.github/workflows/build.yml` (CI setup) - -**For hardware hackers:** -1. Check `docs/HARDWARE.md` (pinout, specs) -2. View `images/product/` (original device photos) -3. Read `README.md` section "Hardware Discovery" - -**For troubleshooters:** -1. Open `docs/INSTALLATION.md` -2. Jump to "Troubleshooting" section -3. Check `images/build/` for reference screenshots - -## Quick Links - -- **Main docs**: [README.md](README.md) -- **Install guide**: [docs/INSTALLATION.md](docs/INSTALLATION.md) -- **Hardware specs**: [docs/HARDWARE.md](docs/HARDWARE.md) -- **Changelog**: [CHANGELOG.md](CHANGELOG.md) -- **Contributing**: [CONTRIBUTING.md](CONTRIBUTING.md) - ---- - -**Last updated**: 2026-01-03 -**Repository**: https://github.com/your-username/esp8266-weather-clock-opensource diff --git a/PUBLISH_TO_GITHUB.md b/PUBLISH_TO_GITHUB.md deleted file mode 100644 index 0325cd4..0000000 --- a/PUBLISH_TO_GITHUB.md +++ /dev/null @@ -1,440 +0,0 @@ -# Publishing to GitHub - Step by Step Guide - -This file contains instructions for publishing this project to GitHub. - -## Prerequisites - -1. **GitHub account** - Sign up at https://github.com if you don't have one -2. **Git installed** - Check with `git --version` in terminal -3. **GitHub CLI (optional)** - Makes repository creation easier: https://cli.github.com/ - ---- - -## Option 1: Using GitHub Web Interface (Easiest) - -### Step 1: Create Repository on GitHub - -1. Go to https://github.com/new -2. Fill in: - - **Repository name**: `esp8266-weather-clock-opensource` - - **Description**: `Secure open-source firmware for ESP8266 weather clock - reverse engineered from AliExpress DIY kit` - - **Visibility**: Public ✅ - - **Initialize**: ❌ Do NOT check "Add README" (we have one) -3. Click: **Create repository** - -### Step 2: Initialize Local Git Repository - -Open terminal and navigate to project directory: - -```bash -cd "/Users/apetrochenko/Library/Mobile Documents/com~apple~CloudDocs/src/arduino/clock/esp8266-weather-clock-opensource" -``` - -Initialize git and add files: - -```bash -# Initialize git -git init - -# Add all files -git add . - -# Create first commit -git commit -m "Initial commit: v1.9.1 production firmware - -- Complete reverse engineering of TJ-56-654 weather clock -- Fixes security issues (WiFi password leak) -- Fully async architecture (zero blocking) -- OTA updates, web interface, REST API -- Open-Meteo weather (free, no API key) -- Comprehensive documentation" -``` - -### Step 3: Connect to GitHub - -Replace `YOUR_USERNAME` with your actual GitHub username: - -```bash -# Add remote -git remote add origin https://github.com/YOUR_USERNAME/esp8266-weather-clock-opensource.git - -# Set main branch -git branch -M main - -# Push to GitHub -git push -u origin main -``` - -**If prompted for credentials:** -- Username: Your GitHub username -- Password: Use **Personal Access Token** (not your password!) - - Create token at: https://github.com/settings/tokens - - Select scopes: `repo` (full control of private repositories) - -### Step 4: Verify Upload - -1. Browse to: `https://github.com/YOUR_USERNAME/esp8266-weather-clock-opensource` -2. You should see: - - README.md rendered nicely - - All directories and files - - First commit visible - ---- - -## Option 2: Using GitHub CLI (Faster) - -If you have GitHub CLI installed: - -```bash -# Navigate to project -cd "/Users/apetrochenko/Library/Mobile Documents/com~apple~CloudDocs/src/arduino/clock/esp8266-weather-clock-opensource" - -# Authenticate (one-time) -gh auth login - -# Create repo and push in one command -gh repo create esp8266-weather-clock-opensource \ - --public \ - --source=. \ - --description="Secure open-source firmware for ESP8266 weather clock" \ - --push -``` - -Done! Repository is created and pushed. - ---- - -## Step 5: Configure Repository Settings - -### Add Topics (Tags) - -1. Go to your repo on GitHub -2. Click: **⚙️ Settings** (top right near About) -3. Under "Topics", add: - - `esp8266` - - `arduino` - - `iot` - - `weather-station` - - `reverse-engineering` - - `security` - - `oled-display` - - `ntp` - - `ota-updates` - - `open-meteo` - -### Update About Section - -1. Go to repo main page -2. Click: **⚙️** (gear icon) next to "About" -3. Set: - - **Description**: `Secure open-source firmware for ESP8266 weather clock - reverse engineered from AliExpress DIY kit to fix security flaws` - - **Website**: `https://open-meteo.com` (or your personal site if you blog about it) - - **Topics**: Should already be set from above - -### Enable Features - -In **Settings → General**: - -**Features**: -- ✅ Issues (for bug reports) -- ✅ Discussions (for questions) -- ❌ Wiki (not needed, we have docs/) -- ❌ Projects (not needed yet) - -**Pull Requests**: -- ✅ Allow squash merging -- ✅ Automatically delete head branches - -### Set Up GitHub Actions - -The CI workflow should activate automatically on first push. Check: -1. Go to: **Actions** tab -2. You should see: "Build Firmware" workflow -3. It should run and ✅ pass (compiles firmware) - -If it fails: -- Check library names in `.github/workflows/build.yml` -- Some libraries may need exact version pinning - ---- - -## Step 6: Create First Release - -### Tag the Release Locally - -```bash -# Create annotated tag -git tag -a v1.9.1 -m "Release v1.9.1: Production-ready firmware - -Features: -- Hybrid WiFi model (sync on boot, async in loop) -- Daylight duration display -- Fully async NTP, weather, WiFi reconnect -- OTA updates, web interface, REST API -- Open-Meteo weather (free API) -- Security fixes (no WiFi password leak) - -Fixes: -- Startup display blank for 10+ seconds -- DNS resolution failed errors -- Sunrise/sunset label cutoff" - -# Push tag to GitHub -git push origin v1.9.1 -``` - -### Create Release on GitHub - -1. Go to: **Releases** (right sidebar) -2. Click: **Draft a new release** -3. Fill in: - - **Tag**: `v1.9.1` (should appear in dropdown) - - **Release title**: `v1.9.1 - Production Ready` - - **Description**: - ```markdown - ## 🎉 First Public Release - - Secure, open-source replacement firmware for ESP8266 weather clocks. - - ### ✨ Highlights - - **Security**: Fixes WiFi password leak in original firmware - - **Performance**: Fully async architecture, <1ms loop time - - **Features**: OTA updates, web UI, REST API, NTP time, weather - - **Free API**: Uses Open-Meteo (no registration required) - - ### 📦 Downloads - - `esp8266-weather-clock-v1.9.1.bin` - Flash this via OTA or FTDI - - ### 📖 Documentation - - [Installation Guide](docs/INSTALLATION.md) - - [Hardware Specs](docs/HARDWARE.md) - - [Full Changelog](CHANGELOG.md) - - ### 🚀 Quick Start - 1. Download `.bin` file - 2. Flash via FTDI (first time) or OTA (updates) - 3. Connect to `TJ56654-Setup` WiFi - 4. Configure your network - 5. Access web UI at `http://tj56654-clock.local` - - See [README](README.md) for complete instructions. - - ### 🐛 Known Issues - None! This release is production-ready and tested 24/7. - ``` - -4. **Attach binary** (if you have it locally): - - Compile firmware first: Arduino IDE → Sketch → Export Compiled Binary - - Or use GitHub Actions artifact - - Drag `build/clock_ntp_ota_v1.9.ino.bin` to release assets - - Rename to: `esp8266-weather-clock-v1.9.1.bin` - -5. Click: **Publish release** - ---- - -## Step 7: Add Shields/Badges to README - -Edit `README.md` and add at the top (after title): - -```markdown -

- - Release - - - License - - - Build - - - Issues - -

-``` - -Replace `YOUR_USERNAME` with actual username. - -Commit and push: -```bash -git add README.md -git commit -m "Add badges to README" -git push -``` - ---- - -## Step 8: Share Your Project - -### Post on Social Media - -**Reddit:** -- r/esp8266 -- r/arduino -- r/selfhosted -- r/homeassistant (when you add HA integration) - -**Hackaday:** -- Submit project tip: https://hackaday.com/submit-a-tip/ - -**Hackster.io:** -- Create project page: https://www.hackster.io/ - -**Twitter/X:** -``` -Just reverse-engineered a $12 AliExpress weather clock and found it was leaking WiFi passwords! - -Replaced the firmware with secure open-source version: -- ✅ No password leak -- ✅ OTA updates -- ✅ Free weather API -- ✅ Full async arch - -Check it out: [your-repo-link] - -#ESP8266 #IoTSecurity #Arduino -``` - -### Add to Awesome Lists - -Search for "awesome ESP8266" and submit PR to add your project. - ---- - -## Maintenance Tips - -### Keep README Updated - -When you add features: -1. Update README.md -2. Update CHANGELOG.md -3. Create new git tag -4. Create GitHub release - -### Respond to Issues - -Enable email notifications: -1. Go to: repo → **Watch** → **Custom** -2. Check: ✅ Issues, ✅ Pull requests, ✅ Discussions - -### Version Numbering - -Use semantic versioning (semver.org): -- `v2.0.0`: Breaking changes (incompatible config) -- `v1.10.0`: New features (backward-compatible) -- `v1.9.2`: Bug fixes only - -### Automated Releases - -GitHub Actions can auto-build on new tags. Check `.github/workflows/build.yml`. - ---- - -## Troubleshooting - -### "Permission denied" when pushing - -**Solution**: Use Personal Access Token instead of password -1. Generate: https://github.com/settings/tokens -2. Scopes: `repo` (full control) -3. Use token as password when prompted - -Or configure SSH keys: -```bash -# Generate SSH key -ssh-keygen -t ed25519 -C "your_email@example.com" - -# Add to GitHub: Settings → SSH Keys → New SSH key -# Paste contents of ~/.ssh/id_ed25519.pub - -# Change remote to SSH -git remote set-url origin git@github.com:YOUR_USERNAME/esp8266-weather-clock-opensource.git -``` - -### "This repository is empty" - -You forgot to push: -```bash -git push -u origin main -``` - -### Files too large - -GitHub has 100MB file size limit. If you accidentally added build artifacts: -```bash -# Remove from staging -git reset HEAD build/ - -# Add to .gitignore -echo "build/" >> .gitignore - -# Commit -git commit -m "Ignore build artifacts" -``` - -### CI build fails - -Check: -- Library names are correct in `build.yml` -- All libraries are available via Arduino Library Manager -- Firmware compiles locally first - ---- - -## Next Steps After Publishing - -1. **Star your own repo** (to make it discoverable) -2. **Watch releases** (be notified of activity) -3. **Enable Discussions** (for community Q&A) -4. **Create SECURITY.md** (if you want responsible disclosure process) -5. **Add funding links** (GitHub Sponsors, Buy Me a Coffee, etc.) - ---- - -## GitHub Repository Best Practices - -### Essential Files (✅ You have these!) -- ✅ README.md -- ✅ LICENSE -- ✅ CONTRIBUTING.md -- ✅ CHANGELOG.md -- ✅ .gitignore -- ✅ Issue templates - -### Nice-to-Have -- CODE_OF_CONDUCT.md (for community standards) -- SECURITY.md (vulnerability disclosure policy) -- FUNDING.yml (donation links) - -### Pin Important Files - -On your repo page, pin: -1. README.md (auto-pinned) -2. INSTALLATION.md (pin in About section) -3. Latest release (pin in sidebar) - ---- - -## Success Checklist - -After publishing, verify: -- [ ] Repository is public and accessible -- [ ] README renders correctly (images, links work) -- [ ] All documentation files are present -- [ ] CI/CD pipeline passes (green checkmark) -- [ ] First release is tagged and published -- [ ] Binary is attached to release -- [ ] Topics/tags are set -- [ ] License is visible -- [ ] Issues and Discussions are enabled - ---- - -**Congratulations!** Your project is now public and ready to help the world build secure IoT devices. 🚀 - ---- - -**Repository URL**: https://github.com/YOUR_USERNAME/esp8266-weather-clock-opensource - -Don't forget to replace `YOUR_USERNAME` with your actual GitHub username!