As weather volatility increasingly affects daily life, energy use, and coastal conditions, I found myself becoming mildly obsessed with understanding what was happening in my immediate environment in real time.
That interest deepened after installing solar panels and wanting better visibility into how weather patterns affected generation, household consumption, and reliance on the grid. What began as a way to log weather station data and monitor BOM radar gradually evolved into something much broader: a self-hosted local intelligence platform.
Today, the system combines live local weather observations, radar-based rain prediction, household energy modelling, coastal and marine conditions, astronomy, and local hazard alerts into a single continuously running dashboard hosted on my own infrastructure..
1. Setting the Goal
The original goal was relatively simple:
- Capture and store real-time weather data from my own station.
- Analyse BOM radar imagery to predict approaching rain.
- Send meaningful alerts before conditions changed locally.
- Combine weather and solar generation data in one place.
Over time, the scope expanded significantly. The platform now also tracks:
- household grid import/export and tariff impact,
- coastal and marine conditions,
- sunrise, sunset, moon phases, and astronomy data,
- NSW fire and weather hazard alerts,
- and long-term environmental and energy trends.
Just as importantly, I wanted the entire system to remain self-hosted, automated, and extensible - something I could continue refining over time rather than relying on a collection of disconnected third-party services.
2. Getting the Hardware and Server Ready
The only on-site hardware is an Ecowitt WS2910 weather station mounted on the roof, publishing observations every 60 seconds.
The original platform ran on a very small AWS Lightsail instance, but as more services, prediction models, dashboards, and integrations were added, the server eventually had to be scaled up to handle the growing workload.
The stack now includes:
- Python and Flask for ingestion and APIs,
- SQLite as the operational datastore,
- scheduled polling and processing jobs,
- nginx and systemd for hosting and reliability,
- and Google Sheets / Looker Studio for historical analysis.
Despite the increasing complexity, the platform still remains lightweight enough to run continuously on a relatively inexpensive VPS.
3. Building the Data Pipeline
The Ecowitt station sends HTTP POST updates every minute to a lightweight Flask ingestion service.
Each observation is written directly into SQLite, where it becomes the central operational record for the rest of the platform.
From there, multiple background processes run continuously:
- syncing weather and energy summaries to Google Sheets,
- importing household energy usage and solar export data,
- polling BOM weather feeds and NSW RFS incident data,
- fetching marine conditions including swell and water temperature,
- caching astronomy data including moon phases and rise/set times,
- and analysing BOM radar imagery for approaching rain systems.
The result is effectively a continuously updating environmental datastore combining both local sensor data and external public feeds.
The overall flow looks roughly like this:
Ecowitt Weather Station
↓
Flask Ingestion API
↓
SQLite Datastore
├── Radar Prediction Engine
├── Energy Modelling
├── Marine & Astronomy Feeds
├── Hazard Monitoring
├── Alert Engine
└── Google Sheets / Looker Studio4. Building a Radar Prediction Engine
The radar-processing pipeline became the most interesting part of the project.
Initially, the system simply checked whether rain-coloured pixels appeared within a certain distance of my location. That worked surprisingly well, but also generated too many false positives and missed fast-moving showers.
Over time, the prediction engine became significantly more sophisticated.
I created a script that downloads the most recent BOM radar image for the Appin (IDR032) site, samples pixels around my location to detect precipitation, and—using recent frames via RadarImageTracker—infers direction, speed, and ETA. If rain is within a configured radius and moving toward Thirroul, it sends a “rain imminent” alert via the notifier and logs the event in SQLite.
The RadarImageTracker module builds on standard optical-flow methods described in the OpenCV documentation and adapted from open examples of radar-frame motion tracking. It applies the Farneback dense optical flow algorithm to successive BOM radar images to estimate the movement, direction, and speed of rain systems near Thirroul.
The system compares consecutive frames to detect movement, direction, and speed of rain cells.
If rain is within 20 km and moving toward Thirroul, the system calculates an ETA and sends an alert:
Instead of relying on a single radar frame, the system now tracks coherent rain clusters across multiple BOM radar images, estimates their movement and direction, projects their closest approach to my location, and calculates likely arrival times.
To reduce noise, the engine suppresses unstable single-frame detections and only alerts when movement patterns remain consistent across multiple updates.
The system also maintains separate concepts of:
- a larger tracking radius,
- a smaller alert radius,
- projected closest approach,
- and confidence levels for likely impact.
If rain is approaching and likely to affect my location, the system sends a push notification such as:
🌧 Rain detected 8 km away, moving SE at 32 km/h. Expected arrival: 15 minutes.Equally important, the system logs:
- confirmed rain events,
- false positives,
- missed showers,
- and completely unpredicted rainfall.
That feedback loop has become critical for tuning the prediction logic against real outcomes rather than intuition.
Radar frames and alert history are also retained for review, making it possible to analyse why certain predictions succeeded or failed.
The underlying movement analysis uses OpenCV optical-flow techniques based on the Farneback dense optical flow algorithm applied to successive BOM radar frames.
5. Adding Marine and Sky Context
Once the weather side stabilised, the platform gradually expanded into broader environmental monitoring.
Additional scripts now pull in:
- BOM tide data,
- swell height, direction, and period,
- water temperature,
- sunrise and sunset times,
- moon phase, rise, set, and upcoming lunar events.
Together, these feeds provide a far richer picture of local conditions — particularly along the coast, where weather, wind, tide, and swell interact constantly.
The astronomy layer also turned out to be surprisingly useful for contextualising seasonal and environmental changes over time.
A separate Python script runs within the same environment to pull in BOM tide tables.
That data complements rainfall and wind trends, offering a complete coastal snapshot—especially useful when the southerlies hit and swell patterns change quickly.
Both the radar and tide scripts use the same virtual environment and interpreter to keep dependencies consistent:
6. Alerts, Notifications, and Dashboards
The platform now includes multiple alerting systems operating simultaneously.
These include:
- rain imminent alerts,
- rain overhead alerts,
- BOM weather warnings,
- NSW RFS major incidents,
- wind and temperature thresholds,
- and system-health notifications.
Alerts are delivered through both:
- Pushover push notifications,
- and formatted HTML email notifications.
Each alert type has configurable cool-down periods and deduplication logic to avoid notification fatigue.
The dashboard layer has also expanded considerably over time.
The system now includes:
- live weather dashboards,
- radar overlays and prediction maps,
- alert history filtering,
- atmospheric history charts,
- marine condition summaries,
- energy overview modals,
- and prediction-accuracy tracking.
Everything is designed to work well on both desktop and mobile devices.
I integrated two alert channels:
- Pushover – instant push notifications to my phone.
- Email (SMTP) – formatted HTML messages with colour-coded headers and data tables.
Each alert type—rain imminent, rain detected, high wind, high/low temperature—has its own cool-down period to prevent spamming .
Meanwhile, Google Sheets acts as the bridge to Looker Studio, turning my SQLite data into live graphs and trend visualisations that span over a year of readings.
7. Energy Intelligence
Once the environmental platform was stable, the obvious next step was energy modelling.
The system now imports household grid usage and solar export data alongside local weather observations, making it possible to correlate weather conditions directly with energy behaviour.
The platform tracks:
- solar generation,
- household consumption,
- grid imports,
- feed-in exports,
- tariff-aware billing estimates,
- supply charges,
- and estimated solar self-consumption.
Five-minute interval imports allow detailed same-day analysis, while billing-cycle views provide longer-term cost modelling.
I also added battery simulations to estimate what different storage capacities — such as a hypothetical 15 kWh battery — would have saved across a billing period.
That turned the platform from a weather dashboard into something much closer to a live environmental and household operating layer.
8. Overcoming blockers
No project like this is complete without a long list of strange problems and edge cases.
Some of the more memorable ones included:
- HTTPS redirect loops caused by the Ecowitt station only supporting HTTP POSTs.
- Colour-indexed BOM radar images requiring RGB conversion before OpenCV analysis.
- SMTP configuration typos that took far too long to diagnose.
- Timezone inconsistencies between weather feeds, logs, and astronomy APIs.
- Expired energy-provider sessions breaking automated imports.
- Aligning radar overlays accurately against geographic coordinates.
- Handling BOM forecast fields occasionally disappearing during daytime feed updates.
- Balancing radar sensitivity to reduce false positives without missing showers.
- Scaling the infrastructure as the system grew far beyond the original scope.
Each failure usually resulted in another layer of logging, monitoring, retry logic, or tuning being added back into the platform.
9. The Result
- The final system is no longer just a weather dashboard.
It has become a continuously evolving local intelligence platform combining:
- minute-level environmental sensing,
- radar-based rain prediction,
- live household energy modelling,
- marine and astronomy context,
- local weather and fire hazard monitoring,
- historical analytics,
- and mobile-friendly dashboards and alerts.
More importantly, the platform keeps improving over time.
Every missed prediction, false alert, unusual weather pattern, or energy anomaly becomes another piece of data that can be reviewed, analysed, and used to refine the system further.
The result is effectively a live environmental operating layer for the house and surrounding coastline — built from a mix of sensors, public feeds, predictive modelling, automation, and a great deal of iterative tuning.
