mirror of
https://github.com/Akkudoktor-EOS/EOS.git
synced 2026-10-09 07:56:40 +00:00
feat(pvforecast): local pvlib provider with measurement calibration
Add PVForecastAkkudoktorLocal, which runs the modelling chain inside EOS on raw Open-Meteo irradiance instead of calling a forecast service: solar position, horizon shading, plane transposition, incidence-angle modifier, cell temperature, PVWatts DC and inverter AC. It needs no API key and serves up to 16 days at 15-minute resolution from a single hourly request, which is what keeps `optimization.tail_horizon_hours` fed - services wrapping Open-Meteo cut the horizon much shorter. Several Open-Meteo models can be listed in `weather_models` and are averaged per variable at no extra request cost. With `calibration_enabled` the provider fits itself against `measurement.pv_production_emr_keys` over the past `calibration_days`: a global scale factor plus optional per-solar-azimuth factors, each weighted by modelled energy, shrunk toward the global factor by `calibration_prior_kwh` and clamped to `[calibration_min_factor, calibration_max_factor]`. The comparison runs on past intervals, where Open-Meteo serves analysed rather than forecast weather, so it corrects the error of the PV model and not that of the weather forecast. Calibration is a scale factor on the output and never touches `userhorizon`, `surface_tilt`, `surface_azimuth` or `peakpower`. The docs say so, and say why a short window and a plant fault inside it are the two ways to end up with a misleading factor. Also add `Measurement.pv_production_total_kwh()` alongside the existing load total, and `scripts/pvforecast_backtest.py`, which scores configuration variants against the stored meter readings without waiting for new forecasts to come true.
This commit is contained in:
Andreas
@@ -34,7 +34,8 @@
|
||||
"PVForecastVrm": null,
|
||||
"PVForecastPVNode": null,
|
||||
"PVForecastForecastSolar": null,
|
||||
"PVForecastSolcast": null
|
||||
"PVForecastSolcast": null,
|
||||
"PVForecastAkkudoktorLocal": null
|
||||
},
|
||||
"planes": [
|
||||
{
|
||||
@@ -102,7 +103,8 @@
|
||||
"PVForecastVrm": null,
|
||||
"PVForecastPVNode": null,
|
||||
"PVForecastForecastSolar": null,
|
||||
"PVForecastSolcast": null
|
||||
"PVForecastSolcast": null,
|
||||
"PVForecastAkkudoktorLocal": null
|
||||
},
|
||||
"planes": [
|
||||
{
|
||||
@@ -157,7 +159,8 @@
|
||||
"PVForecastPVNode",
|
||||
"PVForecastForecastSolar",
|
||||
"PVForecastSolcast",
|
||||
"PVForecastImport"
|
||||
"PVForecastImport",
|
||||
"PVForecastAkkudoktorLocal"
|
||||
],
|
||||
"planes_peakpower": [
|
||||
5.0,
|
||||
@@ -192,6 +195,69 @@
|
||||
```
|
||||
<!-- pyml enable line-length -->
|
||||
|
||||
### Common settings for the local (pvlib) PV forecast provider
|
||||
|
||||
<!-- pyml disable line-length -->
|
||||
:::{table} pvforecast::provider_settings::PVForecastAkkudoktorLocal
|
||||
:widths: 10 10 5 5 30
|
||||
:align: left
|
||||
|
||||
| Name | Type | Read-Only | Default | Description |
|
||||
| ---- | ---- | --------- | ------- | ----------- |
|
||||
| albedo | `float` | `rw` | `0.25` | Ground albedo used for planes that do not set their own. |
|
||||
| apply_iam | `bool` | `rw` | `True` | Apply the ASHRAE incidence-angle modifier to the beam component. |
|
||||
| calibration_azimuth_bin_degrees | `int` | `rw` | `15` | Width of the solar-azimuth bins for the correction. 0 fits a single global factor only. |
|
||||
| calibration_days | `int` | `rw` | `30` | Length of the measurement window used to fit the correction. |
|
||||
| calibration_enabled | `bool` | `rw` | `False` | Correct systematic model error against measured PV production. Requires `measurement.pv_production_emr_keys` to be configured and fed. Fits a global scale factor plus per-solar-azimuth factors, which is what catches near-field shading the horizon profile misses. |
|
||||
| calibration_max_factor | `float` | `rw` | `1.5` | Upper clamp on any fitted correction factor. |
|
||||
| calibration_min_factor | `float` | `rw` | `0.5` | Lower clamp on any fitted correction factor. |
|
||||
| calibration_prior_kwh | `float` | `rw` | `5.0` | Shrinkage strength: a bin needs this much modelled energy before its own factor outweighs the global one. Higher is more conservative. |
|
||||
| forecast_days | `Optional[int]` | `rw` | `None` | Forecast horizon in days (1-16). Leave empty to derive it from `prediction.hours`, which is what keeps the optimizer's tail horizon fed. |
|
||||
| inverter_efficiency | `float` | `rw` | `0.96` | Nominal inverter efficiency (PVWatts eta_inv_nom). |
|
||||
| past_days | `Optional[int]` | `rw` | `None` | Days of past data to request (0-92). Leave empty to derive it from `prediction.historic_hours`. |
|
||||
| resolution_minutes | `int` | `rw` | `15` | Forecast resolution in minutes. 15 requests Open-Meteo's `minutely_15` block (natively resolved over Central Europe and North America, interpolated from hourly elsewhere); 60 requests the `hourly` block. |
|
||||
| shift_to_interval_start | `bool` | `rw` | `True` | Open-Meteo stamps an interval mean with the interval END. EOS labels an interval by its START, so records are shifted back by one interval. Disable only to compare like-for-like against a provider that does not. |
|
||||
| temperature_coefficient | `float` | `rw` | `-0.36` | Module power temperature coefficient in %/degC (negative). Matches the `cellCoEff` the akkudoktor.net forecast uses. |
|
||||
| transposition_model | `str` | `rw` | `perez` | pvlib sky-diffuse transposition model: isotropic, klucher, haydavies, reindl, king or perez. |
|
||||
| weather_models | `list[str]` | `rw` | `['best_match']` | Open-Meteo weather models to request. Listing more than one turns the input into a poor-man's ensemble: the members are averaged per variable, which is the cheapest reliable way to cut irradiance forecast error. Costs no extra API calls. |
|
||||
:::
|
||||
<!-- pyml enable line-length -->
|
||||
|
||||
<!-- pyml disable no-emphasis-as-heading -->
|
||||
**Example Input/Output**
|
||||
<!-- pyml enable no-emphasis-as-heading -->
|
||||
|
||||
<!-- pyml disable line-length -->
|
||||
```json
|
||||
{
|
||||
"pvforecast": {
|
||||
"provider_settings": {
|
||||
"PVForecastAkkudoktorLocal": {
|
||||
"resolution_minutes": 15,
|
||||
"forecast_days": null,
|
||||
"past_days": null,
|
||||
"weather_models": [
|
||||
"best_match"
|
||||
],
|
||||
"transposition_model": "perez",
|
||||
"albedo": 0.25,
|
||||
"inverter_efficiency": 0.96,
|
||||
"temperature_coefficient": -0.36,
|
||||
"apply_iam": true,
|
||||
"shift_to_interval_start": true,
|
||||
"calibration_enabled": true,
|
||||
"calibration_days": 30,
|
||||
"calibration_azimuth_bin_degrees": 15,
|
||||
"calibration_prior_kwh": 5.0,
|
||||
"calibration_min_factor": 0.5,
|
||||
"calibration_max_factor": 1.5
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
<!-- pyml enable line-length -->
|
||||
|
||||
### Common settings for the Solcast PV forecast provider
|
||||
|
||||
<!-- pyml disable line-length -->
|
||||
@@ -368,6 +434,7 @@
|
||||
| ---- | ---- | --------- | ------- | ----------- |
|
||||
| PVForecastForecastSolar | `Optional[akkudoktoreos.prediction.pvforecastforecastsolar.PVForecastForecastSolarCommonSettings]` | `rw` | `None` | PVForecastForecastSolar settings |
|
||||
| PVForecastImport | `Optional[akkudoktoreos.prediction.pvforecastimport.PVForecastImportCommonSettings]` | `rw` | `None` | PVForecastImport settings |
|
||||
| PVForecastAkkudoktorLocal | `Optional[akkudoktoreos.prediction.pvforecastlocal.PVForecastAkkudoktorLocalCommonSettings]` | `rw` | `None` | PVForecastAkkudoktorLocal settings |
|
||||
| PVForecastPVNode | `Optional[akkudoktoreos.prediction.pvforecastpvnode.PVForecastPVNodeCommonSettings]` | `rw` | `None` | PVForecastPVNode settings |
|
||||
| PVForecastSolcast | `Optional[akkudoktoreos.prediction.pvforecastsolcast.PVForecastSolcastCommonSettings]` | `rw` | `None` | PVForecastSolcast settings |
|
||||
| PVForecastVrm | `Optional[akkudoktoreos.prediction.pvforecastvrm.PVForecastVrmCommonSettings]` | `rw` | `None` | PVForecastVrm settings |
|
||||
@@ -387,7 +454,8 @@
|
||||
"PVForecastVrm": null,
|
||||
"PVForecastPVNode": null,
|
||||
"PVForecastForecastSolar": null,
|
||||
"PVForecastSolcast": null
|
||||
"PVForecastSolcast": null,
|
||||
"PVForecastAkkudoktorLocal": null
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -0,0 +1,68 @@
|
||||
// Node-RED function: cumulative PV production meter readings -> EOS measurement API.
|
||||
//
|
||||
// Strang:
|
||||
// inject (repeat 3600 s, msg.topic = die SQL aus dem Tutorial)
|
||||
// -> mysql "MariaDB" (DB `sensor`)
|
||||
// -> DIESE function
|
||||
// -> http request (Method: "- set by msg.method -")
|
||||
//
|
||||
// EOS speichert unter `pv_production_emr_keys` Zaehlerstaende (EMR) in kWh und
|
||||
// bildet die Differenzen selbst. Aus den Momentanleistungen in `data.solarallpower`
|
||||
// muss also erst ein monoton steigender Zaehler werden - das macht die SQL.
|
||||
//
|
||||
// Wichtig: das Startdatum in der SQL bleibt FEST. Ein rollendes
|
||||
// `NOW() - INTERVAL n DAY` verschiebt den Nullpunkt der kumulativen Summe bei
|
||||
// jedem Lauf, und EOS liest den Sprung als Produktion.
|
||||
|
||||
const BASE_URL = "http://192.168.1.151:8503";
|
||||
const KEY = "pv_produktion_emr";
|
||||
const TZ = "Europe/Berlin";
|
||||
|
||||
function toIsoWithOffset(value) {
|
||||
// DATE_FORMAT() kommt als String in lokaler Zeit zurueck. Den Offset aus dem
|
||||
// Datum selbst bilden, damit CEST und CET beide stimmen.
|
||||
const date = value instanceof Date ? value : new Date(String(value).replace(" ", "T"));
|
||||
const pad = n => String(Math.trunc(Math.abs(n))).padStart(2, "0");
|
||||
const offsetMin = -date.getTimezoneOffset();
|
||||
const sign = offsetMin >= 0 ? "+" : "-";
|
||||
return `${date.getFullYear()}-${pad(date.getMonth() + 1)}-${pad(date.getDate())}` +
|
||||
`T${pad(date.getHours())}:${pad(date.getMinutes())}:${pad(date.getSeconds())}` +
|
||||
`${sign}${pad(offsetMin / 60)}:${pad(offsetMin % 60)}`;
|
||||
}
|
||||
|
||||
const rows = Array.isArray(msg.payload) ? msg.payload : [];
|
||||
const data = {};
|
||||
let last = null;
|
||||
let skipped = 0;
|
||||
for (const row of rows) {
|
||||
const value = Number(row.emr);
|
||||
if (!Number.isFinite(value)) {
|
||||
skipped += 1;
|
||||
continue;
|
||||
}
|
||||
// Ein Zaehler laeuft nur vorwaerts. Ein Rueckschritt bedeutet eine Luecke oder
|
||||
// einen kaputten Messwert - EOS wuerde daraus eine negative Produktionsstunde
|
||||
// machen.
|
||||
if (last !== null && value < last) {
|
||||
skipped += 1;
|
||||
continue;
|
||||
}
|
||||
last = value;
|
||||
data[toIsoWithOffset(row.ts)] = Number(value.toFixed(6));
|
||||
}
|
||||
|
||||
const count = Object.keys(data).length;
|
||||
if (count === 0) {
|
||||
node.warn("Keine PV-Messwerte gefunden - topic und Zeitfenster in der SQL pruefen.");
|
||||
return null;
|
||||
}
|
||||
if (skipped > 0) {
|
||||
node.warn(`${skipped} Zeilen uebersprungen (nicht-numerisch oder Zaehler rueckwaerts).`);
|
||||
}
|
||||
node.status({ text: `${count} EMR-Werte, letzter ${last.toFixed(1)} kWh` });
|
||||
|
||||
msg.method = "PUT";
|
||||
msg.url = `${BASE_URL}/v1/measurement/series?key=${encodeURIComponent(KEY)}`;
|
||||
msg.headers = { "Content-Type": "application/json" };
|
||||
msg.payload = { data: data, dtype: "float64", tz: TZ };
|
||||
return msg;
|
||||
@@ -489,6 +489,7 @@ Configuration options:
|
||||
- `PVForecastForecastSolar`: Retrieves forecasts from the free Forecast.Solar API.
|
||||
- `PVForecastSolcast`: Retrieves forecasts from the Solcast rooftop-site API.
|
||||
- `PVForecastImport`: Imports from a file or JSON string or by endpoint data provision.
|
||||
- `PVForecastAkkudoktorLocal`: Computes the forecast inside EOS from Open-Meteo weather with pvlib.
|
||||
|
||||
- `planes[].surface_tilt`: Tilt angle from horizontal plane. Ignored for two-axis tracking.
|
||||
- `planes[].surface_azimuth`: Orientation (azimuth angle) of the (fixed) plane.
|
||||
@@ -522,6 +523,17 @@ Configuration options:
|
||||
- `provider_settings.PVForecastForecastSolar.api_key`: Forecast.Solar API key (optional).
|
||||
- `provider_settings.PVForecastSolcast.api_key`: Solcast API key.
|
||||
- `provider_settings.PVForecastSolcast.site_id`: Solcast rooftop resource (site) id.
|
||||
- `provider_settings.PVForecastAkkudoktorLocal.resolution_minutes`: 15 or 60.
|
||||
- `provider_settings.PVForecastAkkudoktorLocal.forecast_days`: 1-16, empty derives it from `prediction.hours`.
|
||||
- `provider_settings.PVForecastAkkudoktorLocal.past_days`: 0-92, empty derives it from `prediction.historic_hours`.
|
||||
- `provider_settings.PVForecastAkkudoktorLocal.weather_models`: Open-Meteo models; several are averaged.
|
||||
- `provider_settings.PVForecastAkkudoktorLocal.transposition_model`: pvlib sky-diffuse model.
|
||||
- `provider_settings.PVForecastAkkudoktorLocal.albedo`: Fallback albedo for planes without their own.
|
||||
- `provider_settings.PVForecastAkkudoktorLocal.inverter_efficiency`: Nominal inverter efficiency.
|
||||
- `provider_settings.PVForecastAkkudoktorLocal.temperature_coefficient`: Module power coefficient in %/degC.
|
||||
- `provider_settings.PVForecastAkkudoktorLocal.apply_iam`: Apply the ASHRAE incidence-angle modifier.
|
||||
- `provider_settings.PVForecastAkkudoktorLocal.shift_to_interval_start`: Relabel Open-Meteo interval-end stamps.
|
||||
- `provider_settings.PVForecastAkkudoktorLocal.calibration_*`: Self-calibration against measured PV production.
|
||||
|
||||
---
|
||||
|
||||
@@ -752,6 +764,128 @@ The prediction keys for the PV forecast data are:
|
||||
- `pvforecast_ac_power`: Total AC power (W).
|
||||
- `pvforecast_dc_power`: Total DC power (W).
|
||||
|
||||
### PVForecastAkkudoktorLocal Provider
|
||||
|
||||
The `PVForecastAkkudoktorLocal` provider does not call a PV forecast service at all. It fetches raw
|
||||
irradiance and weather from [Open-Meteo](https://open-meteo.com) and runs the whole modelling
|
||||
chain locally with `pvlib`:
|
||||
|
||||
solar position -> horizon shading -> transposition to the module plane ->
|
||||
incidence-angle modifier -> cell temperature -> PVWatts DC -> inverter AC
|
||||
|
||||
Three properties make it the right default for long-horizon optimization:
|
||||
|
||||
- **Horizon.** Up to 16 forecast days at 15-minute resolution from a single request. Services
|
||||
that wrap Open-Meteo cut the horizon much shorter, which starves
|
||||
`optimization.tail_horizon_hours`.
|
||||
- **Call budget.** One request per hour against a ~10k/day non-commercial budget, instead of
|
||||
competing for someone else's upstream quota.
|
||||
- **Honest parameters.** `albedo`, inverter efficiency and the module temperature coefficient are
|
||||
real configuration rather than constants baked into a service URL.
|
||||
|
||||
No API key is required. The location comes from `general.latitude`/`longitude` and the geometry
|
||||
from the configured `planes`, including `userhorizon`, `trackingtype`, `mountingplace` and `loss`.
|
||||
|
||||
```python
|
||||
{
|
||||
"pvforecast": {
|
||||
"provider": "PVForecastAkkudoktorLocal",
|
||||
"provider_settings": {
|
||||
"PVForecastAkkudoktorLocal": {
|
||||
"resolution_minutes": 15,
|
||||
"weather_models": ["icon_seamless", "ecmwf_ifs025", "gfs_seamless"],
|
||||
"calibration_enabled": true
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
#### Improving accuracy
|
||||
|
||||
**Model ensemble.** Listing several models in `weather_models` averages them per variable. This
|
||||
is the cheapest reliable way to cut irradiance forecast error and costs no extra API calls,
|
||||
because Open-Meteo returns all members in the same response. `icon_seamless` (DWD, strong over
|
||||
Central Europe), `ecmwf_ifs025` and `gfs_seamless` are a reasonable trio.
|
||||
|
||||
**Self-calibration.** With `calibration_enabled` the provider compares its own model against
|
||||
measured PV production over the past `calibration_days` and fits a correction:
|
||||
|
||||
- a **global scale factor**, which absorbs a wrong `peakpower`, soiling, degradation and any
|
||||
systematic offset in the loss assumption;
|
||||
- **per-solar-azimuth factors**, which absorb near-field shading that a coarse `userhorizon`
|
||||
cannot express - a chimney, a tree, a neighbouring roof.
|
||||
|
||||
Each bin is weighted by its modelled energy and shrunk toward the global factor by
|
||||
`calibration_prior_kwh`, so a thinly sampled bin cannot swing the forecast on its own, and every
|
||||
factor is clamped to `[calibration_min_factor, calibration_max_factor]` so a broken meter cannot
|
||||
either. The fitted factors and the resulting change in mean absolute error are logged at INFO
|
||||
level on every update.
|
||||
|
||||
Calibration requires `measurement.pv_production_emr_keys` to be configured and fed with
|
||||
cumulative PV production meter readings in kWh:
|
||||
|
||||
```python
|
||||
{
|
||||
"measurement": {
|
||||
"pv_production_emr_keys": ["pv1_emr"]
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Note what this does and does not correct. The comparison runs on past intervals, where the
|
||||
Open-Meteo rows are analysed rather than forecast weather, so it isolates the error of the *PV
|
||||
model* from the error of the *weather forecast*. That is deliberate: only the former is
|
||||
systematic enough to correct. A cloudy day that the weather model got wrong stays wrong.
|
||||
|
||||
**Choosing the window.** `calibration_days` trades responsiveness against stability. A long
|
||||
window averages more weather and gives a steadier factor, but it also reaches back into the
|
||||
plant's own history - and calibration cannot tell a modelling error from a *plant* error. A
|
||||
window that spans a string outage, an inverter derating or a period of heavy soiling will fit
|
||||
that fault as if it were a permanent property of the installation, and the forecast stays
|
||||
depressed long after the plant recovered. Before trusting a factor, compare modelled against
|
||||
measured energy *per day*: a run of days at a markedly different ratio is a plant event, and
|
||||
the window should start after it.
|
||||
|
||||
**Bins need data.** The per-azimuth factors are only worth fitting when the window holds enough
|
||||
daylight hours to populate the bins - roughly a few hundred, so several weeks at the default
|
||||
15 degrees. With a short window, set `calibration_azimuth_bin_degrees` to 0 to fit the global
|
||||
factor alone. One well-determined number beats twenty-four noisy ones.
|
||||
|
||||
**Geometry is out of scope.** Calibration is a scale factor on the model's output; it never
|
||||
touches `userhorizon`, `surface_tilt`, `surface_azimuth` or `peakpower`. That makes it the right
|
||||
tool for *multiplicative* errors and the wrong one for geometric errors. Horizon shading in
|
||||
particular gates the beam component as a hard function of both solar azimuth *and* elevation, so
|
||||
a per-azimuth scale factor cannot move the edge of the shadow to where it belongs, and what it
|
||||
learns in one season is wrong in the next, when the sun crosses the same azimuth at a different
|
||||
height. A wrong horizon should be corrected in `userhorizon`, not calibrated away.
|
||||
|
||||
`scripts/pvforecast_backtest.py` scores configuration variants against the stored meter readings
|
||||
without waiting for new forecasts to come true, which is the quickest way to test a geometry
|
||||
change:
|
||||
|
||||
```bash
|
||||
python scripts/pvforecast_backtest.py --days 30 --tilt 88 --azimuth 175
|
||||
```
|
||||
|
||||
#### Conventions
|
||||
|
||||
Two timing conventions are handled explicitly and are worth knowing when comparing against other
|
||||
providers:
|
||||
|
||||
- Open-Meteo radiation values are the mean over the **preceding** interval, so the representative
|
||||
sun position for a value stamped `t` is `t - interval/2`.
|
||||
- EOS records label an interval by its **start**, so a value stamped `t` by Open-Meteo is stored
|
||||
at `t - interval`. Set `shift_to_interval_start` to false to keep the raw stamps.
|
||||
|
||||
Note also that Open-Meteo's `direct_radiation` is beam irradiance on the *horizontal* plane; the
|
||||
DNI this chain needs is `direct_normal_irradiance`.
|
||||
|
||||
The prediction keys for the PV forecast data are:
|
||||
|
||||
- `pvforecast_ac_power`: Total AC power (W).
|
||||
- `pvforecast_dc_power`: Total DC power (W).
|
||||
|
||||
### PVForecastForecastSolar Provider
|
||||
|
||||
The `PVForecastForecastSolar` provider retrieves PV power forecasts from the free
|
||||
|
||||
Reference in New Issue
Block a user