feat(pvforecast): keep outages out of the local provider's calibration

A battery or inverter failure limits PV to local demand for days. The
calibration read that as the plant's true output and learned it as a permanent
model loss, so one outage degraded the forecast long after the hardware was
fixed.

Calibration now estimates the healthy plant ratio over
`calibration_reference_days`, excludes days below `calibration_outage_threshold`
of it, and falls back to the most recent `calibration_min_healthy_days` when the
normal window is contaminated. `calibration_outage_filter_enabled` turns this
off for plants where measured curtailment, not available potential, is the
prediction target.

The fit also uses native 15-minute meter readings when every configured PV meter
supplies them - never interpolating hourly counters into an invented
quarter-hour profile - interpolates azimuth factors smoothly between bin centres
instead of stepping the EMS input curve, and normalizes the shape per forecast
day so it redistributes energy without changing that day's kWh correction. The
default azimuth bin widens from 15 to 45 degrees, which is what a typical
window actually supports.

Fixes the calibration window itself: it was derived from the measurement store
as a whole rather than from the configured PV production meters. A load meter
reaching further than the PV meter placed the window where no PV reading exists,
so calibration silently fell back to hourly fitting or skipped itself.

Also fixes `Measurement.load()`, which discarded every stored record. It
validated the file into a "temporary" Measurement, but Measurement is a
singleton, so that instance was the live one and the parsed records were
dropped.
This commit is contained in:
Andreas
2026-09-09 07:56:55 +02:00
parent a2f4ef6f54
commit 6dc58c33e2
8 changed files with 700 additions and 77 deletions
+17
View File
@@ -118,6 +118,15 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/).
- Add `scripts/pvforecast_backtest.py`, which scores PV forecast configuration variants against - Add `scripts/pvforecast_backtest.py`, which scores PV forecast configuration variants against
the stored meter readings straight away instead of waiting for new forecasts to come true, and the stored meter readings straight away instead of waiting for new forecasts to come true, and
`Measurement.pv_production_total_kwh()` alongside the existing load total. `Measurement.pv_production_total_kwh()` alongside the existing load total.
- The local PV provider's calibration now excludes probable outage and curtailment days instead of
learning them as permanent model losses: `calibration_outage_filter_enabled` (default on),
`calibration_outage_threshold`, `calibration_reference_days` and `calibration_min_healthy_days`
estimate the healthy plant ratio and fall back to the most recent healthy days. Calibration also
uses native 15-minute meter readings when every configured PV meter supplies them, interpolates
azimuth factors smoothly between bin centres instead of stepping, and normalizes the fitted
shape per forecast day so it redistributes energy without changing that day's kWh correction.
The default `calibration_azimuth_bin_degrees` moves from 15 to 45, which is what a typical
calibration window actually supports.
- Separate the control horizon from the battery lookahead. `optimization.horizon_hours` remains - Separate the control horizon from the battery lookahead. `optimization.horizon_hours` remains
the only span that receives control commands; the new `optimization.tail_horizon_hours` the only span that receives control commands; the new `optimization.tail_horizon_hours`
(default 48 h) is a forecast lookahead that never produces a command. In `AUTO` terminal-value (default 48 h) is a forecast lookahead that never produces a command. In `AUTO` terminal-value
@@ -219,6 +228,14 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/).
provider could 404 the whole load prediction. It now defaults to a cache-aware update and provider could 404 the whole load prediction. It now defaults to a cache-aware update and
accepts an optional `force_update` flag in the request body for callers that still want accepts an optional `force_update` flag in the request body for callers that still want
to force. to force.
- The local PV provider derived its calibration window from the measurement store as a whole
instead of from the configured PV production meters. A load meter reaching further than the PV
meter placed the window where no PV reading exists, so calibration silently fell back to hourly
fitting or skipped itself entirely. The window now follows the PV meters.
- `Measurement.load()` silently discarded every stored record. It validated the file into a
temporary `Measurement`, but `Measurement` is a singleton, so the "temporary" instance was the
already initialized one and the parsed records were dropped. The records are now validated
individually and inserted directly.
- A rejected configuration update no longer damages the running configuration. - A rejected configuration update no longer damages the running configuration.
`merge_settings_from_dict` validated the merged candidate only while reinitializing the `merge_settings_from_dict` validated the merged candidate only while reinitializing the
singleton, so an invalid update could leave EOS half-updated. The candidate is validated first. singleton, so an invalid update could leave EOS half-updated. The candidate is validated first.
+11 -3
View File
@@ -206,12 +206,16 @@
| ---- | ---- | --------- | ------- | ----------- | | ---- | ---- | --------- | ------- | ----------- |
| albedo | `float` | `rw` | `0.25` | Ground albedo used for planes that do not set their own. | | albedo | `float` | `rw` | `0.25` | Ground albedo used for planes that do not set their own. |
| apply_iam | `bool` | `rw` | `True` | Apply the ASHRAE incidence-angle modifier to the beam component. | | apply_iam | `bool` | `rw` | `True` | Apply the ASHRAE incidence-angle modifier to the beam component. |
| calibration_azimuth_bin_degrees | `int` | `rw` | `15` | Width of the solar-azimuth bins for the correction. 0 fits a single global factor only. | | calibration_azimuth_bin_degrees | `int` | `rw` | `45` | Width of the solar-azimuth bins for the correction. 0 fits a single global factor only. |
| calibration_days | `int` | `rw` | `30` | Length of the measurement window used to fit the correction. | | calibration_days | `int` | `rw` | `30` | Length of the measurement window used to fit the correction. |
| calibration_enabled | `bool` | `rw` | `False` | Correct systematic model error against measured PV production. Requires `measurement.pv_production_emr_keys` to be configured and fed. Fits a global scale factor plus per-solar-azimuth factors, which is what catches near-field shading the horizon profile misses. | | calibration_enabled | `bool` | `rw` | `False` | Correct systematic model error against measured PV production. Requires `measurement.pv_production_emr_keys` to be configured and fed. Fits a global scale factor plus per-solar-azimuth factors, which is what catches near-field shading the horizon profile misses. |
| calibration_max_factor | `float` | `rw` | `1.5` | Upper clamp on any fitted correction factor. | | calibration_max_factor | `float` | `rw` | `1.5` | Upper clamp on any fitted correction factor. |
| calibration_min_factor | `float` | `rw` | `0.5` | Lower clamp on any fitted correction factor. | | calibration_min_factor | `float` | `rw` | `0.5` | Lower clamp on any fitted correction factor. |
| calibration_min_healthy_days | `int` | `rw` | `3` | Minimum number of healthy days used for a fit. Older healthy days from the reference window are added when the recent window contains fewer. |
| calibration_outage_filter_enabled | `bool` | `rw` | `True` | Exclude days whose measured production is far below the recent healthy plant level. This prevents inverter, battery and curtailment events from being learned as permanent PV model losses. |
| calibration_outage_threshold | `float` | `rw` | `0.55` | A day is treated as unavailable when its measured/modelled energy ratio is below this fraction of the robust healthy reference ratio. |
| calibration_prior_kwh | `float` | `rw` | `5.0` | Shrinkage strength: a bin needs this much modelled energy before its own factor outweighs the global one. Higher is more conservative. | | calibration_prior_kwh | `float` | `rw` | `5.0` | Shrinkage strength: a bin needs this much modelled energy before its own factor outweighs the global one. Higher is more conservative. |
| calibration_reference_days | `int` | `rw` | `30` | Lookback used to distinguish healthy production from outages or curtailment. If the calibration window contains too few healthy days, the most recent healthy days from this reference window are used. |
| forecast_days | `Optional[int]` | `rw` | `None` | Forecast horizon in days (1-16). Leave empty to derive it from `prediction.hours`, which is what keeps the optimizer's tail horizon fed. | | forecast_days | `Optional[int]` | `rw` | `None` | Forecast horizon in days (1-16). Leave empty to derive it from `prediction.hours`, which is what keeps the optimizer's tail horizon fed. |
| inverter_efficiency | `float` | `rw` | `0.96` | Nominal inverter efficiency (PVWatts eta_inv_nom). | | inverter_efficiency | `float` | `rw` | `0.96` | Nominal inverter efficiency (PVWatts eta_inv_nom). |
| past_days | `Optional[int]` | `rw` | `None` | Days of past data to request (0-92). Leave empty to derive it from `prediction.historic_hours`. | | past_days | `Optional[int]` | `rw` | `None` | Days of past data to request (0-92). Leave empty to derive it from `prediction.historic_hours`. |
@@ -247,7 +251,11 @@
"shift_to_interval_start": true, "shift_to_interval_start": true,
"calibration_enabled": true, "calibration_enabled": true,
"calibration_days": 30, "calibration_days": 30,
"calibration_azimuth_bin_degrees": 15, "calibration_reference_days": 30,
"calibration_outage_filter_enabled": true,
"calibration_outage_threshold": 0.55,
"calibration_min_healthy_days": 3,
"calibration_azimuth_bin_degrees": 45,
"calibration_prior_kwh": 5.0, "calibration_prior_kwh": 5.0,
"calibration_min_factor": 0.5, "calibration_min_factor": 0.5,
"calibration_max_factor": 1.5 "calibration_max_factor": 1.5
@@ -432,9 +440,9 @@
| Name | Type | Read-Only | Default | Description | | Name | Type | Read-Only | Default | Description |
| ---- | ---- | --------- | ------- | ----------- | | ---- | ---- | --------- | ------- | ----------- |
| PVForecastAkkudoktorLocal | `Optional[akkudoktoreos.prediction.pvforecastakkudoktorlocal.PVForecastAkkudoktorLocalCommonSettings]` | `rw` | `None` | PVForecastAkkudoktorLocal settings |
| PVForecastForecastSolar | `Optional[akkudoktoreos.prediction.pvforecastforecastsolar.PVForecastForecastSolarCommonSettings]` | `rw` | `None` | PVForecastForecastSolar settings | | PVForecastForecastSolar | `Optional[akkudoktoreos.prediction.pvforecastforecastsolar.PVForecastForecastSolarCommonSettings]` | `rw` | `None` | PVForecastForecastSolar settings |
| PVForecastImport | `Optional[akkudoktoreos.prediction.pvforecastimport.PVForecastImportCommonSettings]` | `rw` | `None` | PVForecastImport settings | | PVForecastImport | `Optional[akkudoktoreos.prediction.pvforecastimport.PVForecastImportCommonSettings]` | `rw` | `None` | PVForecastImport settings |
| PVForecastAkkudoktorLocal | `Optional[akkudoktoreos.prediction.pvforecastlocal.PVForecastAkkudoktorLocalCommonSettings]` | `rw` | `None` | PVForecastAkkudoktorLocal settings |
| PVForecastPVNode | `Optional[akkudoktoreos.prediction.pvforecastpvnode.PVForecastPVNodeCommonSettings]` | `rw` | `None` | PVForecastPVNode settings | | PVForecastPVNode | `Optional[akkudoktoreos.prediction.pvforecastpvnode.PVForecastPVNodeCommonSettings]` | `rw` | `None` | PVForecastPVNode settings |
| PVForecastSolcast | `Optional[akkudoktoreos.prediction.pvforecastsolcast.PVForecastSolcastCommonSettings]` | `rw` | `None` | PVForecastSolcast settings | | PVForecastSolcast | `Optional[akkudoktoreos.prediction.pvforecastsolcast.PVForecastSolcastCommonSettings]` | `rw` | `None` | PVForecastSolcast settings |
| PVForecastVrm | `Optional[akkudoktoreos.prediction.pvforecastvrm.PVForecastVrmCommonSettings]` | `rw` | `None` | PVForecastVrm settings | | PVForecastVrm | `Optional[akkudoktoreos.prediction.pvforecastvrm.PVForecastVrmCommonSettings]` | `rw` | `None` | PVForecastVrm settings |
+19
View File
@@ -822,6 +822,25 @@ factor is clamped to `[calibration_min_factor, calibration_max_factor]` so a bro
either. The fitted factors and the resulting change in mean absolute error are logged at INFO either. The fitted factors and the resulting change in mean absolute error are logged at INFO
level on every update. level on every update.
By default, calibration also rejects probable outage or curtailment days. It estimates the
healthy plant ratio from `calibration_reference_days`, excludes days below
`calibration_outage_threshold` of that reference, and falls back to the most recent
`calibration_min_healthy_days` when the normal calibration window contains an outage. This keeps
a battery or inverter failure that limits PV to local demand from becoming a permanent forecast
loss. Set `calibration_outage_filter_enabled` to false only when measured curtailed production,
rather than available PV potential, is the intended prediction target.
The measurement cadence controls the detail that can be learned. Hourly cumulative meter
readings calibrate hourly energy while the native Open-Meteo/pvlib chain continues to supply the
15-minute shape. If every configured PV meter supplies genuine 15-minute readings, calibration
automatically uses those native slots as well. It never interpolates hourly counters into an
invented quarter-hour profile. Azimuth factors are interpolated smoothly between bin centres so
they do not introduce steps into the EMS input curve. The shape fit uses all healthy days in the
reference window, while the global factor still follows the shorter recent window. Finally, the
shape is normalized per forecast day: it redistributes the calibrated energy across the day's
15-minute slots without changing that day's global kWh correction (unless the physical inverter
limit clips a peak).
Calibration requires `measurement.pv_production_emr_keys` to be configured and fed with Calibration requires `measurement.pv_production_emr_keys` to be configured and fed with
cumulative PV production meter readings in kWh: cumulative PV production meter readings in kWh:
+24 -4
View File
@@ -46,8 +46,11 @@ ENSEMBLE = ["icon_seamless", "ecmwf_ifs025", "gfs_seamless"]
# Variants scored against the meter. Each entry is a label plus the settings overrides # Variants scored against the meter. Each entry is a label plus the settings overrides
# applied on top of the configured provider settings. # applied on top of the configured provider settings.
VARIANTS: list[tuple[str, dict[str, Any]]] = [ VARIANTS: list[tuple[str, dict[str, Any]]] = [
("best_match", {"weather_models": ["best_match"]}), (
("ensemble", {"weather_models": ENSEMBLE}), "best_match",
{"weather_models": ["best_match"], "calibration_enabled": False},
),
("ensemble", {"weather_models": ENSEMBLE, "calibration_enabled": False}),
("ensemble + calibration", {"weather_models": ENSEMBLE, "calibration_enabled": True}), ("ensemble + calibration", {"weather_models": ENSEMBLE, "calibration_enabled": True}),
( (
"ensemble + calibration (global only)", "ensemble + calibration (global only)",
@@ -57,8 +60,18 @@ VARIANTS: list[tuple[str, dict[str, Any]]] = [
"calibration_azimuth_bin_degrees": 0, "calibration_azimuth_bin_degrees": 0,
}, },
), ),
("ensemble, isotropic sky", {"weather_models": ENSEMBLE, "transposition_model": "isotropic"}), (
("ensemble, no IAM", {"weather_models": ENSEMBLE, "apply_iam": False}), "ensemble, isotropic sky",
{
"weather_models": ENSEMBLE,
"transposition_model": "isotropic",
"calibration_enabled": False,
},
),
(
"ensemble, no IAM",
{"weather_models": ENSEMBLE, "apply_iam": False, "calibration_enabled": False},
),
] ]
@@ -94,6 +107,8 @@ def main(days: int, tilt: Optional[float], azimuth: Optional[float]) -> int:
singletons_init() singletons_init()
config = get_config() config = get_config()
measurement = get_measurement() measurement = get_measurement()
if measurement.max_datetime is None:
measurement.load()
if not config.measurement.pv_production_emr_keys: if not config.measurement.pv_production_emr_keys:
print( print(
@@ -175,6 +190,11 @@ def main(days: int, tilt: Optional[float], azimuth: Optional[float]) -> int:
"on, so their advantage here is optimistic. Re-run with a longer --days to see\n" "on, so their advantage here is optimistic. Re-run with a longer --days to see\n"
"how much of it survives." "how much of it survives."
) )
print(
"Outage or curtailment periods are excluded from calibration but remain in these\n"
"scores, because the script cannot prove the plant's availability without an\n"
"explicit availability measurement."
)
best_label, best, hours = min(rows, key=lambda row: row[1]["mae"]) best_label, best, hours = min(rows, key=lambda row: row[1]["mae"])
print( print(
+7 -8
View File
@@ -6,6 +6,7 @@ data records for measurements.
The measurements can be added programmatically or imported from a file or JSON string. The measurements can be added programmatically or imported from a file or JSON string.
""" """
import json
from pathlib import Path from pathlib import Path
from typing import Any, Optional from typing import Any, Optional
@@ -359,14 +360,12 @@ class Measurement(SingletonMixin, DataImportMixin, DataSequence):
if not measurement_file_path.exists(): if not measurement_file_path.exists():
return False return False
try: try:
# Validate into a temporary instance # Do not validate the complete Measurement model here. Measurement is a
loaded = self.__class__.model_validate_json( # singleton, so constructing a temporary instance returns the already
measurement_file_path.read_text(encoding="utf-8") # initialized singleton and silently discards the serialized records.
) payload = json.loads(measurement_file_path.read_text(encoding="utf-8"))
for record_data in payload.get("records", []):
# Explicitly add data records to the existing singleton self.insert_by_datetime(MeasurementDataRecord.model_validate(record_data))
for record in loaded.records:
self.insert_by_datetime(record)
except Exception as e: except Exception as e:
logger.exception("Cannot load measurements") logger.exception("Cannot load measurements")
return True return True
@@ -203,8 +203,56 @@ class PVForecastAkkudoktorLocalCommonSettings(SettingsBaseModel):
"examples": [30, 14], "examples": [30, 14],
}, },
) )
calibration_reference_days: int = Field(
default=30,
ge=3,
le=MAX_PAST_DAYS,
json_schema_extra={
"description": (
"Lookback used to distinguish healthy production from outages or "
"curtailment. If the calibration window contains too few healthy days, "
"the most recent healthy days from this reference window are used."
),
"examples": [30, 14],
},
)
calibration_outage_filter_enabled: bool = Field(
default=True,
json_schema_extra={
"description": (
"Exclude days whose measured production is far below the recent healthy "
"plant level. This prevents inverter, battery and curtailment events from "
"being learned as permanent PV model losses."
),
"examples": [True],
},
)
calibration_outage_threshold: float = Field(
default=0.55,
gt=0.0,
lt=1.0,
json_schema_extra={
"description": (
"A day is treated as unavailable when its measured/modelled energy ratio "
"is below this fraction of the robust healthy reference ratio."
),
"examples": [0.55, 0.5],
},
)
calibration_min_healthy_days: int = Field(
default=3,
ge=1,
le=31,
json_schema_extra={
"description": (
"Minimum number of healthy days used for a fit. Older healthy days from "
"the reference window are added when the recent window contains fewer."
),
"examples": [3],
},
)
calibration_azimuth_bin_degrees: int = Field( calibration_azimuth_bin_degrees: int = Field(
default=15, default=45,
ge=0, ge=0,
le=180, le=180,
json_schema_extra={ json_schema_extra={
@@ -212,7 +260,7 @@ class PVForecastAkkudoktorLocalCommonSettings(SettingsBaseModel):
"Width of the solar-azimuth bins for the correction. 0 fits a single " "Width of the solar-azimuth bins for the correction. 0 fits a single "
"global factor only." "global factor only."
), ),
"examples": [15, 30, 0], "examples": [45, 30, 15, 0],
}, },
) )
calibration_prior_kwh: float = Field( calibration_prior_kwh: float = Field(
@@ -295,7 +343,12 @@ class PVForecastAkkudoktorLocal(PVForecastProvider):
if settings.calibration_enabled: if settings.calibration_enabled:
# The fit compares modelled against measured power over the same past # The fit compares modelled against measured power over the same past
# intervals, so the weather for that window has to come back with the request. # intervals, so the weather for that window has to come back with the request.
past_days = max(past_days, settings.calibration_days) calibration_lookback = settings.calibration_days
if settings.calibration_outage_filter_enabled:
calibration_lookback = max(
calibration_lookback, settings.calibration_reference_days
)
past_days = max(past_days, calibration_lookback)
return forecast_days, min(MAX_PAST_DAYS, past_days) return forecast_days, min(MAX_PAST_DAYS, past_days)
@@ -305,7 +358,9 @@ class PVForecastAkkudoktorLocal(PVForecastProvider):
latitude = self.config.general.latitude latitude = self.config.general.latitude
longitude = self.config.general.longitude longitude = self.config.general.longitude
if latitude is None or longitude is None: if latitude is None or longitude is None:
raise ValueError("PVForecastAkkudoktorLocal needs general.latitude and general.longitude") raise ValueError(
"PVForecastAkkudoktorLocal needs general.latitude and general.longitude"
)
settings = self._settings settings = self._settings
block = "minutely_15" if settings.resolution_minutes == 15 else "hourly" block = "minutely_15" if settings.resolution_minutes == 15 else "hourly"
@@ -602,8 +657,14 @@ class PVForecastAkkudoktorLocal(PVForecastProvider):
# The fit needs the raw model to compare against, which is exactly `frame`. # The fit needs the raw model to compare against, which is exactly `frame`.
calibration = self._fit_calibration(frame) calibration = self._fit_calibration(frame)
if calibration is not None: if calibration is not None:
_, factors = calibration global_factor, factors = calibration
frame = self._apply_calibration(frame, factors, self._installed_ac_capacity_w()) frame = self._apply_calibration(
frame,
factors,
self._installed_ac_capacity_w(),
global_factor=global_factor,
timezone=self.config.general.timezone,
)
return frame return frame
# ------------------------------------------------------------ calibration # ------------------------------------------------------------ calibration
@@ -618,6 +679,127 @@ class PVForecastAkkudoktorLocal(PVForecastProvider):
total += float(plane.peakpower) * 1000.0 total += float(plane.peakpower) * 1000.0
return total return total
def _pv_measurement_window(self) -> Optional[tuple[Any, Any]]:
"""First and last timestamp that actually carries PV production readings.
The measurement store holds every meter, not just the PV ones. Load meters
routinely reach further than the PV meter in both directions, so deriving
the calibration window from the store as a whole would place it where no
PV reading exists - which silently skips calibration or downgrades it to
hourly fitting.
"""
measurement = self.measurement
if measurement.min_datetime is None or measurement.max_datetime is None:
return None
earliest: Any = None
latest: Any = None
for key in self.config.measurement.pv_production_emr_keys or []:
dates, _ = measurement.key_to_lists(
key=key,
start_datetime=measurement.min_datetime,
end_datetime=measurement.max_datetime.add(minutes=1),
)
if not dates:
continue
if earliest is None or compare_datetimes(dates[0], earliest).lt:
earliest = dates[0]
if latest is None or compare_datetimes(dates[-1], latest).gt:
latest = dates[-1]
if earliest is None or latest is None:
return None
return earliest, latest
def _calibration_interval_minutes(self, start: Any, end: Any) -> int:
"""Use native forecast slots only when every PV meter resolves them.
Interpolating an hourly cumulative meter onto quarter hours would create a
perfectly flat, but invented, intrahour profile. Fall back to hourly fitting
until all configured production meters actually provide native slot readings.
"""
native_minutes = self._settings.resolution_minutes
meter_resolutions: list[float] = []
for key in self.config.measurement.pv_production_emr_keys or []:
dates, _ = self.measurement.key_to_lists(
key=key, start_datetime=start, end_datetime=end
)
if len(dates) < 3:
return 60
deltas = np.asarray(
[
(dates[index] - dates[index - 1]).total_seconds() / 60.0
for index in range(1, len(dates))
if dates[index] > dates[index - 1]
],
dtype=float,
)
if deltas.size == 0:
return 60
meter_resolutions.append(float(np.median(deltas)))
if not meter_resolutions:
return 60
if max(meter_resolutions) <= native_minutes * 1.5:
return native_minutes
return 60
def _fit_azimuth_factors(
self,
modelled_kwh: np.ndarray,
measured_kwh: np.ndarray,
azimuth: np.ndarray,
global_factor: float,
) -> np.ndarray:
"""Fit an energy-weighted intraday shape while preserving global energy."""
settings = self._settings
bin_degrees = settings.calibration_azimuth_bin_degrees
if bin_degrees <= 0:
return np.array([global_factor])
bin_count = max(1, int(round(360 / bin_degrees)))
bin_index = np.clip((azimuth % 360.0) / (360.0 / bin_count), 0, bin_count - 1).astype(int)
# Fit the shape as a residual around the independently determined global
# energy factor. Day-level availability filtering has already removed outages;
# energy sums now give productive intervals the influence relevant to the EMS.
# The prior shrinks sparse bins back toward a neutral relative factor of one.
relative_shape = np.ones(bin_count, dtype=float)
prior = settings.calibration_prior_kwh
for bin_number in range(bin_count):
in_bin = bin_index == bin_number
weight = float(modelled_kwh[in_bin].sum())
if weight <= 0.0:
continue
measured_sum = float(measured_kwh[in_bin].sum())
raw_shape = measured_sum / (weight * global_factor)
relative_shape[bin_number] = (weight * raw_shape + prior) / (weight + prior)
factors = np.clip(
global_factor * relative_shape,
settings.calibration_min_factor,
settings.calibration_max_factor,
)
# Smooth interpolation changes the exact weighted mean of the bin-centre
# values. Renormalize after interpolation so shape correction cannot silently
# change the global kWh calibration. Re-clipping is iterated to respect bounds.
target_energy = global_factor * float(modelled_kwh.sum())
for _ in range(8):
scale = self._interpolate_azimuth_factors(azimuth, factors)
corrected_energy = float(np.dot(modelled_kwh, scale))
if corrected_energy <= 0.0:
break
correction = target_energy / corrected_energy
updated = np.clip(
factors * correction,
settings.calibration_min_factor,
settings.calibration_max_factor,
)
if np.allclose(updated, factors, rtol=1e-6, atol=1e-8):
factors = updated
break
factors = updated
return factors
def _fit_calibration(self, frame: pd.DataFrame) -> Optional[tuple[float, np.ndarray]]: def _fit_calibration(self, frame: pd.DataFrame) -> Optional[tuple[float, np.ndarray]]:
"""Fit correction factors from measured PV production against the model. """Fit correction factors from measured PV production against the model.
@@ -642,21 +824,35 @@ class PVForecastAkkudoktorLocal(PVForecastProvider):
return None return None
measurement = self.measurement measurement = self.measurement
if measurement.max_datetime is None or measurement.min_datetime is None: pv_window = self._pv_measurement_window()
if pv_window is None:
logger.info("PVForecastAkkudoktorLocal calibration: no PV measurements yet - skipping.") logger.info("PVForecastAkkudoktorLocal calibration: no PV measurements yet - skipping.")
return None return None
pv_min_datetime, reference_end = pv_window
interval = to_duration("1 hour") lookback_days = settings.calibration_days
end = measurement.max_datetime.start_of("hour") if settings.calibration_outage_filter_enabled:
start = end.subtract(days=settings.calibration_days) lookback_days = max(lookback_days, settings.calibration_reference_days)
if compare_datetimes(start, measurement.min_datetime).lt: reference_start = reference_end.subtract(days=lookback_days)
start = measurement.min_datetime.start_of("hour").add(hours=1) interval_minutes = self._calibration_interval_minutes(reference_start, reference_end)
interval_hours = interval_minutes / 60.0
interval = to_duration(f"{interval_minutes} minutes")
end = reference_end.start_of("hour")
if interval_minutes < 60:
end = reference_end.start_of("minute").set(
minute=(reference_end.minute // interval_minutes) * interval_minutes
)
start = end.subtract(days=lookback_days)
if compare_datetimes(start, pv_min_datetime).lt:
start = pv_min_datetime.start_of("minute").add(minutes=interval_minutes)
# The model side only exists for the weather window that was requested. # The model side only exists for the weather window that was requested.
model_start = to_datetime(frame.index[0].to_pydatetime()) model_start = to_datetime(frame.index[0].to_pydatetime())
if compare_datetimes(start, model_start).lt: if compare_datetimes(start, model_start).lt:
start = model_start.start_of("hour").add(hours=1) start = model_start.start_of("minute").add(minutes=interval_minutes)
if compare_datetimes(start, end).ge: if compare_datetimes(start, end).ge:
logger.info("PVForecastAkkudoktorLocal calibration: measurement window too short - skipping.") logger.info(
"PVForecastAkkudoktorLocal calibration: measurement window too short - skipping."
)
return None return None
measured_kwh = np.asarray( measured_kwh = np.asarray(
@@ -666,24 +862,32 @@ class PVForecastAkkudoktorLocal(PVForecastProvider):
dtype=float, dtype=float,
) )
if measured_kwh.size == 0 or not np.isfinite(measured_kwh).any(): if measured_kwh.size == 0 or not np.isfinite(measured_kwh).any():
logger.info("PVForecastAkkudoktorLocal calibration: no usable PV measurements - skipping.") logger.info(
"PVForecastAkkudoktorLocal calibration: no usable PV measurements - skipping."
)
return None return None
# Model side on the same hourly grid. `ac_power` is a mean power per interval, # Model side on the same grid as the real meter. Mean power is converted to
# so the hourly mean in W is directly the hourly energy in Wh. # interval energy below; hourly meters stay hourly and native 15-minute meters
hourly = frame[["ac_power", "solar_azimuth"]].resample("1h").mean() # retain the shape that matters to the EMS.
samples_frame = (
frame[["ac_power", "solar_azimuth"]].resample(f"{interval_minutes}min").mean()
)
grid = pd.date_range( grid = pd.date_range(
start=pd.Timestamp(start.in_timezone("UTC").isoformat()), start=pd.Timestamp(start.in_timezone("UTC").isoformat()),
periods=len(measured_kwh), periods=len(measured_kwh),
freq="1h", freq=f"{interval_minutes}min",
) )
hourly = hourly.reindex(grid) samples_frame = samples_frame.reindex(grid)
modelled_kwh = hourly["ac_power"].to_numpy(dtype=float) / 1000.0 modelled_kwh = samples_frame["ac_power"].to_numpy(dtype=float) / 1000.0 * interval_hours
azimuth = hourly["solar_azimuth"].to_numpy(dtype=float) azimuth = samples_frame["solar_azimuth"].to_numpy(dtype=float)
# Only fit where the model says something meaningful is being produced. Dawn and # Only fit where the model says something meaningful is being produced. Dawn and
# dusk intervals otherwise dominate the ratio with noise. # dusk intervals otherwise dominate the ratio with noise.
floor_kwh = max(0.02 * self._installed_ac_capacity_w() / 1000.0, 0.05) floor_kwh = max(
0.02 * self._installed_ac_capacity_w() / 1000.0 * interval_hours,
0.05 * interval_hours,
)
usable = ( usable = (
np.isfinite(modelled_kwh) np.isfinite(modelled_kwh)
& np.isfinite(measured_kwh) & np.isfinite(measured_kwh)
@@ -691,12 +895,108 @@ class PVForecastAkkudoktorLocal(PVForecastProvider):
& (modelled_kwh > floor_kwh) & (modelled_kwh > floor_kwh)
& (measured_kwh >= 0.0) & (measured_kwh >= 0.0)
) )
if usable.sum() < 12: shape_usable = usable.copy()
# Calibration represents the available PV potential. A battery or inverter
# outage can make an otherwise healthy plant cover only local demand; those
# intervals must not be learned as a permanent model loss. Detect this at day
# level, because individual cloudy hours are much too noisy for a reliable
# availability decision.
local_days = samples_frame.index.tz_convert(self.config.general.timezone).normalize()
fit_start = pd.Timestamp(
end.subtract(days=settings.calibration_days)
.in_timezone(self.config.general.timezone)
.isoformat()
).normalize()
if settings.calibration_outage_filter_enabled and usable.any():
samples = pd.DataFrame(
{
"modelled_kwh": modelled_kwh[usable],
"measured_kwh": measured_kwh[usable],
"local_day": local_days[usable],
}
)
daily = samples.groupby("local_day").agg(
modelled_kwh=("modelled_kwh", "sum"),
measured_kwh=("measured_kwh", "sum"),
usable_intervals=("modelled_kwh", "size"),
)
daily["ratio"] = daily["measured_kwh"] / daily["modelled_kwh"]
# Low-yield weather days do not carry enough evidence to call an outage.
# Half an equivalent full-load hour scales naturally with plant size.
minimum_day_kwh = max(0.5 * self._installed_ac_capacity_w() / 1000.0, 1.0)
reference_candidates = daily[
(daily["modelled_kwh"] >= minimum_day_kwh) & np.isfinite(daily["ratio"])
]
outage_days = pd.DatetimeIndex([])
reference_ratio = float("nan")
if len(reference_candidates) >= settings.calibration_min_healthy_days:
# The upper quartile is a robust estimate of the available plant level:
# outages and curtailment only pull the ratio down, while a few weather
# outliers cannot dominate it as a maximum would.
reference_ratio = float(reference_candidates["ratio"].quantile(0.75))
outage_limit = reference_ratio * settings.calibration_outage_threshold
outage_days = pd.DatetimeIndex(
reference_candidates.index[reference_candidates["ratio"] < outage_limit]
)
# A cloudy day inside a known low-production block can accidentally
# resemble a healthy ratio because both numerator and denominator are
# small. Bridge a single-day gap between two detected outage days so a
# continuous plant event is not partly admitted into the fit.
ordered_days = pd.DatetimeIndex(daily.index).sort_values()
bridged_days = []
for index in range(1, len(ordered_days) - 1):
previous_day = ordered_days[index - 1]
day = ordered_days[index]
next_day = ordered_days[index + 1]
if (
previous_day in outage_days
and next_day in outage_days
and (day.date() - previous_day.date()).days == 1
and (next_day.date() - day.date()).days == 1
):
bridged_days.append(day)
if bridged_days:
outage_days = outage_days.union(pd.DatetimeIndex(bridged_days)).sort_values()
healthy_days = pd.DatetimeIndex(daily.index).difference(outage_days).sort_values()
recent_healthy_days = healthy_days[healthy_days >= fit_start]
if len(recent_healthy_days) < settings.calibration_min_healthy_days:
fit_days = healthy_days[-settings.calibration_min_healthy_days :]
else:
fit_days = recent_healthy_days
# The recent healthy window tracks the current energy level. The stable
# intraday signature uses the full healthy reference window, avoiding noisy
# shape factors learned from only a handful of days.
shape_usable &= np.asarray(local_days.isin(healthy_days), dtype=bool)
usable &= np.asarray(local_days.isin(fit_days), dtype=bool)
if len(outage_days) > 0:
day_list = ", ".join(day.strftime("%Y-%m-%d") for day in outage_days)
logger.info(
"PVForecastAkkudoktorLocal calibration: excluded probable outage/"
f"curtailment days [{day_list}] (healthy reference "
f"{reference_ratio:.3f})."
)
else:
usable &= np.asarray(local_days >= fit_start, dtype=bool)
shape_usable = usable.copy()
minimum_samples = math.ceil(12 / interval_hours)
if usable.sum() < minimum_samples:
logger.info( logger.info(
f"PVForecastAkkudoktorLocal calibration: only {int(usable.sum())} usable hours - skipping." "PVForecastAkkudoktorLocal calibration: only "
f"{int(usable.sum())} usable {interval_minutes}-minute intervals - skipping."
) )
return None return None
shape_modelled_kwh = modelled_kwh[shape_usable]
shape_measured_kwh = measured_kwh[shape_usable]
shape_azimuth = azimuth[shape_usable]
modelled_kwh = modelled_kwh[usable] modelled_kwh = modelled_kwh[usable]
measured_kwh = measured_kwh[usable] measured_kwh = measured_kwh[usable]
azimuth = azimuth[usable] azimuth = azimuth[usable]
@@ -712,54 +1012,83 @@ class PVForecastAkkudoktorLocal(PVForecastProvider):
) )
) )
bin_degrees = settings.calibration_azimuth_bin_degrees factors = self._fit_azimuth_factors(
if bin_degrees <= 0: shape_modelled_kwh,
shape_measured_kwh,
shape_azimuth,
global_factor,
)
if len(factors) == 1:
logger.info( logger.info(
f"PVForecastAkkudoktorLocal calibration: global factor {global_factor:.3f} " f"PVForecastAkkudoktorLocal calibration: global factor {global_factor:.3f} "
f"from {usable.sum()} hours." f"from {usable.sum()} {interval_minutes}-minute intervals."
)
return global_factor, np.array([global_factor])
bin_count = max(1, int(round(360 / bin_degrees)))
bin_index = np.clip((azimuth % 360.0) / (360.0 / bin_count), 0, bin_count - 1).astype(int)
# Weight each bin by its modelled energy and shrink toward the global factor, so
# a thinly sampled bin cannot swing the forecast on its own.
prior = settings.calibration_prior_kwh
factors = np.full(bin_count, global_factor, dtype=float)
for b in range(bin_count):
in_bin = bin_index == b
weight = float(modelled_kwh[in_bin].sum())
if weight <= 0.0:
continue
raw = float(measured_kwh[in_bin].sum()) / weight
factors[b] = np.clip(
(weight * raw + prior * global_factor) / (weight + prior),
settings.calibration_min_factor,
settings.calibration_max_factor,
) )
return global_factor, factors
# Report how much of the bias the fit actually removes on its own training window. # Report how much of the bias the fit actually removes on its own training window.
before = float(np.abs(modelled_kwh - measured_kwh).mean()) before = float(np.abs(modelled_kwh - measured_kwh).mean())
after = float(np.abs(modelled_kwh * factors[bin_index] - measured_kwh).mean()) fitted_scale = self._interpolate_azimuth_factors(azimuth, factors)
after = float(np.abs(modelled_kwh * fitted_scale - measured_kwh).mean())
logger.info( logger.info(
f"PVForecastAkkudoktorLocal calibration over {usable.sum()} h: global factor " f"PVForecastAkkudoktorLocal calibration over {usable.sum()} intervals: global factor "
f"{global_factor:.3f}, {bin_count} azimuth bins, " f"{global_factor:.3f}, {len(factors)} azimuth bins at {interval_minutes}-minute "
f"MAE {before:.3f} -> {after:.3f} kWh/h" "measurement resolution, "
f"MAE {before:.3f} -> {after:.3f} kWh per interval"
) )
return global_factor, factors return global_factor, factors
@staticmethod @staticmethod
def _apply_calibration( def _interpolate_azimuth_factors(azimuth: np.ndarray, factors: np.ndarray) -> np.ndarray:
frame: pd.DataFrame, factors: np.ndarray, ac_cap_w: float """Interpolate fitted bin-centre factors without a discontinuity at north."""
) -> pd.DataFrame:
"""Scale the modelled power by the per-azimuth factor of each interval."""
bin_count = len(factors) bin_count = len(factors)
if bin_count == 1:
return np.full(len(azimuth), factors[0], dtype=float)
bin_width = 360.0 / bin_count
centers = (np.arange(bin_count, dtype=float) + 0.5) * bin_width
interpolation_azimuths = np.concatenate(
([centers[-1] - 360.0], centers, [centers[0] + 360.0])
)
interpolation_factors = np.concatenate(([factors[-1]], factors, [factors[0]]))
return np.interp(
np.nan_to_num(azimuth % 360.0),
interpolation_azimuths,
interpolation_factors,
)
@staticmethod
def _apply_calibration(
frame: pd.DataFrame,
factors: np.ndarray,
ac_cap_w: float,
global_factor: Optional[float] = None,
timezone: Optional[str] = None,
) -> pd.DataFrame:
"""Scale power with a smooth, circular interpolation of azimuth factors."""
azimuth = frame["solar_azimuth"].to_numpy(dtype=float) azimuth = frame["solar_azimuth"].to_numpy(dtype=float)
bin_index = np.clip( scale = PVForecastAkkudoktorLocal._interpolate_azimuth_factors(azimuth, factors)
np.nan_to_num(azimuth % 360.0) / (360.0 / bin_count), 0, bin_count - 1
).astype(int) # Preserve the independently fitted kWh correction for every forecast day.
scale = factors[bin_index] # Azimuth factors may redistribute energy within a day, but cannot change its
# calibrated total (apart from the physical inverter cap applied below).
if global_factor is not None and len(frame) > 0:
ac_power = frame["ac_power"].to_numpy(dtype=float)
if isinstance(frame.index, pd.DatetimeIndex):
day_index = frame.index
if timezone is not None and day_index.tz is not None:
day_index = day_index.tz_convert(timezone)
groups = pd.Series(np.arange(len(frame)), index=frame.index).groupby(
day_index.normalize()
)
group_indices = (group.to_numpy() for _, group in groups)
else:
group_indices = (np.arange(len(frame)),)
for indices in group_indices:
model_energy = float(ac_power[indices].sum())
shaped_energy = float(np.dot(ac_power[indices], scale[indices]))
if model_energy > 0.0 and shaped_energy > 0.0:
scale[indices] *= global_factor * model_energy / shaped_energy
frame = frame.copy() frame = frame.copy()
frame["dc_power"] = frame["dc_power"] * scale frame["dc_power"] = frame["dc_power"] * scale
frame["ac_power"] = frame["ac_power"] * scale frame["ac_power"] = frame["ac_power"] * scale
+29
View File
@@ -1,3 +1,5 @@
import json
import numpy as np import numpy as np
import pytest import pytest
from pendulum import datetime, duration from pendulum import datetime, duration
@@ -430,3 +432,30 @@ class TestMeasurement:
result = measurement_eos.load_total_kwh(start_datetime=start_datetime, end_datetime=end_datetime, interval=interval) result = measurement_eos.load_total_kwh(start_datetime=start_datetime, end_datetime=end_datetime, interval=interval)
expected = np.array([100]) # Only one complete interval covered expected = np.array([100]) # Only one complete interval covered
np.testing.assert_array_equal(result, expected) np.testing.assert_array_equal(result, expected)
def test_load_restores_file_records_into_singleton(self, measurement_eos, config_eos, tmp_path):
"""File loading must not lose records when validating the Measurement singleton."""
# ConfigEOS and Measurement are singletons that outlive this module, so
# every global this test touches has to be put back; otherwise later
# modules read their measurements from a deleted tmp_path.
previous_folder = config_eos.general.data_folder_path
previous_keys = config_eos.measurement.load_emr_keys
previous_records = measurement_eos.records
config_eos.general.data_folder_path = tmp_path
config_eos.measurement.load_emr_keys = ["load0_mr"]
record = MeasurementDataRecord(date_time=to_datetime("2026-08-01T12:00:00Z"))
record["load0_mr"] = 123.5
payload = {"records": [record.model_dump(mode="json")]}
(tmp_path / "measurement.json").write_text(
json.dumps(payload), encoding="utf-8", newline="\n"
)
try:
measurement_eos.records = []
assert measurement_eos.load() is True
assert len(measurement_eos.records) == 1
assert measurement_eos.records[0]["load0_mr"] == pytest.approx(123.5)
finally:
measurement_eos.records = previous_records
config_eos.measurement.load_emr_keys = previous_keys
config_eos.general.data_folder_path = previous_folder
+204 -2
View File
@@ -126,7 +126,9 @@ def test_records_are_shifted_to_interval_start(pvforecast_instance):
shifted = pvforecast_instance._forecast_frame(synthetic_openmeteo()) shifted = pvforecast_instance._forecast_frame(synthetic_openmeteo())
pvforecast_instance.config.pvforecast.provider_settings.PVForecastAkkudoktorLocal = ( pvforecast_instance.config.pvforecast.provider_settings.PVForecastAkkudoktorLocal = (
PVForecastAkkudoktorLocalCommonSettings(resolution_minutes=15, shift_to_interval_start=False) PVForecastAkkudoktorLocalCommonSettings(
resolution_minutes=15, shift_to_interval_start=False
)
) )
raw = pvforecast_instance._forecast_frame(synthetic_openmeteo()) raw = pvforecast_instance._forecast_frame(synthetic_openmeteo())
@@ -173,7 +175,9 @@ def test_horizon_shading_reduces_yield(pvforecast_instance):
def test_update_data_writes_records(pvforecast_instance): def test_update_data_writes_records(pvforecast_instance):
with patch.object(PVForecastAkkudoktorLocal, "_request_forecast", return_value=synthetic_openmeteo()): with patch.object(
PVForecastAkkudoktorLocal, "_request_forecast", return_value=synthetic_openmeteo()
):
pvforecast_instance._update_data(force_update=True) pvforecast_instance._update_data(force_update=True)
assert len(pvforecast_instance.records) > 0 assert len(pvforecast_instance.records) > 0
@@ -210,6 +214,59 @@ def _feed_measurements(
) )
def _feed_measurements_with_recent_outage(
instance: PVForecastAkkudoktorLocal,
frame: pd.DataFrame,
healthy_bias: float,
outage_bias: float,
outage_days: int,
key: str,
) -> None:
"""Write a healthy meter history followed by demand-limited PV production."""
instance.config.measurement.pv_production_emr_keys = [key]
measurement = get_measurement()
hourly = frame["ac_power"].resample("1h").mean()
hourly = hourly.loc[hourly.index < START]
outage_start = START.subtract(days=outage_days)
healthy_looking_gap = outage_start.add(days=2).date()
cumulative = 0.0
for timestamp, power_w in hourly.items():
measurement.update_value(
pendulum.instance(timestamp.to_pydatetime()), key, round(cumulative, 6)
)
in_outage = timestamp >= outage_start and timestamp.date() != healthy_looking_gap
bias = outage_bias if in_outage else healthy_bias
cumulative += float(power_w) * bias / 1000.0
measurement.update_value(
pendulum.instance(hourly.index[-1].to_pydatetime()).add(hours=1),
key,
round(cumulative, 6),
)
def _feed_native_quarter_hour_measurements(
instance: PVForecastAkkudoktorLocal, frame: pd.DataFrame, bias: float, key: str
) -> None:
"""Write cumulative PV readings at the provider's native 15-minute cadence."""
instance.config.measurement.pv_production_emr_keys = [key]
measurement = get_measurement()
slots = frame.loc[frame.index < START, "ac_power"]
cumulative = 0.0
for timestamp, power_w in slots.items():
measurement.update_value(
pendulum.instance(timestamp.to_pydatetime()), key, round(cumulative, 6)
)
cumulative += float(power_w) * 0.25 * bias / 1000.0
measurement.update_value(
pendulum.instance(slots.index[-1].to_pydatetime()).add(minutes=15),
key,
round(cumulative, 6),
)
def test_calibration_is_off_by_default(pvforecast_instance): def test_calibration_is_off_by_default(pvforecast_instance):
frame = pvforecast_instance._forecast_frame(synthetic_openmeteo(), calibrate=False) frame = pvforecast_instance._forecast_frame(synthetic_openmeteo(), calibrate=False)
assert pvforecast_instance._fit_calibration(frame) is None assert pvforecast_instance._fit_calibration(frame) is None
@@ -246,6 +303,84 @@ def test_calibration_recovers_a_systematic_bias(pvforecast_instance):
assert corrected["ac_power"].sum() == pytest.approx(frame["ac_power"].sum() * global_factor) assert corrected["ac_power"].sum() == pytest.approx(frame["ac_power"].sum() * global_factor)
def test_calibration_excludes_recent_demand_limited_outage(pvforecast_instance, caplog):
"""A battery outage must not teach demand-limited PV as available generation."""
pvforecast_instance.config.pvforecast.provider_settings.PVForecastAkkudoktorLocal = (
PVForecastAkkudoktorLocalCommonSettings(
calibration_enabled=True,
calibration_days=5,
calibration_reference_days=14,
calibration_azimuth_bin_degrees=0,
calibration_min_factor=0.2,
)
)
frame = pvforecast_instance._forecast_frame(synthetic_openmeteo(), calibrate=False)
_feed_measurements_with_recent_outage(
pvforecast_instance,
frame,
healthy_bias=0.8,
outage_bias=0.2,
outage_days=5,
key="pv_demand_limited_emr",
)
with caplog.at_level("INFO"):
calibration = pvforecast_instance._fit_calibration(frame)
assert calibration is not None
global_factor, factors = calibration
assert global_factor == pytest.approx(0.8, abs=0.03)
assert factors == pytest.approx([global_factor])
assert "excluded probable outage/curtailment days" in caplog.text
# A single statistically healthy-looking day inside the outage is bridged.
assert "2025-06-12" in caplog.text
# Filtering only selects training data. Applying a global factor preserves the
# native quarter-hour shape instead of replacing it with hourly bucket values.
corrected = pvforecast_instance._apply_calibration(frame, factors, 10000.0)
producing = frame["ac_power"] > 0.0
assert corrected.index.to_series().diff().dropna().unique().tolist() == [
pd.Timedelta(minutes=15)
]
assert (
corrected.loc[producing, "ac_power"] / frame.loc[producing, "ac_power"]
).to_numpy() == pytest.approx(np.full(producing.sum(), global_factor))
def test_calibration_uses_real_quarter_hour_measurements(pvforecast_instance, caplog):
"""Native meter slots permit shape calibration without inventing intrahour data."""
pvforecast_instance.config.pvforecast.provider_settings.PVForecastAkkudoktorLocal = (
PVForecastAkkudoktorLocalCommonSettings(
calibration_enabled=True,
calibration_days=14,
calibration_reference_days=14,
calibration_azimuth_bin_degrees=0,
)
)
frame = pvforecast_instance._forecast_frame(synthetic_openmeteo(), calibrate=False)
_feed_native_quarter_hour_measurements(
pvforecast_instance, frame, bias=0.8, key="pv_quarter_hour_emr"
)
assert (
pvforecast_instance._calibration_interval_minutes(
START.subtract(days=14), START
)
== 15
)
with caplog.at_level("INFO"):
calibration = pvforecast_instance._fit_calibration(frame)
assert calibration is not None
global_factor, _ = calibration
assert global_factor == pytest.approx(0.8, abs=0.03)
assert (
"15-minute measurement resolution" in caplog.text
or "15-minute intervals" in caplog.text
)
def test_calibration_factor_is_clamped(pvforecast_instance): def test_calibration_factor_is_clamped(pvforecast_instance):
"""A wildly wrong meter must not be allowed to swing the forecast.""" """A wildly wrong meter must not be allowed to swing the forecast."""
pvforecast_instance.config.pvforecast.provider_settings.PVForecastAkkudoktorLocal = ( pvforecast_instance.config.pvforecast.provider_settings.PVForecastAkkudoktorLocal = (
@@ -273,6 +408,73 @@ def test_calibration_respects_the_inverter_cap(pvforecast_instance):
assert corrected["ac_power"].max() <= 10000.0 + 1e-6 assert corrected["ac_power"].max() <= 10000.0 + 1e-6
def test_azimuth_calibration_is_interpolated_smoothly():
"""Azimuth correction must not introduce steps into the quarter-hour plan."""
frame = pd.DataFrame(
{
"solar_azimuth": [44.9, 45.0, 45.1, 134.9, 135.0, 135.1],
"dc_power": [1000.0] * 6,
"ac_power": [1000.0] * 6,
}
)
corrected = PVForecastAkkudoktorLocal._apply_calibration(
frame, np.array([0.5, 1.0, 1.5, 1.0]), ac_cap_w=2000.0
)
assert corrected.loc[1, "ac_power"] == pytest.approx(500.0)
assert corrected.loc[4, "ac_power"] == pytest.approx(1000.0)
assert abs(corrected.loc[2, "ac_power"] - corrected.loc[0, "ac_power"]) < 2.0
assert abs(corrected.loc[5, "ac_power"] - corrected.loc[3, "ac_power"]) < 2.0
def test_azimuth_shape_preserves_each_days_global_energy():
"""The EMS gets a changed shape without a changed daily energy budget."""
index = pd.date_range("2025-06-01", periods=8, freq="12h", tz="UTC")
frame = pd.DataFrame(
{
"solar_azimuth": [45.0, 225.0, 45.0, 225.0, 45.0, 225.0, 45.0, 225.0],
"dc_power": [100.0, 300.0, 200.0, 200.0, 300.0, 100.0, 150.0, 250.0],
"ac_power": [100.0, 300.0, 200.0, 200.0, 300.0, 100.0, 150.0, 250.0],
},
index=index,
)
global_factor = 0.8
corrected = PVForecastAkkudoktorLocal._apply_calibration(
frame,
np.array([0.5, 1.0, 1.5, 1.0]),
ac_cap_w=10_000.0,
global_factor=global_factor,
timezone="UTC",
)
raw_daily = frame["ac_power"].resample("1D").sum()
corrected_daily = corrected["ac_power"].resample("1D").sum()
assert corrected_daily.to_numpy() == pytest.approx(
raw_daily.to_numpy() * global_factor
)
assert np.std(corrected["ac_power"] / frame["ac_power"]) > 0.01
def test_azimuth_shape_fit_preserves_global_energy(pvforecast_instance):
"""Intraday correction must not undo the independently fitted daily kWh."""
centers = np.arange(22.5, 360.0, 45.0)
azimuth = np.repeat(centers, 20)
modelled_kwh = np.ones(len(azimuth))
expected_shape = np.repeat([0.8, 0.9, 1.0, 1.1, 1.2, 1.1, 1.0, 0.9], 20)
global_factor = 0.8
measured_kwh = modelled_kwh * global_factor * expected_shape
factors = pvforecast_instance._fit_azimuth_factors(
modelled_kwh, measured_kwh, azimuth, global_factor
)
fitted_scale = pvforecast_instance._interpolate_azimuth_factors(azimuth, factors)
assert np.std(factors) > 0.01
assert np.dot(modelled_kwh, fitted_scale) == pytest.approx(
global_factor * modelled_kwh.sum(), rel=1e-6
)
def test_forecast_frame_applies_the_calibration(pvforecast_instance): def test_forecast_frame_applies_the_calibration(pvforecast_instance):
"""The correction must reach every caller of the chain, not just `_update_data`.""" """The correction must reach every caller of the chain, not just `_update_data`."""
pvforecast_instance.config.pvforecast.provider_settings.PVForecastAkkudoktorLocal = ( pvforecast_instance.config.pvforecast.provider_settings.PVForecastAkkudoktorLocal = (