On a converged population the boost was permanently on, so there was nothing
left for it to intervene in. Its trigger, DIVERSITY_BOOST_THRESHOLD at 0.35, sat
above SELECTION_DIVERSITY_FLOOR at 0.30 - the floor the selection itself
guarantees - so `diversity < threshold` was true in every generation after
convergence. The threshold now sits below the floor.
The immigrants the boost injects are by construction the worst individuals in
the pool, and `_select_diverse` ran a plain tournament over parents and
offspring together, so they were removed in the very generation that created
them and their genes never recombined. A bounded share of seats
(IMMIGRANT_PROTECTION_FRACTION) is now reserved for them for
IMMIGRANT_PROTECTION_GENERATIONS selections. The incumbent is protected by
genome key, so no immigrant can evict the best solution or an equal-genome twin,
and offspring do not inherit the protection.
The log line was edge-triggered on `diversity_boost_active`, which was also
cleared on every fitness improvement, so a running boost re-announced itself
with "stagnation 0" while a boost that never stopped looked like several short
ones. It now tracks the boost alone, and the end of a boost is logged too.
Measured on a converged population: without protection 0 of 12 immigrants
survive the selection, with it all 12 do, and the incumbent is kept either way.
A battery or inverter failure limits PV to local demand for days. The
calibration read that as the plant's true output and learned it as a permanent
model loss, so one outage degraded the forecast long after the hardware was
fixed.
Calibration now estimates the healthy plant ratio over
`calibration_reference_days`, excludes days below `calibration_outage_threshold`
of it, and falls back to the most recent `calibration_min_healthy_days` when the
normal window is contaminated. `calibration_outage_filter_enabled` turns this
off for plants where measured curtailment, not available potential, is the
prediction target.
The fit also uses native 15-minute meter readings when every configured PV meter
supplies them - never interpolating hourly counters into an invented
quarter-hour profile - interpolates azimuth factors smoothly between bin centres
instead of stepping the EMS input curve, and normalizes the shape per forecast
day so it redistributes energy without changing that day's kWh correction. The
default azimuth bin widens from 15 to 45 degrees, which is what a typical
window actually supports.
Fixes the calibration window itself: it was derived from the measurement store
as a whole rather than from the configured PV production meters. A load meter
reaching further than the PV meter placed the window where no PV reading exists,
so calibration silently fell back to hourly fitting or skipped itself.
Also fixes `Measurement.load()`, which discarded every stored record. It
validated the file into a "temporary" Measurement, but Measurement is a
singleton, so that instance was the live one and the parsed records were
dropped.
The optimizer treated the end of `optimization.horizon_hours` as the end of the
world: energy left in the battery there was worth a single configured price per
kWh, so it either dumped the battery into the last hours or hoarded it,
depending on that one number.
The horizon is now two spans. `horizon_hours` still receives every control
command. The new `optimization.tail_horizon_hours` (default 48 h) is a pure
lookahead that never produces a command. In AUTO terminal-value mode a
deterministic dynamic program solves that tail backwards on a 101-point SoC
grid using the production battery and inverter models - SoC bounds, power caps,
conversion losses, configured charge and export rates, direct-marketing
permission and LCOS on delivered DC energy - and the existing AUTO proxy
supplies the continuation value at the tail end. Genetic fitness reads the
resulting curve. `tail_horizon_hours: 0` restores the plain proxy at the control
end, FIXED is unchanged.
The forecast budget is reported, never enforced by refusal: a tail that does not
fit is shortened to what the forecast covers and reported as
`effective_tail_hours`, and a control horizon that does not fit is warned about
at configuration time and rejected by the optimizer at run time, which knows
which series ran out. `prediction.hours` defaults to 72 so the new defaults fit
out of the box; existing shorter configurations keep starting.
Control arrays and warm-start genomes now begin at the run timestamp rather than
midnight, flagged by `controls_start_at_now` so the adapters still read older
solutions. `forecast_interval_seconds` declares the resolution of shortened
native quarter-hour inputs.
Required forecasts are no longer silently replaced by demo providers. A missing
PV, price, load, feed-in or weather forecast used to rewrite the configured
provider and retry, so a run could quietly optimize against invented data.
Missing values now stay missing, and provider values are held only within their
own source interval instead of being extended indefinitely.
Also fixes a config update that could leave EOS half-updated: the merged
candidate is validated before the singleton is reinitialized.
Four provider tests that hard-coded the old 48 h prediction default are rewritten
to derive their expectations from the configured horizon.
test_configmigrate leaves a <fixture>.new working copy behind whenever a
migration test fails, test_visualize writes example_report.pdf into the current
directory, and test_config creates tests/testdata/docs/. All three showed up as
untracked noise in every status and invited accidental commits.
Add PVForecastAkkudoktorLocal, which runs the modelling chain inside EOS on
raw Open-Meteo irradiance instead of calling a forecast service: solar
position, horizon shading, plane transposition, incidence-angle modifier,
cell temperature, PVWatts DC and inverter AC.
It needs no API key and serves up to 16 days at 15-minute resolution from a
single hourly request, which is what keeps `optimization.tail_horizon_hours`
fed - services wrapping Open-Meteo cut the horizon much shorter. Several
Open-Meteo models can be listed in `weather_models` and are averaged per
variable at no extra request cost.
With `calibration_enabled` the provider fits itself against
`measurement.pv_production_emr_keys` over the past `calibration_days`: a
global scale factor plus optional per-solar-azimuth factors, each weighted by
modelled energy, shrunk toward the global factor by `calibration_prior_kwh`
and clamped to `[calibration_min_factor, calibration_max_factor]`. The
comparison runs on past intervals, where Open-Meteo serves analysed rather
than forecast weather, so it corrects the error of the PV model and not that
of the weather forecast.
Calibration is a scale factor on the output and never touches `userhorizon`,
`surface_tilt`, `surface_azimuth` or `peakpower`. The docs say so, and say
why a short window and a plant fault inside it are the two ways to end up
with a misleading factor.
Also add `Measurement.pv_production_total_kwh()` alongside the existing load
total, and `scripts/pvforecast_backtest.py`, which scores configuration
variants against the stored meter readings without waiting for new forecasts
to come true.
Everything up to the knee is backed by residual load, everything beyond
it only by an export at the median feed-in tariff - the two halves have
very different confidence. The breakpoint list alone does not say where
the split is, so consumers had to guess it from the segment lengths.
A run whose price forecast is all zeros produces no priced residual load,
so AUTO cannot derive a curve and quietly credits the request scalar
instead. The reported mode was then "FIXED" - indistinguishable from a
run actually configured that way.
The solution now carries a reason, and the fallback is logged as a
warning instead of passing unnoticed.
The energy still stored when the horizon ends keeps its worth: it replaces
grid imports that are paid for afterwards. Crediting that with a single
price per kWh cannot describe it, because the value is not linear in the
amount stored. The first kWh replaces the most expensive hour that PV
cannot cover, the next one the second most expensive, and once every such
hour is served, further energy replaces nothing.
A scalar has to pick one slope for all of it. High enough for the first kWh
means hoarding a full battery; low enough for the last kWh means running it
empty by the end of the horizon - which is exactly what the previous default
of 0 EUR/kWh did.
terminal_value_mode = AUTO (the new default) builds the curve instead. There
is no forecast beyond the horizon, so its trailing window stands in for the
day that follows: residual load max(load - PV, 0) per slot, priced at its
import price, sorted and accumulated. LCOS is subtracted from every marginal
value so stored energy is not credited twice, and the tail beyond the
residual load is only credited when direct marketing allows an export. The
curve is built once per run; the search only interpolates on it.
The solution reports what a run used as terminal_value, curve included, so
the shape can be inspected instead of guessed. FIXED restores the previous
scalar behaviour.
In a 48 h scenario with two cheap slots at the end, AUTO keeps the battery
at 50 % and credits 3.85 EUR where FIXED with 0 EUR/kWh drains it to empty.
The stored optimization results move accordingly - the objective changed.
With grid charging switched off (inverter.max_ac_charge_power_w = 0) the
simulation zeroes the AC charge array before using it, but it did so by
rebinding a local name. The solution is read back from the original array,
so it still carried the optimizer's AC charge genes - genes the fitness
never evaluated, because the simulation ignored them, and which are
therefore arbitrary.
The effect was visible as a slot with ac_charge = 0.8 where the battery SoC
does not move. Harmless inside EOS, but a controller that follows the plan
would grid-charge the battery at a time nobody planned or paid for.
Zero the array in place so the reported plan matches what was simulated.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Two questions that came up while using the new fields: how binding the EV
charging deadline actually is (soft, but the default penalty outweighs the
energy cost by roughly forty to one, and the target is met tightly), and
why grid_export_rates can vanish from the saved config file (values equal
to the default are omitted; /v1/config shows what is effective) together
with when a partial export level pays off at all.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Three related scheduling improvements, all opt-in and behaviour-preserving
when the new fields are not set.
Flexible consumers get absolute time bounds next to the recurring
time_windows: earliest_start_datetime and deadline_datetime, where the
deadline requires the complete run to have *finished* before that moment
("clean dishes by 03:00 tonight"). When no start can meet it,
deadline_policy decides between BEST_EFFORT (run as early as possible, so
the delay rather than the cost is minimized) and STRICT (keep the
deadline; a ONCE consumer then fails the optimization). The solution
reports appliance_deadline_missed per device.
The EV charging target can be given the same kind of deadline, as an
absolute min_soc_deadline_datetime and/or a relative min_soc_max_duration_h
("full in 6 hours"), the earlier of the two winning. The ev_soc_miss
penalty is then evaluated at that slot instead of at the end of the
horizon, and the seeding heuristic only proposes charge slots before it.
Battery-to-grid export under direct marketing is no longer all-or-nothing:
grid_export_rates configures the selectable export levels as a factor of
the rated discharge power (default [0.25, 0.5, 0.75, 1.0]). Each rate is
its own optimizer state, with the full-power state keeping its previous
index so existing seeds and heuristics are unaffected. The chosen level
per slot is reported in battery_grid_export_factor and as the
GRID_SUPPORT_EXPORT operation factor.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
A plane's userhorizon uses the same convention as the Forecast.Solar
horizon query parameter (evenly distributed heights in degrees, starting
north, clockwise), so pass it through instead of dropping it. Without it
a shaded plane is forecast as if it had a free horizon.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Replace the single hourly "dishwasher" home appliance with a list of
flexible consumers (home_appliances). Each consumer defines its load
either as an explicit power profile (energy-preservingly resampled onto
the optimization slot grid, incl. 15-min and non-integer interval ratios)
or the flat consumption_wh/duration_h fallback, and runs ONCE or DAILY
within its time windows and the optimization horizon.
- ConsumerScheduleMode + shared load-definition validation (XOR of
profile/fallback, reject negative/NaN/inf, unique device_id)
- ApplianceGeneLayout: variable appliance gene block (index into
allowed_start_slots), ONCE/DAILY calendar-day based, no snapping
- per-device output: result.home_appliance_energy_wh, appliance_starts
(absolute local times), per-device solution columns and DDBC RUN/OFF
instructions on state transitions only
- deprecate dishwasher/washingstart/Home_appliance_wh_per_hour with
backward-compatible mapping and explicit conflict rejection
- max_home_appliances is now an upper bound only; no demo appliance and
no on/off behaviour
- docs, openapi.json, CHANGELOG and optimize_result_2* fixtures updated;
new tests/test_homeappliance.py covers the mandatory test matrix
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Request QUARTER_HOURLY exchange prices from Tibber and store them at their
native resolution instead of pre-averaging to hourly values. EOS resamples the
stored records onto the optimization grid on demand (key_to_array), so keeping
the native step size lets both the hourly (interval=3600) and the 15-minute
(interval=900) optimizer be fed the correct grid automatically.
- GraphQL: priceInfoRange resolution HOURLY -> QUARTER_HOURLY (last 960).
- _hourly_series -> _normalize_series: dedupe by timestamp (mean) + sort, no
1h aggregation; add _resolution_seconds (median of timestamp diffs, fallback
3600s).
- Resolution-agnostic ETS extrapolation: seasonal windows and history
thresholds are scaled by slots_per_hour, needed forecast length and the
prediction index step are computed in slots. Hourly behaviour is unchanged
(slots_per_hour=1 -> 168/24 seasonal periods, hourly steps).
- Tests: replace the 1h-averaging test with resolution-preserving + dedup
tests, assert QUARTER_HOURLY in the query, add a 15-min end-to-end test
(native storage stays 15min, slot-based seasonal periods = 96, 15-min
forecast index). Hourly backward-compat tests stay green unchanged.
The genetic optimizer was hard-wired to an hourly grid and forced
optimization.interval to 3600 s. Generalize it to a configurable slot grid
of length prediction.hours * (3600 / interval), accepting 900 (15 min) in
addition to the default 3600 (1 hour) so the optimizer can schedule on a
quarter-hour grid for 15-minute dynamic electricity tariffs.
- genetic.py: slot_duration_h / slots_per_hour / total_slots helpers; all GA
vectors sized by total_slots; simulate()/evaluate() indexed by start slot.
- geneticparams.py: allow {900, 3600}; scale the load power series to per-slot
energy, mirroring the PV series.
- battery.py / inverter.py: scale power caps to per-slot energy caps via
slot_duration_h; homeappliance.py carries the hook.
- geneticsolution.py: serialize solution and plan on the slot grid (interval
freq, start-slot offset, second-based instruction instants).
The default 3600 s interval keeps the previous hourly behaviour; the genetic
regression suite is unchanged. Adds tests for the 15-minute slot grid.
Fügt einen Direktvermarktungs-Modus (feedintariff.direct_marketing_enabled)
hinzu, der den Börsenpreis als Einspeisevergütung nutzt und aktive
Batterie-Entladung ins Netz (battery_grid_export_allowed) sowie
DC-Charge-Bypass optimiert.
- FeedInTariffEnergyCharts-Provider (Börsen-Einspeisetarif inkl. Prognose)
- Inverter: DC/AC-Wirkungsgrade und Batterie-Grid-Export in process_energy
- Genetik: Export-/DC-Charge-Zustände, Restwert-Bewertung des Akkus
- Solution-Result: neues Feld Feed_in_tariff (verwendeter Tarif je Stunde)
- Tests für neue Provider, Solution und Simulation
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
tests/test_docstringrst.py scans every class/function member of each
module via inspect.getmembers() with no __module__ filter, so
`from urllib.parse import quote` pulled the stdlib quote() into the
pvnode and Solcast provider namespaces and its non-reST docstring failed
the docstring-compliance check.
Import `urllib.parse` as a module and call `urllib.parse.quote(...)`
instead: a module member is skipped by the isfunction/isclass scan, and
the fully-qualified call keeps mypy happy (the requests stubs lack quote,
which is why urllib.parse was chosen over requests.utils in the first place).
test_all_docstrings_rst_compliant now passes; isort/ruff/ruff-format/mypy
pre-commit hooks all green.
- Use urllib.parse.quote instead of requests.utils.quote in the pvnode and
Solcast providers: the runtime re-export exists, but the requests type stubs
do not declare it, so the pre-commit mypy hook failed with
'Module has no attribute "quote"'.
- Mark the force_update keyword in the new provider tests with `# type: ignore`
— it is consumed by the cache_in_file decorator at runtime; same call
convention and ignore style as pvforecastakkudoktor.py.
- Add PVForecastPVNode, PVForecastForecastSolar and PVForecastSolcast to the
expected provider sequence in tests/test_prediction.py (fixture + index
assertions) — the two sequence tests failed because the new providers were
registered in prediction.py but missing from the hardcoded expectations.
Add provider descriptions and configuration examples for the three new PV
forecast providers to the prediction guide, a CHANGELOG entry, and regenerate
the affected auto-generated config docs.
Add PVForecastSolcast for the Solcast rooftop-site API. The operator registers a
site in the Solcast web app and enters the API key + resource (site) id:
GET /rooftop_sites/{site_id}/forecasts. pv_estimate (kW) is converted to watts
and fed as pvforecast_ac_power; the timestamp is normalised to the period start
(period_end - period) so it aligns with the resample axis. period_end is UTC.
Completes the set of selectable cloud PV forecast providers
(Akkudoktor, VRM, Import, pvnode, Forecast.Solar, Solcast). Adds tests for the
kW->W conversion, period-start normalisation, ISO-8601 period parsing, the
request URL/auth and HTTP-error handling.
Andreas
AndreasandGitHub
ChristinandAndreas
NormannandGitHub
