mirror of
https://github.com/Akkudoktor-EOS/EOS.git
synced 2026-11-13 01:03:45 +01:00
eb4b34f194af89938495fc6d6369a4cfbb9ef4db
174
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
  
|
43cb0f04b2 |
docs(eosdash): describe the setup assistant (#1516)
Add a how-to page for the first setup with the EOSdash setup assistant and link it from the documentation index, the getting started tutorial and the README. The status of the measurements step is derived from the configuration only. Say what that means for a system that sends the state of charge on its own. Add the new Home Assistant step to the list of steps, say that the Home Assistant adapter needs EOS to run as an add-on and that an automation has to act on the instructions, and update the screenshot to the eleven steps. The devices step of the setup assistant now asks for the size of each unit, because the packaged battery and inverter profiles are generic and give no capacity or nominal power. Say so on the setup page, point to the Profiles page and link the battery configuration and the inverter profiles. Signed-off-by: SchaafAlexander <86607129+SchaafAlexander@users.noreply.github.com> Co-authored-by: Claude Fable 5.2 <noreply@anthropic.com> |
||
 Bobby NoelteandClaude Opus 5.5
|
faf37abfe7 |
feat(devices): add inverter profiles
An inverter profile describes the technical data of an inverter model: its nominal AC power, the DC→AC and AC→DC efficiency curves over the load fraction (AC power relative to the nominal power), the AC charge power limit of the model, whether that limit also limits PV charging, and an optional power derating over the ambient temperature. Curves are read linearly and clamped at their ends, so every lookup yields a value. The packaged profile generic-hybrid has typical hybrid inverter efficiencies, no nominal power (the inverter device has to provide it) and full power up to 45 degC. Inverter settings name their profile (inverter_profile, default generic-hybrid) and hold the differences of the unit in profile_adjustment: nominal_power_w (replaces the profile value, required for a generic profile), max_power_w and max_ac_charge_power_w (installation limits that can only lower; an AC charge limit of 0 disables AC charging). The effective parameters are resolved at the ambient temperature of the inverter, given by the new measurement key <device_id>-temperature-degc (45 degC without a fresh value); the power derating lowers the maximum power and the AC charge limit. Optimization parameters are always valid: the GENETIC and GENETIC0 parameter preparation resolve the inverter with resolve_current_operating_parameters_or_defaults, which never raises. An unusable profile gives the generic-hybrid profile, a missing nominal power the fallback of 10000 W, an inverter derated to zero power the parameters at 45 degC, and no usable profile at all the profile adjustment with constant efficiency; each is logged as error. The GENETIC inverter takes the AC→DC efficiency curve of the profile, solved exactly on each linear piece of the curve (about 2 % longer runs); GENETIC0 keeps constant efficiencies and no longer replaces the configured inverters with the demo inverter; a battery, electric vehicle or inverter that cannot be prepared is left out of the GENETIC0 run, never replaced by a demo device. Configurations migrate: max_power_w becomes profile_adjustment.nominal_power_w, max_ac_charge_power_w moves into the adjustment, an ac_to_dc_efficiency of 0 keeps AC charging disabled, and the efficiency fields are dropped with an error log, as for batteries, so the profile efficiencies apply. Home Assistant config paths follow. Additional changes - Move the curve algorithms of the battery profile into profilecurve, so battery and inverter profiles share them; the battery profile keeps its document format and classes. - Move ProfileBaseModel, the strict base of profile documents, into profilestore. - Serve inverter profiles through the profile REST API. Refuse to delete a profile that a configured device uses and that has no packaged profile to revert to (409). - Share the measurement access of the device settings in devicemeasurement. - Rename check_battery_profiles to check_device_profiles; the device profile check at configuration setup covers inverters, too. - Document inverter profiles, their configuration, defaults and sharing; regenerate the configuration and OpenAPI documentation. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Assisted-by: Claude Code (claude-opus-5-5) |
||
|
Bobby NoelteandClaude Opus 5.5
|
e255f987bf |
feat(devices): add battery profiles
A battery profile describes a battery model: the charge and discharge limit
maps over temperature and SoC, the charge and discharge efficiency curves
over C-rate, the technical SoC bounds, the hard temperature cutoffs and,
for a model, its nominal capacity and power limits. EOS packages the
generic profile generic-lfp; users add their own profiles.
A battery or electric vehicle selects its profile by battery_profile and
describes how the installation differs from it in profile_adjustment
(capacity, power limits of the installation, periodic full charge). The
effective values are resolved from both at the measured operating point
(temperature, SoC, power) and shown as effective_* fields.
The GENETIC simulation uses the limits and efficiencies of the profile over
SoC, with precomputed tables as a fast path; GENETIC0 gets the values at the
operating point. Optimization parameters are always valid: a battery whose
profile is missing, unreadable or invalid uses the packaged generic-lfp
profile, and the former defaults only if that is not usable either; each is
logged as error, and an optimization run is never cancelled because of a
battery.
Additional changes
- Add a generic device profile store: user and packaged profiles, atomic
saves, compiled profiles cached per file and checked again at most once a
second, plain file names as profile names ('schema' is reserved). On
Windows, a save or read that collides with the other is retried briefly.
- Add the REST API /v1/profile to list the profile kinds and to list, read
(document and JSON schema), save and delete profiles.
- Check the hard temperature cutoffs on the exact temperature.
- Resolve inhibited charging or discharging to a power of 0, e.g. for a cold
battery; give GENETIC0 its own discharge limit for that.
- Limit generic profiles without own power limits to the former 5000 W.
- Accept only the implemented units; current limits need the nominal
voltage.
- Migrate legacy battery settings into the profile adjustment, keep the
former charge power limit as discharge limit (also for a Home Assistant
entity on it) and report dropped efficiencies as errors.
- Limit GENETIC battery discharge by the discharge power.
- Allow battery profile lookups while the configuration is created.
- Resolve a battery once per configuration dump.
- Package the bundled profiles, document battery profiles and regenerate
the configuration and OpenAPI documentation.
- Ignore the report PDFs written by the visualization tests.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Assisted-by: Claude Code (claude-opus-5-5)
|
||
|
Bobby NoelteandClaude Opus 5.5
|
dd41e5d57c |
chore(ai): add agent self-review and a CI check for agent-assisted PRs
Additional changes Agents write a task plan to .scratch/<branch>/plan.md and self-review with code-review Agents fix hard violations, report judgement calls and rerun the self-review before every push Agents declare the review in a Self-review section of the pull request body Add the ai-self-review workflow and scripts/check_pr_self_review.py for pull requests into main Detect agents by Assisted-by or Anthropic Co-Authored-By trailers, body markers or the AI box Fail only on a missing Self-review section or a missing, malformed or stale Reviewed SHA Note an unticked AI-assisted box and malformed optional lines without failing Comment on the pull request how to fix a failure and update that one comment on later runs Show the same report in the job summary and accept Windows line endings in the body Skip dependabot and release pull requests Run on pull_request_target with the base branch's script, never pull request code Add the pull request template: the pr skill's sections plus Checklist and Self-review Require always valid optimization parameters in the energy-domain skill Require datetimeutil for date and time handling in new and changed code in AGENTS.md Require the Assisted-by trailer on agent commits in the commit-messages skill Add tests for the check in tests/test_ai.py Add GLOSSARY.md and the agent tracker docs in docs/agents/ Add decision record docs/adr/0001 with an ADR index in the docs toctree Document the workflow in docs/develop/ai.md and CONTRIBUTING.md Exclude docs/agents from Sphinx and PyMarkdown and ignore .scratch/ Document cz check instead of make gitlint for commit messages in docs/develop/ai.md Assisted-by: Claude Code (claude-opus-5-5) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
62cbf4cf2a |
fix(homeassistant): apply 0 and off from config entities, never add settings
The configuration sync of the Home Assistant adapter skipped every value that is false in Python, so an entity state of 0 or off never reached the configuration. It also set any path it was given: the documented example devices/batteries/0/capacity_wh named a device "0" that does not exist, so EOS silently added a new battery with a random device id. Apply every value except an unavailable or unknown entity, and only update settings that exist; report other paths as an error. The example now addresses the battery by its device_id. Signed-off-by: SchaafAlexander <86607129+SchaafAlexander@users.noreply.github.com> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
Bobby Noelte
|
93ccb8d592 |
chore: development version update (incl. home assistant config)
Signed-off-by: Bobby Noelte <b0661n0e17e@gmail.com> |
||
 ChenYi-NnergyandGitHub
|
38b07f5a5b |
fix: apply inverter_paco cap and system loss in PVForecastPVLib - #1504
Planes sharing one inverter_paco feed a single physical inverter, but their AC output was summed without any cap, so a 15 kW inverter could report 21-25 kW. This PR groups the per-plane AC output by inverter_paco and clips each group's sum to its rating; planes without a rating are left alone. It also applies the configured system loss (PVGIS-style percent, same semantics as PVForecastAkkudoktorLocal) to DC and AC output instead of warning that it is unsupported. |
||
|
Bobby NoelteandClaude Opus 5.5
|
d68f040206 |
feat(ai): add the mattpocock engineering skills for AI agents
Vendor the engineering skills of https://github.com/mattpocock/skills into ai/skills/engineering, with their MIT license, and list them in AGENTS.md. The agent-specific metadata (agents/openai.yaml) is kept for other agents. Additional changes - Add scripts/update_mattpocock_skills.py to compare the copy with upstream by git blob hashes from a blobless clone and to update it on request. - Add tests/test_ai.py; it warns when the copy differs from upstream and caches the upstream hashes in the pytest cache for a day. - Document the engineering skills in docs/develop/ai.md. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
Bobby NoelteandClaude Opus 5.5
|
57132df44e |
fix(config): enable documentation mode only explicitly
ConfigEOS.documentation_mode() was true as soon as Sphinx was imported anywhere in the process. A test importing Sphinx therefore switched all later tests into documentation mode, and test_config_md_current failed depending on test order. Documentation mode is now enabled only by the documentation generators (_force_documentation_mode) or by the Sphinx configuration, which sets sys._called_from_sphinx in docs/conf.py. Declare _force_documentation_mode as a ClassVar. Without the annotation pydantic turned it into a private attribute, and the class attribute was a truthy descriptor object. Behavior change: because of that descriptor, every running EOS was in documentation mode, so the get_adapter/config/measurement/prediction/ems/ database/resource_registry() guards never raised "access before init" and created the singletons lazily on first access. They now raise for every access before initialization. The worker process of uvicorn's auto-reload (server.reload, make run-dev) relied on that: the app's lifespan now sets up the configuration and initializes the singletons that do not exist yet. Leave runtime state out of the documentation: the Home Assistant entity ID properties and OptimizationCommonSettings.keys return an empty list in documentation mode, so entity IDs and solution keys of a running system no longer reach the generated configuration docs. Keep the tests isolated: test_doc sets documentation mode on the class and test_docstringrst restores sys._called_from_sphinx after reading the Sphinx configuration. Add tests for how documentation mode is enabled and for the runtime values it leaves out. Additional changes - Generate openapi.json in documentation mode, as the config docs. - Share the documentation mode check as coreabc.documentation_mode(). - Move the logging and server address setup of run_eos() to setup_eos_config(). - Test that EOS starts with server.reload. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
Bobby NoelteandClaude Opus 5.5
|
4475774c06 |
test(eosdash): run EOSdash in a headless Chromium browser
Run EOSdash in a headless Chromium browser for testing. Every menu tab must load with HTTP 200 and render content, in light and dark mode, without JavaScript errors. Additional changes Enable the browser tests by --browser-test (marker browser), skip them otherwise Add the CI job browser-test that installs Chromium and runs them Add the make target test-browser, run with --extra dev so playwright is installed Add playwright to the dev dependencies Document how to run the browser tests with --extra dev, incl. the browser system libraries on Linux Accept a child PID of the started server in the server startup checks, on Windows only: there the venv python.exe runs the real interpreter as a child process Share one Chromium per test class, so Playwright stops before other tests run Request chromium before servers so a missing browser fails before EOS starts Make the missing-browser hint platform neutral (Linux: add --with-deps) Set PYTHONIOENCODING=utf-8 for the test server to avoid encoding errors on Windows Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
2245976de4 |
docs(ai): move agent rules to ai/skills and add an AI development sandbox (#1498)
The sandbox provides an isolated Docker environment for AI-assisted development (OpenCode + Claude Code + Qwen Code) against the local AkkudoktorEOS repository. The sandbox does not protect the repository from harmful changes by the agent. It limits what a misled agent can reach on the machine, but the agent can still write any change into the repository. Primary security defence is to check every change before it is run on the host. Same holds for opening a pull request or updating one. Additional changes - Reduce AGENTS.md to the project summary, baseline rules, completion checklist, and an index of the skills. - Add skills for the Python environment and REST API error responses. - Add the development guide page docs/develop/ai.md: the AI coding workflow, how agents get the project rules from AGENTS.md and ai/skills, and how to use the sandbox (start, agent logins, project commands with a virtual environment outside the workspace, running EOS and EOSdash, GitHub access, persistent data, security model). - Add a checklist for reviewing agent changes before pushing, starting with .git, which stays writable in the sandbox. - Check out all text files with LF line endings (.gitattributes), also on Windows with core.autocrlf, so the sandbox scripts run in the container and git in the container agrees with git on the host. Batch files keep CRLF. - Publish the ports of ai/compose.yaml on request (AI_SANDBOX_PORTS=1). docker compose run publishes them only with --service-ports. Off by default, so that several sandboxes can run in parallel. Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
628226d3d9 |
feat(pvforecast): model snow on the modules in the local backend (#1484)
No PV forecast modelled snow lying on the modules, so after a snowfall the optimizer planned with PV that never came. The local Open-Meteo/pvlib backend can now reduce the forecast by the snow coverage (opt-in via pvforecast.akkudoktor.snow_enabled): - Request hourly snowfall and snow depth with the weather and run the NREL snow coverage model (pvlib.snow) per plane. - Correct the coverage with measured PV production: production above the model clears it at once; low production raises it only while snow is plausible, so curtailment and inverter outages are not learned as snow. Persisting snow damps the slide rate, fading with snow_feedback_memory_hours. - Rebuild the coverage from snow_lookback_days on every update instead of storing state. - Exclude intervals with possible snow from the calibration fit. - Warn when PV meter readings are missing or stale; ignore impossible meter jumps. _plane_power keeps its two-value signature (used by the adaptive provider in #1422); the irradiance comes from the new _plane_power_and_poa. Tests replay real February 2026 data from one plant (snow on 02-19/20). Signed-off-by: SchaafAlexander <86607129+SchaafAlexander@users.noreply.github.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
a06473b6d6 |
feat(battery): plan a periodic full charge of the stationary battery (#1486)
LFP batteries can only recalibrate their state of charge close to full charge, so many systems expect a full charge every few weeks and some start one on their own at an arbitrary, possibly expensive time. EOS can now plan it instead (GENETIC only, off by default). New battery settings: full_charge_interval_days (unset = off), full_charge_soc_percentage, full_charge_detect_percentage, full_charge_allow_grid and the informational cell_chemistry. The last full charge is read from the measured SoC history; nothing is stored. When the next one is due within the horizon (or overdue), the maximum SoC is raised to the target. With full_charge_allow_grid the optimizer must reach the target once before the deadline, which is at least 24 h away (penalty battery_full_charge_miss on the best SoC reached), so it picks the cheapest time itself. Without it, PV surplus may fill the battery but the target is not enforced. Signed-off-by: SchaafAlexander <86607129+SchaafAlexander@users.noreply.github.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
 NormannandGitHub
|
b33953d445 |
fix(pvforecast): normalize Forecast.Solar power before resampling (#1487)
Forecast.Solar supplies instantaneous power with irregular sunrise/sunset timestamps. GENETIC interprets stored values as interval averages and infers cadence from the shortest gap, so these samples create artificial coverage gaps. The prediction list API also averages away the sunset zero before interpolating, producing positive nighttime PV. Integrate each roof plane's piecewise-linear power curve into UTC-contiguous 15-minute interval-average W before summing planes. Both hourly and quarter-hour GENETIC slots then consume the same normalized data as the prediction API, with exactly one downstream W-to-Wh conversion. Explicit sunrise/sunset zero boundaries establish the adjacent night within each forecast day. Daylight gaps longer than an hour, invalid power, absent planes and unavailable forecast days remain missing; the strict GENETIC coverage check stays in place. Refreshes replace the current forecast and remove stale tails. Because cadence detection also sees historical records, the first refresh clears legacy forecasts containing non-aligned timestamps, including their history. Later refreshes retain normalized history. This upgrade behavior is documented. Forecast validation errors now identify the first invalid interval and the number of invalid slots. |
||
   
|
f804d48072 |
feat(pvforecast): add adaptive per-plane PV learning and diagnostics (#1422)
PV arrays with different orientations and curtailment need separate plant calibration and weather correction. This opt-in provider publishes normal EOS AC/DC forecast keys at 15-minute resolution. - Preserve daily per-plane energy/shape calibration and validated short-term local/trend selection. Curtailment-sensitive arrays require a fresh release flag or an explicitly configured learning guard. - Add switchable empirical solar-angle response relative to separate eligible reference arrays, removing their common weather error. This learns angular residuals, not physical angles or a measured geometric horizon. - Archive actually received weather model inputs, first local receipt and frozen per-member PV candidates. Returned past weather is excluded; unknown source issuance remains explicit. Learn model weights and forecast-regime bias separately at 2–6, 6–24 and 24–48 hours. - New long-horizon candidates remain in shadow until their genuinely issued outputs improve equal-day MAE by the configured margin, win on at least 60% of validation days and pass a matched-support energy-bias guard. Insufficient evidence retains calibration. Daily yields still require complete local days, including DST. - Expose fitted parameters, independent training/validation day counts, errors, selected methods and persistence in the existing status API, with additional normal prediction keys for Grafana. Retain physical DC through angular/weather corrections an enforce individual inverter AC limits. Co-authored-by: Normann <github@koldrack.com> Co-authored-by: Christin <chrissi@von-perbandt.de> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Co-authored-by: YMT <yn276364@dal.ca> Co-authored-by: sarankumar1325 <120080287+sarankumar1325@users.noreply.github.com> |
||
|
Bobby NoelteandGitHub
|
ef852c45a5 |
chore: development version update (incl. home assistant config) (#1480)
Signed-off-by: Bobby Noelte <b0661n0e17e@gmail.com> |
||
|
|
527c26d424 |
feat(eosdash): plan page with EMS plan timeline and persisted settings (#1460)
* feat(eosdash): plan page with EMS plan timeline and persisted settings Fixes: - SolutionCard no longer appends to the module-level solution_excludes list on every render (the list grew without limit). - All EOSdash pages: keep the scroll position after htmx actions instead of jumping to the bottom; Bokeh() reserves the plot height. - Default visible PV column pvforecast_power_w -> pvforecast_ac_energy_wh (the old column no longer exists). - All EOSdash pages: remove w-screen from components.Page (horizontal scrollbar). - Solution chart: energy is the main axis; the always empty power axis is only shown with power columns. Typo "Electricty" fixed. Features: - EMS plan timeline per device, plan table as collapsible section. - Solution chart rework: KPI row, legend, totals view, value axes for the displayed columns only, units menu, currency from the timezone, aligned time axes with the timeline, themed tooltips, system font. - Per-column and per-command settings: visibility, alias, color, style; device aliases. - New config section eosdash.plan (server/dash/dashsettings.py) stores the plan page settings in the EOS configuration; docs regenerated. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: SchaafAlexander <86607129+SchaafAlexander@users.noreply.github.com> |
||
|
Bobby NoelteandGitHub
|
7554e2dc47 |
fix(database): make delete_by_datetime(None, None) a true full wipe (#1476)
delete_by_datetime() with no bounds was documented as "delete all
records", but db_delete_records() assumed that
_db_ensure_loaded(start=None, end=None) loads every record into memory
before building the delete list. That is wrong: _db_ensure_loaded()
implements a NONE -> INITIAL -> FULL phase machine, and with both bounds
None in the NONE phase it loads only an *initial window* and leaves the
phase at INITIAL. Records outside that window never enter
_db_sorted_timestamps, so they are never marked for deletion and survive
in the physical store. Symptom: after a "wipe", db_count_records() still
returned N records (observed: 4 leftovers from a previous dense fixture).
A second delete_by_datetime() call happened to escalate INITIAL -> FULL
and deleted the rest, which is why the bug looked intermittent and why
stacked test runs accumulated rows.
Fix: in db_delete_records(), when both bounds are None, call
self._db_load_full() directly; otherwise keep the existing
_db_ensure_loaded(start, end) path. The delete list is then complete and
the "wipe" fixture actually wipes.
delete_by_datetime() remains a soft delete (rows are only added to
_db_deleted_timestamps; the physical DELETE happens on the next flush),
unchanged by this commit.
Besides this an additional fix was added:
* fix(tests): assert UTC-instant ordering in test_timestamps_remain_sorted
The test asserted `dts == sorted(dts)` on seq.records[].date_time, which
are tz-aware DateTime values in Europe/Berlin. Compaction buckets on a
UTC grid but records are materialized in local time, so across a DST
fall-back the local wall-clock sequence is non-monotonic even though the
UTC instants are ordered (e.g. 00:45 UTC -> 02:45 CEST, 01:00 UTC ->
02:00 CET). sorted() compares wall-clock fields, so it flags the pair at
the seam (index 1248 on 2026-10-25 for Europe/Berlin) as out of order.
The assertion is unsatisfiable for any 15-min grid spanning a fall-back
boundary when records are labelled in local time.
The actual invariant is "ascending by UTC instant", which is what
_db_sorted_timestamps guarantees and what the adjacent alignment tests
already assert via int(rec.date_time.timestamp()). Change the assertion
to compare integer UTC epochs:
epochs = [int(r.date_time.timestamp()) for r in seq.records if r.date_time]
assert epochs == sorted(epochs)
Note: this test was seasonally flaky. It only fails while a DST
transition lies inside the [now - 4 weeks, now] window used by
dense_energy_seq, which is why it newly started failing without any code
change. Upstream should ideally emit compacted records in UTC (or order
by UTC instant) so that wall-clock ordering is not silently assumed.
Signed-off-by: Bobby Noelte <b0661n0e17e@gmail.com>
|
||
|
NormannandGitHub
|
5d7198d090 |
test: consolidate suites and remove redundant cases (#1459)
Regression tests were scattered across feature- and bug-specific files, and some suites repeated the same checks. Group tests by responsibility, isolate feature fixtures in test classes, and remove duplicate or obsolete cases. The review covered all 127 collected test modules on main. This reduces them and removes or combines definitions:. No application code changes. |
||
|
NormannandBobby Noelte
|
d22628b312 | fix: honor effective time window boundaries in cycle scheduling | ||
|
Bobby Noelte
|
c02d250639 |
feat: Add month and end_time to time window configuration
Extend TimeWindow with two new optional fields: - month (1–12): restricts the window to a start date in the given calendar month. Useful for seasonal schedules. - end_time: explicit exclusive end-of-window clock time. The effective end is the later of end_time and start_time + duration. Primary motivation is DST handling — a wall-clock end_time stays correct across DST transitions, whereas a fixed duration does not. Rework window semantics so date, weekday, and month restrictions apply to the window's start date, not to every timestamp covered by the window. This makes overnight windows (e.g. Friday 22:00 → Saturday 06:00) behave intuitively under calendar constraints; contains() now checks both today's and yesterday's candidate start. duration becomes optional (default 0). All start/end/duration computations route through _window_start_end and a new _matches_start_date helper, so contains, earliest_start_time, latest_start_time, and available_duration stay consistent. Dashboard: add end_time and month inputs to the time-window editor, render the collapsed window summary as icons + pills. Docs, OpenAPI spec, and generated config docs are regenerated for the new fields. Tests cover month filtering, end_time boundary cases (incl. midnight and overnight), and start-date semantics for overnight windows. Bump version to support automatic updates in Home Assistant. Signed-off-by: Bobby Noelte <b0661n0e17e@gmail.com> |
||
|
Bobby NoelteandGitHub
|
8a8cd43a19 | chore: development version update (incl. home assistant config) (#1456) | ||
|
|
58b968562e |
fix(adapter): stamp Home Assistant device states with run time, not slot start (#1452)
* fix(adapter): stamp Home Assistant device states with run time, not slot start The Home Assistant adapter stored every reading with ems_start_datetime, which set_start_datetime() rounds down to the optimization slot (e.g. 14:00 for a run at 14:11). The GENETIC configuration request checks SoC freshness against the unrounded observation_datetime with measurement_max_age_seconds (default 300 s). Any run starting more than five minutes into a slot therefore discarded the freshly read SoC and optimized with 0 % initial SoC. Stamp device state measurements (device_measurement_entity_ids, e.g. SoC) with observation_datetime instead. Not to_datetime(): a reading taken a few seconds after the observation time would be rejected as a future sample. Meter readings (load, grid import/export, PV) keep the slot start, because their energy is aggregated per slot. With device states at their real time, a record holding only some keys can now be the newest one (e.g. the Home Assistant battery SoC while the EV SoC or a cycle count comes through the measurement API). Readers therefore skip records without the value they need: - GENETIC SoC and completed appliance cycles: such a record no longer hides a fresh value (the cycle count case aborted the optimization). This also covers a meter reading at the slot start that is newer than a SoC supplied through the measurement API. Values stored as pandas NA count as missing instead of raising TypeError; an explicit NaN stays an invalid sample. - GENETIC0 SoC: looks up the reading nearest to observation_datetime instead of the slot start, skipping records without SoC and converting only the selected sample. Before, a run late in the slot (14:40 in the 14:00 slot) would pick the previous run's reading (13:40). This also fixes SoC values PUT via the REST API with their real time. LoadAkkudoktorAdjusted anchored its hourly comparison window on the newest record of any key. An off-hour device state would shift every bucket (e.g. to :11) against hour-aligned meter readings; the window now starts and ends on full hours, and less than one full hour of data yields no adjustment instead of an error. (15-minute slot starts and SoC values PUT with their real time had the same effect before.) Regenerated openapi.json and docs/_generated/openapi.md (version only). Fixes #1447 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EYWR1HVM8a8src23xuyL3T * test(genetic0): use pv_battery instead of deprecated pv_akku mypy flags attribute access on the pv_akku computed field (attr-defined), which failed the pre-commit check of the PR. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EYWR1HVM8a8src23xuyL3T --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
Bobby NoelteandGitHub
|
024706b467 |
fix: use of inverter config dict in genetic0 (#1432)
Adapt GENETIC0 to inverters configuration in dict - used to be a list. Signed-off-by: Bobby Noelte <b0661n0e17e@gmail.com> |
||
|
Bobby NoelteandGitHub
|
5ad109cd22 |
chore: development version update (incl. home assistant config) (#1424)
Signed-off-by: Bobby Noelte <b0661n0e17e@gmail.com> |
||
|
AndreasandGitHub
|
cab088f09b |
fix(planning): accept full SoC targets and clarify unlimited EV SoC age (#1407)
* fix(devices): allow charging targets equal to the maximum SoC * test(planning): protect EV SoC without age limits |
||
|
AndreasandGitHub
|
49ea062dc8 | fix(genetic): allow optimization without fresh SoC (#1403) | ||
 
|
70b72bfbce |
feat(ems): notify a URL after every completed optimization (#1383)
* feat(devices): add load-dependent DC-to-AC efficiency curve for GENETIC inverters Real inverters are much less efficient at low load than at rated power (e.g. 0.80 at 2 % vs. 0.93 at 7 % load). A constant dc_to_ac_efficiency overrates night-time battery discharge into a small base load and, with it, arbitrage and grid charging. Add two optional inverter settings for the GENETIC optimizer: - dc_to_ac_efficiency_curve: list of (load_fraction, efficiency) points, load fraction = AC power / max_power_w. Points must be strictly increasing in [0, 1] with efficiencies in (0, 1]; at least two points. Linear interpolation, clamped at both ends. When set, the curve replaces dc_to_ac_efficiency. - dc_to_ac_efficiency_reference_load_fraction (default 0.06): load at which the curve values stored energy without a specific conversion. Battery discharges in Inverter.process_energy() use the efficiency at the AC energy of each conversion. The end-of-horizon value, the terminal and tail value curves and the AC charge break-even penalty use Inverter.reference_dc_to_ac_efficiency. Without a curve, dc_to_ac_efficiency_at() and the reference efficiency return dc_to_ac_efficiency itself, so results are bit-identical to the previous model. GENETIC0 is unchanged. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * feat(genetic): make the fitness cache size configurable The per-run fitness cache of the GENETIC optimizer is unbounded. It grows with individuals x generations (~110k keys at 300 x 400, about 1.1 kB each, 1.6 kB with an EV) and adds 120-170 MB to the peak memory of a run, while only 6-15 % of the lookups hit. On 512 MB boards this is the largest part of the optimizer's memory. Add optimization.genetic.fitness_cache_max_entries: - None (default): unbounded, the previous behaviour. - 0: no cache; every genome is evaluated. - N > 0: at most N keys. The cache is then an OrderedDict and evicts the oldest key first (FIFO, O(1)); LRU would cost bookkeeping on every hit for a cache that rarely hits. A cache hit returns exactly what the deterministic evaluation would return, so the plan for a fixed seed is identical with any setting. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * perf: import heavy optional dependencies on first use A GENETIC server with external (import) providers and EOSdash disabled loads several large packages at start that it never uses, which costs a noticeable part of the memory on 512 MB boards: - matplotlib via the GENETIC/GENETIC0 PDF report modules (server/eos.py) - statsmodels via the Holt-Winters fallback in prediction/priceabc.py - pvlib via the weather base class and the pvlib-based providers - bs4/lxml via the ClearOutside provider - scipy via the self-consumption interpolator singletons, which were loaded at import time - the GENETIC0 optimizer, imported by core/ems.py - tzfpy, whose polygon data the computed general.timezone loads Import these where they are used. The interpolator tables are loaded on first use (thread-safe). Nothing changes in what is computed. general.timezone is computed from latitude/longitude and cannot be set. Add general.timezone_override (IANA name, validated with pendulum): when set, general.timezone returns it and the lookup is skipped. When unset (default) the lookup is unchanged. Tests that patched the moved names now patch them at their source. A new test starts the server module in a subprocess and checks that none of these modules is loaded. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * perf(prediction): evaluate self-consumption tables without SciPy The GENETIC inverter (and the GENETIC0 one) evaluates a pickled scipy.interpolate.RegularGridInterpolator for the self-consumption probabilities. SciPy is used nowhere else in EOS, yet loading it costs about 35 MB RSS (21 MB anonymous) on Linux arm64, which every optimization run pays. Add utils/gridinterpolator.py: - LinearGridInterpolator: 2-D linear interpolation with the same steps as SciPy's linear 2-D path (interval search, normalised distances, corner weights summed in the same order, fill value outside the grid, NaN for NaN input). - load_grid_interpolator(): reads the shipped pickles with a restricted unpickler that maps the SciPy class to a stand-in and accepts only the NumPy globals needed for the arrays. SciPy is not imported, and the data files no longer depend on SciPy's private module paths. Results are bit-identical to SciPy on Linux (aarch64, checked on 3.6M points per table including the query pattern of the inverter). The macOS arm64 SciPy wheel fuses multiply-add, so there SciPy itself differs by <= 1.1e-16; GENETIC plans for fixed seeds are unchanged on both platforms. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * feat(prediction): make the SciPy-free table evaluation opt-in Add optimization.self_consumption_interpolator (scipy | numpy, default scipy). The default keeps evaluating the self-consumption tables with SciPy's RegularGridInterpolator, built from the same grid, values and fill value the pickle holds, so results are unchanged. With numpy the tables are evaluated by LinearGridInterpolator and SciPy is never imported. The backend is read on every evaluation, so a config change applies to the next run. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * feat(genetic): evaluate candidates in parallel worker processes A GENETIC run evaluated every candidate in the server process, so a run used one CPU core however many the machine had. On small ARM boards (4x Cortex-A55, thermally limited) a 300x200 run took 12-15 minutes. Candidates are now evaluated in up to two worker processes, forked once per run after the run is fully prepared. Each worker owns a copy of the prepared optimizer; only genomes and results cross the process boundary. The fitness cache stays in the server process: hits are answered there, and identical genomes in a batch are evaluated once, as a serial run would hit the cache for the repeats. Results are applied only after the whole batch returned. If a batch fails or times out, the run continues serially. For a fixed seed the result is identical to a serial run. optimization.genetic.workers sets the number of processes. Unset (default) is automatic: one CPU core always stays free and at most two are used (1-2 cores -> 1, 3 or more -> 2), capped by a cgroup v2 CPU limit of the process or any parent (container limit, systemd CPUQuota). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EYWR1HVM8a8src23xuyL3T * feat(ems): notify a URL after every completed optimization A client that consumes EOS plans (here DVhub) had to poll /v1/energy-management/optimization/solution to notice a new plan. That delays the plan by up to the poll interval and makes EOS serialize the complete solution on every poll, also while it is computing. ems.notify_url (default None = off) receives an HTTP POST with a small JSON event ({"event": "optimization_completed", "algorithm", "completed_at", "duration_s"}) when a run has published its solution. The POST runs in a daemon thread with a 5 s timeout, so a slow or unreachable receiver never delays energy management; a failure is logged once per failure streak. Only http(s) URLs are accepted. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EYWR1HVM8a8src23xuyL3T * fix(genetic): freeze the optimization configuration for the duration of a run The genetic optimization reads horizon_hours, tail_horizon_hours, interval_sec and prediction.hours from the live configuration on every access (control_slots, tail_slots, ... are properties). A client that updates the configuration while a run is in progress - DVhub shrinks or grows the control horizon as the known price horizon moves during the night - changed these values mid-run. Genomes and device arrays sized at the start then no longer matched the simulation loop and the final evaluation failed: IndexError: index 84 is out of bounds for axis 0 with size 84 Long runs on slow hardware hit this almost every night. optimize_ems() now takes a deep copy of the genetic configuration and the prediction hours at the start of a run, all reads in genetic.py go through it, and it is released when the run ends (also on errors). New values apply to the next run. Tests: tests/test_genetic_config_frozen_per_run.py reproduces the error without the fix (horizon 20 -> 24 mid-run: index 20 out of bounds) and passes with it; genetic/optimization tests 439 passed. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * feat(genetic): pin evaluation workers, keep the server responsive Workers are pinned to the last `workers` CPU cores (new setting optimization.genetic.pin_workers, default on); the server process stays unpinned. The run never spreads beyond `workers` cores without a CPU quota, and the server and the rest of the system keep the other cores. No pinning without a spare core. With a CPU limit (container cpu.max, systemd CPUQuota) the automatic worker count now keeps a quarter core of it for the server process (limit 2 -> 1 worker, 2.5 -> 2): two workers used a 2-core limit fully. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EYWR1HVM8a8src23xuyL3T * fix(ems): tolerate settings without notify_url Test doubles and older configuration objects have no ems.notify_url. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EYWR1HVM8a8src23xuyL3T * fix(devices): evaluate the efficiency curve at the delivered AC load When the battery limits a discharge (low state of charge or its power limit), the conversion ran with the efficiency of the requested AC energy. With a 1000 Wh battery at 10 % SoC, a 1000 W inverter and the curve [(0, .8), (.5, .95), (1, .95)], a request of 500 Wh AC returned 95 Wh AC from the available 100 Wh DC, although the curve gives 82.5 % at that load. The delivered AC energy now solves a = dc * efficiency(a / rated) exactly on the piecewise-linear curve (solve_ac_for_dc_energy), bounded by the request. Unlimited discharges and inverters without a curve are unchanged. Regression tests use a real Battery: low SoC and a power-limited discharge, each with 60- and 15-minute slots, battery discharge losses, the load path through process_energy, and constant efficiency. * docs: regenerate OpenAPI for the efficiency-curve fix * docs: regenerate OpenAPI after merging the efficiency-curve fix * docs: regenerate OpenAPI after merging the efficiency-curve fix * docs: regenerate OpenAPI after merging the efficiency-curve fix * test(genetic): cover serial fallback on platforms without fork --------- Co-authored-by: Christin <christin@BigBrother.bikini.bottom.zone> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Co-authored-by: Andreas <drbacke@gmx.de> |
||
|
|
1f8742a54b |
feat(genetic): evaluate candidates in parallel worker processes (#1382)
* feat(devices): add load-dependent DC-to-AC efficiency curve for GENETIC inverters Real inverters are much less efficient at low load than at rated power (e.g. 0.80 at 2 % vs. 0.93 at 7 % load). A constant dc_to_ac_efficiency overrates night-time battery discharge into a small base load and, with it, arbitrage and grid charging. Add two optional inverter settings for the GENETIC optimizer: - dc_to_ac_efficiency_curve: list of (load_fraction, efficiency) points, load fraction = AC power / max_power_w. Points must be strictly increasing in [0, 1] with efficiencies in (0, 1]; at least two points. Linear interpolation, clamped at both ends. When set, the curve replaces dc_to_ac_efficiency. - dc_to_ac_efficiency_reference_load_fraction (default 0.06): load at which the curve values stored energy without a specific conversion. Battery discharges in Inverter.process_energy() use the efficiency at the AC energy of each conversion. The end-of-horizon value, the terminal and tail value curves and the AC charge break-even penalty use Inverter.reference_dc_to_ac_efficiency. Without a curve, dc_to_ac_efficiency_at() and the reference efficiency return dc_to_ac_efficiency itself, so results are bit-identical to the previous model. GENETIC0 is unchanged. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * feat(genetic): make the fitness cache size configurable The per-run fitness cache of the GENETIC optimizer is unbounded. It grows with individuals x generations (~110k keys at 300 x 400, about 1.1 kB each, 1.6 kB with an EV) and adds 120-170 MB to the peak memory of a run, while only 6-15 % of the lookups hit. On 512 MB boards this is the largest part of the optimizer's memory. Add optimization.genetic.fitness_cache_max_entries: - None (default): unbounded, the previous behaviour. - 0: no cache; every genome is evaluated. - N > 0: at most N keys. The cache is then an OrderedDict and evicts the oldest key first (FIFO, O(1)); LRU would cost bookkeeping on every hit for a cache that rarely hits. A cache hit returns exactly what the deterministic evaluation would return, so the plan for a fixed seed is identical with any setting. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * perf: import heavy optional dependencies on first use A GENETIC server with external (import) providers and EOSdash disabled loads several large packages at start that it never uses, which costs a noticeable part of the memory on 512 MB boards: - matplotlib via the GENETIC/GENETIC0 PDF report modules (server/eos.py) - statsmodels via the Holt-Winters fallback in prediction/priceabc.py - pvlib via the weather base class and the pvlib-based providers - bs4/lxml via the ClearOutside provider - scipy via the self-consumption interpolator singletons, which were loaded at import time - the GENETIC0 optimizer, imported by core/ems.py - tzfpy, whose polygon data the computed general.timezone loads Import these where they are used. The interpolator tables are loaded on first use (thread-safe). Nothing changes in what is computed. general.timezone is computed from latitude/longitude and cannot be set. Add general.timezone_override (IANA name, validated with pendulum): when set, general.timezone returns it and the lookup is skipped. When unset (default) the lookup is unchanged. Tests that patched the moved names now patch them at their source. A new test starts the server module in a subprocess and checks that none of these modules is loaded. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * perf(prediction): evaluate self-consumption tables without SciPy The GENETIC inverter (and the GENETIC0 one) evaluates a pickled scipy.interpolate.RegularGridInterpolator for the self-consumption probabilities. SciPy is used nowhere else in EOS, yet loading it costs about 35 MB RSS (21 MB anonymous) on Linux arm64, which every optimization run pays. Add utils/gridinterpolator.py: - LinearGridInterpolator: 2-D linear interpolation with the same steps as SciPy's linear 2-D path (interval search, normalised distances, corner weights summed in the same order, fill value outside the grid, NaN for NaN input). - load_grid_interpolator(): reads the shipped pickles with a restricted unpickler that maps the SciPy class to a stand-in and accepts only the NumPy globals needed for the arrays. SciPy is not imported, and the data files no longer depend on SciPy's private module paths. Results are bit-identical to SciPy on Linux (aarch64, checked on 3.6M points per table including the query pattern of the inverter). The macOS arm64 SciPy wheel fuses multiply-add, so there SciPy itself differs by <= 1.1e-16; GENETIC plans for fixed seeds are unchanged on both platforms. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * feat(prediction): make the SciPy-free table evaluation opt-in Add optimization.self_consumption_interpolator (scipy | numpy, default scipy). The default keeps evaluating the self-consumption tables with SciPy's RegularGridInterpolator, built from the same grid, values and fill value the pickle holds, so results are unchanged. With numpy the tables are evaluated by LinearGridInterpolator and SciPy is never imported. The backend is read on every evaluation, so a config change applies to the next run. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * feat(genetic): evaluate candidates in parallel worker processes A GENETIC run evaluated every candidate in the server process, so a run used one CPU core however many the machine had. On small ARM boards (4x Cortex-A55, thermally limited) a 300x200 run took 12-15 minutes. Candidates are now evaluated in up to two worker processes, forked once per run after the run is fully prepared. Each worker owns a copy of the prepared optimizer; only genomes and results cross the process boundary. The fitness cache stays in the server process: hits are answered there, and identical genomes in a batch are evaluated once, as a serial run would hit the cache for the repeats. Results are applied only after the whole batch returned. If a batch fails or times out, the run continues serially. For a fixed seed the result is identical to a serial run. optimization.genetic.workers sets the number of processes. Unset (default) is automatic: one CPU core always stays free and at most two are used (1-2 cores -> 1, 3 or more -> 2), capped by a cgroup v2 CPU limit of the process or any parent (container limit, systemd CPUQuota). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EYWR1HVM8a8src23xuyL3T * fix(genetic): freeze the optimization configuration for the duration of a run The genetic optimization reads horizon_hours, tail_horizon_hours, interval_sec and prediction.hours from the live configuration on every access (control_slots, tail_slots, ... are properties). A client that updates the configuration while a run is in progress - DVhub shrinks or grows the control horizon as the known price horizon moves during the night - changed these values mid-run. Genomes and device arrays sized at the start then no longer matched the simulation loop and the final evaluation failed: IndexError: index 84 is out of bounds for axis 0 with size 84 Long runs on slow hardware hit this almost every night. optimize_ems() now takes a deep copy of the genetic configuration and the prediction hours at the start of a run, all reads in genetic.py go through it, and it is released when the run ends (also on errors). New values apply to the next run. Tests: tests/test_genetic_config_frozen_per_run.py reproduces the error without the fix (horizon 20 -> 24 mid-run: index 20 out of bounds) and passes with it; genetic/optimization tests 439 passed. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * feat(genetic): pin evaluation workers, keep the server responsive Workers are pinned to the last `workers` CPU cores (new setting optimization.genetic.pin_workers, default on); the server process stays unpinned. The run never spreads beyond `workers` cores without a CPU quota, and the server and the rest of the system keep the other cores. No pinning without a spare core. With a CPU limit (container cpu.max, systemd CPUQuota) the automatic worker count now keeps a quarter core of it for the server process (limit 2 -> 1 worker, 2.5 -> 2): two workers used a 2-core limit fully. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EYWR1HVM8a8src23xuyL3T * fix(devices): evaluate the efficiency curve at the delivered AC load When the battery limits a discharge (low state of charge or its power limit), the conversion ran with the efficiency of the requested AC energy. With a 1000 Wh battery at 10 % SoC, a 1000 W inverter and the curve [(0, .8), (.5, .95), (1, .95)], a request of 500 Wh AC returned 95 Wh AC from the available 100 Wh DC, although the curve gives 82.5 % at that load. The delivered AC energy now solves a = dc * efficiency(a / rated) exactly on the piecewise-linear curve (solve_ac_for_dc_energy), bounded by the request. Unlimited discharges and inverters without a curve are unchanged. Regression tests use a real Battery: low SoC and a power-limited discharge, each with 60- and 15-minute slots, battery discharge losses, the load path through process_energy, and constant efficiency. * docs: regenerate OpenAPI for the efficiency-curve fix * docs: regenerate OpenAPI after merging the efficiency-curve fix * docs: regenerate OpenAPI after merging the efficiency-curve fix * test(genetic): cover serial fallback on platforms without fork --------- Co-authored-by: Christin <christin@BigBrother.bikini.bottom.zone> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Co-authored-by: Andreas <drbacke@gmx.de> |
||
|
|
7f26ea3f3f |
feat(devices): add load-dependent DC-to-AC efficiency curve for GENETIC inverters (#1375)
* feat(devices): add load-dependent DC-to-AC efficiency curve for GENETIC inverters Real inverters are much less efficient at low load than at rated power (e.g. 0.80 at 2 % vs. 0.93 at 7 % load). A constant dc_to_ac_efficiency overrates night-time battery discharge into a small base load and, with it, arbitrage and grid charging. Add two optional inverter settings for the GENETIC optimizer: - dc_to_ac_efficiency_curve: list of (load_fraction, efficiency) points, load fraction = AC power / max_power_w. Points must be strictly increasing in [0, 1] with efficiencies in (0, 1]; at least two points. Linear interpolation, clamped at both ends. When set, the curve replaces dc_to_ac_efficiency. - dc_to_ac_efficiency_reference_load_fraction (default 0.06): load at which the curve values stored energy without a specific conversion. Battery discharges in Inverter.process_energy() use the efficiency at the AC energy of each conversion. The end-of-horizon value, the terminal and tail value curves and the AC charge break-even penalty use Inverter.reference_dc_to_ac_efficiency. Without a curve, dc_to_ac_efficiency_at() and the reference efficiency return dc_to_ac_efficiency itself, so results are bit-identical to the previous model. GENETIC0 is unchanged. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(devices): evaluate the efficiency curve at the delivered AC load When the battery limits a discharge (low state of charge or its power limit), the conversion ran with the efficiency of the requested AC energy. With a 1000 Wh battery at 10 % SoC, a 1000 W inverter and the curve [(0, .8), (.5, .95), (1, .95)], a request of 500 Wh AC returned 95 Wh AC from the available 100 Wh DC, although the curve gives 82.5 % at that load. The delivered AC energy now solves a = dc * efficiency(a / rated) exactly on the piecewise-linear curve (solve_ac_for_dc_energy), bounded by the request. Unlimited discharges and inverters without a curve are unchanged. Regression tests use a real Battery: low SoC and a power-limited discharge, each with 60- and 15-minute slots, battery discharge losses, the load path through process_energy, and constant efficiency. * docs: regenerate OpenAPI for the efficiency-curve fix --------- Co-authored-by: Christin <christin@BigBrother.bikini.bottom.zone> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Co-authored-by: Andreas <drbacke@gmx.de> |
||
|
|
560370281f |
perf(prediction): optional SciPy-free evaluation of the self-consumption tables (#1378)
* perf(prediction): evaluate self-consumption tables without SciPy The GENETIC inverter (and the GENETIC0 one) evaluates a pickled scipy.interpolate.RegularGridInterpolator for the self-consumption probabilities. SciPy is used nowhere else in EOS, yet loading it costs about 35 MB RSS (21 MB anonymous) on Linux arm64, which every optimization run pays. Add utils/gridinterpolator.py: - LinearGridInterpolator: 2-D linear interpolation with the same steps as SciPy's linear 2-D path (interval search, normalised distances, corner weights summed in the same order, fill value outside the grid, NaN for NaN input). - load_grid_interpolator(): reads the shipped pickles with a restricted unpickler that maps the SciPy class to a stand-in and accepts only the NumPy globals needed for the arrays. SciPy is not imported, and the data files no longer depend on SciPy's private module paths. Results are bit-identical to SciPy on Linux (aarch64, checked on 3.6M points per table including the query pattern of the inverter). The macOS arm64 SciPy wheel fuses multiply-add, so there SciPy itself differs by <= 1.1e-16; GENETIC plans for fixed seeds are unchanged on both platforms. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * feat(prediction): make the SciPy-free table evaluation opt-in Add optimization.self_consumption_interpolator (scipy | numpy, default scipy). The default keeps evaluating the self-consumption tables with SciPy's RegularGridInterpolator, built from the same grid, values and fill value the pickle holds, so results are unchanged. With numpy the tables are evaluated by LinearGridInterpolator and SciPy is never imported. The backend is read on every evaluation, so a config change applies to the next run. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Christin <christin@BigBrother.bikini.bottom.zone> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Co-authored-by: Andreas <drbacke@gmx.de> |
||
|
|
14d8587c9c |
feat(genetic): make the fitness cache size configurable (#1376)
The per-run fitness cache of the GENETIC optimizer is unbounded. It grows with individuals x generations (~110k keys at 300 x 400, about 1.1 kB each, 1.6 kB with an EV) and adds 120-170 MB to the peak memory of a run, while only 6-15 % of the lookups hit. On 512 MB boards this is the largest part of the optimizer's memory. Add optimization.genetic.fitness_cache_max_entries: - None (default): unbounded, the previous behaviour. - 0: no cache; every genome is evaluated. - N > 0: at most N keys. The cache is then an OrderedDict and evicts the oldest key first (FIFO, O(1)); LRU would cost bookkeeping on every hit for a cache that rarely hits. A cache hit returns exactly what the deterministic evaluation would return, so the plan for a fixed seed is identical with any setting. Co-authored-by: Christin <christin@BigBrother.bikini.bottom.zone> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Co-authored-by: Andreas <drbacke@gmx.de> |
||
|
|
ecec39fcd8 |
docs: add a guide for running EOS on small devices (#1379)
Collect the settings that bring EOS from about 290 MB idle / 400-450 MB during a GENETIC run down to about 140 MB, with the measurements behind them: fitness cache, timezone override and SciPy-free self-consumption tables (#1376, #1377, #1378), autosave for SD/eMMC storage, and the glibc allocator and BLAS environment. Includes run times on a Raspberry Pi 4 and an x86 VM with data of a real installation. Co-authored-by: Christin <christin@BigBrother.bikini.bottom.zone> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
3651d33915 |
perf: import heavy optional dependencies on first use (#1377)
A GENETIC server with external (import) providers and EOSdash disabled loads several large packages at start that it never uses, which costs a noticeable part of the memory on 512 MB boards: - matplotlib via the GENETIC/GENETIC0 PDF report modules (server/eos.py) - statsmodels via the Holt-Winters fallback in prediction/priceabc.py - pvlib via the weather base class and the pvlib-based providers - bs4/lxml via the ClearOutside provider - scipy via the self-consumption interpolator singletons, which were loaded at import time - the GENETIC0 optimizer, imported by core/ems.py - tzfpy, whose polygon data the computed general.timezone loads Import these where they are used. The interpolator tables are loaded on first use (thread-safe). Nothing changes in what is computed. general.timezone is computed from latitude/longitude and cannot be set. Add general.timezone_override (IANA name, validated with pendulum): when set, general.timezone returns it and the lookup is skipped. When unset (default) the lookup is unchanged. Tests that patched the moved names now patch them at their source. A new test starts the server module in a subprocess and checks that none of these modules is loaded. Co-authored-by: Christin <christin@BigBrother.bikini.bottom.zone> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
NormannandGitHub
|
e203a81666 |
fix: surface DVhub feed-in tariff update failures (#1357)
* fix: surface DVhub feed-in tariff update failures * test: annotate DVhub sample records for mypy * fix: require real feed-in tariff coverage in GENETIC0 * test: satisfy GENETIC0 tariff coverage checks * test: patch DVhub storage method on its class |
||
|
NormannandGitHub
|
4fe879ddd2 |
fix: allow EOSdash through an external reverse proxy (#1355)
Support externally proxied EOSdash (#1320) * test: verify optional dashboard port stays unpublished * docs: describe optional proxy access and public URL |
||
|
AndreasandGitHub
|
f5c6d2bc1e |
docs: explain energy planner upgrade and compatibility changes (#1331)
* docs: explain energy planner upgrade and compatibility changes * docs: add upgrade guide to documentation navigation * docs: resolve changelog upgrade link in generated site |
||
 
|
04f28997ea |
feat: deliver complete GENETIC optimization to main (#1330)
* feat: adapt configuration for multi optimization algorithms Decouple configuration from optimization algorithm parameters. Add to_[algorithm]_param() methods to the configuration that derive optimization algorithm specific parameters from the configuration. Add x-scope tags to the configuration options that describe for which specific algorithms the configuration option is for. The whole device settings are restructured. There are now general settings for the device classes with the afore mentioned to_[algorithm]_param() methods. The general device settings got their own directory `devices/settings`. By this the parameter class also does not have to be a pydantic model which can be used for future optimization/ simulations speed up. Also the parameter class for a device is now part of the device module. This better decouples and also is the natural place for parameters of a device. Besides this feature there are also fixes and improvements: * feat: extend home appliance time window settings and simulation Home appliance can now be configured for multiple runs with per-cycle allowed time windows. The number of remaining cycles to plan is determined at runtime by reading the ``cycles_completed_measurement_key`` from the measurement store. * feat: specialiced CycleTimeWindowSequence for time window sequences Sequence of time windows associated to cycles. This model specializes ``ValueTimeWindowSequence`` so that the ``value`` field of each ``ValueTimeWindow`` encodes the **cycle index** (0-based integer) the window belongs to. Typical use: an appliance that must run ``n`` times per day, each run constrained to a distinct time window. Assign ``value=0`` to windows for the first cycle, ``value=1`` for the second, and so on. Multiple windows may share the same cycle index (their allowed regions are unioned). Windows with ``value=None`` are silently ignored by all cycle-aware methods. * fix: Make test_configmigrate also regard the _ANY_SENTENIEL in key values * chore: Make devices configurations a map instead of a list This makes config paths stable regardless of declaration order and lets each device settings class build its own config path from ``self.device_id`` without needing an external index. Tests are adapted likewise. Devices configurations are automatically migrated from lists to maps. * chore: rename levelized_cost_of_storage_kwh to levelized_cost_of_storage_amt kwh This better fits in the naming scheme and also makes clear the costs are money. Signed-off-by: Bobby Noelte <b0661n0e17e@gmail.com> * fix: runtime config update ignored by config file Runtime settings were handed back to pydantic-settings as init settings, which rank below the config file and the environment. Any key already present in EOS.config.json or in the environment silently discarded the update, so a bulk PUT /v1/config returned 200 without applying anything, while the granular PUT /v1/config/{path} endpoint kept working. Add a dedicated runtime settings source ranked directly below the command line arguments and record granular updates there as well, so both endpoints share one store that survives re-evaluation of the settings sources. Environment variables keep precedence over the config file for all keys that were not set at runtime. Also repairs revert_settings() and update(), which passed their data through the same init settings. Closes #1303 * fix: env vars ignored on first config build ConfigEOS.__init__ passed self as first positional argument to _setup, which forwards it to pydantic_settings.BaseSettings.__init__. Its first positional parameter is _case_sensitive, so the environment source matched the upper case variable names against the lower case field names and returned nothing. Environment settings only took effect after the next configuration setup. * docs: changelog for config priority fixes * fix(config): preserve device identities and storage costs during migration * fix(measurement): restore JSON records into the existing singleton * fix(devices): preserve charge-rate typing and public import compatibility * feat(measurement): integrate typed energy quality and capacity APIs Port the locally backed-up measurement extensions to main async storage and PR #1256 device maps. Preserve runtime capacity estimates across #1305 bulk updates. Confirm JSON singleton restore defect on unchanged main and add regression. No production configuration or measurements included. Co-authored-by: Andreas <drbacke@gmx.de> * docs(measurement): describe household settings and consolidate regression coverage * docs(measurement): regenerate configuration and API contracts * test(measurement): isolate capacity database state between tests * ruff format fix * fix(measurement): restore JSON records into the existing singleton * test(measurement): assert restored timestamps before timezone conversion * test(measurement): assert restored timestamps before timezone conversion * fix: preserve imported feed-in revenue during parameter preparation Cancel GENETIC preparation when imported revenue cannot be read or contains invalid values, preserving the chosen provider instead of replacing it with demo tariffs. Keep valid positive, zero and negative amount/Wh series unchanged. Adapt the revenue-preservation regressions from PRs #1224 and #1304 to the async main API, including real timestamped imports and simulation repricing. The feature-only direct-marketing override remains outside this main fix. Co-authored-by: Christin <info@bikinibottom.capital> Co-authored-by: Normann <github@koldrack.com> * feat(devices): port slot-aware battery export and direct-use physics Port scoped device changes from |
||
|
|
3c862543a1 |
feat(devices): add bounded slot physics for GENETIC (#1327)
* feat: adapt configuration for multi optimization algorithms
Decouple configuration from optimization algorithm parameters. Add to_[algorithm]_param() methods
to the configuration that derive optimization algorithm specific parameters from the configuration.
Add x-scope tags to the configuration options that describe for which specific algorithms the
configuration option is for.
The whole device settings are restructured. There are now general settings for the device classes
with the afore mentioned to_[algorithm]_param() methods. The general device settings got their own
directory `devices/settings`. By this the parameter class also does not have to be a pydantic model
which can be used for future optimization/ simulations speed up.
Also the parameter class for a device is now part of the device module. This better decouples and
also is the natural place for parameters of a device.
Besides this feature there are also fixes and improvements:
* feat: extend home appliance time window settings and simulation
Home appliance can now be configured for multiple runs with per-cycle allowed time windows. The
number of remaining cycles to plan is determined at runtime by reading the
``cycles_completed_measurement_key`` from the measurement store.
* feat: specialiced CycleTimeWindowSequence for time window sequences
Sequence of time windows associated to cycles.
This model specializes ``ValueTimeWindowSequence`` so that the ``value``
field of each ``ValueTimeWindow`` encodes the **cycle index** (0-based
integer) the window belongs to.
Typical use: an appliance that must run ``n`` times per day, each run
constrained to a distinct time window. Assign ``value=0`` to windows
for the first cycle, ``value=1`` for the second, and so on. Multiple
windows may share the same cycle index (their allowed regions are unioned).
Windows with ``value=None`` are silently ignored by all cycle-aware methods.
* fix: Make test_configmigrate also regard the _ANY_SENTENIEL in key values
* chore: Make devices configurations a map instead of a list
This makes config paths stable regardless of declaration order and lets each device settings
class build its own config path from ``self.device_id`` without needing an external index.
Tests are adapted likewise.
Devices configurations are automatically migrated from lists to maps.
* chore: rename levelized_cost_of_storage_kwh to levelized_cost_of_storage_amt kwh
This better fits in the naming scheme and also makes clear the costs are money.
Signed-off-by: Bobby Noelte <b0661n0e17e@gmail.com>
* fix: runtime config update ignored by config file
Runtime settings were handed back to pydantic-settings as init settings,
which rank below the config file and the environment. Any key already
present in EOS.config.json or in the environment silently discarded the
update, so a bulk PUT /v1/config returned 200 without applying anything,
while the granular PUT /v1/config/{path} endpoint kept working.
Add a dedicated runtime settings source ranked directly below the command
line arguments and record granular updates there as well, so both
endpoints share one store that survives re-evaluation of the settings
sources. Environment variables keep precedence over the config file for
all keys that were not set at runtime.
Also repairs revert_settings() and update(), which passed their data
through the same init settings.
Closes #1303
* fix: env vars ignored on first config build
ConfigEOS.__init__ passed self as first positional argument to _setup,
which forwards it to pydantic_settings.BaseSettings.__init__. Its first
positional parameter is _case_sensitive, so the environment source
matched the upper case variable names against the lower case field names
and returned nothing. Environment settings only took effect after the
next configuration setup.
* docs: changelog for config priority fixes
* fix(config): preserve device identities and storage costs during migration
* fix(devices): preserve charge-rate typing and public import compatibility
* ruff format fix
* feat(devices): port slot-aware battery export and direct-use physics
Port scoped device changes from
|
||
|
|
1a18935667 |
feat(measurement): add typed energy, quality and capacity APIs (#1326)
* feat: adapt configuration for multi optimization algorithms Decouple configuration from optimization algorithm parameters. Add to_[algorithm]_param() methods to the configuration that derive optimization algorithm specific parameters from the configuration. Add x-scope tags to the configuration options that describe for which specific algorithms the configuration option is for. The whole device settings are restructured. There are now general settings for the device classes with the afore mentioned to_[algorithm]_param() methods. The general device settings got their own directory `devices/settings`. By this the parameter class also does not have to be a pydantic model which can be used for future optimization/ simulations speed up. Also the parameter class for a device is now part of the device module. This better decouples and also is the natural place for parameters of a device. Besides this feature there are also fixes and improvements: * feat: extend home appliance time window settings and simulation Home appliance can now be configured for multiple runs with per-cycle allowed time windows. The number of remaining cycles to plan is determined at runtime by reading the ``cycles_completed_measurement_key`` from the measurement store. * feat: specialiced CycleTimeWindowSequence for time window sequences Sequence of time windows associated to cycles. This model specializes ``ValueTimeWindowSequence`` so that the ``value`` field of each ``ValueTimeWindow`` encodes the **cycle index** (0-based integer) the window belongs to. Typical use: an appliance that must run ``n`` times per day, each run constrained to a distinct time window. Assign ``value=0`` to windows for the first cycle, ``value=1`` for the second, and so on. Multiple windows may share the same cycle index (their allowed regions are unioned). Windows with ``value=None`` are silently ignored by all cycle-aware methods. * fix: Make test_configmigrate also regard the _ANY_SENTENIEL in key values * chore: Make devices configurations a map instead of a list This makes config paths stable regardless of declaration order and lets each device settings class build its own config path from ``self.device_id`` without needing an external index. Tests are adapted likewise. Devices configurations are automatically migrated from lists to maps. * chore: rename levelized_cost_of_storage_kwh to levelized_cost_of_storage_amt kwh This better fits in the naming scheme and also makes clear the costs are money. Signed-off-by: Bobby Noelte <b0661n0e17e@gmail.com> * fix: runtime config update ignored by config file Runtime settings were handed back to pydantic-settings as init settings, which rank below the config file and the environment. Any key already present in EOS.config.json or in the environment silently discarded the update, so a bulk PUT /v1/config returned 200 without applying anything, while the granular PUT /v1/config/{path} endpoint kept working. Add a dedicated runtime settings source ranked directly below the command line arguments and record granular updates there as well, so both endpoints share one store that survives re-evaluation of the settings sources. Environment variables keep precedence over the config file for all keys that were not set at runtime. Also repairs revert_settings() and update(), which passed their data through the same init settings. Closes #1303 * fix: env vars ignored on first config build ConfigEOS.__init__ passed self as first positional argument to _setup, which forwards it to pydantic_settings.BaseSettings.__init__. Its first positional parameter is _case_sensitive, so the environment source matched the upper case variable names against the lower case field names and returned nothing. Environment settings only took effect after the next configuration setup. * docs: changelog for config priority fixes * fix(config): preserve device identities and storage costs during migration * fix(measurement): restore JSON records into the existing singleton * fix(devices): preserve charge-rate typing and public import compatibility * feat(measurement): integrate typed energy quality and capacity APIs Port the locally backed-up measurement extensions to main async storage and PR #1256 device maps. Preserve runtime capacity estimates across #1305 bulk updates. Confirm JSON singleton restore defect on unchanged main and add regression. No production configuration or measurements included. Co-authored-by: Andreas <drbacke@gmx.de> * docs(measurement): describe household settings and consolidate regression coverage * docs(measurement): regenerate configuration and API contracts * test(measurement): isolate capacity database state between tests * ruff format fix * test(measurement): assert restored timestamps before timezone conversion * docs(measurement): align API version with refreshed prerequisites * fix(config): satisfy typed device conversion and migration contracts * docs(config): refresh validated configuration prerequisite schemas * fix(measurement): enforce typed capacity and sample validation * style(measurement): normalize imports for CI * docs(measurement): refresh typed measurement API schemas --------- Signed-off-by: Bobby Noelte <b0661n0e17e@gmail.com> Co-authored-by: Bobby Noelte <b0661n0e17e@gmail.com> Co-authored-by: r0b2g1t <r0b2g1t@users.noreply.github.com> |
||
|
|
8224c64654 |
feat: integrate device and runtime configuration foundation on main (#1328)
* feat: adapt configuration for multi optimization algorithms Decouple configuration from optimization algorithm parameters. Add to_[algorithm]_param() methods to the configuration that derive optimization algorithm specific parameters from the configuration. Add x-scope tags to the configuration options that describe for which specific algorithms the configuration option is for. The whole device settings are restructured. There are now general settings for the device classes with the afore mentioned to_[algorithm]_param() methods. The general device settings got their own directory `devices/settings`. By this the parameter class also does not have to be a pydantic model which can be used for future optimization/ simulations speed up. Also the parameter class for a device is now part of the device module. This better decouples and also is the natural place for parameters of a device. Besides this feature there are also fixes and improvements: * feat: extend home appliance time window settings and simulation Home appliance can now be configured for multiple runs with per-cycle allowed time windows. The number of remaining cycles to plan is determined at runtime by reading the ``cycles_completed_measurement_key`` from the measurement store. * feat: specialiced CycleTimeWindowSequence for time window sequences Sequence of time windows associated to cycles. This model specializes ``ValueTimeWindowSequence`` so that the ``value`` field of each ``ValueTimeWindow`` encodes the **cycle index** (0-based integer) the window belongs to. Typical use: an appliance that must run ``n`` times per day, each run constrained to a distinct time window. Assign ``value=0`` to windows for the first cycle, ``value=1`` for the second, and so on. Multiple windows may share the same cycle index (their allowed regions are unioned). Windows with ``value=None`` are silently ignored by all cycle-aware methods. * fix: Make test_configmigrate also regard the _ANY_SENTENIEL in key values * chore: Make devices configurations a map instead of a list This makes config paths stable regardless of declaration order and lets each device settings class build its own config path from ``self.device_id`` without needing an external index. Tests are adapted likewise. Devices configurations are automatically migrated from lists to maps. * chore: rename levelized_cost_of_storage_kwh to levelized_cost_of_storage_amt kwh This better fits in the naming scheme and also makes clear the costs are money. Signed-off-by: Bobby Noelte <b0661n0e17e@gmail.com> * fix: runtime config update ignored by config file Runtime settings were handed back to pydantic-settings as init settings, which rank below the config file and the environment. Any key already present in EOS.config.json or in the environment silently discarded the update, so a bulk PUT /v1/config returned 200 without applying anything, while the granular PUT /v1/config/{path} endpoint kept working. Add a dedicated runtime settings source ranked directly below the command line arguments and record granular updates there as well, so both endpoints share one store that survives re-evaluation of the settings sources. Environment variables keep precedence over the config file for all keys that were not set at runtime. Also repairs revert_settings() and update(), which passed their data through the same init settings. Closes #1303 * fix: env vars ignored on first config build ConfigEOS.__init__ passed self as first positional argument to _setup, which forwards it to pydantic_settings.BaseSettings.__init__. Its first positional parameter is _case_sensitive, so the environment source matched the upper case variable names against the lower case field names and returned nothing. Environment settings only took effect after the next configuration setup. * docs: changelog for config priority fixes * fix(config): preserve device identities and storage costs during migration * fix(devices): preserve charge-rate typing and public import compatibility * ruff format fix * fix(config): satisfy typed device conversion and migration contracts * docs(config): refresh validated configuration prerequisite schemas --------- Signed-off-by: Bobby Noelte <b0661n0e17e@gmail.com> Co-authored-by: Bobby Noelte <b0661n0e17e@gmail.com> Co-authored-by: r0b2g1t <r0b2g1t@users.noreply.github.com> |
||
|
AndreasandAndreas
|
431d7d57e5 | docs(pvforecast): refresh API version after CI formatting | ||
|
AndreasandAndreas
|
2d6acb51b9 | docs(pvforecast): regenerate local backend configuration schema | ||
|
AndreasandAndreas
|
6529c39df4 |
feat(pvforecast): add calibrated local Akkudoktor backend
Port local PV modeling and outage calibration from feature commits |
||
|
1abdd345c4 |
fix: unify mypy environments for local checks and CI (#1291)
The isolated pre-commit mypy hook previously omitted runtime type information that make mypy used, hiding errors involving dependencies such as Pydantic and Pendulum. Makefile, pre-commit and CI now run the same full-project typing policy in the development environment defined by uv.lock. - Use uv run --locked --exact --extra dev and the same mypy arguments for Makefile and the local hook. Check all of src and tests, including on configuration-only changes. - Pin Python 3.13 for local development and the pre-commit CI job, and install the locked pre-commit version in CI. - Disable incremental analysis because existing Pendulum cache state changes mypy 2.3.1 diagnostics. Document the policy, the performance tradeoff and the existing typing debt. - Add a regression test that exercises Makefile, the hook and the CI command in a temporary project, accepting valid dependency types and detecting deliberate Pydantic/Pendulum assignment errors. Resolve the newly detected mypy diagnostics. - Enable the numpydantic and Pydantic mypy plugins, retaining strict Pydantic constructor typing with init_typed = true. Validate raw/coercible payloads through model_validate. - Propagate concrete record, provider and time-window types through generic collections, factories and lookup methods. Preserve runtime field inspection and generated time-window documentation. - Align Pendulum annotations with actual factory/arithmetic results while retaining Pydantic validation adapters at runtime. Correct optional values, array boundaries, REST handlers and plotting interfaces. - Add pinned scipy-stubs and types-psutil, update uv.lock, and supply the plugins' dependencies. - Add runtime regression coverage for validated path defaults, normalized time-series metadata, generic field inspection, invalid timestamps and unsupported provider imports. Runtime and compatibility details: - Validate path defaults as Path objects while retaining raw string defaults needed by migration serialization with exclude_defaults. - Normalize feed-in tariff lists and default charge rates to NumPy arrays; reject missing timestamps/uninitialized values explicitly. Importing into a provider without import support returns HTTP 400. - Public JSON schemas and OpenAPI structure match main (excluding the generated version). Signed-off-by: dr-dimitry Signed-off-by: dr-dimitry Signed-off-by: Bobby Noelte <b0661n0e17e@gmail.com> Co-authored-by: dr-dimitri <87113560+dr-dimitri@users.noreply.github.com> Co-authored-by: Normann <github@koldrack.com> |
||
  
|
7bafc8d02d |
feat: add PVForecastHomeAssistant provider (#1258)
* feat: add PVForecastHomeAssistant provider
Reads a PV forecast time series directly from a Home Assistant entity
attribute (matching the {"forecast": [{"datetime", "watts"}]} shape
already exposed by common HA PV forecast integrations, e.g. Helios
Forecast and Solcast) and feeds it into pvforecast_ac_power, following
the same self-polling provider pattern as PVForecastVrm.
Closes #1232.
* fix: use MagicMock instead of monkeypatched Response for mypy
requests.Response().json is a typed bound method; reassigning it to a
lambda fails mypy's method-assign check. Use MagicMock(spec=...) instead,
which mocks the response without fighting its static type.
* fix: address PR review on PVForecastHomeAssistant provider
Fixes two issues raised in review on PR #1258:
- _update_data() left stale pvforecast_ac_power values in place when a
refreshed forecast was empty or shorter than a previous one; it now
clears the active forecast window before writing and raises instead
of silently no-op'ing when the response has no usable data.
- pvforecast.homeassistant.entity_id used a "select" widget with no
manual-entry fallback in EOSdash, leaving it unusable in standalone
mode where the entity list can't be resolved without SUPERVISOR_TOKEN;
switched to a plain text field.
Also fills in the config docs and openapi.json for the new
pvforecast.homeassistant.* fields, which were missing from the
original commit.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* fix: preserve retained forecast history when clearing stale entries
The previous fix cleared pvforecast_ac_power from start-of-day, but
PredictionProvider deliberately retains historical records back to
keep_datetime (prediction.historic_hours). Since Home Assistant
forecasts are future-only, that clear wiped out retained history
between midnight and the EMS start on every refresh.
Narrow the clear to [ems_start_datetime, end_datetime) - the actual
active forecast window, DST-adjusted - instead of the day boundary.
Addresses review feedback from @NormannK on PR #1258.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
---------
Co-authored-by: Mathias <mathias@Mathiass-MacBook-Air.local>
Co-authored-by: Mathias <mathias@Mathiass-Air.localdomain>
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
|
||
|
Bobby NoelteandGitHub
|
ba76087db9 |
feat: electricity fee provider framework and generic providers (#1235)
Add new provider class for electricity fees providers. Add the generic providers: - ElecFeeFixed - ElecFeeImport The providers provide predictions for: - elecfee_consumption_amt_wh: Total fixed fee for consumed energy per Wh [amount/Wh]. This is the accumulation of all fixed per-Wh fees payable on "consumed energy - such as network charge, concession fee, and electricity charge - into a single amount. - elecfee_consumption_percent_amt: Total fixed surcharge on consumed energy, given as a percentage of the monetary amount already charged for that energy [%]. This is the accumulation of all percentage-based surcharges payable on top of the consumed-energy fee - such as VAT - into a single percentage. This is a percentage of the fee amount, not a per-Wh rate. - elecfee_feedin_amt_wh: Total fixed deduction from feed-in energy per Wh [amount/Wh]. This is the accumulation of all fixed per-Wh charges deducted from feed-in energy - such as metering fees or grid-operator handling "charges - into a single amount. Applied after the percentage-based deduction, i.e. it reduces the price by a flat amount per Wh rather than by a share of the raw price. - elecfee_feedin_percent_amt: Total percentage deducted from the raw feed-in price (spot price) [%]. This is the accumulation of all percentage-based deductions payable on the feed-in tariff - such as a marketing or balancing fee retained by the aggregator - into a single percentage. It is applied as `raw_price * (100 - percent) / 100`, i.e. it scales down the raw price rather than adding a surcharge to it. A new _apply_fee() method is added to the base class for ElecPrice and FeedInTariff to be used to add the fees in a consistent way. Fees are taken from the active ElecFee provider and applied to the raw prices given to the _apply_fee() method. The optional application of fees is added to: - ElecPriceAkkudoktor - ElecPriceFixed - ElecPriceEnergyCharts - ElecPriceSMARD - FeedInTariffEnergyCharts - FeedInTariffFixed - FeedInTariffSMARD The import providers ElecPriceImport and FeedInTariffImport do not apply fees by intentention. The following providers currently do not handle fees defined by ElecFee: - ElecPriceTibber - FeedInTariffAkkudoktor - FeedInTariffDvhubOnline - FeedInTariffTibber The tests for this feature are either added or existing tests are extended. The documentation was extended for the electricity fee provider settings. Besides this feature further improvements are added: * feat: add SMARD quarter-hour electricty price and feed-in tariff provider * feat: to_series method for TimeWindows and ValueTimeWindows Additional to to_array the time window sequence can now also produce a pandas series. Test have been extended to cover the series generation. * feat: use time windows in fixed feedin tariff provider Feedin tariff can now be configured by time windows - not a single value. * feat: EOSdash select for PVLib inverters and modules Provide PVLib inverter and module names in config selection. * feat: EOSdash lazy select for big option sets Add a new form for lazy selection of big option sets. Filtering and generation of the option set is done server-side. * fix: use raw data for ETS/ median prediction Use to raw time series data for ETS/ median prediction to avoid interference by e.g. dynamic grid charges. * fix: EOSdash config drops by type only on details resolve Drop configuration by type and path. Prevents dropping of configuration items with same type and level but different path. * fix: EOSdash configuration section closes on update Open section if searching or if last update touched this category — including updates on deeply nested sub-fields. * chore: make elecfeefixed, elecpricefixed and feedintarifffixed warn about no windows and default to 0 Missining configuration creates default 0 value and a warning instead of an exception. * fix: test setup for providers Reset db state on each test run. * chore: improve config option naming for elecpricefixed. * chore: adapt elecpricefixed test to changed time_windows naming * chore: factorized common price provider helpers to priceabc.py Factorized common price provider helpers to priceabc.py. Add tests for these helpers. Reduce/ change testing of elecpriceabc.py and feedintariffabc.py to cover only specifics. Rest of testing is already covered by test_priceabc.py. * chore: update version Signed-off-by: Bobby Noelte <b0661n0e17e@gmail.com> |
||
|
Bobby NoelteandGitHub
|
886c93c92b |
fix: default server settings prevent env var config (#1234)
Change configuration source priorities to: - cli - environment vars - dotenv settings - config file settings - init settings By this the environment vars supersede any configuration var provided by the configuration file or by the initialisation with pydantic. The test_config.py::test_computed_path was fixed to to not use the defaul env var overwrite defined by conftest.py. This seemed to indicate non working env vars, but in fact was a test setupt fault. Besides this fix there are other fixes and changes added: * fix: exclude computed fields when merging settings Pydantic may overwrite settings by values given for computed fields and use these values instead of re-computing the field. Avoid computed fields in merging settings. * chore: improve Windows compatability of development setup Improve scripts to better run also on Windows. When doing path checks keep compatibility also to Windows pathes. A lot of changes to avoid the famous Windows CRLF handling and keep line endings to LF. * chore: add development hint for Windows Windows developers should set core.autocrlf to false. * chore: update version Signed-off-by: b0661 <b0661n0e17e@gmail.com> |
||
|
Bobby NoelteandGitHub
|
269d2162a9 |
fix: server crash on prediction update failure (#1222)
Catch exceptions from prediction update and log as error message. Keep executing the energy management run. Signed-off-by: Bobby Noelte <b0661n0e17e@gmail.com> |
||
|
Bobby NoelteandGitHub
|
894790f577 |
feat: add pvlib pv forecast provider (#1214)
Add a PV forecast provider that calculates the forecast using a PVLib system model and weather forecast from the EOS weather forecast provider. Additional module and inverter models can be easily added as the database is build from PVLib and SAM databases and a bundled csv file. The module model and inververt model names are provided by new endpoints to be used in configuration. The provider is based on the fantastic work of EMHASS. See https://github.com/davidusb-geek/emhass/blob/master/src/emhass/forecast.py A short description of the provider is added to the documentation. Besides the new features there are the fixes and improvements: * feat: improve EOSdash config page * fix: kex_to_series for start_datetime Make key_to_series always start the series at start_datetime. * fix: default provider for GENETIC and GENETIC0 optimization To make the default less dependent on internet servers (with API changes and availability issues) the default for PVForecast is set to PVForecastPVLib and for ElecPrice to ElecPriceFixed. The default weather provider is changed to OpenMeteo. * fix: EOSdash display resampled prediction values Make EOSdash display resampled prediction values where resampling fits to the prediction value type. Use bar width that fits to 15 minutes value samples. * chore: add a UI hints system to EOSdash The UI hints system eases the definition of forms for configuration items. There are also forms for items in maps and lists. These forms allow to add and delete items to/ from maps and lists. The forms ensure that all required fields of newly added items are filled. * chore: Create an enum for valid optimization algorithms * chore. Make config also provide the available energy management modes. Used for configuration hints. * chore: Randomize default device id in configuration Signed-off-by: Bobby Noelte <b0661n0e17e@gmail.com> |