Commit Graph
1079 Commits
Author SHA1 Message Date
NormannandGitHub fca07a0c62 fix: show Home Assistant add-on version in EOSdash (#1356)
Align add-on and EOSdash version.
2026-09-26 16:57:49 +02:00
NormannandGitHub 4fe879ddd2 fix: allow EOSdash through an external reverse proxy (#1355)
Support externally proxied EOSdash (#1320)

* test: verify optional dashboard port stays unpublished

* docs: describe optional proxy access and public URL
2026-09-26 16:56:53 +02:00
NormannandGitHub 3d2daa694b fix(dash): use reachable host in footer API docs link (#1354)
link to EOS API using reachable host
2026-09-26 16:54:27 +02:00
efab8cd0a0 fix(genetic): keep fitness-cache memory within pymalloc and release arena after each run (#1353)
* fix(genetic): keep fitness-cache memory in pymalloc and release arena after each run

At fine time resolution (interval_sec=900, ~192 control slots over a multi-day
horizon) the fitness-cache keys are ~1.6 KB int tuples, above CPython's 512-byte
pymalloc threshold, so they are served by glibc malloc in the optimization worker
thread's arena and are not returned to the OS on `self._fitness_cache.clear()`.
With re-optimization every 15 min, RSS stair-steps up to the memory limit within
about a day (OOM / forced restart). At hourly resolution the tuples stay < 512 B,
so pymalloc reclaims them and the effect is negligible. See #1352.

- Store the cache key/genome compactly as bytes (1 byte per gene, 8-byte fallback
  for larger state spaces) so entries stay within pymalloc regardless of resolution.
- After each optimization run, gc.collect() + malloc_trim(0) (guarded, glibc-only)
  to return freed arena pages to the OS.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* fix(genetic): pack fitness-cache genome as bounded chunks

Storing the genome as a single bytes object still exceeds pymalloc's
512-byte threshold once the genome grows: at 15-min resolution over a
60 h horizon with EV genes the key is ~480 genes, so even the one-byte
encoding is 481 bytes (514 with the object header) and any value >255
switches the whole genome to 8 bytes per gene. Such keys land in glibc
malloc, which does not reliably return the pages (malloc_trim is
glibc-only, absent on musl) — the platform-independent guarantee did
not actually hold for supported settings.

Encode the genome (key and FitnessCacheEntry.genome) as a tuple of
bounded byte chunks instead — 256 one-byte genes or 32 signed-64-bit
genes per chunk, 256 bytes each — built per chunk so the encoder never
materialises an oversized temporary. Every object then stays inside
pymalloc regardless of horizon, on every platform. malloc_trim after a
run is kept as a secondary release for the rest of the run's heap.

Tests: round-trips incl. 480-gene narrow/wide and negative genes, and
sys.getsizeof for the key AND every chunk at 480 genes in both
encodings.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-09-26 12:39:29 +02:00
NormannandGitHub 986b4eb0d4 build(deps): combine eleven Dependabot updates and synchronize uv.lock (#1340)
* build(deps): consolidate Dependabot updates with synchronized uv lock

* build(deps): add four pending Dependabot updates and refresh uv lock

* build(deps): add platformdirs and Commitizen updates
2026-09-25 19:08:22 +02:00
Hoffis-WeltandGitHub 38915a620b fix(docker): bind EOSdash to 0.0.0.0 so HA ingress can reach it (#1346)
server.eosdash_host defaults to 127.0.0.1 (not None), so the fallback in run_eos() that inherits server.host never triggers. The --host 0.0.0.0 CLI argument only sets server.host. As a result EOSdash binds to 127.0.0.1 inside the container and the Home Assistant Supervisor ingress (ingress_port 8504) cannot reach it (502). Set EOS_SERVER__EOSDASH_HOST=0.0.0.0 in the image.
2026-09-24 19:40:03 +02:00
AndreasandGitHub 35f77c9b99 fix(optimization): let the AC setpoint cap the total charge where the inverter does (#1334)
* fix(optimization): let the AC setpoint cap the total charge where the inverter does

Some hybrid inverters limit the battery's whole charge current to the grid
charge setpoint while grid charging is on. A Deye 12K in time-of-use grid
charging with max_grid_charge_current = 75 A charges at ~80 A even with 8 kW
of PV, and exports the rest - including the surplus of micro inverters on
the grid side. GENETIC modelled an AC slot as "PV surplus first, grid adds
ac_charge x the remaining charge power", so under PV surplus an AC slot
looked at least as good as a DC slot and the optimizer picked a low AC
factor, while the real system exported what the plan meant to store.

New inverter config option devices.inverters[].ac_charge_limits_total_charge
(GENETIC scope, default False, so the existing model is unchanged). When
True, an AC slot caps the battery's raw charge at ac_charge x
max_charge_power_w from all sources: PV surplus above the cap is exported,
the grid only fills what PV leaves of it. The option reaches the optimizer
through InverterCommonSettings.to_genetic_param, i.e. POST /v1/optimize and
the EMS run. GENETIC0 (POST /optimize) is left as it is.

The AC slot logic moves into Inverter (ac_charge_factor,
begin_ac_charge_slot, charge_battery_from_grid) so the simulation and the
tail value curve share it; Battery gets a per-slot charge cap that reset()
lifts again.

* test(optimization): narrow optional simulation arrays for mypy

The annotated fixture indexes GeneticSimulation arrays typed as Optional; assert them first so the locked mypy hook passes.
2026-09-21 16:15:00 +02:00
AndreasandGitHub f5c6d2bc1e docs: explain energy planner upgrade and compatibility changes (#1331)
* docs: explain energy planner upgrade and compatibility changes

* docs: add upgrade guide to documentation navigation

* docs: resolve changelog upgrade link in generated site
2026-09-17 21:17:41 +02:00
04f28997ea feat: deliver complete GENETIC optimization to main (#1330)
* feat: adapt configuration for multi optimization algorithms

Decouple configuration from optimization algorithm parameters. Add to_[algorithm]_param() methods
to the configuration that derive optimization algorithm specific parameters from the configuration.
Add x-scope tags to the configuration options that describe for which specific algorithms the
configuration option is for.

The whole device settings are restructured. There are now general settings for the device classes
with the afore mentioned to_[algorithm]_param() methods. The general device settings got their own
directory `devices/settings`. By this the parameter class also does not have to be a pydantic model
which can be used for future optimization/ simulations speed up.

Also the parameter class for a device is now part of the device module. This better decouples and
also is the natural place for parameters of a device.

Besides this feature there are also fixes and improvements:

* feat: extend home appliance time window settings and simulation

  Home appliance can now be configured for multiple runs with per-cycle allowed time windows. The
  number of remaining cycles to plan is determined at runtime by reading the
  ``cycles_completed_measurement_key`` from the measurement store.

* feat: specialiced CycleTimeWindowSequence for time window sequences

  Sequence of time windows associated to cycles.

  This model specializes ``ValueTimeWindowSequence`` so that the ``value``
  field of each ``ValueTimeWindow`` encodes the **cycle index** (0-based
  integer) the window belongs to.

  Typical use: an appliance that must run ``n`` times per day, each run
  constrained to a distinct time window.  Assign ``value=0`` to windows
  for the first cycle, ``value=1`` for the second, and so on.  Multiple
  windows may share the same cycle index (their allowed regions are unioned).
  Windows with ``value=None`` are silently ignored by all cycle-aware methods.

* fix: Make test_configmigrate also regard the _ANY_SENTENIEL in key values

* chore: Make devices configurations a map instead of a list

  This makes config paths stable regardless of declaration order and lets each device settings
  class build its own config path from ``self.device_id`` without needing an external index.
  Tests are adapted likewise.

  Devices configurations are automatically migrated from lists to maps.

* chore: rename levelized_cost_of_storage_kwh to levelized_cost_of_storage_amt kwh

  This better fits in the naming scheme and also makes clear the costs are money.

Signed-off-by: Bobby Noelte <b0661n0e17e@gmail.com>

* fix: runtime config update ignored by config file

Runtime settings were handed back to pydantic-settings as init settings,
which rank below the config file and the environment. Any key already
present in EOS.config.json or in the environment silently discarded the
update, so a bulk PUT /v1/config returned 200 without applying anything,
while the granular PUT /v1/config/{path} endpoint kept working.

Add a dedicated runtime settings source ranked directly below the command
line arguments and record granular updates there as well, so both
endpoints share one store that survives re-evaluation of the settings
sources. Environment variables keep precedence over the config file for
all keys that were not set at runtime.

Also repairs revert_settings() and update(), which passed their data
through the same init settings.

Closes #1303

* fix: env vars ignored on first config build

ConfigEOS.__init__ passed self as first positional argument to _setup,
which forwards it to pydantic_settings.BaseSettings.__init__. Its first
positional parameter is _case_sensitive, so the environment source
matched the upper case variable names against the lower case field names
and returned nothing. Environment settings only took effect after the
next configuration setup.

* docs: changelog for config priority fixes

* fix(config): preserve device identities and storage costs during migration

* fix(measurement): restore JSON records into the existing singleton

* fix(devices): preserve charge-rate typing and public import compatibility

* feat(measurement): integrate typed energy quality and capacity APIs

Port the locally backed-up measurement extensions to main async storage and PR #1256 device maps. Preserve runtime capacity estimates across #1305 bulk updates. Confirm JSON singleton restore defect on unchanged main and add regression. No production configuration or measurements included.

Co-authored-by: Andreas <drbacke@gmx.de>

* docs(measurement): describe household settings and consolidate regression coverage

* docs(measurement): regenerate configuration and API contracts

* test(measurement): isolate capacity database state between tests

* ruff format fix

* fix(measurement): restore JSON records into the existing singleton

* test(measurement): assert restored timestamps before timezone conversion

* test(measurement): assert restored timestamps before timezone conversion

* fix: preserve imported feed-in revenue during parameter preparation

Cancel GENETIC preparation when imported revenue cannot be read or contains invalid values, preserving the chosen provider instead of replacing it with demo tariffs. Keep valid positive, zero and negative amount/Wh series unchanged.

Adapt the revenue-preservation regressions from PRs #1224 and #1304 to the async main API, including real timestamped imports and simulation repricing. The feature-only direct-marketing override remains outside this main fix.

Co-authored-by: Christin <info@bikinibottom.capital>

Co-authored-by: Normann <github@koldrack.com>

* feat(devices): port slot-aware battery export and direct-use physics

Port scoped device changes from d2e2d58237. Keep PR #1256 parameter conversion structure and separate GENETIC0 devices. Validate physical flows and reprice changed simulation results.

Co-authored-by: Andreas <drbacke@gmx.de>

Co-authored-by: Christin <info@bikinibottom.capital>

* docs(measurement): align API version with refreshed prerequisites

* fix: return only completed optimization results per run

* feat(pvforecast): add calibrated local Akkudoktor backend

Port local PV modeling and outage calibration from feature commits f976335, 6dc58c3 and faed0fd by Andreas. Keep PVForecastAkkudoktor identity and remote default, adapt to async storage, and migrate legacy provider settings.

* fix(cache): distinguish callables in the shared EMS cache

Include the function object in cache keys so methods of one interpolator cannot reuse a probability as a power value. Cover both call orders, keyword arguments, cache hits and separate closures with identical qualified names.

* fix(devices): constrain the physics port and validate export levels

Defer inactive EV deadline fields to the optimizer port, reject nonfinite export rates, and document the hourly Optimize boundary. Verify converter IDs, rates and LCOS, separate GENETIC0 interpolation, physical boundary flows and independent GENETIC repricing.

* docs(pvforecast): regenerate local backend configuration schema

* docs(devices): regenerate slot-physics configuration and OpenAPI schemas

* test: type dynamic Optimize regression arguments

* style: wrap imported tariff test parameter import

* style(pvforecast): apply CI import formatting

* docs(pvforecast): refresh API version after CI formatting

* fix(config): satisfy typed device conversion and migration contracts

* docs(config): refresh validated configuration prerequisite schemas

* fix(measurement): enforce typed capacity and sample validation

* test(devices): align physics regressions with strict type checking

* style(measurement): normalize imports for CI

* docs(measurement): refresh typed measurement API schemas

* docs(devices): refresh API version after prerequisite merge

* test: make optimization dispatch timezones explicit

* docs(interpolator): use portable reStructuredText markup

* docs(devices): refresh API version after docstring compatibility fix

* feat: complete configuration-driven GENETIC optimization and reports (#1329)

* feat(devices): port slot-aware battery export and direct-use physics

Port scoped device changes from d2e2d58237. Keep PR #1256 parameter conversion structure and separate GENETIC0 devices. Validate physical flows and reprice changed simulation results.

Co-authored-by: Andreas <drbacke@gmx.de>

Co-authored-by: Christin <info@bikinibottom.capital>

* feat(optimization): port tested terminal and tail value primitives

Source d2e2d58237. 22 primitive tests pass; integration with the optimizer, forecast horizon and API is still pending.

Co-authored-by: Andreas <drbacke@gmx.de>

Co-authored-by: Christin <info@bikinibottom.capital>

* fix(devices): preserve charge-rate typing and public import compatibility

* feat(measurement): integrate typed energy quality and capacity APIs

Port the locally backed-up measurement extensions to main async storage and PR #1256 device maps. Preserve runtime capacity estimates across #1305 bulk updates. Confirm JSON singleton restore defect on unchanged main and add regression. No production configuration or measurements included.

Co-authored-by: Andreas <drbacke@gmx.de>

* test(integration): validate optimizer economics and document measurement settings

* docs(integration): record tested checkpoint and remaining consolidation work

* docs(development): define isolated PR packages and remaining porting gates

* docs(integration): refresh API version after measurement reconciliation

* docs(integration): record PR readiness verification results

* docs(development): record publication and verification of PR 1322

* test(measurement): assert restored timestamps before timezone conversion

* docs(development): record corrected PR head and CI progress

* docs(integration): refresh API version after prerequisite alignment

* docs(integration): define parallel packages and Optimize compatibility gates

* fix: preserve imported feed-in revenue during parameter preparation

Cancel GENETIC preparation when imported revenue cannot be read or contains invalid values, preserving the chosen provider instead of replacing it with demo tariffs. Keep valid positive, zero and negative amount/Wh series unchanged.

Adapt the revenue-preservation regressions from PRs #1224 and #1304 to the async main API, including real timestamped imports and simulation repricing. The feature-only direct-marketing override remains outside this main fix.

Co-authored-by: Christin <info@bikinibottom.capital>

Co-authored-by: Normann <github@koldrack.com>

* test(integration): verify tariff protection with mapped device physics

* fix: return only completed optimization results per run

* test(integration): verify algorithm aliases and mapped-device contracts

* fix(cache): distinguish callables in the shared EMS cache

Include the function object in cache keys so methods of one interpolator cannot reuse a probability as a power value. Cover both call orders, keyword arguments, cache hits and separate closures with identical qualified names.

* fix(devices): constrain the physics port and validate export levels

Defer inactive EV deadline fields to the optimizer port, reject nonfinite export rates, and document the hourly Optimize boundary. Verify converter IDs, rates and LCOS, separate GENETIC0 interpolation, physical boundary flows and independent GENETIC repricing.

* feat(pvforecast): add calibrated local Akkudoktor backend

Port local PV modeling and outage calibration from feature commits f976335, 6dc58c3 and faed0fd by Andreas. Keep PVForecastAkkudoktor identity and remote default, adapt to async storage, and migrate legacy provider settings.

* docs(integration): record combined compatibility checks and green JSON PR CI

* test: type dynamic Optimize regression arguments

* docs(integration): record Optimize fix PR publication

* docs(integration): record imported tariff protection PR

* style(pvforecast): apply CI import formatting

* style(integration): align combined regression imports

* test: make optimization dispatch timezones explicit

* docs(interpolator): use portable reStructuredText markup

* chore: validate combined integration with locked mypy

* docs: hand off six validated pull requests for manual review

* feat: report genetic interval and terminal value diagnostics

* feat(devices): reconcile flexible profiles and EV deadlines with cycle scheduling

Adapt the flexible consumer primitives from d2e2d582 while retaining the keyed settings and per-cycle scheduling introduced by #1256. Preserve slot battery physics and GENETIC0 flat-load conversion. Cover energy conservation, deadlines, window intersections, DST, completed cycles and EV converters.

* test: satisfy typed genetic PDF chart contracts

* feat(optimization): resolve quarter-hour GENETIC requests from configuration

* test(genetic): verify real device scheduling, measurement and export contracts

Register appliance completed-cycle measurement keys so the real store accepts both default and custom counters. Exercise complete low-budget optimizer runs, persisted measurements, generic solution output and instructions, including zero-power phases, EV departure boundaries, per-cycle windows and LCOS.

* fix: bound genetic report forecasts to executable horizon

* feat: complete native genetic scheduling and retained result contracts

* fix: retain missing raw samples when dropna is disabled

* fix: align local optimization slots and measurement instants

* docs: explain complete genetic rollout and PR dependencies

* feat: expose retained GENETIC report through the versioned API

* docs: regenerate complete genetic configuration and API schema

* docs: format consolidation and review handoff markdown

* test: align isolated EMS fixture with native genetic run options

* test(genetic): clean up singleton measurements after device integration tests

* test: freeze the clock without replacing timestamp conversion

* fix: preserve explicit warmstart timezones in runtime requests

* test(genetic): validate device schedules in UTC and Berlin

Use explicit Berlin origins for Berlin wall-clock windows, compare absolute deadline instants correctly, and run all real device optimizer scenarios under both UTC and Europe/Berlin. Compare exported starts in the run timezone instead of assuming the output timezone matches the host.

* fix: start automatic genetic runs in the site timezone

* Preserve aware GENETIC snapshot times across host timezones

* docs: specify site clock and rehearsed merge resolutions

* test: isolate invalid measurement records and refresh API version

* fix: render single-slot genetic tail diagnostics

* docs: refresh schema version after report fix

* fix: preserve configuration-only Optimize API contract

* docs: refresh configuration request schema

---------

Co-authored-by: Christin <info@bikinibottom.capital>
Co-authored-by: Normann <github@koldrack.com>

---------

Signed-off-by: Bobby Noelte <b0661n0e17e@gmail.com>
Co-authored-by: Bobby Noelte <b0661n0e17e@gmail.com>
Co-authored-by: r0b2g1t <r0b2g1t@users.noreply.github.com>
Co-authored-by: Normann <github@koldrack.com>
Co-authored-by: Christin <info@bikinibottom.capital>
2026-09-17 20:14:24 +02:00
3c862543a1 feat(devices): add bounded slot physics for GENETIC (#1327)
* feat: adapt configuration for multi optimization algorithms

Decouple configuration from optimization algorithm parameters. Add to_[algorithm]_param() methods
to the configuration that derive optimization algorithm specific parameters from the configuration.
Add x-scope tags to the configuration options that describe for which specific algorithms the
configuration option is for.

The whole device settings are restructured. There are now general settings for the device classes
with the afore mentioned to_[algorithm]_param() methods. The general device settings got their own
directory `devices/settings`. By this the parameter class also does not have to be a pydantic model
which can be used for future optimization/ simulations speed up.

Also the parameter class for a device is now part of the device module. This better decouples and
also is the natural place for parameters of a device.

Besides this feature there are also fixes and improvements:

* feat: extend home appliance time window settings and simulation

  Home appliance can now be configured for multiple runs with per-cycle allowed time windows. The
  number of remaining cycles to plan is determined at runtime by reading the
  ``cycles_completed_measurement_key`` from the measurement store.

* feat: specialiced CycleTimeWindowSequence for time window sequences

  Sequence of time windows associated to cycles.

  This model specializes ``ValueTimeWindowSequence`` so that the ``value``
  field of each ``ValueTimeWindow`` encodes the **cycle index** (0-based
  integer) the window belongs to.

  Typical use: an appliance that must run ``n`` times per day, each run
  constrained to a distinct time window.  Assign ``value=0`` to windows
  for the first cycle, ``value=1`` for the second, and so on.  Multiple
  windows may share the same cycle index (their allowed regions are unioned).
  Windows with ``value=None`` are silently ignored by all cycle-aware methods.

* fix: Make test_configmigrate also regard the _ANY_SENTENIEL in key values

* chore: Make devices configurations a map instead of a list

  This makes config paths stable regardless of declaration order and lets each device settings
  class build its own config path from ``self.device_id`` without needing an external index.
  Tests are adapted likewise.

  Devices configurations are automatically migrated from lists to maps.

* chore: rename levelized_cost_of_storage_kwh to levelized_cost_of_storage_amt kwh

  This better fits in the naming scheme and also makes clear the costs are money.

Signed-off-by: Bobby Noelte <b0661n0e17e@gmail.com>

* fix: runtime config update ignored by config file

Runtime settings were handed back to pydantic-settings as init settings,
which rank below the config file and the environment. Any key already
present in EOS.config.json or in the environment silently discarded the
update, so a bulk PUT /v1/config returned 200 without applying anything,
while the granular PUT /v1/config/{path} endpoint kept working.

Add a dedicated runtime settings source ranked directly below the command
line arguments and record granular updates there as well, so both
endpoints share one store that survives re-evaluation of the settings
sources. Environment variables keep precedence over the config file for
all keys that were not set at runtime.

Also repairs revert_settings() and update(), which passed their data
through the same init settings.

Closes #1303

* fix: env vars ignored on first config build

ConfigEOS.__init__ passed self as first positional argument to _setup,
which forwards it to pydantic_settings.BaseSettings.__init__. Its first
positional parameter is _case_sensitive, so the environment source
matched the upper case variable names against the lower case field names
and returned nothing. Environment settings only took effect after the
next configuration setup.

* docs: changelog for config priority fixes

* fix(config): preserve device identities and storage costs during migration

* fix(devices): preserve charge-rate typing and public import compatibility

* ruff format fix

* feat(devices): port slot-aware battery export and direct-use physics

Port scoped device changes from d2e2d58237. Keep PR #1256 parameter conversion structure and separate GENETIC0 devices. Validate physical flows and reprice changed simulation results.

Co-authored-by: Andreas <drbacke@gmx.de>

Co-authored-by: Christin <info@bikinibottom.capital>

* fix(cache): distinguish callables in the shared EMS cache

Include the function object in cache keys so methods of one interpolator cannot reuse a probability as a power value. Cover both call orders, keyword arguments, cache hits and separate closures with identical qualified names.

* fix(devices): constrain the physics port and validate export levels

Defer inactive EV deadline fields to the optimizer port, reject nonfinite export rates, and document the hourly Optimize boundary. Verify converter IDs, rates and LCOS, separate GENETIC0 interpolation, physical boundary flows and independent GENETIC repricing.

* docs(devices): regenerate slot-physics configuration and OpenAPI schemas

* fix(config): satisfy typed device conversion and migration contracts

* docs(config): refresh validated configuration prerequisite schemas

* test(devices): align physics regressions with strict type checking

* docs(devices): refresh API version after prerequisite merge

* docs(interpolator): use portable reStructuredText markup

* docs(devices): refresh API version after docstring compatibility fix

---------

Signed-off-by: Bobby Noelte <b0661n0e17e@gmail.com>
Co-authored-by: Bobby Noelte <b0661n0e17e@gmail.com>
Co-authored-by: r0b2g1t <r0b2g1t@users.noreply.github.com>
Co-authored-by: Christin <info@bikinibottom.capital>
2026-09-17 19:14:01 +02:00
1a18935667 feat(measurement): add typed energy, quality and capacity APIs (#1326)
* feat: adapt configuration for multi optimization algorithms

Decouple configuration from optimization algorithm parameters. Add to_[algorithm]_param() methods
to the configuration that derive optimization algorithm specific parameters from the configuration.
Add x-scope tags to the configuration options that describe for which specific algorithms the
configuration option is for.

The whole device settings are restructured. There are now general settings for the device classes
with the afore mentioned to_[algorithm]_param() methods. The general device settings got their own
directory `devices/settings`. By this the parameter class also does not have to be a pydantic model
which can be used for future optimization/ simulations speed up.

Also the parameter class for a device is now part of the device module. This better decouples and
also is the natural place for parameters of a device.

Besides this feature there are also fixes and improvements:

* feat: extend home appliance time window settings and simulation

  Home appliance can now be configured for multiple runs with per-cycle allowed time windows. The
  number of remaining cycles to plan is determined at runtime by reading the
  ``cycles_completed_measurement_key`` from the measurement store.

* feat: specialiced CycleTimeWindowSequence for time window sequences

  Sequence of time windows associated to cycles.

  This model specializes ``ValueTimeWindowSequence`` so that the ``value``
  field of each ``ValueTimeWindow`` encodes the **cycle index** (0-based
  integer) the window belongs to.

  Typical use: an appliance that must run ``n`` times per day, each run
  constrained to a distinct time window.  Assign ``value=0`` to windows
  for the first cycle, ``value=1`` for the second, and so on.  Multiple
  windows may share the same cycle index (their allowed regions are unioned).
  Windows with ``value=None`` are silently ignored by all cycle-aware methods.

* fix: Make test_configmigrate also regard the _ANY_SENTENIEL in key values

* chore: Make devices configurations a map instead of a list

  This makes config paths stable regardless of declaration order and lets each device settings
  class build its own config path from ``self.device_id`` without needing an external index.
  Tests are adapted likewise.

  Devices configurations are automatically migrated from lists to maps.

* chore: rename levelized_cost_of_storage_kwh to levelized_cost_of_storage_amt kwh

  This better fits in the naming scheme and also makes clear the costs are money.

Signed-off-by: Bobby Noelte <b0661n0e17e@gmail.com>

* fix: runtime config update ignored by config file

Runtime settings were handed back to pydantic-settings as init settings,
which rank below the config file and the environment. Any key already
present in EOS.config.json or in the environment silently discarded the
update, so a bulk PUT /v1/config returned 200 without applying anything,
while the granular PUT /v1/config/{path} endpoint kept working.

Add a dedicated runtime settings source ranked directly below the command
line arguments and record granular updates there as well, so both
endpoints share one store that survives re-evaluation of the settings
sources. Environment variables keep precedence over the config file for
all keys that were not set at runtime.

Also repairs revert_settings() and update(), which passed their data
through the same init settings.

Closes #1303

* fix: env vars ignored on first config build

ConfigEOS.__init__ passed self as first positional argument to _setup,
which forwards it to pydantic_settings.BaseSettings.__init__. Its first
positional parameter is _case_sensitive, so the environment source
matched the upper case variable names against the lower case field names
and returned nothing. Environment settings only took effect after the
next configuration setup.

* docs: changelog for config priority fixes

* fix(config): preserve device identities and storage costs during migration

* fix(measurement): restore JSON records into the existing singleton

* fix(devices): preserve charge-rate typing and public import compatibility

* feat(measurement): integrate typed energy quality and capacity APIs

Port the locally backed-up measurement extensions to main async storage and PR #1256 device maps. Preserve runtime capacity estimates across #1305 bulk updates. Confirm JSON singleton restore defect on unchanged main and add regression. No production configuration or measurements included.

Co-authored-by: Andreas <drbacke@gmx.de>

* docs(measurement): describe household settings and consolidate regression coverage

* docs(measurement): regenerate configuration and API contracts

* test(measurement): isolate capacity database state between tests

* ruff format fix

* test(measurement): assert restored timestamps before timezone conversion

* docs(measurement): align API version with refreshed prerequisites

* fix(config): satisfy typed device conversion and migration contracts

* docs(config): refresh validated configuration prerequisite schemas

* fix(measurement): enforce typed capacity and sample validation

* style(measurement): normalize imports for CI

* docs(measurement): refresh typed measurement API schemas

---------

Signed-off-by: Bobby Noelte <b0661n0e17e@gmail.com>
Co-authored-by: Bobby Noelte <b0661n0e17e@gmail.com>
Co-authored-by: r0b2g1t <r0b2g1t@users.noreply.github.com>
2026-09-17 18:28:43 +02:00
8224c64654 feat: integrate device and runtime configuration foundation on main (#1328)
* feat: adapt configuration for multi optimization algorithms

Decouple configuration from optimization algorithm parameters. Add to_[algorithm]_param() methods
to the configuration that derive optimization algorithm specific parameters from the configuration.
Add x-scope tags to the configuration options that describe for which specific algorithms the
configuration option is for.

The whole device settings are restructured. There are now general settings for the device classes
with the afore mentioned to_[algorithm]_param() methods. The general device settings got their own
directory `devices/settings`. By this the parameter class also does not have to be a pydantic model
which can be used for future optimization/ simulations speed up.

Also the parameter class for a device is now part of the device module. This better decouples and
also is the natural place for parameters of a device.

Besides this feature there are also fixes and improvements:

* feat: extend home appliance time window settings and simulation

  Home appliance can now be configured for multiple runs with per-cycle allowed time windows. The
  number of remaining cycles to plan is determined at runtime by reading the
  ``cycles_completed_measurement_key`` from the measurement store.

* feat: specialiced CycleTimeWindowSequence for time window sequences

  Sequence of time windows associated to cycles.

  This model specializes ``ValueTimeWindowSequence`` so that the ``value``
  field of each ``ValueTimeWindow`` encodes the **cycle index** (0-based
  integer) the window belongs to.

  Typical use: an appliance that must run ``n`` times per day, each run
  constrained to a distinct time window.  Assign ``value=0`` to windows
  for the first cycle, ``value=1`` for the second, and so on.  Multiple
  windows may share the same cycle index (their allowed regions are unioned).
  Windows with ``value=None`` are silently ignored by all cycle-aware methods.

* fix: Make test_configmigrate also regard the _ANY_SENTENIEL in key values

* chore: Make devices configurations a map instead of a list

  This makes config paths stable regardless of declaration order and lets each device settings
  class build its own config path from ``self.device_id`` without needing an external index.
  Tests are adapted likewise.

  Devices configurations are automatically migrated from lists to maps.

* chore: rename levelized_cost_of_storage_kwh to levelized_cost_of_storage_amt kwh

  This better fits in the naming scheme and also makes clear the costs are money.

Signed-off-by: Bobby Noelte <b0661n0e17e@gmail.com>

* fix: runtime config update ignored by config file

Runtime settings were handed back to pydantic-settings as init settings,
which rank below the config file and the environment. Any key already
present in EOS.config.json or in the environment silently discarded the
update, so a bulk PUT /v1/config returned 200 without applying anything,
while the granular PUT /v1/config/{path} endpoint kept working.

Add a dedicated runtime settings source ranked directly below the command
line arguments and record granular updates there as well, so both
endpoints share one store that survives re-evaluation of the settings
sources. Environment variables keep precedence over the config file for
all keys that were not set at runtime.

Also repairs revert_settings() and update(), which passed their data
through the same init settings.

Closes #1303

* fix: env vars ignored on first config build

ConfigEOS.__init__ passed self as first positional argument to _setup,
which forwards it to pydantic_settings.BaseSettings.__init__. Its first
positional parameter is _case_sensitive, so the environment source
matched the upper case variable names against the lower case field names
and returned nothing. Environment settings only took effect after the
next configuration setup.

* docs: changelog for config priority fixes

* fix(config): preserve device identities and storage costs during migration

* fix(devices): preserve charge-rate typing and public import compatibility

* ruff format fix

* fix(config): satisfy typed device conversion and migration contracts

* docs(config): refresh validated configuration prerequisite schemas

---------

Signed-off-by: Bobby Noelte <b0661n0e17e@gmail.com>
Co-authored-by: Bobby Noelte <b0661n0e17e@gmail.com>
Co-authored-by: r0b2g1t <r0b2g1t@users.noreply.github.com>
2026-09-17 17:51:40 +02:00
AndreasandAndreas 431d7d57e5 docs(pvforecast): refresh API version after CI formatting 2026-09-17 13:53:54 +02:00
AndreasandAndreas 5a3a3bf07e style(pvforecast): apply CI import formatting 2026-09-17 13:53:54 +02:00
AndreasandAndreas 2d6acb51b9 docs(pvforecast): regenerate local backend configuration schema 2026-09-17 13:53:54 +02:00
AndreasandAndreas 6529c39df4 feat(pvforecast): add calibrated local Akkudoktor backend
Port local PV modeling and outage calibration from feature commits f976335, 6dc58c3 and faed0fd by Andreas. Keep PVForecastAkkudoktor identity and remote default, adapt to async storage, and migrate legacy provider settings.
2026-09-17 13:53:54 +02:00
AndreasandAndreas 75a65f4ff4 style: wrap imported tariff test parameter import 2026-09-17 13:53:35 +02:00
4b332d4c8a fix: preserve imported feed-in revenue during parameter preparation
Cancel GENETIC preparation when imported revenue cannot be read or contains invalid values, preserving the chosen provider instead of replacing it with demo tariffs. Keep valid positive, zero and negative amount/Wh series unchanged.

Adapt the revenue-preservation regressions from PRs #1224 and #1304 to the async main API, including real timestamped imports and simulation repricing. The feature-only direct-marketing override remains outside this main fix.

Co-authored-by: Christin <info@bikinibottom.capital>

Co-authored-by: Normann <github@koldrack.com>
2026-09-17 13:53:35 +02:00
AndreasandAndreas 0a3ced689c test: make optimization dispatch timezones explicit 2026-09-17 13:53:10 +02:00
AndreasandAndreas e5578e330c test: type dynamic Optimize regression arguments 2026-09-17 13:53:10 +02:00
AndreasandAndreas cfa245b41a fix: return only completed optimization results per run 2026-09-17 13:53:10 +02:00
AndreasandAndreas 1e0579f6b6 test(measurement): assert restored timestamps before timezone conversion 2026-09-17 13:52:48 +02:00
AndreasandAndreas 1f29663eb5 fix(measurement): restore JSON records into the existing singleton 2026-09-17 13:52:48 +02:00
NormannandGitHub 4a3724424f build(deps): combine Dependabot updates and refresh uv lock (#1321) 2026-09-16 15:44:36 +02:00
dr-dimitriandGitHub 7ebe6d714b fix: detect recent Energy-Charts source cadence (#1315)
Infer interval coverage from consistent recent original price spacings while preserving history and forecasting resolution policies. Reuse source data until a successful refresh and cover cadence transitions, fallback, and history repair.

Fixes #1279
2026-09-16 12:22:39 +02:00
NormannandGitHub afb7bcb8bc build(deps): consolidate Dependabot updates (#1311)
Consolidates seven compatible Dependabot updates into one reviewable change:
Runtime dependencies

    platformdirs 4.11.7 → 4.11.8 (build(deps): bump platformdirs from 4.11.7 to 4.11.8 #1307)
    rich-toolkit 0.20.4 → 0.20.5 (build(deps): bump rich-toolkit from 0.20.4 to 0.20.5 #1313)

Development, documentation, and CI dependencies

    types-requests 2.33.0.20260712 → 2.33.0.20260906 (build(deps-dev): bump types-requests from 2.33.0.20260712 to 2.33.0.20260906 #1306)
    pypdf 6.17.0 → 6.18.0 (build(deps-dev): bump pypdf from 6.17.0 to 6.18.0 #1308)
    types-PyYaml 6.0.12.20260815 → 6.0.12.20260906 (build(deps-dev): bump types-pyyaml from 6.0.12.20260815 to 6.0.12.20260906 #1309)
    sphinxcontrib-mermaid 2.1.0 → 2.1.1 (build(deps-dev): bump sphinxcontrib-mermaid from 2.1.0 to 2.1.1 #1310)
    GitPython 3.1.61 → 3.1.62 (build(deps-dev): bump gitpython from 3.1.61 to 3.1.62 #1312)

The committed uv.lock file is regenerated for all seven updates. The pre-commit mypy hook already executes the locked project development environment, so the type-stub pins have a single source of truth in pyproject.toml and uv.lock.
2026-09-14 08:26:16 +02:00
dr-dimitriandGitHub 446ea3da9d fix: preserve local time when parsing naive datetime strings (#1294)
to_datetime() previously interpreted naive strings differently depending on their precision. For example, 2026-01-15 23:45 in Europe/Berlin became 2026-01-16 00:45+01:00, while the equivalent string with seconds kept the intended date and time. Minutes, seconds, and fractional seconds now consistently represent 2026-01-15 23:45+01:00.

Pass the resolved target/local timezone to Pendulum's fallback parser. Explicit Z and numeric offsets continue to determine the input instant, and Unix timestamps retain their UTC semantics. Update the affected docstring examples to distinguish local wall time from UTC conversion.

Regression coverage includes space and T separators, explicit and default Europe/Berlin timezones, winter/summer dates near midnight, fractional-second precision, explicit offsets, and Unix timestamps. The Open-Meteo integration test now independently checks the timestamps and all three irradiance series for all 72 fixture hours, with both UTC and Europe/Berlin as the host timezone. This replaces an expectation that incorrectly associated the 08:00 values with 09:00. The previous parser misplaces all 72 source hours; the corrected parser maps all 72 correctly.
2026-09-14 08:24:55 +02:00
dr-dimitriandGitHub 3b9eecc52b fix: handle denied inspection in test server cleanup (#1301)
Track owned test processes and restrict fallback cleanup to verified EOS servers using the test configuration. Add regression coverage for denied inspection and cleanup failures.

Fixes #1297
2026-09-13 00:22:32 +02:00
dr-dimitriandGitHub 5469ba836c fix: check port availability without system-wide process inspection (#1300)
* fix: check port availability without process inspection

Probe local TCP addresses and use process inspection only for optional diagnostics. Preserve occupied-port detection, bounded waits, and port reuse after closed connections.

Fixes #1298

* fix: preserve IPv6 scope IDs in port availability probes
2026-09-11 22:07:25 +02:00
NormannandGitHub fe7f0fd821 docs: correct mypy regression test filename (#1293)
The mypy regression-test command in CONTRIBUTING.md points to the nonexistent tests/test_typingmypytoolchain.py, causing copied commands to fail.

Change the path to tests/test_typingmypytooling.py, the existing integration test.
2026-09-11 07:50:54 +02:00
1abdd345c4 fix: unify mypy environments for local checks and CI (#1291)
The isolated pre-commit mypy hook previously omitted runtime type information that
make mypy used, hiding errors involving dependencies such as Pydantic and Pendulum.
Makefile, pre-commit and CI now run the same full-project typing policy in the
development environment defined by uv.lock.

- Use uv run --locked --exact --extra dev and the same mypy arguments for Makefile
  and the local hook. Check all of src and tests, including on configuration-only
  changes.
- Pin Python 3.13 for local development and the pre-commit CI job, and install the
  locked pre-commit version in CI.
- Disable incremental analysis because existing Pendulum cache state changes mypy 2.3.1
  diagnostics. Document the policy, the performance tradeoff and the existing typing debt.
- Add a regression test that exercises Makefile, the hook and the CI command in a
  temporary project, accepting valid dependency types and detecting deliberate
  Pydantic/Pendulum assignment errors.

Resolve the newly detected mypy diagnostics.

- Enable the numpydantic and Pydantic mypy plugins, retaining strict Pydantic
  constructor typing with init_typed = true. Validate raw/coercible payloads through model_validate.
- Propagate concrete record, provider and time-window types through generic collections,
  factories and lookup methods. Preserve runtime field inspection and generated time-window
  documentation.
- Align Pendulum annotations with actual factory/arithmetic results while retaining Pydantic
  validation adapters at runtime. Correct optional values, array boundaries, REST handlers
  and plotting interfaces.
- Add pinned scipy-stubs and types-psutil, update uv.lock, and supply the plugins' dependencies.
- Add runtime regression coverage for validated path defaults, normalized time-series metadata,
  generic field inspection, invalid timestamps and unsupported provider imports.

Runtime and compatibility details:

- Validate path defaults as Path objects while retaining raw string defaults needed by
  migration serialization with exclude_defaults.
- Normalize feed-in tariff lists and default charge rates to NumPy arrays; reject missing
  timestamps/uninitialized values explicitly. Importing into a provider without import support
  returns HTTP 400.
- Public JSON schemas and OpenAPI structure match main (excluding the generated version).

Signed-off-by: dr-dimitry

Signed-off-by: dr-dimitry
Signed-off-by: Bobby Noelte <b0661n0e17e@gmail.com>
Co-authored-by: dr-dimitri <87113560+dr-dimitri@users.noreply.github.com>
Co-authored-by: Normann <github@koldrack.com>
2026-09-10 23:20:35 +02:00
5b584cbb57 chore: ignore the artifacts the test suite writes into the work tree (#1292)
test_configmigrate leaves a <fixture>.new working copy behind whenever a
migration test fails, test_visualize writes example_report.pdf into the current
directory, and test_config creates tests/testdata/docs/. All three showed up as
untracked noise in every status and invited accidental commits.

Co-authored-by: Andreas <drbacke@gmx.de>
2026-09-10 22:49:42 +02:00
NormannandGitHub d94ffae565 build(deps): consolidate Dependabot updates (#1290) 2026-09-09 07:26:05 +02:00
2d18547aaa build: multi-stage Docker image with uv and BuildKit caching (#1261)
* build: multi-stage Docker image with uv and BuildKit caching

Builder/runtime split keeps the toolchain out of the runtime image
(standalone ~1.8GB -> ~1.2GB). Dependencies install with uv from
uv.lock in a cache-mounted, source-independent layer. Runtime keeps
the editable install because akkudoktoreos.core.version needs the
src/ layout. Also fixes the io.hass.version label and adds a
HEALTHCHECK.

* build: address Docker image review feedback

- Healthcheck honours EOS_SERVER__PORT instead of hardcoding 8503.
- BUILD_VERSION defaults to "dev" rather than the literal "VERSION";
  docker-compose passes the real version, and the redundant
  org.opencontainers.image.version label is dropped (CI sets it via
  docker/metadata-action).

* fix: use resolved server port for container healthcheck

---------

Co-authored-by: Normann <github@koldrack.com>
Co-authored-by: Normann Koldrack <normann.koldrack@desy.de>
2026-09-07 08:13:09 +02:00
e9bd55caac fix: normalize expected configuration paths on macOS (#1282)
Co-authored-by: Normann <github@koldrack.com>
2026-09-06 22:10:33 +02:00
NormannandGitHub 25430bd42d fix: infer Energy-Charts coverage from recent source intervals (#1280) 2026-09-06 08:21:48 +02:00
dr-dimitriandGitHub 9d816acbd9 fix: honor Energy-Charts electricity price interval coverage (#1278)
* fix: honor Energy-Charts electricity price interval coverage

* test: make Energy-Charts coverage checks timezone-independent
2026-09-05 15:50:43 +02:00
dr-dimitriandGitHub 940aa1021f fix: honor Energy-Charts feed-in interval coverage (#1275) 2026-09-05 01:05:25 +02:00
NormannandGitHub d5df763c6c build(deps): consolidate Dependabot updates (#1266)
* build(deps): consolidate Dependabot updates

* build(deps): extend rollup and synchronize lockfile

* build(deps): bump linkify-it-py to 2.2.0
2026-09-04 09:28:23 +02:00
7bafc8d02d feat: add PVForecastHomeAssistant provider (#1258)
* feat: add PVForecastHomeAssistant provider

Reads a PV forecast time series directly from a Home Assistant entity
attribute (matching the {"forecast": [{"datetime", "watts"}]} shape
already exposed by common HA PV forecast integrations, e.g. Helios
Forecast and Solcast) and feeds it into pvforecast_ac_power, following
the same self-polling provider pattern as PVForecastVrm.

Closes #1232.

* fix: use MagicMock instead of monkeypatched Response for mypy

requests.Response().json is a typed bound method; reassigning it to a
lambda fails mypy's method-assign check. Use MagicMock(spec=...) instead,
which mocks the response without fighting its static type.

* fix: address PR review on PVForecastHomeAssistant provider

Fixes two issues raised in review on PR #1258:
- _update_data() left stale pvforecast_ac_power values in place when a
  refreshed forecast was empty or shorter than a previous one; it now
  clears the active forecast window before writing and raises instead
  of silently no-op'ing when the response has no usable data.
- pvforecast.homeassistant.entity_id used a "select" widget with no
  manual-entry fallback in EOSdash, leaving it unusable in standalone
  mode where the entity list can't be resolved without SUPERVISOR_TOKEN;
  switched to a plain text field.

Also fills in the config docs and openapi.json for the new
pvforecast.homeassistant.* fields, which were missing from the
original commit.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

* fix: preserve retained forecast history when clearing stale entries

The previous fix cleared pvforecast_ac_power from start-of-day, but
PredictionProvider deliberately retains historical records back to
keep_datetime (prediction.historic_hours). Since Home Assistant
forecasts are future-only, that clear wiped out retained history
between midnight and the EMS start on every refresh.

Narrow the clear to [ems_start_datetime, end_datetime) - the actual
active forecast window, DST-adjusted - instead of the day boundary.

Addresses review feedback from @NormannK on PR #1258.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

---------

Co-authored-by: Mathias <mathias@Mathiass-MacBook-Air.local>
Co-authored-by: Mathias <mathias@Mathiass-Air.localdomain>
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-03 00:15:40 +02:00
Bobby NoelteandGitHub ba76087db9 feat: electricity fee provider framework and generic providers (#1235)
Add new provider class for electricity fees providers.

Add the generic providers:
- ElecFeeFixed
- ElecFeeImport

The providers provide predictions for:

- elecfee_consumption_amt_wh:
  Total fixed fee for consumed energy per Wh [amount/Wh]. This is the accumulation of all
  fixed per-Wh fees payable on "consumed energy - such as network charge, concession fee,
  and electricity charge - into a single amount.
- elecfee_consumption_percent_amt:
  Total fixed surcharge on consumed energy, given as a percentage of the monetary amount
  already charged for that energy [%]. This is the accumulation of all percentage-based
  surcharges payable on top of the consumed-energy fee - such as VAT - into a single
  percentage. This is a percentage of the fee amount, not a per-Wh rate.
- elecfee_feedin_amt_wh:
  Total fixed deduction from feed-in energy per Wh [amount/Wh]. This is the accumulation of
  all fixed per-Wh charges deducted from feed-in energy - such as metering fees or
  grid-operator handling "charges - into a single amount. Applied after the percentage-based
  deduction, i.e. it reduces the price by a flat amount per Wh rather than by a share of the
  raw price.
- elecfee_feedin_percent_amt:
  Total percentage deducted from the raw feed-in price (spot price) [%]. This is the
  accumulation of all percentage-based deductions payable on the feed-in tariff - such as a
  marketing or balancing fee retained by the aggregator - into a single percentage. It is
  applied as `raw_price * (100 - percent) / 100`, i.e. it scales down the raw price rather
  than adding a surcharge to it.

A new _apply_fee() method is added to the base class for ElecPrice and FeedInTariff to be used to
add the fees in a consistent way. Fees are taken from the active ElecFee provider and applied
to the raw prices given to the _apply_fee() method.

The optional application of fees is added to:

- ElecPriceAkkudoktor
- ElecPriceFixed
- ElecPriceEnergyCharts
- ElecPriceSMARD
- FeedInTariffEnergyCharts
- FeedInTariffFixed
- FeedInTariffSMARD

The import providers ElecPriceImport and FeedInTariffImport do not apply fees by intentention.

The following providers currently do not handle fees defined by ElecFee:

- ElecPriceTibber
- FeedInTariffAkkudoktor
- FeedInTariffDvhubOnline
- FeedInTariffTibber

The tests for this feature are either added or existing tests are extended.

The documentation was extended for the electricity fee provider settings.

Besides this feature further improvements are added:

* feat: add SMARD quarter-hour electricty price and feed-in tariff provider

* feat: to_series method for TimeWindows and ValueTimeWindows

  Additional to to_array the time window sequence can now also produce a pandas series.
  Test have been extended to cover the series generation.

* feat: use time windows in fixed feedin tariff provider

  Feedin tariff can now be configured by time windows - not a single value.

* feat: EOSdash select for PVLib inverters and modules

  Provide PVLib inverter and module names in config selection.

* feat: EOSdash lazy select for big option sets

  Add a new form for lazy selection of big option sets. Filtering and
  generation of the option set is done server-side.

* fix: use raw data for ETS/ median prediction

  Use to raw time series data for ETS/ median prediction to avoid interference
  by e.g. dynamic grid charges.

* fix: EOSdash config drops by type only on details resolve

  Drop configuration by type and path. Prevents dropping of configuration items
  with same type and level but different path.

* fix: EOSdash configuration section closes on update

  Open section if searching or if last update touched this category — including
  updates on deeply nested sub-fields.

* chore: make elecfeefixed, elecpricefixed and feedintarifffixed warn about no windows and default to 0

  Missining configuration creates default 0 value and a warning instead of an exception.

* fix: test setup for providers

  Reset db state on each test run.

* chore: improve config option naming for elecpricefixed.

* chore: adapt elecpricefixed test to changed time_windows naming

* chore: factorized common price provider helpers to priceabc.py

  Factorized common price provider helpers to priceabc.py. Add tests for these helpers.
  Reduce/ change testing of elecpriceabc.py and feedintariffabc.py to cover
  only specifics. Rest of testing is already covered by test_priceabc.py.

* chore: update version

Signed-off-by: Bobby Noelte <b0661n0e17e@gmail.com>
2026-08-23 02:26:16 +02:00
NormannandGitHub 6849b731b0 chore(deps): combine Dependabot updates (#1250)
### Significant changes / EOS impact

The dependency bumps are mostly patch/tooling updates, but a few are worth explicit review:

- **`pydantic-settings` 2.14.2 → 2.15.0 — highest runtime impact.** EOS directly subclasses `pydantic_settings.BaseSettings` for `SettingsEOS` / `ConfigEOS`, with `case_sensitive` left at its default. In 2.15.0, `case_sensitive` now also applies to init kwargs and config-file sources, so top-level setting keys are now matched case-insensitively by default where those sources previously did not behave that way. This can change how differently-cased config keys are accepted/resolved. The release also adds warnings for unresolved forward references and changes strict non-JSON env-value failures to `ValidationError`. **Recommended:** exercise JSON config loading, init/update paths, env overrides, and config round-trips.

- **`tzfpy` 1.3.2 → 1.3.3 — runtime correctness change.** EOS uses `tzfpy.get_tz()` in `to_timezone()` and exposes the result through `GeneralSettings.timezone`. Queries exactly on timezone polygon borders now resolve instead of returning an empty result, and the `America/Argentina/Ushuaia` boundary is corrected. This can intentionally change timezone output for users on/near affected boundaries.

- **`cachebox` 6.2.2 → 6.2.5 — runtime cache correctness/safety.** EOS uses `cachebox.LRUCache` and `cachebox.cached` for the energy-management cache. The update fixes iterator lifetime safety, LRU iterator invalidation when reads promote entries, and a potential `setdefault_with` locking issue. EOS does not appear to rely on those edge cases directly, so this is expected to be low-risk and mostly corrective; existing cache tests are the relevant regression coverage.

- **`GitPython` 3.1.58 → 3.1.59 — dev/tooling security hardening.** This release blocks file-reading Git options, separate git-directory use during clone, and hardens config parsing. It appears to be a dev/docs dependency rather than EOS runtime code, but CI/tooling that intentionally passes unusual Git options could be affected.

- **`pre-commit` 4.6.1 → 4.6.2 — dev-only bug fix.** Fixes a regression in Node-language hooks using npm 11.x build scripts.

- **`mypy` 2.3.1, `commitizen` 4.17.1, and `types-PyYaml` stub update** are tooling/type-checking changes with no expected EOS runtime behavior change.

Overall, the main compatibility focus should be **configuration handling (`pydantic-settings`)**, followed by **timezone edge cases (`tzfpy`)**. The remaining updates are primarily correctness, security, or developer-tooling fixes.
2026-08-22 00:11:08 +02:00
NormannandGitHub 9b3f37e172 test: clean up git and artifact warnings (#1244)
* initialize the temporary Git repository in test_workflow_git with --initial-branch=main
* avoid Git's default-branch advisory in pytest logs without changing global Git configuration or suppressing stderr
* upload generated optimization result artifacts only when the test job fails
* use the actual nested tests/testdata/**/new_optimize_result* paths
* ignore the no-files case so unrelated test failures do not produce an artifact warning
2026-08-22 00:09:22 +02:00
NormannGitHubgithub-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
9eb3c7e483 build(deps): refresh dependencies and clean up test warnings (#1243)
Consolidates the currently applicable dependency updates into one PR, including the closed Dependabot backlog such as #1241, plus dependency surfaces that were not covered by the repository's previous pip-only Dependabot configuration.

Cleanup of test warnings.

Most importent:

* GitPython + pypdf security hardening.
* Uvicorn WebSocket close/backpressure/header fixes for server/dashboard reliability.
* FastAPI dependency-memory/OpenAPI improvements for the API process.
* Bokeh WebSocket/resource-leak/prefix fixes for EOSdash and proxied deployments.
* cachebox cancellation/lock cleanup fixes for long-running/concurrent work.
* pandas 3.0.5 avoiding the yanked 3.0.4 datetime/segfault build.
* Ruff security-lint and pydocstyle correctness fixes, plus faster release builds via PGO.
* platformdirs malformed-XDG and duplicate-directory fixes for deployment portability.
* CI action modernization, regenerated uv.lock, and expanded Dependabot coverage.

Runtime dependencies

    cachebox: 6.1.2 → 6.2.2
    fastapi: 0.139.2 → 0.141.1
    python-fasthtml: 0.14.9 → 0.14.11
    MonsterUI: 1.0.46 → 1.0.47
    bokeh: 3.9.1 → 3.9.2
    uvicorn: 0.51.0 → 0.52.4 (build(deps): bump uvicorn from 0.51.0 to 0.52.3 #1241, refreshed to latest patch)
    pandas: 3.0.3 → 3.0.5
    platformdirs: 4.11.0 → 4.11.3

Development/test dependencies

    pandas-stubs: 3.0.3.260530 → 3.0.5.260730
    types-PyYAML: 6.0.12.20260518 → 6.0.12.20260724
    GitPython: 3.1.53 → 3.1.58 (security/fix releases)
    coverage: 7.15.2 → 7.15.4
    pypdf: 6.14.2 → 6.16.1 (includes security fixes)

Pre-commit/tooling

    ruff-pre-commit: v0.15.21 → v0.16.3
    synchronize pandas-stubs, types-docutils, and types-PyYAML pins with pyproject.toml

CI / repository dependencies

    Python 3.13.9 → 3.13.15 in CI, Docker, .env, and local Docker Make targets
    actions/checkout → v7 in pytest, pre-commit, CodeQL, and release workflows
    actions/setup-python → v7 in pytest, pre-commit, and release workflows
    actions/upload-artifact → v7 in pytest workflow
    actions/stale: v9.1.0 → v11.0.0 (SHA-pinned)
    regenerate uv.lock from the final dependency pins so locked/frozen installs match pyproject.toml

Future update coverage

Expand Dependabot from pip-only to also monitor:

    GitHub Actions
    Docker

The existing open docutils 0.23 update (#1085) is intentionally excluded because it has separate compatibility/ignore handling and should remain isolated.

docker-build.yml was audited and is already using the newer action generations, so no changes were needed there.

---------

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2026-08-21 07:07:34 +02:00
NormannandGitHub 230698a70d docs: use uv for development setup (#1236)
* docs: use uv for development setup

* docs: restore contributor footer

* docs: run development commands with uv
2026-08-20 02:02:49 +02:00
Bobby NoelteandGitHub 886c93c92b fix: default server settings prevent env var config (#1234)
Change configuration source priorities to:

- cli
- environment vars
- dotenv settings
- config file settings
- init settings

By this the environment vars supersede any configuration var
provided by the configuration file or by the initialisation
with pydantic.

The test_config.py::test_computed_path was fixed to to not
use the defaul env var overwrite defined by conftest.py.
This seemed to indicate non working env vars, but in fact
was a test setupt fault.

Besides this fix there are other fixes and changes added:

* fix: exclude computed fields when merging settings

  Pydantic may overwrite settings by values given for computed
  fields and use these values instead of re-computing the field.
  Avoid computed fields in merging settings.

* chore: improve Windows compatability of development setup

  Improve scripts to better run also on Windows. When doing path
  checks keep compatibility also to Windows pathes. A lot of
  changes to avoid the famous Windows CRLF handling and keep
  line endings to LF.

* chore: add development hint for Windows

  Windows developers should set core.autocrlf to false.

* chore: update version

Signed-off-by: b0661 <b0661n0e17e@gmail.com>
2026-08-16 13:56:25 +02:00
Bobby NoelteandGitHub 269d2162a9 fix: server crash on prediction update failure (#1222)
Catch exceptions from prediction update and log as error message. Keep
executing the energy management run.

Signed-off-by: Bobby Noelte <b0661n0e17e@gmail.com>
2026-08-08 11:37:50 +02:00
NormannandGitHub 74d31e2fac fix PVLib mountingplace None handling (#1220)
Handle an explicit None value for pvforecast.planes[].mountingplace before calling .lower().
2026-08-07 15:26:03 +02:00
NormannandGitHub 682e0ccc2b fix PVLib inverter error message (#1219) 2026-08-07 13:55:26 +02:00
Bobby NoelteandGitHub 894790f577 feat: add pvlib pv forecast provider (#1214)
Add a PV forecast provider that calculates the forecast using a PVLib system model
and weather forecast from the EOS weather forecast provider.

Additional module and inverter models can be easily added as the database
is build from PVLib and SAM databases and a bundled csv file.

The module model and inververt model names are provided by new endpoints
to be used in configuration.

The provider is based on the fantastic work of EMHASS. See
https://github.com/davidusb-geek/emhass/blob/master/src/emhass/forecast.py

A short description of the provider is added to the documentation.

Besides the new features there are the fixes and improvements:

* feat: improve EOSdash config page

* fix: kex_to_series for start_datetime

  Make key_to_series always start the series at start_datetime.

* fix: default provider for GENETIC and GENETIC0 optimization

  To make the default less dependent on internet servers (with API changes and
  availability issues) the default for PVForecast is set to PVForecastPVLib
  and for ElecPrice to ElecPriceFixed. The default weather provider is changed
  to OpenMeteo.

* fix: EOSdash display resampled prediction values

  Make EOSdash display resampled prediction values where  resampling fits to
  the prediction value type. Use bar width that fits to 15 minutes value samples.

* chore: add a UI hints system to EOSdash

  The UI hints system eases the definition of forms for configuration items.
  There are also forms for items in maps and lists. These forms allow to add and delete
  items to/ from  maps and lists. The forms ensure that all required fields of newly
  added items are filled.

* chore: Create an enum for valid optimization algorithms

* chore. Make config also provide the available energy management modes.

  Used for configuration hints.

* chore: Randomize default device id in configuration

Signed-off-by: Bobby Noelte <b0661n0e17e@gmail.com>
2026-08-07 13:13:17 +02:00