11 Commits

Include renames

Author	SHA1	Message	Date
7 people	04f28997ea	feat: deliver complete GENETIC optimization to main (#1330) ... * feat: adapt configuration for multi optimization algorithms Decouple configuration from optimization algorithm parameters. Add to_[algorithm]_param() methods to the configuration that derive optimization algorithm specific parameters from the configuration. Add x-scope tags to the configuration options that describe for which specific algorithms the configuration option is for. The whole device settings are restructured. There are now general settings for the device classes with the afore mentioned to_[algorithm]_param() methods. The general device settings got their own directory `devices/settings`. By this the parameter class also does not have to be a pydantic model which can be used for future optimization/ simulations speed up. Also the parameter class for a device is now part of the device module. This better decouples and also is the natural place for parameters of a device. Besides this feature there are also fixes and improvements: * feat: extend home appliance time window settings and simulation Home appliance can now be configured for multiple runs with per-cycle allowed time windows. The number of remaining cycles to plan is determined at runtime by reading the ``cycles_completed_measurement_key`` from the measurement store. * feat: specialiced CycleTimeWindowSequence for time window sequences Sequence of time windows associated to cycles. This model specializes ``ValueTimeWindowSequence`` so that the ``value`` field of each ``ValueTimeWindow`` encodes the **cycle index** (0-based integer) the window belongs to. Typical use: an appliance that must run ``n`` times per day, each run constrained to a distinct time window. Assign ``value=0`` to windows for the first cycle, ``value=1`` for the second, and so on. Multiple windows may share the same cycle index (their allowed regions are unioned). Windows with ``value=None`` are silently ignored by all cycle-aware methods. * fix: Make test_configmigrate also regard the _ANY_SENTENIEL in key values * chore: Make devices configurations a map instead of a list This makes config paths stable regardless of declaration order and lets each device settings class build its own config path from ``self.device_id`` without needing an external index. Tests are adapted likewise. Devices configurations are automatically migrated from lists to maps. * chore: rename levelized_cost_of_storage_kwh to levelized_cost_of_storage_amt kwh This better fits in the naming scheme and also makes clear the costs are money. Signed-off-by: Bobby Noelte <b0661n0e17e@gmail.com> * fix: runtime config update ignored by config file Runtime settings were handed back to pydantic-settings as init settings, which rank below the config file and the environment. Any key already present in EOS.config.json or in the environment silently discarded the update, so a bulk PUT /v1/config returned 200 without applying anything, while the granular PUT /v1/config/{path} endpoint kept working. Add a dedicated runtime settings source ranked directly below the command line arguments and record granular updates there as well, so both endpoints share one store that survives re-evaluation of the settings sources. Environment variables keep precedence over the config file for all keys that were not set at runtime. Also repairs revert_settings() and update(), which passed their data through the same init settings. Closes #1303 * fix: env vars ignored on first config build ConfigEOS.__init__ passed self as first positional argument to _setup, which forwards it to pydantic_settings.BaseSettings.__init__. Its first positional parameter is _case_sensitive, so the environment source matched the upper case variable names against the lower case field names and returned nothing. Environment settings only took effect after the next configuration setup. * docs: changelog for config priority fixes * fix(config): preserve device identities and storage costs during migration * fix(measurement): restore JSON records into the existing singleton * fix(devices): preserve charge-rate typing and public import compatibility * feat(measurement): integrate typed energy quality and capacity APIs Port the locally backed-up measurement extensions to main async storage and PR #1256 device maps. Preserve runtime capacity estimates across #1305 bulk updates. Confirm JSON singleton restore defect on unchanged main and add regression. No production configuration or measurements included. Co-authored-by: Andreas <drbacke@gmx.de> * docs(measurement): describe household settings and consolidate regression coverage * docs(measurement): regenerate configuration and API contracts * test(measurement): isolate capacity database state between tests * ruff format fix * fix(measurement): restore JSON records into the existing singleton * test(measurement): assert restored timestamps before timezone conversion * test(measurement): assert restored timestamps before timezone conversion * fix: preserve imported feed-in revenue during parameter preparation Cancel GENETIC preparation when imported revenue cannot be read or contains invalid values, preserving the chosen provider instead of replacing it with demo tariffs. Keep valid positive, zero and negative amount/Wh series unchanged. Adapt the revenue-preservation regressions from PRs #1224 and #1304 to the async main API, including real timestamped imports and simulation repricing. The feature-only direct-marketing override remains outside this main fix. Co-authored-by: Christin <info@bikinibottom.capital> Co-authored-by: Normann <github@koldrack.com> * feat(devices): port slot-aware battery export and direct-use physics Port scoped device changes from d2e2d58237. Keep PR #1256 parameter conversion structure and separate GENETIC0 devices. Validate physical flows and reprice changed simulation results. Co-authored-by: Andreas <drbacke@gmx.de> Co-authored-by: Christin <info@bikinibottom.capital> * docs(measurement): align API version with refreshed prerequisites * fix: return only completed optimization results per run * feat(pvforecast): add calibrated local Akkudoktor backend Port local PV modeling and outage calibration from feature commits f976335, 6dc58c3 and faed0fd by Andreas. Keep PVForecastAkkudoktor identity and remote default, adapt to async storage, and migrate legacy provider settings. * fix(cache): distinguish callables in the shared EMS cache Include the function object in cache keys so methods of one interpolator cannot reuse a probability as a power value. Cover both call orders, keyword arguments, cache hits and separate closures with identical qualified names. * fix(devices): constrain the physics port and validate export levels Defer inactive EV deadline fields to the optimizer port, reject nonfinite export rates, and document the hourly Optimize boundary. Verify converter IDs, rates and LCOS, separate GENETIC0 interpolation, physical boundary flows and independent GENETIC repricing. * docs(pvforecast): regenerate local backend configuration schema * docs(devices): regenerate slot-physics configuration and OpenAPI schemas * test: type dynamic Optimize regression arguments * style: wrap imported tariff test parameter import * style(pvforecast): apply CI import formatting * docs(pvforecast): refresh API version after CI formatting * fix(config): satisfy typed device conversion and migration contracts * docs(config): refresh validated configuration prerequisite schemas * fix(measurement): enforce typed capacity and sample validation * test(devices): align physics regressions with strict type checking * style(measurement): normalize imports for CI * docs(measurement): refresh typed measurement API schemas * docs(devices): refresh API version after prerequisite merge * test: make optimization dispatch timezones explicit * docs(interpolator): use portable reStructuredText markup * docs(devices): refresh API version after docstring compatibility fix * feat: complete configuration-driven GENETIC optimization and reports (#1329) * feat(devices): port slot-aware battery export and direct-use physics Port scoped device changes from d2e2d58237. Keep PR #1256 parameter conversion structure and separate GENETIC0 devices. Validate physical flows and reprice changed simulation results. Co-authored-by: Andreas <drbacke@gmx.de> Co-authored-by: Christin <info@bikinibottom.capital> * feat(optimization): port tested terminal and tail value primitives Source d2e2d58237. 22 primitive tests pass; integration with the optimizer, forecast horizon and API is still pending. Co-authored-by: Andreas <drbacke@gmx.de> Co-authored-by: Christin <info@bikinibottom.capital> * fix(devices): preserve charge-rate typing and public import compatibility * feat(measurement): integrate typed energy quality and capacity APIs Port the locally backed-up measurement extensions to main async storage and PR #1256 device maps. Preserve runtime capacity estimates across #1305 bulk updates. Confirm JSON singleton restore defect on unchanged main and add regression. No production configuration or measurements included. Co-authored-by: Andreas <drbacke@gmx.de> * test(integration): validate optimizer economics and document measurement settings * docs(integration): record tested checkpoint and remaining consolidation work * docs(development): define isolated PR packages and remaining porting gates * docs(integration): refresh API version after measurement reconciliation * docs(integration): record PR readiness verification results * docs(development): record publication and verification of PR 1322 * test(measurement): assert restored timestamps before timezone conversion * docs(development): record corrected PR head and CI progress * docs(integration): refresh API version after prerequisite alignment * docs(integration): define parallel packages and Optimize compatibility gates * fix: preserve imported feed-in revenue during parameter preparation Cancel GENETIC preparation when imported revenue cannot be read or contains invalid values, preserving the chosen provider instead of replacing it with demo tariffs. Keep valid positive, zero and negative amount/Wh series unchanged. Adapt the revenue-preservation regressions from PRs #1224 and #1304 to the async main API, including real timestamped imports and simulation repricing. The feature-only direct-marketing override remains outside this main fix. Co-authored-by: Christin <info@bikinibottom.capital> Co-authored-by: Normann <github@koldrack.com> * test(integration): verify tariff protection with mapped device physics * fix: return only completed optimization results per run * test(integration): verify algorithm aliases and mapped-device contracts * fix(cache): distinguish callables in the shared EMS cache Include the function object in cache keys so methods of one interpolator cannot reuse a probability as a power value. Cover both call orders, keyword arguments, cache hits and separate closures with identical qualified names. * fix(devices): constrain the physics port and validate export levels Defer inactive EV deadline fields to the optimizer port, reject nonfinite export rates, and document the hourly Optimize boundary. Verify converter IDs, rates and LCOS, separate GENETIC0 interpolation, physical boundary flows and independent GENETIC repricing. * feat(pvforecast): add calibrated local Akkudoktor backend Port local PV modeling and outage calibration from feature commits f976335, 6dc58c3 and faed0fd by Andreas. Keep PVForecastAkkudoktor identity and remote default, adapt to async storage, and migrate legacy provider settings. * docs(integration): record combined compatibility checks and green JSON PR CI * test: type dynamic Optimize regression arguments * docs(integration): record Optimize fix PR publication * docs(integration): record imported tariff protection PR * style(pvforecast): apply CI import formatting * style(integration): align combined regression imports * test: make optimization dispatch timezones explicit * docs(interpolator): use portable reStructuredText markup * chore: validate combined integration with locked mypy * docs: hand off six validated pull requests for manual review * feat: report genetic interval and terminal value diagnostics * feat(devices): reconcile flexible profiles and EV deadlines with cycle scheduling Adapt the flexible consumer primitives from d2e2d582 while retaining the keyed settings and per-cycle scheduling introduced by #1256. Preserve slot battery physics and GENETIC0 flat-load conversion. Cover energy conservation, deadlines, window intersections, DST, completed cycles and EV converters. * test: satisfy typed genetic PDF chart contracts * feat(optimization): resolve quarter-hour GENETIC requests from configuration * test(genetic): verify real device scheduling, measurement and export contracts Register appliance completed-cycle measurement keys so the real store accepts both default and custom counters. Exercise complete low-budget optimizer runs, persisted measurements, generic solution output and instructions, including zero-power phases, EV departure boundaries, per-cycle windows and LCOS. * fix: bound genetic report forecasts to executable horizon * feat: complete native genetic scheduling and retained result contracts * fix: retain missing raw samples when dropna is disabled * fix: align local optimization slots and measurement instants * docs: explain complete genetic rollout and PR dependencies * feat: expose retained GENETIC report through the versioned API * docs: regenerate complete genetic configuration and API schema * docs: format consolidation and review handoff markdown * test: align isolated EMS fixture with native genetic run options * test(genetic): clean up singleton measurements after device integration tests * test: freeze the clock without replacing timestamp conversion * fix: preserve explicit warmstart timezones in runtime requests * test(genetic): validate device schedules in UTC and Berlin Use explicit Berlin origins for Berlin wall-clock windows, compare absolute deadline instants correctly, and run all real device optimizer scenarios under both UTC and Europe/Berlin. Compare exported starts in the run timezone instead of assuming the output timezone matches the host. * fix: start automatic genetic runs in the site timezone * Preserve aware GENETIC snapshot times across host timezones * docs: specify site clock and rehearsed merge resolutions * test: isolate invalid measurement records and refresh API version * fix: render single-slot genetic tail diagnostics * docs: refresh schema version after report fix * fix: preserve configuration-only Optimize API contract * docs: refresh configuration request schema --------- Co-authored-by: Christin <info@bikinibottom.capital> Co-authored-by: Normann <github@koldrack.com> --------- Signed-off-by: Bobby Noelte <b0661n0e17e@gmail.com> Co-authored-by: Bobby Noelte <b0661n0e17e@gmail.com> Co-authored-by: r0b2g1t <r0b2g1t@users.noreply.github.com> Co-authored-by: Normann <github@koldrack.com> Co-authored-by: Christin <info@bikinibottom.capital>	2026-09-17 20:14:24 +02:00
4 people	1abdd345c4	fix: unify mypy environments for local checks and CI (#1291) ... The isolated pre-commit mypy hook previously omitted runtime type information that make mypy used, hiding errors involving dependencies such as Pydantic and Pendulum. Makefile, pre-commit and CI now run the same full-project typing policy in the development environment defined by uv.lock. - Use uv run --locked --exact --extra dev and the same mypy arguments for Makefile and the local hook. Check all of src and tests, including on configuration-only changes. - Pin Python 3.13 for local development and the pre-commit CI job, and install the locked pre-commit version in CI. - Disable incremental analysis because existing Pendulum cache state changes mypy 2.3.1 diagnostics. Document the policy, the performance tradeoff and the existing typing debt. - Add a regression test that exercises Makefile, the hook and the CI command in a temporary project, accepting valid dependency types and detecting deliberate Pydantic/Pendulum assignment errors. Resolve the newly detected mypy diagnostics. - Enable the numpydantic and Pydantic mypy plugins, retaining strict Pydantic constructor typing with init_typed = true. Validate raw/coercible payloads through model_validate. - Propagate concrete record, provider and time-window types through generic collections, factories and lookup methods. Preserve runtime field inspection and generated time-window documentation. - Align Pendulum annotations with actual factory/arithmetic results while retaining Pydantic validation adapters at runtime. Correct optional values, array boundaries, REST handlers and plotting interfaces. - Add pinned scipy-stubs and types-psutil, update uv.lock, and supply the plugins' dependencies. - Add runtime regression coverage for validated path defaults, normalized time-series metadata, generic field inspection, invalid timestamps and unsupported provider imports. Runtime and compatibility details: - Validate path defaults as Path objects while retaining raw string defaults needed by migration serialization with exclude_defaults. - Normalize feed-in tariff lists and default charge rates to NumPy arrays; reject missing timestamps/uninitialized values explicitly. Importing into a provider without import support returns HTTP 400. - Public JSON schemas and OpenAPI structure match main (excluding the generated version). Signed-off-by: dr-dimitry Signed-off-by: dr-dimitry Signed-off-by: Bobby Noelte <b0661n0e17e@gmail.com> Co-authored-by: dr-dimitri <87113560+dr-dimitri@users.noreply.github.com> Co-authored-by: Normann <github@koldrack.com>	2026-09-10 23:20:35 +02:00
Bobby NoelteandGitHub	894790f577	feat: add pvlib pv forecast provider (#1214) ... Add a PV forecast provider that calculates the forecast using a PVLib system model and weather forecast from the EOS weather forecast provider. Additional module and inverter models can be easily added as the database is build from PVLib and SAM databases and a bundled csv file. The module model and inververt model names are provided by new endpoints to be used in configuration. The provider is based on the fantastic work of EMHASS. See https://github.com/davidusb-geek/emhass/blob/master/src/emhass/forecast.py A short description of the provider is added to the documentation. Besides the new features there are the fixes and improvements: * feat: improve EOSdash config page * fix: kex_to_series for start_datetime Make key_to_series always start the series at start_datetime. * fix: default provider for GENETIC and GENETIC0 optimization To make the default less dependent on internet servers (with API changes and availability issues) the default for PVForecast is set to PVForecastPVLib and for ElecPrice to ElecPriceFixed. The default weather provider is changed to OpenMeteo. * fix: EOSdash display resampled prediction values Make EOSdash display resampled prediction values where resampling fits to the prediction value type. Use bar width that fits to 15 minutes value samples. * chore: add a UI hints system to EOSdash The UI hints system eases the definition of forms for configuration items. There are also forms for items in maps and lists. These forms allow to add and delete items to/ from maps and lists. The forms ensure that all required fields of newly added items are filled. * chore: Create an enum for valid optimization algorithms * chore. Make config also provide the available energy management modes. Used for configuration hints. * chore: Randomize default device id in configuration Signed-off-by: Bobby Noelte <b0661n0e17e@gmail.com>	2026-08-07 13:13:17 +02:00
Bobby NoelteandGitHub	e23bb7b497	chore: prepare for update of genetic algorithm (#1190) ... Andreas will update the genetic algorithm for 15-minutes optimization intervals. Copy the current GENETIC optimization algorithm to GENETIC0 to enable to keep the algorithm with the current functionality. Also copy resources like the load interpolator to the GENETIC0 algorithm to keep them despite possible later changes to the interpolator. Make the deprecated legacy /optimize endpoint use the GENETIC0 optimization algorithm to in-fact behave the same way even if there will later be changes to the GENETIC algorithm by Andreas. Add a new REST endpoint to provide the unprocessed optimisation results of the GENETIC and GENETIC0 algorithm in case one wants to use them as done with the deprecated /optimize endpoint. Adapt the optimization configuration to have distinct configurations for the GENETIC and the GENETIC0 algorithm. Create a copy of the current tests for the GENETIC algorithm to be used for the GENETIC0 algorithm. This avoids the tests for the GENETIC0 algorithm to be influenced by later changes by Andreas. Signed-off-by: Bobby Noelte <b0661n0e17e@gmail.com>	2026-07-29 12:56:08 +02:00
Bobby NoelteandGitHub	7e5aa2f218	feat: add Tibber feed-in tariff provider (#1189) ... The `FeedInTariffTibber` provider requests `priceInfo` and `priceInfoRange` with `resolution: QUARTER_HOURLY` and preserves the native 15-minute timestamps. It uses Tibber's `energy` spot-price component without the `tax` part or EOS electricity-price charges. The end-customer `total` component is deliberately ignored. The provider deliberately rejects hourly API responses instead of silently repeating them. It reuses `elecprice.tibber.access_token` and `elecprice.tibber.home_id`, so no duplicate credentials are needed. Signed-off-by: Andreas Schmitz <akkudoktor.net> Signed-off-by: Bobby Noelte <b0661n0e17e@gmail.com>	2026-07-24 15:52:26 +02:00
3 people	6f28022ed0	fix: optimization fail after restart (#1007) ... Fix documentation for the loadforecast_power_w key. Fix documentation to explain the usage of import file/ JSON string to primarily initialise prediction data. Fix code scanning alert no. 6: URL redirection from remote source Enable to automatically save the configuration to the configuration file by default, which is a widespread user expectation. Make the genetic parameters non optional for better pydantic compliance. Update: - bump pytest to 9.0.3 - bump pillow to 12.2.0 - bump platformdirs to 4.9.6 - bump typespyyaml to 6.0.12.20260408 - bump tzfpy to 1.2.0 - bump pydantic to 2.13.0 - bump types-requests to 2.33.0.20260408 Signed-off-by: Bobby Noelte <b0661n0e17e@gmail.com> Co-authored-by: Normann <github@koldrack.com>	2026-04-15 08:48:56 +02:00
Bobby NoelteandGitHub	997e7646e9	fix: prevent exception when load prediction data is missing (#925) ... Validate solution prediction data before processing. If required prediction data is missing, the prediction is skipped instead of raising an exception. Introduce a new configuration file saving policy to improve loading robustness: - Exclude computed fields - Exclude fields set to their default values - Exclude fields with value None - Use field aliases - Recursively remove empty dictionaries and lists - Ensure general.version is always present and correctly set When loading older configuration files, computed fields are now stripped before migration. This further improves backward compatibility and loading robustness. Signed-off-by: Bobby Noelte <b0661n0e17e@gmail.com>	2026-03-07 14:46:30 +01:00
Bobby NoelteandGitHub	237af5289f	fix: eosdash startup (#915) ... Adapt uvicorn log level to allowed levels. Ensure that EOSdash is started after EOS configuration is available. Signed-off-by: Bobby Noelte <b0661n0e17e@gmail.com>	2026-02-28 11:31:51 +01:00
Christopher NadlerandGitHub	3ccc25d731	Adds inverter AC/DC efficiency and break-even penalty (#888) ... * feat: add inverter AC/DC efficiency and break-even penalty * test: update tests/test_geneticoptimize.py with new ac_charge_break_even parameter * docs: update documentation * chore: update version numbers in configuration files to v0.2.0.dev2602272006923535	2026-02-27 23:12:08 +01:00
Bobby NoelteandGitHub	58d70e417b	feat: add Home Assistant and NodeRED adapters (#764) ... Adapters for Home Assistant and NodeRED integration are added. Akkudoktor-EOS can now be run as Home Assistant add-on and standalone. As Home Assistant add-on EOS uses ingress to fully integrate the EOSdash dashboard in Home Assistant. The fix includes several bug fixes that are not directly related to the adapter implementation but are necessary to keep EOS running properly and to test and document the changes. * fix: development version scheme The development versioning scheme is adaptet to fit to docker and home assistant expectations. The new scheme is x.y.z and x.y.z.dev<hash>. Hash is only digits as expected by home assistant. Development version is appended by .dev as expected by docker. * fix: use mean value in interval on resampling for array When downsampling data use the mean value of all values within the new sampling interval. * fix: default battery ev soc and appliance wh Make the genetic simulation return default values for the battery SoC, electric vehicle SoC and appliance load if these assets are not used. * fix: import json string Strip outer quotes from JSON strings on import to be compliant to json.loads() expectation. * fix: default interval definition for import data Default interval must be defined in lowercase human definition to be accepted by pendulum. * fix: clearoutside schema change * feat: add adapters for integrations Adapters for Home Assistant and NodeRED integration are added. Akkudoktor-EOS can now be run as Home Assistant add-on and standalone. As Home Assistant add-on EOS uses ingress to fully integrate the EOSdash dashboard in Home Assistant. * feat: allow eos to be started with root permissions and drop priviledges Home assistant starts all add-ons with root permissions. Eos now drops root permissions if an applicable user is defined by paramter --run_as_user. The docker image defines the user eos to be used. * feat: make eos supervise and monitor EOSdash Eos now not only starts EOSdash but also monitors EOSdash during runtime and restarts EOSdash on fault. EOSdash logging is captured by EOS and forwarded to the EOS log to provide better visibility. * feat: add duration to string conversion Make to_duration to also return the duration as string on request. * chore: Use info logging to report missing optimization parameters In parameter preparation for automatic optimization an error was logged for missing paramters. Log is now down using the info level. * chore: make EOSdash use the EOS data directory for file import/ export EOSdash use the EOS data directory for file import/ export by default. This allows to use the configuration import/ export function also within docker images. * chore: improve EOSdash config tab display Improve display of JSON code and add more forms for config value update. * chore: make docker image file system layout similar to home assistant Only use /data directory for persistent data. This is handled as a docker volume. The /data volume is mapped to ~/.local/share/net.akkudoktor.eos if using docker compose. * chore: add home assistant add-on development environment Add VSCode devcontainer and task definition for home assistant add-on development. * chore: improve documentation	2025-12-30 22:08:21 +01:00
Bobby NoelteandGitHub	7bf9dd723e	chore: improve doc generation and test (#762) ... Improve documentation generation and add tests for documentation. Extend sphinx by todo directive. The configuration table is now split into several tables. The test is adapted accordingly. There is a new test that checks the docstrings to be compliant to the RST format as used by sphinx to create the documentation. We can not use Markdown in docstrings. The docstrings are adapted accordingly. An additional test checks that the documentation can be build with sphinx. This test takes very long is only enabled in full run (aka. ci) mode. Signed-off-by: Bobby Noelte <b0661n0e17e@gmail.com>	2025-11-13 22:53:46 +01:00