From b397b5d43e6d59ccaead983493d5ac9b02b8c8b2 Mon Sep 17 00:00:00 2001 From: Bobby Noelte Date: Tue, 28 Oct 2025 02:50:31 +0100 Subject: [PATCH] fix: automatic optimization (#596) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit This fix implements the long term goal to have the EOS server run optimization (or energy management) on regular intervals automatically. Thus clients can request the current energy management plan at any time and it is updated on regular intervals without interaction by the client. This fix started out to "only" make automatic optimization (or energy management) runs working. It turned out there are several endpoints that in some way update predictions or run the optimization. To lock against such concurrent attempts the code had to be refactored to allow control of execution. During refactoring it became clear that some classes and files are named without a proper reference to their usage. Thus not only refactoring but also renaming became necessary. The names are still not the best, but I hope they are more intuitive. The fix includes several bug fixes that are not directly related to the automatic optimization but are necessary to keep EOS running properly to do the automatic optimization and to test and document the changes. This is a breaking change as the configuration structure changed once again and the server API was also enhanced and streamlined. The server API that is used by Andreas and Jörg in their videos has not changed. * fix: automatic optimization Allow optimization to automatically run on configured intervals gathering all optimization parameters from configuration and predictions. The automatic run can be configured to only run prediction updates skipping the optimization. Extend documentaion to also cover automatic optimization. Lock automatic runs against runs initiated by the /optimize or other endpoints. Provide new endpoints to retrieve the energy management plan and the genetic solution of the latest automatic optimization run. Offload energy management to thread pool executor to keep the app more responsive during the CPU heavy optimization run. * fix: EOS servers recognize environment variables on startup Force initialisation of EOS configuration on server startup to assure all sources of EOS configuration are properly set up and read. Adapt server tests and configuration tests to also test for environment variable configuration. * fix: Remove 0.0.0.0 to localhost translation under Windows EOS imposed a 0.0.0.0 to localhost translation under Windows for convenience. This caused some trouble in user configurations. Now, as the default IP address configuration is 127.0.0.1, the user is responsible for to set up the correct Windows compliant IP address. * fix: allow names for hosts additional to IP addresses * fix: access pydantic model fields by class Access by instance is deprecated. * fix: down sampling key_to_array * fix: make cache clear endpoint clear all cache files Make /v1/admin/cache/clear clear all cache files. Before it only cleared expired cache files by default. Add new endpoint /v1/admin/clear-expired to only clear expired cache files. * fix: timezonefinder returns Europe/Paris instead of Europe/Berlin timezonefinder 8.10 got more inaccurate for timezones in europe as there is a common timezone. Use new package tzfpy instead which is still returning Europe/Berlin if you are in Germany. tzfpy also claims to be faster than timezonefinder. * fix: provider settings configuration Provider configuration used to be a union holding the settings for several providers. Pydantic union handling does not always find the correct type for a provider setting. This led to exceptions in specific configurations. Now provider settings are explicit comfiguration items for each possible provider. This is a breaking change as the configuration structure was changed. * fix: ClearOutside weather prediction irradiance calculation Pvlib needs a pandas time index. Convert time index. * fix: test config file priority Do not use config_eos fixture as this fixture already creates a config file. * fix: optimization sample request documentation Provide all data in documentation of optimization sample request. * fix: gitlint blocking pip dependency resolution Replace gitlint by commitizen. Gitlint is not actively maintained anymore. Gitlint dependencies blocked pip from dependency resolution. * fix: sync pre-commit config to actual dependency requirements .pre-commit-config.yaml was out of sync, also requirements-dev.txt. * fix: missing babel in requirements.txt Add babel to requirements.txt * feat: setup default device configuration for automatic optimization In case the parameters for automatic optimization are not fully defined a default configuration is setup to allow the automatic energy management run. The default configuration may help the user to correctly define the device configuration. * feat: allow configuration of genetic algorithm parameters The genetic algorithm parameters for number of individuals, number of generations, the seed and penalty function parameters are now avaliable as configuration options. * feat: allow configuration of home appliance time windows The time windows a home appliance is allowed to run are now configurable by the configuration (for /v1 API) and also by the home appliance parameters (for the classic /optimize API). If there is no such configuration the time window defaults to optimization hours, which was the standard before the change. Documentation on how to configure time windows is added. * feat: standardize mesaurement keys for battery/ ev SoC measurements The standardized measurement keys to report battery SoC to the device simulations can now be retrieved from the device configuration as a read-only config option. * feat: feed in tariff prediction Add feed in tarif predictions needed for automatic optimization. The feed in tariff can be retrieved as fixed feed in tarif or can be imported. Also add tests for the different feed in tariff providers. Extend documentation to cover the feed in tariff providers. * feat: add energy management plan based on S2 standard instructions EOS can generate an energy management plan as a list of simple instructions. May be retrieved by the /v1/energy-management/plan endpoint. The instructions loosely follow the S2 energy management standard. * feat: make measurement keys configurable by EOS configuration. The fixed measurement keys are replaced by configurable measurement keys. * feat: make pendulum DateTime, Date, Duration types usable for pydantic models Use pydantic_extra_types.pendulum_dt to get pydantic pendulum types. Types are added to the datetimeutil utility. Remove custom made pendulum adaptations from EOS pydantic module. Make EOS modules use the pydantic pendulum types managed by the datetimeutil module instead of the core pendulum types. * feat: Add Time, TimeWindow, TimeWindowSequence and to_time to datetimeutil. The time windows are are added to support home appliance time window configuration. All time classes are also pydantic models. Time is the base class for time definition derived from pendulum.Time. * feat: Extend DataRecord by configurable field like data. Configurable field like data was added to support the configuration of measurement records. * feat: Add additional information to health information Version information is added to the health endpoints of eos and eosDash. The start time of the last optimization and the latest run time of the energy management is added to the EOS health information. * feat: add pydantic merge model tests * feat: add plan tab to EOSdash The plan tab displays the current energy management instructions. * feat: add predictions tab to EOSdash The predictions tab displays the current predictions. * feat: add cache management to EOSdash admin tab The admin tab is extended by a section for cache management. It allows to clear the cache. * feat: add about tab to EOSdash The about tab resembles the former hello tab and provides extra information. * feat: Adapt changelog and prepare for release management Release management using commitizen is added. The changelog file is adapted and teh changelog and a description for release management is added in the documentation. * feat(doc): Improve install and devlopment documentation Provide a more concise installation description in Readme.md and add extra installation page and development page to documentation. * chore: Use memory cache for interpolation instead of dict in inverter Decorate calculate_self_consumption() with @cachemethod_until_update to cache results in memory during an energy management/ optimization run. Replacement of dict type caching in inverter is now possible because all optimization runs are properly locked and the memory cache CacheUntilUpdateStore is properly cleared at the start of any energy management/ optimization operation. * chore: refactor genetic Refactor the genetic algorithm modules for enhanced module structure and better readability. Removed unnecessary and overcomplex devices singleton. Also split devices configuration from genetic algorithm parameters to allow further development independently from genetic algorithm parameter format. Move charge rates configuration for electric vehicles from optimization to devices configuration to allow to have different charge rates for different cars in the future. * chore: Rename memory cache to CacheEnergyManagementStore The name better resembles the task of the cache to chache function and method results for an energy management run. Also the decorator functions are renamed accordingly: cachemethod_energy_management, cache_energy_management * chore: use class properties for config/ems/prediction mixin classes * chore: skip debug logs from mathplotlib Mathplotlib is very noisy in debug mode. * chore: automatically sync bokeh js to bokeh python package bokeh was updated to 3.8.0, make JS CDN automatically follow the package version. * chore: rename hello.py to about.py Make hello.py the adapted EOSdash about page. * chore: remove demo page from EOSdash As no the plan and prediction pages are working without configuration, the demo page is no longer necessary * chore: split test_server.py for system test Split test_server.py to create explicit test_system.py for system tests. * chore: move doc utils to generate_config_md.py The doc utils are only used in scripts/generate_config_md.py. Move it there to attribute for strong cohesion. * chore: improve pydantic merge model documentation * chore: remove pendulum warning from readme * chore: remove GitHub discussions from contributing documentation Github discussions is to be replaced by Akkudoktor.net. * chore(release): bump version to 0.1.0+dev for development * build(deps): bump fastapi[standard] from 0.115.14 to 0.117.1 bump fastapi and make coverage version (for pytest-cov) explicit to avoid pip break. * build(deps): bump uvicorn from 0.36.0 to 0.37.0 BREAKING CHANGE: EOS configuration changed. V1 API changed. - The available_charge_rates_percent configuration is removed from optimization. Use the new charge_rate configuration for the electric vehicle - Optimization configuration parameter hours renamed to horizon_hours - Device configuration now has to provide the number of devices and device properties per device. - Specific prediction provider configuration to be provided by explicit configuration item (no union for all providers). - Measurement keys to be provided as a list. - New feed in tariff providers have to be configured. - /v1/measurement/loadxxx endpoints are removed. Use generic mesaurement endpoints. - /v1/admin/cache/clear now clears all cache files. Use /v1/admin/cache/clear-expired to only clear all expired cache files. Signed-off-by: Bobby Noelte --- .pre-commit-config.yaml | 62 +- CHANGELOG.md | 267 +- CONTRIBUTING.md | 20 +- Dockerfile | 11 +- Makefile | 25 +- README.md | 178 +- docker-compose.yaml | 20 +- docs/_generated/config.md | 1138 +++- docs/_generated/openapi.md | 257 +- docs/akkudoktoreos/configtimewindow.md | 849 +++ docs/akkudoktoreos/configuration.md | 3 +- docs/akkudoktoreos/integration.md | 3 +- docs/akkudoktoreos/logging.md | 6 + docs/akkudoktoreos/measurement.md | 109 +- docs/akkudoktoreos/optimauto.md | 448 ++ .../{optimization.md => optimpost.md} | 102 +- docs/akkudoktoreos/prediction.md | 28 +- docs/akkudoktoreos/resource.md | 258 + docs/akkudoktoreos/serverapi.md | 1 + docs/develop/CHANGELOG.md | 4 + docs/develop/develop.md | 572 ++ docs/develop/getting_started.md | 157 +- docs/develop/install.md | 288 + docs/develop/release.md | 187 + docs/index.md | 17 +- openapi.json | 5053 +++++++++++++---- pyproject.toml | 28 + requirements-dev.txt | 30 +- requirements.txt | 12 +- scripts/convert_lightweight_tags.py | 69 + scripts/cz_check_branch.py | 47 + scripts/cz_check_commit_message.py | 63 + scripts/cz_check_new_commits.py | 70 + scripts/generate_config_md.py | 57 +- single_test_optimization.py | 155 +- single_test_prediction.py | 20 + src/akkudoktoreos/config/config.py | 150 +- src/akkudoktoreos/config/configmigrate.py | 225 + src/akkudoktoreos/core/cache.py | 76 +- src/akkudoktoreos/core/coreabc.py | 87 +- src/akkudoktoreos/core/dataabc.py | 275 +- src/akkudoktoreos/core/emplan.py | 1914 +++++++ src/akkudoktoreos/core/ems.py | 641 +-- src/akkudoktoreos/core/emsettings.py | 14 + src/akkudoktoreos/core/logging.py | 4 + src/akkudoktoreos/core/logsettings.py | 5 - src/akkudoktoreos/core/pydantic.py | 312 +- src/akkudoktoreos/core/version.py | 5 + src/akkudoktoreos/data/default.config.json | 3 + src/akkudoktoreos/devices/devices.py | 405 +- src/akkudoktoreos/devices/devicesabc.py | 266 +- src/akkudoktoreos/devices/generic.py | 79 - src/akkudoktoreos/devices/genetic/__init__.py | 0 .../devices/{ => genetic}/battery.py | 148 +- .../devices/{ => genetic}/heatpump.py | 0 .../devices/genetic/homeappliance.py | 112 + .../devices/{ => genetic}/inverter.py | 94 +- src/akkudoktoreos/devices/settings.py | 24 - src/akkudoktoreos/measurement/measurement.py | 164 +- .../optimization/{ => genetic}/genetic.py | 545 +- .../optimization/genetic/geneticabc.py | 11 + .../optimization/genetic/geneticdevices.py | 127 + .../optimization/genetic/geneticparams.py | 630 ++ .../optimization/genetic/geneticsolution.py | 480 ++ .../optimization/optimization.py | 126 +- .../optimization/optimizationabc.py | 9 +- src/akkudoktoreos/prediction/elecprice.py | 24 +- .../prediction/elecpriceakkudoktor.py | 16 +- .../prediction/elecpriceenergycharts.py | 18 +- .../prediction/elecpriceimport.py | 11 +- src/akkudoktoreos/prediction/feedintariff.py | 61 + .../prediction/feedintariffabc.py | 58 + .../prediction/feedintarifffixed.py | 48 + .../prediction/feedintariffimport.py | 76 + src/akkudoktoreos/prediction/interpolator.py | 19 +- src/akkudoktoreos/prediction/load.py | 31 +- .../prediction/loadakkudoktor.py | 8 +- src/akkudoktoreos/prediction/loadimport.py | 14 +- src/akkudoktoreos/prediction/loadvrm.py | 11 +- src/akkudoktoreos/prediction/prediction.py | 9 + src/akkudoktoreos/prediction/predictionabc.py | 25 +- src/akkudoktoreos/prediction/pvforecast.py | 72 +- .../prediction/pvforecastakkudoktor.py | 8 +- .../prediction/pvforecastimport.py | 10 +- src/akkudoktoreos/prediction/pvforecastvrm.py | 13 +- src/akkudoktoreos/prediction/weather.py | 19 +- .../prediction/weatherbrightsky.py | 11 +- .../prediction/weatherclearoutside.py | 3 +- src/akkudoktoreos/prediction/weatherimport.py | 12 +- .../server/dash/{hello.py => about.py} | 14 +- src/akkudoktoreos/server/dash/admin.py | 96 + src/akkudoktoreos/server/dash/bokeh.py | 68 +- src/akkudoktoreos/server/dash/components.py | 14 +- .../server/dash/data/democonfig.json | 86 - src/akkudoktoreos/server/dash/eosstatus.py | 13 + src/akkudoktoreos/server/dash/footer.py | 6 +- src/akkudoktoreos/server/dash/plan.py | 496 ++ .../server/dash/{demo.py => prediction.py} | 115 +- src/akkudoktoreos/server/eos.py | 747 ++- src/akkudoktoreos/server/eosdash.py | 342 +- src/akkudoktoreos/server/server.py | 83 +- src/akkudoktoreos/utils/datetimeutil.py | 1467 ++++- src/akkudoktoreos/utils/docs.py | 44 - src/akkudoktoreos/utils/stringutil.py | 53 + src/akkudoktoreos/utils/visualize.py | 12 +- tests/conftest.py | 334 +- tests/test_battery.py | 18 +- tests/test_cache.py | 106 +- tests/test_class_optimize.py | 104 - tests/test_config.py | 158 +- tests/test_configmigrate.py | 229 + tests/test_dataabc.py | 289 +- tests/test_datetimeutil.py | 1279 ++++- tests/test_elecpriceakkudoktor.py | 6 +- tests/test_elecpriceenergycharts.py | 6 +- tests/test_elecpriceimport.py | 22 +- tests/test_emplan.py | 235 + tests/test_eosdashserver.py | 43 +- tests/test_feedintarifffixed.py | 62 + tests/test_geneticoptimize.py | 150 + ...class_ems.py => test_geneticsimulation.py} | 79 +- ...ss_ems_2.py => test_geneticsimulation2.py} | 159 +- tests/test_heatpump.py | 2 +- tests/test_inverter.py | 14 +- tests/test_loadakkudoktor.py | 10 +- tests/test_loadvrm.py | 6 +- tests/test_measurement.py | 551 +- tests/test_prediction.py | 26 +- tests/test_predictionabc.py | 6 +- tests/test_pvforecastakkudoktor.py | 4 +- tests/test_pvforecastimport.py | 22 +- tests/test_pvforecastvrm.py | 6 +- tests/test_pydantic.py | 202 +- tests/test_server.py | 289 +- tests/test_stringutil.py | 51 + tests/test_system.py | 210 + tests/test_weatherclearoutside.py | 2 +- tests/test_weatherimport.py | 22 +- tests/testdata/eos_config_andreas_0_1_0.json | 101 + tests/testdata/eos_config_andreas_now.json | 100 + tests/testdata/eos_config_minimal_0_1_0.json | 24 + tests/testdata/eos_config_minimal_now.json | 24 + tests/testdata/eosserver_config_1.json | 6 +- tests/testdata/optimize_input_1.json | 4 +- .../testdata/weatherforecast_brightsky_1.json | 408 +- .../testdata/weatherforecast_brightsky_2.json | 216 +- 146 files changed, 22024 insertions(+), 5339 deletions(-) create mode 100644 docs/akkudoktoreos/configtimewindow.md create mode 100644 docs/akkudoktoreos/optimauto.md rename docs/akkudoktoreos/{optimization.md => optimpost.md} (56%) create mode 100644 docs/akkudoktoreos/resource.md create mode 100644 docs/develop/CHANGELOG.md create mode 100644 docs/develop/develop.md create mode 100644 docs/develop/install.md create mode 100644 docs/develop/release.md create mode 100644 scripts/convert_lightweight_tags.py create mode 100644 scripts/cz_check_branch.py create mode 100644 scripts/cz_check_commit_message.py create mode 100644 scripts/cz_check_new_commits.py create mode 100644 src/akkudoktoreos/config/configmigrate.py create mode 100644 src/akkudoktoreos/core/emplan.py create mode 100644 src/akkudoktoreos/core/version.py delete mode 100644 src/akkudoktoreos/devices/generic.py create mode 100644 src/akkudoktoreos/devices/genetic/__init__.py rename src/akkudoktoreos/devices/{ => genetic}/battery.py (50%) rename src/akkudoktoreos/devices/{ => genetic}/heatpump.py (100%) create mode 100644 src/akkudoktoreos/devices/genetic/homeappliance.py rename src/akkudoktoreos/devices/{ => genetic}/inverter.py (52%) delete mode 100644 src/akkudoktoreos/devices/settings.py rename src/akkudoktoreos/optimization/{ => genetic}/genetic.py (53%) create mode 100644 src/akkudoktoreos/optimization/genetic/geneticabc.py create mode 100644 src/akkudoktoreos/optimization/genetic/geneticdevices.py create mode 100644 src/akkudoktoreos/optimization/genetic/geneticparams.py create mode 100644 src/akkudoktoreos/optimization/genetic/geneticsolution.py create mode 100644 src/akkudoktoreos/prediction/feedintariff.py create mode 100644 src/akkudoktoreos/prediction/feedintariffabc.py create mode 100644 src/akkudoktoreos/prediction/feedintarifffixed.py create mode 100644 src/akkudoktoreos/prediction/feedintariffimport.py rename src/akkudoktoreos/server/dash/{hello.py => about.py} (72%) delete mode 100644 src/akkudoktoreos/server/dash/data/democonfig.json create mode 100644 src/akkudoktoreos/server/dash/eosstatus.py create mode 100644 src/akkudoktoreos/server/dash/plan.py rename src/akkudoktoreos/server/dash/{demo.py => prediction.py} (68%) delete mode 100644 src/akkudoktoreos/utils/docs.py create mode 100644 src/akkudoktoreos/utils/stringutil.py delete mode 100644 tests/test_class_optimize.py create mode 100644 tests/test_configmigrate.py create mode 100644 tests/test_emplan.py create mode 100644 tests/test_feedintarifffixed.py create mode 100644 tests/test_geneticoptimize.py rename tests/{test_class_ems.py => test_geneticsimulation.py} (80%) rename tests/{test_class_ems_2.py => test_geneticsimulation2.py} (62%) create mode 100644 tests/test_stringutil.py create mode 100644 tests/test_system.py create mode 100644 tests/testdata/eos_config_andreas_0_1_0.json create mode 100644 tests/testdata/eos_config_andreas_now.json create mode 100644 tests/testdata/eos_config_minimal_0_1_0.json create mode 100644 tests/testdata/eos_config_minimal_now.json diff --git a/.pre-commit-config.yaml b/.pre-commit-config.yaml index 9d1098a..79569fe 100644 --- a/.pre-commit-config.yaml +++ b/.pre-commit-config.yaml @@ -1,8 +1,9 @@ # Exclude some file types from automatic code style exclude: \.(json|csv)$ repos: + # --- Basic sanity checks --- - repo: https://github.com/pre-commit/pre-commit-hooks - rev: v5.0.0 + rev: v6.0.0 hooks: - id: check-merge-conflict - id: check-toml @@ -10,31 +11,38 @@ repos: - id: end-of-file-fixer - id: trailing-whitespace - id: check-merge-conflict - exclude: '\.rst$' # Exclude .rst files + exclude: '\.rst$' # Exclude .rst files from whitespace cleanup + + # --- Import sorting --- - repo: https://github.com/PyCQA/isort - rev: 6.0.0 + rev: 7.0.0 hooks: - id: isort - name: isort + + # --- Linting + Formatting via Ruff --- - repo: https://github.com/astral-sh/ruff-pre-commit - rev: v0.9.6 + rev: v0.14.1 hooks: - # Run the linter and fix simple issues automatically + # Run the linter and fix simple isssues automatically - id: ruff args: [--fix] - # Run the formatter. + # Run the formatter - id: ruff-format + + # --- Static type checking --- - repo: https://github.com/pre-commit/mirrors-mypy - rev: 'v1.15.0' + rev: v1.18.2 hooks: - id: mypy additional_dependencies: - - "types-requests==2.32.0.20241016" - - "pandas-stubs==2.2.3.241009" - - "numpy==2.1.3" + - types-requests==2.32.4.20250913 + - pandas-stubs==2.3.2.250926 + - tokenize-rt==3.2.0 pass_filenames: false + + # --- Markdown linter --- - repo: https://github.com/jackdewinter/pymarkdown - rev: v0.9.29 + rev: v0.9.32 hooks: - id: pymarkdown files: ^docs/ @@ -42,7 +50,31 @@ repos: args: - --config=docs/pymarkdown.json - scan - - repo: https://github.com/jorisroovers/gitlint - rev: v0.19.1 + + # --- Commit message linting --- + # - Local cross-platform hooks + - repo: local hooks: - - id: gitlint + # Validate commit messages (using Python wrapper) + - id: commitizen-commit + name: Commitizen (venv-aware) + entry: python3 scripts/cz_check_commit_message.py + language: system + stages: [commit-msg] + pass_filenames: false + + # Branch name check on push (using Python wrapper) + - id: commitizen-branch + name: Commitizen branch check + entry: python3 scripts/cz_check_branch.py + language: system + stages: [pre-push] + pass_filenames: false + + # Validate new commit messages before push (using Python wrapper) + - id: commitizen-new-commits + name: Commitizen (check new commits only, .venv aware) + entry: python3 -m scripts.cz_check_new_commits + language: system + stages: [pre-push] + pass_filenames: false diff --git a/CHANGELOG.md b/CHANGELOG.md index b98cfc7..b6af810 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -5,114 +5,189 @@ All notable changes to the akkudoktoreos project will be documented in this file The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/), and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html). -## [0.1.0] - 2025-09-30 +## 0.1.0+dev (2025-10-26) -### Added +### feat +- setup default device configuration for automatic optimization +- allow configuration of genetic algorithm parameters +- allow configuration of home appliance time windows +- mitigate old config +- standardize measurement keys for battery/EV SoC measurements +- feed-in tariff prediction support (incl. tests and docs) +- energy management plan generation based on S2 standard instructions +- make measurement keys configurable through EOS configuration +- use pendulum types with pydantic via pydantic_extra_types.pendulum_dt +- add Time, TimeWindow, TimeWindowSequence and to_time to datetimeutil +- extend DataRecord with configurable field-like data +- enrich health endpoints with version and optimization timestamps +- add pydantic merge model tests +- add plan tab to EOSdash +- add predictions tab to EOSdash +- add cache management to EOSdash admin tab +- add about tab to EOSdash +- adapt changelog & documentation for commitizen release flow +- improve install and development documentation -- Added Changelog for 0.0.0 amd 0.1.0 +### fix +- automatic optimization (interval execution, locking, new endpoints) +- recognize environment variables on EOS server startup +- remove 0.0.0.0 → localhost translation on Windows +- allow hostnames as well as IPs +- access pydantic model fields via class instead of instance +- downsampling in key_to_array +- /v1/admin/cache/clear now clears all cache files; new /clear-expired endpoint +- replace timezonefinder with tzfpy for accurate European timezones +- explicit provider settings in config versus union +- ClearOutside weather prediction irradiance calculation +- test config file priority without config_eos fixture +- complete optimization sample request documentation +- replace gitlint with commitizen +- synchronize pre-commit config with real dependencies +- add missing babel to requirements +- fix documentation, tests, and implementation around optimization and predictions -## [0.0.0] - 2025-09-30 +### chore +- use memory cache for inverter interpolation +- refactor genetic algorithm modules (split config, remove device singleton) +- rename memory cache to CacheEnergyManagementStore +- use class properties for config/ems/prediction mixins +- skip matplotlib debug logs +- auto-sync Bokeh JS CDN version +- rename hello.py to about.py in EOSdash +- remove demo page from EOSdash +- split server test for system test +- move doc utils to generate_config_md.py +- improve documentation for pydantic merge models +- remove pendulum warning from README +- drop GitHub Discussions from contributing docs +- bump version to 0.1.0+dev +- rename or reorganize files/classes during refactoring + +### build +- bump fastapi[standard] 0.115.14 → 0.117.1 and fix pytest-cov version +- bump uvicorn 0.36.0 → 0.37.0 + +### BREAKING CHANGE +EOS configuration and v1 API were changed: + +- available_charge_rates_percent removed; replaced by new charge_rate config +- optimization param hours renamed to horizon_hours +- device config must now list devices and their properties explicitly +- specific prediction provider configuration versus union +- measurement keys provided as lists +- new feed-in tariff providers must be configured +- /v1/measurement/loadxxx endpoints removed (use generic measurement endpoints) +- /v1/admin/cache/clear clears all cache files; use /v1/admin/cache/clear-expired for expired-only clearing + +## v0.1.0 (2025-09-30) + +### Feat + +- added Changelog for 0.0.0 and 0.1.0 + +## v0.0.0 (2025-09-30) This version represents one year of development of EOS (Energy Optimization System). From this point forward, release management will be introduced. -### Added +### Feat #### Core Features -- Energy Management System (EMS) with battery optimization +- energy Management System (EMS) with battery optimization - PV (Photovoltaic) forecast integration with multiple providers -- Load prediction and forecasting capabilities -- Electricity price integration +- load prediction and forecasting capabilities +- electricity price integration - VRM API integration for load and PV forecasting -- Battery State of Charge (SoC) prediction and optimization -- Inverter class with AC/DC charging logic -- Electric vehicle (EV) charging optimization with configurable currents -- Home appliance scheduling optimization -- Horizon validation for shading calculations +- battery State of Charge (SoC) prediction and optimization +- inverter class with AC/DC charging logic +- electric vehicle (EV) charging optimization with configurable currents +- home appliance scheduling optimization +- horizon validation for shading calculations #### API & Server -- Migration from Flask to FastAPI +- migration from Flask to FastAPI - RESTful API with comprehensive endpoints - EOSdash web interface for configuration and visualization - Docker support with multi-architecture builds -- Web-based visualization with interactive charts +- web-based visualization with interactive charts - OpenAPI/Swagger documentation -- Configurable server settings (port, host) +- configurable server settings (port, host) #### Configuration & Data Management - JSON-based configuration system with nested support -- Configuration validation with Pydantic -- Device registry for managing multiple devices -- Persistent caching for predictions and prices -- Manual prediction updates -- Timezone support with automatic detection -- Configurable VAT rates for electricity prices +- configuration validation with Pydantic +- device registry for managing multiple devices +- persistent caching for predictions and prices +- manual prediction updates +- timezone support with automatic detection +- configurable VAT rates for electricity prices #### Optimization - DEAP-based genetic algorithm optimization -- Multi-objective optimization (cost, battery usage, self-consumption) +- multi-objective optimization (cost, battery usage, self-consumption) - 48-hour prediction and optimization window - AC/DC charging decision optimization -- Discharge hour optimization -- Start solution enforcement -- Fitness visualization with violin plots -- Self-consumption probability interpolator +- discharge hour optimization +- start solution enforcement +- fitness visualization with violin plots +- self-consumption probability interpolator #### Testing & Quality -- Comprehensive test suite with pytest -- Unit tests for major components (EMS, battery, inverter, load, optimization) -- Integration tests for server endpoints -- Pre-commit hooks for code quality -- Type checking with mypy -- Code formatting with ruff and isort -- Markdown linting +- comprehensive test suite with pytest +- unit tests for major components (EMS, battery, inverter, load, optimization) +- integration tests for server endpoints +- pre-commit hooks for code quality +- type checking with mypy +- code formatting with ruff and isort +- markdown linting #### Documentation -- Conceptual documentation +- conceptual documentation - API documentation with Sphinx - ReadTheDocs integration - Docker setup instructions -- Contributing guidelines +- contributing guidelines - English README translation #### Providers & Integrations - PVForecast.Akkudoktor provider - BrightSky weather provider - ClearOutside weather provider -- Electricity price provider +- electricity price provider -### Changed -- Python version requirement updated to 3.10+ -- Optimized Inverter class for improved SCR calculation performance -- Improved caching mechanisms for better performance -- Enhanced visualization with proper timestamp handling -- Updated dependency management with automatic Dependabot updates -- Restructured code into logical submodules -- Package directory structure reorganization -- Improved error handling and logging +### Refactor + +- optimized Inverter class for improved SCR calculation performance +- improved caching mechanisms for better performance +- enhanced visualization with proper timestamp handling +- updated dependency management with automatic Dependabot updates +- restructured code into logical submodules +- package directory structure reorganization +- improved error handling and logging - Windows compatibility improvements -### Fixed -- Cross-site scripting (XSS) vulnerabilities +### Fix + +- cross-site scripting (XSS) vulnerabilities - ReDoS vulnerability in duration parsing -- Timezone and daylight saving time handling +- timezone and daylight saving time handling - BrightSky provider with None humidity data -- Negative values in load mean adjusted calculations +- negative values in load mean adjusted calculations - SoC calculation bugs - AC charge efficiency in price calculations -- Optimization timing bugs +- optimization timing bugs - Docker BuildKit compatibility -- Float value handling in user horizon configuration -- Circular runtime import issues -- Load simulation data return issues -- Multiple optimization-related bugs +- float value handling in user horizon configuration +- circular runtime import issues +- load simulation data return issues +- multiple optimization-related bugs -### Security -- Added Bandit security checks -- Fixed XSS vulnerabilities -- Mitigated ReDoS attacks with input length validation -- Improved credential management with environment variables +### Build -### Dependencies +- Python version requirement updated to 3.10+ +- added Bandit security checks +- improved credential management with environment variables + +#### Dependencies Major dependencies included in this release: - FastAPI 0.115.14 - Pydantic 2.11.9 @@ -125,69 +200,15 @@ Major dependencies included in this release: - PVLib 0.13.1 - Python-FastHTML 0.12.29 -### Development Notes +### Notes + +#### Development Notes This version encompasses all development from the initial commit (February 16, 2024) through September 29, 2025. The project evolved from a basic energy optimization concept to a comprehensive energy management system with: - 698+ commits -- Multiple contributor involvement -- Continuous integration/deployment setup -- Automated dependency updates -- Comprehensive testing infrastructure +- multiple contributor involvement +- continuous integration/deployment setup +- automated dependency updates +- comprehensive testing infrastructure -### Migration Notes +#### Migration Notes As this is the initial versioned release, no migration is required. Future releases will include migration guides as needed. - ---- - -**Full Changelog**: Initial development phase (v0.0.0) - - -## v0.1.0-a0 (2025-09-30) - -### BREAKING CHANGE - -- This is a BREAKING CHANGE as the configuration structure changed -once again and the server API was also enhanced and streamlined. The server API -that is used by Andreas and Jörg in their videos has not changed -- This is a BREAKING CHANGE as the configuration structure changed -once again and the server API was also enhanced and streamlined. The server API -that is used by Andreas and Jörg in their videos has not changed. -- EOS configuration changed. V1 API changed. -- Default IP address for EOS and EOSdash changed to 127.0.0.1 -- Azimuth configurations that followed the PVForecastAkkudoktor convention -(north=+-180, east=-90, south=0, west=90) must be converted to the general azimuth definition: -north=0, east=90, south=180, west=270. - -### Feat - -- **VRM forecast**: add load and pv forecast by VRM API (#611) -- run pytest for PRs -- be helpful, provide a list of valid routes when visiting / -- add documentation, enable makefile driven usage -- Detailliertere README -- andere ports/bind ips erlauben - -### Fix - -- dependencies and optimization solution beginning -- typos in bokeh.py -- automatic optimization -- handle float values in userhorizon configuration (#657) -- **docker**: make EOSDash accessible in Docker containers (#656) -- **ElecPriceEnergyCharts**: get history series, update docs (#606) -- logging, prediction update, multiple bugs (#584) -- add required fields to example optimization request (#574) -- pvforecast fails when there is only a single plane (#569) -- delete empty inverter from testdata optimize_input_2.json (#568) -- azimuth setting of pvforecastakkudoktor provider (#567) -- BrightSky with None humidity data (#555) -- Catch optimize error and return error message. (#534) -- Circular runtime import Closes #533 (#535) -- **docker**: enable BuildKit to support --mount (closes #493) -- mitigate ReDoS in to_duration via max input length check (closes #494) (#523) -- relax stale issue/pr handling -- remove verbose comment -- make port configurable via env - -### Refactor - -- remove `README-DE.md` diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index ecc78ec..bb96f99 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -14,9 +14,10 @@ Please report flaws or vulnerabilities in the [GitHub Issue Tracker](https://git ## Ideas & Features -Please first discuss the idea in a [GitHub Discussion](https://github.com/Akkudoktor-EOS/EOS/discussions) or the [Akkudoktor Forum](https://www.akkudoktor.net/forum/diy-energie-optimierungssystem-opensource-projekt/) before opening an issue. +Issues in the [GitHub Issue Tracker](https://github.com/Akkudoktor-EOS/EOS/issues) are also fine +to discuss ideas and features. -There are just too many possibilities and the project would drown in tickets otherwise. +You may first discuss the idea in the [Akkudoktor Forum](https://www.akkudoktor.net/forum/diy-energie-optimierungssystem-opensource-projekt/) before opening an issue. ## Code Contributions @@ -27,7 +28,6 @@ message style checks. ### Setup development environment Setup virtual environment, then activate virtual environment and install development dependencies. -See also [README.md](README.md). ```bash python -m venv .venv @@ -60,7 +60,7 @@ To run formatting automatically before every commit: ```bash pre-commit install -pre-commit install --hook-type commit-msg +pre-commit install --hook-type commit-msg --hook-type pre-push ``` Or run them manually: @@ -79,5 +79,15 @@ python -m pytest -vs --cov src --cov-report term-missing tests/ ### Commit message style -Our commit message checks use [`gitlint`](https://github.com/jorisroovers/gitlint). The checks +Our commit message checks use +[`commitizen`](https://commitizen-tools.github.io/commitizen/#pre-commit-integration). The checks enforce the [`Conventional Commits`](https://www.conventionalcommits.org) commit message style. + +You may use [`commitizen`](https://commitizen-tools.github.io/commitizen) also to create a +commit message and commit your change. + +## Thank you! + +And last but not least thanks to all our contributors + +[![Contributors](https://contrib.rocks/image?repo=Akkudoktor-EOS/EOS)](https://github.com/Akkudoktor-EOS/EOS/graphs/contributors) diff --git a/Dockerfile b/Dockerfile index 791b73c..f442517 100644 --- a/Dockerfile +++ b/Dockerfile @@ -13,6 +13,13 @@ ENV EOS_CONFIG_DIR="${EOS_DIR}/config" # Overwrite when starting the container in a production environment ENV EOS_SERVER__EOSDASH_SESSKEY=s3cr3t +# Set environment variables to reduce threading needs +ENV OPENBLAS_NUM_THREADS=1 +ENV OMP_NUM_THREADS=1 +ENV MKL_NUM_THREADS=1 +ENV PIP_PROGRESS_BAR=off +ENV PIP_NO_COLOR=1 + WORKDIR ${EOS_DIR} RUN adduser --system --group --no-create-home eos \ @@ -28,10 +35,10 @@ RUN adduser --system --group --no-create-home eos \ COPY requirements.txt . RUN --mount=type=cache,target=/root/.cache/pip \ - pip install -r requirements.txt + pip install --no-cache-dir -r requirements.txt COPY pyproject.toml . -RUN mkdir -p src && pip install -e . +RUN mkdir -p src && pip install --no-cache-dir -e . COPY src src diff --git a/Makefile b/Makefile index 4529c96..fae24a2 100644 --- a/Makefile +++ b/Makefile @@ -1,5 +1,5 @@ # Define the targets -.PHONY: help venv pip install dist test test-full docker-run docker-build docs read-docs clean format gitlint mypy run run-dev +.PHONY: help venv pip install dist test test-full test-system test-ci docker-run docker-build docs read-docs clean format gitlint mypy run run-dev run-dash run-dash-dev bumps # Default target all: help @@ -26,9 +26,11 @@ help: @echo " run-dash-dev - Run EOSdash development server in virtual environment (automatically reloads)." @echo " test - Run tests." @echo " test-full - Run tests with full optimization." + @echo " test-system - Run tests with system tests enabled." @echo " test-ci - Run tests as CI does. No user config file allowed." @echo " dist - Create distribution (in dist/)." @echo " clean - Remove generated documentation, distribution and virtual environment." + @echo " bump - Bump version to next release version." # Target to set up a Python 3 virtual environment venv: @@ -47,7 +49,7 @@ pip-dev: pip @echo "Dependencies installed from requirements-dev.txt." # Target to install EOS in editable form (development mode) into virtual environment. -install: pip +install: pip-dev .venv/bin/pip install build .venv/bin/pip install -e . @echo "EOS installed in editable form (development mode)." @@ -100,7 +102,7 @@ run: run-dev: @echo "Starting EOS development server, please wait..." - .venv/bin/python -m akkudoktoreos.server.eos --host localhost --port 8503 --reload true + .venv/bin/python -m akkudoktoreos.server.eos --host localhost --port 8503 --log_level DEBUG --startup_eosdash false --reload true run-dash: @echo "Starting EOSdash production server, please wait..." @@ -108,7 +110,7 @@ run-dash: run-dash-dev: @echo "Starting EOSdash development server, please wait..." - .venv/bin/python -m akkudoktoreos.server.eosdash --host localhost --port 8504 --reload true + .venv/bin/python -m akkudoktoreos.server.eosdash --host localhost --port 8504 --log_level DEBUG --reload true # Target to setup tests. test-setup: pip-dev @@ -124,6 +126,11 @@ test-ci: @echo "Running tests as CI..." .venv/bin/pytest --full-run --check-config-side-effect -vs --cov src --cov-report term-missing +# Target to run tests including the system tests. +test-system: + @echo "Running tests incl. system tests..." + .venv/bin/pytest --system-test -vs --cov src --cov-report term-missing + # Target to run all tests. test-full: @echo "Running all tests..." @@ -133,9 +140,9 @@ test-full: format: .venv/bin/pre-commit run --all-files -# Target to trigger gitlint using pre-commit for the last commit message +# Target to trigger gitlint using pre-commit for the latest commit messages gitlint: - .venv/bin/pre-commit run gitlint --hook-stage commit-msg --commit-msg-filename .git/COMMIT_EDITMSG + .venv/bin/cz check --rev-range main..HEAD # Target to format code. mypy: @@ -147,3 +154,9 @@ docker-run: docker-build: @docker compose build --pull + +# Bump Akkudoktoreos version +bump: pip-dev + @echo "Bumping akkudoktoreos version to release version" + .venv/bin/python scripts/convert_lightweight_tags.py + .venv/bin/cz bump diff --git a/README.md b/README.md index a004bd8..317e13c 100644 --- a/README.md +++ b/README.md @@ -1,119 +1,157 @@ -# Energy System Simulation and Optimization +![AkkudoktorEOS](docs/_static/logo.png) -This project provides a comprehensive solution for simulating and optimizing an energy system based on renewable energy sources. With a focus on photovoltaic (PV) systems, battery storage (batteries), load management (consumer requirements), heat pumps, electric vehicles, and consideration of electricity price data, this system enables forecasting and optimization of energy flow and costs over a specified period. +**Build optimized energy management plans for your home automation** -Documentation can be found at [Akkudoktor-EOS](https://akkudoktor-eos.readthedocs.io/en/latest/). +AkkudoktorEOS is a comprehensive solution for simulating and optimizing energy systems based on +renewable sources. Optimize your photovoltaic systems, battery storage, load management, and +electric vehicles while considering real-time electricity pricing. -## Getting Involved +## Why use AkkudoktorEOS? -See [CONTRIBUTING.md](CONTRIBUTING.md). +AkkudoktorEOS can be used to build energy management plans that are optimized for your specific +setup of PV system, battery, electric vehicle, household load and electricity pricing. It can +be integrated into home automation systems such as NodeRED, Home Assistant, EVCC. -## System requirements +## 🏘️ Community -- Python >= 3.11, < 3.13 -- Architecture: amd64, aarch64 (armv8) -- OS: Linux, Windows, macOS +We are an open-source community-driven project and we love to hear from you. Here are some ways to +get involved: -Note: For Python 3.13 some dependencies (e.g. [Pendulum](https://github.com/python-pendulum/Pendulum)) are not yet available on https://pypi.org and have to be manually compiled (a recent [Rust](https://www.rust-lang.org/tools/install) installation is required). +- [GitHub Issue Tracker](https://github.com/Akkudoktor-EOS/EOS/issues): discuss ideas and features, +and report bugs. -Other architectures (e.g. armv6, armv7) are unsupported for now, because a multitude of dependencies are not available on https://piwheels.org and have to be built manually (a recent Rust installation and [GCC](https://gcc.gnu.org/) are required, Python 3.11 is recommended). +- [Akkudoktor Forum](https://www.akkudoktor.net/forum/diy-energie-optimierungssystem-opensource-projekt/): +get direct suppport from the cummunity. + +## What do people build with AkkudoktorEOS + +The community uses AkkudoktorEOS to minimize grid energy consumption and to maximize the revenue +from grid energy feed in with their home automation system. + +- Andreas Schmitz, [the Akkudoktor](https://www.youtube.com/@Akkudoktor), uses + EOS integrated in his NodeRED home automation system for + [OpenSource Energieoptimierung](https://www.youtube.com/watch?v=sHtv0JCxAYk). +- Jörg, [meintechblog](https://www.youtube.com/@meintechblog), uses EOS for + day-ahead optimization for time-variable energy prices. See: + [So installiere ich EOS von Andreas Schmitz](https://www.youtube.com/watch?v=9XCPNU9UqSs) + +## Why not use AkkudoktorEOS? + +AkkudoktorEOS does not control your home automation assets. It must be integrated into a home +automation system. If you do not use a home automation system or you feel uncomfortable with +the configuration effort needed for the integration you should better use other solutions. + +## Quick Start + +Run EOS with Docker (access dashboard at `http://localhost:8504`): + +```bash +docker run -d \ + --name akkudoktoreos \ + -p 8503:8503 \ + -p 8504:8504 \ + -e OPENBLAS_NUM_THREADS=1 \ + -e OMP_NUM_THREADS=1 \ + -e MKL_NUM_THREADS=1 \ + -e EOS_SERVER__HOST=0.0.0.0 \ + -e EOS_SERVER__EOSDASH_HOST=0.0.0.0 \ + -e EOS_SERVER__EOSDASH_PORT=8504 \ + --ulimit nproc=65535:65535 \ + --ulimit nofile=65535:65535 \ + --security-opt seccomp=unconfined \ + akkudoktor/eos:latest +``` + +## System Requirements + +- **Python**: 3.11 or higher +- **Architecture**: amd64, aarch64 (armv8) +- **OS**: Linux, Windows, macOS + +> **Note**: Other architectures (armv6, armv7) require manual compilation of dependencies with Rust and GCC. ## Installation -The project requires Python 3.11 or newer. Docker images (amd64/aarch64) can be found at [akkudoktor/eos](https://hub.docker.com/r/akkudoktor/eos). +### Docker (Recommended) -Following sections describe how to locally start the EOS server on `http://localhost:8503`. +```bash +docker pull akkudoktor/eos:latest +docker compose up -d +``` -### Run from source +Access the API at `http://localhost:8503` (docs at `http://localhost:8503/docs`) -Install dependencies in virtual environment: +### From Source -Linux: +```bash +git clone https://github.com/Akkudoktor-EOS/EOS.git +cd EOS +``` + +**Linux:** ```bash python -m venv .venv .venv/bin/pip install -r requirements.txt .venv/bin/pip install -e . +.venv/bin/python -m akkudoktoreos.server.eos ``` -Windows: +**Windows:** ```cmd python -m venv .venv -.venv\Scripts\Activate .venv\Scripts\pip install -r requirements.txt .venv\Scripts\pip install -e . +.venv\Scripts\python -m akkudoktoreos.server.eos ``` -Finally, start the EOS server to access it at `http://localhost:8503` (API docs at `http://localhost:8503/docs`): - -Linux: - -```bash -.venv/bin/python src/akkudoktoreos/server/eos.py -``` - -Windows: - -```cmd -.venv\Scripts\python src/akkudoktoreos/server/eos.py -``` - -### Docker - -Start EOS with following command to access it at `http://localhost:8503` (API docs at `http://localhost:8503/docs`): - -```bash -docker compose up -``` - -If you are running the EOS container on a system hosting multiple services, such as a Synology NAS, and want to allow external network access to EOS, please ensure that the default exported ports (8503, 8504) are available on the host. On Synology systems, these ports might already be in use (refer to [this guide](https://kb.synology.com/en-me/DSM/tutorial/What_network_ports_are_used_by_Synology_services)). If the ports are occupied, you will need to reconfigure the exported ports accordingly. - ## Configuration -This project uses the `EOS.config.json` file to manage configuration settings. +EOS uses `EOS.config.json` for configuration. If the file doesn't exist, a default configuration is +created automatically. -### Default Configuration +### Custom Configuration Directory -A default configuration file `default.config.json` is provided. This file contains all the necessary configuration keys with their default values. +```bash +export EOS_DIR=/path/to/your/config +``` -### Custom Configuration +### Configuration Methods -Users can specify a custom configuration directory by setting the environment variable `EOS_DIR`. +1. **EOSdash** (Recommended) - Web interface at `http://localhost:8504` +2. **Manual** - Edit `EOS.config.json` directly +3. **API** - Use the [Server API](https://petstore3.swagger.io/?url=https://raw.githubusercontent.com/Akkudoktor-EOS/EOS/refs/heads/main/openapi.json) -- If the directory specified by `EOS_DIR` contains an existing `config.json` file, the application will use this configuration file. -- If the `EOS.config.json` file does not exist in the specified directory, the `default.config.json` file will be copied to the directory as `EOS.config.json`. +See the [documentation](https://akkudoktor-eos.readthedocs.io/) for all configuration options. -### Configuration Updates +## Port Configuration -If the configuration keys in the `EOS.config.json` file are missing or different from those in `default.config.json`, they will be automatically updated to match the default settings, ensuring that all required keys are present. +**Default ports**: 8503 (API), 8504 (Dashboard) -## Classes and Functionalities +If running on shared systems (e.g., Synology NAS), these ports may conflict with system services. Reconfigure port mappings as needed: -This project uses various classes to simulate and optimize the components of an energy system. Each class represents a specific aspect of the system, as described below: +```bash +docker run -p 8505:8503 -p 8506:8504 ... +``` -- `Battery`: Simulates a battery storage system, including capacity, state of charge, and now charge and discharge losses. +## API Documentation -- `PVForecast`: Provides forecast data for photovoltaic generation, based on weather data and historical generation data. +Interactive API docs available at: +- Swagger UI: `http://localhost:8503/docs` +- OpenAPI Spec: [View Online](https://petstore3.swagger.io/?url=https://raw.githubusercontent.com/Akkudoktor-EOS/EOS/refs/heads/main/openapi.json) -- `Load`: Models the load requirements of a household or business, enabling the prediction of future energy demand. +## Resources -- `Heatpump`: Simulates a heat pump, including its energy consumption and efficiency under various operating conditions. +- [Full Documentation](https://akkudoktor-eos.readthedocs.io/) +- [Installation Guide (German)](https://www.youtube.com/watch?v=9XCPNU9UqSs) -- `Strompreis`: Provides information on electricity prices, enabling optimization of energy consumption and generation based on tariff information. +## Contributing -- `EMS`: The Energy Management System (EMS) coordinates the interaction between the various components, performs optimization, and simulates the operation of the entire energy system. +We welcome contributions! See [CONTRIBUTING](CONTRIBUTING.md) for guidelines. -These classes work together to enable a detailed simulation and optimization of the energy system. For each class, specific parameters and settings can be adjusted to test different scenarios and strategies. +[![Contributors](https://contrib.rocks/image?repo=Akkudoktor-EOS/EOS)](https://github.com/Akkudoktor-EOS/EOS/graphs/contributors) -### Customization and Extension +## License -Each class is designed to be easily customized and extended to integrate additional functions or improvements. For example, new methods can be added for more accurate modeling of PV system or battery behavior. Developers are invited to modify and extend the system according to their needs. - -## Server API - -See the Swagger API documentation for detailed information: [EOS OpenAPI Spec](https://petstore3.swagger.io/?url=https://raw.githubusercontent.com/Akkudoktor-EOS/EOS/refs/heads/main/openapi.json) - -## Further resources - -- [Installation guide (de)](https://meintechblog.de/2024/09/05/andreas-schmitz-joerg-installiert-mein-energieoptimierungssystem/) +This project is licensed under the Apache License 2.0 - see the LICENSE file for details. diff --git a/docker-compose.yaml b/docker-compose.yaml index bdfd0c0..ca7d3ff 100644 --- a/docker-compose.yaml +++ b/docker-compose.yaml @@ -5,6 +5,7 @@ networks: services: eos: image: "akkudoktor/eos:${EOS_VERSION}" + container_name: "akkudoktoreos" read_only: true build: context: . @@ -14,12 +15,23 @@ services: env_file: - .env environment: + - OPENBLAS_NUM_THREADS=1 + - OMP_NUM_THREADS=1 + - MKL_NUM_THREADS=1 + - PIP_PROGRESS_BAR=off + - PIP_NO_COLOR=1 - EOS_CONFIG_DIR=config - EOS_SERVER__EOSDASH_SESSKEY=s3cr3t - - EOS_PREDICTION__LATITUDE=52.2 - - EOS_PREDICTION__LONGITUDE=13.4 - - EOS_ELECPRICE__PROVIDER=ElecPriceAkkudoktor - - EOS_ELECPRICE__CHARGES_KWH=0.21 + - EOS_SERVER__HOST=0.0.0.0 + - EOS_SERVER__PORT=8503 + - EOS_SERVER__EOSDASH_HOST=0.0.0.0 + - EOS_SERVER__EOSDASH_PORT=8504 + ulimits: + nproc: 65535 + nofile: 65535 + security_opt: + - seccomp:unconfined + restart: unless-stopped ports: # Configure what ports to expose on host - "${EOS_SERVER__PORT}:8503" diff --git a/docs/_generated/config.md b/docs/_generated/config.md index f76d213..f54edcd 100644 --- a/docs/_generated/config.md +++ b/docs/_generated/config.md @@ -21,6 +21,7 @@ Properties: | Name | Environment Variable | Type | Read-Only | Default | Description | | ---- | -------------------- | ---- | --------- | ------- | ----------- | +| version | `EOS_GENERAL__VERSION` | `str` | `rw` | `0.1.0+dev` | Configuration file version. Used to check compatibility. | | data_folder_path | `EOS_GENERAL__DATA_FOLDER_PATH` | `Optional[pathlib.Path]` | `rw` | `None` | Path to EOS data directory. | | data_output_subpath | `EOS_GENERAL__DATA_OUTPUT_SUBPATH` | `Optional[pathlib.Path]` | `rw` | `output` | Sub-path for the EOS output data directory. | | latitude | `EOS_GENERAL__LATITUDE` | `Optional[float]` | `rw` | `52.52` | Latitude in decimal degrees, between -90 and 90, north is positive (ISO 19115) (°) | @@ -38,6 +39,7 @@ Properties: { "general": { + "version": "0.1.0+dev", "data_folder_path": null, "data_output_subpath": "output", "latitude": 52.52, @@ -53,6 +55,7 @@ Properties: { "general": { + "version": "0.1.0+dev", "data_folder_path": null, "data_output_subpath": "output", "latitude": 52.52, @@ -100,6 +103,7 @@ Properties: | ---- | -------------------- | ---- | --------- | ------- | ----------- | | startup_delay | `EOS_EMS__STARTUP_DELAY` | `float` | `rw` | `5` | Startup delay in seconds for EOS energy management runs. | | interval | `EOS_EMS__INTERVAL` | `Optional[float]` | `rw` | `None` | Intervall in seconds between EOS energy management runs. | +| mode | `EOS_EMS__MODE` | `Optional[akkudoktoreos.core.emsettings.EnergyManagementMode]` | `rw` | `None` | Energy management mode [OPTIMIZATION | PREDICTION]. | ::: ### Example Input/Output @@ -110,7 +114,8 @@ Properties: { "ems": { "startup_delay": 5.0, - "interval": 300.0 + "interval": 300.0, + "mode": "OPTIMIZATION" } } ``` @@ -123,7 +128,6 @@ Properties: | Name | Environment Variable | Type | Read-Only | Default | Description | | ---- | -------------------- | ---- | --------- | ------- | ----------- | -| level | `EOS_LOGGING__LEVEL` | `Optional[str]` | `rw` | `None` | This is deprecated. Use console_level and file_level instead. | | console_level | `EOS_LOGGING__CONSOLE_LEVEL` | `Optional[str]` | `rw` | `None` | Logging level when logging to console. | | file_level | `EOS_LOGGING__FILE_LEVEL` | `Optional[str]` | `rw` | `None` | Logging level when logging to file. | | file_path | | `Optional[pathlib.Path]` | `ro` | `N/A` | Computed log file path based on data output path. | @@ -136,7 +140,6 @@ Properties: { "logging": { - "level": null, "console_level": "TRACE", "file_level": "TRACE" } @@ -150,7 +153,6 @@ Properties: { "logging": { - "level": null, "console_level": "TRACE", "file_level": "TRACE", "file_path": "/home/user/.local/share/net.akkudoktoreos.net/output/eos.log" @@ -166,12 +168,18 @@ Properties: | Name | Environment Variable | Type | Read-Only | Default | Description | | ---- | -------------------- | ---- | --------- | ------- | ----------- | -| batteries | `EOS_DEVICES__BATTERIES` | `Optional[list[akkudoktoreos.devices.battery.BaseBatteryParameters]]` | `rw` | `None` | List of battery/ev devices | -| inverters | `EOS_DEVICES__INVERTERS` | `Optional[list[akkudoktoreos.devices.inverter.InverterParameters]]` | `rw` | `None` | List of inverters | -| home_appliances | `EOS_DEVICES__HOME_APPLIANCES` | `Optional[list[akkudoktoreos.devices.generic.HomeApplianceParameters]]` | `rw` | `None` | List of home appliances | +| batteries | `EOS_DEVICES__BATTERIES` | `Optional[list[akkudoktoreos.devices.devices.BatteriesCommonSettings]]` | `rw` | `None` | List of battery devices | +| max_batteries | `EOS_DEVICES__MAX_BATTERIES` | `Optional[int]` | `rw` | `None` | Maximum number of batteries that can be set | +| electric_vehicles | `EOS_DEVICES__ELECTRIC_VEHICLES` | `Optional[list[akkudoktoreos.devices.devices.BatteriesCommonSettings]]` | `rw` | `None` | List of electric vehicle devices | +| max_electric_vehicles | `EOS_DEVICES__MAX_ELECTRIC_VEHICLES` | `Optional[int]` | `rw` | `None` | Maximum number of electric vehicles that can be set | +| inverters | `EOS_DEVICES__INVERTERS` | `Optional[list[akkudoktoreos.devices.devices.InverterCommonSettings]]` | `rw` | `None` | List of inverters | +| max_inverters | `EOS_DEVICES__MAX_INVERTERS` | `Optional[int]` | `rw` | `None` | Maximum number of inverters that can be set | +| home_appliances | `EOS_DEVICES__HOME_APPLIANCES` | `Optional[list[akkudoktoreos.devices.devices.HomeApplianceCommonSettings]]` | `rw` | `None` | List of home appliances | +| max_home_appliances | `EOS_DEVICES__MAX_HOME_APPLIANCES` | `Optional[int]` | `rw` | `None` | Maximum number of home_appliances that can be set | +| measurement_keys | | `Optional[list[str]]` | `ro` | `N/A` | Return the measurement keys for the resource/ device stati that are measurements. | ::: -### Example Input/Output +### Example Input ```{eval-rst} .. code-block:: json @@ -181,23 +189,147 @@ Properties: "batteries": [ { "device_id": "battery1", - "hours": null, "capacity_wh": 8000, "charging_efficiency": 0.88, "discharging_efficiency": 0.88, + "levelized_cost_of_storage_kwh": 0.0, "max_charge_power_w": 5000, - "initial_soc_percentage": 0, + "min_charge_power_w": 50, + "charge_rates": null, "min_soc_percentage": 0, - "max_soc_percentage": 100 + "max_soc_percentage": 100, + "measurement_key_soc_factor": "battery1-soc-factor", + "measurement_key_power_l1_w": "battery1-power-l1-w", + "measurement_key_power_l2_w": "battery1-power-l2-w", + "measurement_key_power_l3_w": "battery1-power-l3-w", + "measurement_key_power_3_phase_sym_w": "battery1-power-3-phase-sym-w", + "measurement_keys": [ + "battery1-soc-factor", + "battery1-power-l1-w", + "battery1-power-l2-w", + "battery1-power-l3-w", + "battery1-power-3-phase-sym-w" + ] } ], + "max_batteries": 1, + "electric_vehicles": [ + { + "device_id": "battery1", + "capacity_wh": 8000, + "charging_efficiency": 0.88, + "discharging_efficiency": 0.88, + "levelized_cost_of_storage_kwh": 0.0, + "max_charge_power_w": 5000, + "min_charge_power_w": 50, + "charge_rates": null, + "min_soc_percentage": 0, + "max_soc_percentage": 100, + "measurement_key_soc_factor": "battery1-soc-factor", + "measurement_key_power_l1_w": "battery1-power-l1-w", + "measurement_key_power_l2_w": "battery1-power-l2-w", + "measurement_key_power_l3_w": "battery1-power-l3-w", + "measurement_key_power_3_phase_sym_w": "battery1-power-3-phase-sym-w", + "measurement_keys": [ + "battery1-soc-factor", + "battery1-power-l1-w", + "battery1-power-l2-w", + "battery1-power-l3-w", + "battery1-power-3-phase-sym-w" + ] + } + ], + "max_electric_vehicles": 1, "inverters": [], - "home_appliances": [] + "max_inverters": 1, + "home_appliances": [], + "max_home_appliances": 1 } } ``` -### Home Appliance Device Simulation Configuration +### Example Output + +```{eval-rst} +.. code-block:: json + + { + "devices": { + "batteries": [ + { + "device_id": "battery1", + "capacity_wh": 8000, + "charging_efficiency": 0.88, + "discharging_efficiency": 0.88, + "levelized_cost_of_storage_kwh": 0.0, + "max_charge_power_w": 5000, + "min_charge_power_w": 50, + "charge_rates": null, + "min_soc_percentage": 0, + "max_soc_percentage": 100, + "measurement_key_soc_factor": "battery1-soc-factor", + "measurement_key_power_l1_w": "battery1-power-l1-w", + "measurement_key_power_l2_w": "battery1-power-l2-w", + "measurement_key_power_l3_w": "battery1-power-l3-w", + "measurement_key_power_3_phase_sym_w": "battery1-power-3-phase-sym-w", + "measurement_keys": [ + "battery1-soc-factor", + "battery1-power-l1-w", + "battery1-power-l2-w", + "battery1-power-l3-w", + "battery1-power-3-phase-sym-w" + ] + } + ], + "max_batteries": 1, + "electric_vehicles": [ + { + "device_id": "battery1", + "capacity_wh": 8000, + "charging_efficiency": 0.88, + "discharging_efficiency": 0.88, + "levelized_cost_of_storage_kwh": 0.0, + "max_charge_power_w": 5000, + "min_charge_power_w": 50, + "charge_rates": null, + "min_soc_percentage": 0, + "max_soc_percentage": 100, + "measurement_key_soc_factor": "battery1-soc-factor", + "measurement_key_power_l1_w": "battery1-power-l1-w", + "measurement_key_power_l2_w": "battery1-power-l2-w", + "measurement_key_power_l3_w": "battery1-power-l3-w", + "measurement_key_power_3_phase_sym_w": "battery1-power-3-phase-sym-w", + "measurement_keys": [ + "battery1-soc-factor", + "battery1-power-l1-w", + "battery1-power-l2-w", + "battery1-power-l3-w", + "battery1-power-3-phase-sym-w" + ] + } + ], + "max_electric_vehicles": 1, + "inverters": [], + "max_inverters": 1, + "home_appliances": [], + "max_home_appliances": 1, + "measurement_keys": [ + "battery1-soc-factor", + "battery1-power-l1-w", + "battery1-power-l2-w", + "battery1-power-l3-w", + "battery1-power-3-phase-sym-w", + "battery1-soc-factor", + "battery1-power-l1-w", + "battery1-power-l2-w", + "battery1-power-l3-w", + "battery1-power-3-phase-sym-w" + ] + } + } +``` + +### Home Appliance devices base settings :::{table} devices::home_appliances::list :widths: 10 10 5 5 30 @@ -205,13 +337,14 @@ Properties: | Name | Type | Read-Only | Default | Description | | ---- | ---- | --------- | ------- | ----------- | -| device_id | `str` | `rw` | `required` | ID of home appliance | -| hours | `Optional[int]` | `rw` | `None` | Number of prediction hours. Defaults to global config prediction hours. | -| consumption_wh | `int` | `rw` | `required` | An integer representing the energy consumption of a household device in watt-hours. | -| duration_h | `int` | `rw` | `required` | An integer representing the usage duration of a household device in hours. | +| device_id | `str` | `rw` | `` | ID of device | +| consumption_wh | `int` | `rw` | `required` | Energy consumption [Wh]. | +| duration_h | `int` | `rw` | `required` | Usage duration in hours [0 ... 24]. | +| time_windows | `Optional[akkudoktoreos.utils.datetimeutil.TimeWindowSequence]` | `rw` | `None` | Sequence of allowed time windows. Defaults to optimization general time window. | +| measurement_keys | `Optional[list[str]]` | `ro` | `N/A` | Measurement keys for the home appliance stati that are measurements. | ::: -#### Example Input/Output +#### Example Input ```{eval-rst} .. code-block:: json @@ -220,17 +353,156 @@ Properties: "devices": { "home_appliances": [ { - "device_id": "dishwasher", - "hours": null, + "device_id": "battery1", "consumption_wh": 2000, - "duration_h": 3 + "duration_h": 1, + "time_windows": { + "windows": [ + { + "start_time": "10:00:00.000000 Europe/Berlin", + "duration": "2 hours", + "day_of_week": null, + "date": null, + "locale": null + } + ] + } + }, + { + "device_id": "ev1", + "consumption_wh": 2000, + "duration_h": 1, + "time_windows": { + "windows": [ + { + "start_time": "10:00:00.000000 Europe/Berlin", + "duration": "2 hours", + "day_of_week": null, + "date": null, + "locale": null + } + ] + } + }, + { + "device_id": "inverter1", + "consumption_wh": 2000, + "duration_h": 1, + "time_windows": { + "windows": [ + { + "start_time": "10:00:00.000000 Europe/Berlin", + "duration": "2 hours", + "day_of_week": null, + "date": null, + "locale": null + } + ] + } + }, + { + "device_id": "dishwasher", + "consumption_wh": 2000, + "duration_h": 1, + "time_windows": { + "windows": [ + { + "start_time": "10:00:00.000000 Europe/Berlin", + "duration": "2 hours", + "day_of_week": null, + "date": null, + "locale": null + } + ] + } } ] } } ``` -### Inverter Device Simulation Configuration +#### Example Output + +```{eval-rst} +.. code-block:: json + + { + "devices": { + "home_appliances": [ + { + "device_id": "battery1", + "consumption_wh": 2000, + "duration_h": 1, + "time_windows": { + "windows": [ + { + "start_time": "10:00:00.000000 Europe/Berlin", + "duration": "2 hours", + "day_of_week": null, + "date": null, + "locale": null + } + ] + }, + "measurement_keys": [] + }, + { + "device_id": "ev1", + "consumption_wh": 2000, + "duration_h": 1, + "time_windows": { + "windows": [ + { + "start_time": "10:00:00.000000 Europe/Berlin", + "duration": "2 hours", + "day_of_week": null, + "date": null, + "locale": null + } + ] + }, + "measurement_keys": [] + }, + { + "device_id": "inverter1", + "consumption_wh": 2000, + "duration_h": 1, + "time_windows": { + "windows": [ + { + "start_time": "10:00:00.000000 Europe/Berlin", + "duration": "2 hours", + "day_of_week": null, + "date": null, + "locale": null + } + ] + }, + "measurement_keys": [] + }, + { + "device_id": "dishwasher", + "consumption_wh": 2000, + "duration_h": 1, + "time_windows": { + "windows": [ + { + "start_time": "10:00:00.000000 Europe/Berlin", + "duration": "2 hours", + "day_of_week": null, + "date": null, + "locale": null + } + ] + }, + "measurement_keys": [] + } + ] + } + } +``` + +### Inverter devices base settings :::{table} devices::inverters::list :widths: 10 10 5 5 30 @@ -238,13 +510,13 @@ Properties: | Name | Type | Read-Only | Default | Description | | ---- | ---- | --------- | ------- | ----------- | -| device_id | `str` | `rw` | `required` | ID of inverter | -| hours | `Optional[int]` | `rw` | `None` | Number of prediction hours. Defaults to global config prediction hours. | -| max_power_wh | `float` | `rw` | `required` | - | -| battery_id | `Optional[str]` | `rw` | `None` | ID of battery | +| device_id | `str` | `rw` | `` | ID of device | +| max_power_w | `Optional[float]` | `rw` | `None` | Maximum power [W]. | +| battery_id | `Optional[str]` | `rw` | `None` | ID of battery controlled by this inverter. | +| measurement_keys | `Optional[list[str]]` | `ro` | `N/A` | Measurement keys for the inverter stati that are measurements. | ::: -#### Example Input/Output +#### Example Input ```{eval-rst} .. code-block:: json @@ -253,17 +525,68 @@ Properties: "devices": { "inverters": [ { - "device_id": "inverter1", - "hours": null, - "max_power_wh": 10000.0, + "device_id": "battery1", + "max_power_w": 10000.0, "battery_id": null + }, + { + "device_id": "ev1", + "max_power_w": 10000.0, + "battery_id": "battery1" + }, + { + "device_id": "inverter1", + "max_power_w": 10000.0, + "battery_id": "battery1" + }, + { + "device_id": "dishwasher", + "max_power_w": 10000.0, + "battery_id": "battery1" } ] } } ``` -### Battery Device Simulation Configuration +#### Example Output + +```{eval-rst} +.. code-block:: json + + { + "devices": { + "inverters": [ + { + "device_id": "battery1", + "max_power_w": 10000.0, + "battery_id": null, + "measurement_keys": [] + }, + { + "device_id": "ev1", + "max_power_w": 10000.0, + "battery_id": "battery1", + "measurement_keys": [] + }, + { + "device_id": "inverter1", + "max_power_w": 10000.0, + "battery_id": "battery1", + "measurement_keys": [] + }, + { + "device_id": "dishwasher", + "max_power_w": 10000.0, + "battery_id": "battery1", + "measurement_keys": [] + } + ] + } + } +``` + +### Battery devices base settings :::{table} devices::batteries::list :widths: 10 10 5 5 30 @@ -271,18 +594,27 @@ Properties: | Name | Type | Read-Only | Default | Description | | ---- | ---- | --------- | ------- | ----------- | -| device_id | `str` | `rw` | `required` | ID of battery | -| hours | `Optional[int]` | `rw` | `None` | Number of prediction hours. Defaults to global config prediction hours. | -| capacity_wh | `int` | `rw` | `required` | An integer representing the capacity of the battery in watt-hours. | -| charging_efficiency | `float` | `rw` | `0.88` | A float representing the charging efficiency of the battery. | -| discharging_efficiency | `float` | `rw` | `0.88` | A float representing the discharge efficiency of the battery. | -| max_charge_power_w | `Optional[float]` | `rw` | `5000` | Maximum charging power in watts. | -| initial_soc_percentage | `int` | `rw` | `0` | An integer representing the state of charge of the battery at the **start** of the current hour (not the current state). | -| min_soc_percentage | `int` | `rw` | `0` | An integer representing the minimum state of charge (SOC) of the battery in percentage. | -| max_soc_percentage | `int` | `rw` | `100` | An integer representing the maximum state of charge (SOC) of the battery in percentage. | +| device_id | `str` | `rw` | `` | ID of device | +| capacity_wh | `int` | `rw` | `8000` | Capacity [Wh]. | +| charging_efficiency | `float` | `rw` | `0.88` | Charging efficiency [0.01 ... 1.00]. | +| discharging_efficiency | `float` | `rw` | `0.88` | Discharge efficiency [0.01 ... 1.00]. | +| levelized_cost_of_storage_kwh | `float` | `rw` | `0.0` | Levelized cost of storage (LCOS), the average lifetime cost of delivering one kWh [€/kWh]. | +| max_charge_power_w | `Optional[float]` | `rw` | `5000` | Maximum charging power [W]. | +| min_charge_power_w | `Optional[float]` | `rw` | `50` | Minimum charging power [W]. | +| charge_rates | `Optional[list[float]]` | `rw` | `None` | Charge rates as factor of maximum charging power [0.00 ... 1.00]. None denotes all charge rates are available. | +| min_soc_percentage | `int` | `rw` | `0` | Minimum state of charge (SOC) as percentage of capacity [%]. | +| max_soc_percentage | `int` | `rw` | `100` | Maximum state of charge (SOC) as percentage of capacity [%]. | +| measurement_key_soc_factor | `str` | `ro` | `N/A` | Measurement key for the battery state of charge (SoC) as factor of total capacity [0.0 ... 1.0]. | +| measurement_key_power_l1_w | `str` | `ro` | `N/A` | Measurement key for the L1 power the battery is charged or discharged with [W]. | +| measurement_key_power_l2_w | `str` | `ro` | `N/A` | Measurement key for the L2 power the battery is charged or discharged with [W]. | +| measurement_key_power_l3_w | `str` | `ro` | `N/A` | Measurement key for the L3 power the battery is charged or discharged with [W]. | +| measurement_key_power_3_phase_sym_w | `str` | `ro` | `N/A` | Measurement key for the symmetric 3 phase power the battery is charged or discharged with [W]. | +| measurement_keys | `Optional[list[str]]` | `ro` | `N/A` | Measurement keys for the battery stati that are measurements. + +Battery SoC, power. | ::: -#### Example Input/Output +#### Example Input ```{eval-rst} .. code-block:: json @@ -292,14 +624,172 @@ Properties: "batteries": [ { "device_id": "battery1", - "hours": null, "capacity_wh": 8000, "charging_efficiency": 0.88, "discharging_efficiency": 0.88, + "levelized_cost_of_storage_kwh": 0.12, "max_charge_power_w": 5000.0, - "initial_soc_percentage": 42, + "min_charge_power_w": 50.0, + "charge_rates": [ + 0.0, + 0.25, + 0.5, + 0.75, + 1.0 + ], "min_soc_percentage": 10, "max_soc_percentage": 100 + }, + { + "device_id": "ev1", + "capacity_wh": 8000, + "charging_efficiency": 0.88, + "discharging_efficiency": 0.88, + "levelized_cost_of_storage_kwh": 0.12, + "max_charge_power_w": 5000.0, + "min_charge_power_w": 50.0, + "charge_rates": null, + "min_soc_percentage": 10, + "max_soc_percentage": 100 + }, + { + "device_id": "inverter1", + "capacity_wh": 8000, + "charging_efficiency": 0.88, + "discharging_efficiency": 0.88, + "levelized_cost_of_storage_kwh": 0.12, + "max_charge_power_w": 5000.0, + "min_charge_power_w": 50.0, + "charge_rates": null, + "min_soc_percentage": 10, + "max_soc_percentage": 100 + }, + { + "device_id": "dishwasher", + "capacity_wh": 8000, + "charging_efficiency": 0.88, + "discharging_efficiency": 0.88, + "levelized_cost_of_storage_kwh": 0.12, + "max_charge_power_w": 5000.0, + "min_charge_power_w": 50.0, + "charge_rates": null, + "min_soc_percentage": 10, + "max_soc_percentage": 100 + } + ] + } + } +``` + +#### Example Output + +```{eval-rst} +.. code-block:: json + + { + "devices": { + "batteries": [ + { + "device_id": "battery1", + "capacity_wh": 8000, + "charging_efficiency": 0.88, + "discharging_efficiency": 0.88, + "levelized_cost_of_storage_kwh": 0.12, + "max_charge_power_w": 5000.0, + "min_charge_power_w": 50.0, + "charge_rates": [ + 0.0, + 0.25, + 0.5, + 0.75, + 1.0 + ], + "min_soc_percentage": 10, + "max_soc_percentage": 100, + "measurement_key_soc_factor": "battery1-soc-factor", + "measurement_key_power_l1_w": "battery1-power-l1-w", + "measurement_key_power_l2_w": "battery1-power-l2-w", + "measurement_key_power_l3_w": "battery1-power-l3-w", + "measurement_key_power_3_phase_sym_w": "battery1-power-3-phase-sym-w", + "measurement_keys": [ + "battery1-soc-factor", + "battery1-power-l1-w", + "battery1-power-l2-w", + "battery1-power-l3-w", + "battery1-power-3-phase-sym-w" + ] + }, + { + "device_id": "ev1", + "capacity_wh": 8000, + "charging_efficiency": 0.88, + "discharging_efficiency": 0.88, + "levelized_cost_of_storage_kwh": 0.12, + "max_charge_power_w": 5000.0, + "min_charge_power_w": 50.0, + "charge_rates": null, + "min_soc_percentage": 10, + "max_soc_percentage": 100, + "measurement_key_soc_factor": "ev1-soc-factor", + "measurement_key_power_l1_w": "ev1-power-l1-w", + "measurement_key_power_l2_w": "ev1-power-l2-w", + "measurement_key_power_l3_w": "ev1-power-l3-w", + "measurement_key_power_3_phase_sym_w": "ev1-power-3-phase-sym-w", + "measurement_keys": [ + "ev1-soc-factor", + "ev1-power-l1-w", + "ev1-power-l2-w", + "ev1-power-l3-w", + "ev1-power-3-phase-sym-w" + ] + }, + { + "device_id": "inverter1", + "capacity_wh": 8000, + "charging_efficiency": 0.88, + "discharging_efficiency": 0.88, + "levelized_cost_of_storage_kwh": 0.12, + "max_charge_power_w": 5000.0, + "min_charge_power_w": 50.0, + "charge_rates": null, + "min_soc_percentage": 10, + "max_soc_percentage": 100, + "measurement_key_soc_factor": "inverter1-soc-factor", + "measurement_key_power_l1_w": "inverter1-power-l1-w", + "measurement_key_power_l2_w": "inverter1-power-l2-w", + "measurement_key_power_l3_w": "inverter1-power-l3-w", + "measurement_key_power_3_phase_sym_w": "inverter1-power-3-phase-sym-w", + "measurement_keys": [ + "inverter1-soc-factor", + "inverter1-power-l1-w", + "inverter1-power-l2-w", + "inverter1-power-l3-w", + "inverter1-power-3-phase-sym-w" + ] + }, + { + "device_id": "dishwasher", + "capacity_wh": 8000, + "charging_efficiency": 0.88, + "discharging_efficiency": 0.88, + "levelized_cost_of_storage_kwh": 0.12, + "max_charge_power_w": 5000.0, + "min_charge_power_w": 50.0, + "charge_rates": null, + "min_soc_percentage": 10, + "max_soc_percentage": 100, + "measurement_key_soc_factor": "dishwasher-soc-factor", + "measurement_key_power_l1_w": "dishwasher-power-l1-w", + "measurement_key_power_l2_w": "dishwasher-power-l2-w", + "measurement_key_power_l3_w": "dishwasher-power-l3-w", + "measurement_key_power_3_phase_sym_w": "dishwasher-power-3-phase-sym-w", + "measurement_keys": [ + "dishwasher-soc-factor", + "dishwasher-power-l1-w", + "dishwasher-power-l2-w", + "dishwasher-power-l3-w", + "dishwasher-power-3-phase-sym-w" + ] } ] } @@ -314,43 +804,76 @@ Properties: | Name | Environment Variable | Type | Read-Only | Default | Description | | ---- | -------------------- | ---- | --------- | ------- | ----------- | -| load0_name | `EOS_MEASUREMENT__LOAD0_NAME` | `Optional[str]` | `rw` | `None` | Name of the load0 source | -| load1_name | `EOS_MEASUREMENT__LOAD1_NAME` | `Optional[str]` | `rw` | `None` | Name of the load1 source | -| load2_name | `EOS_MEASUREMENT__LOAD2_NAME` | `Optional[str]` | `rw` | `None` | Name of the load2 source | -| load3_name | `EOS_MEASUREMENT__LOAD3_NAME` | `Optional[str]` | `rw` | `None` | Name of the load3 source | -| load4_name | `EOS_MEASUREMENT__LOAD4_NAME` | `Optional[str]` | `rw` | `None` | Name of the load4 source | +| load_emr_keys | `EOS_MEASUREMENT__LOAD_EMR_KEYS` | `Optional[list[str]]` | `rw` | `None` | The keys of the measurements that are energy meter readings of a load [kWh]. | +| grid_export_emr_keys | `EOS_MEASUREMENT__GRID_EXPORT_EMR_KEYS` | `Optional[list[str]]` | `rw` | `None` | The keys of the measurements that are energy meter readings of energy export to grid [kWh]. | +| grid_import_emr_keys | `EOS_MEASUREMENT__GRID_IMPORT_EMR_KEYS` | `Optional[list[str]]` | `rw` | `None` | The keys of the measurements that are energy meter readings of energy import from grid [kWh]. | +| pv_production_emr_keys | `EOS_MEASUREMENT__PV_PRODUCTION_EMR_KEYS` | `Optional[list[str]]` | `rw` | `None` | The keys of the measurements that are PV production energy meter readings [kWh]. | +| keys | | `list[str]` | `ro` | `N/A` | The keys of the measurements that can be stored. | ::: -### Example Input/Output +### Example Input ```{eval-rst} .. code-block:: json { "measurement": { - "load0_name": "Household", - "load1_name": null, - "load2_name": null, - "load3_name": null, - "load4_name": null + "load_emr_keys": [ + "load0_emr" + ], + "grid_export_emr_keys": [ + "grid_export_emr" + ], + "grid_import_emr_keys": [ + "grid_import_emr" + ], + "pv_production_emr_keys": [ + "pv1_emr" + ] + } + } +``` + +### Example Output + +```{eval-rst} +.. code-block:: json + + { + "measurement": { + "load_emr_keys": [ + "load0_emr" + ], + "grid_export_emr_keys": [ + "grid_export_emr" + ], + "grid_import_emr_keys": [ + "grid_import_emr" + ], + "pv_production_emr_keys": [ + "pv1_emr" + ], + "keys": [ + "grid_export_emr", + "grid_import_emr", + "load0_emr", + "pv1_emr" + ] } } ``` ## General Optimization Configuration -Attributes: - hours (int): Number of hours for optimizations. - :::{table} optimization :widths: 10 20 10 5 5 30 :align: left | Name | Environment Variable | Type | Read-Only | Default | Description | | ---- | -------------------- | ---- | --------- | ------- | ----------- | -| hours | `EOS_OPTIMIZATION__HOURS` | `Optional[int]` | `rw` | `48` | Number of hours into the future for optimizations. | -| penalty | `EOS_OPTIMIZATION__PENALTY` | `Optional[int]` | `rw` | `10` | Penalty factor used in optimization. | -| ev_available_charge_rates_percent | `EOS_OPTIMIZATION__EV_AVAILABLE_CHARGE_RATES_PERCENT` | `Optional[List[float]]` | `rw` | `[0.0, 0.375, 0.5, 0.625, 0.75, 0.875, 1.0]` | Charge rates available for the EV in percent of maximum charge. | +| horizon_hours | `EOS_OPTIMIZATION__HORIZON_HOURS` | `Optional[int]` | `rw` | `24` | The general time window within which the energy optimization goal shall be achieved [h]. Defaults to 24 hours. | +| interval | `EOS_OPTIMIZATION__INTERVAL` | `Optional[int]` | `rw` | `3600` | The optimization interval [sec]. | +| genetic | `EOS_OPTIMIZATION__GENETIC` | `Optional[akkudoktoreos.optimization.optimization.GeneticCommonSettings]` | `rw` | `None` | Genetic optimization algorithm configuration. | ::: ### Example Input/Output @@ -360,17 +883,49 @@ Attributes: { "optimization": { - "hours": 48, - "penalty": 10, - "ev_available_charge_rates_percent": [ - 0.0, - 0.375, - 0.5, - 0.625, - 0.75, - 0.875, - 1.0 - ] + "horizon_hours": 24, + "interval": 3600, + "genetic": { + "individuals": 400, + "generations": 400, + "seed": null, + "penalties": { + "ev_soc_miss": 10 + } + } + } + } +``` + +### General Genetic Optimization Algorithm Configuration + +:::{table} optimization::genetic +:widths: 10 10 5 5 30 +:align: left + +| Name | Type | Read-Only | Default | Description | +| ---- | ---- | --------- | ------- | ----------- | +| individuals | `Optional[int]` | `rw` | `300` | Number of individuals (solutions) to generate for the (initial) generation [>= 10]. Defaults to 300. | +| generations | `Optional[int]` | `rw` | `400` | Number of generations to evaluate the optimal solution [>= 10]. Defaults to 400. | +| seed | `Optional[int]` | `rw` | `None` | Fixed seed for genetic algorithm. Defaults to 'None' which means random seed. | +| penalties | `Optional[dict[str, Union[float, int, str]]]` | `rw` | `None` | A dictionary of penalty function parameters consisting of a penalty function parameter name and the associated value. | +::: + +#### Example Input/Output + +```{eval-rst} +.. code-block:: json + + { + "optimization": { + "genetic": { + "individuals": 300, + "generations": 400, + "seed": null, + "penalties": { + "ev_soc_miss": 10 + } + } } } ``` @@ -423,9 +978,9 @@ Validators: | Name | Environment Variable | Type | Read-Only | Default | Description | | ---- | -------------------- | ---- | --------- | ------- | ----------- | | provider | `EOS_ELECPRICE__PROVIDER` | `Optional[str]` | `rw` | `None` | Electricity price provider id of provider to be used. | -| charges_kwh | `EOS_ELECPRICE__CHARGES_KWH` | `Optional[float]` | `rw` | `None` | Electricity price charges (€/kWh). | +| charges_kwh | `EOS_ELECPRICE__CHARGES_KWH` | `Optional[float]` | `rw` | `None` | Electricity price charges [€/kWh]. Will be added to variable market price. | | vat_rate | `EOS_ELECPRICE__VAT_RATE` | `Optional[float]` | `rw` | `1.19` | VAT rate factor applied to electricity price when charges are used. | -| provider_settings | `EOS_ELECPRICE__PROVIDER_SETTINGS` | `Optional[akkudoktoreos.prediction.elecpriceimport.ElecPriceImportCommonSettings]` | `rw` | `None` | Provider settings | +| provider_settings | `EOS_ELECPRICE__PROVIDER_SETTINGS` | `ElecPriceCommonProviderSettings` | `rw` | `required` | Provider settings | ::: ### Example Input/Output @@ -438,14 +993,16 @@ Validators: "provider": "ElecPriceAkkudoktor", "charges_kwh": 0.21, "vat_rate": 1.19, - "provider_settings": null + "provider_settings": { + "ElecPriceImport": null + } } } ``` ### Common settings for elecprice data import from file or JSON String -:::{table} elecprice::provider_settings +:::{table} elecprice::provider_settings::ElecPriceImport :widths: 10 10 5 5 30 :align: left @@ -463,8 +1020,146 @@ Validators: { "elecprice": { "provider_settings": { - "import_file_path": null, - "import_json": "{\"elecprice_marketprice_wh\": [0.0003384, 0.0003318, 0.0003284]}" + "ElecPriceImport": { + "import_file_path": null, + "import_json": "{\"elecprice_marketprice_wh\": [0.0003384, 0.0003318, 0.0003284]}" + } + } + } + } +``` + +### Electricity Price Prediction Provider Configuration + +:::{table} elecprice::provider_settings +:widths: 10 10 5 5 30 +:align: left + +| Name | Type | Read-Only | Default | Description | +| ---- | ---- | --------- | ------- | ----------- | +| ElecPriceImport | `Optional[akkudoktoreos.prediction.elecpriceimport.ElecPriceImportCommonSettings]` | `rw` | `None` | ElecPriceImport settings | +::: + +#### Example Input/Output + +```{eval-rst} +.. code-block:: json + + { + "elecprice": { + "provider_settings": { + "ElecPriceImport": null + } + } + } +``` + +## Feed In Tariff Prediction Configuration + +:::{table} feedintariff +:widths: 10 20 10 5 5 30 +:align: left + +| Name | Environment Variable | Type | Read-Only | Default | Description | +| ---- | -------------------- | ---- | --------- | ------- | ----------- | +| provider | `EOS_FEEDINTARIFF__PROVIDER` | `Optional[str]` | `rw` | `None` | Feed in tariff provider id of provider to be used. | +| provider_settings | `EOS_FEEDINTARIFF__PROVIDER_SETTINGS` | `FeedInTariffCommonProviderSettings` | `rw` | `required` | Provider settings | +::: + +### Example Input/Output + +```{eval-rst} +.. code-block:: json + + { + "feedintariff": { + "provider": "FeedInTariffFixed", + "provider_settings": { + "FeedInTariffFixed": null, + "FeedInTariffImport": null + } + } + } +``` + +### Common settings for feed in tariff data import from file or JSON string + +:::{table} feedintariff::provider_settings::FeedInTariffImport +:widths: 10 10 5 5 30 +:align: left + +| Name | Type | Read-Only | Default | Description | +| ---- | ---- | --------- | ------- | ----------- | +| import_file_path | `Union[str, pathlib.Path, NoneType]` | `rw` | `None` | Path to the file to import feed in tariff data from. | +| import_json | `Optional[str]` | `rw` | `None` | JSON string, dictionary of feed in tariff forecast value lists. | +::: + +#### Example Input/Output + +```{eval-rst} +.. code-block:: json + + { + "feedintariff": { + "provider_settings": { + "FeedInTariffImport": { + "import_file_path": null, + "import_json": "{\"fead_in_tariff_wh\": [0.000078, 0.000078, 0.000023]}" + } + } + } + } +``` + +### Common settings for elecprice fixed price + +:::{table} feedintariff::provider_settings::FeedInTariffFixed +:widths: 10 10 5 5 30 +:align: left + +| Name | Type | Read-Only | Default | Description | +| ---- | ---- | --------- | ------- | ----------- | +| feed_in_tariff_kwh | `Optional[float]` | `rw` | `None` | Electricity price feed in tariff [€/kWH]. | +::: + +#### Example Input/Output + +```{eval-rst} +.. code-block:: json + + { + "feedintariff": { + "provider_settings": { + "FeedInTariffFixed": { + "feed_in_tariff_kwh": 0.078 + } + } + } + } +``` + +### Feed In Tariff Prediction Provider Configuration + +:::{table} feedintariff::provider_settings +:widths: 10 10 5 5 30 +:align: left + +| Name | Type | Read-Only | Default | Description | +| ---- | ---- | --------- | ------- | ----------- | +| FeedInTariffFixed | `Optional[akkudoktoreos.prediction.feedintarifffixed.FeedInTariffFixedCommonSettings]` | `rw` | `None` | FeedInTariffFixed settings | +| FeedInTariffImport | `Optional[akkudoktoreos.prediction.feedintariffimport.FeedInTariffImportCommonSettings]` | `rw` | `None` | FeedInTariffImport settings | +::: + +#### Example Input/Output + +```{eval-rst} +.. code-block:: json + + { + "feedintariff": { + "provider_settings": { + "FeedInTariffFixed": null, + "FeedInTariffImport": null } } } @@ -479,7 +1174,7 @@ Validators: | Name | Environment Variable | Type | Read-Only | Default | Description | | ---- | -------------------- | ---- | --------- | ------- | ----------- | | provider | `EOS_LOAD__PROVIDER` | `Optional[str]` | `rw` | `None` | Load provider id of provider to be used. | -| provider_settings | `EOS_LOAD__PROVIDER_SETTINGS` | `Union[akkudoktoreos.prediction.loadakkudoktor.LoadAkkudoktorCommonSettings, akkudoktoreos.prediction.loadvrm.LoadVrmCommonSettings, akkudoktoreos.prediction.loadimport.LoadImportCommonSettings, NoneType]` | `rw` | `None` | Provider settings | +| provider_settings | `EOS_LOAD__PROVIDER_SETTINGS` | `LoadCommonProviderSettings` | `rw` | `required` | Provider settings | ::: ### Example Input/Output @@ -490,14 +1185,18 @@ Validators: { "load": { "provider": "LoadAkkudoktor", - "provider_settings": null + "provider_settings": { + "LoadAkkudoktor": null, + "LoadVrm": null, + "LoadImport": null + } } } ``` ### Common settings for load data import from file or JSON string -:::{table} load::provider_settings +:::{table} load::provider_settings::LoadImport :widths: 10 10 5 5 30 :align: left @@ -515,8 +1214,10 @@ Validators: { "load": { "provider_settings": { - "import_file_path": null, - "import_json": "{\"load0_mean\": [676.71, 876.19, 527.13]}" + "LoadImport": { + "import_file_path": null, + "import_json": "{\"load0_mean\": [676.71, 876.19, 527.13]}" + } } } } @@ -524,7 +1225,7 @@ Validators: ### Common settings for VRM API -:::{table} load::provider_settings +:::{table} load::provider_settings::LoadVrm :widths: 10 10 5 5 30 :align: left @@ -542,8 +1243,10 @@ Validators: { "load": { "provider_settings": { - "load_vrm_token": "your-token", - "load_vrm_idsite": 12345 + "LoadVrm": { + "load_vrm_token": "your-token", + "load_vrm_idsite": 12345 + } } } } @@ -551,7 +1254,7 @@ Validators: ### Common settings for load data import from file -:::{table} load::provider_settings +:::{table} load::provider_settings::LoadAkkudoktor :widths: 10 10 5 5 30 :align: left @@ -568,7 +1271,38 @@ Validators: { "load": { "provider_settings": { - "loadakkudoktor_year_energy": 40421.0 + "LoadAkkudoktor": { + "loadakkudoktor_year_energy": 40421.0 + } + } + } + } +``` + +### Load Prediction Provider Configuration + +:::{table} load::provider_settings +:widths: 10 10 5 5 30 +:align: left + +| Name | Type | Read-Only | Default | Description | +| ---- | ---- | --------- | ------- | ----------- | +| LoadAkkudoktor | `Optional[akkudoktoreos.prediction.loadakkudoktor.LoadAkkudoktorCommonSettings]` | `rw` | `None` | LoadAkkudoktor settings | +| LoadVrm | `Optional[akkudoktoreos.prediction.loadvrm.LoadVrmCommonSettings]` | `rw` | `None` | LoadVrm settings | +| LoadImport | `Optional[akkudoktoreos.prediction.loadimport.LoadImportCommonSettings]` | `rw` | `None` | LoadImport settings | +::: + +#### Example Input/Output + +```{eval-rst} +.. code-block:: json + + { + "load": { + "provider_settings": { + "LoadAkkudoktor": null, + "LoadVrm": null, + "LoadImport": null } } } @@ -583,7 +1317,7 @@ Validators: | Name | Environment Variable | Type | Read-Only | Default | Description | | ---- | -------------------- | ---- | --------- | ------- | ----------- | | provider | `EOS_PVFORECAST__PROVIDER` | `Optional[str]` | `rw` | `None` | PVForecast provider id of provider to be used. | -| provider_settings | `EOS_PVFORECAST__PROVIDER_SETTINGS` | `Union[akkudoktoreos.prediction.pvforecastimport.PVForecastImportCommonSettings, akkudoktoreos.prediction.pvforecastvrm.PVforecastVrmCommonSettings, NoneType]` | `rw` | `None` | Provider settings | +| provider_settings | `EOS_PVFORECAST__PROVIDER_SETTINGS` | `PVForecastCommonProviderSettings` | `rw` | `required` | Provider settings | | planes | `EOS_PVFORECAST__PLANES` | `Optional[list[akkudoktoreos.prediction.pvforecast.PVForecastPlaneSetting]]` | `rw` | `None` | Plane configuration. | | max_planes | `EOS_PVFORECAST__MAX_PLANES` | `Optional[int]` | `rw` | `0` | Maximum number of planes that can be set | | planes_peakpower | | `List[float]` | `ro` | `N/A` | Compute a list of the peak power per active planes. | @@ -601,7 +1335,10 @@ Validators: { "pvforecast": { "provider": "PVForecastAkkudoktor", - "provider_settings": null, + "provider_settings": { + "PVForecastImport": null, + "PVForecastVrm": null + }, "planes": [ { "surface_tilt": 10.0, @@ -648,7 +1385,7 @@ Validators: "strings_per_inverter": 2 } ], - "max_planes": 0 + "max_planes": 1 } } ``` @@ -661,7 +1398,10 @@ Validators: { "pvforecast": { "provider": "PVForecastAkkudoktor", - "provider_settings": null, + "provider_settings": { + "PVForecastImport": null, + "PVForecastVrm": null + }, "planes": [ { "surface_tilt": 10.0, @@ -708,7 +1448,7 @@ Validators: "strings_per_inverter": 2 } ], - "max_planes": 0, + "max_planes": 1, "planes_peakpower": [ 5.0, 3.5 @@ -826,7 +1566,7 @@ Validators: ### Common settings for VRM API -:::{table} pvforecast::provider_settings +:::{table} pvforecast::provider_settings::PVForecastVrm :widths: 10 10 5 5 30 :align: left @@ -844,8 +1584,10 @@ Validators: { "pvforecast": { "provider_settings": { - "pvforecast_vrm_token": "your-token", - "pvforecast_vrm_idsite": 12345 + "PVForecastVrm": { + "pvforecast_vrm_token": "your-token", + "pvforecast_vrm_idsite": 12345 + } } } } @@ -853,7 +1595,7 @@ Validators: ### Common settings for pvforecast data import from file or JSON string -:::{table} pvforecast::provider_settings +:::{table} pvforecast::provider_settings::PVForecastImport :widths: 10 10 5 5 30 :align: left @@ -871,8 +1613,37 @@ Validators: { "pvforecast": { "provider_settings": { - "import_file_path": null, - "import_json": "{\"pvforecast_ac_power\": [0, 8.05, 352.91]}" + "PVForecastImport": { + "import_file_path": null, + "import_json": "{\"pvforecast_ac_power\": [0, 8.05, 352.91]}" + } + } + } + } +``` + +### PV Forecast Provider Configuration + +:::{table} pvforecast::provider_settings +:widths: 10 10 5 5 30 +:align: left + +| Name | Type | Read-Only | Default | Description | +| ---- | ---- | --------- | ------- | ----------- | +| PVForecastImport | `Optional[akkudoktoreos.prediction.pvforecastimport.PVForecastImportCommonSettings]` | `rw` | `None` | PVForecastImport settings | +| PVForecastVrm | `Optional[akkudoktoreos.prediction.pvforecastvrm.PVForecastVrmCommonSettings]` | `rw` | `None` | PVForecastVrm settings | +::: + +#### Example Input/Output + +```{eval-rst} +.. code-block:: json + + { + "pvforecast": { + "provider_settings": { + "PVForecastImport": null, + "PVForecastVrm": null } } } @@ -887,7 +1658,7 @@ Validators: | Name | Environment Variable | Type | Read-Only | Default | Description | | ---- | -------------------- | ---- | --------- | ------- | ----------- | | provider | `EOS_WEATHER__PROVIDER` | `Optional[str]` | `rw` | `None` | Weather provider id of provider to be used. | -| provider_settings | `EOS_WEATHER__PROVIDER_SETTINGS` | `Optional[akkudoktoreos.prediction.weatherimport.WeatherImportCommonSettings]` | `rw` | `None` | Provider settings | +| provider_settings | `EOS_WEATHER__PROVIDER_SETTINGS` | `WeatherCommonProviderSettings` | `rw` | `required` | Provider settings | ::: ### Example Input/Output @@ -898,14 +1669,16 @@ Validators: { "weather": { "provider": "WeatherImport", - "provider_settings": null + "provider_settings": { + "WeatherImport": null + } } } ``` ### Common settings for weather data import from file or JSON string -:::{table} weather::provider_settings +:::{table} weather::provider_settings::WeatherImport :widths: 10 10 5 5 30 :align: left @@ -923,8 +1696,35 @@ Validators: { "weather": { "provider_settings": { - "import_file_path": null, - "import_json": "{\"weather_temp_air\": [18.3, 17.8, 16.9]}" + "WeatherImport": { + "import_file_path": null, + "import_json": "{\"weather_temp_air\": [18.3, 17.8, 16.9]}" + } + } + } + } +``` + +### Weather Forecast Provider Configuration + +:::{table} weather::provider_settings +:widths: 10 10 5 5 30 +:align: left + +| Name | Type | Read-Only | Default | Description | +| ---- | ---- | --------- | ------- | ----------- | +| WeatherImport | `Optional[akkudoktoreos.prediction.weatherimport.WeatherImportCommonSettings]` | `rw` | `None` | WeatherImport settings | +::: + +#### Example Input/Output + +```{eval-rst} +.. code-block:: json + + { + "weather": { + "provider_settings": { + "WeatherImport": null } } } @@ -938,12 +1738,12 @@ Validators: | Name | Environment Variable | Type | Read-Only | Default | Description | | ---- | -------------------- | ---- | --------- | ------- | ----------- | -| host | `EOS_SERVER__HOST` | `Optional[pydantic.networks.IPvAnyAddress]` | `rw` | `127.0.0.1` | EOS server IP address. | -| port | `EOS_SERVER__PORT` | `Optional[int]` | `rw` | `8503` | EOS server IP port number. | +| host | `EOS_SERVER__HOST` | `Optional[str]` | `rw` | `127.0.0.1` | EOS server IP address. Defaults to 127.0.0.1. | +| port | `EOS_SERVER__PORT` | `Optional[int]` | `rw` | `8503` | EOS server IP port number. Defaults to 8503. | | verbose | `EOS_SERVER__VERBOSE` | `Optional[bool]` | `rw` | `False` | Enable debug output | -| startup_eosdash | `EOS_SERVER__STARTUP_EOSDASH` | `Optional[bool]` | `rw` | `True` | EOS server to start EOSdash server. | -| eosdash_host | `EOS_SERVER__EOSDASH_HOST` | `Optional[pydantic.networks.IPvAnyAddress]` | `rw` | `127.0.0.1` | EOSdash server IP address. | -| eosdash_port | `EOS_SERVER__EOSDASH_PORT` | `Optional[int]` | `rw` | `8504` | EOSdash server IP port number. | +| startup_eosdash | `EOS_SERVER__STARTUP_EOSDASH` | `Optional[bool]` | `rw` | `True` | EOS server to start EOSdash server. Defaults to True. | +| eosdash_host | `EOS_SERVER__EOSDASH_HOST` | `Optional[str]` | `rw` | `None` | EOSdash server IP address. Defaults to EOS server IP address. | +| eosdash_port | `EOS_SERVER__EOSDASH_PORT` | `Optional[int]` | `rw` | `None` | EOSdash server IP port number. Defaults to EOS server IP port number + 1. | ::: ### Example Input/Output @@ -990,6 +1790,7 @@ Validators: { "general": { + "version": "0.1.0+dev", "data_folder_path": null, "data_output_subpath": "output", "latitude": 52.52, @@ -1001,10 +1802,10 @@ Validators: }, "ems": { "startup_delay": 5.0, - "interval": 300.0 + "interval": 300.0, + "mode": "OPTIMIZATION" }, "logging": { - "level": null, "console_level": "TRACE", "file_level": "TRACE" }, @@ -1012,38 +1813,87 @@ Validators: "batteries": [ { "device_id": "battery1", - "hours": null, "capacity_wh": 8000, "charging_efficiency": 0.88, "discharging_efficiency": 0.88, + "levelized_cost_of_storage_kwh": 0.0, "max_charge_power_w": 5000, - "initial_soc_percentage": 0, + "min_charge_power_w": 50, + "charge_rates": null, "min_soc_percentage": 0, - "max_soc_percentage": 100 + "max_soc_percentage": 100, + "measurement_key_soc_factor": "battery1-soc-factor", + "measurement_key_power_l1_w": "battery1-power-l1-w", + "measurement_key_power_l2_w": "battery1-power-l2-w", + "measurement_key_power_l3_w": "battery1-power-l3-w", + "measurement_key_power_3_phase_sym_w": "battery1-power-3-phase-sym-w", + "measurement_keys": [ + "battery1-soc-factor", + "battery1-power-l1-w", + "battery1-power-l2-w", + "battery1-power-l3-w", + "battery1-power-3-phase-sym-w" + ] } ], + "max_batteries": 1, + "electric_vehicles": [ + { + "device_id": "battery1", + "capacity_wh": 8000, + "charging_efficiency": 0.88, + "discharging_efficiency": 0.88, + "levelized_cost_of_storage_kwh": 0.0, + "max_charge_power_w": 5000, + "min_charge_power_w": 50, + "charge_rates": null, + "min_soc_percentage": 0, + "max_soc_percentage": 100, + "measurement_key_soc_factor": "battery1-soc-factor", + "measurement_key_power_l1_w": "battery1-power-l1-w", + "measurement_key_power_l2_w": "battery1-power-l2-w", + "measurement_key_power_l3_w": "battery1-power-l3-w", + "measurement_key_power_3_phase_sym_w": "battery1-power-3-phase-sym-w", + "measurement_keys": [ + "battery1-soc-factor", + "battery1-power-l1-w", + "battery1-power-l2-w", + "battery1-power-l3-w", + "battery1-power-3-phase-sym-w" + ] + } + ], + "max_electric_vehicles": 1, "inverters": [], - "home_appliances": [] + "max_inverters": 1, + "home_appliances": [], + "max_home_appliances": 1 }, "measurement": { - "load0_name": "Household", - "load1_name": null, - "load2_name": null, - "load3_name": null, - "load4_name": null + "load_emr_keys": [ + "load0_emr" + ], + "grid_export_emr_keys": [ + "grid_export_emr" + ], + "grid_import_emr_keys": [ + "grid_import_emr" + ], + "pv_production_emr_keys": [ + "pv1_emr" + ] }, "optimization": { - "hours": 48, - "penalty": 10, - "ev_available_charge_rates_percent": [ - 0.0, - 0.375, - 0.5, - 0.625, - 0.75, - 0.875, - 1.0 - ] + "horizon_hours": 24, + "interval": 3600, + "genetic": { + "individuals": 400, + "generations": 400, + "seed": null, + "penalties": { + "ev_soc_miss": 10 + } + } }, "prediction": { "hours": 48, @@ -1053,15 +1903,31 @@ Validators: "provider": "ElecPriceAkkudoktor", "charges_kwh": 0.21, "vat_rate": 1.19, - "provider_settings": null + "provider_settings": { + "ElecPriceImport": null + } + }, + "feedintariff": { + "provider": "FeedInTariffFixed", + "provider_settings": { + "FeedInTariffFixed": null, + "FeedInTariffImport": null + } }, "load": { "provider": "LoadAkkudoktor", - "provider_settings": null + "provider_settings": { + "LoadAkkudoktor": null, + "LoadVrm": null, + "LoadImport": null + } }, "pvforecast": { "provider": "PVForecastAkkudoktor", - "provider_settings": null, + "provider_settings": { + "PVForecastImport": null, + "PVForecastVrm": null + }, "planes": [ { "surface_tilt": 10.0, @@ -1108,11 +1974,13 @@ Validators: "strings_per_inverter": 2 } ], - "max_planes": 0 + "max_planes": 1 }, "weather": { "provider": "WeatherImport", - "provider_settings": null + "provider_settings": { + "WeatherImport": null + } }, "server": { "host": "127.0.0.1", diff --git a/docs/_generated/openapi.md b/docs/_generated/openapi.md index 554ad1b..b29cbb8 100644 --- a/docs/_generated/openapi.md +++ b/docs/_generated/openapi.md @@ -1,6 +1,6 @@ # Akkudoktor-EOS -**Version**: `0.0.1` +**Version**: `v0.1.0+dev` **Description**: This project provides a comprehensive solution for simulating and optimizing an energy system based on renewable energy sources. With a focus on photovoltaic (PV) systems, battery storage (batteries), load management (consumer requirements), heat pumps, electric vehicles, and consideration of electricity price data, this system enables forecasting and optimization of energy flow and costs over a specified period. @@ -26,8 +26,10 @@ filled with the first available prediction value. Note: Use '/v1/prediction/list?key=load_mean_adjusted' instead. Load energy meter readings to be added to EOS measurement by: - '/v1/measurement/load-mr/value/by-name' or - '/v1/measurement/value' + '/v1/measurement/value' or + '/v1/measurement/series' or + '/v1/measurement/dataframe' or + '/v1/measurement/data' ``` **Request Body**: @@ -87,16 +89,25 @@ Note: Fastapi Optimize +``` +Deprecated: Optimize. + +Endpoint to handle optimization. + +Note: + Use automatic optimization instead. +``` + **Parameters**: - `start_hour` (query, optional): Defaults to current hour of the day. -- `ngen` (query, optional): No description provided. +- `ngen` (query, optional): Number of indivuals to generate for genetic algorithm. **Request Body**: - `application/json`: { - "$ref": "#/components/schemas/OptimizationParameters" + "$ref": "#/components/schemas/GeneticOptimizationParameters" } **Responses**: @@ -192,26 +203,38 @@ Returns: Fastapi Admin Cache Clear Post ``` -Clear the cache from expired data. +Clear the cache. -Deletes expired cache files. - -Args: - clear_all (Optional[bool]): Delete all cached files. Default is False. +Deletes all cache files. Returns: data (dict): The management data after cleanup. ``` -**Parameters**: - -- `clear_all` (query, optional): No description provided. - **Responses**: - **200**: Successful Response -- **422**: Validation Error +--- + +## POST /v1/admin/cache/clear-expired + +**Links**: [local](http://localhost:8503/docs#/default/fastapi_admin_cache_clear_expired_post_v1_admin_cache_clear-expired_post), [eos](https://petstore3.swagger.io/?url=https://raw.githubusercontent.com/Akkudoktor-EOS/EOS/refs/heads/main/openapi.json#/default/fastapi_admin_cache_clear_expired_post_v1_admin_cache_clear-expired_post) + +Fastapi Admin Cache Clear Expired Post + +``` +Clear the cache from expired data. + +Deletes expired cache files. + +Returns: + data (dict): The management data after cleanup. +``` + +**Responses**: + +- **200**: Successful Response --- @@ -448,6 +471,38 @@ Returns: --- +## GET /v1/energy-management/optimization/solution + +**Links**: [local](http://localhost:8503/docs#/default/fastapi_energy_management_optimization_solution_get_v1_energy-management_optimization_solution_get), [eos](https://petstore3.swagger.io/?url=https://raw.githubusercontent.com/Akkudoktor-EOS/EOS/refs/heads/main/openapi.json#/default/fastapi_energy_management_optimization_solution_get_v1_energy-management_optimization_solution_get) + +Fastapi Energy Management Optimization Solution Get + +``` +Get the latest solution of the optimization. +``` + +**Responses**: + +- **200**: Successful Response + +--- + +## GET /v1/energy-management/plan + +**Links**: [local](http://localhost:8503/docs#/default/fastapi_energy_management_plan_get_v1_energy-management_plan_get), [eos](https://petstore3.swagger.io/?url=https://raw.githubusercontent.com/Akkudoktor-EOS/EOS/refs/heads/main/openapi.json#/default/fastapi_energy_management_plan_get_v1_energy-management_plan_get) + +Fastapi Energy Management Plan Get + +``` +Get the latest energy management plan. +``` + +**Responses**: + +- **200**: Successful Response + +--- + ## GET /v1/health **Links**: [local](http://localhost:8503/docs#/default/fastapi_health_get_v1_health_get), [eos](https://petstore3.swagger.io/?url=https://raw.githubusercontent.com/Akkudoktor-EOS/EOS/refs/heads/main/openapi.json#/default/fastapi_health_get_v1_health_get) @@ -577,82 +632,6 @@ Get a list of available measurement keys. --- -## GET /v1/measurement/load-mr/series/by-name - -**Links**: [local](http://localhost:8503/docs#/default/fastapi_measurement_load_mr_series_by_name_get_v1_measurement_load-mr_series_by-name_get), [eos](https://petstore3.swagger.io/?url=https://raw.githubusercontent.com/Akkudoktor-EOS/EOS/refs/heads/main/openapi.json#/default/fastapi_measurement_load_mr_series_by_name_get_v1_measurement_load-mr_series_by-name_get) - -Fastapi Measurement Load Mr Series By Name Get - -``` -Get the meter reading of given load name as series. -``` - -**Parameters**: - -- `name` (query, required): Load name. - -**Responses**: - -- **200**: Successful Response - -- **422**: Validation Error - ---- - -## PUT /v1/measurement/load-mr/series/by-name - -**Links**: [local](http://localhost:8503/docs#/default/fastapi_measurement_load_mr_series_by_name_put_v1_measurement_load-mr_series_by-name_put), [eos](https://petstore3.swagger.io/?url=https://raw.githubusercontent.com/Akkudoktor-EOS/EOS/refs/heads/main/openapi.json#/default/fastapi_measurement_load_mr_series_by_name_put_v1_measurement_load-mr_series_by-name_put) - -Fastapi Measurement Load Mr Series By Name Put - -``` -Merge the meter readings series of given load name into EOS measurements at given datetime. -``` - -**Parameters**: - -- `name` (query, required): Load name. - -**Request Body**: - -- `application/json`: { - "$ref": "#/components/schemas/PydanticDateTimeSeries" -} - -**Responses**: - -- **200**: Successful Response - -- **422**: Validation Error - ---- - -## PUT /v1/measurement/load-mr/value/by-name - -**Links**: [local](http://localhost:8503/docs#/default/fastapi_measurement_load_mr_value_by_name_put_v1_measurement_load-mr_value_by-name_put), [eos](https://petstore3.swagger.io/?url=https://raw.githubusercontent.com/Akkudoktor-EOS/EOS/refs/heads/main/openapi.json#/default/fastapi_measurement_load_mr_value_by_name_put_v1_measurement_load-mr_value_by-name_put) - -Fastapi Measurement Load Mr Value By Name Put - -``` -Merge the meter reading of given load name and value into EOS measurements at given datetime. -``` - -**Parameters**: - -- `datetime` (query, required): Datetime. - -- `name` (query, required): Load name. - -- `value` (query, required): No description provided. - -**Responses**: - -- **200**: Successful Response - -- **422**: Validation Error - ---- - ## GET /v1/measurement/series **Links**: [local](http://localhost:8503/docs#/default/fastapi_measurement_series_get_v1_measurement_series_get), [eos](https://petstore3.swagger.io/?url=https://raw.githubusercontent.com/Akkudoktor-EOS/EOS/refs/heads/main/openapi.json#/default/fastapi_measurement_series_get_v1_measurement_series_get) @@ -665,7 +644,7 @@ Get the measurements of given key as series. **Parameters**: -- `key` (query, required): Prediction key. +- `key` (query, required): Measurement key. **Responses**: @@ -687,7 +666,7 @@ Merge measurement given as series into given key. **Parameters**: -- `key` (query, required): Prediction key. +- `key` (query, required): Measurement key. **Request Body**: @@ -717,7 +696,7 @@ Merge the measurement of given key and value into EOS measurements at given date - `datetime` (query, required): Datetime. -- `key` (query, required): Prediction key. +- `key` (query, required): Measurement key. - `value` (query, required): No description provided. @@ -990,6 +969,96 @@ Args: --- +## GET /v1/resource/status + +**Links**: [local](http://localhost:8503/docs#/default/fastapi_devices_status_get_v1_resource_status_get), [eos](https://petstore3.swagger.io/?url=https://raw.githubusercontent.com/Akkudoktor-EOS/EOS/refs/heads/main/openapi.json#/default/fastapi_devices_status_get_v1_resource_status_get) + +Fastapi Devices Status Get + +``` +Get the latest status of a resource/ device. + +Return: + latest_status: The latest status of a resource/ device. +``` + +**Parameters**: + +- `resource_id` (query, required): Resource ID. + +- `actuator_id` (query, optional): Actuator ID. + +**Responses**: + +- **200**: Successful Response + +- **422**: Validation Error + +--- + +## PUT /v1/resource/status + +**Links**: [local](http://localhost:8503/docs#/default/fastapi_devices_status_put_v1_resource_status_put), [eos](https://petstore3.swagger.io/?url=https://raw.githubusercontent.com/Akkudoktor-EOS/EOS/refs/heads/main/openapi.json#/default/fastapi_devices_status_put_v1_resource_status_put) + +Fastapi Devices Status Put + +``` +Update the status of a resource/ device. + +Return: + latest_status: The latest status of a resource/ device. +``` + +**Parameters**: + +- `resource_id` (query, required): Resource ID. + +- `actuator_id` (query, optional): Actuator ID. + +**Request Body**: + +- `application/json`: { + "anyOf": [ + { + "$ref": "#/components/schemas/PowerMeasurement-Input" + }, + { + "$ref": "#/components/schemas/EnergyMeasurement-Input" + }, + { + "$ref": "#/components/schemas/PPBCPowerProfileStatus-Input" + }, + { + "$ref": "#/components/schemas/OMBCStatus" + }, + { + "$ref": "#/components/schemas/FRBCActuatorStatus" + }, + { + "$ref": "#/components/schemas/FRBCEnergyStatus-Input" + }, + { + "$ref": "#/components/schemas/FRBCStorageStatus" + }, + { + "$ref": "#/components/schemas/FRBCTimerStatus" + }, + { + "$ref": "#/components/schemas/DDBCActuatorStatus" + } + ], + "description": "Resource Status.", + "title": "Status" +} + +**Responses**: + +- **200**: Successful Response + +- **422**: Validation Error + +--- + ## GET /visualization_results.pdf **Links**: [local](http://localhost:8503/docs#/default/get_pdf_visualization_results_pdf_get), [eos](https://petstore3.swagger.io/?url=https://raw.githubusercontent.com/Akkudoktor-EOS/EOS/refs/heads/main/openapi.json#/default/get_pdf_visualization_results_pdf_get) diff --git a/docs/akkudoktoreos/configtimewindow.md b/docs/akkudoktoreos/configtimewindow.md new file mode 100644 index 0000000..ebf245e --- /dev/null +++ b/docs/akkudoktoreos/configtimewindow.md @@ -0,0 +1,849 @@ +% SPDX-License-Identifier: Apache-2.0 +(configtimewindow-page)= + +# Time Window Sequence Configuration + +## Overview + +The `TimeWindowSequence` model is used to configure allowed time slots for home appliance runs. +It contains a collection of `TimeWindow` objects that define when appliances can operate. + +## Basic Structure + +A `TimeWindowSequence` is configured as a JSON object with a `windows` array: + +```json +{ + "windows": [ + { + "start_time": "09:00", + "duration": "PT2H", + "day_of_week": null, + "date": null, + "locale": null + } + ] +} +``` + +## TimeWindow Fields + +Each `TimeWindow` object has the following fields: + +- **`start_time`** (required): Time when the window begins +- **`duration`** (required): How long the window lasts +- **`day_of_week`** (optional): Restrict to specific day of week +- **`date`** (optional): Restrict to specific calendar date +- **`locale`** (optional): Language for day name parsing + +## Time Formats + +### Start Time (`start_time`) + +The `start_time` field accepts various time formats: + +#### 24-Hour Format + +```json +{ + "start_time": "14:30" // 2:30 PM +} +``` + +#### 12-Hour Format with AM/PM + +```json +{ + "start_time": "2:30 PM" // 2:30 PM +} +``` + +#### Compact Format + +```json +{ + "start_time": "1430" // 2:30 PM +} +``` + +#### With Seconds + +```json +{ + "start_time": "14:30:45" // 2:30:45 PM +} +``` + +#### With Microseconds + +```json +{ + "start_time": "14:30:45.123456" +} +``` + +#### European Format + +```json +{ + "start_time": "14h30" // 2:30 PM +} +``` + +#### Short Formats + +```json +{ + "start_time": "14" // 2:00 PM +} +``` + +```json +{ + "start_time": "2PM" // 2:00 PM +} +``` + +#### Decimal Time + +```json +{ + "start_time": "14.5" // 2:30 PM (14:30) +} +``` + +#### With Timezones + +```json +{ + "start_time": "14:30 UTC" +} +``` + +```json +{ + "start_time": "2:30 PM EST" +} +``` + +```json +{ + "start_time": "14:30 +05:30" +} +``` + +### Duration (`duration`) + +The `duration` field supports multiple formats for maximum flexibility: + +#### ISO 8601 Duration Format (Recommended) + +```json +{ + "duration": "PT2H30M" // 2 hours 30 minutes +} +``` + +```json +{ + "duration": "PT3H" // 3 hours +} +``` + +```json +{ + "duration": "PT90M" // 90 minutes +} +``` + +```json +{ + "duration": "PT1H30M45S" // 1 hour 30 minutes 45 seconds +} +``` + +#### Human-Readable String Format + +The system accepts natural language duration strings: + +```json +{ + "duration": "2 hours 30 minutes" +} +``` + +```json +{ + "duration": "3 hours" +} +``` + +```json +{ + "duration": "90 minutes" +} +``` + +```json +{ + "duration": "1 hour 30 minutes 45 seconds" +} +``` + +```json +{ + "duration": "2 days 5 hours" +} +``` + +```json +{ + "duration": "1 day 2 hours 30 minutes" +} +``` + +#### Singular and Plural Forms + +Both singular and plural forms are supported: + +```json +{ + "duration": "1 day" // Singular +} +``` + +```json +{ + "duration": "2 days" // Plural +} +``` + +```json +{ + "duration": "1 hour" // Singular +} +``` + +```json +{ + "duration": "5 hours" // Plural +} +``` + +#### Numeric Formats + +##### Seconds as Integer + +```json +{ + "duration": 3600 // 3600 seconds = 1 hour +} +``` + +```json +{ + "duration": 1800 // 1800 seconds = 30 minutes +} +``` + +##### Seconds as Float + +```json +{ + "duration": 3600.5 // 3600.5 seconds = 1 hour 0.5 seconds +} +``` + +##### Tuple Format [days, hours, minutes, seconds] + +```json +{ + "duration": [0, 2, 30, 0] // 0 days, 2 hours, 30 minutes, 0 seconds +} +``` + +```json +{ + "duration": [1, 0, 0, 0] // 1 day +} +``` + +```json +{ + "duration": [0, 0, 45, 30] // 45 minutes 30 seconds +} +``` + +```json +{ + "duration": [2, 5, 15, 45] // 2 days, 5 hours, 15 minutes, 45 seconds +} +``` + +#### Mixed Time Units + +You can combine different time units in string format: + +```json +{ + "duration": "1 day 4 hours 30 minutes 15 seconds" +} +``` + +```json +{ + "duration": "3 days 2 hours" +} +``` + +```json +{ + "duration": "45 minutes 30 seconds" +} +``` + +#### Common Duration Examples + +##### Short Durations + +```json +{ + "duration": "30 minutes" // Quick appliance cycle +} +``` + +```json +{ + "duration": "PT30M" // ISO format equivalent +} +``` + +```json +{ + "duration": 1800 // Numeric equivalent (seconds) +} +``` + +##### Medium Durations + +```json +{ + "duration": "2 hours 15 minutes" +} +``` + +```json +{ + "duration": "PT2H15M" // ISO format equivalent +} +``` + +```json +{ + "duration": [0, 2, 15, 0] // Tuple format equivalent +} +``` + +##### Long Durations + +```json +{ + "duration": "1 day 8 hours" // All-day appliance window +} +``` + +```json +{ + "duration": "PT32H" // ISO format equivalent +} +``` + +```json +{ + "duration": [1, 8, 0, 0] // Tuple format equivalent +} +``` + +#### Validation Rules for Duration + +- **ISO 8601 format**: Must start with `PT` and use valid duration specifiers (H, M, S) +- **String format**: Must contain valid time units (day/days, hour/hours, minute/minutes, second/seconds) +- **Numeric format**: Must be a positive number representing seconds +- **Tuple format**: Must be exactly 4 elements: [days, hours, minutes, seconds] +- **All formats**: Duration must be positive (greater than 0) + +#### Duration Format Recommendations + +1. **Use ISO 8601 format** for API consistency: `"PT2H30M"` +2. **Use human-readable strings** for configuration files: `"2 hours 30 minutes"` +3. **Use numeric format** for programmatic calculations: `9000` (seconds) +4. **Use tuple format** for structured data: `[0, 2, 30, 0]` + +#### Error Handling for Duration + +Common duration errors and solutions: + +- **Invalid ISO format**: Ensure proper `PT` prefix and valid specifiers +- **Unknown time units**: Use day/days, hour/hours, minute/minutes, second/seconds +- **Negative duration**: All durations must be positive +- **Invalid tuple length**: Tuple must have exactly 4 elements +- **String too long**: Duration strings have a maximum length limit for security + +## Day of Week Restrictions + +### Using Numbers (0=Monday, 6=Sunday) + +```json +{ + "day_of_week": 0 // Monday +} +``` + +```json +{ + "day_of_week": 6 // Sunday +} +``` + +### Using English Day Names + +```json +{ + "day_of_week": "Monday" +} +``` + +```json +{ + "day_of_week": "sunday" // Case insensitive +} +``` + +### Using Localized Day Names + +```json +{ + "day_of_week": "Montag", // German for Monday + "locale": "de" +} +``` + +```json +{ + "day_of_week": "Lundi", // French for Monday + "locale": "fr" +} +``` + +## Date Restrictions + +### Specific Date + +```json +{ + "date": "2024-12-25" // Christmas Day 2024 +} +``` + +**Note**: When `date` is specified, `day_of_week` is ignored. + +## Complete Examples + +### Example 1: Basic Daily Window + +Allow appliance to run between 9:00 AM and 11:00 AM every day: + +```json +{ + "windows": [ + { + "start_time": "09:00", + "duration": "PT2H" + } + ] +} +``` + +### Example 2: Weekday Only + +Allow appliance to run between 8:00 AM and 6:00 PM on weekdays: + +```json +{ + "windows": [ + { + "start_time": "08:00", + "duration": "PT10H", + "day_of_week": 0 + }, + { + "start_time": "08:00", + "duration": "PT10H", + "day_of_week": 1 + }, + { + "start_time": "08:00", + "duration": "PT10H", + "day_of_week": 2 + }, + { + "start_time": "08:00", + "duration": "PT10H", + "day_of_week": 3 + }, + { + "start_time": "08:00", + "duration": "PT10H", + "day_of_week": 4 + } + ] +} +``` + +### Example 3: Multiple Daily Windows + +Allow appliance to run during morning and evening hours: + +```json +{ + "windows": [ + { + "start_time": "06:00", + "duration": "PT3H" + }, + { + "start_time": "18:00", + "duration": "PT4H" + } + ] +} +``` + +### Example 4: Weekend Special Hours + +Different hours for weekdays and weekends: + +```json +{ + "windows": [ + { + "start_time": "08:00", + "duration": "PT8H", + "day_of_week": "Monday" + }, + { + "start_time": "08:00", + "duration": "PT8H", + "day_of_week": "Tuesday" + }, + { + "start_time": "08:00", + "duration": "PT8H", + "day_of_week": "Wednesday" + }, + { + "start_time": "08:00", + "duration": "PT8H", + "day_of_week": "Thursday" + }, + { + "start_time": "08:00", + "duration": "PT8H", + "day_of_week": "Friday" + }, + { + "start_time": "10:00", + "duration": "PT6H", + "day_of_week": "Saturday" + }, + { + "start_time": "10:00", + "duration": "PT6H", + "day_of_week": "Sunday" + } + ] +} +``` + +### Example 5: Holiday Schedule + +Special schedule for a specific date: + +```json +{ + "windows": [ + { + "start_time": "10:00", + "duration": "PT4H", + "date": "2024-12-25" + } + ] +} +``` + +### Example 6: Localized Configuration + +Using German day names: + +```json +{ + "windows": [ + { + "start_time": "14:00", + "duration": "PT2H", + "day_of_week": "Montag", + "locale": "de" + }, + { + "start_time": "14:00", + "duration": "PT2H", + "day_of_week": "Mittwoch", + "locale": "de" + }, + { + "start_time": "14:00", + "duration": "PT2H", + "day_of_week": "Freitag", + "locale": "de" + } + ] +} +``` + +### Example 7: Complex Schedule with Timezones + +Multiple windows with different timezones: + +```json +{ + "windows": [ + { + "start_time": "09:00 UTC", + "duration": "PT4H", + "day_of_week": "Monday" + }, + { + "start_time": "2:00 PM EST", + "duration": "PT3H", + "day_of_week": "Friday" + } + ] +} +``` + +### Example 8: Night Shift Schedule + +Crossing midnight (note: each window is within a single day): + +```json +{ + "windows": [ + { + "start_time": "22:00", + "duration": "PT2H" + }, + { + "start_time": "00:00", + "duration": "PT6H" + } + ] +} +``` + +## Advanced Usage Patterns + +### Off-Peak Hours + +Configure appliance to run during off-peak electricity hours: + +```json +{ + "windows": [ + { + "start_time": "23:00", + "duration": "PT1H" + }, + { + "start_time": "00:00", + "duration": "PT7H" + } + ] +} +``` + +### Workday Lunch Break + +Allow appliance to run during lunch break on workdays: + +```json +{ + "windows": [ + { + "start_time": "12:00", + "duration": "PT1H", + "day_of_week": 0 + }, + { + "start_time": "12:00", + "duration": "PT1H", + "day_of_week": 1 + }, + { + "start_time": "12:00", + "duration": "PT1H", + "day_of_week": 2 + }, + { + "start_time": "12:00", + "duration": "PT1H", + "day_of_week": 3 + }, + { + "start_time": "12:00", + "duration": "PT1H", + "day_of_week": 4 + } + ] +} +``` + +### Seasonal Schedule + +Different schedules for different dates: + +```json +{ + "windows": [ + { + "start_time": "08:00", + "duration": "PT10H", + "date": "2024-06-21" + }, + { + "start_time": "09:00", + "duration": "PT8H", + "date": "2024-12-21" + } + ] +} +``` + +## Common Patterns + +### 1. Always Available + +```json +{ + "windows": [ + { + "start_time": "00:00", + "duration": "PT24H" + } + ] +} +``` + +### 2. Business Hours + +```json +{ + "windows": [ + { + "start_time": "09:00", + "duration": "PT8H", + "day_of_week": 0 + }, + { + "start_time": "09:00", + "duration": "PT8H", + "day_of_week": 1 + }, + { + "start_time": "09:00", + "duration": "PT8H", + "day_of_week": 2 + }, + { + "start_time": "09:00", + "duration": "PT8H", + "day_of_week": 3 + }, + { + "start_time": "09:00", + "duration": "PT8H", + "day_of_week": 4 + } + ] +} +``` + +### 3. Never Available + +```json +{ + "windows": [] +} +``` + +## Validation Rules + +- `start_time` must be a valid time format +- `duration` must be a positive duration +- `day_of_week` must be 0-6 (integer) or valid day name (string) +- `date` must be a valid ISO date format (YYYY-MM-DD) +- If `date` is specified, `day_of_week` is ignored +- `locale` must be a valid locale code when using localized day names + +## Tips and Best Practices + +1. **Use 24-hour format** for clarity: `"14:30"` instead of `"2:30 PM"` +2. **Keep durations reasonable** for appliance operation cycles +3. **Test timezone handling** if using timezone-aware times +4. **Use specific dates** for holiday schedules +5. **Consider overlapping windows** for flexibility +6. **Use localization** for international deployments +7. **Document your patterns** for maintenance + +## Error Handling + +Common errors and solutions: + +- **Invalid time format**: Use supported time formats listed above +- **Invalid duration**: Use ISO 8601 duration format (PT1H30M) +- **Invalid day name**: Check spelling and locale settings +- **Invalid date**: Use YYYY-MM-DD format +- **Unknown locale**: Use standard locale codes (en, de, fr, etc.) + +## Integration Examples + +### Python Usage + +```python +from pydantic import ValidationError + +try: + config = TimeWindowSequence.model_validate_json(json_string) + print(f"Configured {len(config.windows)} time windows") +except ValidationError as e: + print(f"Configuration error: {e}") +``` + +### API Configuration + +```json +{ + "device_id": "dishwasher_01", + "time_windows": { + "windows": [ + { + "start_time": "22:00", + "duration": "PT2H" + }, + { + "start_time": "06:00", + "duration": "PT2H" + } + ] + } +} +``` diff --git a/docs/akkudoktoreos/configuration.md b/docs/akkudoktoreos/configuration.md index ff5d111..dd858aa 100644 --- a/docs/akkudoktoreos/configuration.md +++ b/docs/akkudoktoreos/configuration.md @@ -1,6 +1,7 @@ % SPDX-License-Identifier: Apache-2.0 +(configuration-page)= -# Configuration +# Configuration Guideline The configuration controls all aspects of EOS: optimization, prediction, measurement, and energy management. diff --git a/docs/akkudoktoreos/integration.md b/docs/akkudoktoreos/integration.md index f479d23..f7fd6fd 100644 --- a/docs/akkudoktoreos/integration.md +++ b/docs/akkudoktoreos/integration.md @@ -32,8 +32,7 @@ emphasizes local control and user privacy. ### Home Assistant Resources -- Duetting's [EOS Home Assistant Addon](https://github.com/Duetting/ha_eos_addon) — Additional - details can be found in this [discussion thread](https://github.com/Akkudoktor-EOS/EOS/discussions/294). +- Duetting's [EOS Home Assistant Addon](https://github.com/Duetting/ha_eos_addon). ## EOS Connect diff --git a/docs/akkudoktoreos/logging.md b/docs/akkudoktoreos/logging.md index 79f72f1..e47644c 100644 --- a/docs/akkudoktoreos/logging.md +++ b/docs/akkudoktoreos/logging.md @@ -73,3 +73,9 @@ You can also control the log level by setting the `EOS_LOGGING__CONSOLE_LEVEL` a If the `file_level` configuration is set, log records are written to a rotating log file. The log file is in the data output directory and named `eos.log`. You may directly read the file or use the `/v1/logging/log` endpoint to access the file log. + +:::{admonition} Note +:class: note +The `/v1/logging/log` endpoint needs file logging to be enabled. Otherwise old or no logging +information is provided. +::: diff --git a/docs/akkudoktoreos/measurement.md b/docs/akkudoktoreos/measurement.md index 8f4c3da..1035b85 100644 --- a/docs/akkudoktoreos/measurement.md +++ b/docs/akkudoktoreos/measurement.md @@ -1,4 +1,5 @@ % SPDX-License-Identifier: Apache-2.0 +(measurement-page)= # Measurements @@ -12,14 +13,7 @@ accuracy. ## Storing Measurements EOS stores measurements in a **key-value store**, where the term `measurement key` refers to the -unique identifier used to store and retrieve specific measurement data. Note that the key-value -store is memory-based, meaning that all stored data will be lost upon restarting the EOS REST -server. - -:::{admonition} Todo -:class: note -Ensure that measurement data persists across server restarts. -::: +unique identifier used to store and retrieve specific measurement data. Several endpoints of the EOS REST server allow for the management and retrieval of these measurements. @@ -30,14 +24,14 @@ The measurement data must be or is provided in one of the following formats: A dictionary with the following structure: -```python - { - "start_datetime": "2024-01-01 00:00:00", - "interval": "1 Hour", - "": [value, value, ...], - "": [value, value, ...], - ... - } +```json +{ + "start_datetime": "2024-01-01 00:00:00", + "interval": "1 hour", + "": [value, value, ...], + "": [value, value, ...], + ... +} ``` ### 2. DateTimeDataFrame @@ -51,43 +45,84 @@ The column name of the data must be the same as the names of the `measurement ke A JSON string created from a [pandas](https://pandas.pydata.org/docs/index.html) series with a `DatetimeIndex`. Use [pandas.Series.to_json(orient="index")](https://pandas.pydata.org/docs/reference/api/pandas.Series.to_json.html#pandas.Series.to_json). +Creates a dictionary like this: + +```json +{ + "data": { + "2024-01-01T00:00:00+01:00": 1, + "2024-01-02T00:00:00+01:00": 2, + "2024-01-03T00:00:00+01:00": 3, + ... + }, + "dtype": "float64", + "tz": "Europe/Berlin" +} +``` + ## Load Measurement -The EOS measurement store provides for storing meter readings of loads. There are currently five loads -foreseen. The associated `measurement key`s are: +The EOS measurement store provides for storing energy meter readings of loads. -- `load0_mr`: Load0 meter reading [kWh] -- `load1_mr`: Load1 meter reading [kWh] -- `load2_mr`: Load2 meter reading [kWh] -- `load3_mr`: Load3 meter reading [kWh] -- `load4_mr`: Load4 meter reading [kWh] +The associated `measurement key`s can be configured by: -For ease of use, you can assign descriptive names to the `measurement key`s to represent your -system's load sources. Use the following `configuration options` to set these names -(e.g., 'Dish Washer', 'Heat Pump'): - -- `load0_name`: Name of the load0 source -- `load1_name`: Name of the load1 source -- `load2_name`: Name of the load2 source -- `load3_name`: Name of the load3 source -- `load4_name`: Name of the load4 source +```json +{ + "measurement": { + "load_emr_keys": ["load0_emr", "my special load", ...] + } +} +``` Load measurements can be stored for any datetime. The values between different meter readings are -linearly approximated. Since optimization occurs on the hour, storing values between hours is -generally not useful. +linearly approximated. Storing values between optimization intervals is generally not useful. The EOS measurement store automatically sums all given loads to create a total load value series for specified intervals, usually one hour. This aggregated data can be used for load predictions. +:::{admonition} Warning +:class: warning +Only use **actual meter readings** in **kWh**, not energy consumption. +Example: `112345.77`, `112389.23`, `112412.55`, … +::: + ## Grid Export/ Import Measurement The EOS measurement store also allows for the storage of meter readings for grid import and export. -The associated `measurement key`s are: -- `grid_export_mr`: Export to grid meter reading [kWh] -- `grid_import_mr`: Import from grid meter reading [kWh] +The associated `measurement key`s can be configured by: + +```json +{ + "measurement": { + "grid_export_emr_keys": ["grid_export_emr", ...], + "grid_import_emr_keys": ["grid_import_emr", ...], + } +} +``` :::{admonition} Todo :class: note Currently not used. Integrate grid meter readings into the respective predictions. ::: + +## Battery/ Electric Vehicle State of Charge (SoC) Measurement + +The state of charge (SoC) measurement of batteries and electric vehicle batteries can be stored. + +The associated `measurement key` is pre-defined by the device configuration. It can be +determined from the device configuration by the read-only `measurement_key_soc_factor` configuration +option. + +## Battery/ Electric Vehicle Power Measurement + +The charge/ discharge power measurements of batteries and electric vehicle batteries can be stored. +Charging power is denoted by a negative value, discharging power by a positive value. + +The associated `measurement key`s are pre-defined by the device configuration. They can be +determined from the device configuration by read-only configuration options: + +- `measurement_key_power_l1_w` +- `measurement_key_power_l2_w` +- `measurement_key_power_l3_w` +- `measurement_key_power_3_phase_sym_w` diff --git a/docs/akkudoktoreos/optimauto.md b/docs/akkudoktoreos/optimauto.md new file mode 100644 index 0000000..8a14bd8 --- /dev/null +++ b/docs/akkudoktoreos/optimauto.md @@ -0,0 +1,448 @@ +% SPDX-License-Identifier: Apache-2.0 + +# Automatic Optimization + +## Introduction + +EOS offers two approaches to optimize your energy management system: `post /optimize optimization` and +`automatic optimization`. + +The `post /optimize optimization` interface, based on a **POST** request to `/optimize`, is widely +used. It was originally developed by Andreas at the start of the project and is still demonstrated +in his instructional videos. This interface allows users or external systems to trigger an +optimization manually, supplying custom parameters and timing. + +As an alternative, EOS supports `automatic optimization`, which runs automatically at configured +intervals. It retrieves all required input data — including electricity prices, battery storage +capacity, PV production forecasts, and temperature data — based on your system configuration. + +### Genetic Algorithm + +Both optimization modes use the same core optimization engine. + +EOS uses a [genetic algorithm](https://en.wikipedia.org/wiki/Genetic_algorithm) to find an optimal +control strategy for home energy devices such as household loads, batteries, and electric vehicles. + +In this context, each **individual** represents a possible solution — a specific control schedule +that defines how devices should operate over time. These individuals are evaluated using +[resource simulations](#resource-page), which model the system’s energy behavior over a defined time +period divided into fixed intervals. + +The quality of each solution (its *fitness*) is determined by how well it performs during +simulation, based on objectives such as minimizing electricity costs, maximizing self-consumption, +or meeting battery charge targets. + +Through an iterative process of selection, crossover, and mutation, the algorithm gradually evolves +more effective solutions. The final result is an optimized control strategy that balances multiple +system goals within the constraints of the input data and configuration. + +:::{note} +You don’t need to understand the internal workings of the genetic algorithm to benefit from +automatic optimization. EOS handles everything behind the scenes based on your configuration. +However, advanced users can fine-tune the optimization behavior using additional settings like +population size, penalties, and random seed. +::: + +## Energy Management Plan + +Whenever the optimization is run, the energy management plan is updated. The energy management plan +provides a list of energy management instructions in chronological order. The instructions lean on +to the [S2 standard](https://docs.s2standard.org/) to have maximum flexibility and stay completely +independent from any manufacturer. + +### Battery Instructions + +The battery control instructions assume an idealized battery model. Under this model, the battery +can be operated in four discrete operation modes: + +| **Operation Mode ID** | **Description** | +| --------------------- | ------------------------------------------------------------------------------------ | +| **IDLE** | Battery neither charges nor discharges; holds its state of charge. | +| **CHARGE** | Charge at a specified power rate up to the allowable maximum. | +| **DISCHARGE** | Discharge at a specified power rate up to the allowable maximum. | +| **ALLOW_DISCHARGE** | Allow the battery to freely discharge depending on its instantaneous power setpoint. | + +The **operation mode factor** (0.0–1.0) specifies the normalized power rate relative to the +battery's nominal maximum charge or discharge power. A value of 1.0 corresponds to full-rate +charging or discharging, while 0.0 indicates no power transfer. Intermediate values scale the power +proportionally. + +### Electric Vehicle Instructions + +The electric vehicle control instructions assume an idealized EV battery model. Under this model, +the EV battery can be operated in two operation modes: + +| **Operation Mode ID** | **Description** | +| --------------------- | ------------------------------------------------------------------------------------ | +| **IDLE** | Battery neither charges nor discharges; holds its state of charge. | +| **CHARGE** | Charge at a specified power rate up to the allowable maximum. | + +The **operation mode factor** (0.0–1.0) specifies the normalized power rate relative to the +battery's nominal maximum charge power. A value of 1.0 corresponds to full-rate charging, while 0.0 +indicates no power transfer. Intermediate values scale the power proportionally. + +### Home Appliance Instructions + +The home appliance instructions assume an idealized home appliance model. Under this model, +the home appliance can be operated in two operation modes: + +| **Operation Mode ID** | **Description** | +| --------------------- | ------------------------------------------------------------------------------------ | +| **RUN** | The home appliance is started and runs until the end of it's power sequence. | +| **IDLE** | The home appliance does not run. | + +The **operation mode factor** (0.0–1.0) is ignored. + +## Configuration + +### Energy management configuration + +The energy management is run on configured intervals with some startup delay after server start. +Both values are given in seconds. + +:::{admonition} Note +:class: note +If no interval is configured (`None`, `null`) there will be only one energy management run at +startup. +::: + +The energy management can be run in two modes: + +- **OPTIMIZATION**: A full optimization is done. This includes update of predictions. +- **PREDICTION**: Only the predictions are updated. + +**Example:** + +```json +{ + "ems": { + "startup_delay": 5.0, + "interval": 300.0, + "mode": "OPTIMIZATION" + } +} +``` + +### Optimization Configuration + +#### Optimization Time Configuration + +- **horizon_hours**: + The optimization horizon parameter defines the default time window — in hours — within which + the energy optimization goal shall be achieved. + + Specific devices, like the home appliance, have their own configuration for time windows. If + the time windows are not configured the simulation uses the default time window. + + Each device simulation run must ensure that all tasks or appliance cycles (e.g., running a + dishwasher) are completed within the configured time windows. + +- **interval**: Defines the time step in seconds between control actions + (e.g. `3600` for one hour, `900` for 15 minutes). + +:::{warning} +**Current Limitation** + +At present, the `interval` setting is **not used** by the genetic algorithm. Instead: + +- The control interval is fixed to **1 hour**. + +Support for configurable intervals (e.g. 15-minute steps) may be added in a future release. +::: + +#### Genetic Algorithm Parameters + +The behavior of the genetic algorithm can be customized using the following configuration options: + +- **individuals** (`int`, default: `300`): + Sets the number of individuals (candidate solutions) in the (first) generation. A higher number + increases solution diversity and the chance of finding a good result, but also increases + computation time. + +- **generations** (`int`, default: `400`): + Sets the number of generations to evaluate the optimal solution. In each generation, solutions are + evaluated and evolved. More generations can improve optimization quality but increase computation + time. Best results are usually found within a moderate number of generations. + +- **seed** (`int` or `null`, default: `null`): + Sets the random seed for reproducible results. + + - If `null`, a random seed is used (non-reproducible). + - If an integer is provided, it ensures that the same optimization input yields the same output. + + A fixed seed to ensure reproducibility. Runs with the same seed and configuration will + produce the same results. + +- **penalties** (`dict`): + Defines how penalties are applied to solutions that violate constraints (e.g., undercharged + batteries). Penalty function parameter values influence the fitness score, discouraging + undesirable solutions. + +:::{note} +**Supported Penalty Functions** + +Currently, the only supported penalty function parameter is: + +- `ev_soc_miss`: + Applies a penalty when the **state of charge (SOC)** of the electric vehicle battery falls below + the required minimum. This encourages the optimizer to ensure sufficient EV charging. +::: + +#### Value Formats + +- **Time-related values**: + - `hours`: specified in **hours** (e.g. `24`) + - `interval`: specified in **seconds** (e.g. `3600`) + +- **Genetic algorithm parameters**: + - `individuals`: must be an **integer** + - `seed`: must be an **integer** or `null` for random behavior + +- **Penalty function parameter values**: may be `float`, `int`, or `string`, depending on the type + of penalty function. + +#### Optimization configuration example + +```json +{ + "optimization": { + "hours": 24, + "interval": 3600, + "genetic" : { + "individuals": 300, + "generations": 400, + "seed": null, + "penalties": { + "ev_soc_miss": 10 + } + } + } +} +``` + +### Device simulation configuration + +The device simulations are used to evaluate the fitness of the individuals of the solution +population. + +The GENETIC algorithm supports 4 devices: + +- **inverter**: A photovoltaic power inverter that can export to the grid and charge a battery. + The inverter is mandatory. +- **electric_vehicle**: An electric vehicle, basically the battery of an electric vehicle. The + The electrical vehicle is optional. +- **battery**: A battery that can be charged by the inverter. The battery is mandatory. +- **home_appliance**: A home appliance, like a washing machine or a dish washer. The home + appliance is optional. + +:::{admonition} Warning +:class: warning +The GENETIC algorithm can only use the first inverter, electrical vehicle, battery, home appliance +that is configured, even if more devices are configured. +::: + +#### Inverter simulation configuration + +**Example:** + +```json +{ + "devices": { + "max_inverters": 1, + "inverters": [ + { + "device_id": "inv1", + "max_power_w": 10000, + "battery_id": "bat1" + } + ] + } +} +``` + +#### Electric vehicle simulation configuration + +**Example:** + +```json +{ + "devices": { + "max_electric_vehicles": 1, + "electric_vehicles": [ + { + "device_id": "ev1", + "capacity_wh": 50000, + "max_charge_power_w": 10000, + "charge_rates": [0.0, 0.25, 0.5, 0.75, 1.0], + "min_soc_percentage": 10, + "max_soc_percentage": 80 + } + ] + }, + "measurement": { + "electric_vehicle_soc_keys": ["ev1_soc"] + } +} +``` + +#### Battery simulation configuration + +**Example:** + +```json +{ + "devices": { + "max_batteries": 1, + "batteries": [ + { + "device_id": "battery1", + "capacity_wh": 8000, + "charging_efficiency": 0.88, + "discharging_efficiency": 0.88, + "levelized_cost_of_storage_kwh": 0.12, + "max_charge_power_w": 8000, + "min_charge_power_w": 50, + "charge_rates": null, + "min_soc_percentage": 5, + "max_soc_percentage": 95 + } + ] + } +} +``` + +#### Home appliance simulation configuration + +**Example:** + +```json +{ + "devices": { + "max_home_appliances": 1, + "home_appliances": [ + { + "device_id": "washing machine", + "consumption_wh": 600, + "duration_h": 3, + "time_windows": null, + } + ] + } +} +``` + +The time windows the home appliance may run can be [configured](#configtimewindow-page) in several +ways. See the [time window configuration](#configtimewindow-page) for details. + +## Predictions configuration + +The device simulation may rely on predictions to simulate proper behaviour. E.g. the inverter needs +to know the PV forecast. + +Configure the [predictions](#prediction-page) as described on the [prediction page](#prediction-page). + +### Providing your own prediction data + +If EOS does not have a suitable prediction provider you can provide your own data for a prediction. +Configure the respective import provider (ElecPriceImport, LoadImport, PVForecastImport, +WeatherImport) and use one of the following endpoints to provide your own data: + +- **PUT** `/v1/prediction/import/ElecPriceImport` +- **PUT** `/v1/prediction/import/LoadImport` +- **PUT** `/v1/prediction/import/PVForecastImport` +- **PUT** `/v1/prediction/import/WeatherImport` + +## Measurement configuration + +Predictions and device simulations often rely on **measurement data** to produce accurate results. +For example: + +- A **load forecast** requires past energy meter readings. +- A **battery simulation** needs the current **state of charge (SoC)** to start from the correct + condition. + +Before using these features, make sure to configure the [measurement](#measurement-page) as +described on the [measurement page](#measurement-page). + +### Providing your own measurement data + +You can provide your own measurement data to the prediction and simulation engine through the +following REST endpoints (see the [measurement page](#measurement-page) for details on the data +format): + +- **PUT** `/v1/measurement/data` +- **PUT** `/v1/measurement/dataframe` +- **PUT** `/v1/measurement/series` +- **PUT** `/v1/measurement/value` + +### Example: Supplying Battery and EV SoC + +For **batteries** and **electric vehicles**, it is strongly recommended to provide +**current SoC**. This ensures that simulations start with the correct state. + +The simplest way is to use the `/v1/measurement/value` endpoint. +Assuming the battery is named `battery1` and the EV is named `ev11`: + +1. **Use the measurement keys** that are pre-configured for your **devices**. For example: + + ```json + { + "devices": { + "batteries": [ + { + "device_id": "battery1", "capacity_wh": 8000, ... + "measurement_key_soc_factor": "battery1-soc-factor", ... + } + ], + "electric_vehicles": [ + { + "device_id": "ev11", "capacity_wh": 8000, ... + "measurement_key_soc_factor": "ev11-soc-factor", ... + } + ] + } + } + ``` + +2. **Record your SoC readings** to these keys. + + - Enter the values as **factor of total capacity** of the respective **battery**. + +In these examples: + +- datetime specifies the timestamp of the measurement. +- key is the measurement key (e.g. battery1-soc-factor). +- value is the numeric measurement value (e.g. SoC as factor of total capacity). + +#### Raw HTTP request + +```http +PUT http://127.0.0.1:8503/v1/measurement/value?datetime=2025-09-26T16%3A39&key=battery1-soc-factor&value=0.57 +PUT http://127.0.0.1:8503/v1/measurement/value?datetime=2025-09-26T16%3A39&key=ev11-soc-factor&value=0.22 +``` + +#### Equivalent curl commands + +```bash +curl -X PUT "http://127.0.0.1:8503/v1/measurement/value?datetime=2025-09-26T16%3A39&key=battery1-soc-factor&value=0.57" +curl -X PUT "http://127.0.0.1:8503/v1/measurement/value?datetime=2025-09-26T16%3A39&key=ev11-soc-factor&value=0.22" +``` + +### Example: Supplying Load Data + +To provide your actual load measurements in Akkudoktor-EOS: + +1. **Configure the measurement keys** for your load energy meters. For example: + + ```json + { + "measurements": { + "load_emr_keys": ["my_load_meter_reading", "my_other_load_meter_reading"] + } + } + ``` + +2. **Record your meter readings** to these keys. + + - Enter the values exactly as your energy meters report them, in **kWh**. + - Use the same approach as when supplying battery or EV SoC data. diff --git a/docs/akkudoktoreos/optimization.md b/docs/akkudoktoreos/optimpost.md similarity index 56% rename from docs/akkudoktoreos/optimization.md rename to docs/akkudoktoreos/optimpost.md index 5f7c7d1..827ddba 100644 --- a/docs/akkudoktoreos/optimization.md +++ b/docs/akkudoktoreos/optimpost.md @@ -1,12 +1,22 @@ % SPDX-License-Identifier: Apache-2.0 -# Optimization +# `POST /optimize` Optimization ## Introduction The `POST /optimize` API endpoint optimizes your energy management system based on various inputs including electricity prices, battery storage capacity, PV forecast, and temperature data. +The `POST /optimize` optimization interface is the "classical" interface developed by Andreas at the +start of the projects and used and described in his videos. It allows and requires to define all the +optimization paramters on the endpoint request. + +:::{admonition} Warning +:class: warning +The `POST /optimize` endpoint interface does not regard configurations set for the parameters +passed to the request. You have to set the parameters even if given in the configuration. +::: + ## Input Payload ### Sample Request @@ -14,38 +24,70 @@ including electricity prices, battery storage capacity, PV forecast, and tempera ```json { "ems": { - "preis_euro_pro_wh_akku": 0.0007, - "einspeiseverguetung_euro_pro_wh": 0.00007, - "gesamtlast": [500, 500, ..., 500, 500], - "pv_prognose_wh": [300, 0, 0, ..., 2160, 1840], - "strompreis_euro_pro_wh": [0.0003784, 0.0003868, ..., 0.00034102, 0.00033709] + "preis_euro_pro_wh_akku": 0.0001, + "einspeiseverguetung_euro_pro_wh": [ + 0.00007, 0.00007, 0.00007, 0.00007, 0.00007, 0.00007, 0.00007, + 0.00007, 0.00007, 0.00007, 0.00007, 0.00007, 0.00007, 0.00007, + 0.00007, 0.00007, 0.00007, 0.00007, 0.00007, 0.00007, 0.00007, + 0.00007, 0.00007, 0.00007, 0.00007, 0.00007, 0.00007, 0.00007, + 0.00007, 0.00007, 0.00007, 0.00007, 0.00007, 0.00007, 0.00007, + 0.00007, 0.00007, 0.00007, 0.00007, 0.00007, 0.00007, 0.00007, + 0.00007, 0.00007, 0.00007, 0.00007, 0.00007, 0.00007 + ], + "gesamtlast": [ + 676.71, 876.19, 527.13, 468.88, 531.38, 517.95, 483.15, 472.28, + 1011.68, 995.00, 1053.07, 1063.91, 1320.56, 1132.03, 1163.67, + 1176.82, 1216.22, 1103.78, 1129.12, 1178.71, 1050.98, 988.56, 912.38, + 704.61, 516.37, 868.05, 694.34, 608.79, 556.31, 488.89, 506.91, + 804.89, 1141.98, 1056.97, 992.46, 1155.99, 827.01, 1257.98, 1232.67, + 871.26, 860.88, 1158.03, 1222.72, 1221.04, 949.99, 987.01, 733.99, + 592.97 + ], + "pv_prognose_wh": [ + 0, 0, 0, 0, 0, 0, 0, 8.05, 352.91, 728.51, 930.28, 1043.25, 1106.74, + 1161.69, 6018.82, 5519.07, 3969.88, 3017.96, 1943.07, 1007.17, + 319.67, 7.88, 0, 0, 0, 0, 0, 0, 0, 0, 0, 5.04, 335.59, 705.32, + 1121.12, 1604.79, 2157.38, 1433.25, 5718.49, 4553.96, 3027.55, + 2574.46, 1720.4, 963.4, 383.3, 0, 0, 0 + ], + "strompreis_euro_pro_wh": [ + 0.0003384, 0.0003318, 0.0003284, 0.0003283, 0.0003289, 0.0003334, + 0.0003290, 0.0003302, 0.0003042, 0.0002430, 0.0002280, 0.0002212, + 0.0002093, 0.0001879, 0.0001838, 0.0002004, 0.0002198, 0.0002270, + 0.0002997, 0.0003195, 0.0003081, 0.0002969, 0.0002921, 0.0002780, + 0.0003384, 0.0003318, 0.0003284, 0.0003283, 0.0003289, 0.0003334, + 0.0003290, 0.0003302, 0.0003042, 0.0002430, 0.0002280, 0.0002212, + 0.0002093, 0.0001879, 0.0001838, 0.0002004, 0.0002198, 0.0002270, + 0.0002997, 0.0003195, 0.0003081, 0.0002969, 0.0002921, 0.0002780 + ] }, "pv_akku": { "device_id": "battery1", - "capacity_wh": 12000, - "charging_efficiency": 0.92, - "discharging_efficiency": 0.92, - "max_charge_power_w": 5700, - "initial_soc_percentage": 66, - "min_soc_percentage": 5, - "max_soc_percentage": 100 + "capacity_wh": 26400, + "max_charge_power_w": 5000, + "initial_soc_percentage": 80, + "min_soc_percentage": 15 }, "inverter": { "device_id": "inverter1", - "max_power_wh": 15500 - "battery_id": "battery1", + "max_power_wh": 10000, + "battery_id": "battery1" }, "eauto": { - "device_id": "auto1", - "capacity_wh": 64000, - "charging_efficiency": 0.88, - "discharging_efficiency": 0.88, + "device_id": "ev1", + "capacity_wh": 60000, + "charging_efficiency": 0.95, + "discharging_efficiency": 1.0, "max_charge_power_w": 11040, - "initial_soc_percentage": 98, - "min_soc_percentage": 60, - "max_soc_percentage": 100 + "initial_soc_percentage": 54, + "min_soc_percentage": 0 }, - "temperature_forecast": [18.3, 18, ..., 20.16, 19.84], + "temperature_forecast": [ + 18.3, 17.8, 16.9, 16.2, 15.6, 15.1, 14.6, 14.2, 14.3, 14.8, 15.7, 16.7, 17.4, + 18.0, 18.6, 19.2, 19.1, 18.7, 18.5, 17.7, 16.2, 14.6, 13.6, 13.0, 12.6, 12.2, + 11.7, 11.6, 11.3, 11.0, 10.7, 10.2, 11.4, 14.4, 16.4, 18.3, 19.5, 20.7, 21.9, + 22.7, 23.1, 23.1, 22.8, 21.8, 20.2, 19.1, 18.0, 17.4 + ], "start_solution": null } ``` @@ -58,7 +100,8 @@ including electricity prices, battery storage capacity, PV forecast, and tempera - Unit: €/Wh - Purpose: Represents the residual value of energy stored in the battery -- Impact: Lower values encourage battery depletion, higher values preserve charge at the end of the simulation. +- Impact: Lower values encourage battery depletion, higher values preserve charge at the end of the + simulation. #### Feed-in Tariff (`einspeiseverguetung_euro_pro_wh`) @@ -162,23 +205,24 @@ Verify prices against your local tariffs. #### Battery Control -- `ac_charge`: Grid charging schedule (0-1) +- `ac_charge`: Grid charging schedule (0.0-1.0) - `dc_charge`: DC charging schedule (0-1) - `discharge_allowed`: Discharge permission (0 or 1) 0 (no charge) 1 (charge with full load) -`ac_charge` multiplied by the maximum charge power of the battery results in the planned charging power. +`ac_charge` multiplied by the maximum charge power of the battery results in the planned charging +power. #### EV Charging -- `eautocharge_hours_float`: EV charging schedule (0-1) +- `eautocharge_hours_float`: EV charging schedule (0.0-1.0) #### Results -The `result` object contains detailed information about the optimization outcome. -The length of the array is between 25 and 48 and starts at the current hour and ends at 23:00 tomorrow. +The `result` object contains detailed information about the optimization outcome. The length of the +array is between 25 and 48 and starts at the current hour and ends at 23:00 tomorrow. - `Last_Wh_pro_Stunde`: Array of hourly load values in Wh - Shows the total energy consumption per hour diff --git a/docs/akkudoktoreos/prediction.md b/docs/akkudoktoreos/prediction.md index ac68196..4acf043 100644 --- a/docs/akkudoktoreos/prediction.md +++ b/docs/akkudoktoreos/prediction.md @@ -8,21 +8,21 @@ optimization is executed. In EOS, a standard set of predictions is managed, incl - Household Load Prediction - Electricity Price Prediction +- Feed In Tariff Prediction - PV Power Prediction - Weather Prediction ## Storing Predictions EOS stores predictions in a **key-value store**, where the term `prediction key` refers to the -unique key used to retrieve specific prediction data. The key-value store is in memory. Stored -data is lost on re-start of the EOS REST server. +unique key used to retrieve specific prediction data. ## Prediction Providers Most predictions can be sourced from various providers. The specific provider to use is configured in the EOS configuration and can be set by prediction type. For example: -```python +```json { "weather": { "provider": "ClearOutside" @@ -48,7 +48,7 @@ The prediction data must be provided in one of the following formats: A dictionary with the following structure: -```python +```json { "start_datetime": "2024-01-01 00:00:00", "interval": "1 Hour", @@ -170,6 +170,26 @@ The electricity proce forecast data must be provided in one of the formats descr The data may additionally or solely be provided by the **PUT** `/v1/prediction/import/ElecPriceImport` endpoint. +## Feed In Tariff Prediction + +Prediction keys: + +- `feed_in_tarif_wh`: Feed in tarif per Wh (€/Wh). +- `feed_in_tarif_kwh`: Feed in tarif per kWh (€/kWh) + +Configuration options: + +- `feedintarif`: Feed in tariff configuration. + + - `provider`: Feed in tariff provider id of provider to be used. + + - `FeedInTariffFixed`: Provides fixed feed in tariff values. + - `FeedInTariffImport`: Imports from a file or JSON string. + + - `provider_settings.feed_in_tariff_kwh`: Fixed feed in tariff (€/kWh). + - `provider_settings.import_file_path`: Path to the file to import feed in tariff forecast data from. + - `provider_settings.import_json`: JSON string, dictionary of feed in tariff value lists. + ## Load Prediction Prediction keys: diff --git a/docs/akkudoktoreos/resource.md b/docs/akkudoktoreos/resource.md new file mode 100644 index 0000000..2e063fb --- /dev/null +++ b/docs/akkudoktoreos/resource.md @@ -0,0 +1,258 @@ +% SPDX-License-Identifier: Apache-2.0 +(resource-page)= + +# Resources (Device Simulations) + +## Concepts + +The simulations for resources are leaning on general concepts of the [S2 standard]. + +### Control Types + +The control of resources and such what a resource simulation will simulate follows three +basic control principles: + +- Operation Mode Based Control (OMBC) +- Fill Rate Based Control (FRBC) +- Demand Driven Based Control (DDBC) + +Although these control principles differ enough to separate them into three distinct control types, +there are some common aspects that make them similar: + +- Operation Modes +- Transitions and +- Timers. + +The objective for a control type is under which circumstances what things can be adjusted, and what +the constraints are for these adjustments. The three control types model a virtual, abstract resource +for simulation. + +The abstract resource ignores all details of pyhsical device that are not relevant to energy +management. In addition, physical devices have an enormous variety in parameters, sensors, control +strategies, concerns, safeguards, and so on. It would be practically impossible to develop a +simulation that can +understand all the parameters of all the physical devices on the market. By making the resource more +abstract, its concepts can be translated to all sorts of physical devices, even though internally +they function very differently. As a consequence, it not always possible to make a 100% accurate +description of all the behaviors and constraints in these abstractions. But the abstractions used +in the control types are quite powerful, and should allow you to come pretty close. + +The control types basically define how the simulated resource can be described. The user in the end +selects the proper desciption of a physical device using the configuration options provided for +resource simulations. The configuration sets how the simulated resource functions, what it can do and +what kind of constraints it has. + +### Resource Simulation + +Based on the description of this virtual resource, the resource simulation can make predictions of +what the physical device will do in certain situations, and when it is allowed to execute +instructions generated by the optimization as part of the energy management plan evaluation. + +### Resource Status + +Once the physical device has changed it's behavior, the resource simulation should be informed +to make the simulation change it's state accordingly. + +The actual state of a pyhsical device may be reported to the resource simulation by the +**PUT** `/v1/resource/status` API endpoint. + +## Battery + +There is a wealth of possible battery operation modes: + + +| Mode | Purpose / Behavior | Typical Trigger / Context | +| ------------------------- | --------------------------------------------------------------------------------------- | ---------------------------------------------------------------- | +| **IDLE** | Battery neither charges nor discharges (SOC stable). | No active control objective or power imbalance below thresholds. | +| **SELF_CONSUMPTION** | Charge from PV surplus and discharge to cover local load. | PV generation > load (charge) or load > PV (discharge). | +| **NON_EXPORT** | Charge from on-site or local surplus with the goal of minimizing or preventing energy export to the external grid. Discharging to the grid is not allowed. | Export limit reached and SOC < SOC_max. | +| **PEAK_SHAVING** | Discharge to keep grid import below a target threshold. | Predicted or measured site load exceeds peak limit. | +| **GRID_SUPPORT_EXPORT** | Discharge energy to grid for revenue (V2G, wholesale market, flexibility service). | Market or signal permits profitable export. | +| **GRID_SUPPORT_IMPORT** | Charge from grid to absorb surplus or provide up-regulation service. | Low-price or grid-support signal detected. | +| **FREQUENCY_REGULATION** | Rapid charge/discharge response to grid frequency deviations. | Active participation in frequency control. | +| **RAMP_RATE_CONTROL** | Smooth site-level power ramp rates by buffering fluctuations. | Sudden PV/load change exceeding ramp limit. | +| **RESERVE_BACKUP** | Maintain SOC ≥ reserve threshold to ensure backup capacity. | Resilience mode active, grid operational. | +| **OUTAGE_SUPPLY** | Islanded operation: power local loads using stored energy (and PV if available). | Grid failure detected. | +| **FORCED_CHARGE** | Manual or external control command to charge (e.g., pre-event, maintenance). No discharge. | Operator or optimizer command. | +| **FORCED_DISCHARGE** | Manual or external control command to discharge. No charge. | Operator or optimizer command. | +| **FAULT** | Battery unavailable due to fault, safety, or protection state. | Fault detected (thermal, voltage, comms, etc.). | + + +The optimization algorithm, the device simulation and the configuration properties only support the +most important of these modes. + +### Battery Simulation + +The battery simulation assumes an idealized battery model. Under this model, the battery can be +operated in three discrete operation modes with fill rate based control (FRBC): + +| **Operation Mode ID** | **Description** | +| ------------------------ | --------------------------------------------------------------------- | +| **SELF_CONSUMPTION** | Charge from local surplus and discharge to cover local load. | +| **NON_EXPORT** | Charge from local surplus and do not discharge. | +| **FORCED_CHARGE** | Charge. | + +The **operation mode factor** (0.0–1.0) specifies the normalized power rate relative to the +battery's nominal maximum charge or discharge power. A value of 1.0 corresponds to full-rate +charging or discharging, while 0.0 indicates no power transfer. Intermediate values scale the power +proportionally. + +The **fill level** (0.0–1.0) specifies the normalized fill level relative to the +battery's nominal maximum charge. A value of 1.0 corresponds to full while 0.0 indicates empty. +Intermediate values scale the fill level proportionally. + +### Battery Configuration + +### Battery Stati + +To keep the battery simulation in synchonization with the actual stati of the battery the following +resource stati may be reported to EOS by the **PUT** `/v1/resource/status` API endpoint. + +#### Battery FRBCActuatorStatus + +The operation mode the battery is currently operated. + +```json +{ + "type": "FRBCActuatorStatus", + "active_operation_mode_id": "GRID_SUPPORT_IMPORT", + "operation_mode_factor": "0.375", + "previous_operation_mode_id": "SELF_CONSUMPTION", + "transistion_timestamp": "20250725T12:00:12" +} +``` + +#### Battery FRBCStorageStatus + +The current battery state of charge (SoC). + +```json +{ + "type": "FRBCStorageStatus", + "present_fill_level": "0.88" +} +``` + +#### Battery PowerMeasurement + +The current power that the battery is charged or discharged with \[W\]. + +```json +{ + "type": "PowerMeasurement", + "measurement_timestamp": "20250725T12:00:12", + "values": [ + { + "commodity_quantity": "ELECTRIC.POWER.L1", + "value": "887.5" + }, + { + "commodity_quantity": "ELECTRIC.POWER.L2", + "value": "905.5" + }, + { + "commodity_quantity": "ELECTRIC.POWER.L2", + "value": "1100.7" + }, + ] +} +``` + +For symmetric (or unknown) power distribution: + +```json +{ + "type": "PowerMeasurement", + "measurement_timestamp": "20250725T12:00:12", + "values": [ + { + "commodity_quantity": "ELECTRIC.POWER.3_PHASE_SYM", + "value": "1000" + } + ] +} +``` + +## Electric Vehicle + +The electric vehicle is basically a battery with a reduced set of operation modes. + +### Electric Vehicle Instructions + +The electric vehicle control instructions assume an idealized EV battery model. Under this model, +the EV battery can be operated in two operation modes: + +| **Operation Mode ID** | **Description** | +| --------------------- | ----------------------------------------------------------------------- | +| **IDLE** | Battery neither charges nor discharges; holds its state of charge. | +| **FORCED_CHARGE** | Charge at a specified power rate up to the allowable maximum. | + +The **operation mode factor** (0.0–1.0) specifies the normalized power rate relative to the +battery's nominal maximum charge power. A value of 1.0 corresponds to full-rate charging, while 0.0 +indicates no power transfer. Intermediate values scale the power proportionally. + +## Home Appliance + +The optimization algorithm supports one start of the home appliance within the optimization +horizon. + +### Home Appliance Simulation + +### Home Appliance Configuration + +Home appliance to run within the optimization horizon. + +```json +[ + { + "device_id": "dishwasher1", + "consumption_wh": 2000, + "duration_h": 3 + } +] +``` + +Home appliance to run within a time window of 5 hours starting at 8:00 every day and another time +window of 3 hours starting at 15:00 every day. See +[Time Window Sequence Configuration](configtimewindow-page) for more information. + +```json +[ + { + "device_id": "dishwasher1", + "consumption_wh": 2000, + "duration_h": 3, + "time_windows": { + "windows": [ + { + "start_time": "08:00", + "duration": "5 hours" + }, + { + "start_time": "15:00", + "duration": "3 hours" + } + ] + } + } +] +``` + +:::{admonition} Note +:class: note +The optimization algorithm always restricts to one start within the optimization horizon per +energy management run. +::: + +### Home Appliance Instructions + +The home appliance instructions assume an idealized home appliance model. Under this model, +the home appliance can be operated in two operation modes: + +| **Operation Mode ID** | **Description** | +|-----------------------|-------------------------------------------------------------------------| +| **RUN** | The home appliance is started and runs until the end of it's power | +| | sequence. | +| **IDLE** | The home appliance does not run. | + +The **operation mode factor** (0.0–1.0) is ignored. diff --git a/docs/akkudoktoreos/serverapi.md b/docs/akkudoktoreos/serverapi.md index 362da73..7f0dfda 100644 --- a/docs/akkudoktoreos/serverapi.md +++ b/docs/akkudoktoreos/serverapi.md @@ -1,4 +1,5 @@ % SPDX-License-Identifier: Apache-2.0 +(server-api-page)= # Server API diff --git a/docs/develop/CHANGELOG.md b/docs/develop/CHANGELOG.md new file mode 100644 index 0000000..44b4a4f --- /dev/null +++ b/docs/develop/CHANGELOG.md @@ -0,0 +1,4 @@ +```{include} ../../CHANGELOG.md +:relative-docs: ../ +:relative-images: +``` diff --git a/docs/develop/develop.md b/docs/develop/develop.md new file mode 100644 index 0000000..529fab4 --- /dev/null +++ b/docs/develop/develop.md @@ -0,0 +1,572 @@ +% SPDX-License-Identifier: Apache-2.0 +(develop-page)= + +# Development Guide + +## Development Prerequisites + +Have or +[create](https://docs.github.com/en/get-started/start-your-journey/creating-an-account-on-github) +a [GitHub](https://github.com/) account. + +Make shure all the source installation prequistes are installed. See the +[installation guideline](#install-page) for a detailed list of tools. + +Under Linux the [make](https://www.gnu.org/software/make/manual/make.html) tool should be installed +as we have a lot of pre-fabricated commands for it. + +Install your favorite editor or integrated development environment (IDE): + +- Full-Featured IDEs + + - [Eclipse + PyDev](https://www.pydev.org/) + - [KDevelop](https://www.kdevelop.org/) + - [PyCharm](https://www.jetbrains.com/pycharm/) + - ... + +- Code Editors with Python Support + + - [Visual Studio Code (VS Code)](https://code.visualstudio.com/) + - [Sublime Text](https://www.sublimetext.com/) + - [Atom / Pulsar](https://pulsar-edit.dev/) + - ... + +- Python-Focused or Beginner-Friendly IDEs + + - [Spyder](https://www.spyder-ide.org/) + - [Thonny](https://thonny.org/) + - [IDLE](https://www.python.org/downloads/) + - ... + +## Step 1 – Fork the Repository + +[Fork the EOS repository](https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/fork-a-repo) +to your GitHub account. + +Clone your fork locally and add the EOS upstream remote to track updates. + +```{eval-rst} +.. tabs:: + + .. tab:: Windows + + .. code-block:: powershell + + git clone https://github.com//EOS.git + cd EOS + git remote add eos https://github.com/Akkudoktor-EOS/EOS.git + + .. tab:: Linux + + .. code-block:: bash + + git clone https://github.com//EOS.git + cd EOS + git remote add eos https://github.com/Akkudoktor-EOS/EOS.git +``` + +Replace `` with your GitHub username. + +## Step 2 – Development Setup + +This is recommended for developers who want to modify the source code and test changes locally. + +### Step 2.1 – Create a Virtual Environment + +```{eval-rst} +.. tabs:: + + .. tab:: Windows + + .. code-block:: powershell + + python -m venv .venv + .venv\Scripts\pip install --upgrade pip + .venv\Scripts\pip install -r requirements-dev.txt + .venv\Scripts\pip install build + .venv\Scripts\pip install -e . + + .. tab:: Linux + + .. code-block:: bash + + python3 -m venv .venv + .venv/bin/pip install --upgrade pip + .venv/bin/pip install -r requirements-dev.txt + .venv/bin/pip install build + .venv/bin/pip install -e . + + .. tab:: Linux Make + + .. code-block:: bash + + make install +``` + +### Step 2.2 – Activate the Virtual Environment + +```{eval-rst} +.. tabs:: + + .. tab:: Windows + + .. code-block:: powershell + + .venv\Scripts\activate.bat + + .. tab:: Linux + + .. code-block:: bash + + source .venv/bin/activate +``` + +### Step 2.3 - Install pre-commit + +Our code style and commit message checks use [`pre-commit`](https://pre-commit.com). + +```{eval-rst} +.. tabs:: + + .. tab:: Windows + + .. code-block:: powershell + + pre-commit install + pre-commit install --hook-type commit-msg --hook-type pre-push + + .. tab:: Linux + + .. code-block:: bash + + pre-commit install + pre-commit install --hook-type commit-msg --hook-type pre-push +``` + +## Step 3 - Run EOS + +Make EOS accessible at [http://localhost:8503/docs](http://localhost:8503/docs) and EOSdash at +[http://localhost:8504](http://localhost:8504). + +### Option 1 – Using Python Virtual Environment + +```{eval-rst} +.. tabs:: + + .. tab:: Windows + + .. code-block:: powershell + + python -m akkudoktoreos.server.eos + + .. tab:: Linux + + .. code-block:: bash + + python -m akkudoktoreos.server.eos + + .. tab:: Linux Make + + .. code-block:: bash + + make run +``` + +To have full control of the servers during development you may start the servers independently - +e.g. in different terminal windows. Don't forget to activate the virtual environment in your +terminal window. + +:::{admonition} Note +:class: note +If you killed or stopped the servers shortly before, the ports may still be occupied by the last +processes. It may take more than 60 seconds until the ports are released. +::: + +You may add the `--reload true` parameter to have the servers automatically restarted on source code +changes. It is best to also add `--startup_eosdash false` to EOS to prevent the automatic restart +interfere with the EOS server trying to start EOSdash. + + +```{eval-rst} +.. tabs:: + + .. tab:: Windows + + .. code-block:: powershell + + python -m akkudoktoreos.server.eosdash --host localhost --port 8504 --log_level DEBUG --reload true + + .. tab:: Linux + + .. code-block:: bash + + python -m akkudoktoreos.server.eosdash --host localhost --port 8504 --log_level DEBUG --reload true + + .. tab:: Linux Make + + .. code-block:: bash + + make run-dash-dev +``` + +```{eval-rst} +.. tabs:: + + .. tab:: Windows + + .. code-block:: powershell + + python -m akkudoktoreos.server.eos --host localhost --port 8503 --log_level DEBUG --startup_eosdash false --reload true + + .. tab:: Linux + + .. code-block:: bash + + python -m akkudoktoreos.server.eos --host localhost --port 8503 --log_level DEBUG --startup_eosdash false --reload true + + .. tab:: Linux Make + + .. code-block:: bash + + make run-dev +``` + + +### Option 2 – Using Docker + +#### Step 3.1 – Build the Docker Image + +```{eval-rst} +.. tabs:: + + .. tab:: Windows + + .. code-block:: powershell + + docker build -t akkudoktoreos . + + .. tab:: Linux + + .. code-block:: bash + + docker build -t akkudoktoreos . +``` + +#### Step 3.2 – Run the Container + +```{eval-rst} +.. tabs:: + + .. tab:: Windows + + .. code-block:: powershell + + docker run -d ` + --name akkudoktoreos ` + -p 8503:8503 ` + -p 8504:8504 ` + -e OPENBLAS_NUM_THREADS=1 ` + -e OMP_NUM_THREADS=1 ` + -e MKL_NUM_THREADS=1 ` + -e EOS_SERVER__HOST=0.0.0.0 ` + -e EOS_SERVER__PORT=8503 ` + -e EOS_SERVER__EOSDASH_HOST=0.0.0.0 ` + -e EOS_SERVER__EOSDASH_PORT=8504 ` + --ulimit nproc=65535:65535 ` + --ulimit nofile=65535:65535 ` + --security-opt seccomp=unconfined ` + akkudoktor-eos:latest + + .. tab:: Linux + + .. code-block:: bash + + docker run -d \ + --name akkudoktoreos \ + -p 8503:8503 \ + -p 8504:8504 \ + -e OPENBLAS_NUM_THREADS=1 \ + -e OMP_NUM_THREADS=1 \ + -e MKL_NUM_THREADS=1 \ + -e EOS_SERVER__HOST=0.0.0.0 \ + -e EOS_SERVER__PORT=8503 \ + -e EOS_SERVER__EOSDASH_HOST=0.0.0.0 \ + -e EOS_SERVER__EOSDASH_PORT=8504 \ + --ulimit nproc=65535:65535 \ + --ulimit nofile=65535:65535 \ + --security-opt seccomp=unconfined \ + akkudoktor-eos:latest +``` + +#### Step 3.3 – Manage the Container + +```{eval-rst} +.. tabs:: + + .. tab:: Windows + + .. code-block:: powershell + + docker logs -f akkudoktoreos + docker stop akkudoktoreos + docker start akkudoktoreos + docker rm -f akkudoktoreos + + .. tab:: Linux + + .. code-block:: bash + + docker logs -f akkudoktoreos + docker stop akkudoktoreos + docker start akkudoktoreos + docker rm -f akkudoktoreos +``` + +For detailed Docker instructions, refer to +**[Method 3 & 4: Installation with Docker](install.md#method-3-installation-with-docker-dockerhub)** +and +**[Method 4: Docker Compose](install.md#method-4-installation-with-docker-docker-compose)**. + +### Step 4 - Create the changes + +#### Step 4.1 - Create a development branch + +```bash +git checkout -b +``` + +Replace `` with the development branch name. The branch name shall be of the +format (feat|fix|chore|docs|refactor|test)/[a-z0-9._-]+, e.g: + +- feat/my_cool_new_feature +- fix/this_annoying_bug +- ... + +#### Step 4.2 – Edit the sources + +Use your fovourite editor or IDE to edit the sources. + +#### Step 4.3 - Check the source code for correct format + +```{eval-rst} +.. tabs:: + + .. tab:: Windows + + .. code-block:: powershell + + pre-commit run --all-files + + .. tab:: Linux + + .. code-block:: bash + + pre-commit run --all-files + + .. tab:: Linux Make + + .. code-block:: bash + + make format +``` + +#### Step 4.4 - Test the changes + +At a minimum, you should run the module tests: + +```{eval-rst} +.. tabs:: + + .. tab:: Windows + + .. code-block:: powershell + + pytest -vs --cov src --cov-report term-missing + + .. tab:: Linux + + .. code-block:: bash + + pytest -vs --cov src --cov-report term-missing + + .. tab:: Linux Make + + .. code-block:: bash + + make test +``` + +You should also run the system tests. These include additional tests that interact with real +resources: + +```{eval-rst} +.. tabs:: + + .. tab:: Windows + + .. code-block:: powershell + + pytest --system-test -vs --cov src --cov-report term-missing + + .. tab:: Linux + + .. code-block:: bash + + pytest --system-test -vs --cov src --cov-report term-missing + + .. tab:: Linux Make + + .. code-block:: bash + + make test-system +``` + +#### Step 4.5 - Commit the changes + +Add the changed and new files to the commit. + +Create a commit. + +### Step 5 - Pull request + +Before creating a pull request assure the changes are based on the latest EOS upstream. + +Update your local main branch: + +```bash +git checkout main +git pull eos main +``` + +Switch back to your local development branch and rebase to main. + +```bash +git checkout +git rebase -i main +``` + +During rebase you can also squash your changes into one (preferred) or a set of commits that have +proper commit messages and can easily be reviewed. + +After rebase run the tests once again. + +If everything is ok push the commit(s) to your fork on Github. + +```bash +git push -f origin +``` + +If your push by intention does not comply to the rules you can skip the verification by: + +```bash +git push -f --no-verify origin +``` + + +Once ready, [submit a pull request](https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/proposing-changes-to-your-work-with-pull-requests/creating-a-pull-request-from-a-fork) +with your fork to the [Akkudoktor-EOS/EOS@master](https://github.com/Akkudoktor-EOS/EOS) repository. + + +## Developer Tips + +### Keep Your Fork Updated + +Regularly pull changes from the eos repository to avoid merge conflicts: + +```{eval-rst} +.. tabs:: + + .. tab:: Windows + + .. code-block:: powershell + + git checkout main + git pull eos main + git push origin + + .. tab:: Linux + + .. code-block:: bash + + git checkout main + git pull eos main + git push origin +``` + +Rebase your development branch to the latest eos main branch. + +```{eval-rst} +.. tabs:: + + .. tab:: Windows + + .. code-block:: powershell + + git checkout + git rebase -i main + + .. tab:: Linux + + .. code-block:: bash + + git checkout + git rebase -i main +``` + +### Create Feature Branches + +Work in separate branches for each feature or bug fix: + +```{eval-rst} +.. tabs:: + + .. tab:: Windows + + .. code-block:: powershell + + git checkout -b feat/my-feature + + .. tab:: Linux + + .. code-block:: bash + + git checkout -b feat/my-feature +``` + +### Run Tests Frequently + +Ensure your changes do not break existing functionality: + +```{eval-rst} +.. tabs:: + + .. tab:: Windows + + .. code-block:: powershell + + pytest -vs --cov src --cov-report term-missing + + .. tab:: Linux + + .. code-block:: bash + + pytest -vs --cov src --cov-report term-missing + + .. tab:: Linux Make + + .. code-block:: bash + + make test +``` + +### Follow Coding Standards + +Keep your code consistent with existing style and conventions. + +### Use Issues for Discussion + +Before making major changes, open an issue or discuss with maintainers. + +### Document Changes + +Update docstrings, comments, and any relevant documentation. diff --git a/docs/develop/getting_started.md b/docs/develop/getting_started.md index 037da62..20069af 100644 --- a/docs/develop/getting_started.md +++ b/docs/develop/getting_started.md @@ -1,127 +1,86 @@ +% SPDX-License-Identifier: Apache-2.0 +(getting-started-page)= + # Getting Started -## Installation +## Installation and Running -The project requires Python 3.10 or newer. Currently there are no official packages or images published. +AkkudoktorEOS can be installed and run using several different methods: -Following sections describe how to locally start the EOS server on `http://localhost:8503`. +- **Release package** (for stable versions) +- **Docker image** (for easy deployment) +- **From source** (for developers) -### Run from source +See the [installation guideline](#install-page) for detailed instructions on each method. -Install the dependencies in a virtual environment: +### Where to Find AkkudoktorEOS -```{eval-rst} -.. tabs:: - - .. tab:: Windows - - .. code-block:: powershell - - python -m venv .venv - .venv\Scripts\pip install -r requirements.txt - .venv\Scripts\pip install -e . - - .. tab:: Linux - - .. code-block:: bash - - python -m venv .venv - .venv/bin/pip install -r requirements.txt - .venv/bin/pip install -e . - -``` - -Start the EOS fastapi server: - -```{eval-rst} -.. tabs:: - - .. tab:: Windows - - .. code-block:: powershell - - .venv\Scripts\python src/akkudoktoreos/server/eos.py - - .. tab:: Linux - - .. code-block:: bash - - .venv/bin/python src/akkudoktoreos/server/eos.py - -``` - -### Docker - -```{eval-rst} -.. tabs:: - - .. tab:: Windows - - .. code-block:: powershell - - docker compose up --build - - .. tab:: Linux - - .. code-block:: bash - - docker compose up --build - -``` +- **Release Packages**: [GitHub Releases](https://github.com/Akkudoktor-EOS/EOS/releases) +- **Docker Images**: [Docker Hub](https://hub.docker.com/r/akkudoktor/eos) +- **Source Code**: [GitHub Repository](https://github.com/Akkudoktor-EOS/EOS) ## Configuration -This project uses the `EOS.config.json` file to manage configuration settings. +AkkudoktorEOS uses the `EOS.config.json` file to manage all configuration settings. ### Default Configuration -A default configuration file `default.config.json` is provided. This file contains all the necessary -configuration keys with their default values. +If essential configuration settings are missing, the application automatically uses a default +configuration to get you started quickly. -### Custom Configuration +### Custom Configuration Directory -Users can specify a custom configuration directory by setting the environment variable `EOS_DIR`. +You can specify a custom location for your configuration by setting the `EOS_DIR` environment +variable: -- If the directory specified by `EOS_DIR` contains an existing `EOS.config.json` file, the - application will use this configuration file. -- If the `EOS.config.json` file does not exist in the specified directory, the `default.config.json` - file will be copied to the directory as `EOS.config.json`. +```bash +export EOS_DIR=/path/to/your/config +``` -### Configuration Updates +**How it works:** -If the configuration keys in the `EOS.config.json` file are missing or different from those in -`default.config.json`, they will be automatically updated to match the default settings, ensuring -that all required keys are present. +- **If `EOS.config.json` exists** in the `EOS_DIR` directory → the application uses this + configuration +- **If `EOS.config.json` doesn't exist** → the application copies `default.config.json` to `EOS_DIR` + as `EOS.config.json` -## Classes and Functionalities +### Creating Your Configuration -This project uses various classes to simulate and optimize the components of an energy system. Each -class represents a specific aspect of the system, as described below: +There are three ways to configure AkkudoktorEOS: -- `Battery`: Simulates a battery storage system, including capacity, state of charge, and now - charge and discharge losses. +1. **EOSdash (Recommended)** - The easiest method is to use the web-based dashboard at + [http://localhost:8504](http://localhost:8504) -- `PVForecast`: Provides forecast data for photovoltaic generation, based on weather data and - historical generation data. +2. **Manual editing** - Create or edit the `EOS.config.json` file directly in your preferred text + editor -- `Load`: Models the load requirements of a household or business, enabling the prediction of future - energy demand. +3. **Server API** - Programmatically change configuration through the [server API](#server-api-page) -- `Heatpump`: Simulates a heat pump, including its energy consumption and efficiency under various - operating conditions. +For a complete reference of all available configuration options, see the [configuration guideline](#configuration-page). -- `Strompreis`: Provides information on electricity prices, enabling optimization of energy - consumption and generation based on tariff information. +## Quick Start Example -- `EMS`: The Energy Management System (EMS) coordinates the interaction between the various - components, performs optimization, and simulates the operation of the entire energy system. +```bash +# Pull the latest docker image +docker pull akkudoktor/eos:latest -These classes work together to enable a detailed simulation and optimization of the energy system. -For each class, specific parameters and settings can be adjusted to test different scenarios and -strategies. +# Run the application +docker run -d \ + --name akkudoktoreos \ + -p 8503:8503 \ + -p 8504:8504 \ + -e OPENBLAS_NUM_THREADS=1 \ + -e OMP_NUM_THREADS=1 \ + -e MKL_NUM_THREADS=1 \ + -e EOS_SERVER__HOST=0.0.0.0 \ + -e EOS_SERVER__PORT=8503 \ + -e EOS_SERVER__EOSDASH_HOST=0.0.0.0 \ + -e EOS_SERVER__EOSDASH_PORT=8504 \ + --ulimit nproc=65535:65535 \ + --ulimit nofile=65535:65535 \ + --security-opt seccomp=unconfined \ + akkudoktor/eos:latest -### Customization and Extension - -Each class is designed to be easily customized and extended to integrate additional functions or -improvements. For example, new methods can be added for more accurate modeling of PV system or -battery behavior. Developers are invited to modify and extend the system according to their needs. +# Access the dashboard +open http://localhost:8504 +``` diff --git a/docs/develop/install.md b/docs/develop/install.md new file mode 100644 index 0000000..7875969 --- /dev/null +++ b/docs/develop/install.md @@ -0,0 +1,288 @@ +% SPDX-License-Identifier: Apache-2.0 +(install-page)= + +# Installation Guide + +This guide provides four different methods to install AkkudoktorEOS. Choose the method that best +suits your needs. + +## Installation Prerequisites + +Before installing, ensure you have the following: + +- **For Source/Release Installation:** + - Python 3.10 or higher + - pip (Python package manager) + - Git (for source installation) + - Tar/Zip (for release package installation) + +- **For Docker Installation:** + - Docker Engine 20.10 or higher + - Docker Compose (optional, but recommended) + +## Method 1: Installation from Source (GitHub) + +This method is recommended for developers or users who want the latest features. + +### M1-Step 1: Clone the Repository + +```{eval-rst} +.. tabs:: + + .. tab:: Windows + + .. code-block:: powershell + + git clone https://github.com/Akkudoktor-EOS/EOS.git + cd EOS + + .. tab:: Linux + + .. code-block:: bash + + git clone https://github.com/Akkudoktor-EOS/EOS.git + cd EOS +``` + +### M1-Step 2: Create a Virtual Environment and install dependencies + +```{eval-rst} +.. tabs:: + + .. tab:: Windows + + .. code-block:: powershell + + python -m venv .venv + .venv\Scripts\pip install -r requirements.txt + .venv\Scripts\pip install -e . + + .. tab:: Linux + + .. code-block:: bash + + python -m venv .venv + .venv/bin/pip install -r requirements.txt + .venv/bin/pip install -e . + +``` + +### M1-Step 3: Run EOS + +```{eval-rst} +.. tabs:: + + .. tab:: Windows + + .. code-block:: powershell + + .venv\Scripts\python -m akkudoktoreos.server.eos + + .. tab:: Linux + + .. code-block:: bash + + .venv/bin/python -m akkudoktoreos.server.eos + +``` + +EOS should now be accessible at [http://localhost:8503/docs](http://localhost:8503/docs) and EOSdash +should be available at [http://localhost:8504](http://localhost:8504). + +If you want to make EOS and EOSdash accessible from outside of your machine or container at this +stage of the installation provide appropriate IP addresses on startup. + + +```{eval-rst} +.. tabs:: + + .. tab:: Windows + + .. code-block:: powershell + + .venv\Scripts\python -m akkudoktoreos.server.eos --host 0.0.0.0 --eosdash-host 0.0.0.0 + + .. tab:: Linux + + .. code-block:: bash + + .venv/bin/python -m akkudoktoreos.server.eos --host 0.0.0.0 --eosdash-host 0.0.0.0 + +``` + + +### M1-Step 4: Configure EOS + +Use [EOSdash](http://localhost:8504) to configure EOS. + +### Updating from Source + +To update to the latest version: + +```{eval-rst} +.. tabs:: + + .. tab:: Windows + + .. code-block:: powershell + + git pull origin main + .venv\Scripts\pip install -r requirements.txt --upgrade + + .. tab:: Linux + + .. code-block:: bash + + git pull origin main + .venv/bin/pip install -r requirements.txt --upgrade + +``` + +## Method 2: Installation from Release Package (GitHub) + +This method is recommended for users who want a stable, tested version. + +### M2-Step 1: Download the Latest Release + +Visit the [Releases page](https://github.com/Akkudoktor-EOS/EOS/releases) and download the latest +release package (e.g., `akkudoktoreos-v0.1.0.tar.gz` or `akkudoktoreos-v0.1.0.zip`). + +### M2-Step 2: Extract the Package + +```bash +tar -xzf akkudoktoreos-v0.1.0.tar.gz # For .tar.gz +# or +unzip akkudoktoreos-v0.1.0.zip # For .zip + +cd akkudoktoreos-v0.1.0 +``` + +### Follow Step 2, 3 and 4 of Method 1: Installation from source + +Installation from release package now needs the exact same steps 2, 3, 4 of method 1. + +## Method 3: Installation with Docker (DockerHub) + +This method is recommended for easy deployment and containerized environments. + +### M3-Step 1: Pull the Docker Image + +```bash +docker pull akkudoktor/eos:latest +``` + +For a specific version: + +```bash +docker pull akkudoktor/eos:v0.1.0 +``` + +### M3-Step 2: Run the Container + +**Basic run:** + +```bash +docker run -d \ + --name akkudoktoreos \ + -p 8503:8503 \ + -p 8504:8504 \ + -e OPENBLAS_NUM_THREADS=1 \ + -e OMP_NUM_THREADS=1 \ + -e MKL_NUM_THREADS=1 \ + -e EOS_SERVER__HOST=0.0.0.0 \ + -e EOS_SERVER__PORT=8503 \ + -e EOS_SERVER__EOSDASH_HOST=0.0.0.0 \ + -e EOS_SERVER__EOSDASH_PORT=8504 \ + --ulimit nproc=65535:65535 \ + --ulimit nofile=65535:65535 \ + --security-opt seccomp=unconfined \ + akkudoktor/eos:latest +``` + +### M3-Step 3: Verify the Container is Running + +```bash +docker ps +docker logs akkudoktoreos +``` + +EOS should now be accessible at [http://localhost:8503/docs](http://localhost:8503/docs) and EOSdash +should be available at [http://localhost:8504](http://localhost:8504). + +### M3-Step 4: Configure EOS + +Use [EOSdash](http://localhost:8504) to configure EOS. + +### Docker Management Commands + +**View logs:** + +```bash +docker logs -f akkudoktoreos +``` + +**Stop the container:** + +```bash +docker stop akkudoktoreos +``` + +**Start the container:** + +```bash +docker start akkudoktoreos +``` + +**Remove the container:** + +```bash +docker rm -f akkudoktoreos +``` + +**Update to latest version:** + +```bash +docker pull Akkudoktor-EOS/EOS:latest +docker stop akkudoktoreos +docker rm akkudoktoreos +# Then run the container again with the run command +``` + +## Method 4: Installation with Docker (docker-compose) + +### M4-Step 1: Get the akkudoktoreos source code + +You may use either method 1 or method 2 to get the source code. + +### M4-Step 2: Build and run the container + +```{eval-rst} +.. tabs:: + + .. tab:: Windows + + .. code-block:: powershell + + docker compose up --build + + .. tab:: Linux + + .. code-block:: bash + + docker compose up --build + +``` + +### M4-Step 3: Verify the Container is Running + +```bash +docker ps +docker logs akkudoktoreos +``` + +EOS should now be accessible at [http://localhost:8503/docs](http://localhost:8503/docs) and EOSdash +should be available at [http://localhost:8504](http://localhost:8504). + +### M4-Step 4: Configure EOS + +Use [EOSdash](http://localhost:8504) to configure EOS. diff --git a/docs/develop/release.md b/docs/develop/release.md new file mode 100644 index 0000000..045e657 --- /dev/null +++ b/docs/develop/release.md @@ -0,0 +1,187 @@ +% SPDX-License-Identifier: Apache-2.0 +(release-page)= + +# Release Process + +This document describes how to prepare and publish a new release **via a Pull Request from a fork**, +using **Commitizen** to manage versioning and changelogs — and how to set a **development version** after the release. + +## ✅ Overview of the Process + +| Step | Actor | Action | +|------|-------------|--------| +| 1 | Contributor | Prepare a release branch **in your fork** using Commitizen | +| 2 | Contributor | Open a **Pull Request to upstream** (`Akkudoktor-EOS/EOS`) | +| 3 | Maintainer | Review and **merge the release PR** | +| 4 | Maintainer | Create the **GitHub Release and tag** | +| 5 | Maintainer | Set the **development version marker** via a follow-up PR | + +## 🔄 Detailed Workflow + +### 1️⃣ Contributor: Prepare the Release in Your Fork + +**Clone and sync your fork:** + +```bash +git clone https://github.com//EOS +cd EOS +git remote add eos https://github.com/Akkudoktor-EOS/EOS + +git fetch eos +git checkout main +git pull eos main +```` + +**Create the release branch:** + +```bash +git checkout -b release/vX.Y.Z +``` + +**Run Commitizen to bump version and update changelog:** + +```bash +make bump +``` + +> ✅ This updates version files and `CHANGELOG.md` in a single commit. +> 🚫 **Do not push tags** — tags are created by the maintainer via GitHub Releases. + +**Push the branch to your fork:** + +```bash +git push origin release/vX.Y.Z +``` + +### 2️⃣ Contributor: Open the Release Pull Request + +| From | To | +| ------------------------------------ | ------------------------- | +| `/EOS:release/vX.Y.Z` | `Akkudoktor-EOS/EOS:main` | + +**PR Title:** + +```text +Release vX.Y.Z +``` + +**PR Description Template:** + +```markdown +## Release vX.Y.Z + +This pull request prepares release **vX.Y.Z**. + +### Changes +- Version bump via Commitizen +- Changelog update + +### Changelog Summary + + +See `CHANGELOG.md` for full details. +``` + +### 3️⃣ Maintainer: Review and Merge the Release PR + +**Review Checklist:** + +* ✅ Only version files and `CHANGELOG.md` are modified +* ✅ Version numbers are consistent +* ✅ Changelog is complete and properly formatted +* ✅ No unrelated changes are included + +**Merge Strategy:** + +* Prefer **Merge Commit** (or **Squash Merge**, per project preference) +* Use commit message: `Release vX.Y.Z` + +### 4️⃣ Maintainer: Publish the GitHub Release + +1. Go to **GitHub → Releases → Draft a new release** +2. **Choose tag** → enter `vX.Y.Z` (GitHub creates the tag on publish) +3. **Release title:** `vX.Y.Z` +4. **Paste changelog entry** from `CHANGELOG.md` +5. Optionally enable **Set as latest release** +6. Click **Publish release** 🎉 + +### 5️⃣ Maintainer: Prepare the Development Version Marker + +**Sync local copy:** + +```bash +git fetch eos +git checkout main +git pull eos main +``` + +**Create a development version branch:** + +```bash +git checkout -b release/vX.Y.Z_dev +``` + +**Set development marker manually:** + +The following files have to be updated: + +* pyproject.toml +* src/akkudoktoreos/core/version.py +* src/data/default.config.json + +Example for pyproject.toml + +```bash +sed -i 's/version = "\(.*\)"/version = "\1+dev"/' pyproject.toml +git add pyproject.toml +git commit -m "chore: set development version marker" +git push origin release/vX.Y.Z_dev +``` + +### 6️⃣ Maintainer (or Contributor): Open the Development Version PR + +| From | To | +| ---------------------------------------- | ------------------------- | +| `/EOS:release/vX.Y.Z_dev` | `Akkudoktor-EOS/EOS:main` | + +**PR Title:** + +```text +Release vX.Y.Z+dev +``` + +**PR Description Template:** + +```markdown +## Release vX.Y.Z_dev + +This pull request marks the repository as back in active development. + +### Changes +- Set version to `vX.Y.Z+dev` + +No changelog entry is needed. +``` + +### 7️⃣ Maintainer: Review and Merge the Development Version PR + +**Checklist:** + +* ✅ Only version files updated to `+dev` +* ✅ No unintended changes + +**Merge Strategy:** + +* Merge with commit message: `Release vX.Y.Z_dev` + +## ✅ Quick Reference + +| Step | Actor | Action | +| ---- | ----- | ------ | +| **1. Prepare release branch** | Contributor | Bump version & changelog via Commitizen | +| **2. Open release PR** | Contributor | Submit release for review | +| **3. Review & merge release PR** | Maintainer | Finalize changes into `main` | +| **4. Publish GitHub Release** | Maintainer | Create tag & notify users | +| **5. Prepare development version branch** | Maintainer | Set development marker | +| **6. Open development PR** | Maintainer (or Contributor) | Propose returning to development state | +| **7. Review & merge development PR** | Maintainer | Mark repository as back in development | diff --git a/docs/index.md b/docs/index.md index c0cd50c..f22fce9 100644 --- a/docs/index.md +++ b/docs/index.md @@ -36,7 +36,10 @@ develop/CONTRIBUTING.md akkudoktoreos/architecture.md akkudoktoreos/configuration.md -akkudoktoreos/optimization.md +akkudoktoreos/configtimewindow.md +akkudoktoreos/optimpost.md +akkudoktoreos/optimauto.md +akkudoktoreos/resource.md akkudoktoreos/prediction.md akkudoktoreos/measurement.md akkudoktoreos/integration.md @@ -46,6 +49,18 @@ akkudoktoreos/api.rst ``` +```{toctree} +:maxdepth: 2 +:caption: Development + +develop/CONTRIBUTING.md +develop/install.md +develop/develop.md +develop/release.md +develop/CHANGELOG.md + +``` + ## Indices and tables - {ref}`genindex` diff --git a/openapi.json b/openapi.json index fd64401..b0a7516 100644 --- a/openapi.json +++ b/openapi.json @@ -3,7 +3,7 @@ "info": { "title": "Akkudoktor-EOS", "description": "This project provides a comprehensive solution for simulating and optimizing an energy system based on renewable energy sources. With a focus on photovoltaic (PV) systems, battery storage (batteries), load management (consumer requirements), heat pumps, electric vehicles, and consideration of electricity price data, this system enables forecasting and optimization of energy flow and costs over a specified period.", - "version": "0.0.1" + "version": "v0.1.0+dev" }, "paths": { "/v1/admin/cache/clear": { @@ -12,45 +12,41 @@ "admin" ], "summary": "Fastapi Admin Cache Clear Post", - "description": "Clear the cache from expired data.\n\nDeletes expired cache files.\n\nArgs:\n clear_all (Optional[bool]): Delete all cached files. Default is False.\n\nReturns:\n data (dict): The management data after cleanup.", + "description": "Clear the cache.\n\nDeletes all cache files.\n\nReturns:\n data (dict): The management data after cleanup.", "operationId": "fastapi_admin_cache_clear_post_v1_admin_cache_clear_post", - "parameters": [ - { - "name": "clear_all", - "in": "query", - "required": false, - "schema": { - "anyOf": [ - { - "type": "boolean" - }, - { - "type": "null" - } - ], - "title": "Clear All" - } - } - ], "responses": { "200": { "description": "Successful Response", "content": { "application/json": { "schema": { - "type": "object", "additionalProperties": true, + "type": "object", "title": "Response Fastapi Admin Cache Clear Post V1 Admin Cache Clear Post" } } } - }, - "422": { - "description": "Validation Error", + } + } + } + }, + "/v1/admin/cache/clear-expired": { + "post": { + "tags": [ + "admin" + ], + "summary": "Fastapi Admin Cache Clear Expired Post", + "description": "Clear the cache from expired data.\n\nDeletes expired cache files.\n\nReturns:\n data (dict): The management data after cleanup.", + "operationId": "fastapi_admin_cache_clear_expired_post_v1_admin_cache_clear_expired_post", + "responses": { + "200": { + "description": "Successful Response", "content": { "application/json": { "schema": { - "$ref": "#/components/schemas/HTTPValidationError" + "additionalProperties": true, + "type": "object", + "title": "Response Fastapi Admin Cache Clear Expired Post V1 Admin Cache Clear Expired Post" } } } @@ -549,6 +545,256 @@ } } }, + "/v1/resource/status": { + "get": { + "tags": [ + "resource" + ], + "summary": "Fastapi Devices Status Get", + "description": "Get the latest status of a resource/ device.\n\nReturn:\n latest_status: The latest status of a resource/ device.", + "operationId": "fastapi_devices_status_get_v1_resource_status_get", + "parameters": [ + { + "name": "resource_id", + "in": "query", + "required": true, + "schema": { + "type": "string", + "description": "Resource ID.", + "title": "Resource Id" + }, + "description": "Resource ID." + }, + { + "name": "actuator_id", + "in": "query", + "required": false, + "schema": { + "anyOf": [ + { + "type": "string" + }, + { + "type": "null" + } + ], + "description": "Actuator ID.", + "title": "Actuator Id" + }, + "description": "Actuator ID." + } + ], + "responses": { + "200": { + "description": "Successful Response", + "content": { + "application/json": { + "schema": { + "oneOf": [ + { + "$ref": "#/components/schemas/PowerMeasurement-Output" + }, + { + "$ref": "#/components/schemas/EnergyMeasurement-Output" + }, + { + "$ref": "#/components/schemas/PPBCPowerProfileStatus-Output" + }, + { + "$ref": "#/components/schemas/OMBCStatus" + }, + { + "$ref": "#/components/schemas/FRBCActuatorStatus" + }, + { + "$ref": "#/components/schemas/FRBCEnergyStatus-Output" + }, + { + "$ref": "#/components/schemas/FRBCStorageStatus" + }, + { + "$ref": "#/components/schemas/FRBCTimerStatus" + }, + { + "$ref": "#/components/schemas/DDBCActuatorStatus" + } + ], + "discriminator": { + "propertyName": "type", + "mapping": { + "PowerMeasurement": "#/components/schemas/PowerMeasurement-Output", + "EnergyMeasurement": "#/components/schemas/EnergyMeasurement-Output", + "PPBCPowerProfileStatus": "#/components/schemas/PPBCPowerProfileStatus-Output", + "OMBCStatus": "#/components/schemas/OMBCStatus", + "FRBCActuatorStatus": "#/components/schemas/FRBCActuatorStatus", + "FRBCEnergyStatus": "#/components/schemas/FRBCEnergyStatus-Output", + "FRBCStorageStatus": "#/components/schemas/FRBCStorageStatus", + "FRBCTimerStatus": "#/components/schemas/FRBCTimerStatus", + "DDBCActuatorStatus": "#/components/schemas/DDBCActuatorStatus" + } + }, + "title": "Response Fastapi Devices Status Get V1 Resource Status Get" + } + } + } + }, + "422": { + "description": "Validation Error", + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/HTTPValidationError" + } + } + } + } + } + }, + "put": { + "tags": [ + "resource" + ], + "summary": "Fastapi Devices Status Put", + "description": "Update the status of a resource/ device.\n\nReturn:\n latest_status: The latest status of a resource/ device.", + "operationId": "fastapi_devices_status_put_v1_resource_status_put", + "parameters": [ + { + "name": "resource_id", + "in": "query", + "required": true, + "schema": { + "type": "string", + "description": "Resource ID.", + "title": "Resource Id" + }, + "description": "Resource ID." + }, + { + "name": "actuator_id", + "in": "query", + "required": false, + "schema": { + "anyOf": [ + { + "type": "string" + }, + { + "type": "null" + } + ], + "description": "Actuator ID.", + "title": "Actuator Id" + }, + "description": "Actuator ID." + } + ], + "requestBody": { + "required": true, + "content": { + "application/json": { + "schema": { + "anyOf": [ + { + "$ref": "#/components/schemas/PowerMeasurement-Input" + }, + { + "$ref": "#/components/schemas/EnergyMeasurement-Input" + }, + { + "$ref": "#/components/schemas/PPBCPowerProfileStatus-Input" + }, + { + "$ref": "#/components/schemas/OMBCStatus" + }, + { + "$ref": "#/components/schemas/FRBCActuatorStatus" + }, + { + "$ref": "#/components/schemas/FRBCEnergyStatus-Input" + }, + { + "$ref": "#/components/schemas/FRBCStorageStatus" + }, + { + "$ref": "#/components/schemas/FRBCTimerStatus" + }, + { + "$ref": "#/components/schemas/DDBCActuatorStatus" + } + ], + "description": "Resource Status.", + "title": "Status" + } + } + } + }, + "responses": { + "200": { + "description": "Successful Response", + "content": { + "application/json": { + "schema": { + "oneOf": [ + { + "$ref": "#/components/schemas/PowerMeasurement-Output" + }, + { + "$ref": "#/components/schemas/EnergyMeasurement-Output" + }, + { + "$ref": "#/components/schemas/PPBCPowerProfileStatus-Output" + }, + { + "$ref": "#/components/schemas/OMBCStatus" + }, + { + "$ref": "#/components/schemas/FRBCActuatorStatus" + }, + { + "$ref": "#/components/schemas/FRBCEnergyStatus-Output" + }, + { + "$ref": "#/components/schemas/FRBCStorageStatus" + }, + { + "$ref": "#/components/schemas/FRBCTimerStatus" + }, + { + "$ref": "#/components/schemas/DDBCActuatorStatus" + } + ], + "discriminator": { + "propertyName": "type", + "mapping": { + "PowerMeasurement": "#/components/schemas/PowerMeasurement-Output", + "EnergyMeasurement": "#/components/schemas/EnergyMeasurement-Output", + "PPBCPowerProfileStatus": "#/components/schemas/PPBCPowerProfileStatus-Output", + "OMBCStatus": "#/components/schemas/OMBCStatus", + "FRBCActuatorStatus": "#/components/schemas/FRBCActuatorStatus", + "FRBCEnergyStatus": "#/components/schemas/FRBCEnergyStatus-Output", + "FRBCStorageStatus": "#/components/schemas/FRBCStorageStatus", + "FRBCTimerStatus": "#/components/schemas/FRBCTimerStatus", + "DDBCActuatorStatus": "#/components/schemas/DDBCActuatorStatus" + } + }, + "title": "Response Fastapi Devices Status Put V1 Resource Status Put" + } + } + } + }, + "422": { + "description": "Validation Error", + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/HTTPValidationError" + } + } + } + } + } + } + }, "/v1/measurement/keys": { "get": { "tags": [ @@ -575,176 +821,6 @@ } } }, - "/v1/measurement/load-mr/series/by-name": { - "get": { - "tags": [ - "measurement" - ], - "summary": "Fastapi Measurement Load Mr Series By Name Get", - "description": "Get the meter reading of given load name as series.", - "operationId": "fastapi_measurement_load_mr_series_by_name_get_v1_measurement_load_mr_series_by_name_get", - "parameters": [ - { - "name": "name", - "in": "query", - "required": true, - "schema": { - "type": "string", - "description": "Load name.", - "title": "Name" - }, - "description": "Load name." - } - ], - "responses": { - "200": { - "description": "Successful Response", - "content": { - "application/json": { - "schema": { - "$ref": "#/components/schemas/PydanticDateTimeSeries" - } - } - } - }, - "422": { - "description": "Validation Error", - "content": { - "application/json": { - "schema": { - "$ref": "#/components/schemas/HTTPValidationError" - } - } - } - } - } - }, - "put": { - "tags": [ - "measurement" - ], - "summary": "Fastapi Measurement Load Mr Series By Name Put", - "description": "Merge the meter readings series of given load name into EOS measurements at given datetime.", - "operationId": "fastapi_measurement_load_mr_series_by_name_put_v1_measurement_load_mr_series_by_name_put", - "parameters": [ - { - "name": "name", - "in": "query", - "required": true, - "schema": { - "type": "string", - "description": "Load name.", - "title": "Name" - }, - "description": "Load name." - } - ], - "requestBody": { - "required": true, - "content": { - "application/json": { - "schema": { - "$ref": "#/components/schemas/PydanticDateTimeSeries" - } - } - } - }, - "responses": { - "200": { - "description": "Successful Response", - "content": { - "application/json": { - "schema": { - "$ref": "#/components/schemas/PydanticDateTimeSeries" - } - } - } - }, - "422": { - "description": "Validation Error", - "content": { - "application/json": { - "schema": { - "$ref": "#/components/schemas/HTTPValidationError" - } - } - } - } - } - } - }, - "/v1/measurement/load-mr/value/by-name": { - "put": { - "tags": [ - "measurement" - ], - "summary": "Fastapi Measurement Load Mr Value By Name Put", - "description": "Merge the meter reading of given load name and value into EOS measurements at given datetime.", - "operationId": "fastapi_measurement_load_mr_value_by_name_put_v1_measurement_load_mr_value_by_name_put", - "parameters": [ - { - "name": "datetime", - "in": "query", - "required": true, - "schema": { - "type": "string", - "description": "Datetime.", - "title": "Datetime" - }, - "description": "Datetime." - }, - { - "name": "name", - "in": "query", - "required": true, - "schema": { - "type": "string", - "description": "Load name.", - "title": "Name" - }, - "description": "Load name." - }, - { - "name": "value", - "in": "query", - "required": true, - "schema": { - "anyOf": [ - { - "type": "number" - }, - { - "type": "string" - } - ], - "title": "Value" - } - } - ], - "responses": { - "200": { - "description": "Successful Response", - "content": { - "application/json": { - "schema": { - "$ref": "#/components/schemas/PydanticDateTimeSeries" - } - } - } - }, - "422": { - "description": "Validation Error", - "content": { - "application/json": { - "schema": { - "$ref": "#/components/schemas/HTTPValidationError" - } - } - } - } - } - } - }, "/v1/measurement/series": { "get": { "tags": [ @@ -760,10 +836,10 @@ "required": true, "schema": { "type": "string", - "description": "Prediction key.", + "description": "Measurement key.", "title": "Key" }, - "description": "Prediction key." + "description": "Measurement key." } ], "responses": { @@ -803,10 +879,10 @@ "required": true, "schema": { "type": "string", - "description": "Prediction key.", + "description": "Measurement key.", "title": "Key" }, - "description": "Prediction key." + "description": "Measurement key." } ], "requestBody": { @@ -869,10 +945,10 @@ "required": true, "schema": { "type": "string", - "description": "Prediction key.", + "description": "Measurement key.", "title": "Key" }, - "description": "Prediction key." + "description": "Measurement key." }, { "name": "value", @@ -1583,6 +1659,50 @@ } } }, + "/v1/energy-management/optimization/solution": { + "get": { + "tags": [ + "energy-management" + ], + "summary": "Fastapi Energy Management Optimization Solution Get", + "description": "Get the latest solution of the optimization.", + "operationId": "fastapi_energy_management_optimization_solution_get_v1_energy_management_optimization_solution_get", + "responses": { + "200": { + "description": "Successful Response", + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/OptimizationSolution" + } + } + } + } + } + } + }, + "/v1/energy-management/plan": { + "get": { + "tags": [ + "energy-management" + ], + "summary": "Fastapi Energy Management Plan Get", + "description": "Get the latest energy management plan.", + "operationId": "fastapi_energy_management_plan_get_v1_energy_management_plan_get", + "responses": { + "200": { + "description": "Successful Response", + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/EnergyManagementPlan" + } + } + } + } + } + } + }, "/strompreis": { "get": { "tags": [ @@ -1615,7 +1735,7 @@ "prediction" ], "summary": "Fastapi Gesamtlast", - "description": "Deprecated: Total Load Prediction with adjustment.\n\nEndpoint to handle total load prediction adjusted by latest measured data.\n\nTotal load prediction starts at 00.00.00 today and is provided for 48 hours.\nIf no prediction values are available the missing ones at the start of the series are\nfilled with the first available prediction value.\n\nNote:\n Use '/v1/prediction/list?key=load_mean_adjusted' instead.\n Load energy meter readings to be added to EOS measurement by:\n '/v1/measurement/load-mr/value/by-name' or\n '/v1/measurement/value'", + "description": "Deprecated: Total Load Prediction with adjustment.\n\nEndpoint to handle total load prediction adjusted by latest measured data.\n\nTotal load prediction starts at 00.00.00 today and is provided for 48 hours.\nIf no prediction values are available the missing ones at the start of the series are\nfilled with the first available prediction value.\n\nNote:\n Use '/v1/prediction/list?key=load_mean_adjusted' instead.\n Load energy meter readings to be added to EOS measurement by:\n '/v1/measurement/value' or\n '/v1/measurement/series' or\n '/v1/measurement/dataframe' or\n '/v1/measurement/data'", "operationId": "fastapi_gesamtlast_gesamtlast_post", "requestBody": { "content": { @@ -1730,6 +1850,7 @@ "optimize" ], "summary": "Fastapi Optimize", + "description": "Deprecated: Optimize.\n\nEndpoint to handle optimization.\n\nNote:\n Use automatic optimization instead.", "operationId": "fastapi_optimize_optimize_post", "parameters": [ { @@ -1763,8 +1884,10 @@ "type": "null" } ], + "description": "Number of indivuals to generate for genetic algorithm.", "title": "Ngen" - } + }, + "description": "Number of indivuals to generate for genetic algorithm." } ], "requestBody": { @@ -1772,7 +1895,7 @@ "content": { "application/json": { "schema": { - "$ref": "#/components/schemas/OptimizationParameters" + "$ref": "#/components/schemas/GeneticOptimizationParameters" } } } @@ -1783,7 +1906,7 @@ "content": { "application/json": { "schema": { - "$ref": "#/components/schemas/OptimizeResponse" + "$ref": "#/components/schemas/GeneticSolution" } } } @@ -1825,37 +1948,26 @@ }, "components": { "schemas": { - "BaseBatteryParameters": { + "BatteriesCommonSettings-Input": { "properties": { "device_id": { "type": "string", "title": "Device Id", - "description": "ID of battery", + "description": "ID of device", + "default": "", "examples": [ - "battery1" - ] - }, - "hours": { - "anyOf": [ - { - "type": "integer", - "exclusiveMinimum": 0.0 - }, - { - "type": "null" - } - ], - "title": "Hours", - "description": "Number of prediction hours. Defaults to global config prediction hours.", - "examples": [ - null + "battery1", + "ev1", + "inverter1", + "dishwasher" ] }, "capacity_wh": { "type": "integer", "exclusiveMinimum": 0.0, "title": "Capacity Wh", - "description": "An integer representing the capacity of the battery in watt-hours.", + "description": "Capacity [Wh].", + "default": 8000, "examples": [ 8000 ] @@ -1865,16 +1977,31 @@ "maximum": 1.0, "exclusiveMinimum": 0.0, "title": "Charging Efficiency", - "description": "A float representing the charging efficiency of the battery.", - "default": 0.88 + "description": "Charging efficiency [0.01 ... 1.00].", + "default": 0.88, + "examples": [ + 0.88 + ] }, "discharging_efficiency": { "type": "number", "maximum": 1.0, "exclusiveMinimum": 0.0, "title": "Discharging Efficiency", - "description": "A float representing the discharge efficiency of the battery.", - "default": 0.88 + "description": "Discharge efficiency [0.01 ... 1.00].", + "default": 0.88, + "examples": [ + 0.88 + ] + }, + "levelized_cost_of_storage_kwh": { + "type": "number", + "title": "Levelized Cost Of Storage Kwh", + "description": "Levelized cost of storage (LCOS), the average lifetime cost of delivering one kWh [\u20ac/kWh].", + "default": 0.0, + "examples": [ + 0.12 + ] }, "max_charge_power_w": { "anyOf": [ @@ -1887,18 +2014,52 @@ } ], "title": "Max Charge Power W", - "description": "Maximum charging power in watts.", - "default": 5000 - }, - "initial_soc_percentage": { - "type": "integer", - "maximum": 100.0, - "minimum": 0.0, - "title": "Initial Soc Percentage", - "description": "An integer representing the state of charge of the battery at the **start** of the current hour (not the current state).", - "default": 0, + "description": "Maximum charging power [W].", + "default": 5000, "examples": [ - 42 + 5000 + ] + }, + "min_charge_power_w": { + "anyOf": [ + { + "type": "number", + "exclusiveMinimum": 0.0 + }, + { + "type": "null" + } + ], + "title": "Min Charge Power W", + "description": "Minimum charging power [W].", + "default": 50, + "examples": [ + 50 + ] + }, + "charge_rates": { + "anyOf": [ + { + "items": { + "type": "number" + }, + "type": "array" + }, + { + "type": "null" + } + ], + "title": "Charge Rates", + "description": "Charge rates as factor of maximum charging power [0.00 ... 1.00]. None denotes all charge rates are available.", + "examples": [ + [ + 0.0, + 0.25, + 0.5, + 0.75, + 1.0 + ], + null ] }, "min_soc_percentage": { @@ -1906,7 +2067,7 @@ "maximum": 100.0, "minimum": 0.0, "title": "Min Soc Percentage", - "description": "An integer representing the minimum state of charge (SOC) of the battery in percentage.", + "description": "Minimum state of charge (SOC) as percentage of capacity [%].", "default": 0, "examples": [ 10 @@ -1917,18 +2078,211 @@ "maximum": 100.0, "minimum": 0.0, "title": "Max Soc Percentage", - "description": "An integer representing the maximum state of charge (SOC) of the battery in percentage.", - "default": 100 + "description": "Maximum state of charge (SOC) as percentage of capacity [%].", + "default": 100, + "examples": [ + 100 + ] + } + }, + "type": "object", + "title": "BatteriesCommonSettings", + "description": "Battery devices base settings." + }, + "BatteriesCommonSettings-Output": { + "properties": { + "device_id": { + "type": "string", + "title": "Device Id", + "description": "ID of device", + "default": "", + "examples": [ + "battery1", + "ev1", + "inverter1", + "dishwasher" + ] + }, + "capacity_wh": { + "type": "integer", + "exclusiveMinimum": 0.0, + "title": "Capacity Wh", + "description": "Capacity [Wh].", + "default": 8000, + "examples": [ + 8000 + ] + }, + "charging_efficiency": { + "type": "number", + "maximum": 1.0, + "exclusiveMinimum": 0.0, + "title": "Charging Efficiency", + "description": "Charging efficiency [0.01 ... 1.00].", + "default": 0.88, + "examples": [ + 0.88 + ] + }, + "discharging_efficiency": { + "type": "number", + "maximum": 1.0, + "exclusiveMinimum": 0.0, + "title": "Discharging Efficiency", + "description": "Discharge efficiency [0.01 ... 1.00].", + "default": 0.88, + "examples": [ + 0.88 + ] + }, + "levelized_cost_of_storage_kwh": { + "type": "number", + "title": "Levelized Cost Of Storage Kwh", + "description": "Levelized cost of storage (LCOS), the average lifetime cost of delivering one kWh [\u20ac/kWh].", + "default": 0.0, + "examples": [ + 0.12 + ] + }, + "max_charge_power_w": { + "anyOf": [ + { + "type": "number", + "exclusiveMinimum": 0.0 + }, + { + "type": "null" + } + ], + "title": "Max Charge Power W", + "description": "Maximum charging power [W].", + "default": 5000, + "examples": [ + 5000 + ] + }, + "min_charge_power_w": { + "anyOf": [ + { + "type": "number", + "exclusiveMinimum": 0.0 + }, + { + "type": "null" + } + ], + "title": "Min Charge Power W", + "description": "Minimum charging power [W].", + "default": 50, + "examples": [ + 50 + ] + }, + "charge_rates": { + "anyOf": [ + { + "items": { + "type": "number" + }, + "type": "array" + }, + { + "type": "null" + } + ], + "title": "Charge Rates", + "description": "Charge rates as factor of maximum charging power [0.00 ... 1.00]. None denotes all charge rates are available.", + "examples": [ + [ + 0.0, + 0.25, + 0.5, + 0.75, + 1.0 + ], + null + ] + }, + "min_soc_percentage": { + "type": "integer", + "maximum": 100.0, + "minimum": 0.0, + "title": "Min Soc Percentage", + "description": "Minimum state of charge (SOC) as percentage of capacity [%].", + "default": 0, + "examples": [ + 10 + ] + }, + "max_soc_percentage": { + "type": "integer", + "maximum": 100.0, + "minimum": 0.0, + "title": "Max Soc Percentage", + "description": "Maximum state of charge (SOC) as percentage of capacity [%].", + "default": 100, + "examples": [ + 100 + ] + }, + "measurement_key_soc_factor": { + "type": "string", + "title": "Measurement Key Soc Factor", + "description": "Measurement key for the battery state of charge (SoC) as factor of total capacity [0.0 ... 1.0].", + "readOnly": true + }, + "measurement_key_power_l1_w": { + "type": "string", + "title": "Measurement Key Power L1 W", + "description": "Measurement key for the L1 power the battery is charged or discharged with [W].", + "readOnly": true + }, + "measurement_key_power_l2_w": { + "type": "string", + "title": "Measurement Key Power L2 W", + "description": "Measurement key for the L2 power the battery is charged or discharged with [W].", + "readOnly": true + }, + "measurement_key_power_l3_w": { + "type": "string", + "title": "Measurement Key Power L3 W", + "description": "Measurement key for the L3 power the battery is charged or discharged with [W].", + "readOnly": true + }, + "measurement_key_power_3_phase_sym_w": { + "type": "string", + "title": "Measurement Key Power 3 Phase Sym W", + "description": "Measurement key for the symmetric 3 phase power the battery is charged or discharged with [W].", + "readOnly": true + }, + "measurement_keys": { + "anyOf": [ + { + "items": { + "type": "string" + }, + "type": "array" + }, + { + "type": "null" + } + ], + "title": "Measurement Keys", + "description": "Measurement keys for the battery stati that are measurements.\n\nBattery SoC, power.", + "readOnly": true } }, - "additionalProperties": false, "type": "object", "required": [ - "device_id", - "capacity_wh" + "measurement_key_soc_factor", + "measurement_key_power_l1_w", + "measurement_key_power_l2_w", + "measurement_key_power_l3_w", + "measurement_key_power_3_phase_sym_w", + "measurement_keys" ], - "title": "BaseBatteryParameters", - "description": "Battery Device Simulation Configuration." + "title": "BatteriesCommonSettings", + "description": "Battery devices base settings." }, "CacheCommonSettings": { "properties": { @@ -1957,11 +2311,30 @@ "title": "CacheCommonSettings", "description": "Cache Configuration." }, + "CommodityQuantity": { + "type": "string", + "enum": [ + "ELECTRIC.POWER.L1", + "ELECTRIC.POWER.L2", + "ELECTRIC.POWER.L3", + "ELECTRIC.POWER.3_PHASE_SYM", + "NATURAL_GAS.FLOW_RATE", + "HYDROGEN.FLOW_RATE", + "HEAT.TEMPERATURE", + "HEAT.FLOW_RATE", + "HEAT.THERMAL_POWER", + "OIL.FLOW_RATE", + "CURRENCY" + ], + "title": "CommodityQuantity", + "description": "Enumeration of specific commodity quantities and measurement types." + }, "ConfigEOS": { "properties": { "general": { "$ref": "#/components/schemas/GeneralSettings-Output", "default": { + "version": "0.1.0+dev", "data_output_subpath": "output", "latitude": 52.52, "longitude": 13.405, @@ -1990,27 +2363,22 @@ } }, "devices": { - "$ref": "#/components/schemas/DevicesCommonSettings", - "default": {} + "$ref": "#/components/schemas/DevicesCommonSettings-Output", + "default": { + "measurement_keys": [] + } }, "measurement": { - "$ref": "#/components/schemas/MeasurementCommonSettings", - "default": {} + "$ref": "#/components/schemas/MeasurementCommonSettings-Output", + "default": { + "keys": [] + } }, "optimization": { "$ref": "#/components/schemas/OptimizationCommonSettings", "default": { - "hours": 48, - "penalty": 10, - "ev_available_charge_rates_percent": [ - 0.0, - 0.375, - 0.5, - 0.625, - 0.75, - 0.875, - 1.0 - ] + "horizon_hours": 24, + "interval": 3600 } }, "prediction": { @@ -2021,18 +2389,28 @@ } }, "elecprice": { - "$ref": "#/components/schemas/ElecPriceCommonSettings", + "$ref": "#/components/schemas/ElecPriceCommonSettings-Output", "default": { - "vat_rate": 1.19 + "vat_rate": 1.19, + "provider_settings": {} + } + }, + "feedintariff": { + "$ref": "#/components/schemas/FeedInTariffCommonSettings-Output", + "default": { + "provider_settings": {} } }, "load": { - "$ref": "#/components/schemas/LoadCommonSettings", - "default": {} + "$ref": "#/components/schemas/LoadCommonSettings-Output", + "default": { + "provider_settings": {} + } }, "pvforecast": { "$ref": "#/components/schemas/PVForecastCommonSettings-Output", "default": { + "provider_settings": {}, "max_planes": 0, "planes_peakpower": [], "planes_azimuth": [], @@ -2042,8 +2420,10 @@ } }, "weather": { - "$ref": "#/components/schemas/WeatherCommonSettings", - "default": {} + "$ref": "#/components/schemas/WeatherCommonSettings-Output", + "default": { + "provider_settings": {} + } }, "server": { "$ref": "#/components/schemas/ServerCommonSettings", @@ -2051,9 +2431,7 @@ "host": "127.0.0.1", "port": 8503, "verbose": false, - "startup_eosdash": true, - "eosdash_host": "127.0.0.1", - "eosdash_port": 8504 + "startup_eosdash": true } }, "utils": { @@ -2066,13 +2444,134 @@ "title": "ConfigEOS", "description": "Singleton configuration handler for the EOS application.\n\nConfigEOS extends `SettingsEOS` with support for default configuration paths and automatic\ninitialization.\n\n`ConfigEOS` ensures that only one instance of the class is created throughout the application,\nallowing consistent access to EOS configuration settings. This singleton instance loads\nconfiguration data from a predefined set of directories or creates a default configuration if\nnone is found.\n\nInitialization Process:\n - Upon instantiation, the singleton instance attempts to load a configuration file in this order:\n 1. The directory specified by the `EOS_CONFIG_DIR` environment variable\n 2. The directory specified by the `EOS_DIR` environment variable.\n 3. A platform specific default directory for EOS.\n 4. The current working directory.\n - The first available configuration file found in these directories is loaded.\n - If no configuration file is found, a default configuration file is created in the platform\n specific default directory, and default settings are loaded into it.\n\nAttributes from the loaded configuration are accessible directly as instance attributes of\n`ConfigEOS`, providing a centralized, shared configuration object for EOS.\n\nSingleton Behavior:\n - This class uses the `SingletonMixin` to ensure that all requests for `ConfigEOS` return\n the same instance, which contains the most up-to-date configuration. Modifying the configuration\n in one part of the application reflects across all references to this class.\n\nAttributes:\n config_folder_path (Optional[Path]): Path to the configuration directory.\n config_file_path (Optional[Path]): Path to the configuration file.\n\nRaises:\n FileNotFoundError: If no configuration file is found, and creating a default configuration fails.\n\nExample:\n To initialize and access configuration attributes (only one instance is created):\n ```python\n config_eos = ConfigEOS() # Always returns the same instance\n print(config_eos.prediction.hours) # Access a setting from the loaded configuration\n ```" }, - "DevicesCommonSettings": { + "DDBCActuatorStatus": { + "properties": { + "type": { + "type": "string", + "const": "DDBCActuatorStatus", + "title": "Type", + "default": "DDBCActuatorStatus" + }, + "active_operation_mode_id": { + "type": "string", + "title": "Active Operation Mode Id", + "description": "Currently active operation mode ID." + }, + "operation_mode_factor": { + "type": "number", + "maximum": 1.0, + "minimum": 0.0, + "title": "Operation Mode Factor", + "description": "Factor with which the operation mode is configured (0 to 1)." + }, + "previous_operation_mode_id": { + "anyOf": [ + { + "type": "string" + }, + { + "type": "null" + } + ], + "title": "Previous Operation Mode Id", + "description": "Previously active operation mode ID. Required unless this is the first mode." + }, + "transition_timestamp": { + "anyOf": [ + { + "type": "string", + "format": "date-time" + }, + { + "type": "null" + } + ], + "title": "Transition Timestamp", + "description": "Timestamp of transition to the active operation mode." + } + }, + "type": "object", + "required": [ + "active_operation_mode_id", + "operation_mode_factor" + ], + "title": "DDBCActuatorStatus", + "description": "Current status of a DDBC Actuator.\n\nProvides information about the currently active operation mode and transition history.\nUsed to track the current state of the actuator." + }, + "DDBCInstruction": { + "properties": { + "id": { + "anyOf": [ + { + "type": "string" + }, + { + "type": "null" + } + ], + "title": "Id", + "description": "Unique identifier of the instruction in the ResourceManager scope. If not provided and a `resource_id` is passed at instantiation, this will be auto-generated as `{resource_id}@{UUID}`." + }, + "execution_time": { + "type": "string", + "format": "date-time", + "title": "Execution Time", + "description": "Start time of the instruction execution." + }, + "abnormal_condition": { + "type": "boolean", + "title": "Abnormal Condition", + "description": "Indicates if this is an instruction for abnormal conditions. Defaults to False.", + "default": false + }, + "type": { + "type": "string", + "const": "DDBCInstruction", + "title": "Type", + "default": "DDBCInstruction" + }, + "actuator_id": { + "type": "string", + "title": "Actuator Id", + "description": "ID of the actuator this instruction belongs to." + }, + "operation_mode_id": { + "type": "string", + "title": "Operation Mode Id", + "description": "ID of the DDBC.OperationMode to apply." + }, + "operation_mode_factor": { + "type": "number", + "maximum": 1.0, + "minimum": 0.0, + "title": "Operation Mode Factor", + "description": "Factor with which the operation mode should be applied (0 to 1)." + }, + "resource_id": { + "type": "string", + "title": "Resource Id", + "description": "Get the resource identifier component from the instruction's `id`.\n\nAssumes the `id` follows the format `{resource_id}@{UUID}`. Extracts the resource_id part\nof the id by splitting at the last @.\n\nReturns:\n str: The resource identifier prefix of `id`, or an empty string if `id` is None.", + "readOnly": true + } + }, + "type": "object", + "required": [ + "execution_time", + "actuator_id", + "operation_mode_id", + "operation_mode_factor", + "resource_id" + ], + "title": "DDBCInstruction", + "description": "Instruction for Demand Driven Based Control (DDBC).\n\nContains information about when and how to activate a specific operation mode\nfor an actuator. Used to command resources to change their operation at a specified time." + }, + "DevicesCommonSettings-Input": { "properties": { "batteries": { "anyOf": [ { "items": { - "$ref": "#/components/schemas/BaseBatteryParameters" + "$ref": "#/components/schemas/BatteriesCommonSettings-Input" }, "type": "array" }, @@ -2081,7 +2580,7 @@ } ], "title": "Batteries", - "description": "List of battery/ev devices", + "description": "List of battery devices", "examples": [ [ { @@ -2091,11 +2590,68 @@ ] ] }, + "max_batteries": { + "anyOf": [ + { + "type": "integer", + "minimum": 0.0 + }, + { + "type": "null" + } + ], + "title": "Max Batteries", + "description": "Maximum number of batteries that can be set", + "examples": [ + 1, + 2 + ] + }, + "electric_vehicles": { + "anyOf": [ + { + "items": { + "$ref": "#/components/schemas/BatteriesCommonSettings-Input" + }, + "type": "array" + }, + { + "type": "null" + } + ], + "title": "Electric Vehicles", + "description": "List of electric vehicle devices", + "examples": [ + [ + { + "capacity_wh": 8000, + "device_id": "battery1" + } + ] + ] + }, + "max_electric_vehicles": { + "anyOf": [ + { + "type": "integer", + "minimum": 0.0 + }, + { + "type": "null" + } + ], + "title": "Max Electric Vehicles", + "description": "Maximum number of electric vehicles that can be set", + "examples": [ + 1, + 2 + ] + }, "inverters": { "anyOf": [ { "items": { - "$ref": "#/components/schemas/InverterParameters" + "$ref": "#/components/schemas/InverterCommonSettings-Input" }, "type": "array" }, @@ -2109,11 +2665,28 @@ [] ] }, + "max_inverters": { + "anyOf": [ + { + "type": "integer", + "minimum": 0.0 + }, + { + "type": "null" + } + ], + "title": "Max Inverters", + "description": "Maximum number of inverters that can be set", + "examples": [ + 1, + 2 + ] + }, "home_appliances": { "anyOf": [ { "items": { - "$ref": "#/components/schemas/HomeApplianceParameters" + "$ref": "#/components/schemas/HomeApplianceCommonSettings-Input" }, "type": "array" }, @@ -2126,13 +2699,227 @@ "examples": [ [] ] + }, + "max_home_appliances": { + "anyOf": [ + { + "type": "integer", + "minimum": 0.0 + }, + { + "type": "null" + } + ], + "title": "Max Home Appliances", + "description": "Maximum number of home_appliances that can be set", + "examples": [ + 1, + 2 + ] } }, "type": "object", "title": "DevicesCommonSettings", "description": "Base configuration for devices simulation settings." }, - "ElecPriceCommonSettings": { + "DevicesCommonSettings-Output": { + "properties": { + "batteries": { + "anyOf": [ + { + "items": { + "$ref": "#/components/schemas/BatteriesCommonSettings-Output" + }, + "type": "array" + }, + { + "type": "null" + } + ], + "title": "Batteries", + "description": "List of battery devices", + "examples": [ + [ + { + "capacity_wh": 8000, + "device_id": "battery1" + } + ] + ] + }, + "max_batteries": { + "anyOf": [ + { + "type": "integer", + "minimum": 0.0 + }, + { + "type": "null" + } + ], + "title": "Max Batteries", + "description": "Maximum number of batteries that can be set", + "examples": [ + 1, + 2 + ] + }, + "electric_vehicles": { + "anyOf": [ + { + "items": { + "$ref": "#/components/schemas/BatteriesCommonSettings-Output" + }, + "type": "array" + }, + { + "type": "null" + } + ], + "title": "Electric Vehicles", + "description": "List of electric vehicle devices", + "examples": [ + [ + { + "capacity_wh": 8000, + "device_id": "battery1" + } + ] + ] + }, + "max_electric_vehicles": { + "anyOf": [ + { + "type": "integer", + "minimum": 0.0 + }, + { + "type": "null" + } + ], + "title": "Max Electric Vehicles", + "description": "Maximum number of electric vehicles that can be set", + "examples": [ + 1, + 2 + ] + }, + "inverters": { + "anyOf": [ + { + "items": { + "$ref": "#/components/schemas/InverterCommonSettings-Output" + }, + "type": "array" + }, + { + "type": "null" + } + ], + "title": "Inverters", + "description": "List of inverters", + "examples": [ + [] + ] + }, + "max_inverters": { + "anyOf": [ + { + "type": "integer", + "minimum": 0.0 + }, + { + "type": "null" + } + ], + "title": "Max Inverters", + "description": "Maximum number of inverters that can be set", + "examples": [ + 1, + 2 + ] + }, + "home_appliances": { + "anyOf": [ + { + "items": { + "$ref": "#/components/schemas/HomeApplianceCommonSettings-Output" + }, + "type": "array" + }, + { + "type": "null" + } + ], + "title": "Home Appliances", + "description": "List of home appliances", + "examples": [ + [] + ] + }, + "max_home_appliances": { + "anyOf": [ + { + "type": "integer", + "minimum": 0.0 + }, + { + "type": "null" + } + ], + "title": "Max Home Appliances", + "description": "Maximum number of home_appliances that can be set", + "examples": [ + 1, + 2 + ] + }, + "measurement_keys": { + "anyOf": [ + { + "items": { + "type": "string" + }, + "type": "array" + }, + { + "type": "null" + } + ], + "title": "Measurement Keys", + "description": "Return the measurement keys for the resource/ device stati that are measurements.", + "readOnly": true + } + }, + "type": "object", + "required": [ + "measurement_keys" + ], + "title": "DevicesCommonSettings", + "description": "Base configuration for devices simulation settings." + }, + "ElecPriceCommonProviderSettings": { + "properties": { + "ElecPriceImport": { + "anyOf": [ + { + "$ref": "#/components/schemas/ElecPriceImportCommonSettings" + }, + { + "type": "null" + } + ], + "description": "ElecPriceImport settings", + "examples": [ + null + ] + } + }, + "type": "object", + "title": "ElecPriceCommonProviderSettings", + "description": "Electricity Price Prediction Provider Configuration." + }, + "ElecPriceCommonSettings-Input": { "properties": { "provider": { "anyOf": [ @@ -2160,7 +2947,7 @@ } ], "title": "Charges Kwh", - "description": "Electricity price charges (\u20ac/kWh).", + "description": "Electricity price charges [\u20ac/kWh]. Will be added to variable market price.", "examples": [ 0.21 ] @@ -2183,17 +2970,72 @@ ] }, "provider_settings": { + "$ref": "#/components/schemas/ElecPriceCommonProviderSettings", + "description": "Provider settings", + "examples": [ + {} + ] + } + }, + "type": "object", + "title": "ElecPriceCommonSettings", + "description": "Electricity Price Prediction Configuration." + }, + "ElecPriceCommonSettings-Output": { + "properties": { + "provider": { "anyOf": [ { - "$ref": "#/components/schemas/ElecPriceImportCommonSettings" + "type": "string" }, { "type": "null" } ], + "title": "Provider", + "description": "Electricity price provider id of provider to be used.", + "examples": [ + "ElecPriceAkkudoktor" + ] + }, + "charges_kwh": { + "anyOf": [ + { + "type": "number", + "minimum": 0.0 + }, + { + "type": "null" + } + ], + "title": "Charges Kwh", + "description": "Electricity price charges [\u20ac/kWh]. Will be added to variable market price.", + "examples": [ + 0.21 + ] + }, + "vat_rate": { + "anyOf": [ + { + "type": "number", + "minimum": 0.0 + }, + { + "type": "null" + } + ], + "title": "Vat Rate", + "description": "VAT rate factor applied to electricity price when charges are used.", + "default": 1.19, + "examples": [ + 1.19 + ] + }, + "provider_settings": { + "$ref": "#/components/schemas/ElecPriceCommonProviderSettings", "description": "Provider settings", "examples": [ - null + {} ] } }, @@ -2454,69 +3296,632 @@ "examples": [ "300" ] + }, + "mode": { + "anyOf": [ + { + "$ref": "#/components/schemas/EnergyManagementMode" + }, + { + "type": "null" + } + ], + "description": "Energy management mode [OPTIMIZATION | PREDICTION].", + "examples": [ + "OPTIMIZATION", + "PREDICTION" + ] } }, "type": "object", "title": "EnergyManagementCommonSettings", "description": "Energy Management Configuration." }, - "EnergyManagementParameters": { + "EnergyManagementMode": { + "type": "string", + "enum": [ + "PREDICTION", + "OPTIMIZATION" + ], + "title": "EnergyManagementMode", + "description": "Energy management mode." + }, + "EnergyManagementPlan": { "properties": { - "pv_prognose_wh": { - "items": { - "type": "number" - }, - "type": "array", - "title": "Pv Prognose Wh", - "description": "An array of floats representing the forecasted photovoltaic output in watts for different time intervals." + "id": { + "type": "string", + "title": "Id", + "description": "Unique ID for the energy management plan." }, - "strompreis_euro_pro_wh": { - "items": { - "type": "number" - }, - "type": "array", - "title": "Strompreis Euro Pro Wh", - "description": "An array of floats representing the electricity price in euros per watt-hour for different time intervals." + "generated_at": { + "type": "string", + "format": "date-time", + "title": "Generated At", + "description": "Timestamp when the plan was generated." }, - "einspeiseverguetung_euro_pro_wh": { + "valid_from": { "anyOf": [ { - "items": { - "type": "number" - }, - "type": "array" + "type": "string", + "format": "date-time" }, { - "type": "number" + "type": "null" } ], - "title": "Einspeiseverguetung Euro Pro Wh", - "description": "A float or array of floats representing the feed-in compensation in euros per watt-hour." + "title": "Valid From", + "description": "Earliest start time of any instruction." }, - "preis_euro_pro_wh_akku": { - "type": "number", - "title": "Preis Euro Pro Wh Akku", - "description": "A float representing the cost of battery energy per watt-hour." + "valid_until": { + "anyOf": [ + { + "type": "string", + "format": "date-time" + }, + { + "type": "null" + } + ], + "title": "Valid Until", + "description": "Latest end time across all instructions with finite duration; None if all instructions have infinite duration." }, - "gesamtlast": { + "instructions": { "items": { - "type": "number" + "oneOf": [ + { + "$ref": "#/components/schemas/PEBCInstruction" + }, + { + "$ref": "#/components/schemas/PPBCScheduleInstruction" + }, + { + "$ref": "#/components/schemas/PPBCStartInterruptionInstruction" + }, + { + "$ref": "#/components/schemas/PPBCEndInterruptionInstruction" + }, + { + "$ref": "#/components/schemas/OMBCInstruction" + }, + { + "$ref": "#/components/schemas/FRBCInstruction" + }, + { + "$ref": "#/components/schemas/DDBCInstruction" + } + ], + "discriminator": { + "propertyName": "type", + "mapping": { + "DDBCInstruction": "#/components/schemas/DDBCInstruction", + "FRBCInstruction": "#/components/schemas/FRBCInstruction", + "OMBCInstruction": "#/components/schemas/OMBCInstruction", + "PEBCInstruction": "#/components/schemas/PEBCInstruction", + "PPBCEndInterruptionInstruction": "#/components/schemas/PPBCEndInterruptionInstruction", + "PPBCScheduleInstruction": "#/components/schemas/PPBCScheduleInstruction", + "PPBCStartInterruptionInstruction": "#/components/schemas/PPBCStartInterruptionInstruction" + } + } }, "type": "array", - "title": "Gesamtlast", - "description": "An array of floats representing the total load (consumption) in watts for different time intervals." + "title": "Instructions", + "description": "List of control instructions for the plan." + }, + "comment": { + "anyOf": [ + { + "type": "string" + }, + { + "type": "null" + } + ], + "title": "Comment", + "description": "Optional comment or annotation for the plan." } }, - "additionalProperties": false, "type": "object", "required": [ - "pv_prognose_wh", - "strompreis_euro_pro_wh", - "einspeiseverguetung_euro_pro_wh", - "preis_euro_pro_wh_akku", - "gesamtlast" + "id", + "generated_at", + "instructions" ], - "title": "EnergyManagementParameters" + "title": "EnergyManagementPlan", + "description": "A coordinated energy management plan composed of device control instructions.\n\nAttributes:\n plan_id (ID): Unique identifier for this energy management plan.\n generated_at (DateTime): Timestamp when the plan was generated.\n valid_from (Optional[DateTime]): Earliest start time of any instruction.\n valid_until (Optional[DateTime]): Latest end time across all instructions\n with finite duration; None if all instructions have infinite duration.\n instructions (list[BaseInstruction]): List of control instructions for the plan.\n comment (Optional[str]): Optional comment or annotation for the plan." + }, + "EnergyMeasurement-Input": { + "properties": { + "type": { + "type": "string", + "const": "EnergyMeasurement", + "title": "Type", + "default": "EnergyMeasurement" + }, + "measurement_timestamp": { + "type": "string", + "format": "date-time", + "title": "Measurement Timestamp", + "description": "Timestamp when energy values were measured." + }, + "last_reset": { + "anyOf": [ + { + "type": "string", + "format": "date-time" + }, + { + "type": "null" + } + ], + "title": "Last Reset", + "description": "Timestamp when the energy meter's cumulative counter was last reset." + }, + "values": { + "items": { + "$ref": "#/components/schemas/PowerValue" + }, + "type": "array", + "title": "Values", + "description": "Array of measured energy values. Shall contain at least one item and at most one item per 'commodity_quantity' (defined inside the PowerValue)." + } + }, + "type": "object", + "required": [ + "measurement_timestamp", + "values" + ], + "title": "EnergyMeasurement", + "description": "Captures a set of energy meter readouts taken at a specific point in time.\n\nEnergy is defined as the cummulative power per hour as provided by an energy meter.\n\nThis model records multiple energy values (for different commodity quantities)\nalong with the timestamp when the meter readouts were taken, enabling time-series\nanalysis and monitoring of energy consumption or production.\n\nNote: This is an extension to the S2 standard." + }, + "EnergyMeasurement-Output": { + "properties": { + "type": { + "type": "string", + "const": "EnergyMeasurement", + "title": "Type", + "default": "EnergyMeasurement" + }, + "measurement_timestamp": { + "type": "string", + "format": "date-time", + "title": "Measurement Timestamp", + "description": "Timestamp when energy values were measured." + }, + "last_reset": { + "anyOf": [ + { + "type": "string", + "format": "date-time" + }, + { + "type": "null" + } + ], + "title": "Last Reset", + "description": "Timestamp when the energy meter's cumulative counter was last reset." + }, + "values": { + "items": { + "$ref": "#/components/schemas/PowerValue" + }, + "type": "array", + "title": "Values", + "description": "Array of measured energy values. Shall contain at least one item and at most one item per 'commodity_quantity' (defined inside the PowerValue)." + } + }, + "type": "object", + "required": [ + "measurement_timestamp", + "values" + ], + "title": "EnergyMeasurement", + "description": "Captures a set of energy meter readouts taken at a specific point in time.\n\nEnergy is defined as the cummulative power per hour as provided by an energy meter.\n\nThis model records multiple energy values (for different commodity quantities)\nalong with the timestamp when the meter readouts were taken, enabling time-series\nanalysis and monitoring of energy consumption or production.\n\nNote: This is an extension to the S2 standard." + }, + "FRBCActuatorStatus": { + "properties": { + "type": { + "type": "string", + "const": "FRBCActuatorStatus", + "title": "Type", + "default": "FRBCActuatorStatus" + }, + "active_operation_mode_id": { + "type": "string", + "title": "Active Operation Mode Id", + "description": "Currently active operation mode ID." + }, + "operation_mode_factor": { + "type": "number", + "maximum": 1.0, + "minimum": 0.0, + "title": "Operation Mode Factor", + "description": "Factor with which the mode is configured (0 to 1)." + }, + "previous_operation_mode_id": { + "anyOf": [ + { + "type": "string" + }, + { + "type": "null" + } + ], + "title": "Previous Operation Mode Id", + "description": "Previously active operation mode ID." + }, + "transition_timestamp": { + "anyOf": [ + { + "type": "string", + "format": "date-time" + }, + { + "type": "null" + } + ], + "title": "Transition Timestamp", + "description": "Timestamp of the last transition between modes." + } + }, + "type": "object", + "required": [ + "active_operation_mode_id", + "operation_mode_factor" + ], + "title": "FRBCActuatorStatus", + "description": "Current status of an FRBC Actuator.\n\nProvides information about the currently active operation mode and transition history.\nUsed to track the current state of the actuator." + }, + "FRBCEnergyStatus-Input": { + "properties": { + "type": { + "type": "string", + "const": "FRBCEnergyStatus", + "title": "Type", + "default": "FRBCEnergyStatus" + }, + "import_total": { + "anyOf": [ + { + "$ref": "#/components/schemas/EnergyMeasurement-Input" + }, + { + "type": "null" + } + ], + "description": "Total cumulative imported energy from the energy meter start." + }, + "export_total": { + "anyOf": [ + { + "$ref": "#/components/schemas/EnergyMeasurement-Input" + }, + { + "type": "null" + } + ], + "description": "Total cumulative exported energy from the energy meter start." + } + }, + "type": "object", + "title": "FRBCEnergyStatus", + "description": "Energy status of an FRBC storage.\n\nNote: This is an extension to the S2 standard." + }, + "FRBCEnergyStatus-Output": { + "properties": { + "type": { + "type": "string", + "const": "FRBCEnergyStatus", + "title": "Type", + "default": "FRBCEnergyStatus" + }, + "import_total": { + "anyOf": [ + { + "$ref": "#/components/schemas/EnergyMeasurement-Output" + }, + { + "type": "null" + } + ], + "description": "Total cumulative imported energy from the energy meter start." + }, + "export_total": { + "anyOf": [ + { + "$ref": "#/components/schemas/EnergyMeasurement-Output" + }, + { + "type": "null" + } + ], + "description": "Total cumulative exported energy from the energy meter start." + } + }, + "type": "object", + "title": "FRBCEnergyStatus", + "description": "Energy status of an FRBC storage.\n\nNote: This is an extension to the S2 standard." + }, + "FRBCInstruction": { + "properties": { + "id": { + "anyOf": [ + { + "type": "string" + }, + { + "type": "null" + } + ], + "title": "Id", + "description": "Unique identifier of the instruction in the ResourceManager scope. If not provided and a `resource_id` is passed at instantiation, this will be auto-generated as `{resource_id}@{UUID}`." + }, + "execution_time": { + "type": "string", + "format": "date-time", + "title": "Execution Time", + "description": "Start time of the instruction execution." + }, + "abnormal_condition": { + "type": "boolean", + "title": "Abnormal Condition", + "description": "Indicates if this is an instruction for abnormal conditions. Defaults to False.", + "default": false + }, + "type": { + "type": "string", + "const": "FRBCInstruction", + "title": "Type", + "default": "FRBCInstruction" + }, + "actuator_id": { + "type": "string", + "title": "Actuator Id", + "description": "ID of the actuator this instruction belongs to." + }, + "operation_mode_id": { + "type": "string", + "title": "Operation Mode Id", + "description": "ID of the operation mode to activate." + }, + "operation_mode_factor": { + "type": "number", + "maximum": 1.0, + "minimum": 0.0, + "title": "Operation Mode Factor", + "description": "Factor for the operation mode configuration (0 to 1)." + }, + "resource_id": { + "type": "string", + "title": "Resource Id", + "description": "Get the resource identifier component from the instruction's `id`.\n\nAssumes the `id` follows the format `{resource_id}@{UUID}`. Extracts the resource_id part\nof the id by splitting at the last @.\n\nReturns:\n str: The resource identifier prefix of `id`, or an empty string if `id` is None.", + "readOnly": true + } + }, + "type": "object", + "required": [ + "execution_time", + "actuator_id", + "operation_mode_id", + "operation_mode_factor", + "resource_id" + ], + "title": "FRBCInstruction", + "description": "Instruction for Fill Rate Based Control (FRBC).\n\nContains information about when and how to activate a specific operation mode\nfor an actuator. Used to command resources to change their operation at a specified time." + }, + "FRBCStorageStatus": { + "properties": { + "type": { + "type": "string", + "const": "FRBCStorageStatus", + "title": "Type", + "default": "FRBCStorageStatus" + }, + "present_fill_level": { + "type": "number", + "title": "Present Fill Level", + "description": "Current fill level of the storage." + } + }, + "type": "object", + "required": [ + "present_fill_level" + ], + "title": "FRBCStorageStatus", + "description": "Current status of an FRBC Storage.\n\nIndicates the current fill level of the storage, which is essential\nfor determining applicable operation modes and control decisions." + }, + "FRBCTimerStatus": { + "properties": { + "type": { + "type": "string", + "const": "FRBCTimerStatus", + "title": "Type", + "default": "FRBCTimerStatus" + }, + "actuator_id": { + "type": "string", + "title": "Actuator Id", + "description": "ID of the actuator the timer belongs to." + }, + "timer_id": { + "type": "string", + "title": "Timer Id", + "description": "ID of the timer this status refers to." + }, + "finished_at": { + "type": "string", + "format": "date-time", + "title": "Finished At", + "description": "Indicates when the Timer will be finished. If the DateTime is in the future, the timer is not yet finished. If the DateTime is in the past, the timer is finished. If the timer was never started, the value can be an arbitrary DateTimeStamp in the past." + } + }, + "type": "object", + "required": [ + "actuator_id", + "timer_id", + "finished_at" + ], + "title": "FRBCTimerStatus", + "description": "Current status of an FRBC Timer.\n\nIndicates when the Timer will be finished." + }, + "FeedInTariffCommonProviderSettings": { + "properties": { + "FeedInTariffFixed": { + "anyOf": [ + { + "$ref": "#/components/schemas/FeedInTariffFixedCommonSettings" + }, + { + "type": "null" + } + ], + "description": "FeedInTariffFixed settings", + "examples": [ + null + ] + }, + "FeedInTariffImport": { + "anyOf": [ + { + "$ref": "#/components/schemas/FeedInTariffImportCommonSettings" + }, + { + "type": "null" + } + ], + "description": "FeedInTariffImport settings", + "examples": [ + null + ] + } + }, + "type": "object", + "title": "FeedInTariffCommonProviderSettings", + "description": "Feed In Tariff Prediction Provider Configuration." + }, + "FeedInTariffCommonSettings-Input": { + "properties": { + "provider": { + "anyOf": [ + { + "type": "string" + }, + { + "type": "null" + } + ], + "title": "Provider", + "description": "Feed in tariff provider id of provider to be used.", + "examples": [ + "FeedInTariffFixed", + "FeedInTarifImport" + ] + }, + "provider_settings": { + "$ref": "#/components/schemas/FeedInTariffCommonProviderSettings", + "description": "Provider settings", + "examples": [ + {} + ] + } + }, + "type": "object", + "title": "FeedInTariffCommonSettings", + "description": "Feed In Tariff Prediction Configuration." + }, + "FeedInTariffCommonSettings-Output": { + "properties": { + "provider": { + "anyOf": [ + { + "type": "string" + }, + { + "type": "null" + } + ], + "title": "Provider", + "description": "Feed in tariff provider id of provider to be used.", + "examples": [ + "FeedInTariffFixed", + "FeedInTarifImport" + ] + }, + "provider_settings": { + "$ref": "#/components/schemas/FeedInTariffCommonProviderSettings", + "description": "Provider settings", + "examples": [ + {} + ] + } + }, + "type": "object", + "title": "FeedInTariffCommonSettings", + "description": "Feed In Tariff Prediction Configuration." + }, + "FeedInTariffFixedCommonSettings": { + "properties": { + "feed_in_tariff_kwh": { + "anyOf": [ + { + "type": "number", + "minimum": 0.0 + }, + { + "type": "null" + } + ], + "title": "Feed In Tariff Kwh", + "description": "Electricity price feed in tariff [\u20ac/kWH].", + "examples": [ + 0.078 + ] + } + }, + "type": "object", + "title": "FeedInTariffFixedCommonSettings", + "description": "Common settings for elecprice fixed price." + }, + "FeedInTariffImportCommonSettings": { + "properties": { + "import_file_path": { + "anyOf": [ + { + "type": "string" + }, + { + "type": "string", + "format": "path" + }, + { + "type": "null" + } + ], + "title": "Import File Path", + "description": "Path to the file to import feed in tariff data from.", + "examples": [ + null, + "/path/to/feedintariff.json" + ] + }, + "import_json": { + "anyOf": [ + { + "type": "string" + }, + { + "type": "null" + } + ], + "title": "Import Json", + "description": "JSON string, dictionary of feed in tariff forecast value lists.", + "examples": [ + "{\"fead_in_tariff_wh\": [0.000078, 0.000078, 0.000023]}" + ] + } + }, + "type": "object", + "title": "FeedInTariffImportCommonSettings", + "description": "Common settings for feed in tariff data import from file or JSON string." }, "ForecastResponse": { "properties": { @@ -2551,6 +3956,12 @@ }, "GeneralSettings-Input": { "properties": { + "version": { + "type": "string", + "title": "Version", + "description": "Configuration file version. Used to check compatibility.", + "default": "0.1.0+dev" + }, "data_folder_path": { "anyOf": [ { @@ -2619,6 +4030,12 @@ }, "GeneralSettings-Output": { "properties": { + "version": { + "type": "string", + "title": "Version", + "description": "Configuration file version. Used to check compatibility.", + "default": "0.1.0+dev" + }, "data_folder_path": { "anyOf": [ { @@ -2746,514 +4163,43 @@ "title": "GeneralSettings", "description": "Settings for common configuration.\n\nGeneral configuration to set directories of cache and output files and system location (latitude\nand longitude).\nValidators ensure each parameter is within a specified range. A computed property, `timezone`,\ndetermines the time zone based on latitude and longitude.\n\nAttributes:\n latitude (Optional[float]): Latitude in degrees, must be between -90 and 90.\n longitude (Optional[float]): Longitude in degrees, must be between -180 and 180.\n\nProperties:\n timezone (Optional[str]): Computed time zone string based on the specified latitude\n and longitude." }, - "GesamtlastRequest": { + "GeneticCommonSettings": { "properties": { - "year_energy": { - "type": "number", - "title": "Year Energy" - }, - "measured_data": { - "items": { - "additionalProperties": true, - "type": "object" - }, - "type": "array", - "title": "Measured Data" - }, - "hours": { - "type": "integer", - "title": "Hours" - } - }, - "type": "object", - "required": [ - "year_energy", - "measured_data", - "hours" - ], - "title": "GesamtlastRequest" - }, - "HTTPValidationError": { - "properties": { - "detail": { - "items": { - "$ref": "#/components/schemas/ValidationError" - }, - "type": "array", - "title": "Detail" - } - }, - "type": "object", - "title": "HTTPValidationError" - }, - "HomeApplianceParameters": { - "properties": { - "device_id": { - "type": "string", - "title": "Device Id", - "description": "ID of home appliance", - "examples": [ - "dishwasher" - ] - }, - "hours": { + "individuals": { "anyOf": [ { "type": "integer", - "exclusiveMinimum": 0.0 + "minimum": 10.0 }, { "type": "null" } ], - "title": "Hours", - "description": "Number of prediction hours. Defaults to global config prediction hours.", + "title": "Individuals", + "description": "Number of individuals (solutions) to generate for the (initial) generation [>= 10]. Defaults to 300.", + "default": 300, "examples": [ - null + 300 ] }, - "consumption_wh": { - "type": "integer", - "exclusiveMinimum": 0.0, - "title": "Consumption Wh", - "description": "An integer representing the energy consumption of a household device in watt-hours.", - "examples": [ - 2000 - ] - }, - "duration_h": { - "type": "integer", - "exclusiveMinimum": 0.0, - "title": "Duration H", - "description": "An integer representing the usage duration of a household device in hours.", - "examples": [ - 3 - ] - } - }, - "additionalProperties": false, - "type": "object", - "required": [ - "device_id", - "consumption_wh", - "duration_h" - ], - "title": "HomeApplianceParameters", - "description": "Home Appliance Device Simulation Configuration." - }, - "InverterParameters": { - "properties": { - "device_id": { - "type": "string", - "title": "Device Id", - "description": "ID of inverter", - "examples": [ - "inverter1" - ] - }, - "hours": { + "generations": { "anyOf": [ { "type": "integer", - "exclusiveMinimum": 0.0 + "minimum": 10.0 }, { "type": "null" } ], - "title": "Hours", - "description": "Number of prediction hours. Defaults to global config prediction hours.", + "title": "Generations", + "description": "Number of generations to evaluate the optimal solution [>= 10]. Defaults to 400.", + "default": 400, "examples": [ - null + 400 ] }, - "max_power_wh": { - "type": "number", - "exclusiveMinimum": 0.0, - "title": "Max Power Wh", - "examples": [ - 10000 - ] - }, - "battery_id": { - "anyOf": [ - { - "type": "string" - }, - { - "type": "null" - } - ], - "title": "Battery Id", - "description": "ID of battery", - "examples": [ - null, - "battery1" - ] - } - }, - "additionalProperties": false, - "type": "object", - "required": [ - "device_id", - "max_power_wh" - ], - "title": "InverterParameters", - "description": "Inverter Device Simulation Configuration." - }, - "LoadAkkudoktorCommonSettings": { - "properties": { - "loadakkudoktor_year_energy": { - "anyOf": [ - { - "type": "number" - }, - { - "type": "null" - } - ], - "title": "Loadakkudoktor Year Energy", - "description": "Yearly energy consumption (kWh).", - "examples": [ - 40421 - ] - } - }, - "type": "object", - "title": "LoadAkkudoktorCommonSettings", - "description": "Common settings for load data import from file." - }, - "LoadCommonSettings": { - "properties": { - "provider": { - "anyOf": [ - { - "type": "string" - }, - { - "type": "null" - } - ], - "title": "Provider", - "description": "Load provider id of provider to be used.", - "examples": [ - "LoadAkkudoktor" - ] - }, - "provider_settings": { - "anyOf": [ - { - "$ref": "#/components/schemas/LoadAkkudoktorCommonSettings" - }, - { - "$ref": "#/components/schemas/LoadVrmCommonSettings" - }, - { - "$ref": "#/components/schemas/LoadImportCommonSettings" - }, - { - "type": "null" - } - ], - "title": "Provider Settings", - "description": "Provider settings", - "examples": [ - null - ] - } - }, - "type": "object", - "title": "LoadCommonSettings", - "description": "Load Prediction Configuration." - }, - "LoadImportCommonSettings": { - "properties": { - "import_file_path": { - "anyOf": [ - { - "type": "string" - }, - { - "type": "string", - "format": "path" - }, - { - "type": "null" - } - ], - "title": "Import File Path", - "description": "Path to the file to import load data from.", - "examples": [ - null, - "/path/to/yearly_load.json" - ] - }, - "import_json": { - "anyOf": [ - { - "type": "string" - }, - { - "type": "null" - } - ], - "title": "Import Json", - "description": "JSON string, dictionary of load forecast value lists.", - "examples": [ - "{\"load0_mean\": [676.71, 876.19, 527.13]}" - ] - } - }, - "type": "object", - "title": "LoadImportCommonSettings", - "description": "Common settings for load data import from file or JSON string." - }, - "LoadVrmCommonSettings": { - "properties": { - "load_vrm_token": { - "type": "string", - "title": "Load Vrm Token", - "description": "Token for Connecting VRM API", - "default": "your-token", - "examples": [ - "your-token" - ] - }, - "load_vrm_idsite": { - "type": "integer", - "title": "Load Vrm Idsite", - "description": "VRM-Installation-ID", - "default": 12345, - "examples": [ - 12345 - ] - } - }, - "type": "object", - "title": "LoadVrmCommonSettings", - "description": "Common settings for VRM API." - }, - "LoggingCommonSettings-Input": { - "properties": { - "level": { - "anyOf": [ - { - "type": "string" - }, - { - "type": "null" - } - ], - "title": "Level", - "deprecated": true - }, - "console_level": { - "anyOf": [ - { - "type": "string" - }, - { - "type": "null" - } - ], - "title": "Console Level", - "description": "Logging level when logging to console.", - "examples": [ - "TRACE", - "DEBUG", - "INFO", - "WARNING", - "ERROR", - "CRITICAL" - ] - }, - "file_level": { - "anyOf": [ - { - "type": "string" - }, - { - "type": "null" - } - ], - "title": "File Level", - "description": "Logging level when logging to file.", - "examples": [ - "TRACE", - "DEBUG", - "INFO", - "WARNING", - "ERROR", - "CRITICAL" - ] - } - }, - "type": "object", - "title": "LoggingCommonSettings", - "description": "Logging Configuration." - }, - "LoggingCommonSettings-Output": { - "properties": { - "level": { - "anyOf": [ - { - "type": "string" - }, - { - "type": "null" - } - ], - "title": "Level", - "deprecated": true - }, - "console_level": { - "anyOf": [ - { - "type": "string" - }, - { - "type": "null" - } - ], - "title": "Console Level", - "description": "Logging level when logging to console.", - "examples": [ - "TRACE", - "DEBUG", - "INFO", - "WARNING", - "ERROR", - "CRITICAL" - ] - }, - "file_level": { - "anyOf": [ - { - "type": "string" - }, - { - "type": "null" - } - ], - "title": "File Level", - "description": "Logging level when logging to file.", - "examples": [ - "TRACE", - "DEBUG", - "INFO", - "WARNING", - "ERROR", - "CRITICAL" - ] - }, - "file_path": { - "anyOf": [ - { - "type": "string", - "format": "path" - }, - { - "type": "null" - } - ], - "title": "File Path", - "description": "Computed log file path based on data output path.", - "readOnly": true - } - }, - "type": "object", - "required": [ - "file_path" - ], - "title": "LoggingCommonSettings", - "description": "Logging Configuration." - }, - "MeasurementCommonSettings": { - "properties": { - "load0_name": { - "anyOf": [ - { - "type": "string" - }, - { - "type": "null" - } - ], - "title": "Load0 Name", - "description": "Name of the load0 source", - "examples": [ - "Household", - "Heat Pump" - ] - }, - "load1_name": { - "anyOf": [ - { - "type": "string" - }, - { - "type": "null" - } - ], - "title": "Load1 Name", - "description": "Name of the load1 source", - "examples": [ - null - ] - }, - "load2_name": { - "anyOf": [ - { - "type": "string" - }, - { - "type": "null" - } - ], - "title": "Load2 Name", - "description": "Name of the load2 source", - "examples": [ - null - ] - }, - "load3_name": { - "anyOf": [ - { - "type": "string" - }, - { - "type": "null" - } - ], - "title": "Load3 Name", - "description": "Name of the load3 source", - "examples": [ - null - ] - }, - "load4_name": { - "anyOf": [ - { - "type": "string" - }, - { - "type": "null" - } - ], - "title": "Load4 Name", - "description": "Name of the load4 source", - "examples": [ - null - ] - } - }, - "type": "object", - "title": "MeasurementCommonSettings", - "description": "Measurement Configuration." - }, - "OptimizationCommonSettings": { - "properties": { - "hours": { + "seed": { "anyOf": [ { "type": "integer", @@ -3263,24 +4209,66 @@ "type": "null" } ], - "title": "Hours", - "description": "Number of hours into the future for optimizations.", - "default": 48 + "title": "Seed", + "description": "Fixed seed for genetic algorithm. Defaults to 'None' which means random seed.", + "examples": [ + null + ] }, - "penalty": { + "penalties": { "anyOf": [ { - "type": "integer" + "additionalProperties": { + "anyOf": [ + { + "type": "number" + }, + { + "type": "integer" + }, + { + "type": "string" + } + ] + }, + "type": "object" }, { "type": "null" } ], - "title": "Penalty", - "description": "Penalty factor used in optimization.", - "default": 10 + "title": "Penalties", + "description": "A dictionary of penalty function parameters consisting of a penalty function parameter name and the associated value.", + "examples": [ + { + "ev_soc_miss": 10 + } + ] + } + }, + "type": "object", + "title": "GeneticCommonSettings", + "description": "General Genetic Optimization Algorithm Configuration." + }, + "GeneticEnergyManagementParameters": { + "properties": { + "pv_prognose_wh": { + "items": { + "type": "number" + }, + "type": "array", + "title": "Pv Prognose Wh", + "description": "An array of floats representing the forecasted photovoltaic output in watts for different time intervals." }, - "ev_available_charge_rates_percent": { + "strompreis_euro_pro_wh": { + "items": { + "type": "number" + }, + "type": "array", + "title": "Strompreis Euro Pro Wh", + "description": "An array of floats representing the electricity price in euros per watt-hour for different time intervals." + }, + "einspeiseverguetung_euro_pro_wh": { "anyOf": [ { "items": { @@ -3289,30 +4277,42 @@ "type": "array" }, { - "type": "null" + "type": "number" } ], - "title": "Ev Available Charge Rates Percent", - "description": "Charge rates available for the EV in percent of maximum charge.", - "default": [ - 0.0, - 0.375, - 0.5, - 0.625, - 0.75, - 0.875, - 1.0 - ] + "title": "Einspeiseverguetung Euro Pro Wh", + "description": "A float or array of floats representing the feed-in compensation in euros per watt-hour." + }, + "preis_euro_pro_wh_akku": { + "type": "number", + "title": "Preis Euro Pro Wh Akku", + "description": "A float representing the cost of battery energy per watt-hour." + }, + "gesamtlast": { + "items": { + "type": "number" + }, + "type": "array", + "title": "Gesamtlast", + "description": "An array of floats representing the total load (consumption) in watts for different time intervals." } }, + "additionalProperties": false, "type": "object", - "title": "OptimizationCommonSettings", - "description": "General Optimization Configuration.\n\nAttributes:\n hours (int): Number of hours for optimizations." + "required": [ + "pv_prognose_wh", + "strompreis_euro_pro_wh", + "einspeiseverguetung_euro_pro_wh", + "preis_euro_pro_wh_akku", + "gesamtlast" + ], + "title": "GeneticEnergyManagementParameters", + "description": "Encapsulates energy-related forecasts and costs used in GENETIC optimization." }, - "OptimizationParameters": { + "GeneticOptimizationParameters": { "properties": { "ems": { - "$ref": "#/components/schemas/EnergyManagementParameters" + "$ref": "#/components/schemas/GeneticEnergyManagementParameters" }, "pv_akku": { "anyOf": [ @@ -3400,9 +4400,141 @@ "inverter", "eauto" ], - "title": "OptimizationParameters" + "title": "GeneticOptimizationParameters", + "description": "Main parameter class for running the genetic energy optimization.\n\nCollects all model and configuration parameters necessary to run the\noptimization process, such as forecasts, pricing, battery and appliance models." }, - "OptimizeResponse": { + "GeneticSimulationResult": { + "properties": { + "Last_Wh_pro_Stunde": { + "items": { + "type": "number" + }, + "type": "array", + "title": "Last Wh Pro Stunde", + "description": "TBD" + }, + "EAuto_SoC_pro_Stunde": { + "items": { + "type": "number" + }, + "type": "array", + "title": "Eauto Soc Pro Stunde", + "description": "The state of charge of the EV for each hour." + }, + "Einnahmen_Euro_pro_Stunde": { + "items": { + "type": "number" + }, + "type": "array", + "title": "Einnahmen Euro Pro Stunde", + "description": "The revenue from grid feed-in or other sources in euros per hour." + }, + "Gesamt_Verluste": { + "type": "number", + "title": "Gesamt Verluste", + "description": "The total losses in watt-hours over the entire period." + }, + "Gesamtbilanz_Euro": { + "type": "number", + "title": "Gesamtbilanz Euro", + "description": "The total balance of revenues minus costs in euros." + }, + "Gesamteinnahmen_Euro": { + "type": "number", + "title": "Gesamteinnahmen Euro", + "description": "The total revenues in euros." + }, + "Gesamtkosten_Euro": { + "type": "number", + "title": "Gesamtkosten Euro", + "description": "The total costs in euros." + }, + "Home_appliance_wh_per_hour": { + "items": { + "anyOf": [ + { + "type": "number" + }, + { + "type": "null" + } + ] + }, + "type": "array", + "title": "Home Appliance Wh Per Hour", + "description": "The energy consumption of a household appliance in watt-hours per hour." + }, + "Kosten_Euro_pro_Stunde": { + "items": { + "type": "number" + }, + "type": "array", + "title": "Kosten Euro Pro Stunde", + "description": "The costs in euros per hour." + }, + "Netzbezug_Wh_pro_Stunde": { + "items": { + "type": "number" + }, + "type": "array", + "title": "Netzbezug Wh Pro Stunde", + "description": "The grid energy drawn in watt-hours per hour." + }, + "Netzeinspeisung_Wh_pro_Stunde": { + "items": { + "type": "number" + }, + "type": "array", + "title": "Netzeinspeisung Wh Pro Stunde", + "description": "The energy fed into the grid in watt-hours per hour." + }, + "Verluste_Pro_Stunde": { + "items": { + "type": "number" + }, + "type": "array", + "title": "Verluste Pro Stunde", + "description": "The losses in watt-hours per hour." + }, + "akku_soc_pro_stunde": { + "items": { + "type": "number" + }, + "type": "array", + "title": "Akku Soc Pro Stunde", + "description": "The state of charge of the battery (not the EV) in percentage per hour." + }, + "Electricity_price": { + "items": { + "type": "number" + }, + "type": "array", + "title": "Electricity Price", + "description": "Used Electricity Price, including predictions" + } + }, + "additionalProperties": false, + "type": "object", + "required": [ + "Last_Wh_pro_Stunde", + "EAuto_SoC_pro_Stunde", + "Einnahmen_Euro_pro_Stunde", + "Gesamt_Verluste", + "Gesamtbilanz_Euro", + "Gesamteinnahmen_Euro", + "Gesamtkosten_Euro", + "Home_appliance_wh_per_hour", + "Kosten_Euro_pro_Stunde", + "Netzbezug_Wh_pro_Stunde", + "Netzeinspeisung_Wh_pro_Stunde", + "Verluste_Pro_Stunde", + "akku_soc_pro_stunde", + "Electricity_price" + ], + "title": "GeneticSimulationResult", + "description": "This object contains the results of the simulation and provides insights into various parameters over the entire forecast period." + }, + "GeneticSolution": { "properties": { "ac_charge": { "items": { @@ -3410,7 +4542,7 @@ }, "type": "array", "title": "Ac Charge", - "description": "Array with AC charging values as relative power (0-1), other values set to 0." + "description": "Array with AC charging values as relative power (0.0-1.0), other values set to 0." }, "dc_charge": { "items": { @@ -3444,7 +4576,7 @@ "description": "TBD" }, "result": { - "$ref": "#/components/schemas/SimulationResult" + "$ref": "#/components/schemas/GeneticSimulationResult" }, "eauto_obj": { "anyOf": [ @@ -3494,9 +4626,1663 @@ "result", "eauto_obj" ], - "title": "OptimizeResponse", + "title": "GeneticSolution", "description": "**Note**: The first value of \"Last_Wh_per_hour\", \"Netzeinspeisung_Wh_per_hour\", and \"Netzbezug_Wh_per_hour\", will be set to null in the JSON output and represented as NaN or None in the corresponding classes' data returns. This approach is adopted to ensure that the current hour's processing remains unchanged." }, + "GesamtlastRequest": { + "properties": { + "year_energy": { + "type": "number", + "title": "Year Energy" + }, + "measured_data": { + "items": { + "additionalProperties": true, + "type": "object" + }, + "type": "array", + "title": "Measured Data" + }, + "hours": { + "type": "integer", + "title": "Hours" + } + }, + "type": "object", + "required": [ + "year_energy", + "measured_data", + "hours" + ], + "title": "GesamtlastRequest" + }, + "HTTPValidationError": { + "properties": { + "detail": { + "items": { + "$ref": "#/components/schemas/ValidationError" + }, + "type": "array", + "title": "Detail" + } + }, + "type": "object", + "title": "HTTPValidationError" + }, + "HomeApplianceCommonSettings-Input": { + "properties": { + "device_id": { + "type": "string", + "title": "Device Id", + "description": "ID of device", + "default": "", + "examples": [ + "battery1", + "ev1", + "inverter1", + "dishwasher" + ] + }, + "consumption_wh": { + "type": "integer", + "exclusiveMinimum": 0.0, + "title": "Consumption Wh", + "description": "Energy consumption [Wh].", + "examples": [ + 2000 + ] + }, + "duration_h": { + "type": "integer", + "maximum": 24.0, + "exclusiveMinimum": 0.0, + "title": "Duration H", + "description": "Usage duration in hours [0 ... 24].", + "examples": [ + 1 + ] + }, + "time_windows": { + "anyOf": [ + { + "$ref": "#/components/schemas/TimeWindowSequence-Input" + }, + { + "type": "null" + } + ], + "description": "Sequence of allowed time windows. Defaults to optimization general time window.", + "examples": [ + { + "windows": [ + { + "duration": "2 hours", + "start_time": "10:00" + } + ] + } + ] + } + }, + "type": "object", + "required": [ + "consumption_wh", + "duration_h" + ], + "title": "HomeApplianceCommonSettings", + "description": "Home Appliance devices base settings." + }, + "HomeApplianceCommonSettings-Output": { + "properties": { + "device_id": { + "type": "string", + "title": "Device Id", + "description": "ID of device", + "default": "", + "examples": [ + "battery1", + "ev1", + "inverter1", + "dishwasher" + ] + }, + "consumption_wh": { + "type": "integer", + "exclusiveMinimum": 0.0, + "title": "Consumption Wh", + "description": "Energy consumption [Wh].", + "examples": [ + 2000 + ] + }, + "duration_h": { + "type": "integer", + "maximum": 24.0, + "exclusiveMinimum": 0.0, + "title": "Duration H", + "description": "Usage duration in hours [0 ... 24].", + "examples": [ + 1 + ] + }, + "time_windows": { + "anyOf": [ + { + "$ref": "#/components/schemas/TimeWindowSequence-Output" + }, + { + "type": "null" + } + ], + "description": "Sequence of allowed time windows. Defaults to optimization general time window.", + "examples": [ + { + "windows": [ + { + "duration": "2 hours", + "start_time": "10:00" + } + ] + } + ] + }, + "measurement_keys": { + "anyOf": [ + { + "items": { + "type": "string" + }, + "type": "array" + }, + { + "type": "null" + } + ], + "title": "Measurement Keys", + "description": "Measurement keys for the home appliance stati that are measurements.", + "readOnly": true + } + }, + "type": "object", + "required": [ + "consumption_wh", + "duration_h", + "measurement_keys" + ], + "title": "HomeApplianceCommonSettings", + "description": "Home Appliance devices base settings." + }, + "HomeApplianceParameters": { + "properties": { + "device_id": { + "type": "string", + "title": "Device Id", + "description": "ID of home appliance", + "examples": [ + "dishwasher" + ] + }, + "hours": { + "anyOf": [ + { + "type": "integer", + "exclusiveMinimum": 0.0 + }, + { + "type": "null" + } + ], + "title": "Hours", + "description": "Number of prediction hours. Defaults to global config prediction hours.", + "examples": [ + null + ] + }, + "consumption_wh": { + "type": "integer", + "exclusiveMinimum": 0.0, + "title": "Consumption Wh", + "description": "An integer representing the energy consumption of a household device in watt-hours.", + "examples": [ + 2000 + ] + }, + "duration_h": { + "type": "integer", + "exclusiveMinimum": 0.0, + "title": "Duration H", + "description": "An integer representing the usage duration of a household device in hours.", + "examples": [ + 3 + ] + }, + "time_windows": { + "anyOf": [ + { + "$ref": "#/components/schemas/TimeWindowSequence-Input" + }, + { + "type": "null" + } + ], + "description": "List of allowed time windows. Defaults to optimization general time window.", + "examples": [ + [ + { + "duration": "2 hours", + "start_time": "10:00" + } + ] + ] + } + }, + "additionalProperties": false, + "type": "object", + "required": [ + "device_id", + "consumption_wh", + "duration_h" + ], + "title": "HomeApplianceParameters", + "description": "Home Appliance Device Simulation Configuration." + }, + "InverterCommonSettings-Input": { + "properties": { + "device_id": { + "type": "string", + "title": "Device Id", + "description": "ID of device", + "default": "", + "examples": [ + "battery1", + "ev1", + "inverter1", + "dishwasher" + ] + }, + "max_power_w": { + "anyOf": [ + { + "type": "number", + "exclusiveMinimum": 0.0 + }, + { + "type": "null" + } + ], + "title": "Max Power W", + "description": "Maximum power [W].", + "examples": [ + 10000 + ] + }, + "battery_id": { + "anyOf": [ + { + "type": "string" + }, + { + "type": "null" + } + ], + "title": "Battery Id", + "description": "ID of battery controlled by this inverter.", + "examples": [ + null, + "battery1" + ] + } + }, + "type": "object", + "title": "InverterCommonSettings", + "description": "Inverter devices base settings." + }, + "InverterCommonSettings-Output": { + "properties": { + "device_id": { + "type": "string", + "title": "Device Id", + "description": "ID of device", + "default": "", + "examples": [ + "battery1", + "ev1", + "inverter1", + "dishwasher" + ] + }, + "max_power_w": { + "anyOf": [ + { + "type": "number", + "exclusiveMinimum": 0.0 + }, + { + "type": "null" + } + ], + "title": "Max Power W", + "description": "Maximum power [W].", + "examples": [ + 10000 + ] + }, + "battery_id": { + "anyOf": [ + { + "type": "string" + }, + { + "type": "null" + } + ], + "title": "Battery Id", + "description": "ID of battery controlled by this inverter.", + "examples": [ + null, + "battery1" + ] + }, + "measurement_keys": { + "anyOf": [ + { + "items": { + "type": "string" + }, + "type": "array" + }, + { + "type": "null" + } + ], + "title": "Measurement Keys", + "description": "Measurement keys for the inverter stati that are measurements.", + "readOnly": true + } + }, + "type": "object", + "required": [ + "measurement_keys" + ], + "title": "InverterCommonSettings", + "description": "Inverter devices base settings." + }, + "InverterParameters": { + "properties": { + "device_id": { + "type": "string", + "title": "Device Id", + "description": "ID of inverter", + "examples": [ + "inverter1" + ] + }, + "hours": { + "anyOf": [ + { + "type": "integer", + "exclusiveMinimum": 0.0 + }, + { + "type": "null" + } + ], + "title": "Hours", + "description": "Number of prediction hours. Defaults to global config prediction hours.", + "examples": [ + null + ] + }, + "max_power_wh": { + "type": "number", + "exclusiveMinimum": 0.0, + "title": "Max Power Wh", + "examples": [ + 10000 + ] + }, + "battery_id": { + "anyOf": [ + { + "type": "string" + }, + { + "type": "null" + } + ], + "title": "Battery Id", + "description": "ID of battery", + "examples": [ + null, + "battery1" + ] + } + }, + "additionalProperties": false, + "type": "object", + "required": [ + "device_id", + "max_power_wh" + ], + "title": "InverterParameters", + "description": "Inverter Device Simulation Configuration." + }, + "LoadAkkudoktorCommonSettings": { + "properties": { + "loadakkudoktor_year_energy": { + "anyOf": [ + { + "type": "number" + }, + { + "type": "null" + } + ], + "title": "Loadakkudoktor Year Energy", + "description": "Yearly energy consumption (kWh).", + "examples": [ + 40421 + ] + } + }, + "type": "object", + "title": "LoadAkkudoktorCommonSettings", + "description": "Common settings for load data import from file." + }, + "LoadCommonProviderSettings": { + "properties": { + "LoadAkkudoktor": { + "anyOf": [ + { + "$ref": "#/components/schemas/LoadAkkudoktorCommonSettings" + }, + { + "type": "null" + } + ], + "description": "LoadAkkudoktor settings", + "examples": [ + null + ] + }, + "LoadVrm": { + "anyOf": [ + { + "$ref": "#/components/schemas/LoadVrmCommonSettings" + }, + { + "type": "null" + } + ], + "description": "LoadVrm settings", + "examples": [ + null + ] + }, + "LoadImport": { + "anyOf": [ + { + "$ref": "#/components/schemas/LoadImportCommonSettings" + }, + { + "type": "null" + } + ], + "description": "LoadImport settings", + "examples": [ + null + ] + } + }, + "type": "object", + "title": "LoadCommonProviderSettings", + "description": "Load Prediction Provider Configuration." + }, + "LoadCommonSettings-Input": { + "properties": { + "provider": { + "anyOf": [ + { + "type": "string" + }, + { + "type": "null" + } + ], + "title": "Provider", + "description": "Load provider id of provider to be used.", + "examples": [ + "LoadAkkudoktor" + ] + }, + "provider_settings": { + "$ref": "#/components/schemas/LoadCommonProviderSettings", + "description": "Provider settings", + "examples": [ + {} + ] + } + }, + "type": "object", + "title": "LoadCommonSettings", + "description": "Load Prediction Configuration." + }, + "LoadCommonSettings-Output": { + "properties": { + "provider": { + "anyOf": [ + { + "type": "string" + }, + { + "type": "null" + } + ], + "title": "Provider", + "description": "Load provider id of provider to be used.", + "examples": [ + "LoadAkkudoktor" + ] + }, + "provider_settings": { + "$ref": "#/components/schemas/LoadCommonProviderSettings", + "description": "Provider settings", + "examples": [ + {} + ] + } + }, + "type": "object", + "title": "LoadCommonSettings", + "description": "Load Prediction Configuration." + }, + "LoadImportCommonSettings": { + "properties": { + "import_file_path": { + "anyOf": [ + { + "type": "string" + }, + { + "type": "string", + "format": "path" + }, + { + "type": "null" + } + ], + "title": "Import File Path", + "description": "Path to the file to import load data from.", + "examples": [ + null, + "/path/to/yearly_load.json" + ] + }, + "import_json": { + "anyOf": [ + { + "type": "string" + }, + { + "type": "null" + } + ], + "title": "Import Json", + "description": "JSON string, dictionary of load forecast value lists.", + "examples": [ + "{\"load0_mean\": [676.71, 876.19, 527.13]}" + ] + } + }, + "type": "object", + "title": "LoadImportCommonSettings", + "description": "Common settings for load data import from file or JSON string." + }, + "LoadVrmCommonSettings": { + "properties": { + "load_vrm_token": { + "type": "string", + "title": "Load Vrm Token", + "description": "Token for Connecting VRM API", + "default": "your-token", + "examples": [ + "your-token" + ] + }, + "load_vrm_idsite": { + "type": "integer", + "title": "Load Vrm Idsite", + "description": "VRM-Installation-ID", + "default": 12345, + "examples": [ + 12345 + ] + } + }, + "type": "object", + "title": "LoadVrmCommonSettings", + "description": "Common settings for VRM API." + }, + "LoggingCommonSettings-Input": { + "properties": { + "console_level": { + "anyOf": [ + { + "type": "string" + }, + { + "type": "null" + } + ], + "title": "Console Level", + "description": "Logging level when logging to console.", + "examples": [ + "TRACE", + "DEBUG", + "INFO", + "WARNING", + "ERROR", + "CRITICAL" + ] + }, + "file_level": { + "anyOf": [ + { + "type": "string" + }, + { + "type": "null" + } + ], + "title": "File Level", + "description": "Logging level when logging to file.", + "examples": [ + "TRACE", + "DEBUG", + "INFO", + "WARNING", + "ERROR", + "CRITICAL" + ] + } + }, + "type": "object", + "title": "LoggingCommonSettings", + "description": "Logging Configuration." + }, + "LoggingCommonSettings-Output": { + "properties": { + "console_level": { + "anyOf": [ + { + "type": "string" + }, + { + "type": "null" + } + ], + "title": "Console Level", + "description": "Logging level when logging to console.", + "examples": [ + "TRACE", + "DEBUG", + "INFO", + "WARNING", + "ERROR", + "CRITICAL" + ] + }, + "file_level": { + "anyOf": [ + { + "type": "string" + }, + { + "type": "null" + } + ], + "title": "File Level", + "description": "Logging level when logging to file.", + "examples": [ + "TRACE", + "DEBUG", + "INFO", + "WARNING", + "ERROR", + "CRITICAL" + ] + }, + "file_path": { + "anyOf": [ + { + "type": "string", + "format": "path" + }, + { + "type": "null" + } + ], + "title": "File Path", + "description": "Computed log file path based on data output path.", + "readOnly": true + } + }, + "type": "object", + "required": [ + "file_path" + ], + "title": "LoggingCommonSettings", + "description": "Logging Configuration." + }, + "MeasurementCommonSettings-Input": { + "properties": { + "load_emr_keys": { + "anyOf": [ + { + "items": { + "type": "string" + }, + "type": "array" + }, + { + "type": "null" + } + ], + "title": "Load Emr Keys", + "description": "The keys of the measurements that are energy meter readings of a load [kWh].", + "examples": [ + [ + "load0_emr" + ] + ] + }, + "grid_export_emr_keys": { + "anyOf": [ + { + "items": { + "type": "string" + }, + "type": "array" + }, + { + "type": "null" + } + ], + "title": "Grid Export Emr Keys", + "description": "The keys of the measurements that are energy meter readings of energy export to grid [kWh].", + "examples": [ + [ + "grid_export_emr" + ] + ] + }, + "grid_import_emr_keys": { + "anyOf": [ + { + "items": { + "type": "string" + }, + "type": "array" + }, + { + "type": "null" + } + ], + "title": "Grid Import Emr Keys", + "description": "The keys of the measurements that are energy meter readings of energy import from grid [kWh].", + "examples": [ + [ + "grid_import_emr" + ] + ] + }, + "pv_production_emr_keys": { + "anyOf": [ + { + "items": { + "type": "string" + }, + "type": "array" + }, + { + "type": "null" + } + ], + "title": "Pv Production Emr Keys", + "description": "The keys of the measurements that are PV production energy meter readings [kWh].", + "examples": [ + [ + "pv1_emr" + ] + ] + } + }, + "type": "object", + "title": "MeasurementCommonSettings", + "description": "Measurement Configuration." + }, + "MeasurementCommonSettings-Output": { + "properties": { + "load_emr_keys": { + "anyOf": [ + { + "items": { + "type": "string" + }, + "type": "array" + }, + { + "type": "null" + } + ], + "title": "Load Emr Keys", + "description": "The keys of the measurements that are energy meter readings of a load [kWh].", + "examples": [ + [ + "load0_emr" + ] + ] + }, + "grid_export_emr_keys": { + "anyOf": [ + { + "items": { + "type": "string" + }, + "type": "array" + }, + { + "type": "null" + } + ], + "title": "Grid Export Emr Keys", + "description": "The keys of the measurements that are energy meter readings of energy export to grid [kWh].", + "examples": [ + [ + "grid_export_emr" + ] + ] + }, + "grid_import_emr_keys": { + "anyOf": [ + { + "items": { + "type": "string" + }, + "type": "array" + }, + { + "type": "null" + } + ], + "title": "Grid Import Emr Keys", + "description": "The keys of the measurements that are energy meter readings of energy import from grid [kWh].", + "examples": [ + [ + "grid_import_emr" + ] + ] + }, + "pv_production_emr_keys": { + "anyOf": [ + { + "items": { + "type": "string" + }, + "type": "array" + }, + { + "type": "null" + } + ], + "title": "Pv Production Emr Keys", + "description": "The keys of the measurements that are PV production energy meter readings [kWh].", + "examples": [ + [ + "pv1_emr" + ] + ] + }, + "keys": { + "items": { + "type": "string" + }, + "type": "array", + "title": "Keys", + "description": "The keys of the measurements that can be stored.", + "readOnly": true + } + }, + "type": "object", + "required": [ + "keys" + ], + "title": "MeasurementCommonSettings", + "description": "Measurement Configuration." + }, + "OMBCInstruction": { + "properties": { + "id": { + "anyOf": [ + { + "type": "string" + }, + { + "type": "null" + } + ], + "title": "Id", + "description": "Unique identifier of the instruction in the ResourceManager scope. If not provided and a `resource_id` is passed at instantiation, this will be auto-generated as `{resource_id}@{UUID}`." + }, + "execution_time": { + "type": "string", + "format": "date-time", + "title": "Execution Time", + "description": "Start time of the instruction execution." + }, + "abnormal_condition": { + "type": "boolean", + "title": "Abnormal Condition", + "description": "Indicates if this is an instruction for abnormal conditions. Defaults to False.", + "default": false + }, + "type": { + "type": "string", + "const": "OMBCInstruction", + "title": "Type", + "default": "OMBCInstruction" + }, + "operation_mode_id": { + "type": "string", + "title": "Operation Mode Id", + "description": "ID of the OMBC.OperationMode to activate." + }, + "operation_mode_factor": { + "type": "number", + "maximum": 1.0, + "minimum": 0.0, + "title": "Operation Mode Factor", + "description": "Factor with which the operation mode is configured (0 to 1)." + }, + "resource_id": { + "type": "string", + "title": "Resource Id", + "description": "Get the resource identifier component from the instruction's `id`.\n\nAssumes the `id` follows the format `{resource_id}@{UUID}`. Extracts the resource_id part\nof the id by splitting at the last @.\n\nReturns:\n str: The resource identifier prefix of `id`, or an empty string if `id` is None.", + "readOnly": true + } + }, + "type": "object", + "required": [ + "execution_time", + "operation_mode_id", + "operation_mode_factor", + "resource_id" + ], + "title": "OMBCInstruction", + "description": "Instruction for Operation Mode Based Control (OMBC).\n\nContains information about when and how to activate a specific operation mode.\nUsed to command resources to change their operation at a specified time." + }, + "OMBCStatus": { + "properties": { + "type": { + "type": "string", + "const": "OMBCStatus", + "title": "Type", + "default": "OMBCStatus" + }, + "active_operation_mode_id": { + "type": "string", + "title": "Active Operation Mode Id", + "description": "ID of the currently active operation mode." + }, + "operation_mode_factor": { + "type": "number", + "maximum": 1.0, + "minimum": 0.0, + "title": "Operation Mode Factor", + "description": "Factor with which the operation mode is configured (between 0 and 1)." + }, + "previous_operation_mode_id": { + "anyOf": [ + { + "type": "string" + }, + { + "type": "null" + } + ], + "title": "Previous Operation Mode Id", + "description": "ID of the previously active operation mode, if known." + }, + "transition_timestamp": { + "anyOf": [ + { + "type": "string", + "format": "date-time" + }, + { + "type": "null" + } + ], + "title": "Transition Timestamp", + "description": "Timestamp of transition to the active operation mode, if applicable." + } + }, + "type": "object", + "required": [ + "active_operation_mode_id", + "operation_mode_factor" + ], + "title": "OMBCStatus", + "description": "Reports the current operational status of an Operation Mode Based Control system.\n\nThis model provides real-time status information about an OMBC-controlled device,\nincluding which operation mode is currently active, how it is configured,\nand information about recent mode transitions. It enables monitoring of the\ndevice's operational state and tracking mode transition history." + }, + "OptimizationCommonSettings": { + "properties": { + "horizon_hours": { + "anyOf": [ + { + "type": "integer", + "minimum": 0.0 + }, + { + "type": "null" + } + ], + "title": "Horizon Hours", + "description": "The general time window within which the energy optimization goal shall be achieved [h]. Defaults to 24 hours.", + "default": 24, + "examples": [ + 24 + ] + }, + "interval": { + "anyOf": [ + { + "type": "integer", + "maximum": 3600.0, + "minimum": 900.0 + }, + { + "type": "null" + } + ], + "title": "Interval", + "description": "The optimization interval [sec].", + "default": 3600, + "examples": [ + 3600, + 900 + ] + }, + "genetic": { + "anyOf": [ + { + "$ref": "#/components/schemas/GeneticCommonSettings" + }, + { + "type": "null" + } + ], + "description": "Genetic optimization algorithm configuration.", + "examples": [ + { + "individuals": 400, + "penalties": { + "ev_soc_miss": 10 + } + } + ] + } + }, + "type": "object", + "title": "OptimizationCommonSettings", + "description": "General Optimization Configuration." + }, + "OptimizationSolution": { + "properties": { + "id": { + "type": "string", + "title": "Id", + "description": "Unique ID for the optimization solution." + }, + "generated_at": { + "type": "string", + "format": "date-time", + "title": "Generated At", + "description": "Timestamp when the solution was generated." + }, + "comment": { + "anyOf": [ + { + "type": "string" + }, + { + "type": "null" + } + ], + "title": "Comment", + "description": "Optional comment or annotation for the solution." + }, + "valid_from": { + "anyOf": [ + { + "type": "string", + "format": "date-time" + }, + { + "type": "null" + } + ], + "title": "Valid From", + "description": "Start time of the optimization solution." + }, + "valid_until": { + "anyOf": [ + { + "type": "string", + "format": "date-time" + }, + { + "type": "null" + } + ], + "title": "Valid Until", + "description": "End time of the optimization solution." + }, + "total_losses_energy_wh": { + "type": "number", + "title": "Total Losses Energy Wh", + "description": "The total losses in watt-hours over the entire period." + }, + "total_revenues_amt": { + "type": "number", + "title": "Total Revenues Amt", + "description": "The total revenues [money amount]." + }, + "total_costs_amt": { + "type": "number", + "title": "Total Costs Amt", + "description": "The total costs [money amount]." + }, + "data": { + "$ref": "#/components/schemas/PydanticDateTimeDataFrame", + "description": "Datetime data frame with time series optimization data per optimization interval:- load_energy_wh: Load of all energy consumers in wh- grid_energy_wh: Grid energy feed in (negative) or consumption (positive) in wh- pv_prediction_energy_wh: PV energy prediction (positive) in wh- elec_price_prediction_amt_kwh: Electricity price prediction in money per kwh- costs_amt: Costs in money amount- revenue_amt: Revenue in money amount- losses_energy_wh: Energy losses in wh- _operation_mode_id: Operation mode id of the device.- _operation_mode_factor: Operation mode factor of the device.- _soc_factor: State of charge of a battery/ electric vehicle device as factor of total capacity.- _energy_wh: Energy consumption (positive) of a device in wh." + } + }, + "type": "object", + "required": [ + "id", + "generated_at", + "total_losses_energy_wh", + "total_revenues_amt", + "total_costs_amt", + "data" + ], + "title": "OptimizationSolution", + "description": "General Optimization Solution." + }, + "PEBCInstruction": { + "properties": { + "id": { + "anyOf": [ + { + "type": "string" + }, + { + "type": "null" + } + ], + "title": "Id", + "description": "Unique identifier of the instruction in the ResourceManager scope. If not provided and a `resource_id` is passed at instantiation, this will be auto-generated as `{resource_id}@{UUID}`." + }, + "execution_time": { + "type": "string", + "format": "date-time", + "title": "Execution Time", + "description": "Start time of the instruction execution." + }, + "abnormal_condition": { + "type": "boolean", + "title": "Abnormal Condition", + "description": "Indicates if this is an instruction for abnormal conditions. Defaults to False.", + "default": false + }, + "type": { + "type": "string", + "const": "PEBCInstruction", + "title": "Type", + "default": "PEBCInstruction" + }, + "power_constraints_id": { + "type": "string", + "title": "Power Constraints Id", + "description": "ID of the associated PEBC.PowerConstraints." + }, + "power_envelopes": { + "items": { + "$ref": "#/components/schemas/PEBCPowerEnvelope" + }, + "type": "array", + "minItems": 1, + "title": "Power Envelopes", + "description": "List of PowerEnvelopes to follow. One per CommodityQuantity, max one per type." + }, + "resource_id": { + "type": "string", + "title": "Resource Id", + "description": "Get the resource identifier component from the instruction's `id`.\n\nAssumes the `id` follows the format `{resource_id}@{UUID}`. Extracts the resource_id part\nof the id by splitting at the last @.\n\nReturns:\n str: The resource identifier prefix of `id`, or an empty string if `id` is None.", + "readOnly": true + } + }, + "type": "object", + "required": [ + "execution_time", + "power_constraints_id", + "power_envelopes", + "resource_id" + ], + "title": "PEBCInstruction", + "description": "Represents a control instruction for Power Envelope Based Control.\n\nThis model defines a complete instruction for controlling a device using power\nenvelopes. It specifies when the instruction should be executed, which power\nconstraints apply, and the specific power envelopes to follow. It supports\nmultiple power envelopes for different commodity quantities." + }, + "PEBCPowerEnvelope": { + "properties": { + "id": { + "type": "string", + "title": "Id", + "description": "Unique identifier of this PEBC.PowerEnvelope, scoped to the ResourceManager." + }, + "commodity_quantity": { + "$ref": "#/components/schemas/CommodityQuantity", + "description": "Type of power quantity the envelope applies to." + }, + "power_envelope_elements": { + "items": { + "$ref": "#/components/schemas/PEBCPowerEnvelopeElement" + }, + "type": "array", + "minItems": 1, + "title": "Power Envelope Elements", + "description": "Chronologically ordered list of PowerEnvelopeElements. Defines how power should be constrained over time." + } + }, + "type": "object", + "required": [ + "id", + "commodity_quantity", + "power_envelope_elements" + ], + "title": "PEBCPowerEnvelope", + "description": "Defines a complete power envelope constraint for a specific commodity quantity.\n\nThis model specifies a time-series of power limits (upper and lower bounds) that\na device must operate within. The power envelope consists of sequential elements,\neach defining constraints for a specific duration, creating a complete time-varying\noperational boundary for the device." + }, + "PEBCPowerEnvelopeElement": { + "properties": { + "duration": { + "type": "string", + "format": "duration", + "title": "Duration", + "description": "Duration of this power envelope element." + }, + "upper_limit": { + "type": "number", + "title": "Upper Limit", + "description": "Upper power limit for the given commodity_quantity. Shall match PEBC.AllowedLimitRange with limit_type UPPER_LIMIT." + }, + "lower_limit": { + "type": "number", + "title": "Lower Limit", + "description": "Lower power limit for the given commodity_quantity. Shall match PEBC.AllowedLimitRange with limit_type LOWER_LIMIT." + } + }, + "type": "object", + "required": [ + "duration", + "upper_limit", + "lower_limit" + ], + "title": "PEBCPowerEnvelopeElement", + "description": "Defines a segment of a power envelope for a specific duration.\n\nThis model specifies the upper and lower power limits for a specific time duration,\nforming part of a complete power envelope. A sequence of these elements creates\na time-varying power envelope that constrains device power consumption or production." + }, + "PPBCEndInterruptionInstruction": { + "properties": { + "id": { + "anyOf": [ + { + "type": "string" + }, + { + "type": "null" + } + ], + "title": "Id", + "description": "Unique identifier of the instruction in the ResourceManager scope. If not provided and a `resource_id` is passed at instantiation, this will be auto-generated as `{resource_id}@{UUID}`." + }, + "execution_time": { + "type": "string", + "format": "date-time", + "title": "Execution Time", + "description": "Start time of the instruction execution." + }, + "abnormal_condition": { + "type": "boolean", + "title": "Abnormal Condition", + "description": "Indicates if this is an instruction for abnormal conditions. Defaults to False.", + "default": false + }, + "type": { + "type": "string", + "const": "PPBCEndInterruptionInstruction", + "title": "Type", + "default": "PPBCEndInterruptionInstruction" + }, + "power_profile_id": { + "type": "string", + "title": "Power Profile Id", + "description": "ID of the PowerProfileDefinition related to the ended interruption." + }, + "sequence_container_id": { + "type": "string", + "title": "Sequence Container Id", + "description": "ID of the container containing the sequence." + }, + "power_sequence_id": { + "type": "string", + "title": "Power Sequence Id", + "description": "ID of the PowerSequence for which the interruption ends." + }, + "resource_id": { + "type": "string", + "title": "Resource Id", + "description": "Get the resource identifier component from the instruction's `id`.\n\nAssumes the `id` follows the format `{resource_id}@{UUID}`. Extracts the resource_id part\nof the id by splitting at the last @.\n\nReturns:\n str: The resource identifier prefix of `id`, or an empty string if `id` is None.", + "readOnly": true + } + }, + "type": "object", + "required": [ + "execution_time", + "power_profile_id", + "sequence_container_id", + "power_sequence_id", + "resource_id" + ], + "title": "PPBCEndInterruptionInstruction", + "description": "Represents an instruction to resume execution of a previously interrupted power sequence.\n\nThis model defines a control instruction that ends an interruption and resumes\nexecution of a previously interrupted power sequence. It complements the start\ninterruption instruction, enabling the complete interruption-resumption cycle\nfor flexible sequence execution control." + }, + "PPBCPowerProfileStatus-Input": { + "properties": { + "type": { + "type": "string", + "const": "PPBCPowerProfileStatus", + "title": "Type", + "default": "PPBCPowerProfileStatus" + }, + "sequence_container_status": { + "items": { + "$ref": "#/components/schemas/PPBCPowerSequenceContainerStatus" + }, + "type": "array", + "title": "Sequence Container Status", + "description": "Status list for all sequence containers in the PowerProfileDefinition." + } + }, + "type": "object", + "required": [ + "sequence_container_status" + ], + "title": "PPBCPowerProfileStatus", + "description": "Reports the current status of a power profile execution.\n\nThis model provides comprehensive status information for all sequence containers\nin a power profile definition, enabling monitoring of profile execution progress.\nIt tracks which sequences have been selected and their current execution status." + }, + "PPBCPowerProfileStatus-Output": { + "properties": { + "type": { + "type": "string", + "const": "PPBCPowerProfileStatus", + "title": "Type", + "default": "PPBCPowerProfileStatus" + }, + "sequence_container_status": { + "items": { + "$ref": "#/components/schemas/PPBCPowerSequenceContainerStatus" + }, + "type": "array", + "title": "Sequence Container Status", + "description": "Status list for all sequence containers in the PowerProfileDefinition." + } + }, + "type": "object", + "required": [ + "sequence_container_status" + ], + "title": "PPBCPowerProfileStatus", + "description": "Reports the current status of a power profile execution.\n\nThis model provides comprehensive status information for all sequence containers\nin a power profile definition, enabling monitoring of profile execution progress.\nIt tracks which sequences have been selected and their current execution status." + }, + "PPBCPowerSequenceContainerStatus": { + "properties": { + "power_profile_id": { + "type": "string", + "title": "Power Profile Id", + "description": "ID of the related PowerProfileDefinition." + }, + "sequence_container_id": { + "type": "string", + "title": "Sequence Container Id", + "description": "ID of the PowerSequenceContainer being reported on." + }, + "selected_sequence_id": { + "anyOf": [ + { + "type": "string" + }, + { + "type": "null" + } + ], + "title": "Selected Sequence Id", + "description": "ID of the selected PowerSequence, if any." + }, + "progress": { + "anyOf": [ + { + "type": "string", + "format": "duration" + }, + { + "type": "null" + } + ], + "title": "Progress", + "description": "Elapsed time since the selected sequence started, if applicable." + }, + "status": { + "$ref": "#/components/schemas/PPBCPowerSequenceStatus", + "description": "Status of the selected PowerSequence." + } + }, + "type": "object", + "required": [ + "power_profile_id", + "sequence_container_id", + "status" + ], + "title": "PPBCPowerSequenceContainerStatus", + "description": "Reports the status of a specific power sequence container execution.\n\nThis model provides detailed status information for a single sequence container,\nincluding which sequence was selected, the current execution progress, and the\noperational status. It enables fine-grained monitoring of sequence execution\nwithin the broader power profile." + }, + "PPBCPowerSequenceStatus": { + "type": "string", + "enum": [ + "NOT_SCHEDULED", + "SCHEDULED", + "EXECUTING", + "INTERRUPTED", + "FINISHED", + "ABORTED" + ], + "title": "PPBCPowerSequenceStatus", + "description": "Enumeration of status values for Power Profile Based Control sequences." + }, + "PPBCScheduleInstruction": { + "properties": { + "id": { + "anyOf": [ + { + "type": "string" + }, + { + "type": "null" + } + ], + "title": "Id", + "description": "Unique identifier of the instruction in the ResourceManager scope. If not provided and a `resource_id` is passed at instantiation, this will be auto-generated as `{resource_id}@{UUID}`." + }, + "execution_time": { + "type": "string", + "format": "date-time", + "title": "Execution Time", + "description": "Start time of the instruction execution." + }, + "abnormal_condition": { + "type": "boolean", + "title": "Abnormal Condition", + "description": "Indicates if this is an instruction for abnormal conditions. Defaults to False.", + "default": false + }, + "type": { + "type": "string", + "const": "PPBCScheduleInstruction", + "title": "Type", + "default": "PPBCScheduleInstruction" + }, + "power_profile_id": { + "type": "string", + "title": "Power Profile Id", + "description": "ID of the PowerProfileDefinition being scheduled." + }, + "sequence_container_id": { + "type": "string", + "title": "Sequence Container Id", + "description": "ID of the container with the selected sequence." + }, + "power_sequence_id": { + "type": "string", + "title": "Power Sequence Id", + "description": "ID of the selected PowerSequence." + }, + "resource_id": { + "type": "string", + "title": "Resource Id", + "description": "Get the resource identifier component from the instruction's `id`.\n\nAssumes the `id` follows the format `{resource_id}@{UUID}`. Extracts the resource_id part\nof the id by splitting at the last @.\n\nReturns:\n str: The resource identifier prefix of `id`, or an empty string if `id` is None.", + "readOnly": true + } + }, + "type": "object", + "required": [ + "execution_time", + "power_profile_id", + "sequence_container_id", + "power_sequence_id", + "resource_id" + ], + "title": "PPBCScheduleInstruction", + "description": "Represents an instruction to schedule execution of a specific power sequence.\n\nThis model defines a control instruction that schedules the execution of a\nselected power sequence from a power profile. It specifies which sequence\nhas been selected and when it should begin execution, enabling precise control\nof device power behavior according to the predefined sequence." + }, + "PPBCStartInterruptionInstruction": { + "properties": { + "id": { + "anyOf": [ + { + "type": "string" + }, + { + "type": "null" + } + ], + "title": "Id", + "description": "Unique identifier of the instruction in the ResourceManager scope. If not provided and a `resource_id` is passed at instantiation, this will be auto-generated as `{resource_id}@{UUID}`." + }, + "execution_time": { + "type": "string", + "format": "date-time", + "title": "Execution Time", + "description": "Start time of the instruction execution." + }, + "abnormal_condition": { + "type": "boolean", + "title": "Abnormal Condition", + "description": "Indicates if this is an instruction for abnormal conditions. Defaults to False.", + "default": false + }, + "type": { + "type": "string", + "const": "PPBCStartInterruptionInstruction", + "title": "Type", + "default": "PPBCStartInterruptionInstruction" + }, + "power_profile_id": { + "type": "string", + "title": "Power Profile Id", + "description": "ID of the PowerProfileDefinition whose sequence is being interrupted." + }, + "sequence_container_id": { + "type": "string", + "title": "Sequence Container Id", + "description": "ID of the container containing the sequence." + }, + "power_sequence_id": { + "type": "string", + "title": "Power Sequence Id", + "description": "ID of the PowerSequence to be interrupted." + }, + "resource_id": { + "type": "string", + "title": "Resource Id", + "description": "Get the resource identifier component from the instruction's `id`.\n\nAssumes the `id` follows the format `{resource_id}@{UUID}`. Extracts the resource_id part\nof the id by splitting at the last @.\n\nReturns:\n str: The resource identifier prefix of `id`, or an empty string if `id` is None.", + "readOnly": true + } + }, + "type": "object", + "required": [ + "execution_time", + "power_profile_id", + "sequence_container_id", + "power_sequence_id", + "resource_id" + ], + "title": "PPBCStartInterruptionInstruction", + "description": "Represents an instruction to interrupt execution of a running power sequence.\n\nThis model defines a control instruction that interrupts the execution of an\nactive power sequence. It enables dynamic control over sequence execution,\nallowing temporary suspension of a sequence in response to changing system conditions\nor requirements, particularly for sequences marked as interruptible." + }, + "PVForecastCommonProviderSettings": { + "properties": { + "PVForecastImport": { + "anyOf": [ + { + "$ref": "#/components/schemas/PVForecastImportCommonSettings" + }, + { + "type": "null" + } + ], + "description": "PVForecastImport settings", + "examples": [ + null + ] + }, + "PVForecastVrm": { + "anyOf": [ + { + "$ref": "#/components/schemas/PVForecastVrmCommonSettings" + }, + { + "type": "null" + } + ], + "description": "PVForecastVrm settings", + "examples": [ + null + ] + } + }, + "type": "object", + "title": "PVForecastCommonProviderSettings", + "description": "PV Forecast Provider Configuration." + }, "PVForecastCommonSettings-Input": { "properties": { "provider": { @@ -3515,21 +6301,10 @@ ] }, "provider_settings": { - "anyOf": [ - { - "$ref": "#/components/schemas/PVForecastImportCommonSettings" - }, - { - "$ref": "#/components/schemas/PVforecastVrmCommonSettings" - }, - { - "type": "null" - } - ], - "title": "Provider Settings", + "$ref": "#/components/schemas/PVForecastCommonProviderSettings", "description": "Provider settings", "examples": [ - null + {} ] }, "planes": { @@ -3601,7 +6376,11 @@ ], "title": "Max Planes", "description": "Maximum number of planes that can be set", - "default": 0 + "default": 0, + "examples": [ + 1, + 2 + ] } }, "type": "object", @@ -3626,21 +6405,10 @@ ] }, "provider_settings": { - "anyOf": [ - { - "$ref": "#/components/schemas/PVForecastImportCommonSettings" - }, - { - "$ref": "#/components/schemas/PVforecastVrmCommonSettings" - }, - { - "type": "null" - } - ], - "title": "Provider Settings", + "$ref": "#/components/schemas/PVForecastCommonProviderSettings", "description": "Provider settings", "examples": [ - null + {} ] }, "planes": { @@ -3712,7 +6480,11 @@ ], "title": "Max Planes", "description": "Maximum number of planes that can be set", - "default": 0 + "default": 0, + "examples": [ + 1, + 2 + ] }, "planes_peakpower": { "items": { @@ -4077,7 +6849,7 @@ "title": "PVForecastPlaneSetting", "description": "PV Forecast Plane Configuration." }, - "PVforecastVrmCommonSettings": { + "PVForecastVrmCommonSettings": { "properties": { "pvforecast_vrm_token": { "type": "string", @@ -4099,9 +6871,91 @@ } }, "type": "object", - "title": "PVforecastVrmCommonSettings", + "title": "PVForecastVrmCommonSettings", "description": "Common settings for VRM API." }, + "PowerMeasurement-Input": { + "properties": { + "type": { + "type": "string", + "const": "PowerMeasurement", + "title": "Type", + "default": "PowerMeasurement" + }, + "measurement_timestamp": { + "type": "string", + "format": "date-time", + "title": "Measurement Timestamp", + "description": "Timestamp when PowerValues were measured." + }, + "values": { + "items": { + "$ref": "#/components/schemas/PowerValue" + }, + "type": "array", + "title": "Values", + "description": "Array of measured PowerValues. Shall contain at least one item and at most one item per 'commodity_quantity' (defined inside the PowerValue)." + } + }, + "type": "object", + "required": [ + "measurement_timestamp", + "values" + ], + "title": "PowerMeasurement", + "description": "Captures a set of power measurements taken at a specific point in time.\n\nThis model records multiple power values (for different commodity quantities)\nalong with the timestamp when the measurements were taken, enabling time-series\nanalysis and monitoring of power consumption or production." + }, + "PowerMeasurement-Output": { + "properties": { + "type": { + "type": "string", + "const": "PowerMeasurement", + "title": "Type", + "default": "PowerMeasurement" + }, + "measurement_timestamp": { + "type": "string", + "format": "date-time", + "title": "Measurement Timestamp", + "description": "Timestamp when PowerValues were measured." + }, + "values": { + "items": { + "$ref": "#/components/schemas/PowerValue" + }, + "type": "array", + "title": "Values", + "description": "Array of measured PowerValues. Shall contain at least one item and at most one item per 'commodity_quantity' (defined inside the PowerValue)." + } + }, + "type": "object", + "required": [ + "measurement_timestamp", + "values" + ], + "title": "PowerMeasurement", + "description": "Captures a set of power measurements taken at a specific point in time.\n\nThis model records multiple power values (for different commodity quantities)\nalong with the timestamp when the measurements were taken, enabling time-series\nanalysis and monitoring of power consumption or production." + }, + "PowerValue": { + "properties": { + "commodity_quantity": { + "$ref": "#/components/schemas/CommodityQuantity", + "description": "The power quantity the value refers to." + }, + "value": { + "type": "number", + "title": "Value", + "description": "Power value expressed in the unit associated with the CommodityQuantity." + } + }, + "type": "object", + "required": [ + "commodity_quantity", + "value" + ], + "title": "PowerValue", + "description": "Represents a specific power value measurement with its associated commodity quantity.\n\nThis class links a numerical power value to a specific type of power quantity (such as\nactive power, reactive power, etc.) and its unit of measurement." + }, "PredictionCommonSettings": { "properties": { "hours": { @@ -4249,16 +7103,19 @@ "host": { "anyOf": [ { - "type": "string", - "format": "ipvanyaddress" + "type": "string" }, { "type": "null" } ], "title": "Host", - "description": "EOS server IP address.", - "default": "127.0.0.1" + "description": "EOS server IP address. Defaults to 127.0.0.1.", + "default": "127.0.0.1", + "examples": [ + "127.0.0.1", + "localhost" + ] }, "port": { "anyOf": [ @@ -4270,8 +7127,11 @@ } ], "title": "Port", - "description": "EOS server IP port number.", - "default": 8503 + "description": "EOS server IP port number. Defaults to 8503.", + "default": 8503, + "examples": [ + 8503 + ] }, "verbose": { "anyOf": [ @@ -4296,22 +7156,24 @@ } ], "title": "Startup Eosdash", - "description": "EOS server to start EOSdash server.", + "description": "EOS server to start EOSdash server. Defaults to True.", "default": true }, "eosdash_host": { "anyOf": [ { - "type": "string", - "format": "ipvanyaddress" + "type": "string" }, { "type": "null" } ], "title": "Eosdash Host", - "description": "EOSdash server IP address.", - "default": "127.0.0.1" + "description": "EOSdash server IP address. Defaults to EOS server IP address.", + "examples": [ + "127.0.0.1", + "localhost" + ] }, "eosdash_port": { "anyOf": [ @@ -4323,8 +7185,10 @@ } ], "title": "Eosdash Port", - "description": "EOSdash server IP port number.", - "default": 8504 + "description": "EOSdash server IP port number. Defaults to EOS server IP port number + 1.", + "examples": [ + 8504 + ] } }, "type": "object", @@ -4380,7 +7244,7 @@ "devices": { "anyOf": [ { - "$ref": "#/components/schemas/DevicesCommonSettings" + "$ref": "#/components/schemas/DevicesCommonSettings-Input" }, { "type": "null" @@ -4391,7 +7255,7 @@ "measurement": { "anyOf": [ { - "$ref": "#/components/schemas/MeasurementCommonSettings" + "$ref": "#/components/schemas/MeasurementCommonSettings-Input" }, { "type": "null" @@ -4424,7 +7288,7 @@ "elecprice": { "anyOf": [ { - "$ref": "#/components/schemas/ElecPriceCommonSettings" + "$ref": "#/components/schemas/ElecPriceCommonSettings-Input" }, { "type": "null" @@ -4432,10 +7296,21 @@ ], "description": "Electricity Price Settings" }, + "feedintariff": { + "anyOf": [ + { + "$ref": "#/components/schemas/FeedInTariffCommonSettings-Input" + }, + { + "type": "null" + } + ], + "description": "Feed In Tariff Settings" + }, "load": { "anyOf": [ { - "$ref": "#/components/schemas/LoadCommonSettings" + "$ref": "#/components/schemas/LoadCommonSettings-Input" }, { "type": "null" @@ -4457,7 +7332,7 @@ "weather": { "anyOf": [ { - "$ref": "#/components/schemas/WeatherCommonSettings" + "$ref": "#/components/schemas/WeatherCommonSettings-Input" }, { "type": "null" @@ -4491,201 +7366,7 @@ "additionalProperties": false, "type": "object", "title": "SettingsEOS", - "description": "Settings for all EOS.\n\nUsed by updating the configuration with specific settings only." - }, - "SimulationResult": { - "properties": { - "Last_Wh_pro_Stunde": { - "items": { - "anyOf": [ - { - "type": "number" - }, - { - "type": "null" - } - ] - }, - "type": "array", - "title": "Last Wh Pro Stunde", - "description": "TBD" - }, - "EAuto_SoC_pro_Stunde": { - "items": { - "anyOf": [ - { - "type": "number" - }, - { - "type": "null" - } - ] - }, - "type": "array", - "title": "Eauto Soc Pro Stunde", - "description": "The state of charge of the EV for each hour." - }, - "Einnahmen_Euro_pro_Stunde": { - "items": { - "anyOf": [ - { - "type": "number" - }, - { - "type": "null" - } - ] - }, - "type": "array", - "title": "Einnahmen Euro Pro Stunde", - "description": "The revenue from grid feed-in or other sources in euros per hour." - }, - "Gesamt_Verluste": { - "type": "number", - "title": "Gesamt Verluste", - "description": "The total losses in watt-hours over the entire period." - }, - "Gesamtbilanz_Euro": { - "type": "number", - "title": "Gesamtbilanz Euro", - "description": "The total balance of revenues minus costs in euros." - }, - "Gesamteinnahmen_Euro": { - "type": "number", - "title": "Gesamteinnahmen Euro", - "description": "The total revenues in euros." - }, - "Gesamtkosten_Euro": { - "type": "number", - "title": "Gesamtkosten Euro", - "description": "The total costs in euros." - }, - "Home_appliance_wh_per_hour": { - "items": { - "anyOf": [ - { - "type": "number" - }, - { - "type": "null" - } - ] - }, - "type": "array", - "title": "Home Appliance Wh Per Hour", - "description": "The energy consumption of a household appliance in watt-hours per hour." - }, - "Kosten_Euro_pro_Stunde": { - "items": { - "anyOf": [ - { - "type": "number" - }, - { - "type": "null" - } - ] - }, - "type": "array", - "title": "Kosten Euro Pro Stunde", - "description": "The costs in euros per hour." - }, - "Netzbezug_Wh_pro_Stunde": { - "items": { - "anyOf": [ - { - "type": "number" - }, - { - "type": "null" - } - ] - }, - "type": "array", - "title": "Netzbezug Wh Pro Stunde", - "description": "The grid energy drawn in watt-hours per hour." - }, - "Netzeinspeisung_Wh_pro_Stunde": { - "items": { - "anyOf": [ - { - "type": "number" - }, - { - "type": "null" - } - ] - }, - "type": "array", - "title": "Netzeinspeisung Wh Pro Stunde", - "description": "The energy fed into the grid in watt-hours per hour." - }, - "Verluste_Pro_Stunde": { - "items": { - "anyOf": [ - { - "type": "number" - }, - { - "type": "null" - } - ] - }, - "type": "array", - "title": "Verluste Pro Stunde", - "description": "The losses in watt-hours per hour." - }, - "akku_soc_pro_stunde": { - "items": { - "anyOf": [ - { - "type": "number" - }, - { - "type": "null" - } - ] - }, - "type": "array", - "title": "Akku Soc Pro Stunde", - "description": "The state of charge of the battery (not the EV) in percentage per hour." - }, - "Electricity_price": { - "items": { - "anyOf": [ - { - "type": "number" - }, - { - "type": "null" - } - ] - }, - "type": "array", - "title": "Electricity Price", - "description": "Used Electricity Price, including predictions" - } - }, - "additionalProperties": false, - "type": "object", - "required": [ - "Last_Wh_pro_Stunde", - "EAuto_SoC_pro_Stunde", - "Einnahmen_Euro_pro_Stunde", - "Gesamt_Verluste", - "Gesamtbilanz_Euro", - "Gesamteinnahmen_Euro", - "Gesamtkosten_Euro", - "Home_appliance_wh_per_hour", - "Kosten_Euro_pro_Stunde", - "Netzbezug_Wh_pro_Stunde", - "Netzeinspeisung_Wh_pro_Stunde", - "Verluste_Pro_Stunde", - "akku_soc_pro_stunde", - "Electricity_price" - ], - "title": "SimulationResult", - "description": "This object contains the results of the simulation and provides insights into various parameters over the entire forecast period." + "description": "Settings for all EOS.\n\nOnly used to update the configuration with specific settings." }, "SolarPanelBatteryParameters": { "properties": { @@ -4789,7 +7470,174 @@ "device_id", "capacity_wh" ], - "title": "SolarPanelBatteryParameters" + "title": "SolarPanelBatteryParameters", + "description": "PV battery device simulation configuration." + }, + "TimeWindow-Input": { + "properties": { + "start_time": { + "title": "Start Time", + "description": "Start time of the time window (time of day)." + }, + "duration": { + "type": "string", + "format": "duration", + "title": "Duration", + "description": "Duration of the time window starting from `start_time`." + }, + "day_of_week": { + "anyOf": [ + { + "type": "integer" + }, + { + "type": "string" + }, + { + "type": "null" + } + ], + "title": "Day Of Week", + "description": "Optional day of the week restriction. Can be specified as integer (0=Monday to 6=Sunday) or localized weekday name. If None, applies every day unless `date` is set." + }, + "date": { + "anyOf": [ + { + "type": "string", + "format": "date" + }, + { + "type": "null" + } + ], + "title": "Date", + "description": "Optional specific calendar date for the time window. Overrides `day_of_week` if set." + }, + "locale": { + "anyOf": [ + { + "type": "string" + }, + { + "type": "null" + } + ], + "title": "Locale", + "description": "Locale used to parse weekday names in `day_of_week` when given as string. If not set, Pendulum's default locale is used. Examples: 'en', 'de', 'fr', etc." + } + }, + "type": "object", + "required": [ + "start_time", + "duration" + ], + "title": "TimeWindow", + "description": "Model defining a daily or specific date time window with optional localization support.\n\nRepresents a time interval starting at `start_time` and lasting for `duration`.\nCan restrict applicability to a specific day of the week or a specific calendar date.\nSupports day names in multiple languages via locale-aware parsing." + }, + "TimeWindow-Output": { + "properties": { + "start_time": { + "type": "string", + "title": "Start Time", + "description": "Start time of the time window (time of day)." + }, + "duration": { + "type": "string", + "title": "Duration", + "description": "Duration of the time window starting from `start_time`." + }, + "day_of_week": { + "anyOf": [ + { + "type": "integer" + }, + { + "type": "string" + }, + { + "type": "null" + } + ], + "title": "Day Of Week", + "description": "Optional day of the week restriction. Can be specified as integer (0=Monday to 6=Sunday) or localized weekday name. If None, applies every day unless `date` is set." + }, + "date": { + "anyOf": [ + { + "type": "string", + "format": "date" + }, + { + "type": "null" + } + ], + "title": "Date", + "description": "Optional specific calendar date for the time window. Overrides `day_of_week` if set." + }, + "locale": { + "anyOf": [ + { + "type": "string" + }, + { + "type": "null" + } + ], + "title": "Locale", + "description": "Locale used to parse weekday names in `day_of_week` when given as string. If not set, Pendulum's default locale is used. Examples: 'en', 'de', 'fr', etc." + } + }, + "type": "object", + "required": [ + "start_time", + "duration" + ], + "title": "TimeWindow", + "description": "Model defining a daily or specific date time window with optional localization support.\n\nRepresents a time interval starting at `start_time` and lasting for `duration`.\nCan restrict applicability to a specific day of the week or a specific calendar date.\nSupports day names in multiple languages via locale-aware parsing." + }, + "TimeWindowSequence-Input": { + "properties": { + "windows": { + "anyOf": [ + { + "items": { + "$ref": "#/components/schemas/TimeWindow-Input" + }, + "type": "array" + }, + { + "type": "null" + } + ], + "title": "Windows", + "description": "List of TimeWindow objects that make up this sequence." + } + }, + "type": "object", + "title": "TimeWindowSequence", + "description": "Model representing a sequence of time windows with collective operations.\n\nManages multiple TimeWindow objects and provides methods to work with them\nas a cohesive unit for scheduling and availability checking." + }, + "TimeWindowSequence-Output": { + "properties": { + "windows": { + "anyOf": [ + { + "items": { + "$ref": "#/components/schemas/TimeWindow-Output" + }, + "type": "array" + }, + { + "type": "null" + } + ], + "title": "Windows", + "description": "List of TimeWindow objects that make up this sequence." + } + }, + "type": "object", + "title": "TimeWindowSequence", + "description": "Model representing a sequence of time windows with collective operations.\n\nManages multiple TimeWindow objects and provides methods to work with them\nas a cohesive unit for scheduling and availability checking." }, "UtilsCommonSettings": { "properties": {}, @@ -4830,7 +7678,28 @@ ], "title": "ValidationError" }, - "WeatherCommonSettings": { + "WeatherCommonProviderSettings": { + "properties": { + "WeatherImport": { + "anyOf": [ + { + "$ref": "#/components/schemas/WeatherImportCommonSettings" + }, + { + "type": "null" + } + ], + "description": "WeatherImport settings", + "examples": [ + null + ] + } + }, + "type": "object", + "title": "WeatherCommonProviderSettings", + "description": "Weather Forecast Provider Configuration." + }, + "WeatherCommonSettings-Input": { "properties": { "provider": { "anyOf": [ @@ -4848,17 +7717,39 @@ ] }, "provider_settings": { + "$ref": "#/components/schemas/WeatherCommonProviderSettings", + "description": "Provider settings", + "examples": [ + {} + ] + } + }, + "type": "object", + "title": "WeatherCommonSettings", + "description": "Weather Forecast Configuration." + }, + "WeatherCommonSettings-Output": { + "properties": { + "provider": { "anyOf": [ { - "$ref": "#/components/schemas/WeatherImportCommonSettings" + "type": "string" }, { "type": "null" } ], + "title": "Provider", + "description": "Weather provider id of provider to be used.", + "examples": [ + "WeatherImport" + ] + }, + "provider_settings": { + "$ref": "#/components/schemas/WeatherCommonProviderSettings", "description": "Provider settings", "examples": [ - null + {} ] } }, diff --git a/pyproject.toml b/pyproject.toml index da426e9..1575cb1 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -107,3 +107,31 @@ ignore_missing_imports = true [[tool.mypy.overrides]] module = "xprocess.*" ignore_missing_imports = true + +[tool.commitizen] +name = "cz_conventional_commits" +version_scheme = "semver" +version = "0.1.0+dev" # <-- Set your current version heretag_format = "v$version" + +# Files to automatically update when bumping version +update_changelog_on_bump = true +changelog_incremental = true +annotated_tag = true +bump_message = "chore(release): $current_version → $new_version" + +# Branch validation settings +branch_validation = true +branch_pattern = "^(feat|fix|chore|docs|refactor|test)/[a-z0-9._-]+$" + +# Customize changelog generation +[tool.commitizen.changelog] +path = "CHANGELOG.md" +template = "keepachangelog" + +# If your version is stored in multiple files (Python modules, docs etc.), add them here +[tool.commitizen.files] +version = [ + "pyproject.toml", # Auto-update project version + "src/akkudoktoreos/core/version.py", + "src/data/default.config.json" +] diff --git a/requirements-dev.txt b/requirements-dev.txt index 46c2c0b..9a9f616 100644 --- a/requirements-dev.txt +++ b/requirements-dev.txt @@ -1,15 +1,29 @@ -r requirements.txt -gitlint==0.19.1 -GitPython==3.1.45 -myst-parser==4.0.1 + +# Pre-commit framework - basic package requirements handled by pre-commit itself +# - pre-commit-hooks +# - isort +# - ruff +# - mypy (mirrors-mypy) - sync with requirements-dev.txt (if on pypi) +# - pymarkdown +# - commitizen - sync with requirements-dev.txt (if on pypi) +pre-commit==4.3.0 +mypy==1.18.2 +types-requests==2.32.4.20250913 # for mypy +pandas-stubs==2.3.2.250926 # for mypy +tokenize-rt==3.2.0 # for mypy +commitizen==4.9.1 +deprecated==1.2.18 # for commitizen + +# Sphinx sphinx==8.2.3 sphinx_rtd_theme==3.0.2 sphinx-tabs==3.4.7 -pymarkdownlnt==0.9.32 +GitPython==3.1.45 +myst-parser==4.0.1 + +# Pytest pytest==8.4.2 pytest-cov==7.0.0 +coverage==7.10.7 pytest-xprocess==1.0.2 -pre-commit -mypy==1.18.2 -types-requests==2.32.4.20250913 -pandas-stubs==2.3.2.250827 diff --git a/requirements.txt b/requirements.txt index 91e3ad8..1cdc50c 100644 --- a/requirements.txt +++ b/requirements.txt @@ -1,16 +1,21 @@ +babel==2.17.0 +beautifulsoup4==4.14.2 cachebox==5.0.2 numpy==2.3.3 numpydantic==1.6.11 matplotlib==3.10.6 -fastapi[standard]==0.115.14 +contourpy==1.3.3 +fastapi[standard-no-fastapi-cloud-cli]==0.117.1 +fastapi_cli==0.0.13 +rich-toolkit==0.15.1 python-fasthtml==0.12.29 MonsterUI==1.0.29 markdown-it-py==3.0.0 mdit-py-plugins==0.5.0 bokeh==3.8.0 -uvicorn==0.36.0 +uvicorn==0.37.0 scikit-learn==1.7.2 -timezonefinder==7.0.2 +tzfpy==1.0.0 deap==1.4.3 requests==2.32.5 pandas==2.3.2 @@ -19,6 +24,7 @@ platformdirs==4.4.0 psutil==7.1.0 pvlib==0.13.1 pydantic==2.11.9 +pydantic_extra_types==2.10.5 statsmodels==0.14.5 pydantic-settings==2.11.0 linkify-it-py==2.0.3 diff --git a/scripts/convert_lightweight_tags.py b/scripts/convert_lightweight_tags.py new file mode 100644 index 0000000..bd12a65 --- /dev/null +++ b/scripts/convert_lightweight_tags.py @@ -0,0 +1,69 @@ +#!/usr/bin/env python3 + +import subprocess +import sys + +MESSAGE_PREFIX = "Converted to annotated tag:" + +def run(cmd, capture_output=False): + """Run a shell command and return output if needed.""" + result = subprocess.run(cmd, shell=True, check=True, text=True, capture_output=capture_output) + return result.stdout.strip() if capture_output else None + +def get_all_tags(): + """Return a list of all tags.""" + return run("git tag", capture_output=True).splitlines() + +def is_lightweight(tag): + """Return True if a tag is lightweight (points to commit, not tag object).""" + return run(f"git cat-file -t {tag}", capture_output=True) == "commit" + +def get_commit_of_tag(tag): + """Return the commit SHA a tag points to.""" + return run(f"git rev-list -n 1 {tag}", capture_output=True) + +def convert_tag(tag): + """Delete and recreate a tag as annotated.""" + commit = get_commit_of_tag(tag) + print(f"Converting {tag} -> annotated ({commit})") + run(f"git tag -d {tag}") + run(f'git tag -a {tag} -m "{MESSAGE_PREFIX} {tag}" {commit}') + +def main(): + dry_run = "--dry-run" in sys.argv + push = "--push" in sys.argv + + tags = get_all_tags() + lightweight_tags = [t for t in tags if is_lightweight(t)] + + if not lightweight_tags: + print("✅ No lightweight tags found.") + return + + print("🔍 Lightweight tags found:\n " + "\n ".join(lightweight_tags)) + + if dry_run: + print("\n📝 Dry run: No changes will be made.") + return + + confirm = input("\n⚠️ Convert ALL of these tags to annotated? (y/N): ").lower() + if confirm != "y": + print("❌ Aborted.") + return + + for tag in lightweight_tags: + convert_tag(tag) + + print("\n✅ Conversion complete.") + + if push: + print("📤 Pushing updated tags to origin (force)...") + run("git push origin --tags --force") + print("✅ Tags pushed.") + else: + print("\n🚀 To push changes, run:\n git push origin --tags --force") + +if __name__ == "__main__": + print("=== Lightweight Tag Converter ===") + print("Usage: python convert_lightweight_tags.py [--dry-run] [--push]\n") + main() diff --git a/scripts/cz_check_branch.py b/scripts/cz_check_branch.py new file mode 100644 index 0000000..607f367 --- /dev/null +++ b/scripts/cz_check_branch.py @@ -0,0 +1,47 @@ +#!/usr/bin/env python3 +"""Branch name checker using regex (compatible with Commitizen v4.9.1). + +Cross-platform + .venv aware. +""" + +import os +import re +import subprocess +import sys +from pathlib import Path + + +def find_cz() -> str: + venv = os.getenv("VIRTUAL_ENV") + paths = [Path(venv)] if venv else [] + paths.append(Path.cwd() / ".venv") + + for base in paths: + cz = base / ("Scripts" if os.name == "nt" else "bin") / ("cz.exe" if os.name == "nt" else "cz") + if cz.exists(): + return str(cz) + return "cz" + + +def main(): + # Get current branch name + try: + branch = subprocess.check_output(["git", "rev-parse", "--abbrev-ref", "HEAD"], text=True).strip() + except subprocess.CalledProcessError: + print("❌ Could not determine current branch name.") + return 1 + + # Regex pattern + pattern = r"^(feat|fix|chore|docs|refactor|test)/[a-z0-9._-]+$" + + print(f"🔍 Checking branch name '{branch}'...") + if not re.match(pattern, branch): + print(f"❌ Branch name '{branch}' does not match pattern '{pattern}'") + return 1 + + print("✅ Branch name is valid.") + return 0 + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/scripts/cz_check_commit_message.py b/scripts/cz_check_commit_message.py new file mode 100644 index 0000000..be249f4 --- /dev/null +++ b/scripts/cz_check_commit_message.py @@ -0,0 +1,63 @@ +#!/usr/bin/env python3 +"""Commitizen commit message checker that is .venv aware. + +Works for commits with -m or commit message file. +""" + +import os +import subprocess +import sys +from pathlib import Path + + +def find_cz() -> str: + """Find Commitizen executable, preferring virtualenv.""" + venv = os.getenv("VIRTUAL_ENV") + paths = [] + if venv: + paths.append(Path(venv)) + paths.append(Path.cwd() / ".venv") + + for base in paths: + cz = base / ("Scripts" if os.name == "nt" else "bin") / ("cz.exe" if os.name == "nt" else "cz") + if cz.exists(): + return str(cz) + return "cz" + + +def main(): + cz = find_cz() + + # 1️⃣ Try commit-msg file (interactive commit) + commit_msg_file = sys.argv[1] if len(sys.argv) > 1 else None + + # 2️⃣ If not file, fallback to -m message (Git sets GIT_COMMIT_MSG in some environments, or we create a temp file) + if not commit_msg_file: + msg = os.getenv("GIT_COMMIT_MSG") or "" + if not msg: + print("⚠️ No commit message file or environment message found. Skipping Commitizen check.") + return 0 + import tempfile + + with tempfile.NamedTemporaryFile("w+", delete=False) as tmp: + tmp.write(msg) + tmp.flush() + commit_msg_file = tmp.name + + print(f"🔍 Checking commit message using {cz}...") + + try: + subprocess.check_call([cz, "check", "--commit-msg-file", commit_msg_file]) + print("✅ Commit message follows Commitizen convention.") + return 0 + except subprocess.CalledProcessError: + print("❌ Commit message validation failed.") + return 1 + finally: + # Clean up temp file if we created one + if 'tmp' in locals(): + os.unlink(tmp.name) + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/scripts/cz_check_new_commits.py b/scripts/cz_check_new_commits.py new file mode 100644 index 0000000..46473e8 --- /dev/null +++ b/scripts/cz_check_new_commits.py @@ -0,0 +1,70 @@ +#!/usr/bin/env python3 +"""Pre-push hook: Commitizen check for *new commits only*. + +Cross-platform + virtualenv-aware: +- Prefers activated virtual environment (VIRTUAL_ENV) +- Falls back to ./.venv if found +- Falls back to global cz otherwise +""" + +import os +import subprocess +import sys +from pathlib import Path + + +def find_cz_executable() -> str: + """Return path to Commitizen executable, preferring virtual environments.""" + # 1️⃣ Active virtual environment (if running inside one) + venv_env = os.getenv("VIRTUAL_ENV") + if venv_env: + cz_path = Path(venv_env) / ("Scripts" if os.name == "nt" else "bin") / ("cz.exe" if os.name == "nt" else "cz") + if cz_path.exists(): + return str(cz_path) + + # 2️⃣ Local .venv in repo root + repo_venv = Path.cwd() / ".venv" + cz_path = repo_venv / ("Scripts" if os.name == "nt" else "bin") / ("cz.exe" if os.name == "nt" else "cz") + if cz_path.exists(): + return str(cz_path) + + # 3️⃣ Global fallback + return "cz" + + +def get_merge_base() -> str | None: + """Return merge-base between HEAD and upstream branch, or None if unavailable.""" + try: + return ( + subprocess.check_output( + ["git", "merge-base", "@{u}", "HEAD"], + stderr=subprocess.DEVNULL, + text=True, + ) + .strip() + ) + except subprocess.CalledProcessError: + return None + + +def main() -> int: + cz = find_cz_executable() + base = get_merge_base() + + if not base: + print("⚠️ No upstream found; skipping Commitizen check for new commits.") + return 0 + + print(f"🔍 Using {cz} to check new commits from {base}..HEAD ...") + + try: + subprocess.check_call([cz, "check", "--rev-range", f"{base}..HEAD"]) + print("✅ All new commits follow Commitizen conventions.") + return 0 + except subprocess.CalledProcessError as e: + print("❌ Commitizen check failed for one or more new commits.") + return e.returncode + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/scripts/generate_config_md.py b/scripts/generate_config_md.py index eac87fb..8804f26 100755 --- a/scripts/generate_config_md.py +++ b/scripts/generate_config_md.py @@ -16,7 +16,7 @@ from pydantic_core import PydanticUndefined from akkudoktoreos.config.config import ConfigEOS, GeneralSettings, get_config from akkudoktoreos.core.pydantic import PydanticBaseModel -from akkudoktoreos.utils.docs import get_model_structure_from_examples +from akkudoktoreos.utils.datetimeutil import to_datetime documented_types: set[PydanticBaseModel] = set() undocumented_types: dict[PydanticBaseModel, tuple[str, list[str]]] = dict() @@ -52,6 +52,45 @@ def resolve_nested_types(field_type: Any, parent_types: list[str]) -> list[tuple return resolved_types +def get_example_or_default(field_name: str, field_info: FieldInfo, example_ix: int) -> Any: + """Generate a default value for a field, considering constraints.""" + if field_info.examples is not None: + try: + return field_info.examples[example_ix] + except IndexError: + return field_info.examples[-1] + + if field_info.default is not None: + return field_info.default + + raise NotImplementedError(f"No default or example provided '{field_name}': {field_info}") + + +def get_model_structure_from_examples( + model_class: type[PydanticBaseModel], multiple: bool +) -> list[dict[str, Any]]: + """Create a model instance with default or example values, respecting constraints.""" + example_max_length = 1 + + # Get first field with examples (non-default) to get example_max_length + if multiple: + for _, field_info in model_class.model_fields.items(): + if field_info.examples is not None: + example_max_length = len(field_info.examples) + break + + example_data: list[dict[str, Any]] = [{} for _ in range(example_max_length)] + + for field_name, field_info in model_class.model_fields.items(): + if field_info.deprecated: + continue + for example_ix in range(example_max_length): + example_data[example_ix][field_name] = get_example_or_default( + field_name, field_info, example_ix + ) + return example_data + + def create_model_from_examples( model_class: PydanticBaseModel, multiple: bool ) -> list[PydanticBaseModel]: @@ -163,6 +202,7 @@ def generate_config_table_md( inner_types: dict[PydanticBaseModel, tuple[str, list[str]]] = dict() def extract_nested_models(subtype: Any, subprefix: str, parent_types: list[str]): + """Extract nested models.""" if subtype in inner_types.keys(): return nested_types = resolve_nested_types(subtype, []) @@ -174,9 +214,9 @@ def generate_config_table_md( else: new_prefix = f"{subprefix}" inner_types.setdefault(nested_type, (new_prefix, new_parent_types)) - for nested_field_name, nested_field_info in list( - nested_type.model_fields.items() - ) + list(nested_type.model_computed_fields.items()): + + # Handle normal fields + for nested_field_name, nested_field_info in nested_type.model_fields.items(): nested_field_type = nested_field_info.annotation if new_prefix: new_prefix += f"{nested_field_name.upper()}__" @@ -186,6 +226,8 @@ def generate_config_table_md( new_parent_types + [nested_field_name], ) + # Do not extract computed fields + extract_nested_models(field_type, f"{prefix}{config_name}__", toplevel_keys + [field_name]) for new_type, info in inner_types.items(): @@ -290,6 +332,13 @@ def generate_config_md(config_eos: ConfigEOS) -> str: markdown ) + # Assure timezone name does not leak to documentation + tz_name = to_datetime().timezone_name + markdown = re.sub(re.escape(tz_name), "Europe/Berlin", markdown, flags=re.IGNORECASE) + # Also replace UTC, as GitHub CI always is on UTC + markdown = re.sub(re.escape("UTC"), "Europe/Berlin", markdown, flags=re.IGNORECASE) + + return markdown diff --git a/single_test_optimization.py b/single_test_optimization.py index ccc9db9..642c5cb 100755 --- a/single_test_optimization.py +++ b/single_test_optimization.py @@ -1,6 +1,7 @@ #!/usr/bin/env python3 import argparse +import asyncio import cProfile import json import pstats @@ -9,21 +10,27 @@ import time from typing import Any import numpy as np +from loguru import logger from akkudoktoreos.config.config import get_config from akkudoktoreos.core.ems import get_ems +from akkudoktoreos.core.emsettings import EnergyManagementMode from akkudoktoreos.optimization.genetic import ( - OptimizationParameters, - optimization_problem, + GeneticOptimizationParameters, ) from akkudoktoreos.prediction.prediction import get_prediction +from akkudoktoreos.utils.datetimeutil import to_datetime + +config_eos = get_config() +prediction_eos = get_prediction() +ems_eos = get_ems() -def prepare_optimization_real_parameters() -> OptimizationParameters: +def prepare_optimization_real_parameters() -> GeneticOptimizationParameters: """Prepare and return optimization parameters with real world data. Returns: - OptimizationParameters: Configured optimization parameters + GeneticOptimizationParameters: Configured optimization parameters """ # Make a config settings = { @@ -35,6 +42,18 @@ def prepare_optimization_real_parameters() -> OptimizationParameters: "hours": 48, "historic_hours": 24, }, + "optimization": { + "horizon_hours": 24, + "interval": 3600, + "genetic": { + "individuals": 300, + "generations": 400, + "seed": None, + "penalties": { + "ev_soc_miss": 10, + }, + }, + }, # PV Forecast "pvforecast": { "provider": "PVForecastAkkudoktor", @@ -81,14 +100,30 @@ def prepare_optimization_real_parameters() -> OptimizationParameters: "load": { "provider": "LoadAkkudoktor", "provider_settings": { - "loadakkudoktor_year_energy": 5000, # Energy consumption per year in kWh + "LoadAkkudoktor": { + "loadakkudoktor_year_energy": 5000, # Energy consumption per year in kWh + }, }, }, # -- Simulations -- + # Assure we have charge rates for the EV + "devices": { + "max_electric_vehicles": 1, + "electric_vehicles": [ + { + "charge_rates": [ + 0.0, + 6.0 / 16.0, + 8.0 / 16.0, + 10.0 / 16.0, + 12.0 / 16.0, + 14.0 / 16.0, + 1.0, + ], + }, + ], + }, } - config_eos = get_config() - prediction_eos = get_prediction() - ems_eos = get_ems() # Update/ set configuration config_eos.merge_settings_from_dict(settings) @@ -96,14 +131,14 @@ def prepare_optimization_real_parameters() -> OptimizationParameters: # Get current prediction data for optimization run ems_eos.set_start_datetime() print( - f"Real data prediction from {prediction_eos.start_datetime} to {prediction_eos.end_datetime}" + f"Real data prediction from {prediction_eos.ems_start_datetime} to {prediction_eos.end_datetime}" ) prediction_eos.update_data() # PV Forecast (in W) pv_forecast = prediction_eos.key_to_array( key="pvforecast_ac_power", - start_datetime=prediction_eos.start_datetime, + start_datetime=prediction_eos.ems_start_datetime, end_datetime=prediction_eos.end_datetime, ) print(f"pv_forecast: {pv_forecast}") @@ -111,7 +146,7 @@ def prepare_optimization_real_parameters() -> OptimizationParameters: # Temperature Forecast (in degree C) temperature_forecast = prediction_eos.key_to_array( key="weather_temp_air", - start_datetime=prediction_eos.start_datetime, + start_datetime=prediction_eos.ems_start_datetime, end_datetime=prediction_eos.end_datetime, ) print(f"temperature_forecast: {temperature_forecast}") @@ -119,7 +154,7 @@ def prepare_optimization_real_parameters() -> OptimizationParameters: # Electricity Price (in Euro per Wh) strompreis_euro_pro_wh = prediction_eos.key_to_array( key="elecprice_marketprice_wh", - start_datetime=prediction_eos.start_datetime, + start_datetime=prediction_eos.ems_start_datetime, end_datetime=prediction_eos.end_datetime, ) print(f"strompreis_euro_pro_wh: {strompreis_euro_pro_wh}") @@ -127,7 +162,7 @@ def prepare_optimization_real_parameters() -> OptimizationParameters: # Overall System Load (in W) gesamtlast = prediction_eos.key_to_array( key="load_mean", - start_datetime=prediction_eos.start_datetime, + start_datetime=prediction_eos.ems_start_datetime, end_datetime=prediction_eos.end_datetime, ) print(f"gesamtlast: {gesamtlast}") @@ -137,7 +172,7 @@ def prepare_optimization_real_parameters() -> OptimizationParameters: print(f"start_solution: {start_solution}") # Define parameters for the optimization problem - return OptimizationParameters( + return GeneticOptimizationParameters( **{ "ems": { "preis_euro_pro_wh_akku": 0e-05, @@ -147,14 +182,18 @@ def prepare_optimization_real_parameters() -> OptimizationParameters: "strompreis_euro_pro_wh": strompreis_euro_pro_wh, }, "pv_akku": { - "device_id": "battery1", + "device_id": "battery 1", "capacity_wh": 26400, "initial_soc_percentage": 15, "min_soc_percentage": 15, }, - "inverter": {"device_id": "iv1", "max_power_wh": 10000, "battery_id": "battery1"}, + "inverter": { + "device_id": "inverter 1", + "max_power_wh": 10000, + "battery_id": "battery 1", + }, "eauto": { - "device_id": "ev1", + "device_id": "electric vehicle 1", "min_soc_percentage": 50, "capacity_wh": 60000, "charging_efficiency": 0.95, @@ -167,12 +206,49 @@ def prepare_optimization_real_parameters() -> OptimizationParameters: ) -def prepare_optimization_parameters() -> OptimizationParameters: +def prepare_optimization_parameters() -> GeneticOptimizationParameters: """Prepare and return optimization parameters with predefined data. Returns: - OptimizationParameters: Configured optimization parameters + GeneticOptimizationParameters: Configured optimization parameters """ + # Initialize the optimization problem using the default configuration + config_eos.merge_settings_from_dict( + { + "prediction": {"hours": 48}, + "optimization": { + "horizon_hours": 48, + "interval": 3600, + "genetic": { + "individuals": 300, + "generations": 400, + "seed": None, + "penalties": { + "ev_soc_miss": 10, + }, + }, + }, + # Assure we have charge rates for the EV + "devices": { + "max_electric_vehicles": 1, + "electric_vehicles": [ + { + "device_id": "Default EV", + "charge_rates": [ + 0.0, + 6.0 / 16.0, + 8.0 / 16.0, + 10.0 / 16.0, + 12.0 / 16.0, + 14.0 / 16.0, + 1.0, + ], + }, + ], + }, + } + ) + # PV Forecast (in W) pv_forecast = np.zeros(48) pv_forecast[12] = 5000 @@ -291,7 +367,7 @@ def prepare_optimization_parameters() -> OptimizationParameters: start_solution = None # Define parameters for the optimization problem - return OptimizationParameters( + return GeneticOptimizationParameters( **{ "ems": { "preis_euro_pro_wh_akku": 0e-05, @@ -301,14 +377,18 @@ def prepare_optimization_parameters() -> OptimizationParameters: "strompreis_euro_pro_wh": strompreis_euro_pro_wh, }, "pv_akku": { - "device_id": "battery1", + "device_id": "battery 1", "capacity_wh": 26400, "initial_soc_percentage": 15, "min_soc_percentage": 15, }, - "inverter": {"device_id": "iv1", "max_power_wh": 10000, "battery_id": "battery1"}, + "inverter": { + "device_id": "inverter 1", + "max_power_wh": 10000, + "battery_id": "battery 1", + }, "eauto": { - "device_id": "ev1", + "device_id": "electric vehicle 1", "min_soc_percentage": 50, "capacity_wh": 60000, "charging_efficiency": 0.95, @@ -336,27 +416,30 @@ def run_optimization( # Prepare parameters if parameters_file: with open(parameters_file, "r") as f: - parameters = OptimizationParameters(**json.load(f)) + parameters = GeneticOptimizationParameters(**json.load(f)) elif real_world: parameters = prepare_optimization_real_parameters() else: parameters = prepare_optimization_parameters() + logger.info("Optimization Parameters:") + logger.info(parameters.model_dump_json(indent=4)) - if verbose: - print("\nOptimization Parameters:") - print(parameters.model_dump_json(indent=4)) + if start_hour is None: + start_datetime = None + else: + start_datetime = to_datetime().set(hour=start_hour) - # Initialize the optimization problem using the default configuration - config_eos = get_config() - config_eos.merge_settings_from_dict( - {"prediction": {"hours": 48}, "optimization": {"hours": 48}} + asyncio.run( + ems_eos.run( + start_datetime=start_datetime, + mode=EnergyManagementMode.OPTIMIZATION, + genetic_parameters=parameters, + genetic_individuals=ngen, + genetic_seed=seed, + ) ) - opt_class = optimization_problem(verbose=verbose, fixed_seed=seed) - # Perform the optimisation based on the provided parameters and start hour - result = opt_class.optimierung_ems(parameters=parameters, start_hour=start_hour, ngen=ngen) - - return result.model_dump_json() + return ems_eos.genetic_solution().model_dump_json() def main(): diff --git a/single_test_prediction.py b/single_test_prediction.py index ec2382f..af1c745 100644 --- a/single_test_prediction.py +++ b/single_test_prediction.py @@ -93,6 +93,22 @@ def config_elecprice() -> dict: return settings +def config_feedintarifffixed() -> dict: + """Configure settings for feed in tariff forecast.""" + settings = { + "general": { + "latitude": 52.52, + "longitude": 13.405, + }, + "prediction": { + "hours": 48, + "historic_hours": 24, + }, + "feedintariff": dict(), + } + return settings + + def config_load() -> dict: """Configure settings for load forecast.""" settings = { @@ -130,6 +146,9 @@ def run_prediction(provider_id: str, verbose: bool = False) -> str: elif provider_id in ("ElecPriceAkkudoktor",): settings = config_elecprice() forecast = "elecprice" + elif provider_id in ("FeedInTariffFixed",): + settings = config_feedintarifffixed() + forecast = "elecprice" elif provider_id in ("LoadAkkudoktor",): settings = config_elecprice() forecast = "load" @@ -151,6 +170,7 @@ def run_prediction(provider_id: str, verbose: bool = False) -> str: print(settings) print("\nProvider\n----------") print(f"elecprice.provider: {config_eos.elecprice.provider}") + print(f"feedintariff.provider: {config_eos.feedintariff.provider}") print(f"load.provider: {config_eos.load.provider}") print(f"pvforecast.provider: {config_eos.pvforecast.provider}") print(f"weather.provider: {config_eos.weather.provider}") diff --git a/src/akkudoktoreos/config/config.py b/src/akkudoktoreos/config/config.py index 4260650..d0b6ff0 100644 --- a/src/akkudoktoreos/config/config.py +++ b/src/akkudoktoreos/config/config.py @@ -14,28 +14,28 @@ import shutil from pathlib import Path from typing import Any, ClassVar, Optional, Type +import pydantic_settings from loguru import logger from platformdirs import user_config_dir, user_data_dir -from pydantic import Field, computed_field -from pydantic_settings import ( - BaseSettings, - JsonConfigSettingsSource, - PydanticBaseSettingsSource, - SettingsConfigDict, -) +from pydantic import Field, computed_field, field_validator # settings from akkudoktoreos.config.configabc import SettingsBaseModel +from akkudoktoreos.config.configmigrate import migrate_config_file from akkudoktoreos.core.cachesettings import CacheCommonSettings from akkudoktoreos.core.coreabc import SingletonMixin from akkudoktoreos.core.decorators import classproperty -from akkudoktoreos.core.emsettings import EnergyManagementCommonSettings +from akkudoktoreos.core.emsettings import ( + EnergyManagementCommonSettings, +) from akkudoktoreos.core.logsettings import LoggingCommonSettings from akkudoktoreos.core.pydantic import PydanticModelNestedValueMixin, merge_models -from akkudoktoreos.devices.settings import DevicesCommonSettings +from akkudoktoreos.core.version import __version__ +from akkudoktoreos.devices.devices import DevicesCommonSettings from akkudoktoreos.measurement.measurement import MeasurementCommonSettings from akkudoktoreos.optimization.optimization import OptimizationCommonSettings from akkudoktoreos.prediction.elecprice import ElecPriceCommonSettings +from akkudoktoreos.prediction.feedintariff import FeedInTariffCommonSettings from akkudoktoreos.prediction.load import LoadCommonSettings from akkudoktoreos.prediction.prediction import PredictionCommonSettings from akkudoktoreos.prediction.pvforecast import PVForecastCommonSettings @@ -83,6 +83,10 @@ class GeneralSettings(SettingsBaseModel): _config_folder_path: ClassVar[Optional[Path]] = None _config_file_path: ClassVar[Optional[Path]] = None + version: str = Field( + default=__version__, description="Configuration file version. Used to check compatibility." + ) + data_folder_path: Optional[Path] = Field( default=None, description="Path to EOS data directory.", examples=[None, "/home/eos/data"] ) @@ -131,11 +135,25 @@ class GeneralSettings(SettingsBaseModel): """Path to EOS configuration file.""" return self._config_file_path + compatible_versions: ClassVar[list[str]] = [__version__] -class SettingsEOS(BaseSettings, PydanticModelNestedValueMixin): + @field_validator("version") + @classmethod + def check_version(cls, v: str) -> str: + if v not in cls.compatible_versions: + error = ( + f"Incompatible configuration version '{v}'. " + f"Expected one of: {', '.join(cls.compatible_versions)}." + ) + logger.error(error) + raise ValueError(error) + return v + + +class SettingsEOS(pydantic_settings.BaseSettings, PydanticModelNestedValueMixin): """Settings for all EOS. - Used by updating the configuration with specific settings only. + Only used to update the configuration with specific settings. """ general: Optional[GeneralSettings] = Field( @@ -174,6 +192,10 @@ class SettingsEOS(BaseSettings, PydanticModelNestedValueMixin): default=None, description="Electricity Price Settings", ) + feedintariff: Optional[FeedInTariffCommonSettings] = Field( + default=None, + description="Feed In Tariff Settings", + ) load: Optional[LoadCommonSettings] = Field( default=None, description="Load Settings", @@ -195,7 +217,7 @@ class SettingsEOS(BaseSettings, PydanticModelNestedValueMixin): description="Utilities Settings", ) - model_config = SettingsConfigDict( + model_config = pydantic_settings.SettingsConfigDict( env_nested_delimiter="__", nested_model_default_partial_update=True, env_prefix="EOS_", @@ -218,12 +240,18 @@ class SettingsEOSDefaults(SettingsEOS): optimization: OptimizationCommonSettings = OptimizationCommonSettings() prediction: PredictionCommonSettings = PredictionCommonSettings() elecprice: ElecPriceCommonSettings = ElecPriceCommonSettings() + feedintariff: FeedInTariffCommonSettings = FeedInTariffCommonSettings() load: LoadCommonSettings = LoadCommonSettings() pvforecast: PVForecastCommonSettings = PVForecastCommonSettings() weather: WeatherCommonSettings = WeatherCommonSettings() server: ServerCommonSettings = ServerCommonSettings() utils: UtilsCommonSettings = UtilsCommonSettings() + def __hash__(self) -> int: + # Just for usage in configmigrate, finally overwritten when used by ConfigEOS. + # This is mutable, so pydantic does not set a hash. + return id(self) + class ConfigEOS(SingletonMixin, SettingsEOSDefaults): """Singleton configuration handler for the EOS application. @@ -290,33 +318,34 @@ class ConfigEOS(SingletonMixin, SettingsEOSDefaults): @classmethod def settings_customise_sources( cls, - settings_cls: Type[BaseSettings], - init_settings: PydanticBaseSettingsSource, - env_settings: PydanticBaseSettingsSource, - dotenv_settings: PydanticBaseSettingsSource, - file_secret_settings: PydanticBaseSettingsSource, - ) -> tuple[PydanticBaseSettingsSource, ...]: - """Customizes the order and handling of settings sources for a Pydantic BaseSettings subclass. + settings_cls: Type[pydantic_settings.BaseSettings], + init_settings: pydantic_settings.PydanticBaseSettingsSource, + env_settings: pydantic_settings.PydanticBaseSettingsSource, + dotenv_settings: pydantic_settings.PydanticBaseSettingsSource, + file_secret_settings: pydantic_settings.PydanticBaseSettingsSource, + ) -> tuple[pydantic_settings.PydanticBaseSettingsSource, ...]: + """Customizes the order and handling of settings sources for a pydantic_settings.BaseSettings subclass. This method determines the sources for application configuration settings, including environment variables, dotenv files and JSON configuration files. It ensures that a default configuration file exists and creates one if necessary. Args: - settings_cls (Type[BaseSettings]): The Pydantic BaseSettings class for which sources are customized. - init_settings (PydanticBaseSettingsSource): The initial settings source, typically passed at runtime. - env_settings (PydanticBaseSettingsSource): Settings sourced from environment variables. - dotenv_settings (PydanticBaseSettingsSource): Settings sourced from a dotenv file. - file_secret_settings (PydanticBaseSettingsSource): Unused (needed for parent class interface). + settings_cls (Type[pydantic_settings.BaseSettings]): The Pydantic BaseSettings class for + which sources are customized. + init_settings (pydantic_settings.PydanticBaseSettingsSource): The initial settings source, typically passed at runtime. + env_settings (pydantic_settings.PydanticBaseSettingsSource): Settings sourced from environment variables. + dotenv_settings (pydantic_settings.PydanticBaseSettingsSource): Settings sourced from a dotenv file. + file_secret_settings (pydantic_settings.PydanticBaseSettingsSource): Unused (needed for parent class interface). Returns: - tuple[PydanticBaseSettingsSource, ...]: A tuple of settings sources in the order they should be applied. + tuple[pydantic_settings.PydanticBaseSettingsSource, ...]: A tuple of settings sources in the order they should be applied. Behavior: 1. Checks for the existence of a JSON configuration file in the expected location. 2. If the configuration file does not exist, creates the directory (if needed) and attempts to copy a default configuration file to the location. If the copy fails, uses the default configuration file directly. - 3. Creates a `JsonConfigSettingsSource` for both the configuration file and the default configuration file. + 3. Creates a `pydantic_settings.JsonConfigSettingsSource` for both the configuration file and the default configuration file. 4. Updates class attributes `GeneralSettings._config_folder_path` and `GeneralSettings._config_file_path` to reflect the determined paths. 5. Returns a tuple containing all provided and newly created settings sources in the desired order. @@ -325,13 +354,7 @@ class ConfigEOS(SingletonMixin, SettingsEOSDefaults): - This method logs a warning if the default configuration file cannot be copied. - It ensures that a fallback to the default configuration file is always possible. """ - setting_sources = [ - init_settings, - env_settings, - dotenv_settings, - ] - - file_settings: Optional[JsonConfigSettingsSource] = None + # Ensure we know and have the config folder path and the config file config_file, exists = cls._get_config_file_path() config_dir = config_file.parent if not exists: @@ -342,20 +365,38 @@ class ConfigEOS(SingletonMixin, SettingsEOSDefaults): logger.warning(f"Could not copy default config: {exc}. Using default config...") config_file = cls.config_default_file_path config_dir = config_file.parent - try: - file_settings = JsonConfigSettingsSource(settings_cls, json_file=config_file) - setting_sources.append(file_settings) - except Exception as e: - logger.error( - f"Error reading config file '{config_file}' (falling back to default config): {e}" - ) - default_settings = JsonConfigSettingsSource( - settings_cls, json_file=cls.config_default_file_path - ) + # Remember config_dir and config file GeneralSettings._config_folder_path = config_dir GeneralSettings._config_file_path = config_file + # All the settings sources in priority sequence + setting_sources = [ + init_settings, + env_settings, + dotenv_settings, + ] + + # Apend file settings to sources + file_settings: Optional[pydantic_settings.JsonConfigSettingsSource] = None + try: + backup_file = config_file.with_suffix(".bak") + if migrate_config_file(config_file, backup_file): + # If correct version add it as settings source + file_settings = pydantic_settings.JsonConfigSettingsSource( + settings_cls, json_file=config_file + ) + setting_sources.append(file_settings) + except Exception as ex: + logger.error( + f"Error reading config file '{config_file}' (falling back to default config): {ex}" + ) + + # Append default settings to sources + default_settings = pydantic_settings.JsonConfigSettingsSource( + settings_cls, json_file=cls.config_default_file_path + ) setting_sources.append(default_settings) + return tuple(setting_sources) @classproperty @@ -374,28 +415,24 @@ class ConfigEOS(SingletonMixin, SettingsEOSDefaults): Configuration data is loaded from a configuration file or a default one is created if none exists. """ + logger.debug("Config init with parameters {} {}", args, kwargs) + # Check for singleton guard if hasattr(self, "_initialized"): return self._setup(self, *args, **kwargs) def _setup(self, *args: Any, **kwargs: Any) -> None: """Re-initialize global settings.""" - # Check for config file content/ version type - config_file, exists = self._get_config_file_path() - if exists: - with config_file.open("r", encoding="utf-8", newline=None) as f_config: - config_txt = f_config.read() - if '"directories": {' in config_txt or '"server_eos_host": ' in config_txt: - error_msg = f"Configuration file '{config_file}' is outdated. Please remove or update manually." - logger.error(error_msg) - raise ValueError(error_msg) - # Assure settings base knows EOS configuration + logger.debug("Config setup with parameters {} {}", args, kwargs) + # Assure settings base knows the singleton EOS configuration SettingsBaseModel.config = self - # (Re-)load settings + # (Re-)load settings - call base class init SettingsEOSDefaults.__init__(self, *args, **kwargs) # Init config file and data folder pathes self._create_initial_config_file() self._update_data_folder_path() + self._initialized = True + logger.debug("Config setup:\n{}", self) def merge_settings(self, settings: SettingsEOS) -> None: """Merges the provided settings into the global settings for EOS, with optional overwrite. @@ -488,6 +525,11 @@ class ConfigEOS(SingletonMixin, SettingsEOSDefaults): def _get_config_file_path(cls) -> tuple[Path, bool]: """Find a valid configuration file or return the desired path for a new config file. + Searches: + 1. environment variable directory + 2. user configuration directory + 3. current working directory + Returns: tuple[Path, bool]: The path to the configuration file and if there is already a config file there """ diff --git a/src/akkudoktoreos/config/configmigrate.py b/src/akkudoktoreos/config/configmigrate.py new file mode 100644 index 0000000..1367757 --- /dev/null +++ b/src/akkudoktoreos/config/configmigrate.py @@ -0,0 +1,225 @@ +"""Migrate config file to actual version.""" + +import json +import shutil +from pathlib import Path +from typing import Any, Callable, Dict, List, Set, Tuple, Union + +from loguru import logger + +from akkudoktoreos.core.version import __version__ + +# ----------------------------- +# Global migration map constant +# ----------------------------- +# key: old JSON path, value: either +# - str (new model path) +# - tuple[str, Callable[[Any], Any]] (new path + transform) +# - None (drop) +MIGRATION_MAP: Dict[str, Union[str, Tuple[str, Callable[[Any], Any]], None]] = { + # 0.1.0 -> now + "devices/batteries/0/initial_soc_percentage": None, + "devices/electric_vehicles/0/initial_soc_percentage": None, + "elecprice/provider_settings/import_file_path": "elecprice/provider_settings/ElecPriceImport/import_file_path", + "elecprice/provider_settings/import_json": "elecprice/provider_settings/ElecPriceImport/import_json", + "load/provider_settings/import_file_path": "load/provider_settings/LoadImport/import_file_path", + "load/provider_settings/import_json": "load/provider_settings/LoadImport/import_json", + "load/provider_settings/loadakkudoktor_year_energy": "load/provider_settings/LoadAkkudoktor/loadakkudoktor_year_energy", + "load/provider_settings/load_vrm_idsite": "load/provider_settings/LoadVrm/load_vrm_idsite", + "load/provider_settings/load_vrm_token": "load/provider_settings/LoadVrm/load_vrm_token", + "logging/level": "logging/console_level", + "logging/root_level": None, + "measurement/load0_name": "measurement/load_emr_keys/0", + "measurement/load1_name": "measurement/load_emr_keys/1", + "measurement/load2_name": "measurement/load_emr_keys/2", + "measurement/load3_name": "measurement/load_emr_keys/3", + "measurement/load4_name": "measurement/load_emr_keys/4", + "optimization/ev_available_charge_rates_percent": ( + "devices/electric_vehicles/0/charge_rates", + lambda v: [x / 100 for x in v], + ), + "optimization/hours": "optimization/horizon_hours", + "optimization/penalty": ("optimization/genetic/penalties/ev_soc_miss", lambda v: float(v)), + "pvforecast/provider_settings/import_file_path": "pvforecast/provider_settings/PVForecastImport/import_file_path", + "pvforecast/provider_settings/import_json": "pvforecast/provider_settings/PVForecastImport/import_json", + "pvforecast/provider_settings/load_vrm_idsite": "pvforecast/provider_settings/PVForecastVrm/load_vrm_idsite", + "pvforecast/provider_settings/load_vrm_token": "pvforecast/provider_settings/PVForecastVrm/load_vrm_token", + "weather/provider_settings/import_file_path": "weather/provider_settings/WeatherImport/import_file_path", + "weather/provider_settings/import_json": "weather/provider_settings/WeatherImport/import_json", +} + +# ----------------------------- +# Global migration stats +# ----------------------------- +migrated_source_paths: Set[str] = set() +mapped_count: int = 0 +auto_count: int = 0 +skipped_paths: List[str] = [] + + +def migrate_config_file(config_file: Path, backup_file: Path) -> bool: + """Migrate configuration file to the current version. + + Returns: + bool: True if up-to-date or successfully migrated, False on failure. + """ + global migrated_source_paths, mapped_count, auto_count, skipped_paths + + # Reset globals at the start of each migration + migrated_source_paths = set() + mapped_count = 0 + auto_count = 0 + skipped_paths = [] + + try: + with config_file.open("r", encoding="utf-8") as f: + config_data: Dict[str, Any] = json.load(f) + except (FileNotFoundError, json.JSONDecodeError) as e: + logger.error(f"Failed to read configuration file '{config_file}': {e}") + return False + + match config_data: + case {"general": {"version": v}} if v == __version__: + logger.debug(f"Configuration file '{config_file}' is up to date (v{v}).") + return True + case _: + logger.info( + f"Configuration file '{config_file}' is missing current version info. " + f"Starting migration to v{__version__}..." + ) + + try: + # Backup existing file - we already know it is existing + try: + config_file.replace(backup_file) + logger.info(f"Backed up old configuration to '{backup_file}'.") + except Exception as e_replace: + try: + shutil.copy(config_file, backup_file) + logger.info( + f"Could not replace; copied old configuration to '{backup_file}' instead." + ) + except Exception as e_copy: + logger.warning( + f"Failed to backup existing config (replace: {e_replace}; copy: {e_copy}). Continuing without backup." + ) + + from akkudoktoreos.config.config import SettingsEOSDefaults + + new_config = SettingsEOSDefaults() + + # 1) Apply explicit migration map + for old_path, mapping in MIGRATION_MAP.items(): + new_path = None + transform = None + if mapping is None: + migrated_source_paths.add(old_path.strip("/")) + logger.debug(f"🗑️ Migration map: dropping '{old_path}'") + continue + if isinstance(mapping, tuple): + new_path, transform = mapping + else: + new_path = mapping + + old_value = _get_json_nested_value(config_data, old_path) + if old_value is None: + continue + + try: + if transform: + old_value = transform(old_value) + new_config.set_nested_value(new_path, old_value) + migrated_source_paths.add(old_path.strip("/")) + mapped_count += 1 + logger.debug(f"✅ Migrated mapped '{old_path}' → '{new_path}' = {old_value!r}") + except Exception as e: + logger.opt(exception=True).warning( + f"Failed mapped migration '{old_path}' -> '{new_path}': {e}", exc_info=True + ) + + # 2) Automatic migration for remaining fields + auto_count += _migrate_matching_fields( + config_data, new_config, migrated_source_paths, skipped_paths + ) + + # 3) Ensure version + try: + new_config.set_nested_value("general/version", __version__) + except Exception as e: + logger.warning(f"Could not set version on new configuration model: {e}") + + # 4) Write migrated configuration + try: + with config_file.open("w", encoding="utf-8", newline=None) as f_out: + json_str = new_config.model_dump_json(indent=4) + f_out.write(json_str) + except Exception as e_write: + logger.error(f"Failed to write migrated configuration to '{config_file}': {e_write}") + return False + + # 5) Log final migration summary + logger.info( + f"Migration summary for '{config_file}': " + f"mapped fields: {mapped_count}, automatically migrated: {auto_count}, skipped: {len(skipped_paths)}" + ) + if skipped_paths: + logger.debug(f"Skipped paths: {', '.join(skipped_paths)}") + + logger.success(f"Configuration successfully migrated to version {__version__}.") + return True + + except Exception as e: + logger.exception(f"Unexpected error during migration: {e}") + return False + + +def _get_json_nested_value(data: dict, path: str) -> Any: + """Retrieve a nested value from a JSON-like dict using '/'-separated path.""" + current: Any = data + for part in path.strip("/").split("/"): + if isinstance(current, list): + try: + part_idx = int(part) + current = current[part_idx] + except (ValueError, IndexError): + return None + elif isinstance(current, dict): + if part not in current: + return None + current = current[part] + else: + return None + return current + + +def _migrate_matching_fields( + source: Dict[str, Any], + target_model: Any, + migrated_source_paths: Set[str], + skipped_paths: List[str], + prefix: str = "", +) -> int: + """Recursively copy matching keys from source dict into target_model using set_nested_value. + + Returns: + int: number of fields successfully auto-migrated + """ + count: int = 0 + for key, value in source.items(): + full_path = f"{prefix}/{key}".strip("/") + + if full_path in migrated_source_paths: + continue + + if isinstance(value, dict): + count += _migrate_matching_fields( + value, target_model, migrated_source_paths, skipped_paths, full_path + ) + else: + try: + target_model.set_nested_value(full_path, value) + count += 1 + except Exception: + skipped_paths.append(full_path) + continue + return count diff --git a/src/akkudoktoreos/core/cache.py b/src/akkudoktoreos/core/cache.py index 45de386..8b72a09 100644 --- a/src/akkudoktoreos/core/cache.py +++ b/src/akkudoktoreos/core/cache.py @@ -28,12 +28,17 @@ from typing import ( import cachebox from loguru import logger -from pendulum import DateTime, Duration from pydantic import Field from akkudoktoreos.core.coreabc import ConfigMixin, SingletonMixin from akkudoktoreos.core.pydantic import PydanticBaseModel -from akkudoktoreos.utils.datetimeutil import compare_datetimes, to_datetime, to_duration +from akkudoktoreos.utils.datetimeutil import ( + DateTime, + Duration, + compare_datetimes, + to_datetime, + to_duration, +) # --------------------------------- # In-Memory Caching Functionality @@ -43,25 +48,28 @@ from akkudoktoreos.utils.datetimeutil import compare_datetimes, to_datetime, to_ TCallable = TypeVar("TCallable", bound=Callable[..., Any]) -def cache_until_update_store_callback(event: int, key: Any, value: Any) -> None: - """Calback function for CacheUntilUpdateStore.""" - CacheUntilUpdateStore.last_event = event - CacheUntilUpdateStore.last_key = key - CacheUntilUpdateStore.last_value = value +def cache_energy_management_store_callback(event: int, key: Any, value: Any) -> None: + """Calback function for CacheEnergyManagementStore.""" + CacheEnergyManagementStore.last_event = event + CacheEnergyManagementStore.last_key = key + CacheEnergyManagementStore.last_value = value if event == cachebox.EVENT_MISS: - CacheUntilUpdateStore.miss_count += 1 + CacheEnergyManagementStore.miss_count += 1 elif event == cachebox.EVENT_HIT: - CacheUntilUpdateStore.hit_count += 1 + CacheEnergyManagementStore.hit_count += 1 else: # unreachable code raise NotImplementedError -class CacheUntilUpdateStore(SingletonMixin): +class CacheEnergyManagementStore(SingletonMixin): """Singleton-based in-memory LRU (Least Recently Used) cache. This cache is shared across the application to store results of decorated - methods or functions until the next EMS (Energy Management System) update. + methods or functions during energy management runs. + + Energy management tasks shall clear the cache at the start of the energy management + task. The cache uses an LRU eviction strategy, storing up to 100 items, with the oldest items being evicted once the cache reaches its capacity. @@ -75,14 +83,14 @@ class CacheUntilUpdateStore(SingletonMixin): miss_count: ClassVar[int] = 0 def __init__(self, *args: Any, **kwargs: Any) -> None: - """Initializes the `CacheUntilUpdateStore` instance with default parameters. + """Initializes the `CacheEnergyManagementStore` instance with default parameters. The cache uses an LRU eviction strategy with a maximum size of 100 items. This cache is a singleton, meaning only one instance will exist throughout the application lifecycle. Example: - >>> cache = CacheUntilUpdateStore() + >>> cache = CacheEnergyManagementStore() """ if hasattr(self, "_initialized"): return @@ -128,7 +136,7 @@ class CacheUntilUpdateStore(SingletonMixin): Example: >>> value = cache["user_data"] """ - return CacheUntilUpdateStore.cache[key] + return CacheEnergyManagementStore.cache[key] def __setitem__(self, key: Any, value: Any) -> None: """Stores an item in the cache. @@ -140,15 +148,15 @@ class CacheUntilUpdateStore(SingletonMixin): Example: >>> cache["user_data"] = {"name": "Alice", "age": 30} """ - CacheUntilUpdateStore.cache[key] = value + CacheEnergyManagementStore.cache[key] = value def __len__(self) -> int: """Returns the number of items in the cache.""" - return len(CacheUntilUpdateStore.cache) + return len(CacheEnergyManagementStore.cache) def __repr__(self) -> str: - """Provides a string representation of the CacheUntilUpdateStore object.""" - return repr(CacheUntilUpdateStore.cache) + """Provides a string representation of the CacheEnergyManagementStore object.""" + return repr(CacheEnergyManagementStore.cache) def clear(self) -> None: """Clears the cache, removing all stored items. @@ -161,22 +169,22 @@ class CacheUntilUpdateStore(SingletonMixin): >>> cache.clear() """ if hasattr(self.cache, "clear") and callable(getattr(self.cache, "clear")): - CacheUntilUpdateStore.cache.clear() - CacheUntilUpdateStore.last_event = None - CacheUntilUpdateStore.last_key = None - CacheUntilUpdateStore.last_value = None - CacheUntilUpdateStore.miss_count = 0 - CacheUntilUpdateStore.hit_count = 0 + CacheEnergyManagementStore.cache.clear() + CacheEnergyManagementStore.last_event = None + CacheEnergyManagementStore.last_key = None + CacheEnergyManagementStore.last_value = None + CacheEnergyManagementStore.miss_count = 0 + CacheEnergyManagementStore.hit_count = 0 else: raise AttributeError(f"'{self.cache.__class__.__name__}' object has no method 'clear'") -def cachemethod_until_update(method: TCallable) -> TCallable: +def cachemethod_energy_management(method: TCallable) -> TCallable: """Decorator for in memory caching the result of an instance method. - This decorator caches the method's result in `CacheUntilUpdateStore`, ensuring + This decorator caches the method's result in `CacheEnergyManagementStore`, ensuring that subsequent calls with the same arguments return the cached result until the - next EMS update cycle. + next energy management start. Args: method (Callable): The instance method to be decorated. @@ -186,14 +194,14 @@ def cachemethod_until_update(method: TCallable) -> TCallable: Example: >>> class MyClass: - >>> @cachemethod_until_update + >>> @cachemethod_energy_management >>> def expensive_method(self, param: str) -> str: >>> # Perform expensive computation >>> return f"Computed {param}" """ @cachebox.cachedmethod( - cache=CacheUntilUpdateStore().cache, callback=cache_until_update_store_callback + cache=CacheEnergyManagementStore().cache, callback=cache_energy_management_store_callback ) @functools.wraps(method) def wrapper(self: Any, *args: Any, **kwargs: Any) -> Any: @@ -203,12 +211,12 @@ def cachemethod_until_update(method: TCallable) -> TCallable: return wrapper -def cache_until_update(func: TCallable) -> TCallable: +def cache_energy_management(func: TCallable) -> TCallable: """Decorator for in memory caching the result of a standalone function. - This decorator caches the function's result in `CacheUntilUpdateStore`, ensuring + This decorator caches the function's result in `CacheEnergyManagementStore`, ensuring that subsequent calls with the same arguments return the cached result until the - next EMS update cycle. + next energy management start. Args: func (Callable): The function to be decorated. @@ -224,7 +232,7 @@ def cache_until_update(func: TCallable) -> TCallable: """ @cachebox.cached( - cache=CacheUntilUpdateStore().cache, callback=cache_until_update_store_callback + cache=CacheEnergyManagementStore().cache, callback=cache_energy_management_store_callback ) @functools.wraps(func) def wrapper(*args: Any, **kwargs: Any) -> Any: @@ -435,7 +443,7 @@ class CacheFileStore(ConfigMixin, SingletonMixin): ) logger.debug( - f"Search: ttl:{ttl_duration}, until:{until_datetime}, at:{at_datetime}, before:{before_datetime} -> hit: {generated_key == cache_file_key}, item: {cache_item.cache_file.seek(0), cache_item.cache_file.read()}" + f"Search: ttl:{ttl_duration}, until:{until_datetime}, at:{at_datetime}, before:{before_datetime} -> hit: {generated_key == cache_file_key}, item: {cache_item.cache_file.seek(0), cache_item.cache_file.read()[:10]}..." ) if generated_key == cache_file_key: diff --git a/src/akkudoktoreos/core/coreabc.py b/src/akkudoktoreos/core/coreabc.py index 2e90799..c9e4b61 100644 --- a/src/akkudoktoreos/core/coreabc.py +++ b/src/akkudoktoreos/core/coreabc.py @@ -14,13 +14,13 @@ import threading from typing import Any, ClassVar, Dict, Optional, Type from loguru import logger -from pendulum import DateTime -from pydantic import computed_field + +from akkudoktoreos.core.decorators import classproperty +from akkudoktoreos.utils.datetimeutil import DateTime config_eos: Any = None measurement_eos: Any = None prediction_eos: Any = None -devices_eos: Any = None ems_eos: Any = None @@ -46,9 +46,9 @@ class ConfigMixin: ``` """ - @property - def config(self) -> Any: - """Convenience method/ attribute to retrieve the EOS configuration data. + @classproperty + def config(cls) -> Any: + """Convenience class method/ attribute to retrieve the EOS configuration data. Returns: ConfigEOS: The configuration. @@ -86,9 +86,9 @@ class MeasurementMixin: ``` """ - @property - def measurement(self) -> Any: - """Convenience method/ attribute to retrieve the EOS measurement data. + @classproperty + def measurement(cls) -> Any: + """Convenience class method/ attribute to retrieve the EOS measurement data. Returns: Measurement: The measurement. @@ -126,9 +126,9 @@ class PredictionMixin: ``` """ - @property - def prediction(self) -> Any: - """Convenience method/ attribute to retrieve the EOS prediction data. + @classproperty + def prediction(cls) -> Any: + """Convenience class method/ attribute to retrieve the EOS prediction data. Returns: Prediction: The prediction. @@ -143,46 +143,6 @@ class PredictionMixin: return prediction_eos -class DevicesMixin: - """Mixin class for managing EOS devices simulation data. - - This class serves as a foundational component for EOS-related classes requiring access - to global devices simulation data. It provides a `devices` property that dynamically retrieves - the devices instance, ensuring up-to-date access to devices simulation results. - - Usage: - Subclass this base class to gain access to the `devices` attribute, which retrieves the - global devices instance lazily to avoid import-time circular dependencies. - - Attributes: - devices (Devices): Property to access the global EOS devices simulation data. - - Example: - ```python - class MyOptimizationClass(DevicesMixin): - def analyze_mydevicesimulation(self): - device_simulation_data = self.devices.mydevicesresult - # Perform analysis - ``` - """ - - @property - def devices(self) -> Any: - """Convenience method/ attribute to retrieve the EOS devices simulation data. - - Returns: - Devices: The devices simulation. - """ - # avoid circular dependency at import time - global devices_eos - if devices_eos is None: - from akkudoktoreos.devices.devices import get_devices - - devices_eos = get_devices() - - return devices_eos - - class EnergyManagementSystemMixin: """Mixin class for managing EOS energy management system. @@ -207,9 +167,9 @@ class EnergyManagementSystemMixin: ``` """ - @property - def ems(self) -> Any: - """Convenience method/ attribute to retrieve the EOS energy management system. + @classproperty + def ems(cls) -> Any: + """Convenience class method/ attribute to retrieve the EOS energy management system. Returns: EnergyManagementSystem: The energy management system. @@ -231,16 +191,21 @@ class StartMixin(EnergyManagementSystemMixin): - `start_datetime`: The starting datetime of the current or latest energy management. """ - # Computed field for start_datetime - @computed_field # type: ignore[prop-decorator] - @property - def start_datetime(self) -> Optional[DateTime]: - """Returns the start datetime of the current or latest energy management. + @classproperty + def ems_start_datetime(cls) -> Optional[DateTime]: + """Convenience class method/ attribute to retrieve the start datetime of the current or latest energy management. Returns: DateTime: The starting datetime of the current or latest energy management, or None. """ - return self.ems.start_datetime + # avoid circular dependency at import time + global ems_eos + if ems_eos is None: + from akkudoktoreos.core.ems import get_ems + + ems_eos = get_ems() + + return ems_eos.start_datetime class SingletonMixin: diff --git a/src/akkudoktoreos/core/dataabc.py b/src/akkudoktoreos/core/dataabc.py index c82c68b..7cd5245 100644 --- a/src/akkudoktoreos/core/dataabc.py +++ b/src/akkudoktoreos/core/dataabc.py @@ -14,14 +14,23 @@ from abc import abstractmethod from collections.abc import MutableMapping, MutableSequence from itertools import chain from pathlib import Path -from typing import Any, Dict, Iterator, List, Optional, Tuple, Type, Union, overload +from typing import ( + Any, + Dict, + Iterator, + List, + Optional, + Tuple, + Type, + Union, + overload, +) import numpy as np import pandas as pd import pendulum from loguru import logger from numpydantic import NDArray, Shape -from pendulum import DateTime, Duration from pydantic import ( AwareDatetime, ConfigDict, @@ -29,6 +38,7 @@ from pydantic import ( ValidationError, computed_field, field_validator, + model_validator, ) from akkudoktoreos.core.coreabc import ConfigMixin, SingletonMixin, StartMixin @@ -37,7 +47,13 @@ from akkudoktoreos.core.pydantic import ( PydanticDateTimeData, PydanticDateTimeDataFrame, ) -from akkudoktoreos.utils.datetimeutil import compare_datetimes, to_datetime, to_duration +from akkudoktoreos.utils.datetimeutil import ( + DateTime, + Duration, + compare_datetimes, + to_datetime, + to_duration, +) class DataBase(ConfigMixin, StartMixin, PydanticBaseModel): @@ -55,6 +71,11 @@ class DataRecord(DataBase, MutableMapping): Fields can be accessed and mutated both using dictionary-style access (`record['field_name']`) and attribute-style access (`record.field_name`). + The data record also provides configured field like data. Configuration has to be done by the + derived class. Configuration is a list of key strings, which is usually taken from the EOS + configuration. The internal field for these data `configured_data` is mostly hidden from + dictionary-style and attribute-style access. + Attributes: date_time (Optional[DateTime]): Aware datetime indicating when the data record applies. @@ -65,9 +86,42 @@ class DataRecord(DataBase, MutableMapping): date_time: Optional[DateTime] = Field(default=None, description="DateTime") + configured_data: dict[str, Any] = Field( + default_factory=dict, + description="Configured field like data", + examples=[{"load0_mr": 40421}], + ) + # Pydantic v2 model configuration model_config = ConfigDict(arbitrary_types_allowed=True, populate_by_name=True) + @model_validator(mode="before") + @classmethod + def init_configured_field_like_data(cls, data: Any) -> Any: + """Extracts configured data keys from the input and assigns them to `configured_data`. + + This validator is called before the model is initialized. It filters out any keys from the input + dictionary that are listed in the configured data keys, and moves them into + the `configured_data` field of the model. This enables flexible, key-driven population of + dynamic data while keeping the model schema clean. + + Args: + data (Any): The raw input data used to initialize the model. + + Returns: + Any: The modified input data dictionary, with configured keys moved to `configured_data`. + """ + if not isinstance(data, dict): + return data + + configured_keys: Union[list[str], set] = cls.configured_data_keys() or set() + extracted = {k: data.pop(k) for k in list(data.keys()) if k in configured_keys} + + if extracted: + data.setdefault("configured_data", {}).update(extracted) + + return data + @field_validator("date_time", mode="before") @classmethod def transform_to_datetime(cls, value: Any) -> Optional[DateTime]: @@ -77,18 +131,39 @@ class DataRecord(DataBase, MutableMapping): return None return to_datetime(value) + @classmethod + def configured_data_keys(cls) -> Optional[list[str]]: + """Return the keys for the configured field like data. + + Can be overwritten by derived classes to define specific field like data. Usually provided + by configuration data. + """ + return None + @classmethod def record_keys(cls) -> List[str]: """Returns the keys of all fields in the data record.""" key_list = [] key_list.extend(list(cls.model_fields.keys())) key_list.extend(list(cls.__pydantic_decorators__.computed_fields.keys())) + # Add also keys that may be added by configuration + key_list.remove("configured_data") + configured_keys = cls.configured_data_keys() + if configured_keys is not None: + key_list.extend(configured_keys) return key_list @classmethod def record_keys_writable(cls) -> List[str]: """Returns the keys of all fields in the data record that are writable.""" - return list(cls.model_fields.keys()) + keys_writable = [] + keys_writable.extend(list(cls.model_fields.keys())) + # Add also keys that may be added by configuration + keys_writable.remove("configured_data") + configured_keys = cls.configured_data_keys() + if configured_keys is not None: + keys_writable.extend(configured_keys) + return keys_writable def _validate_key_writable(self, key: str) -> None: """Verify that a specified key exists and is writable in the current record keys. @@ -104,6 +179,40 @@ class DataRecord(DataBase, MutableMapping): f"Key '{key}' is not in writable record keys: {self.record_keys_writable()}" ) + def __dir__(self) -> list[str]: + """Extend the default `dir()` output to include configured field like data keys. + + This enables editor auto-completion and interactive introspection, while hiding the internal + `configured_data` dictionary. + + This ensures the configured field like data values appear like native fields, + in line with the base model's attribute behavior. + """ + base = super().__dir__() + keys = set(base) + # Expose configured data keys as attributes + configured_keys = self.configured_data_keys() + if configured_keys is not None: + keys.update(configured_keys) + # Explicitly hide the 'configured_data' internal dict + keys.discard("configured_data") + return sorted(keys) + + def __eq__(self, other: Any) -> bool: + """Ensure equality comparison includes the contents of the `configured_data` dict. + + Contents of the `configured_data` dict are in addition to the base model fields. + """ + if not isinstance(other, self.__class__): + return NotImplemented + # Compare all fields except `configured_data` + if self.model_dump(exclude={"configured_data"}) != other.model_dump( + exclude={"configured_data"} + ): + return False + # Compare `configured_data` explicitly + return self.configured_data == other.configured_data + def __getitem__(self, key: str) -> Any: """Retrieve the value of a field by key name. @@ -116,9 +225,11 @@ class DataRecord(DataBase, MutableMapping): Raises: KeyError: If the specified key does not exist. """ - if key in self.model_fields: - return getattr(self, key) - raise KeyError(f"'{key}' not found in the record fields.") + try: + # Let getattr do the work + return self.__getattr__(key) + except: + raise KeyError(f"'{key}' not found in the record fields.") def __setitem__(self, key: str, value: Any) -> None: """Set the value of a field by key name. @@ -130,9 +241,10 @@ class DataRecord(DataBase, MutableMapping): Raises: KeyError: If the specified key does not exist in the fields. """ - if key in self.model_fields: - setattr(self, key, value) - else: + try: + # Let setattr do the work + self.__setattr__(key, value) + except: raise KeyError(f"'{key}' is not a recognized field.") def __delitem__(self, key: str) -> None: @@ -144,9 +256,9 @@ class DataRecord(DataBase, MutableMapping): Raises: KeyError: If the specified key does not exist in the fields. """ - if key in self.model_fields: - setattr(self, key, None) # Optional: set to None instead of deleting - else: + try: + self.__delattr__(key) + except: raise KeyError(f"'{key}' is not a recognized field.") def __iter__(self) -> Iterator[str]: @@ -155,7 +267,7 @@ class DataRecord(DataBase, MutableMapping): Returns: Iterator[str]: An iterator over field names. """ - return iter(self.model_fields) + return iter(self.record_keys_writable()) def __len__(self) -> int: """Return the number of fields in the data record. @@ -163,7 +275,7 @@ class DataRecord(DataBase, MutableMapping): Returns: int: The number of defined fields. """ - return len(self.model_fields) + return len(self.record_keys_writable()) def __repr__(self) -> str: """Provide a string representation of the data record. @@ -171,7 +283,7 @@ class DataRecord(DataBase, MutableMapping): Returns: str: A string representation showing field names and their values. """ - field_values = {field: getattr(self, field) for field in self.model_fields} + field_values = {field: getattr(self, field) for field in self.__class__.model_fields} return f"{self.__class__.__name__}({field_values})" def __getattr__(self, key: str) -> Any: @@ -186,8 +298,13 @@ class DataRecord(DataBase, MutableMapping): Raises: AttributeError: If the field does not exist. """ - if key in self.model_fields: + if key in self.__class__.model_fields: return getattr(self, key) + if key in self.configured_data.keys(): + return self.configured_data[key] + configured_keys = self.configured_data_keys() + if configured_keys is not None and key in configured_keys: + return None raise AttributeError(f"'{self.__class__.__name__}' object has no attribute '{key}'") def __setattr__(self, key: str, value: Any) -> None: @@ -200,10 +317,14 @@ class DataRecord(DataBase, MutableMapping): Raises: AttributeError: If the attribute/field does not exist. """ - if key in self.model_fields: + if key in self.__class__.model_fields: super().__setattr__(key, value) - else: - raise AttributeError(f"'{self.__class__.__name__}' object has no attribute '{key}'") + return + configured_keys = self.configured_data_keys() + if configured_keys is not None and key in configured_keys: + self.configured_data[key] = value + return + raise AttributeError(f"'{self.__class__.__name__}' object has no attribute '{key}'") def __delattr__(self, key: str) -> None: """Delete an attribute by setting it to None if it exists as a field. @@ -214,10 +335,21 @@ class DataRecord(DataBase, MutableMapping): Raises: AttributeError: If the attribute/field does not exist. """ - if key in self.model_fields: - setattr(self, key, None) # Optional: set to None instead of deleting - else: - super().__delattr__(key) + if key in self.__class__.model_fields: + data: Optional[dict] + if key == "configured_data": + data = dict() + else: + data = None + setattr(self, key, data) + return + if key in self.configured_data: + del self.configured_data[key] + return + configured_keys = self.configured_data_keys() + if configured_keys is not None and key in configured_keys: + return + super().__delattr__(key) @classmethod def key_from_description(cls, description: str, threshold: float = 0.8) -> Optional[str]: @@ -352,10 +484,7 @@ class DataSequence(DataBase, MutableSequence): @property def record_keys(self) -> List[str]: """Returns the keys of all fields in the data records.""" - key_list = [] - key_list.extend(list(self.record_class().model_fields.keys())) - key_list.extend(list(self.record_class().__pydantic_decorators__.computed_fields.keys())) - return key_list + return self.record_class().record_keys() @computed_field # type: ignore[prop-decorator] @property @@ -369,7 +498,7 @@ class DataSequence(DataBase, MutableSequence): Returns: List[str]: A list of field keys that are writable in the data records. """ - return list(self.record_class().model_fields.keys()) + return self.record_class().record_keys_writable() @classmethod def record_class(cls) -> Type: @@ -707,6 +836,38 @@ class DataSequence(DataBase, MutableSequence): return filtered_data + def key_to_value(self, key: str, target_datetime: DateTime) -> Optional[float]: + """Returns the value corresponding to the specified key that is nearest to the given datetime. + + Args: + key (str): The key of the attribute in DataRecord to extract. + target_datetime (datetime): The datetime to search nearest to. + + Returns: + Optional[float]: The value nearest to the given datetime, or None if no valid records are found. + + Raises: + KeyError: If the specified key is not found in any of the DataRecords. + """ + self._validate_key(key) + + # Filter out records with None or NaN values for the key + valid_records = [ + record + for record in self.records + if record.date_time is not None + and getattr(record, key, None) not in (None, float("nan")) + ] + + if not valid_records: + return None + + # Find the record with datetime nearest to target_datetime + target = to_datetime(target_datetime) + nearest_record = min(valid_records, key=lambda r: abs(r.date_time - target)) + + return getattr(nearest_record, key, None) + def key_to_lists( self, key: str, @@ -868,6 +1029,11 @@ class DataSequence(DataBase, MutableSequence): KeyError: If the specified key is not found in any of the DataRecords. """ self._validate_key(key) + + # General check on fill_method + if fill_method not in ("ffill", "bfill", "linear", "none", None): + raise ValueError(f"Unsupported fill method: {fill_method}") + # Ensure datetime objects are normalized start_datetime = to_datetime(start_datetime, to_maxtime=False) if start_datetime else None end_datetime = to_datetime(end_datetime, to_maxtime=False) if end_datetime else None @@ -880,7 +1046,7 @@ class DataSequence(DataBase, MutableSequence): values_len = len(values) if values_len < 1: - # No values, assume at at least one value set to None + # No values, assume at least one value set to None if start_datetime is not None: dates.append(start_datetime - interval) else: @@ -902,6 +1068,11 @@ class DataSequence(DataBase, MutableSequence): # Truncate all values before latest value before start_datetime dates = dates[start_index - 1 :] values = values[start_index - 1 :] + # We have a start_datetime, align to start datetime + resample_origin = start_datetime + else: + # We do not have a start_datetime, align resample buckets to midnight of first day + resample_origin = "start_day" if end_datetime is not None: if compare_datetimes(dates[-1], end_datetime).lt: @@ -922,7 +1093,7 @@ class DataSequence(DataBase, MutableSequence): if fill_method is None: fill_method = "linear" # Resample the series to the specified interval - resampled = series.resample(interval, origin="start").first() + resampled = series.resample(interval, origin=resample_origin).first() if fill_method == "linear": resampled = resampled.interpolate(method="linear") elif fill_method == "ffill": @@ -936,7 +1107,7 @@ class DataSequence(DataBase, MutableSequence): if fill_method is None: fill_method = "ffill" # Resample the series to the specified interval - resampled = series.resample(interval, origin="start").first() + resampled = series.resample(interval, origin=resample_origin).first() if fill_method == "ffill": resampled = resampled.ffill() elif fill_method == "bfill": @@ -944,12 +1115,24 @@ class DataSequence(DataBase, MutableSequence): elif fill_method != "none": raise ValueError(f"Unsupported fill method for non-numeric data: {fill_method}") + logger.debug( + "Resampled for '{}' with length {}: {}...{}", + key, + len(resampled), + resampled[:10], + resampled[-10:], + ) + # Convert the resampled series to a NumPy array if start_datetime is not None and len(resampled) > 0: resampled = resampled.truncate(before=start_datetime) if end_datetime is not None and len(resampled) > 0: resampled = resampled.truncate(after=end_datetime.subtract(seconds=1)) array = resampled.values + logger.debug( + "Array for '{}' with length {}: {}...{}", key, len(array), array[:10], array[-10:] + ) + return array def to_dataframe( @@ -1197,7 +1380,7 @@ class DataImportMixin: the values. `ìnterval` may be used to define the fixed time interval between two values. On import `self.update_value(datetime, key, value)` is called which has to be provided. - Also `self.start_datetime` may be necessary as a default in case `start_datetime`is not given. + Also `self.ems_start_datetime` may be necessary as a default in case `start_datetime`is not given. """ # Attributes required but defined elsehere. @@ -1315,7 +1498,7 @@ class DataImportMixin: raise ValueError(f"Invalid start_datetime in import data: {e}") if start_datetime is None: - start_datetime = self.start_datetime # type: ignore + start_datetime = self.ems_start_datetime # type: ignore if "interval" in import_data: try: @@ -1406,7 +1589,7 @@ class DataImportMixin: raise ValueError(f"Invalid datetime index in DataFrame: {e}") else: if start_datetime is None: - start_datetime = self.start_datetime # type: ignore + start_datetime = self.ems_start_datetime # type: ignore has_datetime_index = False # Filter columns based on key_prefix and record_keys_writable @@ -1463,7 +1646,7 @@ class DataImportMixin: If start_datetime and or interval is given in the JSON dict it will be used. Otherwise the given parameters are used. If None is given start_datetime defaults to - 'self.start_datetime' and interval defaults to 1 hour. + 'self.ems_start_datetime' and interval defaults to 1 hour. Args: json_str (str): The JSON string containing the generic data. @@ -1538,7 +1721,7 @@ class DataImportMixin: If start_datetime and or interval is given in the JSON dict it will be used. Otherwise the given parameters are used. If None is given start_datetime defaults to - 'self.start_datetime' and interval defaults to 1 hour. + 'self.ems_start_datetime' and interval defaults to 1 hour. Args: import_file_path (Path): The path to the JSON file containing the generic data. @@ -1749,7 +1932,12 @@ class DataContainer(SingletonMixin, DataBase, MutableMapping): force_update (bool, optional): If True, forces the providers to update the data even if still cached. """ for provider in self.providers: - provider.update_data(force_enable=force_enable, force_update=force_update) + try: + provider.update_data(force_enable=force_enable, force_update=force_update) + except Exception as ex: + error = f"Provider {provider.provider_id()} fails on update - enabled={provider.enabled()}, force_enable={force_enable}, force_update={force_update}: {ex}" + logger.error(error) + raise RuntimeError(error) def key_to_series( self, @@ -1854,7 +2042,7 @@ class DataContainer(SingletonMixin, DataBase, MutableMapping): ) -> pd.DataFrame: """Retrieve a dataframe indexed by fixed time intervals for specified keys from the data in each DataProvider. - Generates a pandas DataFrame using the NumPy arrays for each specified key, ensuring a common time index.. + Generates a pandas DataFrame using the NumPy arrays for each specified key, ensuring a common time index. Args: keys (list[str]): A list of field names to retrieve. @@ -1903,8 +2091,15 @@ class DataContainer(SingletonMixin, DataBase, MutableMapping): end_datetime.add(seconds=1) # Create a DatetimeIndex based on start, end, and interval + if start_datetime is None or end_datetime is None: + raise ValueError( + f"Can not determine datetime range. Got '{start_datetime}'..'{end_datetime}'." + ) reference_index = pd.date_range( - start=start_datetime, end=end_datetime, freq=interval, inclusive="left" + start=start_datetime, + end=end_datetime, + freq=interval, + inclusive="left", ) data = {} diff --git a/src/akkudoktoreos/core/emplan.py b/src/akkudoktoreos/core/emplan.py new file mode 100644 index 0000000..49772ce --- /dev/null +++ b/src/akkudoktoreos/core/emplan.py @@ -0,0 +1,1914 @@ +"""Energy management plan. + +The energy management plan is leaned on to the S2 standard. + +This module provides data models and enums for energy resource management +following the S2 standard, supporting various control types including Power Envelope Based Control, +Power Profile Based Control, Operation Mode Based Control, Fill Rate Based Control, and +Demand Driven Based Control. +""" + +import uuid +from abc import ABC, abstractmethod +from enum import Enum +from typing import Annotated, Literal, Optional, Union + +from pydantic import Field, computed_field, model_validator + +from akkudoktoreos.core.pydantic import PydanticBaseModel +from akkudoktoreos.utils.datetimeutil import DateTime, Duration, to_datetime + +# S2 Basic Data Types +# - Array -> list +# - Boolean -> bool +# - DateTimeStamp -> DateTime +# - Duration -> Duration +# - ID -> alias on str +# - Number -> float +# - String -> str + +ID = str + + +# S2 Enumerations + + +class RoleType(str, Enum): + """Enumeration of energy resource roles in the system.""" + + ENERGY_PRODUCER = "ENERGY_PRODUCER" + ENERGY_CONSUMER = "ENERGY_CONSUMER" + ENERGY_STORAGE = "ENERGY_STORAGE" + + +class Commodity(str, Enum): + """Enumeration of energy commodities supported in the system.""" + + GAS = "GAS" + HEAT = "HEAT" + ELECTRICITY = "ELECTRICITY" + OIL = "OIL" + + +class CommodityQuantity(str, Enum): + """Enumeration of specific commodity quantities and measurement types.""" + + ELECTRIC_POWER_L1 = "ELECTRIC.POWER.L1" + """Electric power in Watt on phase 1. If a device utilizes only one phase, it should always use L1.""" + + ELECTRIC_POWER_L2 = "ELECTRIC.POWER.L2" + """Electric power in Watt on phase 2. Only applicable for 3-phase devices.""" + + ELECTRIC_POWER_L3 = "ELECTRIC.POWER.L3" + """Electric power in Watt on phase 3. Only applicable for 3-phase devices.""" + + ELECTRIC_POWER_3_PHASE_SYM = "ELECTRIC.POWER.3_PHASE_SYM" + """Electric power in Watt when power is equally shared among the three phases. Only applicable for 3-phase devices.""" + + NATURAL_GAS_FLOW_RATE = "NATURAL_GAS.FLOW_RATE" + """Gas flow rate described in liters per second.""" + + HYDROGEN_FLOW_RATE = "HYDROGEN.FLOW_RATE" + """Hydrogen flow rate described in grams per second.""" + + HEAT_TEMPERATURE = "HEAT.TEMPERATURE" + """Heat temperature described in degrees Celsius.""" + + HEAT_FLOW_RATE = "HEAT.FLOW_RATE" + """Flow rate of heat-carrying gas or liquid in liters per second.""" + + HEAT_THERMAL_POWER = "HEAT.THERMAL_POWER" + """Thermal power in Watt.""" + + OIL_FLOW_RATE = "OIL.FLOW_RATE" + """Oil flow rate described in liters per hour.""" + + CURRENCY = "CURRENCY" + """Currency-related quantity.""" + + +class Currency(str, Enum): + """Enumeration of currency codes following ISO 4217 standard.""" + + AED = "AED" + AFN = "AFN" + ALL = "ALL" + AMD = "AMD" + ANG = "ANG" + AOA = "AOA" + ARS = "ARS" + AUD = "AUD" + AWG = "AWG" + AZN = "AZN" + BAM = "BAM" + BBD = "BBD" + BDT = "BDT" + BGN = "BGN" + BHD = "BHD" + BIF = "BIF" + BMD = "BMD" + BND = "BND" + BOB = "BOB" + BRL = "BRL" + BSD = "BSD" + BTN = "BTN" + BWP = "BWP" + BYN = "BYN" + BZD = "BZD" + CAD = "CAD" + CDF = "CDF" + CHF = "CHF" + CLP = "CLP" + CNY = "CNY" + COP = "COP" + CRC = "CRC" + CUP = "CUP" + CVE = "CVE" + CZK = "CZK" + DJF = "DJF" + DKK = "DKK" + DOP = "DOP" + DZD = "DZD" + EGP = "EGP" + ERN = "ERN" + ETB = "ETB" + EUR = "EUR" + FJD = "FJD" + FKP = "FKP" + FOK = "FOK" + GBP = "GBP" + GEL = "GEL" + GGP = "GGP" + GHS = "GHS" + GIP = "GIP" + GMD = "GMD" + GNF = "GNF" + GTQ = "GTQ" + GYD = "GYD" + HKD = "HKD" + HNL = "HNL" + HRK = "HRK" + HTG = "HTG" + HUF = "HUF" + IDR = "IDR" + ILS = "ILS" + IMP = "IMP" + INR = "INR" + IQD = "IQD" + IRR = "IRR" + ISK = "ISK" + JEP = "JEP" + JMD = "JMD" + JOD = "JOD" + JPY = "JPY" + KES = "KES" + KGS = "KGS" + KHR = "KHR" + KID = "KID" + KMF = "KMF" + KRW = "KRW" + KWD = "KWD" + KYD = "KYD" + KZT = "KZT" + LAK = "LAK" + LBP = "LBP" + LKR = "LKR" + LRD = "LRD" + LSL = "LSL" + LYD = "LYD" + MAD = "MAD" + MDL = "MDL" + MGA = "MGA" + MKD = "MKD" + MMK = "MMK" + MNT = "MNT" + MOP = "MOP" + MRU = "MRU" + MUR = "MUR" + MVR = "MVR" + MWK = "MWK" + MXN = "MXN" + MYR = "MYR" + MZN = "MZN" + NAD = "NAD" + NGN = "NGN" + NIO = "NIO" + NOK = "NOK" + NPR = "NPR" + NZD = "NZD" + OMR = "OMR" + PAB = "PAB" + PEN = "PEN" + PGK = "PGK" + PHP = "PHP" + PKR = "PKR" + PLN = "PLN" + PYG = "PYG" + QAR = "QAR" + RON = "RON" + RSD = "RSD" + RUB = "RUB" + RWF = "RWF" + SAR = "SAR" + SBD = "SBD" + SCR = "SCR" + SDG = "SDG" + SEK = "SEK" + SGD = "SGD" + SHP = "SHP" + SLE = "SLE" + SLL = "SLL" + SOS = "SOS" + SRD = "SRD" + SSP = "SSP" + STN = "STN" + SYP = "SYP" + SZL = "SZL" + THB = "THB" + TJS = "TJS" + TMT = "TMT" + TND = "TND" + TOP = "TOP" + TRY = "TRY" + TTD = "TTD" + TVD = "TVD" + TWD = "TWD" + TZS = "TZS" + UAH = "UAH" + UGX = "UGX" + USD = "USD" + UYU = "UYU" + UZS = "UZS" + VES = "VES" + VND = "VND" + VUV = "VUV" + WST = "WST" + XAF = "XAF" + XCD = "XCD" + XOF = "XOF" + XPF = "XPF" + YER = "YER" + ZAR = "ZAR" + ZMW = "ZMW" + ZWL = "ZWL" + + +class InstructionStatus(str, Enum): + """Enumeration of possible instruction status values.""" + + NEW = "NEW" # Instruction was newly created + ACCEPTED = "ACCEPTED" # Instruction has been accepted + REJECTED = "REJECTED" # Instruction was rejected + REVOKED = "REVOKED" # Instruction was revoked + STARTED = "STARTED" # Instruction was executed + SUCCEEDED = "SUCCEEDED" # Instruction finished successfully + ABORTED = "ABORTED" # Instruction was aborted + + +class ControlType(str, Enum): + """Enumeration of different control types supported by the system.""" + + POWER_ENVELOPE_BASED_CONTROL = ( + "POWER_ENVELOPE_BASED_CONTROL" # Identifier for the Power Envelope Based Control type + ) + POWER_PROFILE_BASED_CONTROL = ( + "POWER_PROFILE_BASED_CONTROL" # Identifier for the Power Profile Based Control type + ) + OPERATION_MODE_BASED_CONTROL = ( + "OPERATION_MODE_BASED_CONTROL" # Identifier for the Operation Mode Based Control type + ) + FILL_RATE_BASED_CONTROL = ( + "FILL_RATE_BASED_CONTROL" # Identifier for the Fill Rate Based Control type + ) + DEMAND_DRIVEN_BASED_CONTROL = ( + "DEMAND_DRIVEN_BASED_CONTROL" # Identifier for the Demand Driven Based Control type + ) + NOT_CONTROLABLE = "NOT_CONTROLABLE" # Used if no control is possible; resource can still provide forecasts and measurements + NO_SELECTION = "NO_SELECTION" # Used if no control type is/has been selected + + +class PEBCPowerEnvelopeLimitType(str, Enum): + """Enumeration of power envelope limit types for Power Envelope Based Control.""" + + UPPER_LIMIT = "UPPER_LIMIT" # Indicates the upper limit of a Power Envelope + LOWER_LIMIT = "LOWER_LIMIT" # Indicates the lower limit of a Power Envelope + + +class PEBCPowerEnvelopeConsequenceType(str, Enum): + """Enumeration of consequences when power is limited for Power Envelope Based Control.""" + + VANISH = "VANISH" # Limited load or generation will be lost and not reappear + DEFER = "DEFER" # Limited load or generation will be postponed to a later moment + + +class ReceptionStatusValues(str, Enum): + """Enumeration of status values for data reception.""" + + SUCCEEDED = "SUCCEEDED" # Data received, complete, and consistent + REJECTED = "REJECTED" # Data could not be parsed or was incomplete/inconsistent + + +class PPBCPowerSequenceStatus(str, Enum): + """Enumeration of status values for Power Profile Based Control sequences.""" + + NOT_SCHEDULED = "NOT_SCHEDULED" # No PowerSequence is scheduled + SCHEDULED = "SCHEDULED" # PowerSequence is scheduled for future execution + EXECUTING = "EXECUTING" # PowerSequence is currently being executed + INTERRUPTED = "INTERRUPTED" # Execution is currently interrupted and will continue later + FINISHED = "FINISHED" # PowerSequence finished successfully + ABORTED = "ABORTED" # PowerSequence was aborted and will not continue + + +# S2 Basic Values + + +class PowerValue(PydanticBaseModel): + """Represents a specific power value measurement with its associated commodity quantity. + + This class links a numerical power value to a specific type of power quantity (such as + active power, reactive power, etc.) and its unit of measurement. + """ + + commodity_quantity: CommodityQuantity = Field( + ..., description="The power quantity the value refers to." + ) + value: float = Field( + ..., description="Power value expressed in the unit associated with the CommodityQuantity." + ) + + +class PowerForecastValue(PydanticBaseModel): + """Represents a forecasted power value with statistical confidence intervals. + + This model provides a complete statistical representation of a power forecast, + including the expected value and multiple confidence intervals (68%, 95%, and absolute limits). + Each forecast is associated with a specific commodity quantity. + """ + + value_upper_limit: Optional[float] = Field( + None, + description="The upper boundary of the range with 100% certainty the power value is in it.", + ) + value_upper_95PPR: Optional[float] = Field( + None, + description="The upper boundary of the range with 95% certainty the power value is in it.", + ) + value_upper_68PPR: Optional[float] = Field( + None, + description="The upper boundary of the range with 68% certainty the power value is in it.", + ) + value_expected: float = Field(..., description="The expected power value.") + value_lower_68PPR: Optional[float] = Field( + None, + description="The lower boundary of the range with 68% certainty the power value is in it.", + ) + value_lower_95PPR: Optional[float] = Field( + None, + description="The lower boundary of the range with 95% certainty the power value is in it.", + ) + value_lower_limit: Optional[float] = Field( + None, + description="The lower boundary of the range with 100% certainty the power value is in it.", + ) + commodity_quantity: CommodityQuantity = Field( + ..., description="The power quantity the value refers to." + ) + + +class PowerRange(PydanticBaseModel): + """Defines a range of acceptable power values for a specific commodity quantity. + + This model specifies the minimum and maximum values for a power parameter, + creating operational boundaries for energy systems. This range is used for + defining permissible operating conditions or constraints. + """ + + start_of_range: float = Field( + ..., description="Power value that defines the start of the range." + ) + end_of_range: float = Field(..., description="Power value that defines the end of the range.") + commodity_quantity: CommodityQuantity = Field( + ..., description="The power quantity the values refer to." + ) + + +class NumberRange(PydanticBaseModel): + """Defines a generic numeric range with start and end values. + + Unlike PowerRange, this model is not tied to a specific commodity quantity + and can be used for any numeric range definition throughout the system. + Used for representing ranges of prices, percentages, or other numeric values. + """ + + start_of_range: float = Field(..., description="Number that defines the start of the range.") + end_of_range: float = Field(..., description="Number that defines the end of the range.") + + +class PowerMeasurement(PydanticBaseModel): + """Captures a set of power measurements taken at a specific point in time. + + This model records multiple power values (for different commodity quantities) + along with the timestamp when the measurements were taken, enabling time-series + analysis and monitoring of power consumption or production. + """ + + type: Literal["PowerMeasurement"] = Field(default="PowerMeasurement") + + measurement_timestamp: DateTime = Field( + ..., description="Timestamp when PowerValues were measured." + ) + values: list[PowerValue] = Field( + ..., + description="Array of measured PowerValues. Shall contain at least one item and at most one item per 'commodity_quantity' (defined inside the PowerValue).", + ) + + +class EnergyMeasurement(PydanticBaseModel): + """Captures a set of energy meter readouts taken at a specific point in time. + + Energy is defined as the cummulative power per hour as provided by an energy meter. + + This model records multiple energy values (for different commodity quantities) + along with the timestamp when the meter readouts were taken, enabling time-series + analysis and monitoring of energy consumption or production. + + Note: This is an extension to the S2 standard. + """ + + type: Literal["EnergyMeasurement"] = Field(default="EnergyMeasurement") + + measurement_timestamp: DateTime = Field( + ..., description="Timestamp when energy values were measured." + ) + last_reset: Optional[DateTime] = Field( + default=None, + description="Timestamp when the energy meter's cumulative counter was last reset.", + ) + values: list[PowerValue] = Field( + ..., + description="Array of measured energy values. Shall contain at least one item and at most one item per 'commodity_quantity' (defined inside the PowerValue).", + ) + + +class Role(PydanticBaseModel): + """Defines an energy system role related to a specific commodity. + + This model links a role type (such as consumer, producer, or storage) with + a specific energy commodity (electricity, gas, heat, etc.), defining how + an entity interacts with the energy system for that commodity. + """ + + role: RoleType = Field(..., description="Role type for the given commodity.") + commodity: Commodity = Field(..., description="Commodity the role refers to.") + + +class ReceptionStatus(PydanticBaseModel): + """Represents the status of a data reception operation with optional diagnostic information. + + This model tracks whether data was successfully received, with additional + diagnostic information for debugging purposes. It serves as a feedback mechanism + for communication operations within the system. + """ + + status: ReceptionStatusValues = Field( + ..., description="Enumeration of status values indicating reception outcome." + ) + diagnostic_label: Optional[str] = Field( + None, + description=( + "Optional diagnostic label providing additional information for debugging. " + "Not intended for Human-Machine Interface (HMI) use." + ), + ) + + +class Transition(PydanticBaseModel): + """Defines a permitted transition between operation modes with associated constraints and costs. + + This model represents the rules and constraints governing how a system can move + between different operation modes. It includes information about timing constraints, + costs associated with the transition, expected duration, and whether the transition + is only allowed during abnormal conditions. + """ + + id: ID = Field( + ..., + description=( + "ID of the Transition. Shall be unique in the scope of the OMBC.SystemDescription, " + "FRBC.ActuatorDescription, or DDBC.ActuatorDescription in which it is used." + ), + ) + from_: ID = Field( + ..., + alias="from", + description=( + "ID of the OperationMode that should be switched from. " + "Exact type depends on the ControlType." + ), + ) + to: ID = Field( + ..., + description=( + "ID of the OperationMode that will be switched to. " + "Exact type depends on the ControlType." + ), + ) + start_timers: list[ID] = Field( + ..., + description=( + "List of IDs of Timers that will be (re)started when this Transition is initiated." + ), + ) + blocking_timers: list[ID] = Field( + ..., + description=( + "List of IDs of Timers that block this Transition from initiating " + "while at least one of them is not yet finished." + ), + ) + transition_costs: Optional[float] = Field( + None, + description=( + "Absolute costs for going through this Transition, in the currency defined in ResourceManagerDetails." + ), + ) + transition_duration: Optional[Duration] = Field( + None, + description=( + "Time between initiation of this Transition and when the device behaves according to the target Operation Mode. " + "Assumed negligible if not provided." + ), + ) + abnormal_condition_only: bool = Field( + ..., + description=( + "Indicates whether this Transition may only be used during an abnormal condition." + ), + ) + + model_config = { + "populate_by_name": True, # Enables using 'from_' as 'from' during model population + "extra": "forbid", + } + + +class Timer(PydanticBaseModel): + """Defines a timing constraint for transitions between operation modes. + + This model implements time-based constraints for state transitions in the system, + tracking both the duration of the timer and when it will complete. Timers are used + to enforce minimum dwell times in states, cooldown periods, or other timing-related + operational constraints. + """ + + id: ID = Field( + ..., + description=( + "ID of the Timer. Shall be unique in the scope of the OMBC.SystemDescription, " + "FRBC.ActuatorDescription, or DDBC.ActuatorDescription in which it is used." + ), + ) + diagnostic_label: Optional[str] = Field( + None, + description=( + "Human readable name/description of the Timer. " + "This element is only intended for diagnostic purposes and not for HMI applications." + ), + ) + duration: Duration = Field( + ..., description=("The time it takes for the Timer to finish after it has been started.") + ) + finished_at: DateTime = Field( + ..., + description=( + "Timestamp indicating when the Timer will be finished. " + "If in the future, the timer is not yet finished. " + "If in the past, the timer is finished. " + "If the timer was never started, this can be an arbitrary timestamp in the past." + ), + ) + + +class InstructionStatusUpdate(PydanticBaseModel): + """Represents an update to the status of a control instruction. + + This model tracks the progress and current state of a control instruction, + including when its status last changed. It enables monitoring of instruction + execution and provides feedback about the system's response to control commands. + """ + + instruction_id: ID = Field(..., description=("ID of this instruction, as provided by the CEM.")) + status_type: InstructionStatus = Field(..., description=("Present status of this instruction.")) + timestamp: DateTime = Field(..., description=("Timestamp when the status_type last changed.")) + + +# ResourceManager + + +class ResourceManagerDetails(PydanticBaseModel): + """Provides comprehensive details about a ResourceManager's capabilities and identity. + + This model defines the core characteristics of a ResourceManager, including its + identification, supported energy roles, control capabilities, and technical specifications. + It serves as the primary descriptor for a device or system that can be controlled + by a Customer Energy Manager (CEM). + """ + + resource_id: ID = Field( + ..., + description="Identifier of the ResourceManager. Shall be unique within the scope of the CEM.", + ) + name: Optional[str] = Field(None, description="Human readable name given by user.") + roles: list[Role] = Field( + ..., description="Each ResourceManager provides one or more energy Roles." + ) + manufacturer: Optional[str] = Field(None, description="Name of Manufacturer.") + model: Optional[str] = Field(None, description="Name of the model of the device.") + serial_number: Optional[str] = Field(None, description="Serial number of the device.") + firmware_version: Optional[str] = Field( + None, description="Version identifier of the firmware used in the device." + ) + instruction_processing_delay: Duration = Field( + ..., + description="The average time the system and device needs to process and execute an instruction.", + ) + available_control_types: list[ControlType] = Field( + ..., description="The control types supported by this ResourceManager." + ) + currency: Optional[Currency] = Field( + None, + description="Currency to be used for all information regarding costs. " + "Mandatory if cost information is published.", + ) + provides_forecast: bool = Field( + ..., description="Indicates whether the ResourceManager is able to provide PowerForecasts." + ) + provides_power_measurement_types: list[CommodityQuantity] = Field( + ..., + description="Array of all CommodityQuantities that this ResourceManager can provide measurements for.", + ) + + +# PowerForecast + + +class PowerForecastElement(PydanticBaseModel): + """Represents a segment of a power forecast covering a specific time duration. + + This model defines power forecast values for a specific time period, with multiple + power values potentially covering different commodity quantities. It is used to + construct time-series forecasts of future power production or consumption. + """ + + duration: Duration = Field( + ..., + description=( + "Duration of the PowerForecastElement. " + "Defines the time window the power values apply to." + ), + ) + power_values: list[PowerForecastValue] = Field( + ..., + min_length=1, + description=( + "The values of power that are expected for the given period. " + "There shall be at least one PowerForecastValue, and at most one per CommodityQuantity." + ), + ) + + +class PowerForecast(PydanticBaseModel): + """Represents a power forecast profile consisting of one or more forecast elements. + + This model defines a time-series forecast of power production or consumption + starting from a specified point in time. It consists of sequential forecast elements, + each covering a specific duration with associated power values for different + commodity quantities. + + Attributes: + start_time (DateTime): Start time of the period covered by the forecast. + elements (list[PowerForecastElement]): Chronologically ordered forecast segments. + """ + + start_time: DateTime = Field( + ..., description="Start time of time period that is covered by the profile." + ) + elements: list[PowerForecastElement] = Field( + ..., + min_length=1, + description=( + "Elements of which this forecast consists. Contains at least one element. " + "Elements shall be placed in chronological order." + ), + ) + + +# Base classes for control types + + +class BaseInstruction(PydanticBaseModel, ABC): + """Base class for S2 control instructions. + + This class defines the common structure for S2 standard control instructions. + An instruction must have a unique identifier (`id`), an `execution_time`, + and a flag indicating abnormal operation (`abnormal_condition`). If a `resource_id` + is provided at instantiation and `id` is not explicitly supplied, a new unique `id` + will be auto-generated as `{resource_id}-{UUID}`. + + Attributes: + id (Optional[ID]): Unique identifier of the instruction in the ResourceManager scope. + execution_time (DateTime): Start time of the instruction execution. + abnormal_condition (bool): Indicates if this is an instruction for abnormal conditions. + """ + + id: Optional[ID] = Field( + default=None, + description=( + "Unique identifier of the instruction in the ResourceManager scope. " + "If not provided and a `resource_id` is passed at instantiation, this will " + "be auto-generated as `{resource_id}@{UUID}`." + ), + ) + execution_time: DateTime = Field(..., description="Start time of the instruction execution.") + abnormal_condition: bool = Field( + default=False, + description="Indicates if this is an instruction for abnormal conditions. Defaults to False.", + ) + + @model_validator(mode="before") + def accept_resource_id(cls, values: dict) -> dict: + """Pre-process the initialization values. + + Accepts an optional `resource_id` and generates an unique instruction `id` if one is not + provided. + + Args: + values (dict): Raw keyword arguments passed to the model constructor. + + Returns: + dict: Updated keyword arguments with `id` set if `resource_id` was present + and `id` was not supplied. + """ + resource_id = values.pop("resource_id", None) + if resource_id and not values.get("id"): + values["id"] = f"{resource_id}@{uuid.uuid4()}" + return values + + # Computed fields + @computed_field # type: ignore[prop-decorator] + @property + def resource_id(self) -> str: + """Get the resource identifier component from the instruction's `id`. + + Assumes the `id` follows the format `{resource_id}@{UUID}`. Extracts the resource_id part + of the id by splitting at the last @. + + Returns: + str: The resource identifier prefix of `id`, or an empty string if `id` is None. + """ + return self.id.rsplit("@", 1)[0] if self.id else "" + + @abstractmethod + def duration(self) -> Optional[Duration]: + """Returns the active duration of this instruction. + + Returns: + Optional[Duration]: + - A finite Duration if the instruction is only active for that period. + - None if the instruction is active indefinitely. + """ + raise NotImplementedError( + f"{self.__class__.__name__} must implement the `duration()` method." + ) + + +# Control Types - Power Envelope Based Control (PEBC) + + +class PEBCAllowedLimitRange(PydanticBaseModel): + """Defines the permissible range for power envelope limits in PEBC. + + This model specifies the range of values that a Customer Energy Manager (CEM) + can select for upper or lower power envelope limits. It establishes the operational + boundaries for controlling a device using Power Envelope Based Control, + with optional flags for use during abnormal conditions. + """ + + commodity_quantity: CommodityQuantity = Field( + ..., description="Type of power quantity this range applies to." + ) + limit_type: PEBCPowerEnvelopeLimitType = Field( + ..., description="Whether this range applies to the upper or lower power envelope limit." + ) + range_boundary: NumberRange = Field( + ..., description="Range of values the CEM can choose for the power envelope." + ) + abnormal_condition_only: Optional[bool] = Field( + False, description="Indicates if this range can only be used during an abnormal condition." + ) + + +class PEBCPowerConstraints(PydanticBaseModel): + """Defines the constraints for power envelope control during a specific time period. + + This model specifies the allowed ranges for power envelope limits and the + consequences of limiting power within those ranges. It provides the CEM with + information about what power limits can be set and how those limits will affect + the controlled device's behavior. + """ + + id: ID = Field(..., description="Unique identifier of this PowerConstraints set.") + valid_from: DateTime = Field(..., description="Timestamp when these constraints become valid.") + valid_until: Optional[DateTime] = Field( + None, description="Optional end time of validity for these constraints." + ) + consequence_type: PEBCPowerEnvelopeConsequenceType = Field( + ..., description="The type of consequence when limiting power." + ) + allowed_limit_ranges: list[PEBCAllowedLimitRange] = Field( + ..., + description="List of allowed power envelope limit ranges. Must contain at least one UPPER_LIMIT and one LOWER_LIMIT.", + ) + + +class PEBCEnergyConstraints(PydanticBaseModel): + """Defines energy constraints over a time period for Power Envelope Based Control. + + This model specifies the minimum and maximum average power over a defined time period, + which translates to energy constraints. It enables the implementation of energy-based + limitations in addition to power-based limitations, supporting more sophisticated + energy management strategies. + """ + + id: ID = Field(..., description="Unique identifier of this EnergyConstraints object.") + valid_from: DateTime = Field(..., description="Start time for which this constraint is valid.") + valid_until: DateTime = Field(..., description="End time for which this constraint is valid.") + upper_average_power: float = Field( + ..., + description=( + "Maximum average power over the given time period. " + "Used to derive maximum energy content." + ), + ) + lower_average_power: float = Field( + ..., + description=( + "Minimum average power over the given time period. " + "Used to derive minimum energy content." + ), + ) + commodity_quantity: CommodityQuantity = Field( + ..., description="The commodity or type of power to which this applies." + ) + + +class PEBCPowerEnvelopeElement(PydanticBaseModel): + """Defines a segment of a power envelope for a specific duration. + + This model specifies the upper and lower power limits for a specific time duration, + forming part of a complete power envelope. A sequence of these elements creates + a time-varying power envelope that constrains device power consumption or production. + """ + + duration: Duration = Field(..., description="Duration of this power envelope element.") + upper_limit: float = Field( + ..., + description=( + "Upper power limit for the given commodity_quantity. " + "Shall match PEBC.AllowedLimitRange with limit_type UPPER_LIMIT." + ), + ) + lower_limit: float = Field( + ..., + description=( + "Lower power limit for the given commodity_quantity. " + "Shall match PEBC.AllowedLimitRange with limit_type LOWER_LIMIT." + ), + ) + + +class PEBCPowerEnvelope(PydanticBaseModel): + """Defines a complete power envelope constraint for a specific commodity quantity. + + This model specifies a time-series of power limits (upper and lower bounds) that + a device must operate within. The power envelope consists of sequential elements, + each defining constraints for a specific duration, creating a complete time-varying + operational boundary for the device. + """ + + id: ID = Field( + ..., + description=( + "Unique identifier of this PEBC.PowerEnvelope, scoped to the ResourceManager." + ), + ) + commodity_quantity: CommodityQuantity = Field( + ..., description="Type of power quantity the envelope applies to." + ) + power_envelope_elements: list[PEBCPowerEnvelopeElement] = Field( + ..., + min_length=1, + description=( + "Chronologically ordered list of PowerEnvelopeElements. " + "Defines how power should be constrained over time." + ), + ) + + +class PEBCInstruction(BaseInstruction): + """Represents a control instruction for Power Envelope Based Control. + + This model defines a complete instruction for controlling a device using power + envelopes. It specifies when the instruction should be executed, which power + constraints apply, and the specific power envelopes to follow. It supports + multiple power envelopes for different commodity quantities. + """ + + type: Literal["PEBCInstruction"] = Field(default="PEBCInstruction") + power_constraints_id: ID = Field(..., description="ID of the associated PEBC.PowerConstraints.") + power_envelopes: list[PEBCPowerEnvelope] = Field( + ..., + min_length=1, + description=( + "List of PowerEnvelopes to follow. One per CommodityQuantity, max one per type." + ), + ) + + def duration(self) -> Optional[Duration]: + envelope_durations: list[Duration] = [] + for power_envelope in self.power_envelopes: + total_duration = Duration(seconds=0) + for power_envelope_element in power_envelope.power_envelope_elements: + total_duration += power_envelope_element.duration + envelope_durations.append(total_duration) + + return max(envelope_durations) if envelope_durations else None + + +# Control Types - Power Profile Based Control (PPBC) + + +class PPBCPowerSequenceElement(PydanticBaseModel): + """Defines a segment of a power sequence with specific duration and power values. + + This model represents a time segment within a power sequence, specifying the + forecasted power values for the duration. Multiple elements arranged sequentially + form a complete power sequence, defining how power will vary over time during + the execution of the sequence. + """ + + duration: Duration = Field(..., description="Duration of the sequence element.") + power_values: list[PowerForecastValue] = Field( + ..., description="Forecasted power values for the duration, one per CommodityQuantity." + ) + + +class PPBCPowerSequence(PydanticBaseModel): + """Defines a specific power sequence pattern with timing and interruptibility properties. + + This model specifies a detailed sequence of power behaviors over time, represented + as a series of power sequence elements. It includes properties that define whether + the sequence can be interrupted and timing constraints related to its execution, + supporting flexible power management strategies. + """ + + id: ID = Field(..., description="Unique identifier of the PowerSequence within its container.") + elements: list[PPBCPowerSequenceElement] = Field( + ..., description="Ordered list of sequence elements representing power behavior." + ) + is_interruptible: bool = Field( + ..., description="Indicates whether this sequence can be interrupted." + ) + max_pause_before: Optional[Duration] = Field( + None, + description="Maximum allowed pause before this sequence starts after the previous one.", + ) + abnormal_condition_only: bool = Field( + ..., description="True if sequence is only applicable in abnormal conditions." + ) + + +class PPBCPowerSequenceContainer(PydanticBaseModel): + """Groups alternative power sequences for a specific phase of operation. + + This model organizes multiple alternative power sequences for a specific operational + phase, allowing the CEM to select one based on system requirements. Containers are + arranged chronologically within a power profile definition to represent sequential + phases of a complete operation. + """ + + id: ID = Field( + ..., + description="Unique identifier of the PowerSequenceContainer within its parent PowerProfileDefinition.", + ) + power_sequences: list[PPBCPowerSequence] = Field( + ..., description="List of alternative PowerSequences. One will be selected by the CEM." + ) + + +class PPBCPowerProfileDefinition(PydanticBaseModel): + """Defines a complete power profile for Power Profile Based Control. + + This model specifies a structured power profile consisting of multiple sequence + containers arranged chronologically. Each container holds alternative power sequences, + allowing the CEM to select the most appropriate sequence based on system needs. + The profile includes timing constraints for when the sequences can be executed. + """ + + id: ID = Field( + ..., + description="Unique identifier of the PowerProfileDefinition within the ResourceManager session.", + ) + start_time: DateTime = Field( + ..., description="Earliest possible start time of the first PowerSequence." + ) + end_time: DateTime = Field( + ..., description="Latest time the last PowerSequence must be completed." + ) + power_sequences_containers: list[PPBCPowerSequenceContainer] = Field( + ..., + description="List of containers for alternative power sequences, in chronological order.", + ) + + +class PPBCPowerSequenceContainerStatus(PydanticBaseModel): + """Reports the status of a specific power sequence container execution. + + This model provides detailed status information for a single sequence container, + including which sequence was selected, the current execution progress, and the + operational status. It enables fine-grained monitoring of sequence execution + within the broader power profile. + """ + + power_profile_id: ID = Field(..., description="ID of the related PowerProfileDefinition.") + sequence_container_id: ID = Field( + ..., description="ID of the PowerSequenceContainer being reported on." + ) + selected_sequence_id: Optional[str] = Field( + None, description="ID of the selected PowerSequence, if any." + ) + progress: Optional[Duration] = Field( + None, description="Elapsed time since the selected sequence started, if applicable." + ) + status: PPBCPowerSequenceStatus = Field( + ..., description="Status of the selected PowerSequence." + ) + + +class PPBCPowerProfileStatus(PydanticBaseModel): + """Reports the current status of a power profile execution. + + This model provides comprehensive status information for all sequence containers + in a power profile definition, enabling monitoring of profile execution progress. + It tracks which sequences have been selected and their current execution status. + """ + + type: Literal["PPBCPowerProfileStatus"] = Field(default="PPBCPowerProfileStatus") + + sequence_container_status: list[PPBCPowerSequenceContainerStatus] = Field( + ..., description="Status list for all sequence containers in the PowerProfileDefinition." + ) + + +class PPBCScheduleInstruction(BaseInstruction): + """Represents an instruction to schedule execution of a specific power sequence. + + This model defines a control instruction that schedules the execution of a + selected power sequence from a power profile. It specifies which sequence + has been selected and when it should begin execution, enabling precise control + of device power behavior according to the predefined sequence. + """ + + type: Literal["PPBCScheduleInstruction"] = Field(default="PPBCScheduleInstruction") + + power_profile_id: ID = Field( + ..., description="ID of the PowerProfileDefinition being scheduled." + ) + sequence_container_id: ID = Field( + ..., description="ID of the container with the selected sequence." + ) + power_sequence_id: ID = Field(..., description="ID of the selected PowerSequence.") + + def duration(self) -> Optional[Duration]: + # @TODO: PPBCPowerProfileDefinition needed + return None + + +class PPBCStartInterruptionInstruction(BaseInstruction): + """Represents an instruction to interrupt execution of a running power sequence. + + This model defines a control instruction that interrupts the execution of an + active power sequence. It enables dynamic control over sequence execution, + allowing temporary suspension of a sequence in response to changing system conditions + or requirements, particularly for sequences marked as interruptible. + """ + + type: Literal["PPBCStartInterruptionInstruction"] = Field( + default="PPBCStartInterruptionInstruction" + ) + power_profile_id: ID = Field( + ..., description="ID of the PowerProfileDefinition whose sequence is being interrupted." + ) + sequence_container_id: ID = Field( + ..., description="ID of the container containing the sequence." + ) + power_sequence_id: ID = Field(..., description="ID of the PowerSequence to be interrupted.") + + def duration(self) -> Optional[Duration]: + # @TODO: PPBCPowerProfileDefinition needed + return None + + +class PPBCEndInterruptionInstruction(BaseInstruction): + """Represents an instruction to resume execution of a previously interrupted power sequence. + + This model defines a control instruction that ends an interruption and resumes + execution of a previously interrupted power sequence. It complements the start + interruption instruction, enabling the complete interruption-resumption cycle + for flexible sequence execution control. + """ + + type: Literal["PPBCEndInterruptionInstruction"] = Field( + default="PPBCEndInterruptionInstruction" + ) + power_profile_id: ID = Field( + ..., description="ID of the PowerProfileDefinition related to the ended interruption." + ) + sequence_container_id: ID = Field( + ..., description="ID of the container containing the sequence." + ) + power_sequence_id: ID = Field( + ..., description="ID of the PowerSequence for which the interruption ends." + ) + + def duration(self) -> Optional[Duration]: + # @TODO: PPBCPowerProfileDefinition needed + return None + + +# Control Types - Operation Mode Based Control (OMBC) + + +class OMBCOperationMode(PydanticBaseModel): + """Operation Mode for Operation Mode Based Control (OMBC). + + Defines a specific operation mode with its power consumption/production characteristics and costs. + Each operation mode represents a distinct way the resource can operate, with an associated power profile. + """ + + id: ID = Field( + ..., description="Unique ID of the OperationMode within the ResourceManager session." + ) + diagnostic_label: Optional[str] = Field( + None, description="Human-readable label for diagnostics (not for HMI)." + ) + power_ranges: list[PowerRange] = Field( + ..., + description="List of power consumption or production ranges mapped to operation_mode_factor 0 to 1.", + ) + running_costs: Optional[NumberRange] = Field( + None, + description="Estimated additional costs per second, excluding commodity cost. Represents uncertainty.", + ) + abnormal_condition_only: bool = Field( + ..., description="True if this mode can only be used during an abnormal condition." + ) + + +class OMBCStatus(PydanticBaseModel): + """Reports the current operational status of an Operation Mode Based Control system. + + This model provides real-time status information about an OMBC-controlled device, + including which operation mode is currently active, how it is configured, + and information about recent mode transitions. It enables monitoring of the + device's operational state and tracking mode transition history. + """ + + type: Literal["OMBCStatus"] = Field(default="OMBCStatus") + + active_operation_mode_id: ID = Field( + ..., description="ID of the currently active operation mode." + ) + operation_mode_factor: float = Field( + ..., + ge=0.0, + le=1.0, + description="Factor with which the operation mode is configured (between 0 and 1).", + ) + previous_operation_mode_id: Optional[str] = Field( + None, description="ID of the previously active operation mode, if known." + ) + transition_timestamp: Optional[DateTime] = Field( + None, description="Timestamp of transition to the active operation mode, if applicable." + ) + + +class OMBCTimerStatus(PydanticBaseModel): + """Current status of an OMBC Timer. + + Indicates when the Timer will be finished. + """ + + type: Literal["OMBCTimerStatus"] = Field(default="OMBCTimerStatus") + + timer_id: ID = Field(..., description="ID of the timer this status refers to.") + + finished_at: DateTime = Field( + ..., + description="Indicates when the Timer will be finished. If the DateTime is in the future, the timer is not yet finished. If the DateTime is in the past, the timer is finished. If the timer was never started, the value can be an arbitrary DateTimeStamp in the past.", + ) + + +class OMBCSystemDefinition(PydanticBaseModel): + """Provides a comprehensive definition of an Operation Mode Based Control system. + + This model defines the complete operational framework for a device controlled using + Operation Mode Based Control. It specifies all available operation modes, permitted + transitions between modes, and the timing constraints via timers. + """ + + valid_from: DateTime = Field( + ..., + description="Start time from which this system description is valid. Must be in the past or present if immediately applicable.", + ) + operation_modes: list[OMBCOperationMode] = Field( + ..., + description="List of operation modes available for the CEM to coordinate device behavior.", + ) + transitions: list[Transition] = Field( + ..., description="Possible transitions between operation modes." + ) + timers: list[Timer] = Field( + ..., description="Timers specifying constraints for when transitions can occur." + ) + + +class OMBCSystemDescription(OMBCSystemDefinition): + """Provides a comprehensive description of an Operation Mode Based Control system. + + This model defines the complete operational framework for a device controlled using + Operation Mode Based Control. It specifies all available operation modes, permitted + transitions between modes, timing constraints via timers, and the current operational + status. + + It serves as the foundation for understanding and controlling the device's behavior. + """ + + status: OMBCStatus = Field( + ..., + description="Current status information, including the active operation mode and transition details.", + ) + + +class OMBCInstruction(BaseInstruction): + """Instruction for Operation Mode Based Control (OMBC). + + Contains information about when and how to activate a specific operation mode. + Used to command resources to change their operation at a specified time. + """ + + type: Literal["OMBCInstruction"] = Field(default="OMBCInstruction") + operation_mode_id: ID = Field(..., description="ID of the OMBC.OperationMode to activate.") + operation_mode_factor: float = Field( + ..., + ge=0.0, + le=1.0, + description="Factor with which the operation mode is configured (0 to 1).", + ) + + def duration(self) -> Optional[Duration]: + # Infinite, until next instruction + return None + + +# Control Types - Fill Rate Based Control (FRBC) + + +class FRBCOperationModeElement(PydanticBaseModel): + """Element of an FRBC Operation Mode with properties dependent on fill level. + + Defines how a resource operates within a specific fill level range, including + its effect on fill rate and associated power consumption/production. + """ + + fill_level_range: NumberRange = Field(..., description="Fill level range for this element.") + fill_rate: NumberRange = Field( + ..., description="Change in fill level per second for this mode." + ) + power_ranges: list[PowerRange] = Field( + ..., description="Power produced/consumed per commodity." + ) + running_costs: Optional[NumberRange] = Field( + None, description="Additional costs per second (excluding commodity cost)." + ) + + +class FRBCOperationMode(PydanticBaseModel): + """Operation Mode for Fill Rate Based Control (FRBC). + + Defines a complete operation mode with properties that may vary based on + the current fill level of the associated storage. Each mode represents a + distinct way to operate the resource affecting the storage fill level. + """ + + id: ID = Field(..., description="Unique ID of the operation mode within the actuator.") + diagnostic_label: Optional[str] = Field( + None, description="Human-readable label for diagnostics." + ) + elements: list[FRBCOperationModeElement] = Field( + ..., description="Properties of the mode depending on fill level." + ) + abnormal_condition_only: bool = Field( + ..., description="True if mode is for abnormal conditions only." + ) + + +class FRBCActuatorStatus(PydanticBaseModel): + """Current status of an FRBC Actuator. + + Provides information about the currently active operation mode and transition history. + Used to track the current state of the actuator. + """ + + type: Literal["FRBCActuatorStatus"] = Field(default="FRBCActuatorStatus") + + active_operation_mode_id: ID = Field(..., description="Currently active operation mode ID.") + operation_mode_factor: float = Field( + ..., ge=0, le=1, description="Factor with which the mode is configured (0 to 1)." + ) + previous_operation_mode_id: Optional[str] = Field( + None, description="Previously active operation mode ID." + ) + transition_timestamp: Optional[DateTime] = Field( + None, description="Timestamp of the last transition between modes." + ) + + +class FRBCActuatorDefinition(PydanticBaseModel): + """Definition of an Actuator for Fill Rate Based Control (FRBC). + + Provides a complete definition of an actuator including its capabilities, + available operation modes, and constraints on transitions between modes. + """ + + id: ID = Field(..., description="Unique actuator ID within the ResourceManager session.") + diagnostic_label: Optional[str] = Field( + None, description="Human-readable actuator description for diagnostics." + ) + supported_commodities: list[str] = Field(..., description="List of supported commodity IDs.") + operation_modes: list[FRBCOperationMode] = Field( + ..., description="Operation modes provided by this actuator." + ) + transitions: list[Transition] = Field( + ..., description="Allowed transitions between operation modes." + ) + timers: list[Timer] = Field(..., description="Timers associated with this actuator.") + + +class FRBCActuatorDescription(FRBCActuatorDefinition): + """Description of an Actuator for Fill Rate Based Control (FRBC). + + Provides a complete definition of an actuator including its capabilities, + available operation modes, constraints on transitions between modes, and the current status + of the actuator. + """ + + status: FRBCActuatorStatus = Field(..., description="Current status of the actuator.") + + +class FRBCEnergyStatus(PydanticBaseModel): + """Energy status of an FRBC storage. + + Note: This is an extension to the S2 standard. + """ + + type: Literal["FRBCEnergyStatus"] = Field(default="FRBCEnergyStatus") + + import_total: Optional[EnergyMeasurement] = Field( + default=None, description="Total cumulative imported energy from the energy meter start." + ) + export_total: Optional[EnergyMeasurement] = Field( + default=None, description="Total cumulative exported energy from the energy meter start." + ) + + +class FRBCStorageStatus(PydanticBaseModel): + """Current status of an FRBC Storage. + + Indicates the current fill level of the storage, which is essential + for determining applicable operation modes and control decisions. + """ + + type: Literal["FRBCStorageStatus"] = Field(default="FRBCStorageStatus") + + present_fill_level: float = Field(..., description="Current fill level of the storage.") + + +class FRBCTimerStatus(PydanticBaseModel): + """Current status of an FRBC Timer. + + Indicates when the Timer will be finished. + """ + + type: Literal["FRBCTimerStatus"] = Field(default="FRBCTimerStatus") + + actuator_id: ID = Field(..., description="ID of the actuator the timer belongs to.") + + timer_id: ID = Field(..., description="ID of the timer this status refers to.") + + finished_at: DateTime = Field( + ..., + description="Indicates when the Timer will be finished. If the DateTime is in the future, the timer is not yet finished. If the DateTime is in the past, the timer is finished. If the timer was never started, the value can be an arbitrary DateTimeStamp in the past.", + ) + + +class FRBCLeakageBehaviourElement(PydanticBaseModel): + """Element of the leakage behavior for an FRBC Storage. + + Describes how leakage varies with fill level, used to model natural + losses in the storage over time. + """ + + fill_level_range: NumberRange = Field( + ..., description="Applicable fill level range for this element." + ) + leakage_rate: float = Field( + ..., description="Rate of fill level decrease per second due to leakage." + ) + + +class FRBCLeakageBehaviour(PydanticBaseModel): + """Complete leakage behavior model for an FRBC Storage. + + Describes how the storage naturally loses its content over time, + with leakage rates that may vary based on fill level. + """ + + valid_from: DateTime = Field(..., description="Start of validity for this leakage behaviour.") + elements: list[FRBCLeakageBehaviourElement] = Field( + ..., description="Contiguous elements modeling leakage." + ) + + +class FRBCUsageForecastElement(PydanticBaseModel): + """Element of a usage forecast for an FRBC Storage. + + Describes expected usage rates for a specific duration, including + probability ranges to represent uncertainty. + """ + + duration: Duration = Field(..., description="How long the given usage rate is valid.") + usage_rate_upper_limit: Optional[float] = Field( + None, description="100% probability upper limit." + ) + usage_rate_upper_95PPR: Optional[float] = Field( + None, description="95% probability upper limit." + ) + usage_rate_upper_68PPR: Optional[float] = Field( + None, description="68% probability upper limit." + ) + usage_rate_expected: float = Field(..., description="Most likely usage rate.") + usage_rate_lower_68PPR: Optional[float] = Field( + None, description="68% probability lower limit." + ) + usage_rate_lower_95PPR: Optional[float] = Field( + None, description="95% probability lower limit." + ) + usage_rate_lower_limit: Optional[float] = Field( + None, description="100% probability lower limit." + ) + + +class FRBCUsageForecast(PydanticBaseModel): + """Complete usage forecast for an FRBC Storage. + + Provides a time-series forecast of expected usage rates, + allowing for planning of optimal resource operation. + """ + + start_time: DateTime = Field(..., description="Start time of the forecast.") + elements: list[FRBCUsageForecastElement] = Field( + ..., description="Chronological forecast profile elements." + ) + + +class FRBCFillLevelTargetProfileElement(PydanticBaseModel): + """Element of a fill level target profile for an FRBC Storage. + + Specifies the desired fill level range for a specific duration, + used to guide resource operation planning. + """ + + duration: Duration = Field(..., description="Duration this target applies for.") + fill_level_range: NumberRange = Field( + ..., description="Target fill level range for the duration." + ) + + +class FRBCFillLevelTargetProfile(PydanticBaseModel): + """Complete fill level target profile for an FRBC Storage. + + Defines a time-series of target fill levels, providing goals + for the control system to achieve through resource operation. + """ + + start_time: DateTime = Field(..., description="Start time of the fill level target profile.") + elements: list[FRBCFillLevelTargetProfileElement] = Field( + ..., description="Chronological list of target ranges." + ) + + +class FRBCStorageDefinition(PydanticBaseModel): + """Definition of a Storage for Fill Rate Based Control (FRBC). + + Provides a complete definition of a storage including its capabilities, + constraints, and behavior characteristics. + """ + + diagnostic_label: Optional[str] = Field( + None, description="Diagnostic description of the storage." + ) + fill_level_label: Optional[str] = Field( + None, description="Description of fill level units (e.g. °C, %)." + ) + fill_level_range: NumberRange = Field( + ..., description="Range in which fill level should remain." + ) + leakage_behaviour: Optional[FRBCLeakageBehaviour] = Field( + None, description="Details of buffer leakage behaviour." + ) + + +class FRBCStorageDescription(FRBCStorageDefinition): + """Description of a Storage for Fill Rate Based Control (FRBC). + + Provides a complete definition of a storage including its capabilities, + constraints, current status, and behavior characteristics. + """ + + status: FRBCStorageStatus = Field(..., description="Current storage status.") + provides_leakage_behaviour: bool = Field( + ..., description="True if leakage behaviour can be provided." + ) + provides_fill_level_target_profile: bool = Field( + ..., description="True if fill level target profile can be provided." + ) + provides_usage_forecast: bool = Field( + ..., description="True if usage forecast can be provided." + ) + + +class FRBCInstruction(BaseInstruction): + """Instruction for Fill Rate Based Control (FRBC). + + Contains information about when and how to activate a specific operation mode + for an actuator. Used to command resources to change their operation at a specified time. + """ + + type: Literal["FRBCInstruction"] = Field(default="FRBCInstruction") + actuator_id: ID = Field(..., description="ID of the actuator this instruction belongs to.") + operation_mode_id: str = Field(..., description="ID of the operation mode to activate.") + operation_mode_factor: float = Field( + ..., ge=0, le=1, description="Factor for the operation mode configuration (0 to 1)." + ) + + def duration(self) -> Optional[Duration]: + # Infinite, until next instruction + return None + + +class FRBCSystemDescription(PydanticBaseModel): + """Complete system description for Fill Rate Based Control (FRBC). + + Provides a comprehensive description of all components in an FRBC system, + including actuators and storage. This is the top-level model for FRBC. + """ + + valid_from: DateTime = Field(..., description="Time this system description becomes valid.") + actuators: list[FRBCActuatorDescription] = Field(..., description="List of all actuators.") + storage: FRBCStorageDescription = Field(..., description="Details of the storage.") + + +# Control Types - Demand Driven Based Control (DDBC) + + +class DDBCOperationMode(PydanticBaseModel): + """Operation Mode for Demand Driven Based Control (DDBC). + + Defines a specific operation mode with its power consumption/production characteristics, + supply capabilities, and costs. Each mode represents a distinct way to operate a resource + to meet demand. + """ + + id: ID = Field( + ..., description="ID of the operation mode. Must be unique within the actuator description." + ) + diagnostic_label: Optional[str] = Field( + None, description="Human-readable name/description for diagnostics (not for HMI)." + ) + power_ranges: list[PowerRange] = Field( + ..., + description="Power ranges associated with this operation mode. At least one per CommodityQuantity.", + ) + supply_range: NumberRange = Field( + ..., description="Supply rate that can match the demand rate, mapped from factor 0 to 1." + ) + running_costs: NumberRange = Field( + ..., + description="Additional cost per second (excluding commodity cost). Represents uncertainty, not linked to factor.", + ) + abnormal_condition_only: Optional[bool] = Field( + False, + description="Whether this operation mode may only be used during abnormal conditions.", + ) + + +class DDBCActuatorStatus(PydanticBaseModel): + """Current status of a DDBC Actuator. + + Provides information about the currently active operation mode and transition history. + Used to track the current state of the actuator. + """ + + type: Literal["DDBCActuatorStatus"] = Field(default="DDBCActuatorStatus") + + active_operation_mode_id: ID = Field(..., description="Currently active operation mode ID.") + operation_mode_factor: float = Field( + ..., ge=0, le=1, description="Factor with which the operation mode is configured (0 to 1)." + ) + previous_operation_mode_id: Optional[str] = Field( + None, + description="Previously active operation mode ID. Required unless this is the first mode.", + ) + transition_timestamp: Optional[DateTime] = Field( + None, description="Timestamp of transition to the active operation mode." + ) + + +class DDBCActuatorDefinition(PydanticBaseModel): + """Definition of an Actuator for Demand Driven Based Control (DDBC). + + Provides a complete definition of an actuator including its capabilities, + available operation modes, and constraints on transitions between modes. + """ + + id: ID = Field( + ..., + description="ID of this actuator. Must be unique in the ResourceManager scope during the session.", + ) + diagnostic_label: Optional[str] = Field( + None, description="Human-readable name/description for diagnostics (not for HMI)." + ) + supported_commodities: list[str] = Field( + ..., description="Commodities supported by this actuator. Must include at least one." + ) + operation_modes: list[DDBCOperationMode] = Field( + ..., + description="List of available operation modes for this actuator. Must include at least one.", + ) + transitions: list[Transition] = Field( + ..., description="List of transitions between operation modes. Must include at least one." + ) + timers: list[Timer] = Field( + ..., description="List of timers associated with transitions. Can be empty." + ) + + +class DDBCActuatorDescription(DDBCActuatorDefinition): + """Description of an Actuator for Demand Driven Based Control (DDBC). + + Provides a complete description of an actuator including its capabilities, + available operation modes, constraints on transitions between modes, and + its present status. + """ + + status: DDBCActuatorStatus = Field(..., description="Present status of this actuator.") + + +class DDBCSystemDescription(PydanticBaseModel): + """Complete system description for Demand Driven Based Control (DDBC). + + Provides a comprehensive description of all components in a DDBC system, + including actuators and demand characteristics. This is the top-level model for DDBC. + """ + + valid_from: DateTime = Field( + ..., + description="Moment this DDBC.SystemDescription starts to be valid. If immediately valid, it should be now or in the past.", + ) + actuators: list[DDBCActuatorDescription] = Field( + ..., + description="List of all available actuators in the system. Shall contain at least one DDBC.ActuatorAggregated.", + ) + present_demand_rate: NumberRange = Field( + ..., description="Present demand rate that needs to be satisfied by the system." + ) + provides_average_demand_rate_forecast: bool = Field( + ..., + description="Indicates whether a demand rate forecast is provided through DDBC.AverageDemandRateForecast.", + ) + + +class DDBCInstruction(BaseInstruction): + """Instruction for Demand Driven Based Control (DDBC). + + Contains information about when and how to activate a specific operation mode + for an actuator. Used to command resources to change their operation at a specified time. + """ + + type: Literal["DDBCInstruction"] = Field(default="DDBCInstruction") + actuator_id: ID = Field(..., description="ID of the actuator this instruction belongs to.") + operation_mode_id: ID = Field(..., description="ID of the DDBC.OperationMode to apply.") + operation_mode_factor: float = Field( + ..., + ge=0, + le=1, + description="Factor with which the operation mode should be applied (0 to 1).", + ) + + def duration(self) -> Optional[Duration]: + # Infinite, until next instruction + return None + + +class DDBCAverageDemandRateForecastElement(PydanticBaseModel): + """Element of a demand rate forecast for DDBC. + + Describes expected demand rates for a specific duration, including + probability ranges to represent uncertainty. + """ + + duration: Duration = Field(..., description="Duration of this forecast element.") + demand_rate_upper_limit: Optional[float] = Field( + None, description="100% upper limit of demand rate range." + ) + demand_rate_upper_95PPR: Optional[float] = Field( + None, description="95% upper limit of demand rate range." + ) + demand_rate_upper_68PPR: Optional[float] = Field( + None, description="68% upper limit of demand rate range." + ) + demand_rate_expected: float = Field( + ..., description="Expected demand rate (fill level increase/decrease per second)." + ) + demand_rate_lower_68PPR: Optional[float] = Field( + None, description="68% lower limit of demand rate range." + ) + demand_rate_lower_95PPR: Optional[float] = Field( + None, description="95% lower limit of demand rate range." + ) + demand_rate_lower_limit: Optional[float] = Field( + None, description="100% lower limit of demand rate range." + ) + + +class DDBCAverageDemandRateForecast(PydanticBaseModel): + """Complete demand rate forecast for DDBC. + + Provides a time-series forecast of expected demand rates, + allowing for planning of optimal resource operation to meet future demands. + """ + + start_time: DateTime = Field( + ..., description="Start time of the average demand rate forecast profile." + ) + elements: list[DDBCAverageDemandRateForecastElement] = Field( + ..., description="List of forecast elements in chronological order." + ) + + +# Resource Status + +# ResourceStatus, discriminated by its type field +ResourceStatus = Annotated[ + Union[ + PowerMeasurement, + EnergyMeasurement, + PPBCPowerProfileStatus, + OMBCStatus, + FRBCActuatorStatus, + FRBCEnergyStatus, + FRBCStorageStatus, + FRBCTimerStatus, + DDBCActuatorStatus, + ], + Field(discriminator="type"), +] + + +# Plan + +# Instruction, discriminated by its type field +EnergyManagementInstruction = Annotated[ + Union[ + PEBCInstruction, + PPBCScheduleInstruction, + PPBCStartInterruptionInstruction, + PPBCEndInterruptionInstruction, + OMBCInstruction, + FRBCInstruction, + DDBCInstruction, + ], + Field(discriminator="type"), +] + + +class EnergyManagementPlan(PydanticBaseModel): + """A coordinated energy management plan composed of device control instructions. + + Attributes: + plan_id (ID): Unique identifier for this energy management plan. + generated_at (DateTime): Timestamp when the plan was generated. + valid_from (Optional[DateTime]): Earliest start time of any instruction. + valid_until (Optional[DateTime]): Latest end time across all instructions + with finite duration; None if all instructions have infinite duration. + instructions (list[BaseInstruction]): List of control instructions for the plan. + comment (Optional[str]): Optional comment or annotation for the plan. + """ + + id: ID = Field(..., description="Unique ID for the energy management plan.") + generated_at: DateTime = Field(..., description="Timestamp when the plan was generated.") + valid_from: Optional[DateTime] = Field( + default=None, description="Earliest start time of any instruction." + ) + valid_until: Optional[DateTime] = Field( + default=None, + description=( + "Latest end time across all instructions with finite duration; " + "None if all instructions have infinite duration." + ), + ) + instructions: list[EnergyManagementInstruction] = Field( + ..., description="List of control instructions for the plan." + ) + comment: Optional[str] = Field( + default=None, description="Optional comment or annotation for the plan." + ) + + def _update_time_range(self) -> None: + """Updates valid_from and valid_until based on the instructions. + + Sets valid_from as the earliest execution_time of the instructions. + Sets valid_until as the latest end time, or None if any instruction is infinite. + """ + if not self.instructions: + self.valid_from = to_datetime() + self.valid_until = None + return + + self.valid_from = min(i.execution_time for i in self.instructions) + + end_times = [] + for instr in self.instructions: + instr_duration = instr.duration() # Returns Optional[Duration] + if instr_duration is None: + # Infinite instruction means valid_until must be None + self.valid_until = None + return + end_times.append(instr.execution_time + instr_duration) + + self.valid_until = max(end_times) if end_times else None + + def add_instruction(self, instruction: EnergyManagementInstruction) -> None: + """Adds a new control instruction and updates time range.""" + self.instructions.append(instruction) + self.instructions.sort(key=lambda i: i.execution_time) + self._update_time_range() + + def clear(self) -> None: + """Removes all control instructions and resets time range.""" + self.instructions.clear() + self.valid_from = to_datetime() + self.valid_until = None + + def get_active_instructions( + self, now: Optional[DateTime] = None + ) -> list[EnergyManagementInstruction]: + """Retrieves all currently active instructions at the specified time.""" + now = now or to_datetime() + active = [] + for instr in self.instructions: + instr_duration = instr.duration() + if instr_duration is None: + if instr.execution_time <= now: + active.append(instr) + else: + if instr.execution_time <= now < instr.execution_time + instr_duration: + active.append(instr) + return active + + def get_next_instruction( + self, now: Optional[DateTime] = None + ) -> Optional[EnergyManagementInstruction]: + """Finds the next instruction scheduled after the specified time.""" + now = now or to_datetime() + future_instructions = [i for i in self.instructions if i.execution_time > now] + return ( + min(future_instructions, key=lambda i: i.execution_time) + if future_instructions + else None + ) + + def get_instructions_for_resource(self, resource_id: ID) -> list[EnergyManagementInstruction]: + """Filters the plan's instructions for a specific resource.""" + return [i for i in self.instructions if i.resource_id == resource_id] diff --git a/src/akkudoktoreos/core/ems.py b/src/akkudoktoreos/core/ems.py index 043637e..6ceb39d 100644 --- a/src/akkudoktoreos/core/ems.py +++ b/src/akkudoktoreos/core/ems.py @@ -1,123 +1,50 @@ import traceback -from typing import Any, ClassVar, Optional +from asyncio import Lock, get_running_loop +from concurrent.futures import ThreadPoolExecutor +from functools import partial +from typing import ClassVar, Optional -import numpy as np from loguru import logger -from numpydantic import NDArray, Shape -from pendulum import DateTime -from pydantic import ConfigDict, Field, computed_field, field_validator, model_validator -from typing_extensions import Self +from pydantic import computed_field -from akkudoktoreos.core.cache import CacheUntilUpdateStore +from akkudoktoreos.core.cache import CacheEnergyManagementStore from akkudoktoreos.core.coreabc import ConfigMixin, PredictionMixin, SingletonMixin -from akkudoktoreos.core.pydantic import ParametersBaseModel, PydanticBaseModel -from akkudoktoreos.devices.battery import Battery -from akkudoktoreos.devices.generic import HomeAppliance -from akkudoktoreos.devices.inverter import Inverter -from akkudoktoreos.utils.datetimeutil import compare_datetimes, to_datetime -from akkudoktoreos.utils.utils import NumpyEncoder +from akkudoktoreos.core.emplan import EnergyManagementPlan +from akkudoktoreos.core.emsettings import EnergyManagementMode +from akkudoktoreos.core.pydantic import PydanticBaseModel +from akkudoktoreos.optimization.genetic.genetic import GeneticOptimization +from akkudoktoreos.optimization.genetic.geneticparams import ( + GeneticOptimizationParameters, +) +from akkudoktoreos.optimization.genetic.geneticsolution import GeneticSolution +from akkudoktoreos.optimization.optimization import OptimizationSolution +from akkudoktoreos.utils.datetimeutil import DateTime, compare_datetimes, to_datetime - -class EnergyManagementParameters(ParametersBaseModel): - pv_prognose_wh: list[float] = Field( - description="An array of floats representing the forecasted photovoltaic output in watts for different time intervals." - ) - strompreis_euro_pro_wh: list[float] = Field( - description="An array of floats representing the electricity price in euros per watt-hour for different time intervals." - ) - einspeiseverguetung_euro_pro_wh: list[float] | float = Field( - description="A float or array of floats representing the feed-in compensation in euros per watt-hour." - ) - preis_euro_pro_wh_akku: float = Field( - description="A float representing the cost of battery energy per watt-hour." - ) - gesamtlast: list[float] = Field( - description="An array of floats representing the total load (consumption) in watts for different time intervals." - ) - - @model_validator(mode="after") - def validate_list_length(self) -> Self: - pv_prognose_length = len(self.pv_prognose_wh) - if ( - pv_prognose_length != len(self.strompreis_euro_pro_wh) - or pv_prognose_length != len(self.gesamtlast) - or ( - isinstance(self.einspeiseverguetung_euro_pro_wh, list) - and pv_prognose_length != len(self.einspeiseverguetung_euro_pro_wh) - ) - ): - raise ValueError("Input lists have different lengths") - return self - - -class SimulationResult(ParametersBaseModel): - """This object contains the results of the simulation and provides insights into various parameters over the entire forecast period.""" - - Last_Wh_pro_Stunde: list[Optional[float]] = Field(description="TBD") - EAuto_SoC_pro_Stunde: list[Optional[float]] = Field( - description="The state of charge of the EV for each hour." - ) - Einnahmen_Euro_pro_Stunde: list[Optional[float]] = Field( - description="The revenue from grid feed-in or other sources in euros per hour." - ) - Gesamt_Verluste: float = Field( - description="The total losses in watt-hours over the entire period." - ) - Gesamtbilanz_Euro: float = Field( - description="The total balance of revenues minus costs in euros." - ) - Gesamteinnahmen_Euro: float = Field(description="The total revenues in euros.") - Gesamtkosten_Euro: float = Field(description="The total costs in euros.") - Home_appliance_wh_per_hour: list[Optional[float]] = Field( - description="The energy consumption of a household appliance in watt-hours per hour." - ) - Kosten_Euro_pro_Stunde: list[Optional[float]] = Field( - description="The costs in euros per hour." - ) - Netzbezug_Wh_pro_Stunde: list[Optional[float]] = Field( - description="The grid energy drawn in watt-hours per hour." - ) - Netzeinspeisung_Wh_pro_Stunde: list[Optional[float]] = Field( - description="The energy fed into the grid in watt-hours per hour." - ) - Verluste_Pro_Stunde: list[Optional[float]] = Field( - description="The losses in watt-hours per hour." - ) - akku_soc_pro_stunde: list[Optional[float]] = Field( - description="The state of charge of the battery (not the EV) in percentage per hour." - ) - Electricity_price: list[Optional[float]] = Field( - description="Used Electricity Price, including predictions" - ) - - @field_validator( - "Last_Wh_pro_Stunde", - "Netzeinspeisung_Wh_pro_Stunde", - "akku_soc_pro_stunde", - "Netzbezug_Wh_pro_Stunde", - "Kosten_Euro_pro_Stunde", - "Einnahmen_Euro_pro_Stunde", - "EAuto_SoC_pro_Stunde", - "Verluste_Pro_Stunde", - "Home_appliance_wh_per_hour", - "Electricity_price", - mode="before", - ) - def convert_numpy(cls, field: Any) -> Any: - return NumpyEncoder.convert_numpy(field)[0] +# The executor to execute the CPU heavy energy management run +executor = ThreadPoolExecutor(max_workers=1) class EnergyManagement(SingletonMixin, ConfigMixin, PredictionMixin, PydanticBaseModel): - # Disable validation on assignment to speed up simulation runs. - model_config = ConfigDict( - validate_assignment=False, - ) + """Energy management.""" # Start datetime. _start_datetime: ClassVar[Optional[DateTime]] = None # last run datetime. Used by energy management task - _last_datetime: ClassVar[Optional[DateTime]] = None + _last_run_datetime: ClassVar[Optional[DateTime]] = None + + # energy management plan of latest energy management run with optimization + _plan: ClassVar[Optional[EnergyManagementPlan]] = None + + # opimization solution of the latest energy management run + _optimization_solution: ClassVar[Optional[OptimizationSolution]] = None + + # Solution of the genetic algorithm of latest energy management run with optimization + # For classic API + _genetic_solution: ClassVar[Optional[GeneticSolution]] = None + + # energy management lock (for energy management run) + _run_lock: ClassVar[Lock] = Lock() @computed_field # type: ignore[prop-decorator] @property @@ -127,9 +54,15 @@ class EnergyManagement(SingletonMixin, ConfigMixin, PredictionMixin, PydanticBas EnergyManagement.set_start_datetime() return EnergyManagement._start_datetime + @computed_field # type: ignore[prop-decorator] + @property + def last_run_datetime(self) -> Optional[DateTime]: + """The datetime the last energy management was run.""" + return EnergyManagement._last_run_datetime + @classmethod def set_start_datetime(cls, start_datetime: Optional[DateTime] = None) -> DateTime: - """Set the start datetime for the next energy management cycle. + """Set the start datetime for the next energy management run. If no datetime is provided, the current datetime is used. @@ -148,142 +81,208 @@ class EnergyManagement(SingletonMixin, ConfigMixin, PredictionMixin, PydanticBas cls._start_datetime = start_datetime.set(minute=0, second=0, microsecond=0) return cls._start_datetime - # ------------------------- - # TODO: Take from prediction - # ------------------------- + @classmethod + def plan(cls) -> Optional[EnergyManagementPlan]: + """Get the latest energy management plan. - load_energy_array: Optional[NDArray[Shape["*"], float]] = Field( - default=None, - description="An array of floats representing the total load (consumption) in watts for different time intervals.", - ) - pv_prediction_wh: Optional[NDArray[Shape["*"], float]] = Field( - default=None, - description="An array of floats representing the forecasted photovoltaic output in watts for different time intervals.", - ) - elect_price_hourly: Optional[NDArray[Shape["*"], float]] = Field( - default=None, - description="An array of floats representing the electricity price in euros per watt-hour for different time intervals.", - ) - elect_revenue_per_hour_arr: Optional[NDArray[Shape["*"], float]] = Field( - default=None, - description="An array of floats representing the feed-in compensation in euros per watt-hour.", - ) + Returns: + Optional[EnergyManagementPlan]: The latest energy management plan or None. + """ + return cls._plan - # ------------------------- - # TODO: Move to devices - # ------------------------- + @classmethod + def optimization_solution(cls) -> Optional[OptimizationSolution]: + """Get the latest optimization solution. - battery: Optional[Battery] = Field(default=None, description="TBD.") - ev: Optional[Battery] = Field(default=None, description="TBD.") - home_appliance: Optional[HomeAppliance] = Field(default=None, description="TBD.") - inverter: Optional[Inverter] = Field(default=None, description="TBD.") + Returns: + Optional[OptimizationSolution]: The latest optimization solution. + """ + return cls._optimization_solution - # ------------------------- - # TODO: Move to devices - # ------------------------- + @classmethod + def genetic_solution(cls) -> Optional[GeneticSolution]: + """Get the latest solution of the genetic algorithm. - ac_charge_hours: Optional[NDArray[Shape["*"], float]] = Field(default=None, description="TBD") - dc_charge_hours: Optional[NDArray[Shape["*"], float]] = Field(default=None, description="TBD") - ev_charge_hours: Optional[NDArray[Shape["*"], float]] = Field(default=None, description="TBD") + Returns: + Optional[GeneticSolution]: The latest solution of the genetic algorithm. + """ + return cls._genetic_solution - def __init__(self, *args: Any, **kwargs: Any) -> None: - if hasattr(self, "_initialized"): - return - super().__init__(*args, **kwargs) - - def set_parameters( - self, - parameters: EnergyManagementParameters, - ev: Optional[Battery] = None, - home_appliance: Optional[HomeAppliance] = None, - inverter: Optional[Inverter] = None, - ) -> None: - self.load_energy_array = np.array(parameters.gesamtlast, float) - self.pv_prediction_wh = np.array(parameters.pv_prognose_wh, float) - self.elect_price_hourly = np.array(parameters.strompreis_euro_pro_wh, float) - self.elect_revenue_per_hour_arr = ( - parameters.einspeiseverguetung_euro_pro_wh - if isinstance(parameters.einspeiseverguetung_euro_pro_wh, list) - else np.full( - len(self.load_energy_array), parameters.einspeiseverguetung_euro_pro_wh, float - ) - ) - if inverter: - self.battery = inverter.battery - else: - self.battery = None - self.ev = ev - self.home_appliance = home_appliance - self.inverter = inverter - self.ac_charge_hours = np.full(self.config.prediction.hours, 0.0) - self.dc_charge_hours = np.full(self.config.prediction.hours, 1.0) - self.ev_charge_hours = np.full(self.config.prediction.hours, 0.0) - - def set_akku_discharge_hours(self, ds: np.ndarray) -> None: - if self.battery: - self.battery.set_discharge_per_hour(ds) - - def set_akku_ac_charge_hours(self, ds: np.ndarray) -> None: - self.ac_charge_hours = ds - - def set_akku_dc_charge_hours(self, ds: np.ndarray) -> None: - self.dc_charge_hours = ds - - def set_ev_charge_hours(self, ds: np.ndarray) -> None: - self.ev_charge_hours = ds - - def set_home_appliance_start(self, ds: int, global_start_hour: int = 0) -> None: - if self.home_appliance: - self.home_appliance.set_starting_time(ds, global_start_hour=global_start_hour) - - def reset(self) -> None: - if self.ev: - self.ev.reset() - if self.battery: - self.battery.reset() - - def run( - self, - start_hour: Optional[int] = None, + @classmethod + def _run( + cls, + start_datetime: Optional[DateTime] = None, + mode: Optional[EnergyManagementMode] = None, + genetic_parameters: Optional[GeneticOptimizationParameters] = None, + genetic_individuals: Optional[int] = None, + genetic_seed: Optional[int] = None, force_enable: Optional[bool] = False, force_update: Optional[bool] = False, ) -> None: - """Run energy management. + """Run the energy management. - Sets `start_datetime` to current hour, updates the configuration and the prediction, and - starts simulation at current hour. + This method initializes the energy management run by setting its + start datetime, updating predictions, and optionally starting + optimization depending on the selected mode or configuration. Args: - start_hour (int, optional): Hour to take as start time for the energy management. Defaults - to now. - force_enable (bool, optional): If True, forces to update even if disabled. This - is mostly relevant to prediction providers. - force_update (bool, optional): If True, forces to update the data even if still cached. + start_datetime (DateTime, optional): The starting timestamp + of the energy management run. Defaults to the current datetime + if not provided. + mode (EnergyManagementMode, optional): The management mode to use. Must be one of: + - "OPTIMIZATION": Runs the optimization process. + - "PREDICTION": Updates the forecast without optimization. + + Defaults to the mode defined in the current configuration. + genetic_parameters (GeneticOptimizationParameters, optional): The + parameter set for the genetic algorithm. If not provided, it will + be constructed based on the current configuration and predictions. + genetic_individuals (int, optional): The number of individuals for the + genetic algorithm. Defaults to the algorithm's internal default (400) + if not specified. + genetic_seed (int, optional): The seed for the genetic algorithm. Defaults + to the algorithm's internal random seed if not specified. + force_enable (bool, optional): If True, bypasses any disabled state + to force the update process. This is mostly applicable to + prediction providers. + force_update (bool, optional): If True, forces data to be refreshed + even if a cached version is still valid. + + Returns: + None """ - # Throw away any cached results of the last run. - CacheUntilUpdateStore().clear() - self.set_start_hour(start_hour=start_hour) + # Ensure there is only one optimization/ energy management run at a time + if mode not in (None, "PREDICTION", "OPTIMIZATION"): + raise ValueError(f"Unknown energy management mode {mode}.") - # Check for run definitions - if self.start_datetime is None: - error_msg = "Start datetime unknown." - logger.error(error_msg) - raise ValueError(error_msg) - if self.config.prediction.hours is None: - error_msg = "Prediction hours unknown." - logger.error(error_msg) - raise ValueError(error_msg) - if self.config.optimization.hours is None: - error_msg = "Optimization hours unknown." - logger.error(error_msg) - raise ValueError(error_msg) + logger.info("Starting energy management run.") - self.prediction.update_data(force_enable=force_enable, force_update=force_update) - # TODO: Create optimisation problem that calls into devices.update_data() for simulations. + # Remember/ set the start datetime of this energy management run. + # None leads + cls.set_start_datetime(start_datetime) - logger.info("Energy management run (crippled version - prediction update only)") + # Throw away any memory cached results of the last energy management run. + CacheEnergyManagementStore().clear() - def manage_energy(self) -> None: + if mode is None: + mode = cls.config.ems.mode + if mode is None or mode == "PREDICTION": + # Update the predictions + cls.prediction.update_data(force_enable=force_enable, force_update=force_update) + logger.info("Energy management run done (predictions updated)") + return + + # Prepare optimization parameters + # This also creates default configurations for missing values and updates the predictions + logger.info( + "Starting energy management prediction update and optimzation parameter preparation." + ) + if genetic_parameters is None: + genetic_parameters = GeneticOptimizationParameters.prepare() + + if not genetic_parameters: + logger.error( + "Energy management run canceled. Could not prepare optimisation parameters." + ) + return + + # Take values from config if not given + if genetic_individuals is None: + genetic_individuals = cls.config.optimization.genetic.individuals + if genetic_seed is None: + genetic_seed = cls.config.optimization.genetic.seed + + if cls._start_datetime is None: # Make mypy happy - already set by us + raise RuntimeError("Start datetime not set.") + + logger.info("Starting energy management optimization.") + try: + optimization = GeneticOptimization( + verbose=bool(cls.config.server.verbose), + fixed_seed=genetic_seed, + ) + solution = optimization.optimierung_ems( + start_hour=cls._start_datetime.hour, + parameters=genetic_parameters, + ngen=genetic_individuals, + ) + except: + logger.exception("Energy management optimization failed.") + return + + # Make genetic solution public + cls._genetic_solution = solution + + # Make optimization solution public + cls._optimization_solution = solution.optimization_solution() + + # Make plan public + cls._plan = solution.energy_management_plan() + + logger.debug("Energy management genetic solution:\n{}", cls._genetic_solution) + logger.debug("Energy management optimization solution:\n{}", cls._optimization_solution) + logger.debug("Energy management plan:\n{}", cls._plan) + logger.info("Energy management run done (optimization updated)") + + async def run( + self, + start_datetime: Optional[DateTime] = None, + mode: Optional[EnergyManagementMode] = None, + genetic_parameters: Optional[GeneticOptimizationParameters] = None, + genetic_individuals: Optional[int] = None, + genetic_seed: Optional[int] = None, + force_enable: Optional[bool] = False, + force_update: Optional[bool] = False, + ) -> None: + """Run the energy management. + + This method initializes the energy management run by setting its + start datetime, updating predictions, and optionally starting + optimization depending on the selected mode or configuration. + + Args: + start_datetime (DateTime, optional): The starting timestamp + of the energy management run. Defaults to the current datetime + if not provided. + mode (EnergyManagementMode, optional): The management mode to use. Must be one of: + - "OPTIMIZATION": Runs the optimization process. + - "PREDICTION": Updates the forecast without optimization. + + Defaults to the mode defined in the current configuration. + genetic_parameters (GeneticOptimizationParameters, optional): The + parameter set for the genetic algorithm. If not provided, it will + be constructed based on the current configuration and predictions. + genetic_individuals (int, optional): The number of individuals for the + genetic algorithm. Defaults to the algorithm's internal default (400) + if not specified. + genetic_seed (int, optional): The seed for the genetic algorithm. Defaults + to the algorithm's internal random seed if not specified. + force_enable (bool, optional): If True, bypasses any disabled state + to force the update process. This is mostly applicable to + prediction providers. + force_update (bool, optional): If True, forces data to be refreshed + even if a cached version is still valid. + + Returns: + None + """ + async with self._run_lock: + loop = get_running_loop() + # Create a partial function with parameters "baked in" + func = partial( + EnergyManagement._run, + start_datetime=start_datetime, + mode=mode, + genetic_parameters=genetic_parameters, + genetic_individuals=genetic_individuals, + genetic_seed=genetic_seed, + force_enable=force_enable, + force_update=force_update, + ) + # Run optimization in background thread to avoid blocking event loop + await loop.run_in_executor(executor, func) + + async def manage_energy(self) -> None: """Repeating task for managing energy. This task should be executed by the server regularly (e.g., every 10 seconds) @@ -304,13 +303,13 @@ class EnergyManagement(SingletonMixin, ConfigMixin, PredictionMixin, PydanticBas current_datetime = to_datetime() interval = self.config.ems.interval # interval maybe changed in between - if EnergyManagement._last_datetime is None: + if EnergyManagement._last_run_datetime is None: # Never run before try: # Remember energy run datetime. - EnergyManagement._last_datetime = current_datetime + EnergyManagement._last_run_datetime = current_datetime # Try to run a first energy management. May fail due to config incomplete. - self.run() + await self.run() except Exception as e: trace = "".join(traceback.TracebackException.from_exception(e).format()) message = f"EOS init: {e}\n{trace}" @@ -322,14 +321,14 @@ class EnergyManagement(SingletonMixin, ConfigMixin, PredictionMixin, PydanticBas return if ( - compare_datetimes(current_datetime, EnergyManagement._last_datetime).time_diff + compare_datetimes(current_datetime, EnergyManagement._last_run_datetime).time_diff < interval ): # Wait for next run return try: - self.run() + await self.run() except Exception as e: trace = "".join(traceback.TracebackException.from_exception(e).format()) message = f"EOS run: {e}\n{trace}" @@ -337,187 +336,13 @@ class EnergyManagement(SingletonMixin, ConfigMixin, PredictionMixin, PydanticBas # Remember the energy management run - keep on interval even if we missed some intervals while ( - compare_datetimes(current_datetime, EnergyManagement._last_datetime).time_diff + compare_datetimes(current_datetime, EnergyManagement._last_run_datetime).time_diff >= interval ): - EnergyManagement._last_datetime = EnergyManagement._last_datetime.add(seconds=interval) - - def set_start_hour(self, start_hour: Optional[int] = None) -> None: - """Sets start datetime to given hour. - - Args: - start_hour (int, optional): Hour to take as start time for the energy management. Defaults - to now. - """ - if start_hour is None: - self.set_start_datetime() - else: - start_datetime = to_datetime().set(hour=start_hour, minute=0, second=0, microsecond=0) - self.set_start_datetime(start_datetime) - - def simulate_start_now(self) -> dict[str, Any]: - start_hour = to_datetime().now().hour - return self.simulate(start_hour) - - def simulate(self, start_hour: int) -> dict[str, Any]: - """Simulate energy usage and costs for the given start hour. - - akku_soc_pro_stunde begin of the hour, initial hour state! - last_wh_pro_stunde integral of last hour (end state) - """ - # Check for simulation integrity - required_attrs = [ - "load_energy_array", - "pv_prediction_wh", - "elect_price_hourly", - "ev_charge_hours", - "ac_charge_hours", - "dc_charge_hours", - "elect_revenue_per_hour_arr", - ] - missing_data = [ - attr.replace("_", " ").title() for attr in required_attrs if getattr(self, attr) is None - ] - - if missing_data: - logger.error("Mandatory data missing - %s", ", ".join(missing_data)) - raise ValueError(f"Mandatory data missing: {', '.join(missing_data)}") - - # Pre-fetch data - load_energy_array = np.array(self.load_energy_array) - pv_prediction_wh = np.array(self.pv_prediction_wh) - elect_price_hourly = np.array(self.elect_price_hourly) - ev_charge_hours = np.array(self.ev_charge_hours) - ac_charge_hours = np.array(self.ac_charge_hours) - dc_charge_hours = np.array(self.dc_charge_hours) - elect_revenue_per_hour_arr = np.array(self.elect_revenue_per_hour_arr) - - # Fetch objects - battery = self.battery - if battery is None: - raise ValueError(f"battery not set: {battery}") - ev = self.ev - home_appliance = self.home_appliance - inverter = self.inverter - - if not (len(load_energy_array) == len(pv_prediction_wh) == len(elect_price_hourly)): - error_msg = f"Array sizes do not match: Load Curve = {len(load_energy_array)}, PV Forecast = {len(pv_prediction_wh)}, Electricity Price = {len(elect_price_hourly)}" - logger.error(error_msg) - raise ValueError(error_msg) - - end_hour = len(load_energy_array) - total_hours = end_hour - start_hour - - # Pre-allocate arrays for the results, optimized for speed - loads_energy_per_hour = np.full((total_hours), np.nan) - feedin_energy_per_hour = np.full((total_hours), np.nan) - consumption_energy_per_hour = np.full((total_hours), np.nan) - costs_per_hour = np.full((total_hours), np.nan) - revenue_per_hour = np.full((total_hours), np.nan) - soc_per_hour = np.full((total_hours), np.nan) - soc_ev_per_hour = np.full((total_hours), np.nan) - losses_wh_per_hour = np.full((total_hours), np.nan) - home_appliance_wh_per_hour = np.full((total_hours), np.nan) - electricity_price_per_hour = np.full((total_hours), np.nan) - - # Set initial state - soc_per_hour[0] = battery.current_soc_percentage() - if ev: - soc_ev_per_hour[0] = ev.current_soc_percentage() - - for hour in range(start_hour, end_hour): - hour_idx = hour - start_hour - - # save begin states - soc_per_hour[hour_idx] = battery.current_soc_percentage() - - if ev: - soc_ev_per_hour[hour_idx] = ev.current_soc_percentage() - - # Accumulate loads and PV generation - consumption = load_energy_array[hour] - losses_wh_per_hour[hour_idx] = 0.0 - - # Home appliances - if home_appliance: - ha_load = home_appliance.get_load_for_hour(hour) - consumption += ha_load - home_appliance_wh_per_hour[hour_idx] = ha_load - - # E-Auto handling - if ev and ev_charge_hours[hour] > 0: - loaded_energy_ev, verluste_eauto = ev.charge_energy( - None, hour, relative_power=ev_charge_hours[hour] - ) - consumption += loaded_energy_ev - losses_wh_per_hour[hour_idx] += verluste_eauto - - # Process inverter logic - energy_feedin_grid_actual = energy_consumption_grid_actual = losses = eigenverbrauch = ( - 0.0 + EnergyManagement._last_run_datetime = EnergyManagement._last_run_datetime.add( + seconds=interval ) - hour_ac_charge = ac_charge_hours[hour] - hour_dc_charge = dc_charge_hours[hour] - hourly_electricity_price = elect_price_hourly[hour] - hourly_energy_revenue = elect_revenue_per_hour_arr[hour] - - battery.set_charge_allowed_for_hour(hour_dc_charge, hour) - - if inverter: - energy_produced = pv_prediction_wh[hour] - ( - energy_feedin_grid_actual, - energy_consumption_grid_actual, - losses, - eigenverbrauch, - ) = inverter.process_energy(energy_produced, consumption, hour) - - # AC PV Battery Charge - if hour_ac_charge > 0.0: - battery.set_charge_allowed_for_hour(1, hour) - battery_charged_energy_actual, battery_losses_actual = battery.charge_energy( - None, hour, relative_power=hour_ac_charge - ) - - total_battery_energy = battery_charged_energy_actual + battery_losses_actual - consumption += total_battery_energy - energy_consumption_grid_actual += total_battery_energy - losses_wh_per_hour[hour_idx] += battery_losses_actual - - # Update hourly arrays - feedin_energy_per_hour[hour_idx] = energy_feedin_grid_actual - consumption_energy_per_hour[hour_idx] = energy_consumption_grid_actual - losses_wh_per_hour[hour_idx] += losses - loads_energy_per_hour[hour_idx] = consumption - electricity_price_per_hour[hour_idx] = hourly_electricity_price - - # Financial calculations - costs_per_hour[hour_idx] = energy_consumption_grid_actual * hourly_electricity_price - revenue_per_hour[hour_idx] = energy_feedin_grid_actual * hourly_energy_revenue - - total_cost = np.nansum(costs_per_hour) - total_losses = np.nansum(losses_wh_per_hour) - total_revenue = np.nansum(revenue_per_hour) - - # Prepare output dictionary - return { - "Last_Wh_pro_Stunde": loads_energy_per_hour, - "Netzeinspeisung_Wh_pro_Stunde": feedin_energy_per_hour, - "Netzbezug_Wh_pro_Stunde": consumption_energy_per_hour, - "Kosten_Euro_pro_Stunde": costs_per_hour, - "akku_soc_pro_stunde": soc_per_hour, - "Einnahmen_Euro_pro_Stunde": revenue_per_hour, - "Gesamtbilanz_Euro": total_cost - total_revenue, - "EAuto_SoC_pro_Stunde": soc_ev_per_hour, - "Gesamteinnahmen_Euro": total_revenue, - "Gesamtkosten_Euro": total_cost, - "Verluste_Pro_Stunde": losses_wh_per_hour, - "Gesamt_Verluste": total_losses, - "Home_appliance_wh_per_hour": home_appliance_wh_per_hour, - "Electricity_price": electricity_price_per_hour, - } - # Initialize the Energy Management System, it is a singleton. ems = EnergyManagement() diff --git a/src/akkudoktoreos/core/emsettings.py b/src/akkudoktoreos/core/emsettings.py index 94cb1e9..99b12bc 100644 --- a/src/akkudoktoreos/core/emsettings.py +++ b/src/akkudoktoreos/core/emsettings.py @@ -3,6 +3,7 @@ Kept in an extra module to avoid cyclic dependencies on package import. """ +from enum import Enum from typing import Optional from pydantic import Field @@ -10,6 +11,13 @@ from pydantic import Field from akkudoktoreos.config.configabc import SettingsBaseModel +class EnergyManagementMode(str, Enum): + """Energy management mode.""" + + PREDICTION = "PREDICTION" + OPTIMIZATION = "OPTIMIZATION" + + class EnergyManagementCommonSettings(SettingsBaseModel): """Energy Management Configuration.""" @@ -24,3 +32,9 @@ class EnergyManagementCommonSettings(SettingsBaseModel): description="Intervall in seconds between EOS energy management runs.", examples=["300"], ) + + mode: Optional[EnergyManagementMode] = Field( + default=None, + description="Energy management mode [OPTIMIZATION | PREDICTION].", + examples=["OPTIMIZATION", "PREDICTION"], + ) diff --git a/src/akkudoktoreos/core/logging.py b/src/akkudoktoreos/core/logging.py index d27b4f2..1ae0df5 100644 --- a/src/akkudoktoreos/core/logging.py +++ b/src/akkudoktoreos/core/logging.py @@ -42,6 +42,10 @@ class InterceptHandler(pylogging.Handler): Args: record (logging.LogRecord): A record object containing log message and metadata. """ + # Skip DEBUG logs from matplotlib - very noisy + if record.name.startswith("matplotlib") and record.levelno <= pylogging.DEBUG: + return + try: level = logger.level(record.levelname).name except AttributeError: diff --git a/src/akkudoktoreos/core/logsettings.py b/src/akkudoktoreos/core/logsettings.py index d2ec48a..0eeab29 100644 --- a/src/akkudoktoreos/core/logsettings.py +++ b/src/akkudoktoreos/core/logsettings.py @@ -15,11 +15,6 @@ from akkudoktoreos.core.logabc import LOGGING_LEVELS class LoggingCommonSettings(SettingsBaseModel): """Logging Configuration.""" - level: Optional[str] = Field( - default=None, - deprecated="This is deprecated. Use console_level and file_level instead.", - ) - console_level: Optional[str] = Field( default=None, description="Logging level when logging to console.", diff --git a/src/akkudoktoreos/core/pydantic.py b/src/akkudoktoreos/core/pydantic.py index ce072e1..b3bddd7 100644 --- a/src/akkudoktoreos/core/pydantic.py +++ b/src/akkudoktoreos/core/pydantic.py @@ -14,7 +14,6 @@ Key Features: import inspect import json -import re import uuid import weakref from copy import deepcopy @@ -32,23 +31,20 @@ from typing import ( from zoneinfo import ZoneInfo import pandas as pd -import pendulum from loguru import logger from pandas.api.types import is_datetime64_any_dtype from pydantic import ( - AwareDatetime, BaseModel, ConfigDict, Field, PrivateAttr, RootModel, - TypeAdapter, ValidationError, ValidationInfo, field_validator, ) -from akkudoktoreos.utils.datetimeutil import to_datetime, to_duration +from akkudoktoreos.utils.datetimeutil import DateTime, to_datetime, to_duration # Global weakref dictionary to hold external state per model instance # Used as a workaround for PrivateAttr not working in e.g. Mixin Classes @@ -56,49 +52,40 @@ _model_private_state: "weakref.WeakKeyDictionary[Union[PydanticBaseModel, Pydant def merge_models(source: BaseModel, update_dict: dict[str, Any]) -> dict[str, Any]: - def deep_update(source_dict: dict[str, Any], update_dict: dict[str, Any]) -> dict[str, Any]: - for key, value in source_dict.items(): - if isinstance(value, dict) and isinstance(update_dict.get(key), dict): - update_dict[key] = deep_update(update_dict[key], value) - else: - update_dict[key] = value - return update_dict + """Merge a Pydantic model instance with an update dictionary. + + Values in update_dict (including None) override source values. + Nested dictionaries are merged recursively. + Lists in update_dict replace source lists entirely. + + Args: + source (BaseModel): Pydantic model instance serving as the source. + update_dict (dict[str, Any]): Dictionary with updates to apply. + + Returns: + dict[str, Any]: Merged dictionary representing combined model data. + """ + + def deep_merge(source_data: Any, update_data: Any) -> Any: + if isinstance(source_data, dict) and isinstance(update_data, dict): + merged = dict(source_data) + for key, update_value in update_data.items(): + if key in merged: + merged[key] = deep_merge(merged[key], update_value) + else: + merged[key] = update_value + return merged + + # If both are lists, replace source list with update list + if isinstance(source_data, list) and isinstance(update_data, list): + return update_data + + # For other types or if update_data is None, override source_data + return update_data source_dict = source.model_dump(exclude_unset=True) - merged_dict = deep_update(source_dict, deepcopy(update_dict)) - - return merged_dict - - -class PydanticTypeAdapterDateTime(TypeAdapter[pendulum.DateTime]): - """Custom type adapter for Pendulum DateTime fields.""" - - @classmethod - def serialize(cls, value: Any) -> str: - """Convert pendulum.DateTime to ISO 8601 string.""" - if isinstance(value, pendulum.DateTime): - return value.to_iso8601_string() - raise ValueError(f"Expected pendulum.DateTime, got {type(value)}") - - @classmethod - def deserialize(cls, value: Any) -> pendulum.DateTime: - """Convert ISO 8601 string to pendulum.DateTime.""" - if isinstance(value, str) and cls.is_iso8601(value): - try: - return pendulum.parse(value) - except pendulum.parsing.exceptions.ParserError as e: - raise ValueError(f"Invalid date format: {value}") from e - elif isinstance(value, pendulum.DateTime): - return value - raise ValueError(f"Expected ISO 8601 string or pendulum.DateTime, got {type(value)}") - - @staticmethod - def is_iso8601(value: str) -> bool: - """Check if the string is a valid ISO 8601 date string.""" - iso8601_pattern = ( - r"^(\d{4}-\d{2}-\d{2}[T ]\d{2}:\d{2}:\d{2}(?:\.\d{1,3})?(?:Z|[+-]\d{2}:\d{2})?)$" - ) - return bool(re.match(iso8601_pattern, value)) + merged_result = deep_merge(source_dict, deepcopy(update_dict)) + return merged_result class PydanticModelNestedValueMixin: @@ -653,67 +640,9 @@ class PydanticBaseModel(PydanticModelNestedValueMixin, BaseModel): """ return hash(self._uuid) - @field_validator("*", mode="before") - def validate_and_convert_pendulum(cls, value: Any, info: ValidationInfo) -> Any: - """Validator to convert fields of type `pendulum.DateTime`. - - Converts fields to proper `pendulum.DateTime` objects, ensuring correct input types. - - This method is invoked for every field before the field value is set. If the field's type - is `pendulum.DateTime`, it tries to convert string or timestamp values to `pendulum.DateTime` - objects. If the value cannot be converted, a validation error is raised. - - Args: - value: The value to be assigned to the field. - info: Validation information for the field. - - Returns: - The converted value, if successful. - - Raises: - ValidationError: If the value cannot be converted to `pendulum.DateTime`. - """ - # Get the field name and expected type - field_name = info.field_name - expected_type = cls.model_fields[field_name].annotation - - # Convert - if expected_type is pendulum.DateTime or expected_type is AwareDatetime: - try: - value = to_datetime(value) - except Exception as e: - raise ValueError(f"Cannot convert {value!r} to datetime: {e}") - return value - - # Override Pydantic’s serialization for all DateTime fields - def model_dump( - self, *args: Any, include_computed_fields: bool = True, **kwargs: Any - ) -> dict[str, Any]: - """Custom dump method to handle serialization for DateTime fields.""" - result = super().model_dump(*args, **kwargs) - - if not include_computed_fields: - for computed_field_name in self.model_computed_fields: - result.pop(computed_field_name, None) - - for key, value in result.items(): - if isinstance(value, pendulum.DateTime): - result[key] = PydanticTypeAdapterDateTime.serialize(value) - return result - - @classmethod - def model_construct( - cls, _fields_set: set[str] | None = None, **values: Any - ) -> "PydanticBaseModel": - """Custom constructor to handle deserialization for DateTime fields.""" - for key, value in values.items(): - if isinstance(value, str) and PydanticTypeAdapterDateTime.is_iso8601(value): - values[key] = PydanticTypeAdapterDateTime.deserialize(value) - return super().model_construct(_fields_set, **values) - def reset_to_defaults(self) -> "PydanticBaseModel": """Resets the fields to their default values.""" - for field_name, field_info in self.model_fields.items(): + for field_name, field_info in self.__class__.model_fields.items(): if field_info.default_factory is not None: # Handle fields with default_factory default_value = field_info.default_factory() else: @@ -725,6 +654,19 @@ class PydanticBaseModel(PydanticModelNestedValueMixin, BaseModel): pass return self + # Override Pydantic’s serialization to include computed fields by default + def model_dump( + self, *args: Any, include_computed_fields: bool = True, **kwargs: Any + ) -> dict[str, Any]: + """Custom dump method to serialize computed fields by default.""" + result = super().model_dump(*args, **kwargs) + + if not include_computed_fields: + for computed_field_name in self.__class__.model_computed_fields: + result.pop(computed_field_name, None) + + return result + def to_dict(self) -> dict: """Convert this PredictionRecord instance to a dictionary representation. @@ -910,16 +852,27 @@ class PydanticDateTimeDataFrame(PydanticBaseModel): if not v: return v - valid_dtypes = {"int64", "float64", "bool", "datetime64[ns]", "object", "string"} - invalid_dtypes = set(v.values()) - valid_dtypes - if invalid_dtypes: - raise ValueError(f"Unsupported dtypes: {invalid_dtypes}") + # Allowed exact dtypes + valid_base_dtypes = {"int64", "float64", "bool", "object", "string"} + def is_valid_dtype(dtype: str) -> bool: + # Allow timezone-aware or naive datetime64 + if dtype.startswith("datetime64[ns"): + return True + return dtype in valid_base_dtypes + + invalid_dtypes = [dtype for dtype in v.values() if not is_valid_dtype(dtype)] + if invalid_dtypes: + raise ValueError(f"Unsupported dtypes: {set(invalid_dtypes)}") + + # Cross-check with data column existence data = info.data.get("data", {}) if data: columns = set(next(iter(data.values())).keys()) - if not all(col in columns for col in v.keys()): - raise ValueError("dtype columns must exist in data columns") + missing_columns = set(v.keys()) - columns + if missing_columns: + raise ValueError(f"dtype columns must exist in data columns: {missing_columns}") + return v def to_dataframe(self) -> pd.DataFrame: @@ -927,7 +880,8 @@ class PydanticDateTimeDataFrame(PydanticBaseModel): df = pd.DataFrame.from_dict(self.data, orient="index") # Convert index to datetime - index = pd.Index([to_datetime(dt, in_timezone=self.tz) for dt in df.index]) + # index = pd.Index([to_datetime(dt, in_timezone=self.tz) for dt in df.index]) + index = [to_datetime(dt, in_timezone=self.tz) for dt in df.index] df.index = index # Check if 'date_time' column exists, if not, create it @@ -943,8 +897,8 @@ class PydanticDateTimeDataFrame(PydanticBaseModel): # Apply dtypes for col, dtype in self.dtypes.items(): - if dtype == "datetime64[ns]": - df[col] = pd.to_datetime(to_datetime(df[col], in_timezone=self.tz)) + if dtype.startswith("datetime64[ns"): + df[col] = pd.to_datetime(df[col], utc=True) elif dtype in dtype_mapping.keys(): df[col] = df[col].astype(dtype_mapping[dtype]) else: @@ -969,6 +923,132 @@ class PydanticDateTimeDataFrame(PydanticBaseModel): datetime_columns=datetime_columns, ) + # --- Direct Manipulation Methods --- + + def _normalize_index(self, index: str | DateTime) -> str: + """Normalize index into timezone-aware datetime string. + + Args: + index (str | DateTime): A datetime-like value. + + Returns: + str: Normalized datetime string based on model timezone. + """ + return to_datetime(index, as_string=True, in_timezone=self.tz) + + def add_row(self, index: str | DateTime, row: Dict[str, Any]) -> None: + """Add a new row to the dataset. + + Args: + index (str | DateTime): Timestamp of the new row. + row (Dict[str, Any]): Dictionary of column values. Must match existing columns. + + Raises: + ValueError: If row does not contain the exact same columns as existing rows. + """ + idx = self._normalize_index(index) + + if self.data: + existing_cols = set(next(iter(self.data.values())).keys()) + if set(row.keys()) != existing_cols: + raise ValueError(f"Row must have exactly these columns: {existing_cols}") + self.data[idx] = row + + def update_row(self, index: str | DateTime, updates: Dict[str, Any]) -> None: + """Update values for an existing row. + + Args: + index (str | DateTime): Timestamp of the row to modify. + updates (Dict[str, Any]): Key/value pairs of columns to update. + + Raises: + KeyError: If row or column does not exist. + """ + idx = self._normalize_index(index) + if idx not in self.data: + raise KeyError(f"Row {idx} not found") + for col, value in updates.items(): + if col not in self.data[idx]: + raise KeyError(f"Column {col} does not exist") + self.data[idx][col] = value + + def delete_row(self, index: str | DateTime) -> None: + """Delete a row from the dataset. + + Args: + index (str | DateTime): Timestamp of the row to delete. + """ + idx = self._normalize_index(index) + if idx in self.data: + del self.data[idx] + + def set_value(self, index: str | DateTime, column: str, value: Any) -> None: + """Set a single cell value. + + Args: + index (str | datetime): Timestamp of the row. + column (str): Column name. + value (Any): New value. + """ + self.update_row(index, {column: value}) + + def get_value(self, index: str | DateTime, column: str) -> Any: + """Retrieve a single cell value. + + Args: + index (str | DateTime): Timestamp of the row. + column (str): Column name. + + Returns: + Any: Value stored at the given location. + """ + idx = self._normalize_index(index) + return self.data[idx][column] + + def add_column(self, name: str, default: Any = None, dtype: Optional[str] = None) -> None: + """Add a new column to all rows. + + Args: + name (str): Name of the column to add. + default (Any, optional): Default value for all rows. Defaults to None. + dtype (Optional[str], optional): Declared data type. Defaults to None. + """ + for row in self.data.values(): + row[name] = default + if dtype: + self.dtypes[name] = dtype + + def rename_column(self, old: str, new: str) -> None: + """Rename a column across all rows. + + Args: + old (str): Existing column name. + new (str): New column name. + + Raises: + KeyError: If column does not exist. + """ + for row in self.data.values(): + if old not in row: + raise KeyError(f"Column {old} does not exist") + row[new] = row.pop(old) + if old in self.dtypes: + self.dtypes[new] = self.dtypes.pop(old) + if old in self.datetime_columns: + self.datetime_columns = [new if c == old else c for c in self.datetime_columns] + + def drop_column(self, name: str) -> None: + """Remove a column from all rows. + + Args: + name (str): Column to remove. + """ + for row in self.data.values(): + if name in row: + del row[name] + self.dtypes.pop(name, None) + self.datetime_columns = [c for c in self.datetime_columns if c != name] + class PydanticDateTimeSeries(PydanticBaseModel): """Pydantic model for validating pandas Series with datetime index in JSON format. @@ -1068,10 +1148,6 @@ class PydanticDateTimeSeries(PydanticBaseModel): ) -class ParametersBaseModel(PydanticBaseModel): - model_config = ConfigDict(extra="forbid") - - def set_private_attr( model: Union[PydanticBaseModel, PydanticModelNestedValueMixin], key: str, value: Any ) -> None: diff --git a/src/akkudoktoreos/core/version.py b/src/akkudoktoreos/core/version.py new file mode 100644 index 0000000..9064d6d --- /dev/null +++ b/src/akkudoktoreos/core/version.py @@ -0,0 +1,5 @@ +"""Version information for akkudoktoreos.""" + +# For development add `+dev` to previous release +# For release omit `+dev`. +__version__ = "0.1.0+dev" diff --git a/src/akkudoktoreos/data/default.config.json b/src/akkudoktoreos/data/default.config.json index 2c63c08..9e3bc94 100644 --- a/src/akkudoktoreos/data/default.config.json +++ b/src/akkudoktoreos/data/default.config.json @@ -1,2 +1,5 @@ { + "general": { + "version": "0.1.0+dev" + } } diff --git a/src/akkudoktoreos/devices/devices.py b/src/akkudoktoreos/devices/devices.py index ff3d38d..c6da249 100644 --- a/src/akkudoktoreos/devices/devices.py +++ b/src/akkudoktoreos/devices/devices.py @@ -1,49 +1,376 @@ -from typing import Optional +"""General configuration settings for simulated devices for optimization.""" -from akkudoktoreos.core.coreabc import SingletonMixin -from akkudoktoreos.devices.battery import Battery -from akkudoktoreos.devices.devicesabc import DevicesBase -from akkudoktoreos.devices.generic import HomeAppliance -from akkudoktoreos.devices.inverter import Inverter -from akkudoktoreos.devices.settings import DevicesCommonSettings +import json +from typing import Any, Optional, TextIO, cast + +from loguru import logger +from pydantic import Field, computed_field, model_validator + +from akkudoktoreos.config.configabc import SettingsBaseModel +from akkudoktoreos.core.cache import CacheFileStore +from akkudoktoreos.core.coreabc import ConfigMixin, SingletonMixin +from akkudoktoreos.core.emplan import ResourceStatus +from akkudoktoreos.core.pydantic import ConfigDict, PydanticBaseModel +from akkudoktoreos.devices.devicesabc import DevicesBaseSettings +from akkudoktoreos.utils.datetimeutil import DateTime, TimeWindowSequence, to_datetime -class Devices(SingletonMixin, DevicesBase): - def __init__(self, settings: Optional[DevicesCommonSettings] = None): - if hasattr(self, "_initialized"): - return - super().__init__() - if settings is None: - settings = self.config.devices - if settings is None: - return +class BatteriesCommonSettings(DevicesBaseSettings): + """Battery devices base settings.""" - # initialize devices - if settings.batteries is not None: - for battery_params in settings.batteries: - self.add_device(Battery(battery_params)) - if settings.inverters is not None: - for inverter_params in settings.inverters: - self.add_device(Inverter(inverter_params)) - if settings.home_appliances is not None: - for home_appliance_params in settings.home_appliances: - self.add_device(HomeAppliance(home_appliance_params)) + capacity_wh: int = Field( + default=8000, + gt=0, + description="Capacity [Wh].", + examples=[8000], + ) - self.post_setup() + charging_efficiency: float = Field( + default=0.88, + gt=0, + le=1, + description="Charging efficiency [0.01 ... 1.00].", + examples=[0.88], + ) - def post_setup(self) -> None: - for device in self.devices.values(): - device.post_setup() + discharging_efficiency: float = Field( + default=0.88, + gt=0, + le=1, + description="Discharge efficiency [0.01 ... 1.00].", + examples=[0.88], + ) + + levelized_cost_of_storage_kwh: float = Field( + default=0.0, + description="Levelized cost of storage (LCOS), the average lifetime cost of delivering one kWh [€/kWh].", + examples=[0.12], + ) + + max_charge_power_w: Optional[float] = Field( + default=5000, + gt=0, + description="Maximum charging power [W].", + examples=[5000], + ) + + min_charge_power_w: Optional[float] = Field( + default=50, + gt=0, + description="Minimum charging power [W].", + examples=[50], + ) + + charge_rates: Optional[list[float]] = Field( + default=None, + description="Charge rates as factor of maximum charging power [0.00 ... 1.00]. None denotes all charge rates are available.", + examples=[[0.0, 0.25, 0.5, 0.75, 1.0], None], + ) + + min_soc_percentage: int = Field( + default=0, + ge=0, + le=100, + description="Minimum state of charge (SOC) as percentage of capacity [%].", + examples=[10], + ) + + max_soc_percentage: int = Field( + default=100, + ge=0, + le=100, + description="Maximum state of charge (SOC) as percentage of capacity [%].", + examples=[100], + ) + + @computed_field # type: ignore[prop-decorator] + @property + def measurement_key_soc_factor(self) -> str: + """Measurement key for the battery state of charge (SoC) as factor of total capacity [0.0 ... 1.0].""" + return f"{self.device_id}-soc-factor" + + @computed_field # type: ignore[prop-decorator] + @property + def measurement_key_power_l1_w(self) -> str: + """Measurement key for the L1 power the battery is charged or discharged with [W].""" + return f"{self.device_id}-power-l1-w" + + @computed_field # type: ignore[prop-decorator] + @property + def measurement_key_power_l2_w(self) -> str: + """Measurement key for the L2 power the battery is charged or discharged with [W].""" + return f"{self.device_id}-power-l2-w" + + @computed_field # type: ignore[prop-decorator] + @property + def measurement_key_power_l3_w(self) -> str: + """Measurement key for the L3 power the battery is charged or discharged with [W].""" + return f"{self.device_id}-power-l3-w" + + @computed_field # type: ignore[prop-decorator] + @property + def measurement_key_power_3_phase_sym_w(self) -> str: + """Measurement key for the symmetric 3 phase power the battery is charged or discharged with [W].""" + return f"{self.device_id}-power-3-phase-sym-w" + + @computed_field # type: ignore[prop-decorator] + @property + def measurement_keys(self) -> Optional[list[str]]: + """Measurement keys for the battery stati that are measurements. + + Battery SoC, power. + """ + keys: list[str] = [ + self.measurement_key_soc_factor, + self.measurement_key_power_l1_w, + self.measurement_key_power_l2_w, + self.measurement_key_power_l3_w, + self.measurement_key_power_3_phase_sym_w, + ] + return keys -# Initialize the Devices simulation, it is a singleton. -devices: Optional[Devices] = None +class InverterCommonSettings(DevicesBaseSettings): + """Inverter devices base settings.""" + + max_power_w: Optional[float] = Field( + default=None, + gt=0, + description="Maximum power [W].", + examples=[10000], + ) + + battery_id: Optional[str] = Field( + default=None, + description="ID of battery controlled by this inverter.", + examples=[None, "battery1"], + ) + + @computed_field # type: ignore[prop-decorator] + @property + def measurement_keys(self) -> Optional[list[str]]: + """Measurement keys for the inverter stati that are measurements.""" + keys: list[str] = [] + return keys -def get_devices() -> Devices: - global devices - # Fix circular import at runtime - if devices is None: - devices = Devices() - """Gets the EOS Devices simulation.""" - return devices +class HomeApplianceCommonSettings(DevicesBaseSettings): + """Home Appliance devices base settings.""" + + consumption_wh: int = Field( + gt=0, + description="Energy consumption [Wh].", + examples=[2000], + ) + + duration_h: int = Field( + gt=0, + le=24, + description="Usage duration in hours [0 ... 24].", + examples=[1], + ) + + time_windows: Optional[TimeWindowSequence] = Field( + default=None, + description="Sequence of allowed time windows. Defaults to optimization general time window.", + examples=[ + { + "windows": [ + {"start_time": "10:00", "duration": "2 hours"}, + ], + }, + ], + ) + + @computed_field # type: ignore[prop-decorator] + @property + def measurement_keys(self) -> Optional[list[str]]: + """Measurement keys for the home appliance stati that are measurements.""" + keys: list[str] = [] + return keys + + +class DevicesCommonSettings(SettingsBaseModel): + """Base configuration for devices simulation settings.""" + + batteries: Optional[list[BatteriesCommonSettings]] = Field( + default=None, + description="List of battery devices", + examples=[[{"device_id": "battery1", "capacity_wh": 8000}]], + ) + + max_batteries: Optional[int] = Field( + default=None, + ge=0, + description="Maximum number of batteries that can be set", + examples=[1, 2], + ) + + electric_vehicles: Optional[list[BatteriesCommonSettings]] = Field( + default=None, + description="List of electric vehicle devices", + examples=[[{"device_id": "battery1", "capacity_wh": 8000}]], + ) + + max_electric_vehicles: Optional[int] = Field( + default=None, + ge=0, + description="Maximum number of electric vehicles that can be set", + examples=[1, 2], + ) + + inverters: Optional[list[InverterCommonSettings]] = Field( + default=None, description="List of inverters", examples=[[]] + ) + + max_inverters: Optional[int] = Field( + default=None, + ge=0, + description="Maximum number of inverters that can be set", + examples=[1, 2], + ) + + home_appliances: Optional[list[HomeApplianceCommonSettings]] = Field( + default=None, description="List of home appliances", examples=[[]] + ) + + max_home_appliances: Optional[int] = Field( + default=None, + ge=0, + description="Maximum number of home_appliances that can be set", + examples=[1, 2], + ) + + @computed_field # type: ignore[prop-decorator] + @property + def measurement_keys(self) -> Optional[list[str]]: + """Return the measurement keys for the resource/ device stati that are measurements.""" + keys: list[str] = [] + + if self.max_batteries and self.batteries: + for battery in self.batteries: + keys.extend(battery.measurement_keys) + if self.max_electric_vehicles and self.electric_vehicles: + for electric_vehicle in self.electric_vehicles: + keys.extend(electric_vehicle.measurement_keys) + return keys + + +# Type used for indexing: (resource_id, optional actuator_id) +class ResourceKey(PydanticBaseModel): + """Key identifying a resource and optionally an actuator.""" + + resource_id: str + actuator_id: Optional[str] = None + + model_config = ConfigDict(frozen=True) + + def __hash__(self) -> int: + """Returns a stable hash based on the resource_id and actuator_id. + + Returns: + int: Hash value derived from the resource_id and actuator_id. + """ + return hash(self.resource_id + self.actuator_id if self.actuator_id else "") + + def as_tuple(self) -> tuple[str, Optional[str]]: + """Return the key as a tuple for internal dictionary indexing.""" + return (self.resource_id, self.actuator_id) + + def __eq__(self, other: Any) -> bool: + if not isinstance(other, ResourceKey): + return NotImplemented + return self.resource_id == other.resource_id and self.actuator_id == other.actuator_id + + +class ResourceRegistry(SingletonMixin, ConfigMixin, PydanticBaseModel): + """Registry for collecting and retrieving device status reports for simulations. + + Maintains the latest and optionally historical status reports for each resource. + """ + + keep_history: bool = False + history_size: int = 100 + + latest: dict[ResourceKey, ResourceStatus] = Field( + default_factory=dict, + description="Latest resource status that was reported per resource key.", + example=[], + ) + history: dict[ResourceKey, list[tuple[DateTime, ResourceStatus]]] = Field( + default_factory=dict, + description="History of resource stati that were reported per resource key.", + example=[], + ) + + @model_validator(mode="after") + def _enforce_history_limits(self) -> "ResourceRegistry": + """Ensure history list lengths respect the history_size limit.""" + if self.keep_history: + for key, records in self.history.items(): + if len(records) > self.history_size: + self.history[key] = records[-self.history_size :] + return self + + def update_status(self, key: ResourceKey, status: ResourceStatus) -> None: + """Update the latest status and optionally store in history. + + Args: + key (ResourceKey): Identifier for the resource. + status (ResourceStatus): Status report to store. + """ + self.latest[key] = status + if self.keep_history: + timestamp = getattr(status, "transition_timestamp", None) or to_datetime() + self.history.setdefault(key, []).append((timestamp, status)) + if len(self.history[key]) > self.history_size: + self.history[key] = self.history[key][-self.history_size :] + + def status_latest(self, key: ResourceKey) -> Optional[ResourceStatus]: + """Retrieve the most recent status for a resource.""" + return self.latest.get(key) + + def status_history(self, key: ResourceKey) -> list[tuple[DateTime, ResourceStatus]]: + """Retrieve historical status reports for a resource.""" + if not self.keep_history: + raise RuntimeError("History tracking is disabled.") + return self.history.get(key, []) + + def status_exists(self, key: ResourceKey) -> bool: + """Check if a status report exists for the given resource. + + Args: + key (ResourceKey): Identifier for the resource. + """ + return key in self.latest + + def save(self) -> None: + """Save the registry to file.""" + # Make explicit cast to make mypy happy + cache_file = cast( + TextIO, CacheFileStore().create(key="resource_registry", mode="w+", suffix=".json") + ) + cache_file.seek(0) + cache_file.write(self.model_dump_json(indent=4)) + cache_file.truncate() # Important to remove leftover data! + + def load(self) -> None: + """Load registry state from file and update the current instance.""" + cache_file = CacheFileStore().get(key="resource_registry") + if cache_file: + try: + cache_file.seek(0) + data = json.load(cache_file) + loaded = self.__class__.model_validate(data) + + self.keep_history = loaded.keep_history + self.history_size = loaded.history_size + self.latest = loaded.latest + self.history = loaded.history + except Exception as e: + logger.error("Can not load resource registry: {}", e) + + +def get_resource_registry() -> ResourceRegistry: + """Gets the EOS resource registry.""" + return ResourceRegistry() diff --git a/src/akkudoktoreos/devices/devicesabc.py b/src/akkudoktoreos/devices/devicesabc.py index a9599fd..f9a6b91 100644 --- a/src/akkudoktoreos/devices/devicesabc.py +++ b/src/akkudoktoreos/devices/devicesabc.py @@ -1,181 +1,131 @@ """Abstract and base classes for devices.""" -from enum import Enum -from typing import Optional, Type +from enum import StrEnum -from loguru import logger -from pendulum import DateTime -from pydantic import Field, computed_field +from pydantic import Field -from akkudoktoreos.core.coreabc import ( - ConfigMixin, - DevicesMixin, - EnergyManagementSystemMixin, - PredictionMixin, -) -from akkudoktoreos.core.pydantic import ParametersBaseModel -from akkudoktoreos.utils.datetimeutil import to_duration +from akkudoktoreos.config.configabc import SettingsBaseModel -class DeviceParameters(ParametersBaseModel): - device_id: str = Field(description="ID of device", examples="device1") - hours: Optional[int] = Field( - default=None, - gt=0, - description="Number of prediction hours. Defaults to global config prediction hours.", - examples=[None], +class DevicesBaseSettings(SettingsBaseModel): + """Base devices setting.""" + + device_id: str = Field( + default="", + description="ID of device", + examples=["battery1", "ev1", "inverter1", "dishwasher"], ) -class DeviceOptimizeResult(ParametersBaseModel): - device_id: str = Field(description="ID of device", examples=["device1"]) - hours: int = Field(gt=0, description="Number of hours in the simulation.", examples=[24]) +class BatteryOperationMode(StrEnum): + """Battery Operation Mode. + Enumerates the operating modes of a battery in a home energy + management simulation. These modes require no direct awareness + of electricity prices or carbon intensity — higher-level + controllers or optimizers decide when to switch modes. -class DeviceState(Enum): - UNINITIALIZED = 0 - PREPARED = 1 - INITIALIZED = 2 + Modes + ----- + - IDLE: + No charging or discharging. + - SELF_CONSUMPTION: + Charge from local surplus and discharge to meet local demand. -class DevicesStartEndMixin(ConfigMixin, EnergyManagementSystemMixin): - """A mixin to manage start, end datetimes for devices data. + - NON_EXPORT: + Charge from on-site or local surplus with the goal of + minimizing or preventing energy export to the external grid. + Discharging to the grid is not allowed. - The starting datetime for devices data generation is provided by the energy management - system. Device data cannot be computed if this value is `None`. + - PEAK_SHAVING: + Discharge during local demand peaks to reduce grid draw. + + - GRID_SUPPORT_EXPORT: + Discharge to support the upstream grid when commanded. + + - GRID_SUPPORT_IMPORT: + Charge from the grid when instructed to absorb excess supply. + + - FREQUENCY_REGULATION: + Perform fast bidirectional power adjustments based on grid + frequency deviations. + + - RAMP_RATE_CONTROL: + Smooth changes in local net load or generation. + + - RESERVE_BACKUP: + Maintain a minimum state of charge for emergency use. + + - OUTAGE_SUPPLY: + Discharge to power critical loads during a grid outage. + + - FORCED_CHARGE: + Override all other logic and charge regardless of conditions. + + - FORCED_DISCHARGE: + Override all other logic and discharge regardless of conditions. + + - FAULT: + Battery is unavailable due to fault or error state. """ - # Computed field for end_datetime and keep_datetime - @computed_field # type: ignore[prop-decorator] - @property - def end_datetime(self) -> Optional[DateTime]: - """Compute the end datetime based on the `start_datetime` and `hours`. - - Ajusts the calculated end time if DST transitions occur within the prediction window. - - Returns: - Optional[DateTime]: The calculated end datetime, or `None` if inputs are missing. - """ - if self.ems.start_datetime and self.config.prediction.hours: - end_datetime = self.ems.start_datetime + to_duration( - f"{self.config.prediction.hours} hours" - ) - dst_change = end_datetime.offset_hours - self.ems.start_datetime.offset_hours - logger.debug( - f"Pre: {self.ems.start_datetime}..{end_datetime}: DST change: {dst_change}" - ) - if dst_change < 0: - end_datetime = end_datetime + to_duration(f"{abs(int(dst_change))} hours") - elif dst_change > 0: - end_datetime = end_datetime - to_duration(f"{abs(int(dst_change))} hours") - logger.debug( - f"Pst: {self.ems.start_datetime}..{end_datetime}: DST change: {dst_change}" - ) - return end_datetime - return None - - @computed_field # type: ignore[prop-decorator] - @property - def total_hours(self) -> Optional[int]: - """Compute the hours from `start_datetime` to `end_datetime`. - - Returns: - Optional[pendulum.period]: The duration hours, or `None` if either datetime is unavailable. - """ - end_dt = self.end_datetime - if end_dt is None: - return None - duration = end_dt - self.ems.start_datetime - return int(duration.total_hours()) + IDLE = "IDLE" + SELF_CONSUMPTION = "SELF_CONSUMPTION" + NON_EXPORT = "NON_EXPORT" + PEAK_SHAVING = "PEAK_SHAVING" + GRID_SUPPORT_EXPORT = "GRID_SUPPORT_EXPORT" + GRID_SUPPORT_IMPORT = "GRID_SUPPORT_IMPORT" + FREQUENCY_REGULATION = "FREQUENCY_REGULATION" + RAMP_RATE_CONTROL = "RAMP_RATE_CONTROL" + RESERVE_BACKUP = "RESERVE_BACKUP" + OUTAGE_SUPPLY = "OUTAGE_SUPPLY" + FORCED_CHARGE = "FORCED_CHARGE" + FORCED_DISCHARGE = "FORCED_DISCHARGE" + FAULT = "FAULT" -class DeviceBase(DevicesStartEndMixin, PredictionMixin, DevicesMixin): - """Base class for device simulations. +class ApplianceOperationMode(StrEnum): + """Appliance operation modes. - Enables access to EOS configuration data (attribute `config`), EOS prediction data (attribute - `prediction`) and EOS device registry (attribute `devices`). + Modes + ----- + - OFF: + Stop or prevent any active operation of the appliance. - Behavior: - - Several initialization phases (setup, post_setup): - - setup: Initialize class attributes from DeviceParameters (pydantic input validation) - - post_setup: Set connections between devices - - NotImplemented: - - hooks during optimization + - RUN: + Start or continue normal operation of the appliance. - Notes: - - This class is base to concrete devices like battery, inverter, etc. that are used in optimization. - - Not a pydantic model for a low footprint during optimization. + - DEFER: + Postpone operation to a later time window based on + scheduling or optimization criteria. + + - PAUSE: + Temporarily suspend an ongoing operation, keeping the + option to resume later. + + - RESUME: + Continue an operation that was previously paused or + deferred. + + - LIMIT_POWER: + Run the appliance under reduced power constraints, + for example in response to load-management or + demand-response signals. + + - FORCED_RUN: + Start or maintain operation even if constraints or + optimization strategies would otherwise delay or limit it. + + - FAULT: + Appliance is unavailable due to fault or error state. """ - def __init__(self, parameters: Optional[DeviceParameters] = None): - self.device_id: str = "" - self.parameters: Optional[DeviceParameters] = None - self.hours = -1 - if self.total_hours is not None: - self.hours = self.total_hours - - self.initialized = DeviceState.UNINITIALIZED - - if parameters is not None: - self.setup(parameters) - - def setup(self, parameters: DeviceParameters) -> None: - if self.initialized != DeviceState.UNINITIALIZED: - return - - self.parameters = parameters - self.device_id = self.parameters.device_id - - if self.parameters.hours is not None: - self.hours = self.parameters.hours - if self.hours < 0: - raise ValueError("hours is unset") - - self._setup() - - self.initialized = DeviceState.PREPARED - - def post_setup(self) -> None: - if self.initialized.value >= DeviceState.INITIALIZED.value: - return - - self._post_setup() - self.initialized = DeviceState.INITIALIZED - - def _setup(self) -> None: - """Implement custom setup in derived device classes.""" - pass - - def _post_setup(self) -> None: - """Implement custom setup in derived device classes that is run when all devices are initialized.""" - pass - - -class DevicesBase(DevicesStartEndMixin, PredictionMixin): - """Base class for handling device data. - - Enables access to EOS configuration data (attribute `config`) and EOS prediction data (attribute - `prediction`). - """ - - def __init__(self) -> None: - super().__init__() - self.devices: dict[str, "DeviceBase"] = dict() - - def get_device_by_id(self, device_id: str) -> Optional["DeviceBase"]: - return self.devices.get(device_id) - - def add_device(self, device: Optional["DeviceBase"]) -> None: - if device is None: - return - if device.device_id in self.devices: - raise ValueError(f"{device.device_id} already registered") - self.devices[device.device_id] = device - - def remove_device(self, device: Type["DeviceBase"] | str) -> bool: - if isinstance(device, DeviceBase): - device = device.device_id - return self.devices.pop(device, None) is not None # type: ignore[arg-type] - - def reset(self) -> None: - self.devices = dict() + OFF = "OFF" + RUN = "RUN" + DEFER = "DEFER" + PAUSE = "PAUSE" + RESUME = "RESUME" + LIMIT_POWER = "LIMIT_POWER" + FORCED_RUN = "FORCED_RUN" + FAULT = "FAULT" diff --git a/src/akkudoktoreos/devices/generic.py b/src/akkudoktoreos/devices/generic.py deleted file mode 100644 index e6c9021..0000000 --- a/src/akkudoktoreos/devices/generic.py +++ /dev/null @@ -1,79 +0,0 @@ -from typing import Optional - -import numpy as np -from pydantic import Field - -from akkudoktoreos.devices.devicesabc import DeviceBase, DeviceParameters - - -class HomeApplianceParameters(DeviceParameters): - """Home Appliance Device Simulation Configuration.""" - - device_id: str = Field(description="ID of home appliance", examples=["dishwasher"]) - consumption_wh: int = Field( - gt=0, - description="An integer representing the energy consumption of a household device in watt-hours.", - examples=[2000], - ) - duration_h: int = Field( - gt=0, - description="An integer representing the usage duration of a household device in hours.", - examples=[3], - ) - - -class HomeAppliance(DeviceBase): - def __init__( - self, - parameters: Optional[HomeApplianceParameters] = None, - ): - self.parameters: Optional[HomeApplianceParameters] = None - super().__init__(parameters) - - def _setup(self) -> None: - if self.parameters is None: - raise ValueError(f"Parameters not set: {self.parameters}") - self.load_curve = np.zeros(self.hours) # Initialize the load curve with zeros - self.duration_h = self.parameters.duration_h - self.consumption_wh = self.parameters.consumption_wh - - def set_starting_time(self, start_hour: int, global_start_hour: int = 0) -> None: - """Sets the start time of the device and generates the corresponding load curve. - - :param start_hour: The hour at which the device should start. - """ - self.reset_load_curve() - # Check if the duration of use is within the available time frame - if start_hour + self.duration_h > self.hours: - raise ValueError("The duration of use exceeds the available time frame.") - if start_hour < global_start_hour: - raise ValueError("The start time is earlier than the available time frame.") - - # Calculate power per hour based on total consumption and duration - power_per_hour = self.consumption_wh / self.duration_h # Convert to watt-hours - - # Set the power for the duration of use in the load curve array - self.load_curve[start_hour : start_hour + self.duration_h] = power_per_hour - - def reset_load_curve(self) -> None: - """Resets the load curve.""" - self.load_curve = np.zeros(self.hours) - - def get_load_curve(self) -> np.ndarray: - """Returns the current load curve.""" - return self.load_curve - - def get_load_for_hour(self, hour: int) -> float: - """Returns the load for a specific hour. - - :param hour: The hour for which the load is queried. - :return: The load in watts for the specified hour. - """ - if hour < 0 or hour >= self.hours: - raise ValueError("The specified hour is outside the available time frame.") - - return self.load_curve[hour] - - def get_latest_starting_point(self) -> int: - """Returns the latest possible start time at which the device can still run completely.""" - return self.hours - self.duration_h diff --git a/src/akkudoktoreos/devices/genetic/__init__.py b/src/akkudoktoreos/devices/genetic/__init__.py new file mode 100644 index 0000000..e69de29 diff --git a/src/akkudoktoreos/devices/battery.py b/src/akkudoktoreos/devices/genetic/battery.py similarity index 50% rename from src/akkudoktoreos/devices/battery.py rename to src/akkudoktoreos/devices/genetic/battery.py index 916a842..ff120d8 100644 --- a/src/akkudoktoreos/devices/battery.py +++ b/src/akkudoktoreos/devices/genetic/battery.py @@ -1,125 +1,23 @@ from typing import Any, Optional import numpy as np -from pydantic import Field, field_validator -from akkudoktoreos.devices.devicesabc import ( - DeviceBase, - DeviceOptimizeResult, - DeviceParameters, +from akkudoktoreos.optimization.genetic.geneticdevices import ( + BaseBatteryParameters, + SolarPanelBatteryParameters, ) -from akkudoktoreos.utils.utils import NumpyEncoder -def max_charging_power_field(description: Optional[str] = None) -> float: - if description is None: - description = "Maximum charging power in watts." - return Field( - default=5000, - gt=0, - description=description, - ) - - -def initial_soc_percentage_field(description: str) -> int: - return Field(default=0, ge=0, le=100, description=description, examples=[42]) - - -def discharging_efficiency_field(default_value: float) -> float: - return Field( - default=default_value, - gt=0, - le=1, - description="A float representing the discharge efficiency of the battery.", - ) - - -class BaseBatteryParameters(DeviceParameters): - """Battery Device Simulation Configuration.""" - - device_id: str = Field(description="ID of battery", examples=["battery1"]) - capacity_wh: int = Field( - gt=0, - description="An integer representing the capacity of the battery in watt-hours.", - examples=[8000], - ) - charging_efficiency: float = Field( - default=0.88, - gt=0, - le=1, - description="A float representing the charging efficiency of the battery.", - ) - discharging_efficiency: float = discharging_efficiency_field(0.88) - max_charge_power_w: Optional[float] = max_charging_power_field() - initial_soc_percentage: int = initial_soc_percentage_field( - "An integer representing the state of charge of the battery at the **start** of the current hour (not the current state)." - ) - min_soc_percentage: int = Field( - default=0, - ge=0, - le=100, - description="An integer representing the minimum state of charge (SOC) of the battery in percentage.", - examples=[10], - ) - max_soc_percentage: int = Field( - default=100, - ge=0, - le=100, - description="An integer representing the maximum state of charge (SOC) of the battery in percentage.", - ) - - -class SolarPanelBatteryParameters(BaseBatteryParameters): - max_charge_power_w: Optional[float] = max_charging_power_field() - - -class ElectricVehicleParameters(BaseBatteryParameters): - """Battery Electric Vehicle Device Simulation Configuration.""" - - device_id: str = Field(description="ID of electric vehicle", examples=["ev1"]) - discharging_efficiency: float = discharging_efficiency_field(1.0) - initial_soc_percentage: int = initial_soc_percentage_field( - "An integer representing the current state of charge (SOC) of the battery in percentage." - ) - - -class ElectricVehicleResult(DeviceOptimizeResult): - """Result class containing information related to the electric vehicle's charging and discharging behavior.""" - - device_id: str = Field(description="ID of electric vehicle", examples=["ev1"]) - charge_array: list[float] = Field( - description="Hourly charging status (0 for no charging, 1 for charging)." - ) - discharge_array: list[int] = Field( - description="Hourly discharging status (0 for no discharging, 1 for discharging)." - ) - discharging_efficiency: float = Field(description="The discharge efficiency as a float..") - capacity_wh: int = Field(description="Capacity of the EV’s battery in watt-hours.") - charging_efficiency: float = Field(description="Charging efficiency as a float..") - max_charge_power_w: int = Field(description="Maximum charging power in watts.") - soc_wh: float = Field( - description="State of charge of the battery in watt-hours at the start of the simulation." - ) - initial_soc_percentage: int = Field( - description="State of charge at the start of the simulation in percentage." - ) - - @field_validator("discharge_array", "charge_array", mode="before") - def convert_numpy(cls, field: Any) -> Any: - return NumpyEncoder.convert_numpy(field)[0] - - -class Battery(DeviceBase): +class Battery: """Represents a battery device with methods to simulate energy charging and discharging.""" - def __init__(self, parameters: Optional[BaseBatteryParameters] = None): - self.parameters: Optional[BaseBatteryParameters] = None - super().__init__(parameters) + def __init__(self, parameters: BaseBatteryParameters, prediction_hours: int): + self.parameters = parameters + self.prediction_hours = prediction_hours + self._setup() def _setup(self) -> None: """Sets up the battery parameters based on configuration or provided parameters.""" - if self.parameters is None: - raise ValueError(f"Parameters not set: {self.parameters}") self.capacity_wh = self.parameters.capacity_wh self.initial_soc_percentage = self.parameters.initial_soc_percentage self.charging_efficiency = self.parameters.charging_efficiency @@ -138,8 +36,8 @@ class Battery(DeviceBase): self.max_charge_power_w = self.parameters.max_charge_power_w else: self.max_charge_power_w = self.capacity_wh # TODO this should not be equal capacity_wh - self.discharge_array = np.full(self.hours, 1) - self.charge_array = np.full(self.hours, 1) + self.discharge_array = np.full(self.prediction_hours, 1) + self.charge_array = np.full(self.prediction_hours, 1) self.soc_wh = (self.initial_soc_percentage / 100) * self.capacity_wh self.min_soc_wh = (self.min_soc_percentage / 100) * self.capacity_wh self.max_soc_wh = (self.max_soc_percentage / 100) * self.capacity_wh @@ -147,11 +45,11 @@ class Battery(DeviceBase): def to_dict(self) -> dict[str, Any]: """Converts the object to a dictionary representation.""" return { - "device_id": self.device_id, + "device_id": self.parameters.device_id, "capacity_wh": self.capacity_wh, "initial_soc_percentage": self.initial_soc_percentage, "soc_wh": self.soc_wh, - "hours": self.hours, + "hours": self.prediction_hours, "discharge_array": self.discharge_array, "charge_array": self.charge_array, "charging_efficiency": self.charging_efficiency, @@ -163,25 +61,31 @@ class Battery(DeviceBase): """Resets the battery state to its initial values.""" self.soc_wh = (self.initial_soc_percentage / 100) * self.capacity_wh self.soc_wh = min(max(self.soc_wh, self.min_soc_wh), self.max_soc_wh) - self.discharge_array = np.full(self.hours, 1) - self.charge_array = np.full(self.hours, 1) + self.discharge_array = np.full(self.prediction_hours, 1) + self.charge_array = np.full(self.prediction_hours, 1) def set_discharge_per_hour(self, discharge_array: np.ndarray) -> None: """Sets the discharge values for each hour.""" - if len(discharge_array) != self.hours: - raise ValueError(f"Discharge array must have exactly {self.hours} elements.") + if len(discharge_array) != self.prediction_hours: + raise ValueError( + f"Discharge array must have exactly {self.prediction_hours} elements. Got {len(discharge_array)} elements." + ) self.discharge_array = np.array(discharge_array) def set_charge_per_hour(self, charge_array: np.ndarray) -> None: """Sets the charge values for each hour.""" - if len(charge_array) != self.hours: - raise ValueError(f"Charge array must have exactly {self.hours} elements.") + if len(charge_array) != self.prediction_hours: + raise ValueError( + f"Charge array must have exactly {self.prediction_hours} elements. Got {len(charge_array)} elements." + ) self.charge_array = np.array(charge_array) def set_charge_allowed_for_hour(self, charge: float, hour: int) -> None: """Sets the charge for a specific hour.""" - if hour >= self.hours: - raise ValueError(f"Hour {hour} is out of range. Must be less than {self.hours}.") + if hour >= self.prediction_hours: + raise ValueError( + f"Hour {hour} is out of range. Must be less than {self.prediction_hours}." + ) self.charge_array[hour] = charge def current_soc_percentage(self) -> float: diff --git a/src/akkudoktoreos/devices/heatpump.py b/src/akkudoktoreos/devices/genetic/heatpump.py similarity index 100% rename from src/akkudoktoreos/devices/heatpump.py rename to src/akkudoktoreos/devices/genetic/heatpump.py diff --git a/src/akkudoktoreos/devices/genetic/homeappliance.py b/src/akkudoktoreos/devices/genetic/homeappliance.py new file mode 100644 index 0000000..d2c1ecd --- /dev/null +++ b/src/akkudoktoreos/devices/genetic/homeappliance.py @@ -0,0 +1,112 @@ +from typing import Optional + +import numpy as np + +from akkudoktoreos.optimization.genetic.geneticdevices import HomeApplianceParameters +from akkudoktoreos.utils.datetimeutil import ( + TimeWindow, + TimeWindowSequence, + to_datetime, + to_duration, + to_time, +) + + +class HomeAppliance: + def __init__( + self, + parameters: HomeApplianceParameters, + optimization_hours: int, + prediction_hours: int, + ): + self.parameters: HomeApplianceParameters = parameters + self.prediction_hours = prediction_hours + self._setup() + + def _setup(self) -> None: + """Sets up the home appliance parameters based provided parameters.""" + self.load_curve = np.zeros(self.prediction_hours) # Initialize the load curve with zeros + self.duration_h = self.parameters.duration_h + self.consumption_wh = self.parameters.consumption_wh + self.appliance_start: Optional[int] = None + # setup possible start times + if self.parameters.time_windows is None: + self.parameters.time_windows = TimeWindowSequence( + windows=[ + TimeWindow( + start_time=to_time("00:00"), + duration=to_duration(f"{self.prediction_hours} hours"), + ), + ] + ) + start_datetime = to_datetime().set(hour=0, minute=0, second=0) + duration = to_duration(f"{self.duration_h} hours") + self.start_allowed: list[bool] = [] + for hour in range(0, self.prediction_hours): + self.start_allowed.append( + self.parameters.time_windows.contains( + start_datetime.add(hours=hour), duration=duration + ) + ) + start_earliest = self.parameters.time_windows.earliest_start_time(duration, start_datetime) + if start_earliest: + self.start_earliest = start_earliest.hour + else: + self.start_earliest = 0 + start_latest = self.parameters.time_windows.latest_start_time(duration, start_datetime) + if start_latest: + self.start_latest = start_latest.hour + else: + self.start_latest = 23 + + def set_starting_time(self, start_hour: int, global_start_hour: int = 0) -> None: + """Sets the start time of the device and generates the corresponding load curve. + + :param start_hour: The hour at which the device should start. + """ + self.reset_load_curve() + + # Check if the duration of use is within the available time windows + if not self.start_allowed[start_hour]: + # No available time window to start home appliance + # Use the earliest one + start_hour = self.start_earliest + + # Check if it is possibility to start the appliance + if start_hour < global_start_hour: + # Start is before current time + # Use the latest one + start_hour = self.start_latest + + # Calculate power per hour based on total consumption and duration + power_per_hour = self.consumption_wh / self.duration_h # Convert to watt-hours + + # Set the power for the duration of use in the load curve array + self.load_curve[start_hour : start_hour + self.duration_h] = power_per_hour + + # Set the selected start hour + self.appliance_start = start_hour + + def reset_load_curve(self) -> None: + """Resets the load curve.""" + self.load_curve = np.zeros(self.prediction_hours) + + def get_load_curve(self) -> np.ndarray: + """Returns the current load curve.""" + return self.load_curve + + def get_load_for_hour(self, hour: int) -> float: + """Returns the load for a specific hour. + + :param hour: The hour for which the load is queried. + :return: The load in watts for the specified hour. + """ + if hour < 0 or hour >= self.prediction_hours: + raise ValueError( + f"The specified hour {hour} is outside the available time frame {self.prediction_hours}." + ) + + return self.load_curve[hour] + + def get_appliance_start(self) -> Optional[int]: + return self.appliance_start diff --git a/src/akkudoktoreos/devices/inverter.py b/src/akkudoktoreos/devices/genetic/inverter.py similarity index 52% rename from src/akkudoktoreos/devices/inverter.py rename to src/akkudoktoreos/devices/genetic/inverter.py index 708a688..6ae88e0 100644 --- a/src/akkudoktoreos/devices/inverter.py +++ b/src/akkudoktoreos/devices/genetic/inverter.py @@ -1,65 +1,32 @@ from typing import Optional from loguru import logger -from pydantic import Field -from akkudoktoreos.devices.devicesabc import DeviceBase, DeviceParameters +from akkudoktoreos.devices.genetic.battery import Battery +from akkudoktoreos.optimization.genetic.geneticdevices import InverterParameters from akkudoktoreos.prediction.interpolator import get_eos_load_interpolator -class InverterParameters(DeviceParameters): - """Inverter Device Simulation Configuration.""" - - device_id: str = Field(description="ID of inverter", examples=["inverter1"]) - max_power_wh: float = Field(gt=0, examples=[10000]) - battery_id: Optional[str] = Field( - default=None, description="ID of battery", examples=[None, "battery1"] - ) - - -class Inverter(DeviceBase): +class Inverter: def __init__( self, - parameters: Optional[InverterParameters] = None, + parameters: InverterParameters, + battery: Optional[Battery] = None, ): - self.parameters: Optional[InverterParameters] = None - super().__init__(parameters) - - self.scr_lookup: dict = {} - - def _calculate_scr(self, consumption: float, generation: float) -> float: - """Check if the consumption and production is in the lookup table. If not, calculate and store the value.""" - if consumption not in self.scr_lookup: - self.scr_lookup[consumption] = {} - - if generation not in self.scr_lookup[consumption]: - scr = self.self_consumption_predictor.calculate_self_consumption( - consumption, generation - ) - self.scr_lookup[consumption][generation] = scr - return scr - - return self.scr_lookup[consumption][generation] + self.parameters: InverterParameters = parameters + self.battery: Optional[Battery] = battery + self._setup() def _setup(self) -> None: - if self.parameters is None: - raise ValueError(f"Parameters not set: {self.parameters}") - if self.parameters.battery_id is None: - # For the moment raise exception - # TODO: Make battery configurable by config - error_msg = "Battery for PV inverter is mandatory." + if self.battery and self.parameters.battery_id != self.battery.parameters.device_id: + error_msg = f"Battery ID mismatch - {self.parameters.battery_id} is configured; got {self.battery.parameters.device_id}." logger.error(error_msg) - raise NotImplementedError(error_msg) + raise ValueError(error_msg) self.self_consumption_predictor = get_eos_load_interpolator() self.max_power_wh = ( self.parameters.max_power_wh ) # Maximum power that the inverter can handle - def _post_setup(self) -> None: - if self.parameters is None: - raise ValueError(f"Parameters not set: {self.parameters}") - self.battery = self.devices.get_device_by_id(self.parameters.battery_id) - def process_energy( self, generation: float, consumption: float, hour: int ) -> tuple[float, float, float, float]: @@ -76,8 +43,10 @@ class Inverter(DeviceBase): grid_import = -remaining_power # Negative indicates feeding into the grid self_consumption = self.max_power_wh else: - # Calculate scr with lookup table - scr = self._calculate_scr(consumption, generation) + # Calculate scr using cached results per energy management/optimization run + scr = self.self_consumption_predictor.calculate_self_consumption( + consumption, generation + ) # Remaining power after consumption remaining_power = (generation - consumption) * scr # EVQ @@ -86,11 +55,12 @@ class Inverter(DeviceBase): if remaining_load_evq > 0: # Akku muss den Restverbrauch decken - from_battery, discharge_losses = self.battery.discharge_energy( - remaining_load_evq, hour - ) - remaining_load_evq -= from_battery # Restverbrauch nach Akkuentladung - losses += discharge_losses + if self.battery: + from_battery, discharge_losses = self.battery.discharge_energy( + remaining_load_evq, hour + ) + remaining_load_evq -= from_battery # Restverbrauch nach Akkuentladung + losses += discharge_losses # Wenn der Akku den Restverbrauch nicht vollständig decken kann, wird der Rest ins Netz gezogen if remaining_load_evq > 0: @@ -101,10 +71,13 @@ class Inverter(DeviceBase): if remaining_power > 0: # Load battery with excess energy - charged_energie, charge_losses = self.battery.charge_energy( - remaining_power, hour - ) - remaining_surplus = remaining_power - (charged_energie + charge_losses) + if self.battery: + charged_energie, charge_losses = self.battery.charge_energy( + remaining_power, hour + ) + remaining_surplus = remaining_power - (charged_energie + charge_losses) + else: + remaining_surplus = remaining_power # Feed-in to the grid based on remaining capacity if remaining_surplus > self.max_power_wh - consumption: @@ -124,10 +97,13 @@ class Inverter(DeviceBase): available_ac_power = max(self.max_power_wh - generation, 0) # Discharge battery to cover shortfall, if possible - battery_discharge, discharge_losses = self.battery.discharge_energy( - min(shortfall, available_ac_power), hour - ) - losses += discharge_losses + if self.battery: + battery_discharge, discharge_losses = self.battery.discharge_energy( + min(shortfall, available_ac_power), hour + ) + losses += discharge_losses + else: + battery_discharge = 0 # Draw remaining required power from the grid (discharge_losses are already substraved in the battery) grid_import = shortfall - battery_discharge diff --git a/src/akkudoktoreos/devices/settings.py b/src/akkudoktoreos/devices/settings.py deleted file mode 100644 index e9443ce..0000000 --- a/src/akkudoktoreos/devices/settings.py +++ /dev/null @@ -1,24 +0,0 @@ -from typing import Optional - -from pydantic import Field - -from akkudoktoreos.config.configabc import SettingsBaseModel -from akkudoktoreos.devices.battery import BaseBatteryParameters -from akkudoktoreos.devices.generic import HomeApplianceParameters -from akkudoktoreos.devices.inverter import InverterParameters - - -class DevicesCommonSettings(SettingsBaseModel): - """Base configuration for devices simulation settings.""" - - batteries: Optional[list[BaseBatteryParameters]] = Field( - default=None, - description="List of battery/ev devices", - examples=[[{"device_id": "battery1", "capacity_wh": 8000}]], - ) - inverters: Optional[list[InverterParameters]] = Field( - default=None, description="List of inverters", examples=[[]] - ) - home_appliances: Optional[list[HomeApplianceParameters]] = Field( - default=None, description="List of home appliances", examples=[[]] - ) diff --git a/src/akkudoktoreos/measurement/measurement.py b/src/akkudoktoreos/measurement/measurement.py index 5b6fb74..70dc543 100644 --- a/src/akkudoktoreos/measurement/measurement.py +++ b/src/akkudoktoreos/measurement/measurement.py @@ -6,90 +6,69 @@ data records for measurements. The measurements can be added programmatically or imported from a file or JSON string. """ -from typing import Any, ClassVar, List, Optional +from typing import Any, Optional import numpy as np from loguru import logger from numpydantic import NDArray, Shape -from pendulum import DateTime, Duration from pydantic import Field, computed_field from akkudoktoreos.config.configabc import SettingsBaseModel from akkudoktoreos.core.coreabc import SingletonMixin from akkudoktoreos.core.dataabc import DataImportMixin, DataRecord, DataSequence -from akkudoktoreos.utils.datetimeutil import to_duration +from akkudoktoreos.utils.datetimeutil import DateTime, Duration, to_duration class MeasurementCommonSettings(SettingsBaseModel): """Measurement Configuration.""" - load0_name: Optional[str] = Field( - default=None, description="Name of the load0 source", examples=["Household", "Heat Pump"] + load_emr_keys: Optional[list[str]] = Field( + default=None, + description="The keys of the measurements that are energy meter readings of a load [kWh].", + examples=[["load0_emr"]], ) - load1_name: Optional[str] = Field( - default=None, description="Name of the load1 source", examples=[None] + + grid_export_emr_keys: Optional[list[str]] = Field( + default=None, + description="The keys of the measurements that are energy meter readings of energy export to grid [kWh].", + examples=[["grid_export_emr"]], ) - load2_name: Optional[str] = Field( - default=None, description="Name of the load2 source", examples=[None] + + grid_import_emr_keys: Optional[list[str]] = Field( + default=None, + description="The keys of the measurements that are energy meter readings of energy import from grid [kWh].", + examples=[["grid_import_emr"]], ) - load3_name: Optional[str] = Field( - default=None, description="Name of the load3 source", examples=[None] - ) - load4_name: Optional[str] = Field( - default=None, description="Name of the load4 source", examples=[None] + + pv_production_emr_keys: Optional[list[str]] = Field( + default=None, + description="The keys of the measurements that are PV production energy meter readings [kWh].", + examples=[["pv1_emr"]], ) + ## Computed fields + @computed_field # type: ignore[prop-decorator] + @property + def keys(self) -> list[str]: + """The keys of the measurements that can be stored.""" + key_list = [] + for key in self.__class__.model_fields.keys(): + if key.endswith("_keys") and (value := getattr(self, key)): + key_list.extend(value) + return sorted(set(key_list)) + class MeasurementDataRecord(DataRecord): - """Represents a measurement data record containing various measurements at a specific datetime. + """Represents a measurement data record containing various measurements at a specific datetime.""" - Attributes: - date_time (Optional[DateTime]): The datetime of the record. - """ - - # Single loads, to be aggregated to total load - load0_mr: Optional[float] = Field( - default=None, ge=0, description="Load0 meter reading [kWh]", examples=[40421] - ) - load1_mr: Optional[float] = Field( - default=None, ge=0, description="Load1 meter reading [kWh]", examples=[None] - ) - load2_mr: Optional[float] = Field( - default=None, ge=0, description="Load2 meter reading [kWh]", examples=[None] - ) - load3_mr: Optional[float] = Field( - default=None, ge=0, description="Load3 meter reading [kWh]", examples=[None] - ) - load4_mr: Optional[float] = Field( - default=None, ge=0, description="Load4 meter reading [kWh]", examples=[None] - ) - - max_loads: ClassVar[int] = 5 # Maximum number of loads that can be set - - grid_export_mr: Optional[float] = Field( - default=None, ge=0, description="Export to grid meter reading [kWh]", examples=[1000] - ) - - grid_import_mr: Optional[float] = Field( - default=None, ge=0, description="Import from grid meter reading [kWh]", examples=[1000] - ) - - # Computed fields - @computed_field # type: ignore[prop-decorator] - @property - def loads(self) -> List[str]: - """Compute a list of active loads.""" - active_loads = [] - - # Loop through loadx - for i in range(self.max_loads): - load_attr = f"load{i}_mr" - - # Check if either attribute is set and add to active loads - if getattr(self, load_attr, None): - active_loads.append(load_attr) - - return active_loads + @classmethod + def configured_data_keys(cls) -> Optional[list[str]]: + """Return the keys for the configured field like data.""" + keys = cls.config.measurement.keys + # Add measurment keys that are needed/ handled by the resource/ device simulations. + if cls.config.devices.measurement_keys: + keys.extend(cls.config.devices.measurement_keys) + return keys class Measurement(SingletonMixin, DataImportMixin, DataSequence): @@ -98,14 +77,10 @@ class Measurement(SingletonMixin, DataImportMixin, DataSequence): Measurements can be provided programmatically or read from JSON string or file. """ - records: List[MeasurementDataRecord] = Field( - default_factory=list, description="List of measurement data records" + records: list[MeasurementDataRecord] = Field( + default_factory=list, description="list of measurement data records" ) - topics: ClassVar[List[str]] = [ - "load", - ] - def __init__(self, *args: Any, **kwargs: Any) -> None: if hasattr(self, "_initialized"): return @@ -141,34 +116,6 @@ class Measurement(SingletonMixin, DataImportMixin, DataSequence): # Return ceiling of division to include partial intervals return int(np.ceil(diff_seconds / interval_seconds)) - def name_to_key(self, name: str, topic: str) -> Optional[str]: - """Provides measurement key for given name and topic.""" - topic = topic.lower() - - if topic not in self.topics: - return None - - topic_keys = [ - key for key in self.config.measurement.model_fields.keys() if key.startswith(topic) - ] - key = None - if topic == "load": - for config_key in topic_keys: - if ( - config_key.endswith("_name") - and getattr(self.config.measurement, config_key) == name - ): - key = topic + config_key[len(topic) : len(topic) + 1] + "_mr" - break - - if key is not None and key not in self.record_keys: - # Should never happen - error_msg = f"Key '{key}' not available." - logger.error(error_msg) - raise KeyError(error_msg) - - return key - def _energy_from_meter_readings( self, key: str, @@ -253,17 +200,20 @@ class Measurement(SingletonMixin, DataImportMixin, DataSequence): end_datetime = self[-1].date_time size = self._interval_count(start_datetime, end_datetime, interval) load_total_array = np.zeros(size) - # Loop through load_mr - for i in range(self.record_class().max_loads): - key = f"load{i}_mr" - # Calculate load per interval - load_array = self._energy_from_meter_readings( - key=key, start_datetime=start_datetime, end_datetime=end_datetime, interval=interval - ) - # Add calculated load to total load - load_total_array += load_array - debug_msg = f"Total load '{key}' calculation: {load_total_array}" - logger.debug(debug_msg) + # Loop through all loads + if isinstance(self.config.measurement.load_emr_keys, list): + for key in self.config.measurement.load_emr_keys: + # Calculate load per interval + load_array = self._energy_from_meter_readings( + key=key, + start_datetime=start_datetime, + end_datetime=end_datetime, + interval=interval, + ) + # Add calculated load to total load + load_total_array += load_array + debug_msg = f"Total load '{key}' calculation: {load_total_array}" + logger.debug(debug_msg) return load_total_array diff --git a/src/akkudoktoreos/optimization/genetic.py b/src/akkudoktoreos/optimization/genetic/genetic.py similarity index 53% rename from src/akkudoktoreos/optimization/genetic.py rename to src/akkudoktoreos/optimization/genetic/genetic.py index dc8497f..83b8692 100644 --- a/src/akkudoktoreos/optimization/genetic.py +++ b/src/akkudoktoreos/optimization/genetic/genetic.py @@ -1,3 +1,5 @@ +"""Genetic algorithm.""" + import random import time from typing import Any, Optional @@ -5,101 +7,302 @@ from typing import Any, Optional import numpy as np from deap import algorithms, base, creator, tools from loguru import logger -from pydantic import Field, field_validator, model_validator -from typing_extensions import Self +from numpydantic import NDArray, Shape +from pydantic import ConfigDict, Field -from akkudoktoreos.core.coreabc import ( - ConfigMixin, - DevicesMixin, - EnergyManagementSystemMixin, +from akkudoktoreos.core.pydantic import PydanticBaseModel +from akkudoktoreos.devices.genetic.battery import Battery +from akkudoktoreos.devices.genetic.homeappliance import HomeAppliance +from akkudoktoreos.devices.genetic.inverter import Inverter +from akkudoktoreos.optimization.genetic.geneticparams import ( + GeneticEnergyManagementParameters, + GeneticOptimizationParameters, ) -from akkudoktoreos.core.ems import EnergyManagementParameters, SimulationResult -from akkudoktoreos.core.pydantic import ParametersBaseModel -from akkudoktoreos.devices.battery import ( - Battery, - ElectricVehicleParameters, - ElectricVehicleResult, - SolarPanelBatteryParameters, +from akkudoktoreos.optimization.genetic.geneticsolution import ( + GeneticSimulationResult, + GeneticSolution, ) -from akkudoktoreos.devices.generic import HomeAppliance, HomeApplianceParameters -from akkudoktoreos.devices.inverter import Inverter, InverterParameters -from akkudoktoreos.utils.utils import NumpyEncoder +from akkudoktoreos.optimization.optimizationabc import OptimizationBase -class OptimizationParameters(ParametersBaseModel): - ems: EnergyManagementParameters - pv_akku: Optional[SolarPanelBatteryParameters] - inverter: Optional[InverterParameters] - eauto: Optional[ElectricVehicleParameters] - dishwasher: Optional[HomeApplianceParameters] = None - temperature_forecast: Optional[list[Optional[float]]] = Field( +class GeneticSimulation(PydanticBaseModel): + """Device simulation for GENETIC optimization algorithm.""" + + # Disable validation on assignment to speed up simulation runs. + model_config = ConfigDict( + validate_assignment=False, + ) + + start_hour: int = Field( + default=0, ge=0, le=23, description="Starting hour on day for optimizations." + ) + + optimization_hours: Optional[int] = Field( + default=24, ge=0, description="Number of hours into the future for optimizations." + ) + + prediction_hours: Optional[int] = Field( + default=48, ge=0, description="Number of hours into the future for predictions" + ) + + load_energy_array: Optional[NDArray[Shape["*"], float]] = Field( default=None, - description="An array of floats representing the temperature forecast in degrees Celsius for different time intervals.", + description="An array of floats representing the total load (consumption) in watts for different time intervals.", ) - start_solution: Optional[list[float]] = Field( - default=None, description="Can be `null` or contain a previous solution (if available)." - ) - - @model_validator(mode="after") - def validate_list_length(self) -> Self: - arr_length = len(self.ems.pv_prognose_wh) - if self.temperature_forecast is not None and arr_length != len(self.temperature_forecast): - raise ValueError("Input lists have different lengths") - return self - - @field_validator("start_solution") - def validate_start_solution( - cls, start_solution: Optional[list[float]] - ) -> Optional[list[float]]: - if start_solution is not None and len(start_solution) < 2: - raise ValueError("Requires at least two values.") - return start_solution - - -class OptimizeResponse(ParametersBaseModel): - """**Note**: The first value of "Last_Wh_per_hour", "Netzeinspeisung_Wh_per_hour", and "Netzbezug_Wh_per_hour", will be set to null in the JSON output and represented as NaN or None in the corresponding classes' data returns. This approach is adopted to ensure that the current hour's processing remains unchanged.""" - - ac_charge: list[float] = Field( - description="Array with AC charging values as relative power (0-1), other values set to 0." - ) - dc_charge: list[float] = Field( - description="Array with DC charging values as relative power (0-1), other values set to 0." - ) - discharge_allowed: list[int] = Field( - description="Array with discharge values (1 for discharge, 0 otherwise)." - ) - eautocharge_hours_float: Optional[list[float]] = Field(description="TBD") - result: SimulationResult - eauto_obj: Optional[ElectricVehicleResult] - start_solution: Optional[list[float]] = Field( + pv_prediction_wh: Optional[NDArray[Shape["*"], float]] = Field( default=None, - description="An array of binary values (0 or 1) representing a possible starting solution for the simulation.", + description="An array of floats representing the forecasted photovoltaic output in watts for different time intervals.", ) - washingstart: Optional[int] = Field( + elect_price_hourly: Optional[NDArray[Shape["*"], float]] = Field( default=None, - description="Can be `null` or contain an object representing the start of washing (if applicable).", + description="An array of floats representing the electricity price in euros per watt-hour for different time intervals.", + ) + elect_revenue_per_hour_arr: Optional[NDArray[Shape["*"], float]] = Field( + default=None, + description="An array of floats representing the feed-in compensation in euros per watt-hour.", ) - @field_validator( - "ac_charge", - "dc_charge", - "discharge_allowed", - mode="before", - ) - def convert_numpy(cls, field: Any) -> Any: - return NumpyEncoder.convert_numpy(field)[0] + battery: Optional[Battery] = Field(default=None, description="TBD.") + ev: Optional[Battery] = Field(default=None, description="TBD.") + home_appliance: Optional[HomeAppliance] = Field(default=None, description="TBD.") + inverter: Optional[Inverter] = Field(default=None, description="TBD.") - @field_validator( - "eauto_obj", - mode="before", - ) - def convert_eauto(cls, field: Any) -> Any: - if isinstance(field, Battery): - return ElectricVehicleResult(**field.to_dict()) - return field + ac_charge_hours: Optional[NDArray[Shape["*"], float]] = Field(default=None, description="TBD") + dc_charge_hours: Optional[NDArray[Shape["*"], float]] = Field(default=None, description="TBD") + ev_charge_hours: Optional[NDArray[Shape["*"], float]] = Field(default=None, description="TBD") + + def prepare( + self, + parameters: GeneticEnergyManagementParameters, + optimization_hours: int, + prediction_hours: int, + ev: Optional[Battery] = None, + home_appliance: Optional[HomeAppliance] = None, + inverter: Optional[Inverter] = None, + ) -> None: + self.optimization_hours = optimization_hours + self.prediction_hours = prediction_hours + self.load_energy_array = np.array(parameters.gesamtlast, float) + self.pv_prediction_wh = np.array(parameters.pv_prognose_wh, float) + self.elect_price_hourly = np.array(parameters.strompreis_euro_pro_wh, float) + self.elect_revenue_per_hour_arr = ( + parameters.einspeiseverguetung_euro_pro_wh + if isinstance(parameters.einspeiseverguetung_euro_pro_wh, list) + else np.full( + len(self.load_energy_array), parameters.einspeiseverguetung_euro_pro_wh, float + ) + ) + if inverter: + self.battery = inverter.battery + else: + self.battery = None + self.ev = ev + self.home_appliance = home_appliance + self.inverter = inverter + self.ac_charge_hours = np.full(self.prediction_hours, 0.0) + self.dc_charge_hours = np.full(self.prediction_hours, 1.0) + self.ev_charge_hours = np.full(self.prediction_hours, 0.0) + """Prepare simulation runs.""" + self.load_energy_array = np.array(parameters.gesamtlast, float) + self.pv_prediction_wh = np.array(parameters.pv_prognose_wh, float) + self.elect_price_hourly = np.array(parameters.strompreis_euro_pro_wh, float) + self.elect_revenue_per_hour_arr = ( + parameters.einspeiseverguetung_euro_pro_wh + if isinstance(parameters.einspeiseverguetung_euro_pro_wh, list) + else np.full( + len(self.load_energy_array), parameters.einspeiseverguetung_euro_pro_wh, float + ) + ) + + def set_akku_discharge_hours(self, ds: np.ndarray) -> None: + if self.battery: + self.battery.set_discharge_per_hour(ds) + + def set_akku_ac_charge_hours(self, ds: np.ndarray) -> None: + self.ac_charge_hours = ds + + def set_akku_dc_charge_hours(self, ds: np.ndarray) -> None: + self.dc_charge_hours = ds + + def set_ev_charge_hours(self, ds: np.ndarray) -> None: + self.ev_charge_hours = ds + + def set_home_appliance_start(self, ds: int, global_start_hour: int = 0) -> None: + if self.home_appliance: + self.home_appliance.set_starting_time(ds, global_start_hour=global_start_hour) + + def reset(self) -> None: + if self.ev: + self.ev.reset() + if self.battery: + self.battery.reset() + + def simulate(self, start_hour: int) -> dict[str, Any]: + """Simulate energy usage and costs for the given start hour. + + akku_soc_pro_stunde begin of the hour, initial hour state! + last_wh_pro_stunde integral of last hour (end state) + """ + # Remember start hour + self.start_hour = start_hour + + # Check for simulation integrity + required_attrs = [ + "load_energy_array", + "pv_prediction_wh", + "elect_price_hourly", + "ev_charge_hours", + "ac_charge_hours", + "dc_charge_hours", + "elect_revenue_per_hour_arr", + ] + missing_data = [ + attr.replace("_", " ").title() for attr in required_attrs if getattr(self, attr) is None + ] + + if missing_data: + logger.error("Mandatory data missing - %s", ", ".join(missing_data)) + raise ValueError(f"Mandatory data missing: {', '.join(missing_data)}") + + # Pre-fetch data + load_energy_array = np.array(self.load_energy_array) + pv_prediction_wh = np.array(self.pv_prediction_wh) + elect_price_hourly = np.array(self.elect_price_hourly) + ev_charge_hours = np.array(self.ev_charge_hours) + ac_charge_hours = np.array(self.ac_charge_hours) + dc_charge_hours = np.array(self.dc_charge_hours) + elect_revenue_per_hour_arr = np.array(self.elect_revenue_per_hour_arr) + + # Fetch objects + battery = self.battery + ev = self.ev + home_appliance = self.home_appliance + inverter = self.inverter + + if not (len(load_energy_array) == len(pv_prediction_wh) == len(elect_price_hourly)): + error_msg = f"Array sizes do not match: Load Curve = {len(load_energy_array)}, PV Forecast = {len(pv_prediction_wh)}, Electricity Price = {len(elect_price_hourly)}" + logger.error(error_msg) + raise ValueError(error_msg) + + end_hour = len(load_energy_array) + total_hours = end_hour - start_hour + + # Pre-allocate arrays for the results, optimized for speed + loads_energy_per_hour = np.full((total_hours), np.nan) + feedin_energy_per_hour = np.full((total_hours), np.nan) + consumption_energy_per_hour = np.full((total_hours), np.nan) + costs_per_hour = np.full((total_hours), np.nan) + revenue_per_hour = np.full((total_hours), np.nan) + soc_per_hour = np.full((total_hours), np.nan) + soc_ev_per_hour = np.full((total_hours), np.nan) + losses_wh_per_hour = np.full((total_hours), np.nan) + home_appliance_wh_per_hour = np.full((total_hours), np.nan) + electricity_price_per_hour = np.full((total_hours), np.nan) + + # Set initial state + if battery: + soc_per_hour[0] = battery.current_soc_percentage() + if ev: + soc_ev_per_hour[0] = ev.current_soc_percentage() + + for hour in range(start_hour, end_hour): + hour_idx = hour - start_hour + + # save begin states + if battery: + soc_per_hour[hour_idx] = battery.current_soc_percentage() + if ev: + soc_ev_per_hour[hour_idx] = ev.current_soc_percentage() + + # Accumulate loads and PV generation + consumption = load_energy_array[hour] + losses_wh_per_hour[hour_idx] = 0.0 + + # Home appliances + if home_appliance: + ha_load = home_appliance.get_load_for_hour(hour) + consumption += ha_load + home_appliance_wh_per_hour[hour_idx] = ha_load + + # E-Auto handling + if ev and ev_charge_hours[hour] > 0: + loaded_energy_ev, verluste_eauto = ev.charge_energy( + None, hour, relative_power=ev_charge_hours[hour] + ) + consumption += loaded_energy_ev + losses_wh_per_hour[hour_idx] += verluste_eauto + + # Process inverter logic + energy_feedin_grid_actual = energy_consumption_grid_actual = losses = eigenverbrauch = ( + 0.0 + ) + + hour_ac_charge = ac_charge_hours[hour] + hour_dc_charge = dc_charge_hours[hour] + hourly_electricity_price = elect_price_hourly[hour] + hourly_energy_revenue = elect_revenue_per_hour_arr[hour] + + if battery: + battery.set_charge_allowed_for_hour(hour_dc_charge, hour) + + if inverter: + energy_produced = pv_prediction_wh[hour] + ( + energy_feedin_grid_actual, + energy_consumption_grid_actual, + losses, + eigenverbrauch, + ) = inverter.process_energy(energy_produced, consumption, hour) + + # AC PV Battery Charge + if battery and hour_ac_charge > 0.0: + battery.set_charge_allowed_for_hour(1, hour) + battery_charged_energy_actual, battery_losses_actual = battery.charge_energy( + None, hour, relative_power=hour_ac_charge + ) + + total_battery_energy = battery_charged_energy_actual + battery_losses_actual + consumption += total_battery_energy + energy_consumption_grid_actual += total_battery_energy + losses_wh_per_hour[hour_idx] += battery_losses_actual + + # Update hourly arrays + feedin_energy_per_hour[hour_idx] = energy_feedin_grid_actual + consumption_energy_per_hour[hour_idx] = energy_consumption_grid_actual + losses_wh_per_hour[hour_idx] += losses + loads_energy_per_hour[hour_idx] = consumption + electricity_price_per_hour[hour_idx] = hourly_electricity_price + + # Financial calculations + costs_per_hour[hour_idx] = energy_consumption_grid_actual * hourly_electricity_price + revenue_per_hour[hour_idx] = energy_feedin_grid_actual * hourly_energy_revenue + + total_cost = np.nansum(costs_per_hour) + total_losses = np.nansum(losses_wh_per_hour) + total_revenue = np.nansum(revenue_per_hour) + + # Prepare output dictionary + return { + "Last_Wh_pro_Stunde": loads_energy_per_hour, + "Netzeinspeisung_Wh_pro_Stunde": feedin_energy_per_hour, + "Netzbezug_Wh_pro_Stunde": consumption_energy_per_hour, + "Kosten_Euro_pro_Stunde": costs_per_hour, + "akku_soc_pro_stunde": soc_per_hour, + "Einnahmen_Euro_pro_Stunde": revenue_per_hour, + "Gesamtbilanz_Euro": total_cost - total_revenue, + "EAuto_SoC_pro_Stunde": soc_ev_per_hour, + "Gesamteinnahmen_Euro": total_revenue, + "Gesamtkosten_Euro": total_cost, + "Verluste_Pro_Stunde": losses_wh_per_hour, + "Gesamt_Verluste": total_losses, + "Home_appliance_wh_per_hour": home_appliance_wh_per_hour, + "Electricity_price": electricity_price_per_hour, + } -class optimization_problem(ConfigMixin, DevicesMixin, EnergyManagementSystemMixin): +class GeneticOptimization(OptimizationBase): + """GENETIC algorithm to solve energy optimization.""" + def __init__( self, verbose: bool = False, @@ -107,8 +310,10 @@ class optimization_problem(ConfigMixin, DevicesMixin, EnergyManagementSystemMixi ): """Initialize the optimization problem with the required parameters.""" self.opti_param: dict[str, Any] = {} - self.fixed_eauto_hours = self.config.prediction.hours - self.config.optimization.hours - self.possible_charge_values = self.config.optimization.ev_available_charge_rates_percent + self.fixed_eauto_hours = ( + self.config.prediction.hours - self.config.optimization.horizon_hours + ) + self.ev_possible_charge_values: list[float] = [1.0] self.verbose = verbose self.fix_seed = fixed_seed self.optimize_ev = True @@ -122,12 +327,15 @@ class optimization_problem(ConfigMixin, DevicesMixin, EnergyManagementSystemMixi self.fix_seed = random.randint(1, 100000000000) # noqa: S311 random.seed(self.fix_seed) + # Create Simulation + self.simulation = GeneticSimulation() + def decode_charge_discharge( self, discharge_hours_bin: np.ndarray ) -> tuple[np.ndarray, np.ndarray, np.ndarray]: """Decode the input array into ac_charge, dc_charge, and discharge arrays.""" discharge_hours_bin_np = np.array(discharge_hours_bin) - len_ac = len(self.possible_charge_values) + len_ac = len(self.ev_possible_charge_values) # Categorization: # Idle: 0 .. len_ac-1 @@ -159,7 +367,7 @@ class optimization_problem(ConfigMixin, DevicesMixin, EnergyManagementSystemMixi discharge[discharge_mask] = 1 # Set Discharge states to 1 ac_charge = np.zeros_like(discharge_hours_bin_np, dtype=float) - ac_charge[ac_mask] = [self.possible_charge_values[i] for i in ac_indices] + ac_charge[ac_mask] = [self.ev_possible_charge_values[i] for i in ac_indices] # Idle is just 0, already default. @@ -168,7 +376,7 @@ class optimization_problem(ConfigMixin, DevicesMixin, EnergyManagementSystemMixi def mutate(self, individual: list[int]) -> tuple[list[int]]: """Custom mutation function for the individual.""" # Calculate the number of states - len_ac = len(self.possible_charge_values) + len_ac = len(self.ev_possible_charge_values) if self.optimize_dc_charge: total_states = 3 * len_ac + 2 else: @@ -303,7 +511,7 @@ class optimization_problem(ConfigMixin, DevicesMixin, EnergyManagementSystemMixi creator.create("Individual", list, fitness=creator.FitnessMin) self.toolbox = base.Toolbox() - len_ac = len(self.possible_charge_values) + len_ac = len(self.ev_possible_charge_values) # Total number of states without DC: # Idle: len_ac states @@ -362,38 +570,39 @@ class optimization_problem(ConfigMixin, DevicesMixin, EnergyManagementSystemMixi This is an internal function. """ - self.ems.reset() + self.simulation.reset() discharge_hours_bin, eautocharge_hours_index, washingstart_int = self.split_individual( individual ) - if self.opti_param.get("home_appliance", 0) > 0: - self.ems.set_home_appliance_start( + if self.opti_param.get("home_appliance", 0) > 0 and washingstart_int: + self.simulation.set_home_appliance_start( washingstart_int, global_start_hour=self.ems.start_datetime.hour ) ac, dc, discharge = self.decode_charge_discharge(discharge_hours_bin) - self.ems.set_akku_discharge_hours(discharge) + self.simulation.set_akku_discharge_hours(discharge) # Set DC charge hours only if DC optimization is enabled if self.optimize_dc_charge: - self.ems.set_akku_dc_charge_hours(dc) - self.ems.set_akku_ac_charge_hours(ac) + self.simulation.set_akku_dc_charge_hours(dc) + self.simulation.set_akku_ac_charge_hours(ac) if eautocharge_hours_index is not None: eautocharge_hours_float = np.array( - [self.possible_charge_values[i] for i in eautocharge_hours_index], + [self.ev_possible_charge_values[i] for i in eautocharge_hours_index], float, ) - self.ems.set_ev_charge_hours(eautocharge_hours_float) + self.simulation.set_ev_charge_hours(eautocharge_hours_float) else: - self.ems.set_ev_charge_hours(np.full(self.config.prediction.hours, 0)) + self.simulation.set_ev_charge_hours(np.full(self.config.prediction.hours, 0)) - return self.ems.simulate(self.ems.start_datetime.hour) + # Do the simulation and return result. + return self.simulation.simulate(self.ems.start_datetime.hour) def evaluate( self, individual: list[int], - parameters: OptimizationParameters, + parameters: GeneticOptimizationParameters, start_hour: int, worst_case: bool, ) -> tuple[float]: @@ -456,7 +665,8 @@ class optimization_problem(ConfigMixin, DevicesMixin, EnergyManagementSystemMixi # # len_ac + 2 # # ) # Activate discharge for these hours - # # When Battery SoC then set the Discharge randomly to 0 or 1. otherwise it's very unlikely to get a state where a battery can store energy for a longer time + # # When Battery SoC then set the Discharge randomly to 0 or 1. otherwise it's very + # # unlikely to get a state where a battery can store energy for a longer time # # Find hours where battery SoC is 0 # zero_soc_mask = battery_soc_per_hour_tail == 0 # # discharge_hours_bin_tail[zero_soc_mask] = ( @@ -478,43 +688,67 @@ class optimization_problem(ConfigMixin, DevicesMixin, EnergyManagementSystemMixi individual.extra_data = ( # type: ignore[attr-defined] o["Gesamtbilanz_Euro"], o["Gesamt_Verluste"], - parameters.eauto.min_soc_percentage - self.ems.ev.current_soc_percentage() - if parameters.eauto and self.ems.ev + parameters.eauto.min_soc_percentage - self.simulation.ev.current_soc_percentage() + if parameters.eauto and self.simulation.ev else 0, ) # Adjust total balance with battery value and penalties for unmet SOC - restwert_akku = ( - self.ems.battery.current_energy_content() * parameters.ems.preis_euro_pro_wh_akku - ) - gesamtbilanz += -restwert_akku + if self.simulation.battery: + restwert_akku = ( + self.simulation.battery.current_energy_content() + * parameters.ems.preis_euro_pro_wh_akku + ) + gesamtbilanz += -restwert_akku if self.optimize_ev: + try: + penalty = self.config.optimization.genetic.penalties["ev_soc_miss"] + except: + # Use default + penalty = 10 + logger.error( + "Penalty function parameter `ev_soc_miss` not configured, using {}.", penalty + ) gesamtbilanz += max( 0, ( - parameters.eauto.min_soc_percentage - self.ems.ev.current_soc_percentage() - if parameters.eauto and self.ems.ev + parameters.eauto.min_soc_percentage + - self.simulation.ev.current_soc_percentage() + if parameters.eauto and self.simulation.ev else 0 ) - * self.config.optimization.penalty, + * penalty, ) return (gesamtbilanz,) def optimize( - self, start_solution: Optional[list[float]] = None, ngen: int = 200 + self, + start_solution: Optional[list[float]] = None, + ngen: int = 200, ) -> tuple[Any, dict[str, list[Any]]]: - """Run the optimization process using a genetic algorithm.""" - population = self.toolbox.population(n=300) + """Run the optimization process using a genetic algorithm. + + @TODO: optimize() ngen default (200) is different from optimierung_ems() ngen default (400). + """ + # Set the number of inviduals in a generation + try: + individuals = self.config.optimization.genetic.individuals + if individuals is None: + raise + except: + individuals = 300 + logger.error("Individuals not configured. Using {}.", individuals) + + population = self.toolbox.population(n=individuals) hof = tools.HallOfFame(1) stats = tools.Statistics(lambda ind: ind.fitness.values) stats.register("min", np.min) stats.register("avg", np.mean) stats.register("max", np.max) - if self.verbose: - print("Start optimize:", start_solution) + logger.debug("Start optimize: {}", start_solution) # Insert the start solution into the population if provided if start_solution is not None: @@ -555,39 +789,63 @@ class optimization_problem(ConfigMixin, DevicesMixin, EnergyManagementSystemMixi def optimierung_ems( self, - parameters: OptimizationParameters, + parameters: GeneticOptimizationParameters, start_hour: Optional[int] = None, worst_case: bool = False, - ngen: int = 400, - ) -> OptimizeResponse: + ngen: Optional[int] = None, + ) -> GeneticSolution: """Perform EMS (Energy Management System) optimization and visualize results.""" if start_hour is None: start_hour = self.ems.start_datetime.hour + # Start hour has to be in sync with energy management + if start_hour != self.ems.start_datetime.hour: + raise ValueError( + f"Start hour not synced. EMS {self.ems.start_datetime.hour} vs. GENETIC {start_hour}." + ) + + # Set the number of generations + generations = ngen + if generations is None: + try: + generations = self.config.optimization.genetic.generations + except: + generations = 400 + logger.error("Generations not configured. Using {}.", generations) einspeiseverguetung_euro_pro_wh = np.full( self.config.prediction.hours, parameters.ems.einspeiseverguetung_euro_pro_wh ) - # TODO: Refactor device setup phase out - self.devices.reset() + self.simulation.reset() # Initialize PV and EV batteries akku: Optional[Battery] = None if parameters.pv_akku: - akku = Battery(parameters.pv_akku) - self.devices.add_device(akku) + akku = Battery( + parameters.pv_akku, + prediction_hours=self.config.prediction.hours, + ) akku.set_charge_per_hour(np.full(self.config.prediction.hours, 1)) eauto: Optional[Battery] = None if parameters.eauto: eauto = Battery( parameters.eauto, + prediction_hours=self.config.prediction.hours, ) - self.devices.add_device(eauto) eauto.set_charge_per_hour(np.full(self.config.prediction.hours, 1)) self.optimize_ev = ( parameters.eauto.min_soc_percentage - parameters.eauto.initial_soc_percentage >= 0 ) + try: + charge_rates = self.config.devices.electric_vehicles[0].charge_rates + if charge_rates is None: + raise + except: + error_msg = "No charge rates provided for electric vehicle." + logger.exception(error_msg) + raise ValueError(error_msg) + self.ev_possible_charge_values = charge_rates else: self.optimize_ev = False @@ -595,29 +853,30 @@ class optimization_problem(ConfigMixin, DevicesMixin, EnergyManagementSystemMixi dishwasher = ( HomeAppliance( parameters=parameters.dishwasher, + optimization_hours=self.config.optimization.horizon_hours, + prediction_hours=self.config.prediction.hours, ) if parameters.dishwasher is not None else None ) - self.devices.add_device(dishwasher) # Initialize the inverter and energy management system inverter: Optional[Inverter] = None if parameters.inverter: inverter = Inverter( parameters.inverter, + battery=akku, ) - self.devices.add_device(inverter) - self.devices.post_setup() - - self.ems.set_parameters( - parameters.ems, - inverter=inverter, + # Prepare device simulation + self.simulation.prepare( + parameters=parameters.ems, + optimization_hours=self.config.optimization.horizon_hours, + prediction_hours=self.config.prediction.hours, + inverter=inverter, # battery is part of inverter ev=eauto, home_appliance=dishwasher, ) - self.ems.set_start_hour(start_hour) # Setup the DEAP environment and optimization process self.setup_deap_environment({"home_appliance": 1 if dishwasher else 0}, start_hour) @@ -626,20 +885,24 @@ class optimization_problem(ConfigMixin, DevicesMixin, EnergyManagementSystemMixi lambda ind: self.evaluate(ind, parameters, start_hour, worst_case), ) - if self.verbose: - start_time = time.time() - start_solution, extra_data = self.optimize(parameters.start_solution, ngen=ngen) + start_time = time.time() + start_solution, extra_data = self.optimize(parameters.start_solution, ngen=generations) + elapsed_time = time.time() - start_time + logger.debug(f"Time evaluate inner: {elapsed_time:.4f} sec.") - if self.verbose: - elapsed_time = time.time() - start_time - print(f"Time evaluate inner: {elapsed_time:.4f} sec.") # Perform final evaluation on the best solution - o = self.evaluate_inner(start_solution) + simulation_result = self.evaluate_inner(start_solution) + + # Prepare results discharge_hours_bin, eautocharge_hours_index, washingstart_int = self.split_individual( start_solution ) + # home appliance may have choosen a different appliance start hour + if self.simulation.home_appliance: + washingstart_int = self.simulation.home_appliance.get_appliance_start() + eautocharge_hours_float = ( - [self.possible_charge_values[i] for i in eautocharge_hours_index] + [self.ev_possible_charge_values[i] for i in eautocharge_hours_index] if eautocharge_hours_index is not None else None ) @@ -651,8 +914,8 @@ class optimization_problem(ConfigMixin, DevicesMixin, EnergyManagementSystemMixi "dc_charge": dc_charge.tolist(), "discharge_allowed": discharge.tolist(), "eautocharge_hours_float": eautocharge_hours_float, - "result": o, - "eauto_obj": self.ems.ev.to_dict(), + "result": simulation_result, + "eauto_obj": self.simulation.ev.to_dict() if self.simulation.ev else None, "start_solution": start_solution, "spuelstart": washingstart_int, "extra_data": extra_data, @@ -663,14 +926,14 @@ class optimization_problem(ConfigMixin, DevicesMixin, EnergyManagementSystemMixi prepare_visualize(parameters, visualize, start_hour=start_hour) - return OptimizeResponse( + return GeneticSolution( **{ "ac_charge": ac_charge, "dc_charge": dc_charge, "discharge_allowed": discharge, "eautocharge_hours_float": eautocharge_hours_float, - "result": SimulationResult(**o), - "eauto_obj": self.ems.ev, + "result": GeneticSimulationResult(**simulation_result), + "eauto_obj": self.simulation.ev, "start_solution": start_solution, "washingstart": washingstart_int, } diff --git a/src/akkudoktoreos/optimization/genetic/geneticabc.py b/src/akkudoktoreos/optimization/genetic/geneticabc.py new file mode 100644 index 0000000..5d343cb --- /dev/null +++ b/src/akkudoktoreos/optimization/genetic/geneticabc.py @@ -0,0 +1,11 @@ +"""Genetic optimization algorithm abstract and base classes.""" + +from pydantic import ConfigDict + +from akkudoktoreos.core.pydantic import PydanticBaseModel + + +class GeneticParametersBaseModel(PydanticBaseModel): + """Pydantic base model for parameters for the GENETIC algorithm.""" + + model_config = ConfigDict(extra="forbid") diff --git a/src/akkudoktoreos/optimization/genetic/geneticdevices.py b/src/akkudoktoreos/optimization/genetic/geneticdevices.py new file mode 100644 index 0000000..9da941a --- /dev/null +++ b/src/akkudoktoreos/optimization/genetic/geneticdevices.py @@ -0,0 +1,127 @@ +"""Genetic optimization algorithm device interfaces/ parameters.""" + +from typing import Optional + +from pydantic import Field + +from akkudoktoreos.optimization.genetic.geneticabc import GeneticParametersBaseModel +from akkudoktoreos.utils.datetimeutil import TimeWindowSequence + + +class DeviceParameters(GeneticParametersBaseModel): + device_id: str = Field(description="ID of device", examples="device1") + hours: Optional[int] = Field( + default=None, + gt=0, + description="Number of prediction hours. Defaults to global config prediction hours.", + examples=[None], + ) + + +def max_charging_power_field(description: Optional[str] = None) -> float: + if description is None: + description = "Maximum charging power in watts." + return Field( + default=5000, + gt=0, + description=description, + ) + + +def initial_soc_percentage_field(description: str) -> int: + return Field(default=0, ge=0, le=100, description=description, examples=[42]) + + +def discharging_efficiency_field(default_value: float) -> float: + return Field( + default=default_value, + gt=0, + le=1, + description="A float representing the discharge efficiency of the battery.", + ) + + +class BaseBatteryParameters(DeviceParameters): + """Battery Device Simulation Configuration.""" + + device_id: str = Field(description="ID of battery", examples=["battery1"]) + capacity_wh: int = Field( + gt=0, + description="An integer representing the capacity of the battery in watt-hours.", + examples=[8000], + ) + charging_efficiency: float = Field( + default=0.88, + gt=0, + le=1, + description="A float representing the charging efficiency of the battery.", + ) + discharging_efficiency: float = discharging_efficiency_field(0.88) + max_charge_power_w: Optional[float] = max_charging_power_field() + initial_soc_percentage: int = initial_soc_percentage_field( + "An integer representing the state of charge of the battery at the **start** of the current hour (not the current state)." + ) + min_soc_percentage: int = Field( + default=0, + ge=0, + le=100, + description="An integer representing the minimum state of charge (SOC) of the battery in percentage.", + examples=[10], + ) + max_soc_percentage: int = Field( + default=100, + ge=0, + le=100, + description="An integer representing the maximum state of charge (SOC) of the battery in percentage.", + ) + + +class SolarPanelBatteryParameters(BaseBatteryParameters): + """PV battery device simulation configuration.""" + + max_charge_power_w: Optional[float] = max_charging_power_field() + + +class ElectricVehicleParameters(BaseBatteryParameters): + """Battery Electric Vehicle Device Simulation Configuration.""" + + device_id: str = Field(description="ID of electric vehicle", examples=["ev1"]) + discharging_efficiency: float = discharging_efficiency_field(1.0) + initial_soc_percentage: int = initial_soc_percentage_field( + "An integer representing the current state of charge (SOC) of the battery in percentage." + ) + + +class HomeApplianceParameters(DeviceParameters): + """Home Appliance Device Simulation Configuration.""" + + device_id: str = Field(description="ID of home appliance", examples=["dishwasher"]) + consumption_wh: int = Field( + gt=0, + description="An integer representing the energy consumption of a household device in watt-hours.", + examples=[2000], + ) + duration_h: int = Field( + gt=0, + description="An integer representing the usage duration of a household device in hours.", + examples=[3], + ) + time_windows: Optional[TimeWindowSequence] = Field( + default=None, + description="List of allowed time windows. Defaults to optimization general time window.", + examples=[ + [ + {"start_time": "10:00", "duration": "2 hours"}, + ], + ], + ) + + +class InverterParameters(DeviceParameters): + """Inverter Device Simulation Configuration.""" + + device_id: str = Field(description="ID of inverter", examples=["inverter1"]) + max_power_wh: float = Field(gt=0, examples=[10000]) + battery_id: Optional[str] = Field( + default=None, description="ID of battery", examples=[None, "battery1"] + ) diff --git a/src/akkudoktoreos/optimization/genetic/geneticparams.py b/src/akkudoktoreos/optimization/genetic/geneticparams.py new file mode 100644 index 0000000..c260466 --- /dev/null +++ b/src/akkudoktoreos/optimization/genetic/geneticparams.py @@ -0,0 +1,630 @@ +"""GENETIC algorithm paramters. + +This module defines the Pydantic-based configuration and input parameter models +used in the energy optimization routines, including photovoltaic forecasts, +electricity pricing, and system component parameters. + +It also provides a method to assemble these parameters from predictions, +forecasts, and fallback defaults, preparing them for optimization runs. +""" + +from typing import Optional, Union + +from loguru import logger +from pydantic import Field, field_validator, model_validator +from typing_extensions import Self + +from akkudoktoreos.core.coreabc import ( + ConfigMixin, + MeasurementMixin, + PredictionMixin, +) +from akkudoktoreos.optimization.genetic.geneticabc import GeneticParametersBaseModel +from akkudoktoreos.optimization.genetic.geneticdevices import ( + ElectricVehicleParameters, + HomeApplianceParameters, + InverterParameters, + SolarPanelBatteryParameters, +) +from akkudoktoreos.utils.datetimeutil import to_duration + +# Do not import directly from akkudoktoreos.core.coreabc +# EnergyManagementSystemMixin - Creates circular dependency with ems.py +# StartMixin - Creates circular dependency with ems.py + + +class GeneticEnergyManagementParameters(GeneticParametersBaseModel): + """Encapsulates energy-related forecasts and costs used in GENETIC optimization.""" + + pv_prognose_wh: list[float] = Field( + description="An array of floats representing the forecasted photovoltaic output in watts for different time intervals." + ) + strompreis_euro_pro_wh: list[float] = Field( + description="An array of floats representing the electricity price in euros per watt-hour for different time intervals." + ) + einspeiseverguetung_euro_pro_wh: Union[list[float], float] = Field( + description="A float or array of floats representing the feed-in compensation in euros per watt-hour." + ) + preis_euro_pro_wh_akku: float = Field( + description="A float representing the cost of battery energy per watt-hour." + ) + gesamtlast: list[float] = Field( + description="An array of floats representing the total load (consumption) in watts for different time intervals." + ) + + @model_validator(mode="after") + def validate_list_length(self) -> Self: + """Validate that all input lists are of the same length. + + Raises: + ValueError: If input list lengths differ. + """ + pv_prognose_length = len(self.pv_prognose_wh) + if ( + pv_prognose_length != len(self.strompreis_euro_pro_wh) + or pv_prognose_length != len(self.gesamtlast) + or ( + isinstance(self.einspeiseverguetung_euro_pro_wh, list) + and pv_prognose_length != len(self.einspeiseverguetung_euro_pro_wh) + ) + ): + raise ValueError("Input lists have different lengths") + return self + + +class GeneticOptimizationParameters( + ConfigMixin, + MeasurementMixin, + PredictionMixin, + # EnergyManagementSystemMixin, # Creates circular dependency with ems.py + # StartMixin, # Creates circular dependency with ems.py + GeneticParametersBaseModel, +): + """Main parameter class for running the genetic energy optimization. + + Collects all model and configuration parameters necessary to run the + optimization process, such as forecasts, pricing, battery and appliance models. + """ + + ems: GeneticEnergyManagementParameters + pv_akku: Optional[SolarPanelBatteryParameters] + inverter: Optional[InverterParameters] + eauto: Optional[ElectricVehicleParameters] + dishwasher: Optional[HomeApplianceParameters] = None + temperature_forecast: Optional[list[Optional[float]]] = Field( + default=None, + description="An array of floats representing the temperature forecast in degrees Celsius for different time intervals.", + ) + start_solution: Optional[list[float]] = Field( + default=None, description="Can be `null` or contain a previous solution (if available)." + ) + + @model_validator(mode="after") + def validate_list_length(self) -> Self: + """Ensure that temperature forecast list matches the PV forecast length. + + Raises: + ValueError: If list lengths mismatch. + """ + arr_length = len(self.ems.pv_prognose_wh) + if self.temperature_forecast is not None and arr_length != len(self.temperature_forecast): + raise ValueError("Input lists have different lengths") + return self + + @field_validator("start_solution") + def validate_start_solution( + cls, start_solution: Optional[list[float]] + ) -> Optional[list[float]]: + """Validate that the starting solution has at least two elements. + + Args: + start_solution (list[float]): Optional list of solution values. + + Returns: + list[float]: Validated list. + + Raises: + ValueError: If the solution is too short. + """ + if start_solution is not None and len(start_solution) < 2: + raise ValueError("Requires at least two values.") + return start_solution + + @classmethod + def prepare(cls) -> "Optional[GeneticOptimizationParameters]": + """Prepare optimization parameters from config, forecast and measurement data. + + Fills in values needed for optimization from available configuration, predictions and + measurements. If some data is missing, default or demo values are used. + + Parameters start by definition of the genetic algorithm at hour 0 of the actual date + (not at start datetime of energy management run) + + Returns: + GeneticOptimizationParameters: The fully prepared optimization parameters. + + Raises: + ValueError: If required configuration values like start time are missing. + """ + # Avoid circular dependency + from akkudoktoreos.core.ems import get_ems + + ems = get_ems() + + # The optimization paramters + oparams: "Optional[GeneticOptimizationParameters]" = None + + # Check for run definitions + if ems.start_datetime is None: + error_msg = "Start datetime unknown." + logger.error(error_msg) + raise ValueError(error_msg) + # Check for general predictions conditions + if cls.config.general.latitude is None: + default_latitude = 52.52 + logger.error(f"Latitude unknown - defaulting to {default_latitude}.") + cls.config.general.latitude = default_latitude + if cls.config.general.longitude is None: + default_longitude = 13.405 + logger.error(f"Longitude unknown - defaulting to {default_longitude}.") + cls.config.general.longitude = default_longitude + if cls.config.prediction.hours is None: + logger.error("Prediction hours unknown - defaulting to 48 hours.") + cls.config.prediction.hours = 48 + if cls.config.prediction.historic_hours is None: + logger.error("Prediction historic hours unknown - defaulting to 24 hours.") + cls.config.prediction.historic_hours = 24 + # Check optimization definitions + if cls.config.optimization.horizon_hours is None: + logger.error("Optimization horizon unknown - defaulting to 24 hours.") + cls.config.optimization.horizon_hours = 24 + if cls.config.optimization.interval is None: + logger.error("Optimization interval unknown - defaulting to 3600 seconds.") + cls.config.optimization.interval = 3600 + if cls.config.optimization.interval != 3600: + logger.error( + "Optimization interval '{}' seconds not supported - forced to 3600 seconds." + ) + cls.config.optimization.interval = 3600 + # Check genetic algorithm definitions + if cls.config.optimization.genetic is None: + logger.error( + "Genetic optimization configuration not configured - defaulting to demo config." + ) + cls.config.optimization.genetic = { + "individuals": 300, + "generations": 400, + "seed": None, + "penalties": { + "ev_soc_miss": 10, + }, + } + if cls.config.optimization.genetic.individuals is None: + logger.error("Genetic individuals unknown - defaulting to 300.") + cls.config.optimization.genetic.individuals = 300 + if cls.config.optimization.genetic.generations is None: + logger.error("Genetic generations unknown - defaulting to 400.") + cls.config.optimization.genetic.generations = 400 + if cls.config.optimization.genetic.penalties is None: + logger.error("Genetic penalties unknown - defaulting to demo config.") + cls.config.optimization.genetic.penalties = {"ev_soc_miss": 10} + if "ev_soc_miss" not in cls.config.optimization.genetic.penalties: + logger.error("ev_soc_miss penalty function parameter unknown - defaulting to 100.") + cls.config.optimization.genetic.penalties["ev_soc_miss"] = 10 + + # Add forecast and device data + interval = to_duration(cls.config.optimization.interval) + power_to_energy_per_interval_factor = cls.config.optimization.interval / 3600 + parameter_start_datetime = ems.start_datetime.set(hour=0, second=0, microsecond=0) + parameter_end_datetime = parameter_start_datetime.add(hours=cls.config.prediction.hours) + max_retries = 10 + + for attempt in range(1, max_retries + 1): + # Collect all the data for optimisation, but do not exceed max retries + if attempt > max_retries: + error_msg = f"Maximum retries {max_retries} for parameter collection exceeded. Parameter preparation attempt {attempt}." + logger.error(error_msg) + raise ValueError(error_msg) + + # Assure predictions are uptodate + cls.prediction.update_data() + + try: + pvforecast_ac_power = ( + cls.prediction.key_to_array( + key="pvforecast_ac_power", + start_datetime=parameter_start_datetime, + end_datetime=parameter_end_datetime, + interval=interval, + fill_method="linear", + ) + * power_to_energy_per_interval_factor + ).tolist() + except: + logger.exception( + "No PV forecast data available - defaulting to demo data. Parameter preparation attempt {}.", + attempt, + ) + cls.config.merge_settings_from_dict( + { + "pvforecast": { + "provider": "PVForecastAkkudoktor", + "planes": [ + { + "peakpower": 5.0, + "surface_azimuth": 170, + "surface_tilt": 7, + "userhorizon": [20, 27, 22, 20], + "inverter_paco": 10000, + }, + { + "peakpower": 4.8, + "surface_azimuth": 90, + "surface_tilt": 7, + "userhorizon": [30, 30, 30, 50], + "inverter_paco": 10000, + }, + { + "peakpower": 1.4, + "surface_azimuth": 140, + "surface_tilt": 60, + "userhorizon": [60, 30, 0, 30], + "inverter_paco": 2000, + }, + { + "peakpower": 1.6, + "surface_azimuth": 185, + "surface_tilt": 45, + "userhorizon": [45, 25, 30, 60], + "inverter_paco": 1400, + }, + ], + }, + } + ) + # Retry + continue + try: + elecprice_marketprice_wh = cls.prediction.key_to_array( + key="elecprice_marketprice_wh", + start_datetime=parameter_start_datetime, + end_datetime=parameter_end_datetime, + interval=interval, + fill_method="ffill", + ).tolist() + except: + logger.exception( + "No Electricity Marketprice forecast data available - defaulting to demo data. Parameter preparation attempt {}.", + attempt, + ) + cls.config.elecprice.provider = "ElecPriceAkkudoktor" + # Retry + continue + try: + load_mean_adjusted = cls.prediction.key_to_array( + key="load_mean_adjusted", + start_datetime=parameter_start_datetime, + end_datetime=parameter_end_datetime, + interval=interval, + fill_method="ffill", + ).tolist() + except: + logger.exception( + "No Load forecast data available - defaulting to demo data. Parameter preparation attempt {}.", + attempt, + ) + cls.config.merge_settings_from_dict( + { + "load": { + "provider": "LoadAkkudoktor", + "provider_settings": { + "LoadAkkudoktor": { + "loadakkudoktor_year_energy": "1000", + }, + }, + }, + } + ) + # Retry + continue + try: + feed_in_tariff_wh = cls.prediction.key_to_array( + key="feed_in_tariff_wh", + start_datetime=parameter_start_datetime, + end_datetime=parameter_end_datetime, + interval=interval, + fill_method="ffill", + ).tolist() + except: + logger.exception( + "No feed in tariff forecast data available - defaulting to demo data. Parameter preparation attempt {}.", + attempt, + ) + cls.config.merge_settings_from_dict( + { + "feedintariff": { + "provider": "FeedInTariffFixed", + "provider_settings": { + "FeedInTariffFixed": { + "feed_in_tariff_kwh": 0.078, + }, + }, + }, + } + ) + # Retry + continue + try: + weather_temp_air = cls.prediction.key_to_array( + key="weather_temp_air", + start_datetime=parameter_start_datetime, + end_datetime=parameter_end_datetime, + interval=interval, + fill_method="ffill", + ).tolist() + except: + logger.exception( + "No weather forecast data available - defaulting to demo data. Parameter preparation attempt {}.", + attempt, + ) + cls.config.weather.provider = "BrightSky" + # Retry + continue + + # Add device data + + # Batteries + # --------- + if cls.config.devices.max_batteries is None: + logger.error("Number of battery devices not configured - defaulting to 1.") + cls.config.devices.max_batteries = 1 + if cls.config.devices.max_batteries == 0: + battery_params = None + battery_lcos_kwh = 0 + else: + if cls.config.devices.batteries is None: + logger.error("No battery device data available - defaulting to demo data.") + cls.config.devices.batteries = [{"device_id": "battery1", "capacity_wh": 8000}] + try: + battery_config = cls.config.devices.batteries[0] + battery_params = SolarPanelBatteryParameters( + device_id=battery_config.device_id, + capacity_wh=battery_config.capacity_wh, + charging_efficiency=battery_config.charging_efficiency, + discharging_efficiency=battery_config.discharging_efficiency, + max_charge_power_w=battery_config.max_charge_power_w, + min_soc_percentage=battery_config.min_soc_percentage, + max_soc_percentage=battery_config.max_soc_percentage, + ) + except: + logger.exception( + "No battery device data available - defaulting to demo data. Parameter preparation attempt {}.", + attempt, + ) + cls.config.devices.batteries = [{"device_id": "battery1", "capacity_wh": 8000}] + # Retry + continue + # Levelized cost of ownership + if battery_config.levelized_cost_of_storage_kwh is None: + logger.error( + "No battery device LCOS data available - defaulting to 0 €/kWh. Parameter preparation attempt {}.", + attempt, + ) + battery_config.levelized_cost_of_storage_kwh = 0 + battery_lcos_kwh = battery_config.levelized_cost_of_storage_kwh + # Initial SOC + try: + initial_soc_factor = cls.measurement.key_to_value( + key=battery_config.measurement_key_soc_factor, + target_datetime=ems.start_datetime, + ) + if initial_soc_factor > 1.0 or initial_soc_factor < 0.0: + logger.error( + f"Invalid battery initial SoC factor {initial_soc_factor} - defaulting to 0.0." + ) + initial_soc_factor = 0.0 + # genetic parameter is 0..100 as int + initial_soc_percentage = int(initial_soc_factor * 100) + except: + initial_soc_percentage = None + if initial_soc_percentage is None: + logger.error( + f"No battery device SoC data (measurement key = '{battery_config.measurement_key_soc_factor}') available - defaulting to 0." + ) + initial_soc_percentage = 0 + battery_params.initial_soc_percentage = initial_soc_percentage + + # Electric Vehicles + # ----------------- + if cls.config.devices.max_electric_vehicles is None: + logger.error("Number of electric_vehicle devices not configured - defaulting to 1.") + cls.config.devices.max_electric_vehicles = 1 + if cls.config.devices.max_electric_vehicles == 0: + electric_vehicle_params = None + else: + if cls.config.devices.electric_vehicles is None: + logger.error( + "No electric vehicle device data available - defaulting to demo data." + ) + cls.config.devices.max_electric_vehicles = 1 + cls.config.devices.electric_vehicles = [ + { + "device_id": "ev11", + "capacity_wh": 50000, + "charge_rates": [0.0, 0.375, 0.5, 0.625, 0.75, 0.875, 1.0], + "min_soc_percentage": 70, + } + ] + try: + electric_vehicle_config = cls.config.devices.electric_vehicles[0] + electric_vehicle_params = ElectricVehicleParameters( + device_id=electric_vehicle_config.device_id, + capacity_wh=electric_vehicle_config.capacity_wh, + charging_efficiency=electric_vehicle_config.charging_efficiency, + discharging_efficiency=electric_vehicle_config.discharging_efficiency, + max_charge_power_w=electric_vehicle_config.max_charge_power_w, + min_soc_percentage=electric_vehicle_config.min_soc_percentage, + max_soc_percentage=electric_vehicle_config.max_soc_percentage, + ) + except: + logger.exception( + "No electric_vehicle device data available - defaulting to demo data. Parameter preparation attempt {}.", + attempt, + ) + cls.config.devices.max_electric_vehicles = 1 + cls.config.devices.electric_vehicles = [ + { + "device_id": "ev12", + "capacity_wh": 50000, + "charge_rates": [0.0, 0.375, 0.5, 0.625, 0.75, 0.875, 1.0], + "min_soc_percentage": 70, + } + ] + # Retry + continue + # Initial SOC + try: + initial_soc_factor = cls.measurement.key_to_value( + key=electric_vehicle_config.measurement_key_soc_factor, + target_datetime=ems.start_datetime, + ) + if initial_soc_factor > 1.0 or initial_soc_factor < 0.0: + logger.error( + f"Invalid electric vehicle initial SoC factor {initial_soc_factor} - defaulting to 0.0." + ) + initial_soc_factor = 0.0 + # genetic parameter is 0..100 as int + initial_soc_percentage = int(initial_soc_factor * 100) + except: + initial_soc_percentage = None + if initial_soc_percentage is None: + logger.error( + f"No electric vehicle device SoC data (measurement key = '{electric_vehicle_config.measurement_key_soc_factor}') available - defaulting to 0." + ) + initial_soc_percentage = 0 + electric_vehicle_params.initial_soc_percentage = initial_soc_percentage + + # Inverters + # --------- + if cls.config.devices.max_inverters is None: + logger.error("Number of inverter devices not configured - defaulting to 1.") + cls.config.devices.max_inverters = 1 + if cls.config.devices.max_inverters == 0: + inverter_params = None + else: + if cls.config.devices.inverters is None: + logger.error("No inverter device data available - defaulting to demo data.") + cls.config.devices.inverters = [ + { + "device_id": "inverter1", + "max_power_w": 10000, + "battery_id": battery_config.device_id, + } + ] + try: + inverter_config = cls.config.devices.inverters[0] + inverter_params = InverterParameters( + device_id=inverter_config.device_id, + max_power_wh=inverter_config.max_power_w, + battery_id=inverter_config.battery_id, + ) + except: + logger.exception( + "No inverter device data available - defaulting to demo data. Parameter preparation attempt {}.", + attempt, + ) + cls.config.devices.inverters = [ + { + "device_id": "inverter1", + "max_power_w": 10000, + "battery_id": battery_config.device_id, + } + ] + # Retry + continue + + # Home Appliances + # --------------- + if cls.config.devices.max_home_appliances is None: + logger.error("Number of home appliance devices not configured - defaulting to 1.") + cls.config.devices.max_home_appliances = 1 + if cls.config.devices.max_home_appliances == 0: + home_appliance_params = None + else: + home_appliance_params = None + if cls.config.devices.home_appliances is None: + logger.error( + "No home appliance device data available - defaulting to demo data." + ) + cls.config.devices.home_appliances = [ + { + "device_id": "dishwasher1", + "consumption_wh": 2000, + "duration_h": 3.0, + "time_windows": { + "windows": [ + { + "start_time": "08:00", + "duration": "5 hours", + }, + { + "start_time": "15:00", + "duration": "3 hours", + }, + ], + }, + } + ] + try: + home_appliance_config = cls.config.devices.home_appliances[0] + home_appliance_params = HomeApplianceParameters( + device_id=home_appliance_config.device_id, + consumption_wh=home_appliance_config.consumption_wh, + duration_h=home_appliance_config.duration_h, + time_windows=home_appliance_config.time_windows, + ) + except: + logger.exception( + "No home appliance device data available - defaulting to demo data. Parameter preparation attempt {}.", + attempt, + ) + cls.config.devices.home_appliances = [ + { + "device_id": "dishwasher1", + "consumption_wh": 2000, + "duration_h": 3.0, + "time_windows": None, + } + ] + # Retry + continue + + # We got all parameter data + try: + oparams = GeneticOptimizationParameters( + ems=GeneticEnergyManagementParameters( + pv_prognose_wh=pvforecast_ac_power, + strompreis_euro_pro_wh=elecprice_marketprice_wh, + einspeiseverguetung_euro_pro_wh=feed_in_tariff_wh, + gesamtlast=load_mean_adjusted, + preis_euro_pro_wh_akku=battery_lcos_kwh / 1000, + ), + temperature_forecast=weather_temp_air, + pv_akku=battery_params, + eauto=electric_vehicle_params, + inverter=inverter_params, + dishwasher=home_appliance_params, + ) + except: + logger.exception( + "Can not prepare optimization parameters - will retry. Parameter preparation attempt {}.", + attempt, + ) + oparams = None + # Retry + continue + + # Parameters prepared + break + + return oparams diff --git a/src/akkudoktoreos/optimization/genetic/geneticsolution.py b/src/akkudoktoreos/optimization/genetic/geneticsolution.py new file mode 100644 index 0000000..cc392ed --- /dev/null +++ b/src/akkudoktoreos/optimization/genetic/geneticsolution.py @@ -0,0 +1,480 @@ +"""Genetic algorithm optimisation solution.""" + +from typing import Any, Optional + +import pandas as pd +from loguru import logger +from pydantic import Field, field_validator + +from akkudoktoreos.config.config import get_config +from akkudoktoreos.core.emplan import ( + DDBCInstruction, + EnergyManagementPlan, + FRBCInstruction, +) +from akkudoktoreos.core.pydantic import PydanticDateTimeDataFrame +from akkudoktoreos.devices.devicesabc import ( + ApplianceOperationMode, + BatteryOperationMode, +) +from akkudoktoreos.devices.genetic.battery import Battery +from akkudoktoreos.optimization.genetic.geneticdevices import GeneticParametersBaseModel +from akkudoktoreos.optimization.optimization import OptimizationSolution +from akkudoktoreos.prediction.prediction import get_prediction +from akkudoktoreos.utils.datetimeutil import to_datetime, to_duration +from akkudoktoreos.utils.utils import NumpyEncoder + + +class DeviceOptimizeResult(GeneticParametersBaseModel): + device_id: str = Field(description="ID of device", examples=["device1"]) + hours: int = Field(gt=0, description="Number of hours in the simulation.", examples=[24]) + + +class ElectricVehicleResult(DeviceOptimizeResult): + """Result class containing information related to the electric vehicle's charging and discharging behavior.""" + + device_id: str = Field(description="ID of electric vehicle", examples=["ev1"]) + charge_array: list[float] = Field( + description="Hourly charging status (0 for no charging, 1 for charging)." + ) + discharge_array: list[int] = Field( + description="Hourly discharging status (0 for no discharging, 1 for discharging)." + ) + discharging_efficiency: float = Field(description="The discharge efficiency as a float..") + capacity_wh: int = Field(description="Capacity of the EV’s battery in watt-hours.") + charging_efficiency: float = Field(description="Charging efficiency as a float..") + max_charge_power_w: int = Field(description="Maximum charging power in watts.") + soc_wh: float = Field( + description="State of charge of the battery in watt-hours at the start of the simulation." + ) + initial_soc_percentage: int = Field( + description="State of charge at the start of the simulation in percentage." + ) + + @field_validator("discharge_array", "charge_array", mode="before") + def convert_numpy(cls, field: Any) -> Any: + return NumpyEncoder.convert_numpy(field)[0] + + +class GeneticSimulationResult(GeneticParametersBaseModel): + """This object contains the results of the simulation and provides insights into various parameters over the entire forecast period.""" + + Last_Wh_pro_Stunde: list[float] = Field(description="TBD") + EAuto_SoC_pro_Stunde: list[float] = Field( + description="The state of charge of the EV for each hour." + ) + Einnahmen_Euro_pro_Stunde: list[float] = Field( + description="The revenue from grid feed-in or other sources in euros per hour." + ) + Gesamt_Verluste: float = Field( + description="The total losses in watt-hours over the entire period." + ) + Gesamtbilanz_Euro: float = Field( + description="The total balance of revenues minus costs in euros." + ) + Gesamteinnahmen_Euro: float = Field(description="The total revenues in euros.") + Gesamtkosten_Euro: float = Field(description="The total costs in euros.") + Home_appliance_wh_per_hour: list[Optional[float]] = Field( + description="The energy consumption of a household appliance in watt-hours per hour." + ) + Kosten_Euro_pro_Stunde: list[float] = Field(description="The costs in euros per hour.") + Netzbezug_Wh_pro_Stunde: list[float] = Field( + description="The grid energy drawn in watt-hours per hour." + ) + Netzeinspeisung_Wh_pro_Stunde: list[float] = Field( + description="The energy fed into the grid in watt-hours per hour." + ) + Verluste_Pro_Stunde: list[float] = Field(description="The losses in watt-hours per hour.") + akku_soc_pro_stunde: list[float] = Field( + description="The state of charge of the battery (not the EV) in percentage per hour." + ) + Electricity_price: list[float] = Field( + description="Used Electricity Price, including predictions" + ) + + @field_validator( + "Last_Wh_pro_Stunde", + "Netzeinspeisung_Wh_pro_Stunde", + "akku_soc_pro_stunde", + "Netzbezug_Wh_pro_Stunde", + "Kosten_Euro_pro_Stunde", + "Einnahmen_Euro_pro_Stunde", + "EAuto_SoC_pro_Stunde", + "Verluste_Pro_Stunde", + "Home_appliance_wh_per_hour", + "Electricity_price", + mode="before", + ) + def convert_numpy(cls, field: Any) -> Any: + return NumpyEncoder.convert_numpy(field)[0] + + +class GeneticSolution(GeneticParametersBaseModel): + """**Note**: The first value of "Last_Wh_per_hour", "Netzeinspeisung_Wh_per_hour", and "Netzbezug_Wh_per_hour", will be set to null in the JSON output and represented as NaN or None in the corresponding classes' data returns. This approach is adopted to ensure that the current hour's processing remains unchanged.""" + + ac_charge: list[float] = Field( + description="Array with AC charging values as relative power (0.0-1.0), other values set to 0." + ) + dc_charge: list[float] = Field( + description="Array with DC charging values as relative power (0-1), other values set to 0." + ) + discharge_allowed: list[int] = Field( + description="Array with discharge values (1 for discharge, 0 otherwise)." + ) + eautocharge_hours_float: Optional[list[float]] = Field(description="TBD") + result: GeneticSimulationResult + eauto_obj: Optional[ElectricVehicleResult] + start_solution: Optional[list[float]] = Field( + default=None, + description="An array of binary values (0 or 1) representing a possible starting solution for the simulation.", + ) + washingstart: Optional[int] = Field( + default=None, + description="Can be `null` or contain an object representing the start of washing (if applicable).", + ) + + @field_validator( + "ac_charge", + "dc_charge", + "discharge_allowed", + mode="before", + ) + def convert_numpy(cls, field: Any) -> Any: + return NumpyEncoder.convert_numpy(field)[0] + + @field_validator( + "eauto_obj", + mode="before", + ) + def convert_eauto(cls, field: Any) -> Any: + if isinstance(field, Battery): + return ElectricVehicleResult(**field.to_dict()) + return field + + def _battery_operation_from_solution( + self, + ac_charge: float, + dc_charge: float, + discharge_allowed: bool, + ) -> tuple[BatteryOperationMode, float]: + """Maps low-level solution to a representative operation mode and factor. + + Args: + ac_charge (float): Allowed AC-side charging power (relative units). + dc_charge (float): Allowed DC-side charging power (relative units). + discharge_allowed (bool): Whether discharging is permitted. + + Returns: + tuple[BatteryOperationMode, float]: + A tuple containing: + - `BatteryOperationMode`: the representative high-level operation mode. + - `float`: the operation factor corresponding to the active signal. + + Notes: + - The mapping prioritizes AC charge > DC charge > discharge. + - Multiple strategies can produce the same low-level signals; this function + returns a representative mode based on a defined priority order. + """ + # (0,0,0) → Nothing allowed + if ac_charge <= 0.0 and dc_charge <= 0.0 and not discharge_allowed: + return BatteryOperationMode.IDLE, 1.0 + + # (0,0,1) → Discharge only + if ac_charge <= 0.0 and dc_charge <= 0.0 and discharge_allowed: + return BatteryOperationMode.PEAK_SHAVING, 1.0 + + # (ac>0,0,0) → AC charge only + if ac_charge > 0.0 and dc_charge <= 0.0 and not discharge_allowed: + return BatteryOperationMode.GRID_SUPPORT_IMPORT, ac_charge + + # (0,dc>0,0) → DC charge only + if ac_charge <= 0.0 and dc_charge > 0.0 and not discharge_allowed: + return BatteryOperationMode.NON_EXPORT, dc_charge + + # (ac>0,dc>0,0) → Both charge paths, no discharge + if ac_charge > 0.0 and dc_charge > 0.0 and not discharge_allowed: + return BatteryOperationMode.FORCED_CHARGE, ac_charge + + # (ac>0,0,1) → AC charge + discharge - does not make sense + if ac_charge > 0.0 and dc_charge <= 0.0 and discharge_allowed: + raise ValueError( + f"Illegal state: ac_charge: {ac_charge} and discharge_allowed: {discharge_allowed}" + ) + + # (0,dc>0,1) → DC charge + discharge + if ac_charge <= 0.0 and dc_charge > 0.0 and discharge_allowed: + return BatteryOperationMode.SELF_CONSUMPTION, dc_charge + + # (ac>0,dc>0,1) → Fully flexible - does not make sense + if ac_charge > 0.0 and dc_charge > 0.0 and discharge_allowed: + raise ValueError( + f"Illegal state: ac_charge: {ac_charge} and discharge_allowed: {discharge_allowed}" + ) + + # Fallback → safe idle + return BatteryOperationMode.IDLE, 1.0 + + def optimization_solution(self) -> OptimizationSolution: + """Provide the genetic solution as a general optimization solution. + + The battery modes are controlled by the grid control triggers: + - ac_charge: charge from grid + - discharge_allowed: discharge to grid + + The following battery modes are supported: + - SELF_CONSUMPTION: ac_charge == 0 and discharge_allowed == 0 + - GRID_SUPPORT_EXPORT: ac_charge == 0 and discharge_allowed == 1 + - GRID_SUPPORT_IMPORT: ac_charge > 0 and discharge_allowed == 0 or 1 + """ + from akkudoktoreos.core.ems import get_ems + + config = get_config() + start_datetime = get_ems().start_datetime + interval_hours = 1 + + # --- Create index based on list length and interval --- + n_points = len(self.result.Kosten_Euro_pro_Stunde) + time_index = pd.date_range( + start=start_datetime, + periods=n_points, + freq=f"{interval_hours}h", + ) + end_datetime = start_datetime.add(hours=n_points) + + # Fill data into dataframe with correct column names + # - load_energy_wh: Load of all energy consumers in wh" + # - grid_energy_wh: Grid energy feed in (negative) or consumption (positive) in wh" + # - pv_prediction_energy_wh: PV energy prediction (positive) in wh" + # - elec_price_prediction_amt_kwh: Electricity price prediction in money per kwh" + # - costs_amt: Costs in money amount" + # - revenue_amt: Revenue in money amount" + # - losses_energy_wh: Energy losses in wh" + # - __op_mode: Operation mode of the device (1.0 when active)." + # - __op_factor: Operation mode factor of the device." + # - _soc_factor: State of charge of a battery/ electric vehicle device as factor of total capacity." + # - _energy_wh: Energy consumption (positive) of a device in wh." + + data = pd.DataFrame( + { + "date_time": time_index, + "load_energy_wh": self.result.Last_Wh_pro_Stunde, + "grid_feedin_energy_wh": self.result.Netzeinspeisung_Wh_pro_Stunde, + "grid_consumption_energy_wh": self.result.Netzbezug_Wh_pro_Stunde, + "elec_price_prediction_amt_kwh": [v * 1000 for v in self.result.Electricity_price], + "costs_amt": self.result.Kosten_Euro_pro_Stunde, + "revenue_amt": self.result.Einnahmen_Euro_pro_Stunde, + "losses_energy_wh": self.result.Verluste_Pro_Stunde, + }, + index=time_index, + ) + + # Add battery data + data["battery1_soc_factor"] = [v / 100 for v in self.result.akku_soc_pro_stunde] + operation: dict[str, list[float]] = {} + for hour, rate in enumerate(self.ac_charge): + if hour >= n_points: + break + operation_mode, operation_mode_factor = self._battery_operation_from_solution( + self.ac_charge[hour], self.dc_charge[hour], bool(self.discharge_allowed[hour]) + ) + for mode in BatteryOperationMode: + mode_key = f"battery1_{mode.lower()}_op_mode" + factor_key = f"battery1_{mode.lower()}_op_factor" + if mode_key not in operation.keys(): + operation[mode_key] = [] + operation[factor_key] = [] + if mode == operation_mode: + operation[mode_key].append(1.0) + operation[factor_key].append(operation_mode_factor) + else: + operation[mode_key].append(0.0) + operation[factor_key].append(0.0) + for key in operation.keys(): + data[key] = operation[key] + + # Add EV battery data + if self.eauto_obj: + if self.eautocharge_hours_float is None: + # Electric vehicle is full enough. No load times. + data[f"{self.eauto_obj.device_id}_soc_factor"] = [ + self.eauto_obj.initial_soc_percentage / 100.0 + ] * n_points + # operation modes + operation_mode = BatteryOperationMode.IDLE + for mode in BatteryOperationMode: + mode_key = f"{self.eauto_obj.device_id}_{mode.lower()}_op_mode" + factor_key = f"{self.eauto_obj.device_id}_{mode.lower()}_op_factor" + if mode == operation_mode: + data[mode_key] = [1.0] * n_points + data[factor_key] = [1.0] * n_points + else: + data[mode_key] = [0.0] * n_points + data[factor_key] = [0.0] * n_points + else: + data[f"{self.eauto_obj.device_id}_soc_factor"] = [ + v / 100 for v in self.result.EAuto_SoC_pro_Stunde + ] + operation = {} + for hour, rate in enumerate(self.eautocharge_hours_float): + if hour >= n_points: + break + operation_mode, operation_mode_factor = self._battery_operation_from_solution( + rate, 0.0, False + ) + for mode in BatteryOperationMode: + mode_key = f"{self.eauto_obj.device_id}_{mode.lower()}_op_mode" + factor_key = f"{self.eauto_obj.device_id}_{mode.lower()}_op_factor" + if mode_key not in operation.keys(): + operation[mode_key] = [] + operation[factor_key] = [] + if mode == operation_mode: + operation[mode_key].append(1.0) + operation[factor_key].append(operation_mode_factor) + else: + operation[mode_key].append(0.0) + operation[factor_key].append(0.0) + for key in operation.keys(): + data[key] = operation[key] + + # Add home appliance data + if self.washingstart: + data["homeappliance1_energy_wh"] = self.result.Home_appliance_wh_per_hour + + # Add important predictions that are not already available from the GenericSolution + prediction = get_prediction() + power_to_energy_per_interval_factor = 1.0 + if "pvforecast_ac_power" in prediction.record_keys: + data["pv_prediction_energy_wh"] = ( + prediction.key_to_array( + key="pvforecast_ac_power", + start_datetime=start_datetime, + end_datetime=end_datetime, + interval=to_duration(f"{interval_hours} hours"), + fill_method="linear", + ) + * power_to_energy_per_interval_factor + ).tolist() + if "weather_temp_air" in prediction.record_keys: + data["weather_temp_air"] = ( + prediction.key_to_array( + key="weather_temp_air", + start_datetime=start_datetime, + end_datetime=end_datetime, + interval=to_duration(f"{interval_hours} hours"), + fill_method="linear", + ) + ).tolist() + + solution = OptimizationSolution( + id=f"optimization-genetic@{to_datetime(as_string=True)}", + generated_at=to_datetime(), + comment="Optimization solution derived from GeneticSolution.", + valid_from=start_datetime, + valid_until=start_datetime.add(hours=config.optimization.horizon_hours), + total_losses_energy_wh=self.result.Gesamt_Verluste, + total_revenues_amt=self.result.Gesamteinnahmen_Euro, + total_costs_amt=self.result.Gesamtkosten_Euro, + data=PydanticDateTimeDataFrame.from_dataframe(data), + ) + + return solution + + def energy_management_plan(self) -> EnergyManagementPlan: + """Provide the genetic solution as an energy management plan.""" + from akkudoktoreos.core.ems import get_ems + + start_datetime = get_ems().start_datetime + plan = EnergyManagementPlan( + id=f"plan-genetic@{to_datetime(as_string=True)}", + generated_at=to_datetime(), + instructions=[], + comment="Energy management plan derived from GeneticSolution.", + ) + + # Add battery instructions (fill rate based control) + last_operation_mode: Optional[str] = None + last_operation_mode_factor: Optional[float] = None + resource_id = "battery1" + logger.debug("BAT: {} - {}", resource_id, self.ac_charge) + for hour, rate in enumerate(self.ac_charge): + operation_mode, operation_mode_factor = self._battery_operation_from_solution( + self.ac_charge[hour], self.dc_charge[hour], bool(self.discharge_allowed[hour]) + ) + if ( + operation_mode == last_operation_mode + and operation_mode_factor == last_operation_mode_factor + ): + # Skip, we already added the instruction + continue + last_operation_mode = operation_mode + last_operation_mode_factor = operation_mode_factor + execution_time = start_datetime.add(hours=hour) + plan.add_instruction( + FRBCInstruction( + resource_id=resource_id, + execution_time=execution_time, + actuator_id=resource_id, + operation_mode_id=operation_mode, + operation_mode_factor=operation_mode_factor, + ) + ) + + # Add EV battery instructions (fill rate based control) + if self.eauto_obj: + resource_id = self.eauto_obj.device_id + if self.eautocharge_hours_float is None: + # Electric vehicle is full enough. No load times. + logger.debug("EV: {} - SoC >= min, no optimization", resource_id) + plan.add_instruction( + FRBCInstruction( + resource_id=resource_id, + execution_time=start_datetime, + actuator_id=resource_id, + operation_mode_id=BatteryOperationMode.IDLE, + operation_mode_factor=1.0, + ) + ) + else: + last_operation_mode = None + last_operation_mode_factor = None + logger.debug("EV: {} - {}", resource_id, self.eauto_obj.charge_array) + for hour, rate in enumerate(self.eautocharge_hours_float): + operation_mode, operation_mode_factor = self._battery_operation_from_solution( + rate, 0.0, False + ) + if ( + operation_mode == last_operation_mode + and operation_mode_factor == last_operation_mode_factor + ): + # Skip, we already added the instruction + continue + last_operation_mode = operation_mode + last_operation_mode_factor = operation_mode_factor + execution_time = start_datetime.add(hours=hour) + plan.add_instruction( + FRBCInstruction( + resource_id=resource_id, + execution_time=execution_time, + actuator_id=resource_id, + operation_mode_id=operation_mode, + operation_mode_factor=operation_mode_factor, + ) + ) + + # Add home appliance instructions (demand driven based control) + if self.washingstart: + resource_id = "homeappliance1" + operation_mode = ApplianceOperationMode.RUN # type: ignore[assignment] + operation_mode_factor = 1.0 + execution_time = start_datetime.add(hours=self.washingstart) + plan.add_instruction( + DDBCInstruction( + resource_id=resource_id, + execution_time=execution_time, + actuator_id=resource_id, + operation_mode_id=operation_mode, + operation_mode_factor=operation_mode_factor, + ) + ) + + return plan diff --git a/src/akkudoktoreos/optimization/optimization.py b/src/akkudoktoreos/optimization/optimization.py index f958e1c..74cf1ba 100644 --- a/src/akkudoktoreos/optimization/optimization.py +++ b/src/akkudoktoreos/optimization/optimization.py @@ -1,37 +1,111 @@ -from typing import List, Optional +from typing import Optional, Union from pydantic import Field from akkudoktoreos.config.configabc import SettingsBaseModel +from akkudoktoreos.core.pydantic import PydanticBaseModel, PydanticDateTimeDataFrame +from akkudoktoreos.utils.datetimeutil import DateTime + + +class GeneticCommonSettings(SettingsBaseModel): + """General Genetic Optimization Algorithm Configuration.""" + + individuals: Optional[int] = Field( + default=300, + ge=10, + description="Number of individuals (solutions) to generate for the (initial) generation [>= 10]. Defaults to 300.", + examples=[300], + ) + + generations: Optional[int] = Field( + default=400, + ge=10, + description="Number of generations to evaluate the optimal solution [>= 10]. Defaults to 400.", + examples=[400], + ) + + seed: Optional[int] = Field( + default=None, + ge=0, + description="Fixed seed for genetic algorithm. Defaults to 'None' which means random seed.", + examples=[None], + ) + + penalties: Optional[dict[str, Union[float, int, str]]] = Field( + default=None, + description="A dictionary of penalty function parameters consisting of a penalty function parameter name and the associated value.", + examples=[ + {"ev_soc_miss": 10}, + ], + ) class OptimizationCommonSettings(SettingsBaseModel): - """General Optimization Configuration. + """General Optimization Configuration.""" - Attributes: - hours (int): Number of hours for optimizations. - """ - - hours: Optional[int] = Field( - default=48, ge=0, description="Number of hours into the future for optimizations." + horizon_hours: Optional[int] = Field( + default=24, + ge=0, + description="The general time window within which the energy optimization goal shall be achieved [h]. Defaults to 24 hours.", + examples=[24], ) - penalty: Optional[int] = Field(default=10, description="Penalty factor used in optimization.") - - ev_available_charge_rates_percent: Optional[List[float]] = Field( - default=[ - 0.0, - 6.0 / 16.0, - # 7.0 / 16.0, - 8.0 / 16.0, - # 9.0 / 16.0, - 10.0 / 16.0, - # 11.0 / 16.0, - 12.0 / 16.0, - # 13.0 / 16.0, - 14.0 / 16.0, - # 15.0 / 16.0, - 1.0, - ], - description="Charge rates available for the EV in percent of maximum charge.", + interval: Optional[int] = Field( + default=3600, + ge=15 * 60, + le=60 * 60, + description="The optimization interval [sec].", + examples=[60 * 60, 15 * 60], + ) + + genetic: Optional[GeneticCommonSettings] = Field( + default=None, + description="Genetic optimization algorithm configuration.", + examples=[{"individuals": 400, "seed": None, "penalties": {"ev_soc_miss": 10}}], + ) + + +class OptimizationSolution(PydanticBaseModel): + """General Optimization Solution.""" + + id: str = Field(..., description="Unique ID for the optimization solution.") + + generated_at: DateTime = Field(..., description="Timestamp when the solution was generated.") + + comment: Optional[str] = Field( + default=None, description="Optional comment or annotation for the solution." + ) + + valid_from: Optional[DateTime] = Field( + default=None, description="Start time of the optimization solution." + ) + + valid_until: Optional[DateTime] = Field( + default=None, + description="End time of the optimization solution.", + ) + + total_losses_energy_wh: float = Field( + description="The total losses in watt-hours over the entire period." + ) + + total_revenues_amt: float = Field(description="The total revenues [money amount].") + + total_costs_amt: float = Field(description="The total costs [money amount].") + + data: PydanticDateTimeDataFrame = Field( + description=( + "Datetime data frame with time series optimization data per optimization interval:" + "- load_energy_wh: Load of all energy consumers in wh" + "- grid_energy_wh: Grid energy feed in (negative) or consumption (positive) in wh" + "- pv_prediction_energy_wh: PV energy prediction (positive) in wh" + "- elec_price_prediction_amt_kwh: Electricity price prediction in money per kwh" + "- costs_amt: Costs in money amount" + "- revenue_amt: Revenue in money amount" + "- losses_energy_wh: Energy losses in wh" + "- _operation_mode_id: Operation mode id of the device." + "- _operation_mode_factor: Operation mode factor of the device." + "- _soc_factor: State of charge of a battery/ electric vehicle device as factor of total capacity." + "- _energy_wh: Energy consumption (positive) of a device in wh." + ) ) diff --git a/src/akkudoktoreos/optimization/optimizationabc.py b/src/akkudoktoreos/optimization/optimizationabc.py index 5e49236..805132c 100644 --- a/src/akkudoktoreos/optimization/optimizationabc.py +++ b/src/akkudoktoreos/optimization/optimizationabc.py @@ -2,11 +2,14 @@ from pydantic import ConfigDict -from akkudoktoreos.core.coreabc import ConfigMixin, PredictionMixin -from akkudoktoreos.core.pydantic import PydanticBaseModel +from akkudoktoreos.core.coreabc import ( + ConfigMixin, + EnergyManagementSystemMixin, + PredictionMixin, +) -class OptimizationBase(ConfigMixin, PredictionMixin, PydanticBaseModel): +class OptimizationBase(ConfigMixin, PredictionMixin, EnergyManagementSystemMixin): """Base class for handling optimization data. Enables access to EOS configuration data (attribute `config`) and EOS prediction data (attribute diff --git a/src/akkudoktoreos/prediction/elecprice.py b/src/akkudoktoreos/prediction/elecprice.py index 83f11a0..9a0f1a0 100644 --- a/src/akkudoktoreos/prediction/elecprice.py +++ b/src/akkudoktoreos/prediction/elecprice.py @@ -17,6 +17,14 @@ elecprice_providers = [ ] +class ElecPriceCommonProviderSettings(SettingsBaseModel): + """Electricity Price Prediction Provider Configuration.""" + + ElecPriceImport: Optional[ElecPriceImportCommonSettings] = Field( + default=None, description="ElecPriceImport settings", examples=[None] + ) + + class ElecPriceCommonSettings(SettingsBaseModel): """Electricity Price Prediction Configuration.""" @@ -26,7 +34,10 @@ class ElecPriceCommonSettings(SettingsBaseModel): examples=["ElecPriceAkkudoktor"], ) charges_kwh: Optional[float] = Field( - default=None, ge=0, description="Electricity price charges (€/kWh).", examples=[0.21] + default=None, + ge=0, + description="Electricity price charges [€/kWh]. Will be added to variable market price.", + examples=[0.21], ) vat_rate: Optional[float] = Field( default=1.19, @@ -35,8 +46,15 @@ class ElecPriceCommonSettings(SettingsBaseModel): examples=[1.19], ) - provider_settings: Optional[ElecPriceImportCommonSettings] = Field( - default=None, description="Provider settings", examples=[None] + provider_settings: ElecPriceCommonProviderSettings = Field( + default_factory=ElecPriceCommonProviderSettings, + description="Provider settings", + examples=[ + # Example 1: Empty/default settings (all providers None) + { + "ElecPriceImport": None, + }, + ], ) # Validators diff --git a/src/akkudoktoreos/prediction/elecpriceakkudoktor.py b/src/akkudoktoreos/prediction/elecpriceakkudoktor.py index 65ffe13..15711fc 100644 --- a/src/akkudoktoreos/prediction/elecpriceakkudoktor.py +++ b/src/akkudoktoreos/prediction/elecpriceakkudoktor.py @@ -102,10 +102,10 @@ class ElecPriceAkkudoktor(ElecPriceProvider): - add the file cache again. """ source = "https://api.akkudoktor.net" - if not self.start_datetime: - raise ValueError(f"Start DateTime not set: {self.start_datetime}") + if not self.ems_start_datetime: + raise ValueError(f"Start DateTime not set: {self.ems_start_datetime}") # Try to take data from 5 weeks back for prediction - date = to_datetime(self.start_datetime - to_duration("35 days"), as_string="YYYY-MM-DD") + date = to_datetime(self.ems_start_datetime - to_duration("35 days"), as_string="YYYY-MM-DD") last_date = to_datetime(self.end_datetime, as_string="YYYY-MM-DD") url = f"{source}/prices?start={date}&end={last_date}&tz={self.config.general.timezone}" response = requests.get(url, timeout=10) @@ -147,8 +147,8 @@ class ElecPriceAkkudoktor(ElecPriceProvider): """ # Get Akkudoktor electricity price data akkudoktor_data = self._request_forecast(force_update=force_update) # type: ignore - if not self.start_datetime: - raise ValueError(f"Start DateTime not set: {self.start_datetime}") + if not self.ems_start_datetime: + raise ValueError(f"Start DateTime not set: {self.ems_start_datetime}") # Assumption that all lists are the same length and are ordered chronologically # in ascending order and have the same timestamps. @@ -186,13 +186,13 @@ class ElecPriceAkkudoktor(ElecPriceProvider): # some of our data is already in the future, so we need to predict less. If we got less data we increase the prediction hours needed_hours = int( self.config.prediction.hours - - ((highest_orig_datetime - self.start_datetime).total_seconds() // 3600) + - ((highest_orig_datetime - self.ems_start_datetime).total_seconds() // 3600) ) if needed_hours <= 0: logger.warning( - f"No prediction needed. needed_hours={needed_hours}, hours={self.config.prediction.hours},highest_orig_datetime {highest_orig_datetime}, start_datetime {self.start_datetime}" - ) # this might keep data longer than self.start_datetime + self.config.prediction.hours in the records + f"No prediction needed. needed_hours={needed_hours}, hours={self.config.prediction.hours},highest_orig_datetime {highest_orig_datetime}, start_datetime {self.ems_start_datetime}" + ) # this might keep data longer than self.ems_start_datetime + self.config.prediction.hours in the records return if amount_datasets > 800: # we do the full ets with seasons of 1 week diff --git a/src/akkudoktoreos/prediction/elecpriceenergycharts.py b/src/akkudoktoreos/prediction/elecpriceenergycharts.py index 5e91883..016d5da 100644 --- a/src/akkudoktoreos/prediction/elecpriceenergycharts.py +++ b/src/akkudoktoreos/prediction/elecpriceenergycharts.py @@ -91,7 +91,7 @@ class ElecPriceEnergyCharts(ElecPriceProvider): if start_date is None: # Try to take data from 5 weeks back for prediction start_date = to_datetime( - self.start_datetime - to_duration("35 days"), as_string="YYYY-MM-DD" + self.ems_start_datetime - to_duration("35 days"), as_string="YYYY-MM-DD" ) last_date = to_datetime(self.end_datetime, as_string="YYYY-MM-DD") @@ -172,17 +172,17 @@ class ElecPriceEnergyCharts(ElecPriceProvider): hours_ahead = 23 if now.time() < pd.Timestamp("14:00").time() else 47 end = midnight + pd.Timedelta(hours=hours_ahead) - if not self.start_datetime: - raise ValueError(f"Start DateTime not set: {self.start_datetime}") + if not self.ems_start_datetime: + raise ValueError(f"Start DateTime not set: {self.ems_start_datetime}") # Determine if update is needed and how many days past_days = 35 if self.highest_orig_datetime: history_series = self.key_to_series( - key="elecprice_marketprice_wh", start_datetime=self.start_datetime + key="elecprice_marketprice_wh", start_datetime=self.ems_start_datetime ) # If history lower, then start_datetime - if history_series.index.min() <= self.start_datetime: + if history_series.index.min() <= self.ems_start_datetime: past_days = 0 needs_update = end > self.highest_orig_datetime @@ -195,7 +195,7 @@ class ElecPriceEnergyCharts(ElecPriceProvider): ) # Set start_date try to take data from 5 weeks back for prediction start_date = to_datetime( - self.start_datetime - to_duration(f"{past_days} days"), as_string="YYYY-MM-DD" + self.ems_start_datetime - to_duration(f"{past_days} days"), as_string="YYYY-MM-DD" ) # Get Energy-Charts electricity price data energy_charts_data = self._request_forecast( @@ -227,13 +227,13 @@ class ElecPriceEnergyCharts(ElecPriceProvider): # some of our data is already in the future, so we need to predict less. If we got less data we increase the prediction hours needed_hours = int( self.config.prediction.hours - - ((self.highest_orig_datetime - self.start_datetime).total_seconds() // 3600) + - ((self.highest_orig_datetime - self.ems_start_datetime).total_seconds() // 3600) ) if needed_hours <= 0: logger.warning( - f"No prediction needed. needed_hours={needed_hours}, hours={self.config.prediction.hours},highest_orig_datetime {self.highest_orig_datetime}, start_datetime {self.start_datetime}" - ) # this might keep data longer than self.start_datetime + self.config.prediction.hours in the records + f"No prediction needed. needed_hours={needed_hours}, hours={self.config.prediction.hours},highest_orig_datetime {self.highest_orig_datetime}, start_datetime {self.ems_start_datetime}" + ) # this might keep data longer than self.ems_start_datetime + self.config.prediction.hours in the records return if amount_datasets > 800: # we do the full ets with seasons of 1 week diff --git a/src/akkudoktoreos/prediction/elecpriceimport.py b/src/akkudoktoreos/prediction/elecpriceimport.py index 0cdfd12..31e416f 100644 --- a/src/akkudoktoreos/prediction/elecpriceimport.py +++ b/src/akkudoktoreos/prediction/elecpriceimport.py @@ -61,15 +61,16 @@ class ElecPriceImport(ElecPriceProvider, PredictionImportProvider): return "ElecPriceImport" def _update_data(self, force_update: Optional[bool] = False) -> None: - if self.config.elecprice.provider_settings is None: + if self.config.elecprice.provider_settings.ElecPriceImport is None: logger.debug(f"{self.provider_id()} data update without provider settings.") return - if self.config.elecprice.provider_settings.import_file_path: + if self.config.elecprice.provider_settings.ElecPriceImport.import_file_path: self.import_from_file( - self.config.elecprice.provider_settings.import_file_path, + self.config.elecprice.provider_settings.ElecPriceImport.import_file_path, key_prefix="elecprice", ) - if self.config.elecprice.provider_settings.import_json: + if self.config.elecprice.provider_settings.ElecPriceImport.import_json: self.import_from_json( - self.config.elecprice.provider_settings.import_json, key_prefix="elecprice" + self.config.elecprice.provider_settings.ElecPriceImport.import_json, + key_prefix="elecprice", ) diff --git a/src/akkudoktoreos/prediction/feedintariff.py b/src/akkudoktoreos/prediction/feedintariff.py new file mode 100644 index 0000000..821f7b8 --- /dev/null +++ b/src/akkudoktoreos/prediction/feedintariff.py @@ -0,0 +1,61 @@ +from typing import Optional + +from pydantic import Field, field_validator + +from akkudoktoreos.config.configabc import SettingsBaseModel +from akkudoktoreos.prediction.feedintariffabc import FeedInTariffProvider +from akkudoktoreos.prediction.feedintarifffixed import FeedInTariffFixedCommonSettings +from akkudoktoreos.prediction.feedintariffimport import FeedInTariffImportCommonSettings +from akkudoktoreos.prediction.prediction import get_prediction + +prediction_eos = get_prediction() + +# Valid feedintariff providers +feedintariff_providers = [ + provider.provider_id() + for provider in prediction_eos.providers + if isinstance(provider, FeedInTariffProvider) +] + + +class FeedInTariffCommonProviderSettings(SettingsBaseModel): + """Feed In Tariff Prediction Provider Configuration.""" + + FeedInTariffFixed: Optional[FeedInTariffFixedCommonSettings] = Field( + default=None, description="FeedInTariffFixed settings", examples=[None] + ) + FeedInTariffImport: Optional[FeedInTariffImportCommonSettings] = Field( + default=None, description="FeedInTariffImport settings", examples=[None] + ) + + +class FeedInTariffCommonSettings(SettingsBaseModel): + """Feed In Tariff Prediction Configuration.""" + + provider: Optional[str] = Field( + default=None, + description="Feed in tariff provider id of provider to be used.", + examples=["FeedInTariffFixed", "FeedInTarifImport"], + ) + + provider_settings: FeedInTariffCommonProviderSettings = Field( + default_factory=FeedInTariffCommonProviderSettings, + description="Provider settings", + examples=[ + # Example 1: Empty/default settings (all providers None) + { + "FeedInTariffFixed": None, + "FeedInTariffImport": None, + }, + ], + ) + + # Validators + @field_validator("provider", mode="after") + @classmethod + def validate_provider(cls, value: Optional[str]) -> Optional[str]: + if value is None or value in feedintariff_providers: + return value + raise ValueError( + f"Provider '{value}' is not a valid feed in tariff provider: {feedintariff_providers}." + ) diff --git a/src/akkudoktoreos/prediction/feedintariffabc.py b/src/akkudoktoreos/prediction/feedintariffabc.py new file mode 100644 index 0000000..ec76a63 --- /dev/null +++ b/src/akkudoktoreos/prediction/feedintariffabc.py @@ -0,0 +1,58 @@ +"""Abstract and base classes for feed in tariff predictions. + +Notes: + - Ensure appropriate API keys or configurations are set up if required by external data sources. +""" + +from abc import abstractmethod +from typing import List, Optional + +from pydantic import Field, computed_field + +from akkudoktoreos.prediction.predictionabc import PredictionProvider, PredictionRecord + + +class FeedInTariffDataRecord(PredictionRecord): + """Represents a feed in tariff data record containing various price attributes at a specific datetime. + + Attributes: + date_time (Optional[AwareDatetime]): The datetime of the record. + + """ + + feed_in_tariff_wh: Optional[float] = Field(None, description="Feed in tariff per Wh (€/Wh)") + + # Computed fields + @computed_field # type: ignore[prop-decorator] + @property + def feed_in_tariff_kwh(self) -> Optional[float]: + """Feed in tariff per kWh (€/kWh). + + Convenience attribute calculated from `feed_in_tariff_wh`. + """ + if self.feed_in_tariff_wh is None: + return None + return self.feed_in_tariff_wh * 1000.0 + + +class FeedInTariffProvider(PredictionProvider): + """Abstract base class for feed in tariff providers. + + FeedInTariffProvider is a thread-safe singleton, ensuring only one instance of this class is created. + + Configuration variables: + feed in tariff_provider (str): Prediction provider for feed in tarif. + """ + + # overload + records: List[FeedInTariffDataRecord] = Field( + default_factory=list, description="List of FeedInTariffDataRecord records" + ) + + @classmethod + @abstractmethod + def provider_id(cls) -> str: + return "FeedInTariffProvider" + + def enabled(self) -> bool: + return self.provider_id() == self.config.feedintariff.provider diff --git a/src/akkudoktoreos/prediction/feedintarifffixed.py b/src/akkudoktoreos/prediction/feedintarifffixed.py new file mode 100644 index 0000000..e80cef6 --- /dev/null +++ b/src/akkudoktoreos/prediction/feedintarifffixed.py @@ -0,0 +1,48 @@ +"""Provides feed in tariff data.""" + +from typing import Optional + +from loguru import logger +from pydantic import Field + +from akkudoktoreos.config.configabc import SettingsBaseModel +from akkudoktoreos.prediction.feedintariffabc import FeedInTariffProvider +from akkudoktoreos.utils.datetimeutil import to_datetime + + +class FeedInTariffFixedCommonSettings(SettingsBaseModel): + """Common settings for elecprice fixed price.""" + + feed_in_tariff_kwh: Optional[float] = Field( + default=None, + ge=0, + description="Electricity price feed in tariff [€/kWH].", + examples=[0.078], + ) + + +class FeedInTariffFixed(FeedInTariffProvider): + """Fixed price feed in tariff data. + + FeedInTariffFixed is a singleton-based class that retrieves elecprice data. + """ + + @classmethod + def provider_id(cls) -> str: + """Return the unique identifier for the FeedInTariffFixed provider.""" + return "FeedInTariffFixed" + + def _update_data(self, force_update: Optional[bool] = False) -> None: + error_msg = "Feed in tariff not provided" + try: + feed_in_tariff = ( + self.config.feedintariff.provider_settings.FeedInTariffFixed.feed_in_tariff_kwh + ) + except: + logger.exception(error_msg) + raise ValueError(error_msg) + if feed_in_tariff is None: + logger.error(error_msg) + raise ValueError(error_msg) + feed_in_tariff_wh = feed_in_tariff / 1000 + self.update_value(to_datetime(), "feed_in_tariff_wh", feed_in_tariff_wh) diff --git a/src/akkudoktoreos/prediction/feedintariffimport.py b/src/akkudoktoreos/prediction/feedintariffimport.py new file mode 100644 index 0000000..4c30cfd --- /dev/null +++ b/src/akkudoktoreos/prediction/feedintariffimport.py @@ -0,0 +1,76 @@ +"""Retrieves feed in tariff forecast data from an import file. + +This module provides classes and mappings to manage feed in tariff data obtained from +an import file. The data is mapped to the `FeedInTariffDataRecord` format, enabling consistent +access to forecasted and historical feed in tariff attributes. +""" + +from pathlib import Path +from typing import Optional, Union + +from loguru import logger +from pydantic import Field, field_validator + +from akkudoktoreos.config.configabc import SettingsBaseModel +from akkudoktoreos.prediction.feedintariffabc import FeedInTariffProvider +from akkudoktoreos.prediction.predictionabc import PredictionImportProvider + + +class FeedInTariffImportCommonSettings(SettingsBaseModel): + """Common settings for feed in tariff data import from file or JSON string.""" + + import_file_path: Optional[Union[str, Path]] = Field( + default=None, + description="Path to the file to import feed in tariff data from.", + examples=[None, "/path/to/feedintariff.json"], + ) + import_json: Optional[str] = Field( + default=None, + description="JSON string, dictionary of feed in tariff forecast value lists.", + examples=['{"fead_in_tariff_wh": [0.000078, 0.000078, 0.000023]}'], + ) + + # Validators + @field_validator("import_file_path", mode="after") + @classmethod + def validate_feedintariffimport_file_path( + cls, value: Optional[Union[str, Path]] + ) -> Optional[Path]: + if value is None: + return None + if isinstance(value, str): + value = Path(value) + """Ensure file is available.""" + value.resolve() + if not value.is_file(): + raise ValueError(f"Import file path '{value}' is not a file.") + return value + + +class FeedInTariffImport(FeedInTariffProvider, PredictionImportProvider): + """Fetch Feed In Tariff data from import file or JSON string. + + FeedInTariffImport is a singleton-based class that retrieves fedd in tariff forecast data + from a file or JSON string and maps it to `FeedInTariffDataRecord` fields. It manages the forecast + over a range of hours into the future and retains historical data. + """ + + @classmethod + def provider_id(cls) -> str: + """Return the unique identifier for the FeedInTariffImport provider.""" + return "FeedInTariffImport" + + def _update_data(self, force_update: Optional[bool] = False) -> None: + if self.config.feedintariff.provider_settings.FeedInTariffImport is None: + logger.debug(f"{self.provider_id()} data update without provider settings.") + return + if self.config.feedintariff.provider_settings.FeedInTariffImport.import_file_path: + self.import_from_file( + self.config.provider_settings.FeedInTariffImport.import_file_path, + key_prefix="feedintariff", + ) + if self.config.feedintariff.provider_settings.FeedInTariffImport.import_json: + self.import_from_json( + self.config.feedintariff.provider_settings.FeedInTariffImport.import_json, + key_prefix="feedintariff", + ) diff --git a/src/akkudoktoreos/prediction/interpolator.py b/src/akkudoktoreos/prediction/interpolator.py index 47d1033..4d1ef7e 100644 --- a/src/akkudoktoreos/prediction/interpolator.py +++ b/src/akkudoktoreos/prediction/interpolator.py @@ -1,11 +1,11 @@ #!/usr/bin/env python import pickle -from functools import lru_cache from pathlib import Path import numpy as np from scipy.interpolate import RegularGridInterpolator +from akkudoktoreos.core.cache import cachemethod_energy_management from akkudoktoreos.core.coreabc import SingletonMixin @@ -16,8 +16,7 @@ class SelfConsumptionProbabilityInterpolator: with open(self.filepath, "rb") as file: self.interpolator: RegularGridInterpolator = pickle.load(file) # noqa: S301 - @lru_cache(maxsize=128) - def generate_points( + def _generate_points( self, load_1h_power: float, pv_power: float ) -> tuple[np.ndarray, np.ndarray]: """Generate the grid points for interpolation.""" @@ -25,8 +24,20 @@ class SelfConsumptionProbabilityInterpolator: points = np.array([np.full_like(partial_loads, load_1h_power), partial_loads]).T return points, partial_loads + @cachemethod_energy_management def calculate_self_consumption(self, load_1h_power: float, pv_power: float) -> float: - points, partial_loads = self.generate_points(load_1h_power, pv_power) + """Calculate the PV self-consumption rate using RegularGridInterpolator. + + The results are cached until the start of the next energy management run/ optimization. + + Args: + - last_1h_power: 1h power levels (W). + - pv_power: Current PV power output (W). + + Returns: + - Self-consumption rate as a float. + """ + points, partial_loads = self._generate_points(load_1h_power, pv_power) probabilities = self.interpolator(points) return probabilities.sum() diff --git a/src/akkudoktoreos/prediction/load.py b/src/akkudoktoreos/prediction/load.py index 495effe..5dc8532 100644 --- a/src/akkudoktoreos/prediction/load.py +++ b/src/akkudoktoreos/prediction/load.py @@ -1,6 +1,6 @@ """Load forecast module for load predictions.""" -from typing import Optional, Union +from typing import Optional from pydantic import Field, field_validator @@ -21,6 +21,20 @@ load_providers = [ ] +class LoadCommonProviderSettings(SettingsBaseModel): + """Load Prediction Provider Configuration.""" + + LoadAkkudoktor: Optional[LoadAkkudoktorCommonSettings] = Field( + default=None, description="LoadAkkudoktor settings", examples=[None] + ) + LoadVrm: Optional[LoadVrmCommonSettings] = Field( + default=None, description="LoadVrm settings", examples=[None] + ) + LoadImport: Optional[LoadImportCommonSettings] = Field( + default=None, description="LoadImport settings", examples=[None] + ) + + class LoadCommonSettings(SettingsBaseModel): """Load Prediction Configuration.""" @@ -30,9 +44,18 @@ class LoadCommonSettings(SettingsBaseModel): examples=["LoadAkkudoktor"], ) - provider_settings: Optional[ - Union[LoadAkkudoktorCommonSettings, LoadVrmCommonSettings, LoadImportCommonSettings] - ] = Field(default=None, description="Provider settings", examples=[None]) + provider_settings: LoadCommonProviderSettings = Field( + default_factory=LoadCommonProviderSettings, + description="Provider settings", + examples=[ + # Example 1: Empty/default settings (all providers None) + { + "LoadAkkudoktor": None, + "LoadVrm": None, + "LoadImport": None, + }, + ], + ) # Validators @field_validator("provider", mode="after") diff --git a/src/akkudoktoreos/prediction/loadakkudoktor.py b/src/akkudoktoreos/prediction/loadakkudoktor.py index 11d8d48..9c5759c 100644 --- a/src/akkudoktoreos/prediction/loadakkudoktor.py +++ b/src/akkudoktoreos/prediction/loadakkudoktor.py @@ -90,7 +90,9 @@ class LoadAkkudoktor(LoadProvider): ) # Calculate values in W by relative profile data and yearly consumption given in kWh data_year_energy = ( - profile_data * self.config.load.provider_settings.loadakkudoktor_year_energy * 1000 + profile_data + * self.config.load.provider_settings.LoadAkkudoktor.loadakkudoktor_year_energy + * 1000 ) except FileNotFoundError: error_msg = f"Error: File {load_file} not found." @@ -108,8 +110,8 @@ class LoadAkkudoktor(LoadProvider): weekday_adjust, weekend_adjust = self._calculate_adjustment(data_year_energy) # We provide prediction starting at start of day, to be compatible to old system. # End date for prediction is prediction hours from now. - date = self.start_datetime.start_of("day") - end_date = self.start_datetime.add(hours=self.config.prediction.hours) + date = self.ems_start_datetime.start_of("day") + end_date = self.ems_start_datetime.add(hours=self.config.prediction.hours) while compare_datetimes(date, end_date).lt: # Extract mean (index 0) and standard deviation (index 1) for the given day and hour # Day indexing starts at 0, -1 because of that diff --git a/src/akkudoktoreos/prediction/loadimport.py b/src/akkudoktoreos/prediction/loadimport.py index 13ce6c4..38e0fb4 100644 --- a/src/akkudoktoreos/prediction/loadimport.py +++ b/src/akkudoktoreos/prediction/loadimport.py @@ -60,10 +60,14 @@ class LoadImport(LoadProvider, PredictionImportProvider): return "LoadImport" def _update_data(self, force_update: Optional[bool] = False) -> None: - if self.config.load.provider_settings is None: + if self.config.load.provider_settings.LoadImport is None: logger.debug(f"{self.provider_id()} data update without provider settings.") return - if self.config.load.provider_settings.import_file_path: - self.import_from_file(self.config.provider_settings.import_file_path, key_prefix="load") - if self.config.load.provider_settings.import_json: - self.import_from_json(self.config.load.provider_settings.import_json, key_prefix="load") + if self.config.load.provider_settings.LoadImport.import_file_path: + self.import_from_file( + self.config.provider_settings.LoadImport.import_file_path, key_prefix="load" + ) + if self.config.load.provider_settings.LoadImport.import_json: + self.import_from_json( + self.config.load.provider_settings.LoadImport.import_json, key_prefix="load" + ) diff --git a/src/akkudoktoreos/prediction/loadvrm.py b/src/akkudoktoreos/prediction/loadvrm.py index 1d84bbf..569f6ab 100644 --- a/src/akkudoktoreos/prediction/loadvrm.py +++ b/src/akkudoktoreos/prediction/loadvrm.py @@ -4,13 +4,12 @@ from typing import Any, Optional, Union import requests from loguru import logger -from pendulum import DateTime from pydantic import Field, ValidationError from akkudoktoreos.config.configabc import SettingsBaseModel from akkudoktoreos.core.pydantic import PydanticBaseModel from akkudoktoreos.prediction.loadabc import LoadProvider -from akkudoktoreos.utils.datetimeutil import to_datetime +from akkudoktoreos.utils.datetimeutil import DateTime, to_datetime class VrmForecastRecords(PydanticBaseModel): @@ -57,8 +56,8 @@ class LoadVrm(LoadProvider): def _request_forecast(self, start_ts: int, end_ts: int) -> VrmForecastResponse: """Fetch forecast data from Victron VRM API.""" base_url = "https://vrmapi.victronenergy.com/v2/installations" - installation_id = self.config.load.provider_settings.load_vrm_idsite - api_token = self.config.load.provider_settings.load_vrm_token + installation_id = self.config.load.provider_settings.LoadVrm.load_vrm_idsite + api_token = self.config.load.provider_settings.LoadVrm.load_vrm_token url = f"{base_url}/{installation_id}/stats?type=forecast&start={start_ts}&end={end_ts}&interval=hours" headers = {"X-Authorization": f"Token {api_token}", "Content-Type": "application/json"} @@ -80,8 +79,8 @@ class LoadVrm(LoadProvider): def _update_data(self, force_update: Optional[bool] = False) -> None: """Fetch and store VRM load forecast as load_mean and related values.""" - start_date = self.start_datetime.start_of("day") - end_date = self.start_datetime.add(hours=self.config.prediction.hours) + start_date = self.ems_start_datetime.start_of("day") + end_date = self.ems_start_datetime.add(hours=self.config.prediction.hours) start_ts = int(start_date.timestamp()) end_ts = int(end_date.timestamp()) diff --git a/src/akkudoktoreos/prediction/prediction.py b/src/akkudoktoreos/prediction/prediction.py index f3f7fde..6a4f829 100644 --- a/src/akkudoktoreos/prediction/prediction.py +++ b/src/akkudoktoreos/prediction/prediction.py @@ -34,6 +34,8 @@ from akkudoktoreos.config.configabc import SettingsBaseModel from akkudoktoreos.prediction.elecpriceakkudoktor import ElecPriceAkkudoktor from akkudoktoreos.prediction.elecpriceenergycharts import ElecPriceEnergyCharts from akkudoktoreos.prediction.elecpriceimport import ElecPriceImport +from akkudoktoreos.prediction.feedintarifffixed import FeedInTariffFixed +from akkudoktoreos.prediction.feedintariffimport import FeedInTariffImport from akkudoktoreos.prediction.loadakkudoktor import LoadAkkudoktor from akkudoktoreos.prediction.loadimport import LoadImport from akkudoktoreos.prediction.loadvrm import LoadVrm @@ -67,6 +69,7 @@ class PredictionCommonSettings(SettingsBaseModel): hours: Optional[int] = Field( default=48, ge=0, description="Number of hours into the future for predictions" ) + historic_hours: Optional[int] = Field( default=48, ge=0, @@ -88,6 +91,8 @@ class Prediction(PredictionContainer): ElecPriceAkkudoktor, ElecPriceEnergyCharts, ElecPriceImport, + FeedInTariffFixed, + FeedInTariffImport, LoadAkkudoktor, LoadVrm, LoadImport, @@ -105,6 +110,8 @@ class Prediction(PredictionContainer): elecprice_akkudoktor = ElecPriceAkkudoktor() elecprice_energy_charts = ElecPriceEnergyCharts() elecprice_import = ElecPriceImport() +feedintariff_fixed = FeedInTariffFixed() +feedintariff_import = FeedInTariffImport() load_akkudoktor = LoadAkkudoktor() load_vrm = LoadVrm() load_import = LoadImport() @@ -125,6 +132,8 @@ def get_prediction() -> Prediction: elecprice_akkudoktor, elecprice_energy_charts, elecprice_import, + feedintariff_fixed, + feedintariff_import, load_akkudoktor, load_vrm, load_import, diff --git a/src/akkudoktoreos/prediction/predictionabc.py b/src/akkudoktoreos/prediction/predictionabc.py index 906eb44..4aa2e12 100644 --- a/src/akkudoktoreos/prediction/predictionabc.py +++ b/src/akkudoktoreos/prediction/predictionabc.py @@ -11,7 +11,6 @@ and manipulation of configuration and prediction data in a clear, scalable, and from typing import List, Optional from loguru import logger -from pendulum import DateTime from pydantic import Field, computed_field from akkudoktoreos.core.coreabc import MeasurementMixin @@ -23,7 +22,7 @@ from akkudoktoreos.core.dataabc import ( DataRecord, DataSequence, ) -from akkudoktoreos.utils.datetimeutil import to_duration +from akkudoktoreos.utils.datetimeutil import DateTime, to_duration class PredictionBase(DataBase, MeasurementMixin): @@ -119,17 +118,21 @@ class PredictionStartEndKeepMixin(PredictionBase): Returns: Optional[DateTime]: The calculated end datetime, or `None` if inputs are missing. """ - if self.start_datetime and self.config.prediction.hours: - end_datetime = self.start_datetime + to_duration( + if self.ems_start_datetime and self.config.prediction.hours: + end_datetime = self.ems_start_datetime + to_duration( f"{self.config.prediction.hours} hours" ) - dst_change = end_datetime.offset_hours - self.start_datetime.offset_hours - logger.debug(f"Pre: {self.start_datetime}..{end_datetime}: DST change: {dst_change}") + dst_change = end_datetime.offset_hours - self.ems_start_datetime.offset_hours + logger.debug( + f"Pre: {self.ems_start_datetime}..{end_datetime}: DST change: {dst_change}" + ) if dst_change < 0: end_datetime = end_datetime + to_duration(f"{abs(int(dst_change))} hours") elif dst_change > 0: end_datetime = end_datetime - to_duration(f"{abs(int(dst_change))} hours") - logger.debug(f"Pst: {self.start_datetime}..{end_datetime}: DST change: {dst_change}") + logger.debug( + f"Pst: {self.ems_start_datetime}..{end_datetime}: DST change: {dst_change}" + ) return end_datetime return None @@ -141,7 +144,7 @@ class PredictionStartEndKeepMixin(PredictionBase): Returns: Optional[DateTime]: The calculated retention cutoff datetime, or `None` if inputs are missing. """ - if self.start_datetime is None: + if self.ems_start_datetime is None: return None historic_hours = self.historic_hours_min() if ( @@ -149,7 +152,7 @@ class PredictionStartEndKeepMixin(PredictionBase): and self.config.prediction.historic_hours > historic_hours ): historic_hours = int(self.config.prediction.historic_hours) - return self.start_datetime - to_duration(f"{historic_hours} hours") + return self.ems_start_datetime - to_duration(f"{historic_hours} hours") @computed_field # type: ignore[prop-decorator] @property @@ -162,7 +165,7 @@ class PredictionStartEndKeepMixin(PredictionBase): end_dt = self.end_datetime if end_dt is None: return None - duration = end_dt - self.start_datetime + duration = end_dt - self.ems_start_datetime return int(duration.total_hours()) @computed_field # type: ignore[prop-decorator] @@ -176,7 +179,7 @@ class PredictionStartEndKeepMixin(PredictionBase): keep_dt = self.keep_datetime if keep_dt is None: return None - duration = self.start_datetime - keep_dt + duration = self.ems_start_datetime - keep_dt return int(duration.total_hours()) diff --git a/src/akkudoktoreos/prediction/pvforecast.py b/src/akkudoktoreos/prediction/pvforecast.py index 1dc9c88..f26b468 100644 --- a/src/akkudoktoreos/prediction/pvforecast.py +++ b/src/akkudoktoreos/prediction/pvforecast.py @@ -1,6 +1,6 @@ """PV forecast module for PV power predictions.""" -from typing import Any, List, Optional, Self, Union +from typing import Any, List, Optional, Self from pydantic import Field, computed_field, field_validator, model_validator @@ -8,8 +8,7 @@ from akkudoktoreos.config.configabc import SettingsBaseModel from akkudoktoreos.prediction.prediction import get_prediction from akkudoktoreos.prediction.pvforecastabc import PVForecastProvider from akkudoktoreos.prediction.pvforecastimport import PVForecastImportCommonSettings -from akkudoktoreos.prediction.pvforecastvrm import PVforecastVrmCommonSettings -from akkudoktoreos.utils.docs import get_model_structure_from_examples +from akkudoktoreos.prediction.pvforecastvrm import PVForecastVrmCommonSettings prediction_eos = get_prediction() @@ -121,6 +120,17 @@ class PVForecastPlaneSetting(SettingsBaseModel): return pvtechchoice +class PVForecastCommonProviderSettings(SettingsBaseModel): + """PV Forecast Provider Configuration.""" + + PVForecastImport: Optional[PVForecastImportCommonSettings] = Field( + default=None, description="PVForecastImport settings", examples=[None] + ) + PVForecastVrm: Optional[PVForecastVrmCommonSettings] = Field( + default=None, description="PVForecastVrm settings", examples=[None] + ) + + class PVForecastCommonSettings(SettingsBaseModel): """PV Forecast Configuration.""" @@ -135,20 +145,68 @@ class PVForecastCommonSettings(SettingsBaseModel): examples=["PVForecastAkkudoktor"], ) - provider_settings: Optional[ - Union[PVForecastImportCommonSettings, PVforecastVrmCommonSettings] - ] = Field(default=None, description="Provider settings", examples=[None]) + provider_settings: PVForecastCommonProviderSettings = Field( + default_factory=PVForecastCommonProviderSettings, + description="Provider settings", + examples=[ + # Example 1: Empty/default settings (all providers None) + { + "PVForecastImport": None, + "PVForecastVrm": None, + }, + ], + ) planes: Optional[list[PVForecastPlaneSetting]] = Field( default=None, description="Plane configuration.", - examples=[get_model_structure_from_examples(PVForecastPlaneSetting, True)], + examples=[ + [ + { + "surface_tilt": 10.0, + "surface_azimuth": 180.0, + "userhorizon": [10.0, 20.0, 30.0], + "peakpower": 5.0, + "pvtechchoice": "crystSi", + "mountingplace": "free", + "loss": 14.0, + "trackingtype": 0, + "optimal_surface_tilt": False, + "optimalangles": False, + "albedo": None, + "module_model": None, + "inverter_model": None, + "inverter_paco": 6000, + "modules_per_string": 20, + "strings_per_inverter": 2, + }, + { + "surface_tilt": 20.0, + "surface_azimuth": 90.0, + "userhorizon": [5.0, 15.0, 25.0], + "peakpower": 3.5, + "pvtechchoice": "crystSi", + "mountingplace": "free", + "loss": 14.0, + "trackingtype": 1, + "optimal_surface_tilt": False, + "optimalangles": False, + "albedo": None, + "module_model": None, + "inverter_model": None, + "inverter_paco": 4000, + "modules_per_string": 20, + "strings_per_inverter": 2, + }, + ] + ], ) max_planes: Optional[int] = Field( default=0, ge=0, description="Maximum number of planes that can be set", + examples=[1, 2], ) # Validators diff --git a/src/akkudoktoreos/prediction/pvforecastakkudoktor.py b/src/akkudoktoreos/prediction/pvforecastakkudoktor.py index bafae24..859fefd 100644 --- a/src/akkudoktoreos/prediction/pvforecastakkudoktor.py +++ b/src/akkudoktoreos/prediction/pvforecastakkudoktor.py @@ -330,8 +330,8 @@ class PVForecastAkkudoktor(PVForecastProvider): logger.error(f"Akkudoktor schema change: {error_msg}") raise ValueError(error_msg) - if not self.start_datetime: - raise ValueError(f"Start DateTime not set: {self.start_datetime}") + if not self.ems_start_datetime: + raise ValueError(f"Start DateTime not set: {self.ems_start_datetime}") # Iterate over forecast data points for forecast_values in zip(*akkudoktor_data.values): @@ -339,7 +339,7 @@ class PVForecastAkkudoktor(PVForecastProvider): dt = to_datetime(original_datetime, in_timezone=self.config.general.timezone) # Skip outdated forecast data - if compare_datetimes(dt, self.start_datetime.start_of("day")).lt: + if compare_datetimes(dt, self.ems_start_datetime.start_of("day")).lt: continue sum_dc_power = sum(values.dcPower for values in forecast_values) @@ -357,7 +357,7 @@ class PVForecastAkkudoktor(PVForecastProvider): if len(self) < self.config.prediction.hours: raise ValueError( f"The forecast must cover at least {self.config.prediction.hours} hours, " - f"but only {len(self)} hours starting from {self.start_datetime} " + f"but only {len(self)} hours starting from {self.ems_start_datetime} " f"were predicted." ) diff --git a/src/akkudoktoreos/prediction/pvforecastimport.py b/src/akkudoktoreos/prediction/pvforecastimport.py index 528a7a4..bccfc2c 100644 --- a/src/akkudoktoreos/prediction/pvforecastimport.py +++ b/src/akkudoktoreos/prediction/pvforecastimport.py @@ -61,16 +61,16 @@ class PVForecastImport(PVForecastProvider, PredictionImportProvider): return "PVForecastImport" def _update_data(self, force_update: Optional[bool] = False) -> None: - if self.config.pvforecast.provider_settings is None: + if self.config.pvforecast.provider_settings.PVForecastImport is None: logger.debug(f"{self.provider_id()} data update without provider settings.") return - if self.config.pvforecast.provider_settings.import_file_path is not None: + if self.config.pvforecast.provider_settings.PVForecastImport.import_file_path is not None: self.import_from_file( - self.config.pvforecast.provider_settings.import_file_path, + self.config.pvforecast.provider_settings.PVForecastImport.import_file_path, key_prefix="pvforecast", ) - if self.config.pvforecast.provider_settings.import_json is not None: + if self.config.pvforecast.provider_settings.PVForecastImport.import_json is not None: self.import_from_json( - self.config.pvforecast.provider_settings.import_json, + self.config.pvforecast.provider_settings.PVForecastImport.import_json, key_prefix="pvforecast", ) diff --git a/src/akkudoktoreos/prediction/pvforecastvrm.py b/src/akkudoktoreos/prediction/pvforecastvrm.py index 7d3ccd4..d5f4832 100644 --- a/src/akkudoktoreos/prediction/pvforecastvrm.py +++ b/src/akkudoktoreos/prediction/pvforecastvrm.py @@ -4,13 +4,12 @@ from typing import Any, Optional, Union import requests from loguru import logger -from pendulum import DateTime from pydantic import Field, ValidationError from akkudoktoreos.config.configabc import SettingsBaseModel from akkudoktoreos.core.pydantic import PydanticBaseModel from akkudoktoreos.prediction.pvforecastabc import PVForecastProvider -from akkudoktoreos.utils.datetimeutil import to_datetime +from akkudoktoreos.utils.datetimeutil import DateTime, to_datetime class VrmForecastRecords(PydanticBaseModel): @@ -24,7 +23,7 @@ class VrmForecastResponse(PydanticBaseModel): totals: dict -class PVforecastVrmCommonSettings(SettingsBaseModel): +class PVForecastVrmCommonSettings(SettingsBaseModel): """Common settings for VRM API.""" pvforecast_vrm_token: str = Field( @@ -60,8 +59,8 @@ class PVForecastVrm(PVForecastProvider): def _request_forecast(self, start_ts: int, end_ts: int) -> VrmForecastResponse: """Fetch forecast data from Victron VRM API.""" source = "https://vrmapi.victronenergy.com/v2/installations" - id_site = self.config.pvforecast.provider_settings.pvforecast_vrm_idsite - api_token = self.config.pvforecast.provider_settings.pvforecast_vrm_token + id_site = self.config.pvforecast.provider_settings.PVForecastVrm.pvforecast_vrm_idsite + api_token = self.config.pvforecast.provider_settings.PVForecastVrm.pvforecast_vrm_token headers = {"X-Authorization": f"Token {api_token}", "Content-Type": "application/json"} url = f"{source}/{id_site}/stats?type=forecast&start={start_ts}&end={end_ts}&interval=hours" logger.debug(f"Requesting VRM forecast: {url}") @@ -82,8 +81,8 @@ class PVForecastVrm(PVForecastProvider): def _update_data(self, force_update: Optional[bool] = False) -> None: """Update forecast data in the PVForecastDataRecord format.""" - start_date = self.start_datetime.start_of("day") - end_date = self.start_datetime.add(hours=self.config.prediction.hours) + start_date = self.ems_start_datetime.start_of("day") + end_date = self.ems_start_datetime.add(hours=self.config.prediction.hours) start_ts = int(start_date.timestamp()) end_ts = int(end_date.timestamp()) diff --git a/src/akkudoktoreos/prediction/weather.py b/src/akkudoktoreos/prediction/weather.py index 60a7eba..fc8244a 100644 --- a/src/akkudoktoreos/prediction/weather.py +++ b/src/akkudoktoreos/prediction/weather.py @@ -19,6 +19,14 @@ weather_providers = [ ] +class WeatherCommonProviderSettings(SettingsBaseModel): + """Weather Forecast Provider Configuration.""" + + WeatherImport: Optional[WeatherImportCommonSettings] = Field( + default=None, description="WeatherImport settings", examples=[None] + ) + + class WeatherCommonSettings(SettingsBaseModel): """Weather Forecast Configuration.""" @@ -28,8 +36,15 @@ class WeatherCommonSettings(SettingsBaseModel): examples=["WeatherImport"], ) - provider_settings: Optional[WeatherImportCommonSettings] = Field( - default=None, description="Provider settings", examples=[None] + provider_settings: WeatherCommonProviderSettings = Field( + default_factory=WeatherCommonProviderSettings, + description="Provider settings", + examples=[ + # Example 1: Empty/default settings (all providers None) + { + "WeatherImport": None, + }, + ], ) # Validators diff --git a/src/akkudoktoreos/prediction/weatherbrightsky.py b/src/akkudoktoreos/prediction/weatherbrightsky.py index 7b30414..93a17f9 100644 --- a/src/akkudoktoreos/prediction/weatherbrightsky.py +++ b/src/akkudoktoreos/prediction/weatherbrightsky.py @@ -94,7 +94,7 @@ class WeatherBrightSky(WeatherProvider): ValueError: If the API response does not include expected `weather` data. """ source = "https://api.brightsky.dev" - date = to_datetime(self.start_datetime, as_string=True) + date = to_datetime(self.ems_start_datetime, as_string=True) last_date = to_datetime(self.end_datetime, as_string=True) response = requests.get( f"{source}/weather?lat={self.config.general.latitude}&lon={self.config.general.longitude}&date={date}&last_date={last_date}&tz={self.config.general.timezone}", @@ -223,7 +223,7 @@ class WeatherBrightSky(WeatherProvider): assert key # noqa: S101 temperature = self.key_to_array( key=key, - start_datetime=self.start_datetime, + start_datetime=self.ems_start_datetime, end_datetime=self.end_datetime, interval=to_duration("1 hour"), ) @@ -236,7 +236,7 @@ class WeatherBrightSky(WeatherProvider): assert key # noqa: S101 humidity = self.key_to_array( key=key, - start_datetime=self.start_datetime, + start_datetime=self.ems_start_datetime, end_datetime=self.end_datetime, interval=to_duration("1 hour"), ) @@ -250,7 +250,10 @@ class WeatherBrightSky(WeatherProvider): data=data, index=pd.DatetimeIndex( pd.date_range( - start=self.start_datetime, end=self.end_datetime, freq="1h", inclusive="left" + start=self.ems_start_datetime, + end=self.end_datetime, + freq="1h", + inclusive="left", ) ), ) diff --git a/src/akkudoktoreos/prediction/weatherclearoutside.py b/src/akkudoktoreos/prediction/weatherclearoutside.py index ec73609..b0b53b8 100644 --- a/src/akkudoktoreos/prediction/weatherclearoutside.py +++ b/src/akkudoktoreos/prediction/weatherclearoutside.py @@ -301,7 +301,8 @@ class WeatherClearOutside(WeatherProvider): # Converting the cloud cover into Irradiance (GHI, DNI, DHI) cloud_cover = pd.Series( - data=clearout_data["Total Clouds (% Sky Obscured)"], index=clearout_data["DateTime"] + data=clearout_data["Total Clouds (% Sky Obscured)"], + index=pd.to_datetime(clearout_data["DateTime"]), ) ghi, dni, dhi = self.estimate_irradiance_from_cloud_cover( self.config.general.latitude, self.config.general.longitude, cloud_cover diff --git a/src/akkudoktoreos/prediction/weatherimport.py b/src/akkudoktoreos/prediction/weatherimport.py index 8241269..098667d 100644 --- a/src/akkudoktoreos/prediction/weatherimport.py +++ b/src/akkudoktoreos/prediction/weatherimport.py @@ -61,14 +61,16 @@ class WeatherImport(WeatherProvider, PredictionImportProvider): return "WeatherImport" def _update_data(self, force_update: Optional[bool] = False) -> None: - if self.config.weather.provider_settings is None: + if self.config.weather.provider_settings.WeatherImport is None: logger.debug(f"{self.provider_id()} data update without provider settings.") return - if self.config.weather.provider_settings.import_file_path: + if self.config.weather.provider_settings.WeatherImport.import_file_path: self.import_from_file( - self.config.weather.provider_settings.import_file_path, key_prefix="weather" + self.config.weather.provider_settings.WeatherImport.import_file_path, + key_prefix="weather", ) - if self.config.weather.provider_settings.import_json: + if self.config.weather.provider_settings.WeatherImport.import_json: self.import_from_json( - self.config.weather.provider_settings.import_json, key_prefix="weather" + self.config.weather.provider_settings.WeatherImport.import_json, + key_prefix="weather", ) diff --git a/src/akkudoktoreos/server/dash/hello.py b/src/akkudoktoreos/server/dash/about.py similarity index 72% rename from src/akkudoktoreos/server/dash/hello.py rename to src/akkudoktoreos/server/dash/about.py index 997a545..da78ee1 100644 --- a/src/akkudoktoreos/server/dash/hello.py +++ b/src/akkudoktoreos/server/dash/about.py @@ -2,9 +2,10 @@ from typing import Any from fasthtml.common import Div +from akkudoktoreos.core.version import __version__ from akkudoktoreos.server.dash.markdown import Markdown -hello_md = """![Logo](/eosdash/assets/logo.png) +about_md = f"""![Logo](/eosdash/assets/logo.png) # Akkudoktor EOSdash @@ -17,8 +18,15 @@ electricity price data, this system enables forecasting and optimization of ener over a specified period. Documentation can be found at [Akkudoktor-EOS](https://akkudoktor-eos.readthedocs.io/en/latest/). + +## Version Information + +**Current Version:** {__version__} + +**License:** Apache License + """ -def Hello(**kwargs: Any) -> Div: - return Markdown(hello_md, **kwargs) +def About(**kwargs: Any) -> Div: + return Markdown(about_md, **kwargs) diff --git a/src/akkudoktoreos/server/dash/admin.py b/src/akkudoktoreos/server/dash/admin.py index d8b1ea2..fd93ccd 100644 --- a/src/akkudoktoreos/server/dash/admin.py +++ b/src/akkudoktoreos/server/dash/admin.py @@ -56,6 +56,101 @@ def AdminButton(*c: Any, cls: Optional[Union[str, tuple]] = None, **kwargs: Any) return Button(*c, submit=False, **kwargs) +def AdminCache( + eos_host: str, eos_port: Union[str, int], data: Optional[dict], config: Optional[dict[str, Any]] +) -> tuple[str, Union[Card, list[Card]]]: + """Creates a cache management card. + + Args: + eos_host (str): The hostname of the EOS server. + eos_port (Union[str, int]): The port of the EOS server. + data (Optional[dict]): Incoming data containing action and category for processing. + + Returns: + tuple[str, Union[Card, list[Card]]]: A tuple containing the cache category label and the `Card` UI component. + """ + server = f"http://{eos_host}:{eos_port}" + eos_hostname = "EOS server" + eosdash_hostname = "EOSdash server" + + category = "cache" + + if data and data.get("category", None) == category: + # This data is for us + if data["action"] == "clear": + # Clear all cache files + try: + result = requests.post(f"{server}/v1/admin/cache/clear", timeout=10) + result.raise_for_status() + status = Success(f"Cleared all cache files on '{eos_hostname}'") + except requests.exceptions.HTTPError as e: + detail = result.json()["detail"] + status = Error(f"Can not clear all cache files on '{eos_hostname}': {e}, {detail}") + except Exception as e: + status = Error(f"Can not clear all cache files on '{eos_hostname}': {e}") + elif data["action"] == "clear-expired": + # Clear expired cache files + try: + result = requests.post(f"{server}/v1/admin/cache/clear-expired", timeout=10) + result.raise_for_status() + status = Success(f"Cleared expired cache files on '{eos_hostname}'") + except requests.exceptions.HTTPError as e: + detail = result.json()["detail"] + status = Error( + f"Can not clear expired cache files on '{eos_hostname}': {e}, {detail}" + ) + except Exception as e: + status = Error(f"Can not clear expired cache files on '{eos_hostname}': {e}") + + return ( + category, + [ + Card( + Details( + Summary( + Grid( + DivHStacked( + UkIcon(icon="play"), + AdminButton( + "Clear all", + hx_post="/eosdash/admin", + hx_target="#page-content", + hx_swap="innerHTML", + hx_vals='{"category": "cache", "action": "clear"}', + ), + P(f"cache files on '{eos_hostname}'"), + ), + ), + cls="list-none", + ), + P(f"Clear all cache files on '{eos_hostname}'."), + ), + ), + Card( + Details( + Summary( + Grid( + DivHStacked( + UkIcon(icon="play"), + AdminButton( + "Clear expired", + hx_post="/eosdash/admin", + hx_target="#page-content", + hx_swap="innerHTML", + hx_vals='{"category": "cache", "action": "clear-expired"}', + ), + P(f"cache files on '{eos_hostname}'"), + ), + ), + cls="list-none", + ), + P(f"Clear expired cache files on '{eos_hostname}'."), + ), + ), + ], + ) + + def AdminConfig( eos_host: str, eos_port: Union[str, int], data: Optional[dict], config: Optional[dict[str, Any]] ) -> tuple[str, Union[Card, list[Card]]]: @@ -282,6 +377,7 @@ def Admin(eos_host: str, eos_port: Union[str, int], data: Optional[dict] = None) rows = [] last_category = "" for category, admin in [ + AdminCache(eos_host, eos_port, data, config), AdminConfig(eos_host, eos_port, data, config), ]: if category != last_category: diff --git a/src/akkudoktoreos/server/dash/bokeh.py b/src/akkudoktoreos/server/dash/bokeh.py index 560b7f8..d6ef211 100644 --- a/src/akkudoktoreos/server/dash/bokeh.py +++ b/src/akkudoktoreos/server/dash/bokeh.py @@ -1,33 +1,87 @@ # Module taken from https://github.com/koaning/fh-altair # MIT license - from typing import Optional +import bokeh from bokeh.embed import components from bokeh.models import Plot from monsterui.franken import H4, Card, NotStr, Script +bokeh_version = bokeh.__version__ + BokehJS = [ - Script(src="https://cdn.bokeh.org/bokeh/release/bokeh-3.7.0.min.js", crossorigin="anonymous"), Script( - src="https://cdn.bokeh.org/bokeh/release/bokeh-widgets-3.7.0.min.js", + src=f"https://cdn.bokeh.org/bokeh/release/bokeh-{bokeh_version}.min.js", crossorigin="anonymous", ), Script( - src="https://cdn.bokeh.org/bokeh/release/bokeh-tables-3.7.0.min.js", crossorigin="anonymous" + src=f"https://cdn.bokeh.org/bokeh/release/bokeh-widgets-{bokeh_version}.min.js", + crossorigin="anonymous", ), Script( - src="https://cdn.bokeh.org/bokeh/release/bokeh-gl-3.7.0.min.js", crossorigin="anonymous" + src=f"https://cdn.bokeh.org/bokeh/release/bokeh-tables-{bokeh_version}.min.js", + crossorigin="anonymous", ), Script( - src="https://cdn.bokeh.org/bokeh/release/bokeh-mathjax-3.7.0.min.js", + src=f"https://cdn.bokeh.org/bokeh/release/bokeh-gl-{bokeh_version}.min.js", + crossorigin="anonymous", + ), + Script( + src=f"https://cdn.bokeh.org/bokeh/release/bokeh-mathjax-{bokeh_version}.min.js", crossorigin="anonymous", ), ] +def bokey_apply_theme_to_plot(plot: Plot, dark: bool) -> None: + """Apply a dark or light theme to a Bokeh plot. + + This function modifies the appearance of a Bokeh `Plot` object in-place, + adjusting background, border, title, axis, and grid colors based on the + `dark` parameter. + + Args: + plot (Plot): The Bokeh plot to style. + dark (bool): Whether to apply the dark theme (`True`) or light theme (`False`). + + Notes: + - This only affects the plot passed in; it does not change other plots + in the same document. + """ + if dark: + plot.background_fill_color = "#1e1e1e" + plot.border_fill_color = "#1e1e1e" + plot.title.text_color = "white" + for ax in plot.xaxis + plot.yaxis: + ax.axis_line_color = "white" + ax.major_tick_line_color = "white" + ax.major_label_text_color = "white" + ax.axis_label_text_color = "white" + # Grid lines + for grid in plot.renderers: + if hasattr(grid, "grid_line_color"): + grid.grid_line_color = "#333" + for grid in plot.xgrid + plot.ygrid: + grid.grid_line_color = "#333" + else: + plot.background_fill_color = "white" + plot.border_fill_color = "white" + plot.title.text_color = "black" + for ax in plot.xaxis + plot.yaxis: + ax.axis_line_color = "black" + ax.major_tick_line_color = "black" + ax.major_label_text_color = "black" + ax.axis_label_text_color = "black" + # Grid lines + for grid in plot.renderers: + if hasattr(grid, "grid_line_color"): + grid.grid_line_color = "#ddd" + for grid in plot.xgrid + plot.ygrid: + grid.grid_line_color = "#ddd" + + def Bokeh(plot: Plot, header: Optional[str] = None) -> Card: - """Converts an Bokeh plot to a FastHTML FT component.""" + """Convert a Bokeh plot to a FastHTML FT component.""" script, div = components(plot) if header: header = H4(header, cls="mt-2") diff --git a/src/akkudoktoreos/server/dash/components.py b/src/akkudoktoreos/server/dash/components.py index 87c489f..2ca05e9 100644 --- a/src/akkudoktoreos/server/dash/components.py +++ b/src/akkudoktoreos/server/dash/components.py @@ -1,15 +1,13 @@ from typing import Any, Optional, Union -from fasthtml.common import H1, Div, Li +from fasthtml.common import H1, Button, Div, Li from monsterui.daisy import ( Alert, AlertT, ) from monsterui.foundations import stringify -from monsterui.franken import ( +from monsterui.franken import ( # Button, Does not pass hx_vals H3, - Button, - ButtonT, Card, Container, ContainerT, @@ -246,7 +244,8 @@ def DashboardTrigger(*c: Any, cls: Optional[Union[str, tuple]] = None, **kwargs: Returns: Button: A styled `Button` component. """ - new_cls = f"{ButtonT.primary}" + # new_cls = f"{ButtonT.primary} uk-border-rounded uk-padding-small" + new_cls = "uk-btn uk-btn-primary uk-border-rounded uk-padding-medium" if cls: new_cls += f" {stringify(cls)}" kwargs["cls"] = new_cls @@ -270,6 +269,7 @@ def DashboardTabs(dashboard_items: dict[str, str]) -> Card: hx_get=f"{path}", hx_target="#page-content", hx_swap="innerHTML", + hx_vals='js:{ "dark": window.matchMedia("(prefers-color-scheme: dark)").matches }', ), ) for menu, path in dashboard_items.items() @@ -286,7 +286,9 @@ def DashboardContent(content: Any) -> Card: Returns: Card: A styled `Card` element containing the content. """ - return Card(ScrollArea(Container(content, id="page-content"), cls="h-[75vh] w-full rounded-md")) + return Card( + ScrollArea(Container(content, id="page-content"), cls="h-[75vh] w-full rounded-md"), + ) def Page( diff --git a/src/akkudoktoreos/server/dash/data/democonfig.json b/src/akkudoktoreos/server/dash/data/democonfig.json deleted file mode 100644 index db79876..0000000 --- a/src/akkudoktoreos/server/dash/data/democonfig.json +++ /dev/null @@ -1,86 +0,0 @@ -{ - "elecprice": { - "charges_kwh": 0.21, - "provider": "ElecPriceAkkudoktor" - }, - "general": { - "latitude": 52.5, - "longitude": 13.4 - }, - "prediction": { - "historic_hours": 48, - "hours": 48 - }, - "load": { - "provider": "LoadAkkudoktor", - "provider_settings": { - "loadakkudoktor_year_energy": 20000 - } - }, - "optimization": { - "hours": 48 - }, - "pvforecast": { - "planes": [ - { - "peakpower": 5.0, - "surface_azimuth": 170, - "surface_tilt": 7, - "userhorizon": [ - 20, - 27, - 22, - 20 - ], - "inverter_paco": 10000 - }, - { - "peakpower": 4.8, - "surface_azimuth": 90, - "surface_tilt": 7, - "userhorizon": [ - 30, - 30, - 30, - 50 - ], - "inverter_paco": 10000 - }, - { - "peakpower": 1.4, - "surface_azimuth": 140, - "surface_tilt": 60, - "userhorizon": [ - 60, - 30, - 0, - 30 - ], - "inverter_paco": 2000 - }, - { - "peakpower": 1.6, - "surface_azimuth": 185, - "surface_tilt": 45, - "userhorizon": [ - 45, - 25, - 30, - 60 - ], - "inverter_paco": 1400 - } - ], - "provider": "PVForecastAkkudoktor" - }, - "server": { - "startup_eosdash": true, - "host": "127.0.0.1", - "port": 8503, - "eosdash_host": "127.0.0.1", - "eosdash_port": 8504 - }, - "weather": { - "provider": "BrightSky" - } -} diff --git a/src/akkudoktoreos/server/dash/eosstatus.py b/src/akkudoktoreos/server/dash/eosstatus.py new file mode 100644 index 0000000..7bac6ee --- /dev/null +++ b/src/akkudoktoreos/server/dash/eosstatus.py @@ -0,0 +1,13 @@ +# EOS server status information + +from typing import Optional + +from akkudoktoreos.config.config import SettingsEOS +from akkudoktoreos.core.emplan import EnergyManagementPlan +from akkudoktoreos.optimization.optimization import OptimizationSolution + +# The latest information from the EOS server +eos_health: Optional[dict] = None +eos_solution: Optional[OptimizationSolution] = None +eos_plan: Optional[EnergyManagementPlan] = None +eos_config: Optional[SettingsEOS] = None diff --git a/src/akkudoktoreos/server/dash/footer.py b/src/akkudoktoreos/server/dash/footer.py index bbfdf6f..c4662a8 100644 --- a/src/akkudoktoreos/server/dash/footer.py +++ b/src/akkudoktoreos/server/dash/footer.py @@ -6,6 +6,7 @@ from monsterui.daisy import Loading, LoadingT from monsterui.franken import A, ButtonT, DivFullySpaced, P from requests.exceptions import RequestException +import akkudoktoreos.server.dash.eosstatus as eosstatus from akkudoktoreos.config.config import get_config config_eos = get_config() @@ -21,12 +22,15 @@ def get_alive(eos_host: str, eos_port: Union[str, int]) -> str: Returns: str: Alive data. """ + global eos_health result = requests.Response() try: result = requests.get(f"http://{eos_host}:{eos_port}/v1/health", timeout=10) if result.status_code == 200: - alive = result.json()["status"] + eosstatus.eos_health = result.json() + alive = eosstatus.eos_health["status"] else: + eosstatus.eos_health = None alive = f"Server responded with status code: {result.status_code}" except RequestException as e: warning_msg = f"{e}" diff --git a/src/akkudoktoreos/server/dash/plan.py b/src/akkudoktoreos/server/dash/plan.py new file mode 100644 index 0000000..eabcb8d --- /dev/null +++ b/src/akkudoktoreos/server/dash/plan.py @@ -0,0 +1,496 @@ +from typing import Optional, Union + +import pandas as pd +import requests +from bokeh.models import ColumnDataSource, LinearAxis, Range1d +from bokeh.plotting import figure +from loguru import logger +from monsterui.franken import ( + Card, + Details, + Div, + DivLAligned, + Grid, + LabelCheckboxX, + P, + Summary, + UkIcon, +) + +import akkudoktoreos.server.dash.eosstatus as eosstatus +from akkudoktoreos.config.config import SettingsEOS +from akkudoktoreos.core.emplan import ( + DDBCInstruction, + EnergyManagementInstruction, + EnergyManagementPlan, + FRBCInstruction, +) +from akkudoktoreos.optimization.optimization import OptimizationSolution +from akkudoktoreos.server.dash.bokeh import Bokeh, bokey_apply_theme_to_plot +from akkudoktoreos.server.dash.components import Error +from akkudoktoreos.utils.datetimeutil import compare_datetimes, to_datetime + +# bar width for 1 hour bars (time given in millseconds) +BAR_WIDTH_1HOUR = 1000 * 60 * 60 + +# Current state of solution displayed +solution_visible: dict[str, bool] = { + "pv_prediction_energy_wh": True, + "elec_price_prediction_amt_kwh": True, +} +solution_color: dict[str, str] = {} + + +def validate_source(source: ColumnDataSource, x_col: str = "date_time") -> None: + data = source.data + + # 1. Source has data at all + if not data: + raise ValueError("ColumnDataSource has no data.") + + # 2. x_col must be present + if x_col not in data: + raise ValueError(f"Missing expected x-axis column '{x_col}' in source.") + + # 3. All columns must have equal length + lengths = {len(v) for v in data.values()} + if len(lengths) != 1: + raise ValueError(f"ColumnDataSource columns have mismatched lengths: {lengths}") + + # 4. Must have at least one non-x column + y_columns = [c for c in data.keys() if c != x_col] + if not y_columns: + raise ValueError("No y-value columns found for plotting (only x-axis present).") + + # 5. Each y-column must have at least one valid value + for col in y_columns: + values = [v for v in data[col] if v is not None] + if not values: + raise ValueError(f"Column '{col}' contains only None/NaN or is empty.") + + +def SolutionCard(solution: OptimizationSolution, config: SettingsEOS, data: Optional[dict]) -> Grid: + """Creates a optimization solution card. + + Args: + data (Optional[dict]): Incoming data containing action and category for processing. + """ + category = "solution" + dark = False + if data and data.get("category", None) == category: + # This data is for us + if data.get("action", None) == "visible": + renderer = data.get("renderer", None) + if renderer: + solution_visible[renderer] = bool(data.get(f"{renderer}-visible", False)) + if data and data.get("dark", None) == "true": + dark = True + + df = solution.data.to_dataframe() + if df.empty or len(df.columns) <= 1: + raise ValueError(f"DataFrame is empty or missing plottable columns: {list(df.columns)}") + if "date_time" not in df.columns: + raise ValueError(f"DataFrame is missing column 'date_time': {list(df.columns)}") + + # Remove time offset from UTC to get naive local time and make bokey plot in local time + dst_offsets = df.index.map(lambda x: x.dst().total_seconds() / 3600) + if config.general is None or config.general.timezone is None: + date_time_tz = "Europe/Berlin" + else: + date_time_tz = config.general.timezone + df["date_time"] = pd.to_datetime(df["date_time"], utc=True).dt.tz_convert(date_time_tz) + + # There is a special case if we have daylight saving time change in the time series + if dst_offsets.nunique() > 1: + date_time_tz += " + DST change" + + source = ColumnDataSource(df) + validate_source(source) + + # Calculate minimum and maximum Range + energy_wh_min = 0.0 + energy_wh_max = 0.0 + amt_kwh_min = 0.0 + amt_kwh_max = 0.0 + amt_min = 0.0 + amt_max = 0.0 + soc_factor_min = 0.0 + soc_factor_max = 1.0 + for col in df.columns: + if col.endswith("energy_wh"): + energy_wh_min = min(energy_wh_min, float(df[col].min())) + energy_wh_max = max(energy_wh_max, float(df[col].max())) + elif col.endswith("amt_kwh"): + amt_kwh_min = min(amt_kwh_min, float(df[col].min())) + amt_kwh_max = max(amt_kwh_max, float(df[col].max())) + elif col.endswith("amt"): + amt_min = min(amt_min, float(df[col].min())) + amt_max = max(amt_max, float(df[col].max())) + else: + continue + # Adjust to similar y-axis 0-point + # First get the maximum factor for the min value related the maximum value + min_max_factor = max( + (energy_wh_min * -1.0) / energy_wh_max, + (amt_kwh_min * -1.0) / amt_kwh_max, + (amt_min * -1.0) / amt_max, + (soc_factor_min * -1.0) / soc_factor_max, + ) + # Adapt the min values to have the same relative min/max factor on all y-axis + energy_wh_min = min_max_factor * energy_wh_max * -1.0 + amt_kwh_min = min_max_factor * amt_kwh_max * -1.0 + amt_min = min_max_factor * amt_max * -1.0 + soc_factor_min = min_max_factor * soc_factor_max * -1.0 + # add 5% to min and max values for better display + energy_wh_range_orig = energy_wh_max - energy_wh_min + energy_wh_max += 0.05 * energy_wh_range_orig + energy_wh_min -= 0.05 * energy_wh_range_orig + amt_kwh_range_orig = amt_kwh_max - amt_kwh_min + amt_kwh_max += 0.05 * amt_kwh_range_orig + amt_kwh_min -= 0.05 * amt_kwh_range_orig + amt_range_orig = amt_max - amt_min + amt_max += 0.05 * amt_range_orig + amt_min -= 0.05 * amt_range_orig + soc_factor_range_orig = soc_factor_max - soc_factor_min + soc_factor_max += 0.05 * soc_factor_range_orig + soc_factor_min -= 0.05 * soc_factor_range_orig + + if eosstatus.eos_health is not None: + last_run_datetime = eosstatus.eos_health["energy-management"]["last_run_datetime"] + start_datetime = eosstatus.eos_health["energy-management"]["start_datetime"] + else: + last_run_datetime = "unknown" + start_datetime = "unknown" + + plot = figure( + title=f"Optimization Solution - last run: {last_run_datetime}", + x_axis_type="datetime", + x_axis_label=f"Datetime [localtime {date_time_tz}] - start: {start_datetime}", + y_axis_label="Energy [Wh]", + sizing_mode="stretch_width", + y_range=Range1d(energy_wh_min, energy_wh_max), + height=400, + ) + + plot.extra_y_ranges = { + "factor": Range1d(soc_factor_min, soc_factor_max), # y2 + "amt_kwh": Range1d(amt_kwh_min, amt_kwh_max), # y3 + "amt": Range1d(amt_min, amt_max), # y4 + } + # y2 axis + y2_axis = LinearAxis(y_range_name="factor", axis_label="Factor [0.0..1.0]") + plot.add_layout(y2_axis, "left") + # y3 axis + y3_axis = LinearAxis(y_range_name="amt_kwh", axis_label="Electricty Price [currency/kWh]") + y3_axis.axis_label_text_color = "red" + plot.add_layout(y3_axis, "right") + # y4 axis + y4_axis = LinearAxis(y_range_name="amt", axis_label="Amount [currency]") + plot.add_layout(y4_axis, "right") + + plot.toolbar.autohide = True + + # Create line renderers for each column + renderers = {} + colors = ["black", "blue", "cyan", "green", "orange", "pink", "purple"] + + for i, col in enumerate(sorted(df.columns)): + # Exclude some columns that are currently not used or are covered by others + excludes = [ + "date_time", + "_op_mode", + "_fault_", + "_forced_discharge_", + "_outage_supply_", + "_reserve_backup_", + "_ramp_rate_control_", + "_frequency_regulation_", + "_grid_support_export_", + "_peak_shaving_", + ] + # excludes = ["date_time"] + if any(exclude in col for exclude in excludes): + continue + if col in solution_visible: + visible = solution_visible[col] + else: + visible = False + solution_visible[col] = visible + if col in solution_color: + color = solution_color[col] + elif col == "pv_prediction_energy_wh": + color = "yellow" + solution_color[col] = color + elif col == "elec_price_prediction_amt_kwh": + color = "red" + solution_color[col] = color + else: + color = colors[i % len(colors)] + solution_color[col] = color + if visible: + if col == "pv_prediction_energy_wh": + r = plot.vbar( + x="date_time", + top=col, + source=source, + width=BAR_WIDTH_1HOUR * 0.8, + legend_label=col, + color=color, + level="underlay", + ) + elif col.endswith("energy_wh"): + r = plot.step( + x="date_time", + y=col, + mode="before", + source=source, + legend_label=col, + color=color, + ) + elif col.endswith("factor"): + r = plot.step( + x="date_time", + y=col, + mode="before", + source=source, + legend_label=col, + color=color, + y_range_name="factor", + ) + elif col.endswith("mode"): + r = plot.step( + x="date_time", + y=col, + mode="before", + source=source, + legend_label=col, + color=color, + y_range_name="factor", + ) + elif col.endswith("amt_kwh"): + r = plot.step( + x="date_time", + y=col, + mode="before", + source=source, + legend_label=col, + color=color, + y_range_name="amt_kwh", + ) + elif col.endswith("amt"): + r = plot.step( + x="date_time", + y=col, + mode="before", + source=source, + legend_label=col, + color=color, + y_range_name="amt", + ) + else: + raise ValueError(f"Unexpected column name: {col}") + + else: + r = None + renderers[col] = r + plot.legend.visible = False # no legend at plot + bokey_apply_theme_to_plot(plot, dark) + + # --- CheckboxGroup to toggle datasets --- + Checkbox = Grid( + *[ + LabelCheckboxX( + label=renderer, + id=f"{renderer}-visible", + name=f"{renderer}-visible", + value="true", + checked=solution_visible[renderer], + hx_post="/eosdash/plan", + hx_target="#page-content", + hx_swap="innerHTML", + hx_vals='js:{ "category": "solution", "action": "visible", "renderer": ' + + '"' + + f"{renderer}" + + '", ' + + '"dark": window.matchMedia("(prefers-color-scheme: dark)").matches ' + + "}", + lbl_cls=f"text-{solution_color[renderer]}-500", + ) + for renderer in list(renderers.keys()) + ], + cols=2, + ) + + return Grid( + Bokeh(plot), + Card( + Checkbox, + ), + cls="w-full space-y-3 space-x-3", + ) + + +def InstructionCard( + instruction: EnergyManagementInstruction, config: SettingsEOS, data: Optional[dict] +) -> Card: + """Creates a styled instruction card for displaying instruction details. + + This function generates a instruction card that is displayed in the UI with + various sections such as instruction name, type, description, default value, + current value, and error details. It supports both read-only and editable modes. + + Args: + instruction (EnergyManagementInstruction): The instruction. + data (Optional[dict]): Incoming data containing action and category for processing. + + Returns: + Card: A styled Card component containing the instruction details. + """ + if instruction.id is None: + return Error("Instruction without id encountered. Can not handle") + idx = instruction.id.find("@") + resource_id = instruction.id[:idx] if idx != -1 else instruction.id + execution_time = to_datetime(instruction.execution_time, as_string=True) + description = instruction.type + summary = None + # Search an icon that fits to device_id + if ( + config.devices + and config.devices.batteries + and any( + battery_config.device_id == resource_id for battery_config in config.devices.batteries + ) + ): + # This is a battery + if instruction.operation_mode_id in ("CHARGE",): + icon = "battery-charging" + else: + icon = "battery" + elif ( + config.devices + and config.devices.electric_vehicles + and any( + electric_vehicle_config.device_id == resource_id + for electric_vehicle_config in config.devices.electric_vehicles + ) + ): + # This is a car battery + icon = "car" + elif ( + config.devices + and config.devices.home_appliances + and any( + home_appliance.device_id == resource_id + for home_appliance in config.devices.home_appliances + ) + ): + # This is a home appliance + icon = "washing-machine" + else: + icon = "play" + if isinstance(instruction, (DDBCInstruction, FRBCInstruction)): + summary = f"{instruction.operation_mode_id}" + summary_detail = f"{instruction.operation_mode_factor}" + return Card( + Details( + Summary( + Grid( + Grid( + DivLAligned( + UkIcon(icon=icon), + P(execution_time), + ), + DivLAligned( + P(resource_id), + ), + ), + P(summary), + P(summary_detail), + ), + cls="list-none", + ), + Grid( + P(description), + P("TBD"), + ), + ), + cls="w-full", + ) + + +def Plan(eos_host: str, eos_port: Union[str, int], data: Optional[dict] = None) -> Div: + """Generates the plan dashboard layout. + + Args: + eos_host (str): The hostname of the EOS server. + eos_port (Union[str, int]): The port of the EOS server. + data (Optional[dict], optional): Incoming data to trigger plan actions. Defaults to None. + + Returns: + Div: A `Div` component containing the assembled admin interface. + """ + server = f"http://{eos_host}:{eos_port}" + + print("Plan: ", data) + + if ( + eosstatus.eos_config is None + or eosstatus.eos_solution is None + or eosstatus.eos_plan is None + or eosstatus.eos_health is None + or compare_datetimes( + to_datetime(eosstatus.eos_plan.generated_at), + to_datetime(eosstatus.eos_health["energy-management"]["last_run_datetime"]), + ).lt + ): + # Get current configuration from server + try: + result = requests.get(f"{server}/v1/config", timeout=10) + result.raise_for_status() + except requests.exceptions.HTTPError as err: + detail = result.json()["detail"] + return Error(f"Can not retrieve configuration from {server}: {err}, {detail}") + eosstatus.eos_config = SettingsEOS(**result.json()) + + # Get the optimization solution + try: + result = requests.get( + f"{server}/v1/energy-management/optimization/solution", timeout=10 + ) + result.raise_for_status() + solution_json = result.json() + except requests.exceptions.HTTPError as e: + detail = result.json()["detail"] + warning_msg = f"Can not retrieve optimization solution from {server}: {e}, {detail}" + logger.warning(warning_msg) + return Error(warning_msg) + except Exception as e: + warning_msg = f"Can not retrieve optimization solution from {server}: {e}" + logger.warning(warning_msg) + return Error(warning_msg) + eosstatus.eos_solution = OptimizationSolution(**solution_json) + + # Get the plan + try: + result = requests.get(f"{server}/v1/energy-management/plan", timeout=10) + result.raise_for_status() + plan_json = result.json() + except requests.exceptions.HTTPError as e: + detail = result.json()["detail"] + warning_msg = f"Can not retrieve plan from {server}: {e}, {detail}" + logger.warning(warning_msg) + return Error(warning_msg) + except Exception as e: + warning_msg = f"Can not retrieve plan from {server}: {e}" + logger.warning(warning_msg) + return Error(warning_msg) + eosstatus.eos_plan = EnergyManagementPlan(**plan_json, data=data) + + rows = [ + SolutionCard(eosstatus.eos_solution, eosstatus.eos_config, data=data), + ] + for instruction in eosstatus.eos_plan.instructions: + rows.append(InstructionCard(instruction, eosstatus.eos_config, data=data)) + return Div(*rows, cls="space-y-4") + + # return Div(f"Plan:\n{json.dumps(plan_json, indent=4)}") diff --git a/src/akkudoktoreos/server/dash/demo.py b/src/akkudoktoreos/server/dash/prediction.py similarity index 68% rename from src/akkudoktoreos/server/dash/demo.py rename to src/akkudoktoreos/server/dash/prediction.py index b7a0c06..ec491e6 100644 --- a/src/akkudoktoreos/server/dash/demo.py +++ b/src/akkudoktoreos/server/dash/prediction.py @@ -1,6 +1,4 @@ -import json -from pathlib import Path -from typing import Union +from typing import Optional, Union import pandas as pd import requests @@ -9,30 +7,25 @@ from bokeh.plotting import figure from monsterui.franken import FT, Grid, P from akkudoktoreos.core.pydantic import PydanticDateTimeDataFrame -from akkudoktoreos.server.dash.bokeh import Bokeh - -DIR_DEMODATA = Path(__file__).absolute().parent.joinpath("data") -FILE_DEMOCONFIG = DIR_DEMODATA.joinpath("democonfig.json") -if not FILE_DEMOCONFIG.exists(): - raise ValueError(f"File does not exist: {FILE_DEMOCONFIG}") +from akkudoktoreos.server.dash.bokeh import Bokeh, bokey_apply_theme_to_plot +from akkudoktoreos.server.dash.components import Error # bar width for 1 hour bars (time given in millseconds) BAR_WIDTH_1HOUR = 1000 * 60 * 60 -def DemoPVForecast(predictions: pd.DataFrame, config: dict) -> FT: +def PVForecast(predictions: pd.DataFrame, config: dict, date_time_tz: str, dark: bool) -> FT: source = ColumnDataSource(predictions) provider = config["pvforecast"]["provider"] plot = figure( x_axis_type="datetime", title=f"PV Power Prediction ({provider})", - x_axis_label="Datetime", + x_axis_label=f"Datetime [localtime {date_time_tz}]", y_axis_label="Power [W]", sizing_mode="stretch_width", height=400, ) - plot.vbar( x="date_time", top="pvforecast_ac_power", @@ -41,11 +34,15 @@ def DemoPVForecast(predictions: pd.DataFrame, config: dict) -> FT: legend_label="AC Power", color="lightblue", ) + plot.toolbar.autohide = True + bokey_apply_theme_to_plot(plot, dark) return Bokeh(plot) -def DemoElectricityPriceForecast(predictions: pd.DataFrame, config: dict) -> FT: +def ElectricityPriceForecast( + predictions: pd.DataFrame, config: dict, date_time_tz: str, dark: bool +) -> FT: source = ColumnDataSource(predictions) provider = config["elecprice"]["provider"] @@ -56,7 +53,7 @@ def DemoElectricityPriceForecast(predictions: pd.DataFrame, config: dict) -> FT: predictions["elecprice_marketprice_kwh"].max() + 0.1, ), title=f"Electricity Price Prediction ({provider})", - x_axis_label="Datetime", + x_axis_label=f"Datetime [localtime {date_time_tz}]", y_axis_label="Price [€/kWh]", sizing_mode="stretch_width", height=400, @@ -69,18 +66,22 @@ def DemoElectricityPriceForecast(predictions: pd.DataFrame, config: dict) -> FT: legend_label="Market Price", color="lightblue", ) + plot.toolbar.autohide = True + bokey_apply_theme_to_plot(plot, dark) return Bokeh(plot) -def DemoWeatherTempAirHumidity(predictions: pd.DataFrame, config: dict) -> FT: +def WeatherTempAirHumidityForecast( + predictions: pd.DataFrame, config: dict, date_time_tz: str, dark: bool +) -> FT: source = ColumnDataSource(predictions) provider = config["weather"]["provider"] plot = figure( x_axis_type="datetime", title=f"Air Temperature and Humidity Prediction ({provider})", - x_axis_label="Datetime", + x_axis_label=f"Datetime [localtime {date_time_tz}]", y_axis_label="Temperature [°C]", sizing_mode="stretch_width", height=400, @@ -94,7 +95,6 @@ def DemoWeatherTempAirHumidity(predictions: pd.DataFrame, config: dict) -> FT: plot.line( "date_time", "weather_temp_air", source=source, legend_label="Air Temperature", color="blue" ) - plot.line( "date_time", "weather_relative_humidity", @@ -103,18 +103,22 @@ def DemoWeatherTempAirHumidity(predictions: pd.DataFrame, config: dict) -> FT: color="green", y_range_name="humidity", ) + plot.toolbar.autohide = True + bokey_apply_theme_to_plot(plot, dark) return Bokeh(plot) -def DemoWeatherIrradiance(predictions: pd.DataFrame, config: dict) -> FT: +def WeatherIrradianceForecast( + predictions: pd.DataFrame, config: dict, date_time_tz: str, dark: bool +) -> FT: source = ColumnDataSource(predictions) provider = config["weather"]["provider"] plot = figure( x_axis_type="datetime", title=f"Irradiance Prediction ({provider})", - x_axis_label="Datetime", + x_axis_label=f"Datetime [localtime {date_time_tz}]", y_axis_label="Irradiance [W/m2]", sizing_mode="stretch_width", height=400, @@ -140,21 +144,25 @@ def DemoWeatherIrradiance(predictions: pd.DataFrame, config: dict) -> FT: legend_label="Diffuse Horizontal Irradiance", color="blue", ) + plot.toolbar.autohide = True + bokey_apply_theme_to_plot(plot, dark) return Bokeh(plot) -def DemoLoad(predictions: pd.DataFrame, config: dict) -> FT: +def LoadForecast(predictions: pd.DataFrame, config: dict, date_time_tz: str, dark: bool) -> FT: source = ColumnDataSource(predictions) provider = config["load"]["provider"] if provider == "LoadAkkudoktor": - year_energy = config["load"]["provider_settings"]["loadakkudoktor_year_energy"] + year_energy = config["load"]["provider_settings"]["LoadAkkudoktor"][ + "loadakkudoktor_year_energy" + ] provider = f"{provider}, {year_energy} kWh" plot = figure( - x_axis_type="datetime", title=f"Load Prediction ({provider})", - x_axis_label="Datetime", + x_axis_type="datetime", + x_axis_label=f"Datetime [localtime {date_time_tz}]", y_axis_label="Load [W]", sizing_mode="stretch_width", height=400, @@ -189,13 +197,19 @@ def DemoLoad(predictions: pd.DataFrame, config: dict) -> FT: color="green", y_range_name="stddev", ) + plot.toolbar.autohide = True + bokey_apply_theme_to_plot(plot, dark) return Bokeh(plot) -def Demo(eos_host: str, eos_port: Union[str, int]) -> str: +def Prediction(eos_host: str, eos_port: Union[str, int], data: Optional[dict] = None) -> str: server = f"http://{eos_host}:{eos_port}" + dark = False + if data and data.get("dark", None) == "true": + dark = True + # Get current configuration from server try: result = requests.get(f"{server}/v1/config", timeout=10) @@ -208,34 +222,6 @@ def Demo(eos_host: str, eos_port: Union[str, int]) -> str: ) config = result.json() - # Set demo configuration - with FILE_DEMOCONFIG.open("r", encoding="utf-8") as fd: - democonfig = json.load(fd) - try: - result = requests.put(f"{server}/v1/config", json=democonfig, timeout=10) - result.raise_for_status() - except requests.exceptions.HTTPError as err: - detail = result.json()["detail"] - # Try to reset to original config - requests.put(f"{server}/v1/config", json=config, timeout=10) - return P( - f"Can not set demo configuration on {server}: {err}, {detail}", - cls="text-center", - ) - - # Update all predictions - try: - result = requests.post(f"{server}/v1/prediction/update", timeout=10) - result.raise_for_status() - except requests.exceptions.HTTPError as err: - detail = result.json()["detail"] - # Try to reset to original config - requests.put(f"{server}/v1/config", json=config, timeout=10) - return P( - f"Can not update predictions on {server}: {err}, {detail}", - cls="text-center", - ) - # Get Forecasts try: params = { @@ -257,24 +243,19 @@ def Demo(eos_host: str, eos_port: Union[str, int]) -> str: predictions = PydanticDateTimeDataFrame(**result.json()).to_dataframe() except requests.exceptions.HTTPError as err: detail = result.json()["detail"] - return P( - f"Can not retrieve predictions from {server}: {err}, {detail}", - cls="text-center", - ) + return Error(f"Can not retrieve predictions from {server}: {err}, {detail}") except Exception as err: - return P( - f"Can not retrieve predictions from {server}: {err}", - cls="text-center", - ) + return Error(f"Can not retrieve predictions from {server}: {err}") - # Reset to original config - requests.put(f"{server}/v1/config", json=config, timeout=10) + # Remove time offset from UTC to get naive local time and make bokeh plot in local time + date_time_tz = predictions["date_time"].dt.tz + predictions["date_time"] = pd.to_datetime(predictions["date_time"]).dt.tz_localize(None) return Grid( - DemoPVForecast(predictions, democonfig), - DemoElectricityPriceForecast(predictions, democonfig), - DemoWeatherTempAirHumidity(predictions, democonfig), - DemoWeatherIrradiance(predictions, democonfig), - DemoLoad(predictions, democonfig), + PVForecast(predictions, config, date_time_tz, dark), + ElectricityPriceForecast(predictions, config, date_time_tz, dark), + WeatherTempAirHumidityForecast(predictions, config, date_time_tz, dark), + WeatherIrradianceForecast(predictions, config, date_time_tz, dark), + LoadForecast(predictions, config, date_time_tz, dark), cols_max=2, ) diff --git a/src/akkudoktoreos/server/eos.py b/src/akkudoktoreos/server/eos.py index 9dede18..40de4e6 100755 --- a/src/akkudoktoreos/server/eos.py +++ b/src/akkudoktoreos/server/eos.py @@ -29,7 +29,9 @@ from loguru import logger from akkudoktoreos.config.config import ConfigEOS, SettingsEOS, get_config from akkudoktoreos.core.cache import CacheFileStore +from akkudoktoreos.core.emplan import EnergyManagementPlan, ResourceStatus from akkudoktoreos.core.ems import get_ems +from akkudoktoreos.core.emsettings import EnergyManagementMode from akkudoktoreos.core.logabc import LOGGING_LEVELS from akkudoktoreos.core.logging import read_file_log, track_logging_config from akkudoktoreos.core.pydantic import ( @@ -38,43 +40,154 @@ from akkudoktoreos.core.pydantic import ( PydanticDateTimeDataFrame, PydanticDateTimeSeries, ) +from akkudoktoreos.core.version import __version__ +from akkudoktoreos.devices.devices import ResourceKey, get_resource_registry from akkudoktoreos.measurement.measurement import get_measurement -from akkudoktoreos.optimization.genetic import ( - OptimizationParameters, - OptimizeResponse, - optimization_problem, +from akkudoktoreos.optimization.genetic.geneticparams import ( + GeneticOptimizationParameters, ) +from akkudoktoreos.optimization.genetic.geneticsolution import GeneticSolution +from akkudoktoreos.optimization.optimization import OptimizationSolution from akkudoktoreos.prediction.elecprice import ElecPriceCommonSettings -from akkudoktoreos.prediction.load import LoadCommonSettings +from akkudoktoreos.prediction.load import LoadCommonProviderSettings, LoadCommonSettings from akkudoktoreos.prediction.loadakkudoktor import LoadAkkudoktorCommonSettings -from akkudoktoreos.prediction.prediction import PredictionCommonSettings, get_prediction +from akkudoktoreos.prediction.prediction import get_prediction from akkudoktoreos.prediction.pvforecast import PVForecastCommonSettings from akkudoktoreos.server.rest.error import create_error_page from akkudoktoreos.server.rest.tasks import repeat_every from akkudoktoreos.server.server import ( get_default_host, - is_valid_ip_or_hostname, + get_host_ip, + validate_ip_or_hostname, wait_for_port_free, ) from akkudoktoreos.utils.datetimeutil import to_datetime, to_duration +from akkudoktoreos.utils.stringutil import str2bool config_eos = get_config() measurement_eos = get_measurement() prediction_eos = get_prediction() ems_eos = get_ems() - -# Command line arguments -args = None +resource_registry_eos = get_resource_registry() -# ---------------------- -# Logging configuration -# ---------------------- +# ------------------------------------ +# Logging configuration at import time +# ------------------------------------ logger.remove() track_logging_config(config_eos, "logging", None, None) config_eos.track_nested_value("/logging", track_logging_config) + +# ---------------------------- +# Safe argparse at import time +# ---------------------------- + +parser = argparse.ArgumentParser(description="Start EOS server.") + +parser.add_argument( + "--host", + type=str, + help="Host for the EOS server (default: value from config)", +) +parser.add_argument( + "--port", + type=int, + help="Port for the EOS server (default: value from config)", +) +parser.add_argument( + "--log_level", + type=str, + default="none", + help='Log level for the server console. Options: "critical", "error", "warning", "info", "debug", "trace" (default: "none")', +) +parser.add_argument( + "--reload", + type=str2bool, + default=False, + help="Enable or disable auto-reload. Useful for development. Options: True or False (default: False)", +) +parser.add_argument( + "--startup_eosdash", + type=str2bool, + default=None, + help="Enable or disable automatic EOSdash startup. Options: True or False (default: value from config)", +) + +# Command line arguments +args: argparse.Namespace +args_unknown: list[str] +args, args_unknown = parser.parse_known_args() + + +# ----------------------------- +# Prepare config at import time +# ----------------------------- + +# Set config to actual environment variable & config file content +config_eos.reset_settings() + +# Setup parameters from args, config_eos and default +# Remember parameters in config + +# Setup EOS logging level - first to have the other logging messages logged +if args and args.log_level is not None: + log_level = args.log_level.upper() + # Ensure log_level from command line is in config settings + if log_level in LOGGING_LEVELS: + # Setup console logging level using nested value + # - triggers logging configuration by track_logging_config + config_eos.set_nested_value("logging/console_level", log_level) + logger.debug(f"logging/console_level configuration set by argument to {log_level}") + +# Setup EOS server host +if args and args.host: + host = args.host + logger.debug(f"server/host configuration set by argument to {host}") +elif config_eos.server.host: + host = config_eos.server.host +else: + host = get_default_host() +# Ensure host from command line is in config settings +config_eos.set_nested_value("server/host", host) + +# Setup EOS server port +if args and args.port: + port = args.port + logger.debug(f"server/port configuration set by argument to {port}") +elif config_eos.server.port: + port = config_eos.server.port +else: + port = 8503 +# Ensure port from command line is in config settings +config_eos.set_nested_value("server/port", port) + +# Setup EOS reload for development +if args is None or args.reload is None: + reload = False +else: + logger.debug(f"reload set by argument to {args.reload}") + reload = args.reload + +# Setup EOSdash startup +if args and args.startup_eosdash is not None: + # Ensure startup_eosdash from command line is in config settings + config_eos.set_nested_value("server/startup_eosdash", args.startup_eosdash) + logger.debug(f"server/startup_eosdash configuration set by argument to {args.startup_eosdash}") + +if config_eos.server.startup_eosdash: + # Ensure EOSdash host and port config settings are at least set to default values + + # Setup EOS server host + if config_eos.server.eosdash_host is None: + config_eos.set_nested_value("server/eosdash_host", host) + + # Setup EOS server host + if config_eos.server.eosdash_port is None: + config_eos.set_nested_value("server/eosdash_port", port + 1) + + # ---------------------- # EOSdash server startup # ---------------------- @@ -94,7 +207,8 @@ def start_eosdash( """Start the EOSdash server as a subprocess. This function starts the EOSdash server by launching it as a subprocess. It checks if the server - is already running on the specified port and either returns the existing process or starts a new one. + is already running on the specified port and either returns the existing process or starts a new + one. Args: host (str): The hostname for the EOSdash server. @@ -113,10 +227,13 @@ def start_eosdash( Raises: RuntimeError: If the EOSdash server fails to start. """ - if not is_valid_ip_or_hostname(host): - raise ValueError(f"Invalid EOSdash host: {host}") - if not is_valid_ip_or_hostname(eos_host): - raise ValueError(f"Invalid EOS host: {eos_host}") + try: + validate_ip_or_hostname(host) + validate_ip_or_hostname(eos_host) + except Exception as ex: + error_msg = f"Could not start EOSdash: {ex}" + logger.error(error_msg) + raise RuntimeError(error_msg) eosdash_path = Path(__file__).parent.resolve().joinpath("eosdash.py") @@ -155,11 +272,18 @@ def start_eosdash( stderr=subprocess.PIPE, start_new_session=True, ) + logger.info(f"Started EOSdash with '{cmd}'.") except subprocess.CalledProcessError as ex: error_msg = f"Could not start EOSdash: {ex}" logger.error(error_msg) raise RuntimeError(error_msg) + # Check EOSdash is still running + if server_process.poll() is not None: + error_msg = f"EOSdash finished immediatedly with code: {server_process.returncode}" + logger.error(error_msg) + raise RuntimeError(error_msg) + return server_process @@ -168,6 +292,29 @@ def start_eosdash( # ---------------------- +def save_eos_state() -> None: + """Save EOS state.""" + resource_registry_eos.save() + cache_save() # keep last + + +def load_eos_state() -> None: + """Load EOS state.""" + cache_load() # keep first + resource_registry_eos.load() + + +def terminate_eos() -> None: + """Gracefully shut down the EOS server process.""" + pid = psutil.Process().pid + if os.name == "nt": + os.kill(pid, signal.CTRL_C_EVENT) # type: ignore[attr-defined,unused-ignore] + else: + os.kill(pid, signal.SIGTERM) # type: ignore[attr-defined,unused-ignore] + + logger.info(f"🚀 EOS terminated, PID {pid}") + + def cache_clear(clear_all: Optional[bool] = None) -> None: """Cleanup expired cache files.""" if clear_all: @@ -209,10 +356,10 @@ def energy_management_on_exception(e: Exception) -> None: wait_first=config_eos.ems.startup_delay, on_exception=energy_management_on_exception, ) -def energy_management_task() -> None: +async def energy_management_task() -> None: """Repeating task for energy management.""" logger.debug("Check EMS run") - ems_eos.manage_energy() + await ems_eos.manage_energy() async def server_shutdown_task() -> None: @@ -228,20 +375,13 @@ async def server_shutdown_task() -> None: Finally, logs a message indicating that the EOS server has been terminated. """ - # Assure cache is saved - cache_save() + save_eos_state() # Give EOS time to finish some work await asyncio.sleep(5) - # Gracefully shut down this process. - pid = psutil.Process().pid - if os.name == "nt": - os.kill(pid, signal.CTRL_C_EVENT) # type: ignore[attr-defined,unused-ignore] - else: - os.kill(pid, signal.SIGTERM) # type: ignore[attr-defined,unused-ignore] + terminate_eos() - logger.info(f"🚀 EOS terminated, PID {pid}") sys.exit(0) @@ -251,51 +391,43 @@ async def lifespan(app: FastAPI) -> AsyncGenerator[None, None]: # On startup if config_eos.server.startup_eosdash: try: - if args is None: - # No command line arguments - host = config_eos.server.eosdash_host - port = config_eos.server.eosdash_port - eos_host = config_eos.server.host - eos_port = config_eos.server.port - log_level = "info" - access_log = False - reload = False - else: - host = args.host - port = ( - config_eos.server.eosdash_port - if config_eos.server.eosdash_port - else (args.port + 1) + if ( + config_eos.server.eosdash_host is None + or config_eos.server.eosdash_port is None + or config_eos.server.host is None + or config_eos.server.port is None + ): + raise ValueError( + f"Invalid configuration for EOSdash server startup.\n" + f"- server/startup_eosdash: {config_eos.server.startup_eosdash}\n" + f"- server/eosdash_host: {config_eos.server.eosdash_host}\n" + f"- server/eosdash_port: {config_eos.server.eosdash_port}\n" + f"- server/host: {config_eos.server.host}\n" + f"- server/port: {config_eos.server.port}" ) - eos_host = args.host - eos_port = args.port - log_level = args.log_level - access_log = args.access_log - reload = args.reload - eos_host = eos_host if eos_host else get_default_host() - eos_port = eos_port if eos_port else 8503 - host = host if host else eos_host - port = port if port else 8504 - - eos_dir = str(config_eos.general.data_folder_path) - eos_config_dir = str(config_eos.general.config_folder_path) + log_level = ( + config_eos.logging.console_level if config_eos.logging.console_level else "info" + ) eosdash_process = start_eosdash( - host=host, - port=port, - eos_host=eos_host, - eos_port=eos_port, + host=str(config_eos.server.eosdash_host), + port=config_eos.server.eosdash_port, + eos_host=str(config_eos.server.host), + eos_port=config_eos.server.port, log_level=log_level, - access_log=access_log, - reload=reload, - eos_dir=eos_dir, - eos_config_dir=eos_config_dir, + access_log=True, + reload=False, + eos_dir=str(config_eos.general.data_folder_path), + eos_config_dir=str(config_eos.general.config_folder_path), ) except Exception as e: logger.error(f"Failed to start EOSdash server. Error: {e}") sys.exit(1) - cache_load() + + load_eos_state() + + # Start EOS tasks if config_eos.cache.cleanup_interval is None: logger.warning("Cache file cleanup disabled. Set cache.cleanup_interval.") else: @@ -306,14 +438,14 @@ async def lifespan(app: FastAPI) -> AsyncGenerator[None, None]: yield # On shutdown - cache_save() + save_eos_state() app = FastAPI( title="Akkudoktor-EOS", description="This project provides a comprehensive solution for simulating and optimizing an energy system based on renewable energy sources. With a focus on photovoltaic (PV) systems, battery storage (batteries), load management (consumer requirements), heat pumps, electric vehicles, and consideration of electricity price data, this system enables forecasting and optimization of energy flow and costs over a specified period.", summary="Comprehensive solution for simulating and optimizing an energy system based on renewable energy sources", - version="0.0.1", + version=f"v{__version__}", license_info={ "name": "Apache 2.0", "url": "https://www.apache.org/licenses/LICENSE-2.0.html", @@ -327,25 +459,39 @@ class PdfResponse(FileResponse): @app.post("/v1/admin/cache/clear", tags=["admin"]) -def fastapi_admin_cache_clear_post(clear_all: Optional[bool] = None) -> dict: - """Clear the cache from expired data. +def fastapi_admin_cache_clear_post() -> dict: + """Clear the cache. - Deletes expired cache files. - - Args: - clear_all (Optional[bool]): Delete all cached files. Default is False. + Deletes all cache files. Returns: data (dict): The management data after cleanup. """ try: - cache_clear(clear_all=clear_all) + cache_clear(clear_all=True) data = CacheFileStore().current_store() except Exception as e: raise HTTPException(status_code=400, detail=f"Error on cache clear: {e}") return data +@app.post("/v1/admin/cache/clear-expired", tags=["admin"]) +def fastapi_admin_cache_clear_expired_post() -> dict: + """Clear the cache from expired data. + + Deletes expired cache files. + + Returns: + data (dict): The management data after cleanup. + """ + try: + cache_clear(clear_all=False) + data = CacheFileStore().current_store() + except Exception as e: + raise HTTPException(status_code=400, detail=f"Error on cache clear expired: {e}") + return data + + @app.post("/v1/admin/cache/save", tags=["admin"]) def fastapi_admin_cache_save_post() -> dict: """Save the current cache management data. @@ -394,23 +540,45 @@ async def fastapi_admin_server_restart_post() -> dict: Restart EOS properly by starting a new instance before exiting the old one. """ - logger.info("🔄 Restarting EOS...") + save_eos_state() # Start a new EOS (Uvicorn) process + logger.info("🔄 Restarting EOS...") + # Force a new process group to make the new process easily distinguishable from the current one # Set environment before any subprocess run, to keep custom config dir env = os.environ.copy() env["EOS_DIR"] = str(config_eos.general.data_folder_path) env["EOS_CONFIG_DIR"] = str(config_eos.general.config_folder_path) - new_process = subprocess.Popen( # noqa: S603 - [ - sys.executable, - ] - + sys.argv, - env=env, - start_new_session=True, - ) + if os.name == "nt": + # Windows + DETACHED_PROCESS = 0x00000008 + # getattr avoids mypy warning on Linux + CREATE_NEW_PROCESS_GROUP = getattr(subprocess, "CREATE_NEW_PROCESS_GROUP", 0x00000200) + + new_process = subprocess.Popen( # noqa: S603 + [ + sys.executable, + ] + + sys.argv, + env=env, + creationflags=DETACHED_PROCESS | CREATE_NEW_PROCESS_GROUP, + close_fds=True, + # stdin, stdout, stderr are inherited by default + ) + else: + # Unix/Linux/macOS + new_process = subprocess.Popen( # noqa: S603 + [ + sys.executable, + ] + + sys.argv, + env=env, + start_new_session=True, + close_fds=True, + # stdin, stdout, stderr are inherited by default + ) logger.info(f"🚀 EOS restarted, PID {new_process.pid}") # Gracefully shut down this process. @@ -445,6 +613,11 @@ def fastapi_health_get(): # type: ignore { "status": "alive", "pid": psutil.Process().pid, + "version": __version__, + "energy-management": { + "start_datetime": to_datetime(ems_eos.start_datetime, as_string=True), + "last_run_datetime": to_datetime(ems_eos.last_run_datetime, as_string=True), + }, } ) @@ -614,68 +787,59 @@ async def fastapi_logging_get_log( return JSONResponse(content={"error": str(e)}, status_code=500) +@app.get("/v1/resource/status", tags=["resource"]) +def fastapi_devices_status_get( + resource_id: Annotated[str, Query(description="Resource ID.")], + actuator_id: Annotated[Optional[str], Query(description="Actuator ID.")] = None, +) -> ResourceStatus: + """Get the latest status of a resource/ device. + + Return: + latest_status: The latest status of a resource/ device. + """ + key = ResourceKey(resource_id=resource_id, actuator_id=actuator_id) + if not resource_registry_eos.status_exists(key): + raise HTTPException(status_code=404, detail=f"Key '{key}' is not available.") + status_latest = resource_registry_eos.status_latest(key) + if status_latest is None: + raise HTTPException(status_code=404, detail=f"Key '{key}' does not have a status.") + return status_latest + + +@app.put("/v1/resource/status", tags=["resource"]) +def fastapi_devices_status_put( + resource_id: Annotated[str, Query(description="Resource ID.")], + status: Annotated[ResourceStatus, Body(description="Resource Status.")], + actuator_id: Annotated[Optional[str], Query(description="Actuator ID.")] = None, +) -> ResourceStatus: + """Update the status of a resource/ device. + + Return: + latest_status: The latest status of a resource/ device. + """ + key = ResourceKey(resource_id=resource_id, actuator_id=actuator_id) + try: + resource_registry_eos.update_status(key, status) + except Exception as e: + raise HTTPException( + status_code=400, + detail=f"Error on resource status update key='{key}', status='{status}': {e}", + ) + status_latest = resource_registry_eos.status_latest(key) + if status_latest is None: + raise HTTPException(status_code=404, detail=f"Key '{key}' does not have a status.") + return status_latest + + @app.get("/v1/measurement/keys", tags=["measurement"]) def fastapi_measurement_keys_get() -> list[str]: """Get a list of available measurement keys.""" return sorted(measurement_eos.record_keys) -@app.get("/v1/measurement/load-mr/series/by-name", tags=["measurement"]) -def fastapi_measurement_load_mr_series_by_name_get( - name: Annotated[str, Query(description="Load name.")], -) -> PydanticDateTimeSeries: - """Get the meter reading of given load name as series.""" - key = measurement_eos.name_to_key(name=name, topic="measurement_load") - if key is None: - raise HTTPException( - status_code=404, detail=f"Measurement load with name '{name}' is not available." - ) - if key not in measurement_eos.record_keys: - raise HTTPException(status_code=404, detail=f"Key '{key}' is not available.") - pdseries = measurement_eos.key_to_series(key=key) - return PydanticDateTimeSeries.from_series(pdseries) - - -@app.put("/v1/measurement/load-mr/value/by-name", tags=["measurement"]) -def fastapi_measurement_load_mr_value_by_name_put( - datetime: Annotated[str, Query(description="Datetime.")], - name: Annotated[str, Query(description="Load name.")], - value: Union[float | str], -) -> PydanticDateTimeSeries: - """Merge the meter reading of given load name and value into EOS measurements at given datetime.""" - key = measurement_eos.name_to_key(name=name, topic="measurement_load") - if key is None: - raise HTTPException( - status_code=404, detail=f"Measurement load with name '{name}' is not available." - ) - if key not in measurement_eos.record_keys: - raise HTTPException(status_code=404, detail=f"Key '{key}' is not available.") - measurement_eos.update_value(datetime, key, value) - pdseries = measurement_eos.key_to_series(key=key) - return PydanticDateTimeSeries.from_series(pdseries) - - -@app.put("/v1/measurement/load-mr/series/by-name", tags=["measurement"]) -def fastapi_measurement_load_mr_series_by_name_put( - name: Annotated[str, Query(description="Load name.")], series: PydanticDateTimeSeries -) -> PydanticDateTimeSeries: - """Merge the meter readings series of given load name into EOS measurements at given datetime.""" - key = measurement_eos.name_to_key(name=name, topic="measurement_load") - if key is None: - raise HTTPException( - status_code=404, detail=f"Measurement load with name '{name}' is not available." - ) - if key not in measurement_eos.record_keys: - raise HTTPException(status_code=404, detail=f"Key '{key}' is not available.") - pdseries = series.to_series() # make pandas series from PydanticDateTimeSeries - measurement_eos.key_from_series(key=key, series=pdseries) - pdseries = measurement_eos.key_to_series(key=key) - return PydanticDateTimeSeries.from_series(pdseries) - - @app.get("/v1/measurement/series", tags=["measurement"]) def fastapi_measurement_series_get( - key: Annotated[str, Query(description="Prediction key.")], + key: Annotated[str, Query(description="Measurement key.")], ) -> PydanticDateTimeSeries: """Get the measurements of given key as series.""" if key not in measurement_eos.record_keys: @@ -687,12 +851,20 @@ def fastapi_measurement_series_get( @app.put("/v1/measurement/value", tags=["measurement"]) def fastapi_measurement_value_put( datetime: Annotated[str, Query(description="Datetime.")], - key: Annotated[str, Query(description="Prediction key.")], + key: Annotated[str, Query(description="Measurement key.")], value: Union[float | str], ) -> PydanticDateTimeSeries: """Merge the measurement of given key and value into EOS measurements at given datetime.""" if key not in measurement_eos.record_keys: raise HTTPException(status_code=404, detail=f"Key '{key}' is not available.") + if isinstance(value, str): + # Try to convert to float + try: + value = float(value) + except: + logger.debug( + f'/v1/measurement/value key: {key} value: "{value}" - string value not convertable to float' + ) measurement_eos.update_value(datetime, key, value) pdseries = measurement_eos.key_to_series(key=key) return PydanticDateTimeSeries.from_series(pdseries) @@ -700,7 +872,7 @@ def fastapi_measurement_value_put( @app.put("/v1/measurement/series", tags=["measurement"]) def fastapi_measurement_series_put( - key: Annotated[str, Query(description="Prediction key.")], series: PydanticDateTimeSeries + key: Annotated[str, Query(description="Measurement key.")], series: PydanticDateTimeSeries ) -> PydanticDateTimeSeries: """Merge measurement given as series into given key.""" if key not in measurement_eos.record_keys: @@ -775,7 +947,7 @@ def fastapi_prediction_series_get( if key not in prediction_eos.record_keys: raise HTTPException(status_code=404, detail=f"Key '{key}' is not available.") if start_datetime is None: - start_datetime = prediction_eos.start_datetime + start_datetime = prediction_eos.ems_start_datetime else: start_datetime = to_datetime(start_datetime) if end_datetime is None: @@ -818,7 +990,7 @@ def fastapi_prediction_dataframe_get( if key not in prediction_eos.record_keys: raise HTTPException(status_code=404, detail=f"Key '{key}' is not available.") if start_datetime is None: - start_datetime = prediction_eos.start_datetime + start_datetime = prediction_eos.ems_start_datetime else: start_datetime = to_datetime(start_datetime) if end_datetime is None: @@ -861,7 +1033,7 @@ def fastapi_prediction_list_get( if key not in prediction_eos.record_keys: raise HTTPException(status_code=404, detail=f"Key '{key}' is not available.") if start_datetime is None: - start_datetime = prediction_eos.start_datetime + start_datetime = prediction_eos.ems_start_datetime else: start_datetime = to_datetime(start_datetime) if end_datetime is None: @@ -912,7 +1084,7 @@ def fastapi_prediction_import_provider( @app.post("/v1/prediction/update", tags=["prediction"]) -def fastapi_prediction_update( +async def fastapi_prediction_update( force_update: Optional[bool] = False, force_enable: Optional[bool] = False ) -> Response: """Update predictions for all providers. @@ -923,19 +1095,25 @@ def fastapi_prediction_update( force_enable: Update data even if provider is disabled. Defaults to False. """ + # Ensure there is only one optimization/ energy management run at a time try: - prediction_eos.update_data(force_update=force_update, force_enable=force_enable) + await ems_eos.run( + mode=EnergyManagementMode.PREDICTION, + force_update=force_update, + force_enable=force_enable, + ) except Exception as e: trace = "".join(traceback.TracebackException.from_exception(e).format()) raise HTTPException( status_code=400, detail=f"Error on prediction update: {e}\n{trace}", ) + return Response() @app.post("/v1/prediction/update/{provider_id}", tags=["prediction"]) -def fastapi_prediction_update_provider( +async def fastapi_prediction_update_provider( provider_id: str, force_update: Optional[bool] = False, force_enable: Optional[bool] = False ) -> Response: """Update predictions for given provider ID. @@ -951,17 +1129,50 @@ def fastapi_prediction_update_provider( provider = prediction_eos.provider_by_id(provider_id) except ValueError: raise HTTPException(status_code=404, detail=f"Provider '{provider_id}' not found.") + + # Ensure there is only one optimization/ energy management run at a time try: - provider.update_data(force_update=force_update, force_enable=force_enable) - except Exception as e: - raise HTTPException( - status_code=400, detail=f"Error on update of provider '{provider_id}': {e}" + await ems_eos.run( + mode=EnergyManagementMode.PREDICTION, + force_update=force_update, + force_enable=force_enable, ) + except Exception as e: + trace = "".join(traceback.TracebackException.from_exception(e).format()) + raise HTTPException( + status_code=400, + detail=f"Error on prediction update: {e}\n{trace}", + ) + return Response() +@app.get("/v1/energy-management/optimization/solution", tags=["energy-management"]) +def fastapi_energy_management_optimization_solution_get() -> OptimizationSolution: + """Get the latest solution of the optimization.""" + solution = ems_eos.optimization_solution() + if solution is None: + raise HTTPException( + status_code=404, + detail="Can not get the optimization solution. Did you configure automatic optimization?", + ) + return solution + + +@app.get("/v1/energy-management/plan", tags=["energy-management"]) +def fastapi_energy_management_plan_get() -> EnergyManagementPlan: + """Get the latest energy management plan.""" + plan = ems_eos.plan() + if plan is None: + raise HTTPException( + status_code=404, + detail="Can not get the energy management plan. Did you configure automatic optimization?", + ) + return plan + + @app.get("/strompreis", tags=["prediction"]) -def fastapi_strompreis() -> list[float]: +async def fastapi_strompreis() -> list[float]: """Deprecated: Electricity Market Price Prediction per Wh (€/Wh). Electricity prices start at 00.00.00 today and are provided for 48 hours. @@ -984,11 +1195,18 @@ def fastapi_strompreis() -> list[float]: ) ) config_eos.merge_settings(settings=settings) - ems_eos.set_start_datetime() # Set energy management start datetime to current hour. - - # Create electricity price forecast - prediction_eos.update_data(force_update=True) + # Ensure there is only one optimization/ energy management run at a time + try: + await ems_eos.run( + mode=EnergyManagementMode.PREDICTION, + force_update=True, + ) + except Exception as e: + raise HTTPException( + status_code=404, + detail=f"Can not update predictions: {e}", + ) # Get the current date and the end date based on prediction hours # Fetch prices for the specified date range start_datetime = to_datetime().start_of("day") @@ -1015,7 +1233,7 @@ class GesamtlastRequest(PydanticBaseModel): @app.post("/gesamtlast", tags=["prediction"]) -def fastapi_gesamtlast(request: GesamtlastRequest) -> list[float]: +async def fastapi_gesamtlast(request: GesamtlastRequest) -> list[float]: """Deprecated: Total Load Prediction with adjustment. Endpoint to handle total load prediction adjusted by latest measured data. @@ -1027,27 +1245,35 @@ def fastapi_gesamtlast(request: GesamtlastRequest) -> list[float]: Note: Use '/v1/prediction/list?key=load_mean_adjusted' instead. Load energy meter readings to be added to EOS measurement by: - '/v1/measurement/load-mr/value/by-name' or - '/v1/measurement/value' + '/v1/measurement/value' or + '/v1/measurement/series' or + '/v1/measurement/dataframe' or + '/v1/measurement/data' """ - settings = SettingsEOS( - prediction=PredictionCommonSettings( - hours=request.hours, - ), - load=LoadCommonSettings( - provider="LoadAkkudoktor", - provider_settings=LoadAkkudoktorCommonSettings( - loadakkudoktor_year_energy=request.year_energy, - ), - ), - ) - config_eos.merge_settings(settings=settings) - ems_eos.set_start_datetime() # Set energy management start datetime to current hour. + settings = { + "prediction": { + "hours": request.hours, + }, + "load": { + "provider": "LoadAkkudoktor", + "provider_settings": { + "LoadAkkudoktor": { + "loadakkudoktor_year_energy": request.year_energy, + }, + }, + }, + "measurement": { + "load_emr_keys": ["gesamtlast_emr"], + }, + } + config_eos.merge_settings_from_dict(settings) # Insert measured data into EOS measurement # Convert from energy per interval to dummy energy meter readings - measurement_key = "load0_mr" - measurement_eos.key_delete_by_datetime(key=measurement_key) # delete all load0_mr measurements + measurement_key = "gesamtlast_emr" + measurement_eos.key_delete_by_datetime( + key=measurement_key + ) # delete all gesamtlast_emr measurements energy = {} try: for data_dict in request.measured_data: @@ -1074,8 +1300,17 @@ def fastapi_gesamtlast(request: GesamtlastRequest) -> list[float]: energy_mr_values.append(energy_mr) measurement_eos.key_from_lists(measurement_key, energy_mr_dates, energy_mr_values) - # Create load forecast - prediction_eos.update_data(force_update=True) + # Ensure there is only one optimization/ energy management run at a time + try: + await ems_eos.run( + mode=EnergyManagementMode.PREDICTION, + force_update=True, + ) + except Exception as e: + raise HTTPException( + status_code=404, + detail=f"Can not update predictions: {e}", + ) # Get the forcast starting at start of day start_datetime = to_datetime().start_of("day") @@ -1096,7 +1331,7 @@ def fastapi_gesamtlast(request: GesamtlastRequest) -> list[float]: @app.get("/gesamtlast_simple", tags=["prediction"]) -def fastapi_gesamtlast_simple(year_energy: float) -> list[float]: +async def fastapi_gesamtlast_simple(year_energy: float) -> list[float]: """Deprecated: Total Load Prediction. Endpoint to handle total load prediction. @@ -1117,16 +1352,26 @@ def fastapi_gesamtlast_simple(year_energy: float) -> list[float]: settings = SettingsEOS( load=LoadCommonSettings( provider="LoadAkkudoktor", - provider_settings=LoadAkkudoktorCommonSettings( - loadakkudoktor_year_energy=year_energy / 1000, # Convert to kWh + provider_settings=LoadCommonProviderSettings( + LoadAkkudoktor=LoadAkkudoktorCommonSettings( + loadakkudoktor_year_energy=year_energy / 1000, # Convert to kWh + ), ), ) ) config_eos.merge_settings(settings=settings) - ems_eos.set_start_datetime() # Set energy management start datetime to current hour. - # Create load forecast - prediction_eos.update_data(force_update=True) + # Ensure there is only one optimization/ energy management run at a time + try: + await ems_eos.run( + mode=EnergyManagementMode.PREDICTION, + force_update=True, + ) + except Exception as e: + raise HTTPException( + status_code=404, + detail=f"Can not update predictions: {e}", + ) # Get the forcast starting at start of day start_datetime = to_datetime().start_of("day") @@ -1152,7 +1397,7 @@ class ForecastResponse(PydanticBaseModel): @app.get("/pvforecast", tags=["prediction"]) -def fastapi_pvforecast() -> ForecastResponse: +async def fastapi_pvforecast() -> ForecastResponse: """Deprecated: PV Forecast Prediction. Endpoint to handle PV forecast prediction. @@ -1171,15 +1416,16 @@ def fastapi_pvforecast() -> ForecastResponse: settings = SettingsEOS(pvforecast=PVForecastCommonSettings(provider="PVForecastAkkudoktor")) config_eos.merge_settings(settings=settings) - ems_eos.set_start_datetime() # Set energy management start datetime to current hour. - - # Create PV forecast + # Ensure there is only one optimization/ energy management run at a time try: - prediction_eos.update_data(force_update=True) - except ValueError as e: + await ems_eos.run( + mode=EnergyManagementMode.PREDICTION, + force_update=True, + ) + except Exception as e: raise HTTPException( status_code=404, - detail=f"Can not get the PV forecast: {e}", + detail=f"Can not update predictions: {e}", ) # Get the forcast starting at start of day @@ -1207,34 +1453,44 @@ def fastapi_pvforecast() -> ForecastResponse: @app.post("/optimize", tags=["optimize"]) -def fastapi_optimize( - parameters: OptimizationParameters, +async def fastapi_optimize( + parameters: GeneticOptimizationParameters, start_hour: Annotated[ Optional[int], Query(description="Defaults to current hour of the day.") ] = None, - ngen: Optional[int] = None, -) -> OptimizeResponse: + ngen: Annotated[ + Optional[int], Query(description="Number of indivuals to generate for genetic algorithm.") + ] = None, +) -> GeneticSolution: + """Deprecated: Optimize. + + Endpoint to handle optimization. + + Note: + Use automatic optimization instead. + """ if start_hour is None: - start_hour = to_datetime().hour - extra_args: dict[str, Any] = dict() - if ngen is not None: - extra_args["ngen"] = ngen + start_datetime = None + else: + start_datetime = to_datetime().set(hour=start_hour) - # TODO: Remove when config and prediction update is done by EMS. - config_eos.update() - prediction_eos.update_data() - - # Perform optimization simulation - opt_class = optimization_problem(verbose=bool(config_eos.server.verbose)) + # Ensure there is only one optimization/ energy management run at a time try: - result = opt_class.optimierung_ems( - parameters=parameters, start_hour=start_hour, **extra_args + await ems_eos.run( + start_datetime=start_datetime, + mode=EnergyManagementMode.OPTIMIZATION, + genetic_parameters=parameters, + genetic_individuals=ngen, ) - # print(result) - return result except Exception as e: raise HTTPException(status_code=400, detail=f"Optimize error: {e}.") + solution = ems_eos.genetic_solution() + if solution is None: + raise HTTPException(status_code=400, detail="Optimize error: no solution stored by run.") + + return solution + @app.get("/visualization_results.pdf", response_class=PdfResponse, tags=["optimize"]) def get_pdf() -> PdfResponse: @@ -1286,9 +1542,9 @@ def redirect(request: Request, path: str) -> Union[HTMLResponse, RedirectRespons port = config_eos.server.eosdash_port if port is None: port = 8504 - # Make hostname Windows friendly - if host == "0.0.0.0" and os.name == "nt": # noqa: S104 - host = "localhost" + if host == "0.0.0.0": # noqa: S104 + # Use IP of EOS host + host = get_host_ip() url = f"http://{host}:{port}/" error_page = create_error_page( status_code="404", @@ -1302,10 +1558,10 @@ Did you want to connect to EOSdash? ) return HTMLResponse(content=error_page, status_code=404) - # Make hostname Windows friendly host = str(config_eos.server.eosdash_host) - if host == "0.0.0.0" and os.name == "nt": # noqa: S104 - host = "localhost" + if host == "0.0.0.0": # noqa: S104 + # Use IP of EOS host + host = get_host_ip() if host and config_eos.server.eosdash_port: # Redirect to EOSdash server url = f"http://{host}:{config_eos.server.eosdash_port}/{path}" @@ -1315,31 +1571,15 @@ Did you want to connect to EOSdash? return RedirectResponse(url="/docs", status_code=303) -def run_eos(host: str, port: int, log_level: str, reload: bool) -> None: +def run_eos() -> None: """Run the EOS server with the specified configurations. Starts the EOS server using the Uvicorn ASGI server. Logs an error and exits if binding to the host and port fails. - Args: - host (str): Hostname to bind the server to. - port (int): Port number to bind the server to. - log_level (str): Log level for the server. One of - "critical", "error", "warning", "info", "debug", or "trace". - reload (bool): Enable or disable auto-reload. True for development. - Returns: None """ - # Make hostname Windows friendly - if host == "0.0.0.0" and os.name == "nt": # noqa: S104 - host = "localhost" - - # Setup console logging level using nested value - # - triggers logging configuration by track_logging_config - if log_level.upper() in LOGGING_LEVELS: - config_eos.set_nested_value("logging/console_level", log_level.upper()) - # Wait for EOS port to be free - e.g. in case of restart wait_for_port_free(port, timeout=120, waiting_app_name="EOS") @@ -1347,10 +1587,10 @@ def run_eos(host: str, port: int, log_level: str, reload: bool) -> None: # Let uvicorn run the fastAPI app uvicorn.run( "akkudoktoreos.server.eos:app", - host=host, - port=port, - log_level="info", - access_log=True, + host=str(config_eos.server.host), + port=config_eos.server.port, + log_level="info", # Fix log level for uvicorn to info + access_log=True, # Fix server access logging to True reload=reload, ) except Exception as e: @@ -1370,47 +1610,14 @@ def main() -> None: --host (str): Host for the EOS server (default: value from config). --port (int): Port for the EOS server (default: value from config). --log_level (str): Log level for the server console. Options: "critical", "error", "warning", "info", "debug", "trace" (default: "info"). - --access_log (bool): Enable or disable access log. Options: True or False (default: False). --reload (bool): Enable or disable auto-reload. Useful for development. Options: True or False (default: False). """ - parser = argparse.ArgumentParser(description="Start EOS server.") - - # Host and port arguments with defaults from config_eos - parser.add_argument( - "--host", - type=str, - default=str(config_eos.server.host), - help="Host for the EOS server (default: value from config)", - ) - parser.add_argument( - "--port", - type=int, - default=config_eos.server.port, - help="Port for the EOS server (default: value from config)", - ) - - # Optional arguments for log_level, access_log, and reload - parser.add_argument( - "--log_level", - type=str, - default="none", - help='Log level for the server console. Options: "critical", "error", "warning", "info", "debug", "trace" (default: "none")', - ) - parser.add_argument( - "--reload", - type=bool, - default=False, - help="Enable or disable auto-reload. Useful for development. Options: True or False (default: False)", - ) - - args = parser.parse_args() - - host = args.host if args.host else get_default_host() - port = args.port if args.port else 8503 - try: - run_eos(host, port, args.log_level, args.reload) - except: + run_eos() + except Exception as ex: + error_msg = f"Failed to run EOS: {ex}" + logger.error(error_msg) + traceback.print_exc() sys.exit(1) diff --git a/src/akkudoktoreos/server/eosdash.py b/src/akkudoktoreos/server/eosdash.py index 82a8663..cba8f8d 100644 --- a/src/akkudoktoreos/server/eosdash.py +++ b/src/akkudoktoreos/server/eosdash.py @@ -3,7 +3,6 @@ import os import sys import traceback from pathlib import Path -from typing import Optional import psutil import uvicorn @@ -12,33 +11,171 @@ from loguru import logger from monsterui.core import FastHTML, Theme from akkudoktoreos.config.config import get_config +from akkudoktoreos.core.logabc import LOGGING_LEVELS +from akkudoktoreos.core.logging import track_logging_config +from akkudoktoreos.core.version import __version__ +from akkudoktoreos.server.dash.about import About # Pages from akkudoktoreos.server.dash.admin import Admin from akkudoktoreos.server.dash.bokeh import BokehJS from akkudoktoreos.server.dash.components import Page from akkudoktoreos.server.dash.configuration import ConfigKeyUpdate, Configuration -from akkudoktoreos.server.dash.demo import Demo from akkudoktoreos.server.dash.footer import Footer -from akkudoktoreos.server.dash.hello import Hello +from akkudoktoreos.server.dash.plan import Plan +from akkudoktoreos.server.dash.prediction import Prediction from akkudoktoreos.server.server import get_default_host, wait_for_port_free +from akkudoktoreos.utils.stringutil import str2bool config_eos = get_config() + +# ------------------------------------ +# Logging configuration at import time +# ------------------------------------ + +logger.remove() +track_logging_config(config_eos, "logging", None, None) +config_eos.track_nested_value("/logging", track_logging_config) + + +# ---------------------------- +# Safe argparse at import time +# ---------------------------- + +parser = argparse.ArgumentParser(description="Start EOSdash server.") + +parser.add_argument( + "--host", + type=str, + help="Host for the EOSdash server (default: value from config)", +) +parser.add_argument( + "--port", + type=int, + help="Port for the EOSdash server (default: value from config)", +) +parser.add_argument( + "--eos-host", + type=str, + help="Host of the EOS server (default: value from config)", +) +parser.add_argument( + "--eos-port", + type=int, + help="Port of the EOS server (default: value from config)", +) +parser.add_argument( + "--log_level", + type=str, + default="INFO", + help='Log level for the server. Options: "critical", "error", "warning", "info", "debug", "trace" (default: "INFO")', +) +parser.add_argument( + "--access_log", + type=str2bool, + default=False, + help="Enable or disable access logging. Options: True or False (default: False)", +) +parser.add_argument( + "--reload", + type=str2bool, + default=False, + help="Enable or disable auto-reload. Useful for development. Options: True or False (default: False)", +) + +# Command line arguments +args: argparse.Namespace +args_unknown: list[str] +args, args_unknown = parser.parse_known_args() + + +# ----------------------------- +# Prepare config at import time +# ----------------------------- + +# Set EOS config to actual environment variable & config file content +config_eos.reset_settings() + +# Setup parameters from args, config_eos and default +# Remember parameters in config +config_eosdash = {} + +# Setup EOS logging level - first to have the other logging messages logged +# - log level +if args and args.log_level is not None: + config_eosdash["log_level"] = args.log_level.upper() +else: + config_eosdash["log_level"] = "info" +# Ensure log_level from command line is in config settings +if config_eosdash["log_level"] in LOGGING_LEVELS: + # Setup console logging level using nested value + # - triggers logging configuration by track_logging_config + config_eos.set_nested_value("logging/console_level", config_eosdash["log_level"]) + logger.debug( + f"logging/console_level configuration set by argument to {config_eosdash['log_level']}" + ) + +# Setup EOS server host +if args and args.eos_host: + config_eosdash["eos_host"] = args.eos_host +elif config_eos.server.host: + config_eosdash["eos_host"] = str(config_eos.server.host) +else: + config_eosdash["eos_host"] = get_default_host() + +# Setup EOS server port +if args and args.eos_port: + config_eosdash["eos_port"] = args.eos_port +elif config_eos.server.port: + config_eosdash["eos_port"] = config_eos.server.port +else: + config_eosdash["eos_port"] = 8503 + +# - EOSdash host +if args and args.host: + config_eosdash["eosdash_host"] = args.host +elif config_eos.server.eosdash_host: + config_eosdash["eosdash_host"] = str(config_eos.server.eosdash_host) +else: + config_eosdash["eosdash_host"] = get_default_host() + +# - EOS port +if args and args.port: + config_eosdash["eosdash_port"] = args.port +elif config_eos.server.eosdash_port: + config_eosdash["eosdash_port"] = config_eos.server.eosdash_port +else: + config_eosdash["eosdash_port"] = 8504 + +# - access log +if args and args.access_log: + config_eosdash["access_log"] = args.access_log +else: + config_eosdash["access_log"] = False + +# - reload +if args is None and args.reload is None: + config_eosdash["reload"] = False +else: + config_eosdash["reload"] = args.reload + + +# --------------------- +# Prepare FastHTML app +# --------------------- + # The favicon for EOSdash favicon_filepath = Path(__file__).parent.joinpath("dash/assets/favicon/favicon.ico") if not favicon_filepath.exists(): raise ValueError(f"Does not exist {favicon_filepath}") -# Command line arguments -args: Optional[argparse.Namespace] = None - -# Get frankenui and tailwind headers via CDN using Theme.green.headers() # Add Bokeh headers +# Get frankenui and tailwind headers via CDN using Theme.green.headers() hdrs = ( + *BokehJS, Theme.green.headers(highlightjs=True), - BokehJS, ) # The EOSdash application @@ -52,24 +189,14 @@ app: FastHTML = FastHTML( def eos_server() -> tuple[str, int]: """Retrieves the EOS server host and port configuration. - If `args` is provided, it uses the `eos_host` and `eos_port` from `args`. - Otherwise, it falls back to the values from `config_eos.server`. + Takes values from `config_eos.server` or default. Returns: tuple[str, int]: A tuple containing: - `eos_host` (str): The EOS server hostname or IP. - `eos_port` (int): The EOS server port. """ - if args is None: - eos_host = str(config_eos.server.host) - eos_port = config_eos.server.port - else: - eos_host = args.eos_host - eos_port = args.eos_port - eos_host = eos_host if eos_host else get_default_host() - eos_port = eos_port if eos_port else 8503 - - return eos_host, eos_port + return config_eosdash["eos_host"], config_eosdash["eos_port"] @app.get("/favicon.ico") @@ -88,12 +215,13 @@ def get_eosdash(): # type: ignore return Page( None, { - "EOSdash": "/eosdash/hello", + "Plan": "/eosdash/plan", + "Prediction": "/eosdash/prediction", "Config": "/eosdash/configuration", - "Demo": "/eosdash/demo", "Admin": "/eosdash/admin", + "About": "/eosdash/about", }, - Hello(), + About(), Footer(*eos_server()), "/eosdash/footer", ) @@ -109,14 +237,14 @@ def get_eosdash_footer(): # type: ignore return Footer(*eos_server()) -@app.get("/eosdash/hello") -def get_eosdash_hello(): # type: ignore - """Serves the EOSdash Hello page. +@app.get("/eosdash/about") +def get_eosdash_about(): # type: ignore + """Serves the EOSdash About page. Returns: - Hello: The Hello page component. + About: The About page component. """ - return Hello() + return About() @app.get("/eosdash/admin") @@ -131,6 +259,13 @@ def get_eosdash_admin(): # type: ignore @app.post("/eosdash/admin") def post_eosdash_admin(data: dict): # type: ignore + """Provide control data to the Admin page. + + This endpoint is called from within the Admin page on user actions. + + Returns: + Admin: The Admin page component. + """ return Admin(*eos_server(), data) @@ -149,14 +284,36 @@ def put_eosdash_configuration(data: dict): # type: ignore return ConfigKeyUpdate(*eos_server(), data["key"], data["value"]) -@app.get("/eosdash/demo") -def get_eosdash_demo(): # type: ignore - """Serves the EOSdash Demo page. +@app.get("/eosdash/plan") +def get_eosdash_plan(data: dict): # type: ignore + """Serves the EOSdash Plan page. Returns: - Demo: The Demo page component. + Plan: The Plan page component. """ - return Demo(*eos_server()) + return Plan(*eos_server(), data) + + +@app.post("/eosdash/plan") +def post_eosdash_plan(data: dict): # type: ignore + """Provide control data to the Plan page. + + This endpoint is called from within the Plan page on user actions. + + Returns: + Plan: The Plan page component. + """ + return Plan(*eos_server(), data) + + +@app.get("/eosdash/prediction") +def get_eosdash_prediction(data: dict): # type: ignore + """Serves the EOSdash Prediction page. + + Returns: + Prediction: The Prediction page component. + """ + return Prediction(*eos_server(), data) @app.get("/eosdash/health") @@ -166,6 +323,7 @@ def get_eosdash_health(): # type: ignore { "status": "alive", "pid": psutil.Process().pid, + "version": __version__, } ) @@ -190,74 +348,22 @@ def run_eosdash() -> None: Returns: None """ - # Setup parameters from args, config_eos and default - # Remember parameters that are also in config - # - EOS host - if args and args.eos_host: - eos_host = args.eos_host - elif config_eos.server.host: - eos_host = config_eos.server.host - else: - eos_host = get_default_host() - config_eos.server.host = eos_host - # - EOS port - if args and args.eos_port: - eos_port = args.eos_port - elif config_eos.server.port: - eos_port = config_eos.server.port - else: - eos_port = 8503 - config_eos.server.port = eos_port - # - EOSdash host - if args and args.host: - eosdash_host = args.host - elif config_eos.server.eosdash.host: - eosdash_host = config_eos.server.eosdash_host - else: - eosdash_host = get_default_host() - config_eos.server.eosdash_host = eosdash_host - # - EOS port - if args and args.port: - eosdash_port = args.port - elif config_eos.server.eosdash_port: - eosdash_port = config_eos.server.eosdash_port - else: - eosdash_port = 8504 - config_eos.server.eosdash_port = eosdash_port - # - log level - if args and args.log_level: - log_level = args.log_level - else: - log_level = "info" - # - access log - if args and args.access_log: - access_log = args.access_log - else: - access_log = False - # - reload - if args and args.reload: - reload = args.reload - else: - reload = False - - # Make hostname Windows friendly - if eosdash_host == "0.0.0.0" and os.name == "nt": # noqa: S104 - eosdash_host = "localhost" - # Wait for EOSdash port to be free - e.g. in case of restart - wait_for_port_free(eosdash_port, timeout=120, waiting_app_name="EOSdash") + wait_for_port_free(config_eosdash["eosdash_port"], timeout=120, waiting_app_name="EOSdash") try: uvicorn.run( "akkudoktoreos.server.eosdash:app", - host=eosdash_host, - port=eosdash_port, - log_level=log_level.lower(), - access_log=access_log, - reload=reload, + host=config_eosdash["eosdash_host"], + port=config_eosdash["eosdash_port"], + log_level=config_eosdash["log_level"].lower(), + access_log=config_eosdash["access_log"], + reload=config_eosdash["reload"], ) except Exception as e: - logger.error(f"Could not bind to host {eosdash_host}:{eosdash_port}. Error: {e}") + logger.error( + f"Could not bind to host {config_eosdash['eosdash_host']}:{config_eosdash['eosdash_port']}. Error: {e}" + ) raise e @@ -278,54 +384,6 @@ def main() -> None: --access_log (bool): Enable or disable access log. Options: True or False (default: False). --reload (bool): Enable or disable auto-reload. Useful for development. Options: True or False (default: False). """ - parser = argparse.ArgumentParser(description="Start EOSdash server.") - - parser.add_argument( - "--host", - type=str, - default=str(config_eos.server.eosdash_host), - help="Host for the EOSdash server (default: value from config)", - ) - parser.add_argument( - "--port", - type=int, - default=config_eos.server.eosdash_port, - help="Port for the EOSdash server (default: value from config)", - ) - parser.add_argument( - "--eos-host", - type=str, - default=str(config_eos.server.host), - help="Host of the EOS server (default: value from config)", - ) - parser.add_argument( - "--eos-port", - type=int, - default=config_eos.server.port, - help="Port of the EOS server (default: value from config)", - ) - parser.add_argument( - "--log_level", - type=str, - default="info", - help='Log level for the server. Options: "critical", "error", "warning", "info", "debug", "trace" (default: "info")', - ) - parser.add_argument( - "--access_log", - type=bool, - default=False, - help="Enable or disable access log. Options: True or False (default: False)", - ) - parser.add_argument( - "--reload", - type=bool, - default=False, - help="Enable or disable auto-reload. Useful for development. Options: True or False (default: False)", - ) - - global args - args = parser.parse_args() - try: run_eosdash() except Exception as ex: diff --git a/src/akkudoktoreos/server/server.py b/src/akkudoktoreos/server/server.py index ba695ac..becd245 100644 --- a/src/akkudoktoreos/server/server.py +++ b/src/akkudoktoreos/server/server.py @@ -2,12 +2,13 @@ import ipaddress import re +import socket import time -from typing import Optional, Union +from typing import Optional import psutil from loguru import logger -from pydantic import Field, IPvAnyAddress, field_validator +from pydantic import Field, field_validator from akkudoktoreos.config.configabc import SettingsBaseModel @@ -17,7 +18,29 @@ def get_default_host() -> str: return "127.0.0.1" -def is_valid_ip_or_hostname(value: str) -> bool: +def get_host_ip() -> str: + """IP address of the host machine. + + This function determines the IP address used to communicate with the outside world + (e.g., for internet access), without sending any actual data. It does so by + opening a UDP socket connection to a public IP address (Google DNS). + + Returns: + str: The local IP address as a string. Returns '127.0.0.1' if unable to determine. + + Example: + >>> get_host_ip() + '192.168.1.42' + """ + try: + with socket.socket(socket.AF_INET, socket.SOCK_DGRAM) as s: + s.connect(("8.8.8.8", 80)) + return s.getsockname()[0] + except Exception: + return "127.0.0.1" + + +def validate_ip_or_hostname(value: str) -> str: """Validate whether a string is a valid IP address (IPv4 or IPv6) or hostname. This function first attempts to interpret the input as an IP address using the @@ -29,23 +52,30 @@ def is_valid_ip_or_hostname(value: str) -> bool: value (str): The input string to validate. Returns: - bool: True if the input is a valid IP address or hostname, False otherwise. + IP address: Valid IP address or hostname. """ try: ipaddress.ip_address(value) - return True + return value except ValueError: pass if len(value) > 253: - return False + raise ValueError(f"Not a valid hostname: {value}") hostname_regex = re.compile( r"^(?=.{1,253}$)(?!-)[A-Z\d-]{1,63}(? bool: @@ -121,28 +151,39 @@ def wait_for_port_free(port: int, timeout: int = 0, waiting_app_name: str = "App class ServerCommonSettings(SettingsBaseModel): """Server Configuration.""" - host: Optional[IPvAnyAddress] = Field( - default=get_default_host(), description="EOS server IP address." + host: Optional[str] = Field( + default=get_default_host(), + description="EOS server IP address. Defaults to 127.0.0.1.", + examples=["127.0.0.1", "localhost"], + ) + port: Optional[int] = Field( + default=8503, + description="EOS server IP port number. Defaults to 8503.", + examples=[ + 8503, + ], ) - port: Optional[int] = Field(default=8503, description="EOS server IP port number.") verbose: Optional[bool] = Field(default=False, description="Enable debug output") startup_eosdash: Optional[bool] = Field( - default=True, description="EOS server to start EOSdash server." + default=True, description="EOS server to start EOSdash server. Defaults to True." ) - eosdash_host: Optional[IPvAnyAddress] = Field( - default=get_default_host(), description="EOSdash server IP address." + eosdash_host: Optional[str] = Field( + default=None, + description="EOSdash server IP address. Defaults to EOS server IP address.", + examples=["127.0.0.1", "localhost"], + ) + eosdash_port: Optional[int] = Field( + default=None, + description="EOSdash server IP port number. Defaults to EOS server IP port number + 1.", + examples=[ + 8504, + ], ) - eosdash_port: Optional[int] = Field(default=8504, description="EOSdash server IP port number.") @field_validator("host", "eosdash_host", mode="before") - def validate_server_host( - cls, value: Optional[Union[str, IPvAnyAddress]] - ) -> Optional[Union[str, IPvAnyAddress]]: + def validate_server_host(cls, value: Optional[str]) -> Optional[str]: if isinstance(value, str): - if not is_valid_ip_or_hostname(value): - raise ValueError(f"Invalid host: {value}") - if value.lower() in ("localhost", "loopback"): - value = "127.0.0.1" + value = validate_ip_or_hostname(value) return value @field_validator("port", "eosdash_port") diff --git a/src/akkudoktoreos/utils/datetimeutil.py b/src/akkudoktoreos/utils/datetimeutil.py index 3f853a1..fc219b7 100644 --- a/src/akkudoktoreos/utils/datetimeutil.py +++ b/src/akkudoktoreos/utils/datetimeutil.py @@ -1,40 +1,1441 @@ -"""Utility functions for date-time conversion tasks. +"""Utility module for date, time, and timezone handling. + +This module provides a unified interface for working with dates, times, durations, and timezones. +It leverages the `pendulum` library to simplify conversions between string representations, +native `datetime`/`date`/`timedelta` types, Unix timestamps, and timezone-aware types. + +Features: +--------- +- Parse and normalize various date or timestamp formats into a `pendulum.DateTime`. +- Convert durations from strings or numerics into `pendulum.Duration`. +- Infer timezone from UTC offset or geolocation. +- Support for custom output formats (ISO 8601, UTC normalized, or user-specified formats). +- Makes pendulum types usable in pydantic models using `pydantic_extra_types.pendulum_dt` + and the `Time` class. + +Types: +------ +- `Time`: Pendulum's time type with timezone awareness. +- `DateTime`: Pendulum's timezone-aware datetime type. +- `Date`: Pendulum's date type. +- `Duration`: Pendulum's representation of a time delta. +- `TimeWindow`: Daily or specific date time window with optional localization support. Functions: ---------- -- to_datetime: Converts various date or time inputs to a timezone-aware or naive `datetime` - object or formatted string. -- to_duration: Converts various time delta inputs to a `timedelta`object. -- to_timezone: Converts utc offset or location latitude and longitude to a `timezone` object. +- `to_time`: Convert diverse time inputs into a `Time` or formatted string. +- `to_datetime`: Convert diverse date/time inputs into a `DateTime` or formatted string. +- `to_duration`: Convert strings or numerics into a `Duration`. +- `to_timezone`: Convert a UTC offset or geographic coordinate into a `Timezone` or its name. -Example usage: --------------- +Usage Examples: +--------------- + >>> to_time("15:30:00", in_timezone="Europe/Berlin") + Time(17, 30, 0, tzinfo=Timezone('Europe/Berlin')) - # Date-time conversion - >>> date_str = "2024-10-15" - >>> date_obj = to_datetime(date_str) - >>> print(date_obj) # Output: datetime object for '2024-10-15' + >>> to_datetime("2024-10-13T15:30:00", in_timezone="Europe/Berlin") + DateTime(2024, 10, 13, 17, 30, 0, tzinfo=Timezone('Europe/Berlin')) - # Time delta conversion >>> to_duration("2 days 5 hours") + Duration(days=2, hours=5) - # Timezone detection - >>> to_timezone(location=(40.7128, -74.0060)) + >>> to_timezone(location=(40.7128, -74.0060), as_string=True) + 'America/New_York' + +See each function's docstring for detailed argument options and examples. """ +import calendar +import datetime import re -from datetime import date, datetime, timedelta -from typing import Any, List, Literal, Optional, Tuple, Union, overload +from typing import Any, Iterator, List, Literal, Optional, Tuple, Union, overload import pendulum +from babel.dates import get_day_names from loguru import logger -from pendulum import Date, DateTime, Duration from pendulum.tz.timezone import Timezone -from timezonefinder import TimezoneFinder +from pydantic import ( + BaseModel, + Field, + GetCoreSchemaHandler, + field_serializer, + field_validator, + model_validator, +) +from pydantic_core import core_schema +from pydantic_extra_types.pendulum_dt import ( # make pendulum types pydantic + Date, + DateTime, + Duration, +) +from tzfpy import get_tz MAX_DURATION_STRING_LENGTH = 350 +class Time(pendulum.Time): + """A timezone-aware Time class derived from pendulum.Time. + + Provides methods to get hour information for specific timezones + with local timezone as default. + Makes Time handled by pydantic. + """ + + def __new__( + cls, + *args: Any, + tzinfo: Optional[Union[datetime.tzinfo, pendulum.Timezone, str]] = None, + **kwargs: Any, + ) -> "Time": + """Create a new Time instance with optional tzinfo parameter. + + Args: + *args: Positional arguments passed to pendulum.Time + tzinfo: Optional timezone information - can be: + - datetime.tzinfo object + - pendulum.Timezone object + - string (timezone name like 'UTC', 'Europe/Berlin') + **kwargs: Keyword arguments passed to pendulum.Time + """ + # Extract tzinfo from args if one of them is a Time-like object + for arg in args: + if isinstance(arg, (pendulum.Time, Time)) and arg.tzinfo: + tzinfo = tzinfo or arg.tzinfo + + # Check if tzinfo is already in kwargs, use it if our tzinfo param is None + existing_tzinfo = kwargs.pop("tzinfo", None) + if tzinfo is None and existing_tzinfo is not None: + tzinfo = existing_tzinfo + + # Create the base instance without tzinfo (pendulum.Time does not understand), + # but with pydantic validation + instance = super().__new__(cls, *args, **kwargs) + + if tzinfo is not None: + # Convert string timezone names to pendulum timezone + if isinstance(tzinfo, str): + tzinfo = pendulum.timezone(tzinfo) + # Convert datetime.tzinfo to pendulum timezone if needed + elif isinstance(tzinfo, datetime.tzinfo) and not isinstance(tzinfo, pendulum.Timezone): + # For standard datetime tzinfo, we need to handle conversion + # This is a simplified approach - you might need more sophisticated handling + tzinfo = pendulum.timezone(str(tzinfo)) + + # Use pendulum.Time.replace() directly to avoid recursive pydantic validation + pend_instance = pendulum.Time(*args, **kwargs) + pend_instance = pend_instance.replace(tzinfo=tzinfo) + # Create Time instance from pendulum Time instance to avoid recursive pydantic validation + instance = cls._create_from_pendulum_time(pend_instance) + + return instance + + @classmethod + def __get_pydantic_core_schema__( + cls, source_type: type, handler: GetCoreSchemaHandler + ) -> core_schema.CoreSchema: + return core_schema.no_info_after_validator_function( + cls._validate, + core_schema.any_schema(), # Accept any input, let _validate handle the logic + serialization=core_schema.plain_serializer_function_ser_schema( + cls._serialize, + return_schema=core_schema.str_schema(), + ), + ) + + @classmethod + def _validate(cls, value: Any) -> "Time": + """Validate input value and convert to Time instance using to_time function.""" + if isinstance(value, cls): + return value + + # Handle None values explicitly + if value is None: + raise ValueError("Time value cannot be None") + + try: + # Use to_time function to convert the value + time_obj = to_time(value, to_naive=False) + + # If to_time returned a string (shouldn't happen) + if isinstance(time_obj, str): + raise ValueError(f"Unexpected string result from to_time: {time_obj}") + + # If it's already our custom Time class, return it + if isinstance(time_obj, cls): + return time_obj + + # Convert pendulum.Time to our custom Time class + if isinstance(time_obj, pendulum.Time): + # Convert to our custom class without triggering validation + return cls._create_from_pendulum_time(time_obj) + + raise ValueError(f"Cannot convert {type(time_obj)} to Time") + + except Exception as e: + raise ValueError(f"Invalid time value: {value}") from e + + @classmethod + def _create_from_pendulum_time(cls, pend_time: pendulum.Time) -> "Time": + """Create a Time instance from a pendulum.Time object. + + This bypasses Pydantic validation and ensures proper internal state. + """ + # Construct a new pendulum.Time instance explicitly + time_obj = pendulum.Time( + pend_time.hour, + pend_time.minute, + pend_time.second, + pend_time.microsecond, + ) + time_obj = time_obj.replace(tzinfo=pend_time.tzinfo) + + # Bypass __init__ and __new__ by directly casting the type + time_obj.__class__ = cls # This is safe since Time inherits from pendulum.Time + + return time_obj + + @classmethod + def _serialize(cls, value: Optional["Time"]) -> str: + """Serialize Time instance to string. + + Returns timezone-aware format if timezone info is present, + otherwise returns naive format. + """ + if value is None: + return "" + + tz = value.tzinfo + if tz is None: + return value.format("HH:mm:ss.SSSSSS") + tz_str = str(tz) + if tz_str in ("UTC", "Etc/UTC"): + return f"{value.format('HH:mm:ss.SSSSSS')} UTC" + if re.match(r"[+-]\d{2}:?\d{2}", tz_str): + return value.format("HH:mm:ss.SSSSSSZZ") + return f"{value.format('HH:mm:ss.SSSSSS')} {tz_str}" + + def __repr__(self) -> str: + """Enhanced repr with more detailed information.""" + tz_info = f", tzinfo={self.tzinfo}" if self.tzinfo else "" + return f"Time({self.hour}, {self.minute}, {self.second}, {self.microsecond}{tz_info})" + + def __str__(self) -> str: + """String representation for user-friendly display.""" + return self._serialize(self) + + def __eq__(self, other: Any) -> bool: + """Enhanced equality comparison that handles timezone conversion.""" + if not isinstance(other, (pendulum.Time, Time)): + return False + + # If both have timezone info, compare in UTC + if self.tzinfo and other.tzinfo: + # Convert both to UTC for comparison + self_utc = self.in_timezone("UTC") + other_utc = other.in_timezone("UTC") + return (self_utc.hour, self_utc.minute, self_utc.second, self_utc.microsecond) == ( + other_utc.hour, + other_utc.minute, + other_utc.second, + other_utc.microsecond, + ) + + # If neither has timezone info, compare directly + if not self.tzinfo and not other.tzinfo: + return super().__eq__(other) + + # Mixed timezone/naive comparison - only equal if times are exactly the same + return super().__eq__(other) + + def __hash__(self) -> int: + """Hash function that considers timezone.""" + if self.tzinfo: + # Hash based on UTC time + utc_time = self.in_timezone("UTC") + return hash( + (utc_time.hour, utc_time.minute, utc_time.second, utc_time.microsecond, "UTC") + ) + return hash((self.hour, self.minute, self.second, self.microsecond, None)) + + def to_local(self) -> "Time": + """Convert to local timezone.""" + if not self.tzinfo: + return self # Already naive, assume local + return self.in_timezone(pendulum.local_timezone()) + + def to_utc(self) -> "Time": + """Convert to UTC timezone.""" + return self.in_timezone("UTC") + + def in_timezone(self, timezone: Union[str, pendulum.Timezone]) -> "Time": + """Convert to specified timezone.""" + if isinstance(timezone, str): + timezone = pendulum.timezone(timezone) + + if self.is_aware(): + # For timezone conversion, we need a reference date + # Use today's date as reference + today = pendulum.today(self.tzinfo) + dt = today.at(self.hour, self.minute, self.second, self.microsecond) + dt = dt.in_timezone(timezone) # Convert to target timezone + t = dt.time() # Extract naiv time component + t = t.replace(tzinfo=timezone) # Add target timezone + time_obj = self._create_from_pendulum_time(t) + else: + # Assume current time is in local timezone + time_obj = self.replace(tzinfo=pendulum.local_timezone()) + + return time_obj + + def is_naive(self) -> bool: + """Check if time is timezone-naive.""" + return self.tzinfo is None + + def is_aware(self) -> bool: + """Check if time is timezone-aware.""" + return self.tzinfo is not None + + def replace_timezone(self, tz: Union[str, pendulum.Timezone, None]) -> "Time": + """Replace timezone without converting the time value.""" + if isinstance(tz, str): + tz = pendulum.timezone(tz) + return self.replace(tzinfo=tz) + + def format_user_friendly( + self, include_seconds: bool = False, include_timezone: Optional[bool] = None + ) -> str: + """Format time in a user-friendly way. + + Args: + include_seconds: Whether to include seconds in the output + include_timezone: Whether to include timezone info (auto-detected if None) + """ + if include_timezone is None: + include_timezone = self.tzinfo is not None + + if include_seconds: + time_format = "HH:mm:ss" + else: + time_format = "HH:mm" + + if include_timezone and self.tzinfo: + time_format += " ZZ" + + return self.format(time_format) + + @classmethod + def now(cls, tz: Union[str, pendulum.Timezone] = None) -> "Time": + """Get current time with optional timezone.""" + if tz: + if isinstance(tz, str): + tz = pendulum.timezone(tz) + now = pendulum.now(tz) + else: + now = pendulum.now() + + return cls(now.hour, now.minute, now.second, now.microsecond, tzinfo=now.tzinfo) + + @classmethod + def parse(cls, time_string: str) -> "Time": + """Parse time string using your enhanced parser.""" + parsed = _parse_time_string(time_string) + return cls( + parsed.hour, parsed.minute, parsed.second, parsed.microsecond, tzinfo=parsed.tzinfo + ) + + +def _parse_time_string(time_str: str, default_date: pendulum.Date = None) -> pendulum.Time: + """Parse various time string formats with comprehensive patterns and timezone support. + + Supports a wide variety of time formats including: + + Basic 24-hour formats: + - "14:30" - Standard HH:MM format + - "14:30:45" - HH:MM:SS format + - "14:30:45.123456" - HH:MM:SS with microseconds + - "1430" - Compact HHMM format + - "143045" - Compact HHMMSS format + - "930" - Short format (9:30) + - "14" - Hour only + - "14.5" - Decimal time (14:30) + - "14h30" - European format with 'h' + - "14-30" - With dash separator + - "14 30" - With space separator + + 12-hour AM/PM formats: + - "2:30 PM" - Standard 12-hour with seconds + - "2:30:45 PM" - 12-hour with seconds + - "2PM" - Short AM/PM format + - "11AM" - Short AM/PM format + + Timezone formats: + - "14:30 UTC" - With UTC timezone + - "14:30 GMT" - With GMT timezone + - "2:30 PM EST" - 12-hour with timezone abbreviation + - "14:30 +05:30" - With offset timezone + - "14:30 -0800" - With compact offset + - "930 PST" - Any format can have timezone + - "14h30 America/New_York" - With full timezone name + + Args: + time_str: The time string to parse + default_date: Default date to use when timezone is present (defaults to today) + + Returns: + pendulum.Time object, optionally with timezone information attached + + Raises: + ValueError: If the time string cannot be parsed or contains invalid time components + """ + time_str = time_str.strip() + original_str = time_str + + # Validate basic format first + if not time_str: + raise ValueError("Empty time string") + + # Extract timezone information first + timezone_info = None + time_part = time_str + + # Pattern for timezone at the end: +HH:MM, -HH:MM, +HHMM, -HHMM, UTC, GMT, EST, PST, etc. + tz_pattern = re.compile( + r"(.+?)\s*([+-]\d{2}:?\d{2}|UTC[+-]?\d{0,2}:?\d{0,2}|GMT[+-]?\d{0,2}:?\d{0,2}|[A-Z]{3,4}|[A-Za-z_]+/[A-Za-z_]+)$", + re.IGNORECASE, + ) + tz_match = tz_pattern.match(time_str) + + if tz_match: + time_part = tz_match.group(1).strip() + timezone_str = tz_match.group(2).strip() + + # Parse timezone + if timezone_str.upper() in ["UTC", "GMT"]: + timezone_info = pendulum.timezone("UTC") + elif timezone_str.upper() in ["EST", "EDT"]: + timezone_info = pendulum.timezone("America/New_York") + elif timezone_str.upper() in ["CST", "CDT"]: + timezone_info = pendulum.timezone("America/Chicago") + elif timezone_str.upper() in ["MST", "MDT"]: + timezone_info = pendulum.timezone("America/Denver") + elif timezone_str.upper() in ["PST", "PDT"]: + timezone_info = pendulum.timezone("America/Los_Angeles") + elif re.match(r"[A-Za-z_]+/[A-Za-z_]+", timezone_str): + # Try to parse as a standard timezone name + try: + timezone_info = pendulum.timezone(timezone_str) + except: + raise ValueError(f"Unknown timezone: {timezone_str}") + elif re.match(r"[+-]\d{2}:?\d{2}", timezone_str): + # Handle offset format like +05:30, -08:00, +0530, -0800 + clean_tz = timezone_str.replace(":", "") + if len(clean_tz) == 5: # +HHMM or -HHMM + sign = clean_tz[0] + hours = int(clean_tz[1:3]) + minutes = int(clean_tz[3:5]) + offset_minutes = hours * 60 + minutes + if sign == "-": + offset_minutes = -offset_minutes + timezone_info = pendulum.tz.timezone.FixedTimezone(offset_minutes * 60) + else: + raise ValueError(f"Unknown timezone: {timezone_str}") + + # Now parse the time part (convert to uppercase for AM/PM matching) + time_part_upper = time_part.upper() + + # Pattern 1: HH:MM:SS.microseconds format + pattern1 = re.compile(r"^(\d{1,2}):(\d{2}):(\d{2})(?:\.(\d{1,6}))?$") + match = pattern1.match(time_part_upper) + if match: + hour, minute, second = int(match.group(1)), int(match.group(2)), int(match.group(3)) + microsecond = int((match.group(4) or "0").ljust(6, "0")[:6]) + if hour > 23 or minute > 59 or second > 59: + raise ValueError(f"Invalid time components: {hour}:{minute}:{second}") + time_obj = pendulum.time(hour, minute, second, microsecond) + + if timezone_info: + return time_obj.replace(tzinfo=timezone_info) + return time_obj + + # Pattern 2: HH:MM format + pattern2 = re.compile(r"^(\d{1,2}):(\d{2})$") + match = pattern2.match(time_part_upper) + if match: + hour, minute = int(match.group(1)), int(match.group(2)) + if hour > 23 or minute > 59: + raise ValueError(f"Invalid time components: {hour}:{minute}") + time_obj = pendulum.time(hour, minute) + + if timezone_info: + return time_obj.replace(tzinfo=timezone_info) + return time_obj + + # Pattern 3: 12-hour format with AM/PM (HH:MM:SS AM/PM or HH:MM AM/PM) + pattern3 = re.compile(r"^(\d{1,2}):(\d{2})(?::(\d{2}))?\s*(AM|PM)$") + match = pattern3.match(time_part_upper) + if match: + hour, minute = int(match.group(1)), int(match.group(2)) + second = int(match.group(3)) if match.group(3) else 0 + am_pm = match.group(4) + + if hour > 12 or hour < 1 or minute > 59 or second > 59: + raise ValueError(f"Invalid 12-hour time: {original_str}") + + # Convert to 24-hour format + if am_pm == "AM": + if hour == 12: + hour = 0 + else: # PM + if hour != 12: + hour += 12 + + time_obj = pendulum.time(hour, minute, second) + + if timezone_info: + return time_obj.replace(tzinfo=timezone_info) + return time_obj + + # Pattern 4: Short AM/PM format (e.g., "2PM", "11AM") + pattern4 = re.compile(r"^(\d{1,2})\s*(AM|PM)$") + match = pattern4.match(time_part_upper) + if match: + hour = int(match.group(1)) + am_pm = match.group(2) + + if hour > 12 or hour < 1: + raise ValueError(f"Invalid 12-hour time: {original_str}") + + # Convert to 24-hour format + if am_pm == "AM": + if hour == 12: + hour = 0 + else: # PM + if hour != 12: + hour += 12 + + time_obj = pendulum.time(hour, 0) + + if timezone_info: + return time_obj.replace(tzinfo=timezone_info) + return time_obj + + # Pattern 5: European format with 'h' (e.g., "14h30", "9h15") + pattern5 = re.compile(r"^(\d{1,2})H(\d{2})$") + match = pattern5.match(time_part_upper) + if match: + hour, minute = int(match.group(1)), int(match.group(2)) + if hour > 23 or minute > 59: + raise ValueError(f"Invalid time components: {hour}:{minute}") + time_obj = pendulum.time(hour, minute) + + if timezone_info: + return time_obj.replace(tzinfo=timezone_info) + return time_obj + + # Pattern 6: Compact format (HHMM, HHMMSS) + if time_part_upper.isdigit(): + if len(time_part_upper) == 4: + hour, minute = int(time_part_upper[:2]), int(time_part_upper[2:]) + if hour > 23 or minute > 59: + raise ValueError(f"Invalid time components: {hour}:{minute}") + time_obj = pendulum.time(hour, minute) + + if timezone_info: + return time_obj.replace(tzinfo=timezone_info) + return time_obj + elif len(time_part_upper) == 6: + hour, minute, second = ( + int(time_part_upper[:2]), + int(time_part_upper[2:4]), + int(time_part_upper[4:6]), + ) + if hour > 23 or minute > 59 or second > 59: + raise ValueError(f"Invalid time components: {hour}:{minute}:{second}") + time_obj = pendulum.time(hour, minute, second) + + if timezone_info: + return time_obj.replace(tzinfo=timezone_info) + return time_obj + elif len(time_part_upper) == 3: + # Handle formats like "930" as 9:30 + hour, minute = int(time_part_upper[0]), int(time_part_upper[1:]) + if hour > 23 or minute > 59: + raise ValueError(f"Invalid time components: {hour}:{minute}") + time_obj = pendulum.time(hour, minute) + + if timezone_info: + return time_obj.replace(tzinfo=timezone_info) + return time_obj + elif len(time_part_upper) == 1 or len(time_part_upper) == 2: + # Handle single/double digit hours + hour = int(time_part_upper) + if hour > 23: + raise ValueError(f"Invalid hour: {hour}") + time_obj = pendulum.time(hour, 0) + + if timezone_info: + return time_obj.replace(tzinfo=timezone_info) + return time_obj + + # Pattern 7: Decimal time (e.g., "14.5" as 14:30) + try: + if "." in time_part and time_part.replace(".", "").isdigit(): + float_val = float(time_part) + if float_val < 0 or float_val >= 24: + raise ValueError(f"Hour must be between 0 and 23.999..., got {float_val}") + hour = int(float_val) + minutes = int((float_val - hour) * 60) + seconds = int(((float_val - hour) * 60 - minutes) * 60) + time_obj = pendulum.time(hour, minutes, seconds) + + if timezone_info: + return time_obj.replace(tzinfo=timezone_info) + return time_obj + except ValueError: + pass + + # Pattern 8: Handle various separators (but be careful with dots) + separators = ["-", " "] # Removed '.' to avoid conflict with decimal times + for sep in separators: + if sep in time_part: + parts = time_part.split(sep) + if len(parts) >= 2 and all(part.isdigit() for part in parts[:3]): + hour = int(parts[0]) + minute = int(parts[1]) + second = int(parts[2]) if len(parts) > 2 else 0 + if hour > 23 or minute > 59 or second > 59: + raise ValueError(f"Invalid time components: {hour}:{minute}:{second}") + time_obj = pendulum.time(hour, minute, second) + + if timezone_info: + return time_obj.replace(tzinfo=timezone_info) + return time_obj + + raise ValueError(f"Unable to parse time string: '{original_str}'") + + +TimeLike = Union[ + str, + int, + float, + Tuple[int, ...], + Time, + datetime.time, + datetime.datetime, + pendulum.Time, + DateTime, +] + + +# Overload 1: Returns Time +@overload +def to_time( + value: TimeLike, + in_timezone: Union[str, pendulum.tz.Timezone, None] = ..., + to_naive: bool = ..., + as_string: Literal[False, None] = ..., +) -> Time: ... + + +# Overload 2: Returns str +@overload +def to_time( + value: TimeLike, + in_timezone: Union[str, pendulum.tz.Timezone, None] = ..., + to_naive: bool = ..., + as_string: Union[str, Literal[True]] = ..., +) -> str: ... + + +# Implementation that satisfies both +def to_time( + value: TimeLike, + in_timezone: Union[str, pendulum.tz.Timezone, None] = None, + to_naive: bool = False, + as_string: Union[str, bool, None] = None, +) -> Union[Time, str]: + """Convert a time-like value into a timezone-aware Time object or formatted string. + + Args: + value: A time representation. Supports: + - Time + - pendulum.Time or pendulum.DateTime + - datetime.time or datetime.datetime + - strings like "14:30", "2:30 PM", "1430", "14:30:00.123", "2PM", "14h30" + - int (e.g. 14 → 14:00) + - float (e.g. 14.5 → 14:30) + - tuple like (14,), (14, 30), (14, 30, 15) + in_timezone: Optional timezone name or object (e.g., "Europe/Berlin"). + Defaults to the local timezone. + to_naive: If True, return a timezone-naive Time object. + as_string: If True, return time as "HH:mm:ss ZZ". + If a format string is provided, it's passed to `pendulum.Time.format()`. + + Returns: + Time or str: A time object or its formatted string. + + Raises: + ValueError: If the input cannot be interpreted as a valid time. + TypeError: If timezone is not a valid type. + """ + # Validate and set timezone + try: + if in_timezone is None: + timezone = pendulum.local_timezone() + if isinstance(timezone, str): + timezone = pendulum.timezone(timezone) + elif isinstance(in_timezone, str): + timezone = pendulum.timezone(in_timezone) + elif isinstance(in_timezone, pendulum.tz.Timezone): + timezone = in_timezone + else: + raise TypeError(f"Invalid timezone type: {type(in_timezone)}") + if not isinstance(timezone, Timezone): + # Should never happen + raise TypeError(f"Invalid timezone conversion to type: {type(timezone)} ({timezone})") + except Exception as e: + raise ValueError(f"Invalid timezone: {in_timezone}") from e + + def finalize(t: pendulum.Time) -> Union[Time, str]: + """Finalize the time object with timezone and formatting.""" + nonlocal timezone, in_timezone + try: + if to_naive: + t = t.replace(tzinfo=None) + # Apply timezone if not naive + elif t.tzinfo: + if in_timezone is not None and t.tzinfo != timezone: + # Convert from original timezone to selected timezone + # For timezone conversion, we need a reference date + # Use today's date as reference + today = pendulum.today(t.tzinfo) + dt = today.at(t.hour, t.minute, t.second, t.microsecond) + dt = dt.in_timezone(timezone) # Convert to target timezone + t = dt.time() # Extract time component (always naive) + t = t.replace(tzinfo=timezone) # Add timezone to naive time + else: + # Just set the timezone + t = t.replace(tzinfo=timezone) + + if as_string: + if isinstance(as_string, str): + return t.format(as_string) + elif t.tzinfo is not None: + return t.format("HH:mm:ss ZZ") + else: + return t.format("HH:mm:ss") + + return Time(t.hour, t.minute, t.second, t.microsecond, tzinfo=t.tzinfo) + except Exception as e: + raise ValueError(f"Failed to finalize time object: {t}") from e + + # Handle different input types + try: + if isinstance(value, Time): + return finalize(value) + + if isinstance(value, pendulum.Time): + return finalize(value) + + # Handle DateTime class if it exists + if hasattr(value, "in_tz") and hasattr(value, "time"): + return finalize(value.in_tz(timezone).time()) + + if isinstance(value, datetime.time): + base = pendulum.time(value.hour, value.minute, value.second, value.microsecond) + return finalize(base) + + if isinstance(value, datetime.datetime): + if value.tzinfo: + # Convert tzinfo to a string (name or offset) + tz_name = value.tzinfo.tzname(value) + # Safely get Pendulum timezone + try: + timezone = pendulum.timezone(tz_name) + except Exception: + # fallback to fixed offset if tz_name is something like 'UTC+02:00' + utc_offset = value.tzinfo.utcoffset(value) + if utc_offset is None: + utc_offset_total_seconds = 0.0 + else: + utc_offset_total_seconds = utc_offset.total_seconds() + timezone = pendulum.FixedTimezone(utc_offset_total_seconds // 60) + pdt = pendulum.instance(value).in_tz(timezone) + return finalize(pdt.time()) + + if isinstance(value, tuple): + if not value: + raise ValueError("Empty tuple provided") + # Pad tuple with zeros if needed + padded = tuple(list(value) + [0] * (4 - len(value)))[:4] + base = pendulum.time(*padded) + return finalize(base) + + if isinstance(value, int): + if value < 0 or value > 23: + raise ValueError(f"Hour must be between 0 and 23, got {value}") + base = pendulum.time(value, 0) + return finalize(base) + + if isinstance(value, float): + if value < 0 or value >= 24: + raise ValueError(f"Hour must be between 0 and 23.999..., got {value}") + hour = int(value) + minutes = int((value - hour) * 60) + seconds = int(((value - hour) * 60 - minutes) * 60) + microseconds = int(((((value - hour) * 60 - minutes) * 60 - seconds) * 1_000_000)) + base = pendulum.time(hour, minutes, seconds, microseconds) + return finalize(base) + + if isinstance(value, str): + # Try our comprehensive string parser first + try: + parsed_time = _parse_time_string(value) + return finalize(parsed_time) + except ValueError as e: + logger.trace(f"Custom parser failed for: {value} - {e}") + + # Fallback to pendulum's parser + try: + dt = pendulum.parse(value, strict=False).in_tz(timezone) + return finalize(dt.time()) + except Exception as e: + logger.trace(f"Pendulum parser failed for '{value}': {e}") + + # Try parsing with ISO time prefix + try: + dt = pendulum.parse(f"T{value}", strict=False).in_tz(timezone) + return finalize(dt.time()) + except Exception as e: + logger.trace(f"ISO time parser failed for 'T{value}': {e}") + + # Try parsing as part of a full datetime + try: + dt = pendulum.parse(f"2000-01-01 {value}", strict=False).in_tz(timezone) + return finalize(dt.time()) + except Exception as e: + logger.trace(f"Full datetime parser failed for '2000-01-01 {value}': {e}") + + # If all parsing attempts fail, raise a more specific error + raise ValueError(f"Unable to parse time string: '{value}'") + + raise ValueError(f"Unsupported type: {type(value)}") + + except ValueError: + raise + except Exception as e: + raise ValueError(f"Invalid time value: {value!r} of type: {type(value)}") from e + + +class TimeWindow(BaseModel): + """Model defining a daily or specific date time window with optional localization support. + + Represents a time interval starting at `start_time` and lasting for `duration`. + Can restrict applicability to a specific day of the week or a specific calendar date. + Supports day names in multiple languages via locale-aware parsing. + """ + + start_time: Time = Field(..., description="Start time of the time window (time of day).") + duration: Duration = Field( + ..., description="Duration of the time window starting from `start_time`." + ) + day_of_week: Optional[Union[int, str]] = Field( + default=None, + description=( + "Optional day of the week restriction. " + "Can be specified as integer (0=Monday to 6=Sunday) or localized weekday name. " + "If None, applies every day unless `date` is set." + ), + ) + date: Optional[Date] = Field( + default=None, + description=( + "Optional specific calendar date for the time window. Overrides `day_of_week` if set." + ), + ) + locale: Optional[str] = Field( + default=None, + description=( + "Locale used to parse weekday names in `day_of_week` when given as string. " + "If not set, Pendulum's default locale is used. " + "Examples: 'en', 'de', 'fr', etc." + ), + ) + + @field_validator("duration", mode="before") + @classmethod + def transform_to_duration(cls, value: Any) -> Duration: + """Converts various duration formats into Duration. + + Args: + value: The value to convert to Duration. + + Returns: + Duration: The converted Duration object. + """ + return to_duration(value) + + @model_validator(mode="after") + def validate_day_of_week_with_locale(self) -> "TimeWindow": + """Validates and normalizes the `day_of_week` field using the specified locale. + + This method supports both integer (0–6) and string inputs for `day_of_week`. + String inputs are matched first against English weekday names (case-insensitive), + and then against localized weekday names using the provided `locale`. + + If a valid match is found, `day_of_week` is converted to its corresponding + integer value (0 for Monday through 6 for Sunday). + + Returns: + TimeWindow: The validated instance with `day_of_week` normalized to an integer. + + Raises: + ValueError: If `day_of_week` is an invalid integer (not in 0–6), + or an unrecognized string (not matching English or localized names), + or of an unsupported type. + """ + if self.day_of_week is None: + return self + + if isinstance(self.day_of_week, int): + if not 0 <= self.day_of_week <= 6: + raise ValueError("day_of_week must be in 0 (Monday) to 6 (Sunday)") + return self + + if isinstance(self.day_of_week, str): + # Try matching against English names first (lowercase) + english_days = {name.lower(): i for i, name in enumerate(calendar.day_name)} + lowercase_value = self.day_of_week.lower() + if lowercase_value in english_days: + self.day_of_week = english_days[lowercase_value] + return self + + # Try localized names + if self.locale: + localized_days = { + get_day_names("wide", locale=self.locale)[i].lower(): i for i in range(7) + } + if lowercase_value in localized_days: + self.day_of_week = localized_days[lowercase_value] + return self + + raise ValueError( + f"Invalid weekday name '{self.day_of_week}' for locale '{self.locale}'. " + f"Expected English names (monday–sunday) or localized names." + ) + + raise ValueError(f"Invalid type for day_of_week: {type(self.day_of_week)}") + + @field_serializer("duration") + def serialize_duration(self, value: Duration) -> str: + """Serialize duration to string.""" + return str(value) + + def _window_start_end(self, reference_date: DateTime) -> tuple[DateTime, DateTime]: + """Get the actual start and end datetimes for the time window on a given date. + + This method computes the concrete start and end datetimes of the configured + time window for a specific date, taking into account timezone information. + + Handles timezone-aware and naive `DateTime` and `Time` objects: + - If both `reference_date` and `start_time` have timezones but differ, + `start_time` is converted to the timezone of `reference_date`. + - If only one has a timezone, the other inherits it. + - If both are naive, UTC is assumed for both. + + Args: + reference_date: The reference date on which to calculate the window. + + Returns: + tuple[DateTime, DateTime]: A tuple containing the start and end datetimes + for the time window, both timezone-aware. + """ + ref_tz = reference_date.timezone + start_tz = self.start_time.tzinfo + + # --- Timezone resolution logic --- + if ref_tz and start_tz: + # Both aware: align start_time to reference_date's tz + if ref_tz != start_tz: + start_time = self.start_time.in_timezone(ref_tz) + else: + start_time = self.start_time + elif ref_tz and not start_tz: + # Only reference_date aware → assume same tz for time + start_time = self.start_time.replace_timezone(ref_tz) + elif not ref_tz and start_tz: + # Only start_time aware → apply its tz to reference_date + reference_date = reference_date.replace(tzinfo=start_tz) + start_time = self.start_time + else: + # Both naive → default to UTC + reference_date = reference_date.replace(tzinfo="UTC") + start_time = self.start_time.replace_timezone("UTC") + + # --- Build window start --- + start = reference_date.replace( + hour=start_time.hour, + minute=start_time.minute, + second=start_time.second, + microsecond=start_time.microsecond, + ) + + # --- Compute window end --- + end = start + self.duration + return start, end + + def contains(self, date_time: DateTime, duration: Optional[Duration] = None) -> bool: + """Check whether a datetime (and optional duration) fits within the time window. + + This method checks if a given datetime `date_time` lies within the start time and duration + defined by the `TimeWindow`. If `duration` is provided, it also ensures that + the full duration starting at `date_time` ends before or at the end of the time window. + + Handles timezone-aware and naive datetimes: + - If both `date_time` and `start_time` are timezone-aware but differ → align `start_time` + to `date_time`’s timezone. + - If only one has a timezone → assign it to the other. + - If both are naive → assume UTC for both. + + If `day_of_week` or `date` are specified in the time window, the method will also + ensure that `date_time` falls on the correct day or matches the exact date. + + Args: + date_time: The datetime to test. + duration: An optional duration that must fit entirely within the time window + starting from `date_time`. + + Returns: + bool: True if the datetime (and optional duration) is fully contained in the + time window, False otherwise. + """ + start_time = self.start_time # work on a local copy to avoid mutating self + start_tz = getattr(start_time, "tzinfo", None) + ref_tz = date_time.timezone + + # --- Handle timezone logic --- + if ref_tz and start_tz: + # Both aware but different → align start_time to date_time's timezone + if ref_tz != start_tz: + start_time = start_time.in_timezone(ref_tz) + elif ref_tz and not start_tz: + # Only date_time aware → assign its timezone to start_time + start_time = start_time.replace_timezone(ref_tz) + elif not ref_tz and start_tz: + # Only start_time aware → assign its timezone to date_time + date_time = date_time.replace(tzinfo=start_tz) + else: + # Both naive → assume UTC + date_time = date_time.replace(tzinfo="UTC") + start_time = start_time.replace_timezone("UTC") + + # --- Date and weekday constraints --- + if self.date and date_time.date() != self.date: + return False + + if self.day_of_week is not None and date_time.day_of_week != self.day_of_week: + return False + + # --- Compute window start and end for this date --- + start, end = self._window_start_end(date_time) + + # --- Check containment --- + if not (start <= date_time < end): + return False + + if duration is not None: + date_time_end = date_time + duration + return date_time_end <= end + + return True + + def earliest_start_time( + self, duration: Duration, reference_date: Optional[DateTime] = None + ) -> Optional[DateTime]: + """Get the earliest datetime that allows a duration to fit within the time window. + + Args: + duration: The duration that needs to fit within the window. + reference_date: The date to check for the time window. Defaults to today. + + Returns: + The earliest start time for the duration, or None if it doesn't fit. + """ + if reference_date is None: + reference_date = pendulum.today() + + # Check if the reference date matches our constraints + if self.date and reference_date.date() != self.date: + return None + + if self.day_of_week is not None and reference_date.day_of_week != self.day_of_week: + return None + + # Check if the duration can fit within the time window + if duration > self.duration: + return None + + window_start, window_end = self._window_start_end(reference_date) + + # The earliest start time is simply the window start time + return window_start + + def latest_start_time( + self, duration: Duration, reference_date: Optional[DateTime] = None + ) -> Optional[DateTime]: + """Get the latest datetime that allows a duration to fit within the time window. + + Args: + duration: The duration that needs to fit within the window. + reference_date: The date to check for the time window. Defaults to today. + + Returns: + The latest start time for the duration, or None if it doesn't fit. + """ + if reference_date is None: + reference_date = pendulum.today() + + # Check if the reference date matches our constraints + if self.date and reference_date.date() != self.date: + return None + + if self.day_of_week is not None and reference_date.day_of_week != self.day_of_week: + return None + + # Check if the duration can fit within the time window + if duration > self.duration: + return None + + window_start, window_end = self._window_start_end(reference_date) + + # The latest start time is the window end minus the duration + latest_start = window_end - duration + + # Ensure the latest start time is not before the window start + if latest_start < window_start: + return None + + return latest_start + + def can_fit_duration( + self, duration: Duration, reference_date: Optional[DateTime] = None + ) -> bool: + """Check if a duration can fit within the time window on a given date. + + Args: + duration: The duration to check. + reference_date: The date to check for the time window. Defaults to today. + + Returns: + bool: True if the duration can fit, False otherwise. + """ + return self.earliest_start_time(duration, reference_date) is not None + + def available_duration(self, reference_date: Optional[DateTime] = None) -> Optional[Duration]: + """Get the total available duration for the time window on a given date. + + Args: + reference_date: The date to check for the time window. Defaults to today. + + Returns: + The available duration, or None if the date doesn't match constraints. + """ + if reference_date is None: + reference_date = pendulum.today() + + if self.date and reference_date.date() != self.date: + return None + + if self.day_of_week is not None and reference_date.day_of_week != self.day_of_week: + return None + + return self.duration + + +class TimeWindowSequence(BaseModel): + """Model representing a sequence of time windows with collective operations. + + Manages multiple TimeWindow objects and provides methods to work with them + as a cohesive unit for scheduling and availability checking. + """ + + windows: Optional[list[TimeWindow]] = Field( + default_factory=list, description="List of TimeWindow objects that make up this sequence." + ) + + @field_validator("windows") + @classmethod + def validate_windows(cls, v: Optional[list[TimeWindow]]) -> list[TimeWindow]: + """Validate windows and convert None to empty list.""" + if v is None: + return [] + return v + + def model_post_init(self, __context: Any) -> None: + """Ensure windows is always a list after initialization.""" + if self.windows is None: + self.windows = [] + + def __iter__(self) -> Iterator[TimeWindow]: + """Allow iteration over the time windows.""" + return iter(self.windows or []) + + def __len__(self) -> int: + """Return the number of time windows in the sequence.""" + return len(self.windows or []) + + def __getitem__(self, index: int) -> TimeWindow: + """Allow indexing into the time windows.""" + if not self.windows: + raise IndexError("list index out of range") + return self.windows[index] + + def contains(self, date_time: DateTime, duration: Optional[Duration] = None) -> bool: + """Check if any time window in the sequence contains the given datetime and duration. + + Args: + date_time: The datetime to test. + duration: An optional duration that must fit entirely within one of the time windows. + + Returns: + bool: True if any time window contains the datetime (and optional duration), False if no windows. + """ + if not self.windows: + return False + return any(window.contains(date_time, duration) for window in self.windows) + + def earliest_start_time( + self, duration: Duration, reference_date: Optional[DateTime] = None + ) -> Optional[DateTime]: + """Get the earliest datetime across all windows that allows a duration to fit. + + Args: + duration: The duration that needs to fit within a window. + reference_date: The date to check for the time windows. Defaults to today. + + Returns: + The earliest start time across all windows, or None if no window can fit the duration. + """ + if not self.windows: + return None + + if reference_date is None: + reference_date = pendulum.today() + + earliest_times = [] + + for window in self.windows: + earliest = window.earliest_start_time(duration, reference_date) + if earliest is not None: + earliest_times.append(earliest) + + return min(earliest_times) if earliest_times else None + + def latest_start_time( + self, duration: Duration, reference_date: Optional[DateTime] = None + ) -> Optional[DateTime]: + """Get the latest datetime across all windows that allows a duration to fit. + + Args: + duration: The duration that needs to fit within a window. + reference_date: The date to check for the time windows. Defaults to today. + + Returns: + The latest start time across all windows, or None if no window can fit the duration. + """ + if not self.windows: + return None + + if reference_date is None: + reference_date = pendulum.today() + + latest_times = [] + + for window in self.windows: + latest = window.latest_start_time(duration, reference_date) + if latest is not None: + latest_times.append(latest) + + return max(latest_times) if latest_times else None + + def can_fit_duration( + self, duration: Duration, reference_date: Optional[DateTime] = None + ) -> bool: + """Check if the duration can fit within any time window in the sequence. + + Args: + duration: The duration to check. + reference_date: The date to check for the time windows. Defaults to today. + + Returns: + bool: True if any window can fit the duration, False if no windows. + """ + if not self.windows: + return False + + return any(window.can_fit_duration(duration, reference_date) for window in self.windows) + + def available_duration(self, reference_date: Optional[DateTime] = None) -> Optional[Duration]: + """Get the total available duration across all applicable windows. + + Args: + reference_date: The date to check for the time windows. Defaults to today. + + Returns: + The sum of available durations from all applicable windows, or None if no windows apply. + """ + if not self.windows: + return None + + if reference_date is None: + reference_date = pendulum.today() + + total_duration = Duration() + has_applicable_windows = False + + for window in self.windows: + window_duration = window.available_duration(reference_date) + if window_duration is not None: + total_duration += window_duration + has_applicable_windows = True + + return total_duration if has_applicable_windows else None + + def get_applicable_windows(self, reference_date: Optional[DateTime] = None) -> list[TimeWindow]: + """Get all windows that apply to the given reference date. + + Args: + reference_date: The date to check for the time windows. Defaults to today. + + Returns: + List of TimeWindow objects that apply to the reference date. + """ + if not self.windows: + return [] + + if reference_date is None: + reference_date = pendulum.today() + + applicable_windows = [] + + for window in self.windows: + if window.available_duration(reference_date) is not None: + applicable_windows.append(window) + + return applicable_windows + + def find_windows_for_duration( + self, duration: Duration, reference_date: Optional[DateTime] = None + ) -> list[TimeWindow]: + """Find all windows that can accommodate the given duration. + + Args: + duration: The duration that needs to fit. + reference_date: The date to check for the time windows. Defaults to today. + + Returns: + List of TimeWindow objects that can fit the duration. + """ + if not self.windows: + return [] + + if reference_date is None: + reference_date = pendulum.today() + + fitting_windows = [] + + for window in self.windows: + if window.can_fit_duration(duration, reference_date): + fitting_windows.append(window) + + return fitting_windows + + def get_all_possible_start_times( + self, duration: Duration, reference_date: Optional[DateTime] = None + ) -> list[tuple[DateTime, DateTime, TimeWindow]]: + """Get all possible start time ranges for a duration across all windows. + + Args: + duration: The duration that needs to fit. + reference_date: The date to check for the time windows. Defaults to today. + + Returns: + List of tuples containing (earliest_start, latest_start, window) for each + window that can accommodate the duration. + """ + if not self.windows: + return [] + + if reference_date is None: + reference_date = pendulum.today() + + possible_times = [] + + for window in self.windows: + earliest = window.earliest_start_time(duration, reference_date) + latest = window.latest_start_time(duration, reference_date) + + if earliest is not None and latest is not None: + possible_times.append((earliest, latest, window)) + + return possible_times + + def add_window(self, window: TimeWindow) -> None: + """Add a new time window to the sequence. + + Args: + window: The TimeWindow to add. + """ + if self.windows is None: + self.windows = [] + self.windows.append(window) + + def remove_window(self, index: int) -> TimeWindow: + """Remove a time window from the sequence by index. + + Args: + index: The index of the window to remove. + + Returns: + The removed TimeWindow. + + Raises: + IndexError: If the index is out of range. + """ + if not self.windows: + raise IndexError("pop from empty list") + return self.windows.pop(index) + + def clear_windows(self) -> None: + """Remove all windows from the sequence.""" + if self.windows is not None: + self.windows.clear() + + def sort_windows_by_start_time(self, reference_date: Optional[DateTime] = None) -> None: + """Sort the windows by their start time on the given reference date. + + Windows that don't apply to the reference date are placed at the end. + + Args: + reference_date: The date to use for sorting. Defaults to today. + """ + if not self.windows: + return + + if reference_date is None: + reference_date = pendulum.today() + + def sort_key(window: TimeWindow) -> tuple[int, DateTime]: + """Sort key: (priority, start_time) where priority 0 = applicable, 1 = not applicable.""" + start_time = window.earliest_start_time(Duration(), reference_date) + if start_time is None: + # Non-applicable windows get a high priority (sorted last) and a dummy time + return (1, reference_date) + return (0, start_time) + + self.windows.sort(key=sort_key) + + @overload def to_datetime( date_input: Optional[Any] = None, @@ -185,11 +1586,14 @@ def to_datetime( raise ValueError(f"Date string {date_input} does not match any known formats.") elif date_input is None: dt = pendulum.now(tz=in_timezone) - elif isinstance(date_input, datetime): + elif isinstance(date_input, datetime.datetime): dt = pendulum.instance(date_input) - elif isinstance(date_input, date): + elif isinstance(date_input, datetime.date): dt = pendulum.instance( - datetime.combine(date_input, datetime.max.time() if to_maxtime else datetime.min.time()) + datetime.datetime.combine( + date_input, + datetime.datetime.max.time() if to_maxtime else datetime.datetime.min.time(), + ) ) elif isinstance(date_input, (int, float)): dt = pendulum.from_timestamp(date_input, tz="UTC") @@ -222,7 +1626,9 @@ def to_datetime( def to_duration( - input_value: Union[Duration, timedelta, str, int, float, Tuple[int, int, int, int], List[int]], + input_value: Union[ + Duration, datetime.timedelta, str, int, float, Tuple[int, int, int, int], List[int] + ], ) -> Duration: """Converts various input types into a Duration object using pendulum. @@ -252,7 +1658,7 @@ def to_duration( if isinstance(input_value, Duration): return input_value - if isinstance(input_value, timedelta): + if isinstance(input_value, datetime.timedelta): return pendulum.duration(seconds=input_value.total_seconds()) if isinstance(input_value, (int, float)): @@ -272,8 +1678,12 @@ def to_duration( elif isinstance(input_value, str): # Use pendulum's parsing for human-readable duration strings try: - duration = pendulum.parse(input_value) - return duration - duration.start_of("day") + parsed = pendulum.parse(input_value) + if isinstance(parsed, pendulum.Duration): + return parsed # Already a duration + else: + # It's a DateTime, calculate duration from start of day + return parsed - parsed.start_of("day") except pendulum.parsing.exceptions.ParserError as e: logger.trace(f"Invalid Pendulum time string format '{input_value}': {e}") @@ -316,9 +1726,6 @@ def to_duration( raise ValueError(error_msg) -timezone_finder = TimezoneFinder() # Static variable for caching - - @overload def to_timezone( utc_offset: Optional[float] = None, @@ -394,7 +1801,7 @@ def to_timezone( lat, lon = location if not (-90 <= lat <= 90 and -180 <= lon <= 180): raise ValueError(f"Invalid latitude/longitude: {lat}, {lon}") - tz_name = timezone_finder.timezone_at(lat=lat, lng=lon) + tz_name = get_tz(lon, lat) if not tz_name: raise ValueError( f"No timezone found for coordinates: latitude {lat}, longitude {lon}" @@ -408,6 +1815,8 @@ def to_timezone( # Fallback to local timezone local_tz = pendulum.local_timezone() + if isinstance(local_tz, str): + local_tz = pendulum.timezone(local_tz) if as_string: return local_tz.name return local_tz diff --git a/src/akkudoktoreos/utils/docs.py b/src/akkudoktoreos/utils/docs.py deleted file mode 100644 index 50af4fd..0000000 --- a/src/akkudoktoreos/utils/docs.py +++ /dev/null @@ -1,44 +0,0 @@ -from typing import Any - -from pydantic.fields import FieldInfo - -from akkudoktoreos.core.pydantic import PydanticBaseModel - - -def get_example_or_default(field_name: str, field_info: FieldInfo, example_ix: int) -> Any: - """Generate a default value for a field, considering constraints.""" - if field_info.examples is not None: - try: - return field_info.examples[example_ix] - except IndexError: - return field_info.examples[-1] - - if field_info.default is not None: - return field_info.default - - raise NotImplementedError(f"No default or example provided '{field_name}': {field_info}") - - -def get_model_structure_from_examples( - model_class: type[PydanticBaseModel], multiple: bool -) -> list[dict[str, Any]]: - """Create a model instance with default or example values, respecting constraints.""" - example_max_length = 1 - - # Get first field with examples (non-default) to get example_max_length - if multiple: - for _, field_info in model_class.model_fields.items(): - if field_info.examples is not None: - example_max_length = len(field_info.examples) - break - - example_data: list[dict[str, Any]] = [{} for _ in range(example_max_length)] - - for field_name, field_info in model_class.model_fields.items(): - if field_info.deprecated: - continue - for example_ix in range(example_max_length): - example_data[example_ix][field_name] = get_example_or_default( - field_name, field_info, example_ix - ) - return example_data diff --git a/src/akkudoktoreos/utils/stringutil.py b/src/akkudoktoreos/utils/stringutil.py new file mode 100644 index 0000000..f01307f --- /dev/null +++ b/src/akkudoktoreos/utils/stringutil.py @@ -0,0 +1,53 @@ +"""Utility module for string-to-boolean conversion.""" + +from typing import Any + + +def str2bool(value: Any) -> bool: + """Convert a string or boolean value to a boolean. + + This function normalizes common textual representations of truthy and + falsy values (case-insensitive). It also accepts an existing boolean + and returns it unchanged. + + Accepted truthy values: + - "yes", "y", "true", "t", "1", "on" + + Accepted falsy values: + - "no", "n", "false", "f", "0", "off" + + Args: + value (Union[str, bool]): The input value to convert. Can be a string + (e.g., "true", "no", "on") or a boolean. + + Returns: + bool: The corresponding boolean value. + + Raises: + ValueError: If the input cannot be interpreted as a boolean. + + Examples: + >>> str2bool("yes") + True + >>> str2bool("OFF") + False + >>> str2bool(True) + True + >>> str2bool("n") + False + >>> str2bool("maybe") + Traceback (most recent call last): + ... + ValueError: Invalid boolean value: maybe + """ + if isinstance(value, bool): + return value + + if isinstance(value, str): + val = value.strip().lower() + if val in ("yes", "y", "true", "t", "1", "on"): + return True + if val in ("no", "n", "false", "f", "0", "off"): + return False + + raise ValueError(f"Invalid boolean value: {value}") diff --git a/src/akkudoktoreos/utils/visualize.py b/src/akkudoktoreos/utils/visualize.py index 25ed877..69929ab 100644 --- a/src/akkudoktoreos/utils/visualize.py +++ b/src/akkudoktoreos/utils/visualize.py @@ -12,9 +12,9 @@ import pendulum from matplotlib.backends.backend_pdf import PdfPages from akkudoktoreos.core.coreabc import ConfigMixin -from akkudoktoreos.core.ems import EnergyManagement -from akkudoktoreos.optimization.genetic import OptimizationParameters -from akkudoktoreos.utils.datetimeutil import to_datetime +from akkudoktoreos.core.ems import get_ems +from akkudoktoreos.optimization.genetic.genetic import GeneticOptimizationParameters +from akkudoktoreos.utils.datetimeutil import DateTime, to_datetime matplotlib.use( "Agg" @@ -137,7 +137,7 @@ class VisualizationReport(ConfigMixin): def create_line_chart_date( self, - start_date: pendulum.DateTime, + start_date: DateTime, y_list: list[Union[np.ndarray, list[Optional[float]], list[float]]], ylabel: str, xlabel: Optional[str] = None, @@ -436,7 +436,7 @@ class VisualizationReport(ConfigMixin): def prepare_visualize( - parameters: OptimizationParameters, + parameters: GeneticOptimizationParameters, results: dict, filename: str = "visualization_results.pdf", start_hour: int = 0, @@ -444,7 +444,7 @@ def prepare_visualize( global debug_visualize report = VisualizationReport(filename) - next_full_hour_date = EnergyManagement.set_start_datetime() + next_full_hour_date = get_ems().start_datetime # Group 1: report.create_line_chart_date( next_full_hour_date, diff --git a/tests/conftest.py b/tests/conftest.py index 332812d..0e85f0c 100644 --- a/tests/conftest.py +++ b/tests/conftest.py @@ -130,25 +130,6 @@ def prediction_eos(): return get_prediction() -@pytest.fixture -def devices_eos(config_mixin): - from akkudoktoreos.devices.devices import get_devices - - devices = get_devices() - print("devices_eos reset!") - devices.reset() - return devices - - -@pytest.fixture -def devices_mixin(devices_eos): - with patch( - "akkudoktoreos.core.coreabc.DevicesMixin.devices", new_callable=PropertyMock - ) as devices_mixin_patch: - devices_mixin_patch.return_value = devices_eos - yield devices_mixin_patch - - # Test if test has side effect of writing to system (user) config file # Before activating, make sure that no user config file exists (e.g. ~/.config/net.akkudoktoreos.eos/EOS.config.json) @pytest.fixture(autouse=True) @@ -273,12 +254,144 @@ def config_default_dirs(tmpdir): ) +# ------------------------------------ +# Provide pytest EOS server management +# ------------------------------------ + + +def cleanup_eos_eosdash( + host: str, + port: int, + eosdash_host: str, + eosdash_port: int, + server_timeout: float = 10.0, +) -> None: + """Clean up any running EOS and EOSdash processes. + + Args: + host (str): EOS server host (e.g., "127.0.0.1"). + port (int): Port number used by the EOS process. + eosdash_hostr (str): EOSdash server host. + eosdash_port (int): Port used by EOSdash. + server_timeout (float): Timeout in seconds before giving up. + """ + server = f"http://{host}:{port}" + eosdash_server = f"http://{eosdash_host}:{eosdash_port}" + + sigkill = signal.SIGTERM if os.name == "nt" else signal.SIGKILL + + # Attempt to shut down EOS via health endpoint + try: + result = requests.get(f"{server}/v1/health", timeout=2) + if result.status_code == HTTPStatus.OK: + pid = result.json()["pid"] + os.kill(pid, sigkill) + time.sleep(1) + result = requests.get(f"{server}/v1/health", timeout=2) + assert result.status_code != HTTPStatus.OK + except Exception: + pass + + # Fallback: kill processes bound to the EOS port + pids: list[int] = [] + for _ in range(int(server_timeout / 3)): + for conn in psutil.net_connections(kind="inet"): + if conn.laddr.port == port and conn.pid is not None: + try: + process = psutil.Process(conn.pid) + cmdline = process.as_dict(attrs=["cmdline"])["cmdline"] + if "akkudoktoreos.server.eos" in " ".join(cmdline): + pids.append(conn.pid) + except Exception: + pass + for pid in pids: + os.kill(pid, sigkill) + running = False + for pid in pids: + try: + proc = psutil.Process(pid) + status = proc.status() + if status != psutil.STATUS_ZOMBIE: + running = True + break + except psutil.NoSuchProcess: + continue + if not running: + break + time.sleep(3) + + # Check for processes still running (maybe zombies). + for pid in pids: + try: + proc = psutil.Process(pid) + status = proc.status() + assert status == psutil.STATUS_ZOMBIE, f"Cleanup EOS expected zombie, got {status} for PID {pid}" + except psutil.NoSuchProcess: + # Process already reaped (possibly by init/systemd) + continue + + # Attempt to shut down EOSdash via health endpoint + for srv in (eosdash_server, "http://127.0.0.1:8504", "http://127.0.0.1:8555"): + try: + result = requests.get(f"{srv}/eosdash/health", timeout=2) + if result.status_code == HTTPStatus.OK: + pid = result.json()["pid"] + os.kill(pid, sigkill) + time.sleep(1) + result = requests.get(f"{srv}/eosdash/health", timeout=2) + assert result.status_code != HTTPStatus.OK + except Exception: + pass + + # Fallback: kill EOSdash processes bound to known ports + pids = [] + for _ in range(int(server_timeout / 3)): + for conn in psutil.net_connections(kind="inet"): + if conn.laddr.port in (eosdash_port, 8504, 8555) and conn.pid is not None: + try: + process = psutil.Process(conn.pid) + cmdline = process.as_dict(attrs=["cmdline"])["cmdline"] + if "akkudoktoreos.server.eosdash" in " ".join(cmdline): + pids.append(conn.pid) + except Exception: + pass + for pid in pids: + os.kill(pid, sigkill) + running = False + for pid in pids: + try: + proc = psutil.Process(pid) + status = proc.status() + if status != psutil.STATUS_ZOMBIE: + running = True + break + except psutil.NoSuchProcess: + continue + if not running: + break + time.sleep(3) + + # Check for processes still running (maybe zombies). + for pid in pids: + try: + proc = psutil.Process(pid) + status = proc.status() + assert status == psutil.STATUS_ZOMBIE, f"Cleanup EOSdash expected zombie, got {status} for PID {pid}" + except psutil.NoSuchProcess: + # Process already reaped (possibly by init/systemd) + continue + + @contextmanager -def server_base(xprocess: XProcess) -> Generator[dict[str, Union[str, int]], None, None]: +def server_base( + xprocess: XProcess, + extra_env: Optional[dict[str, str]] = None +) -> Generator[dict[str, Union[str, int]], None, None]: """Fixture to start the server with temporary EOS_DIR and default config. Args: xprocess (XProcess): The pytest-xprocess fixture to manage the server process. + extra_env (Optional[dict[str, str]]): Environment variables to set before server startup. Yields: dict[str, str]: A dictionary containing: @@ -287,14 +400,22 @@ def server_base(xprocess: XProcess) -> Generator[dict[str, Union[str, int]], Non """ host = get_default_host() port = 8503 - eosdash_port = 8504 + server = f"http://{host}:{port}" # Port of server may be still blocked by a server usage despite the other server already # shut down. CLOSE_WAIT, TIME_WAIT may typically take up to 120 seconds. server_timeout = 120 - server = f"http://{host}:{port}" - eosdash_server = f"http://{host}:{eosdash_port}" + if extra_env and extra_env.get("EOS_SERVER__EOSDASH_HOST", None): + eosdash_host = extra_env["EOS_SERVER__EOSDASH_HOST"] + else: + eosdash_host = host + if extra_env and extra_env.get("EOS_SERVER__EOSDASH_PORT", None): + eosdash_port: int = int(extra_env["EOS_SERVER__EOSDASH_PORT"]) + else: + eosdash_port = 8504 + eosdash_server = f"http://{eosdash_host}:{eosdash_port}" + eos_tmp_dir = tempfile.TemporaryDirectory() eos_dir = str(eos_tmp_dir.name) @@ -324,8 +445,10 @@ def server_base(xprocess: XProcess) -> Generator[dict[str, Union[str, int]], Non env = os.environ.copy() env["EOS_DIR"] = eos_dir env["EOS_CONFIG_DIR"] = eos_dir + if extra_env: + env.update(extra_env) - # command to start server process + # Set command to start server process args = [ sys.executable, "-m", @@ -345,88 +468,17 @@ def server_base(xprocess: XProcess) -> Generator[dict[str, Union[str, int]], Non # checks if our server is ready def startup_check(self): try: - result = requests.get(f"{server}/v1/health", timeout=2) - if result.status_code == 200: + response = requests.get(f"{server}/v1/health", timeout=10) + logger.debug(f"[xprocess] Health check: {response.status_code}") + if response.status_code == 200: return True - except: - pass + logger.debug(f"[xprocess] Health check: {response}") + except Exception as e: + logger.debug(f"[xprocess] Exception during health check: {e}") return False - def cleanup_eos_eosdash(): - # Cleanup any EOS process left. - if os.name == "nt": - # Windows does not provide SIGKILL - sigkill = signal.SIGTERM - else: - sigkill = signal.SIGKILL # type: ignore - # - Use pid on EOS health endpoint - try: - result = requests.get(f"{server}/v1/health", timeout=2) - if result.status_code == HTTPStatus.OK: - pid = result.json()["pid"] - os.kill(pid, sigkill) - time.sleep(1) - result = requests.get(f"{server}/v1/health", timeout=2) - assert result.status_code != HTTPStatus.OK - except: - pass - # - Use pids from processes on EOS port - for retries in range(int(server_timeout / 3)): - pids: list[int] = [] - for conn in psutil.net_connections(kind="inet"): - if conn.laddr.port == port: - if conn.pid not in pids: - # Get fresh process info - try: - process = psutil.Process(conn.pid) - process_info = process.as_dict(attrs=["pid", "cmdline"]) - if "akkudoktoreos.server.eos" in process_info["cmdline"]: - pids.append(conn.pid) - except: - # PID may already be dead - pass - for pid in pids: - os.kill(pid, sigkill) - if len(pids) == 0: - break - time.sleep(3) - assert len(pids) == 0 - # Cleanup any EOSdash processes left. - # - Use pid on EOSdash health endpoint - try: - result = requests.get(f"{eosdash_server}/eosdash/health", timeout=2) - if result.status_code == HTTPStatus.OK: - pid = result.json()["pid"] - os.kill(pid, sigkill) - time.sleep(1) - result = requests.get(f"{eosdash_server}/eosdash/health", timeout=2) - assert result.status_code != HTTPStatus.OK - except: - pass - # - Use pids from processes on EOSdash port - for retries in range(int(server_timeout / 3)): - pids = [] - for conn in psutil.net_connections(kind="inet"): - if conn.laddr.port == eosdash_port: - if conn.pid not in pids: - # Get fresh process info - try: - process = psutil.Process(conn.pid) - process_info = process.as_dict(attrs=["pid", "cmdline"]) - if "akkudoktoreos.server.eosdash" in process_info["cmdline"]: - pids.append(conn.pid) - except: - # PID may already be dead - pass - for pid in pids: - os.kill(pid, sigkill) - if len(pids) == 0: - break - time.sleep(3) - assert len(pids) == 0 - # Kill all running eos and eosdash process - just to be sure - cleanup_eos_eosdash() + cleanup_eos_eosdash(host, port, eosdash_host, eosdash_port, server_timeout) # Ensure there is an empty config file in the temporary EOS directory config_file_path = Path(eos_dir).joinpath(ConfigEOS.CONFIG_FILE_NAME) @@ -440,7 +492,9 @@ def server_base(xprocess: XProcess) -> Generator[dict[str, Union[str, int]], Non yield { "server": server, + "port": port, "eosdash_server": eosdash_server, + "eosdash_port": eosdash_port, "eos_dir": eos_dir, "timeout": server_timeout, } @@ -449,16 +503,21 @@ def server_base(xprocess: XProcess) -> Generator[dict[str, Union[str, int]], Non xprocess.getinfo("eos").terminate() # Cleanup any EOS process left. - cleanup_eos_eosdash() + cleanup_eos_eosdash(host, port, eosdash_host, eosdash_port, server_timeout) # Remove temporary EOS_DIR eos_tmp_dir.cleanup() @pytest.fixture(scope="class") -def server_setup_for_class(xprocess) -> Generator[dict[str, Union[str, int]], None, None]: - """A fixture to start the server for a test class.""" - with server_base(xprocess) as result: +def server_setup_for_class(request, xprocess) -> Generator[dict[str, Union[str, int]], None, None]: + """A fixture to start the server for a test class. + + Get env vars from the test class attribute `eos_env`, if defined + """ + extra_env = getattr(request.cls, "eos_env", None) + + with server_base(xprocess, extra_env=extra_env) as result: yield result @@ -469,66 +528,9 @@ def server_setup_for_function(xprocess) -> Generator[dict[str, Union[str, int]], yield result -@pytest.fixture -def server(xprocess, config_eos, config_default_dirs) -> Generator[str, None, None]: - """Fixture to start the server. - - Provides URL of the server. - """ - # create url/port info to the server - url = "http://127.0.0.1:8503" - - class Starter(ProcessStarter): - # Set environment before any subprocess run, to keep custom config dir - env = os.environ.copy() - env["EOS_DIR"] = str(config_default_dirs[-1]) - project_dir = config_eos.package_root_path.parent.parent - - # assure server to be installed - try: - subprocess.run( - [sys.executable, "-c", "import", "akkudoktoreos.server.eos"], - check=True, - env=env, - stdout=subprocess.PIPE, - stderr=subprocess.PIPE, - cwd=project_dir, - ) - except subprocess.CalledProcessError: - subprocess.run( - [sys.executable, "-m", "pip", "install", "-e", str(project_dir)], - check=True, - stdout=subprocess.PIPE, - stderr=subprocess.PIPE, - ) - - # command to start server process - args = [sys.executable, "-m", "akkudoktoreos.server.eos"] - - # will wait for xx seconds before timing out - timeout = 10 - - # xprocess will now attempt to clean up upon interruptions - terminate_on_interrupt = True - - # checks if our server is ready - def startup_check(self): - try: - result = requests.get(f"{url}/v1/health") - if result.status_code == 200: - return True - except: - pass - return False - - # ensure process is running and return its logfile - pid, logfile = xprocess.ensure("eos", Starter) - print(f"View xprocess logfile at: {logfile}") - - yield url - - # clean up whole process tree afterwards - xprocess.getinfo("eos").terminate() +# ------------------------------ +# Provide pytest timezone change +# ------------------------------ @pytest.fixture diff --git a/tests/test_battery.py b/tests/test_battery.py index e9b87cf..23b489b 100644 --- a/tests/test_battery.py +++ b/tests/test_battery.py @@ -1,7 +1,7 @@ import numpy as np import pytest -from akkudoktoreos.devices.battery import Battery, SolarPanelBatteryParameters +from akkudoktoreos.devices.genetic.battery import Battery, SolarPanelBatteryParameters @pytest.fixture @@ -15,7 +15,10 @@ def setup_pv_battery(): max_charge_power_w=8000, hours=24, ) - battery = Battery(params) + battery = Battery( + params, + prediction_hours=48, + ) battery.reset() return battery @@ -150,7 +153,7 @@ def test_charge_energy_not_allowed_hour(setup_pv_battery): battery = setup_pv_battery # Disable charging for all hours - battery.set_charge_per_hour(np.zeros(battery.hours)) + battery.set_charge_per_hour(np.zeros(battery.prediction_hours)) charged_wh, losses_wh = battery.charge_energy(wh=4000, hour=3) @@ -177,7 +180,9 @@ def test_charge_energy_relative_power(setup_pv_battery): @pytest.fixture def setup_car_battery(): - from akkudoktoreos.devices.battery import ElectricVehicleParameters + from akkudoktoreos.optimization.genetic.geneticparams import ( + ElectricVehicleParameters, + ) params = ElectricVehicleParameters( device_id="ev1", @@ -188,7 +193,10 @@ def setup_car_battery(): max_charge_power_w=7000, hours=24, ) - battery = Battery(params) + battery = Battery( + params, + prediction_hours=48, + ) battery.reset() return battery diff --git a/tests/test_cache.py b/tests/test_cache.py index d2f932d..d97d5c2 100644 --- a/tests/test_cache.py +++ b/tests/test_cache.py @@ -11,12 +11,12 @@ import cachebox import pytest from akkudoktoreos.core.cache import ( + CacheEnergyManagementStore, CacheFileRecord, CacheFileStore, - CacheUntilUpdateStore, + cache_energy_management, cache_in_file, - cache_until_update, - cachemethod_until_update, + cachemethod_energy_management, ) from akkudoktoreos.utils.datetimeutil import compare_datetimes, to_datetime, to_duration @@ -27,103 +27,103 @@ from akkudoktoreos.utils.datetimeutil import compare_datetimes, to_datetime, to_ # Fixtures for testing @pytest.fixture -def cache_until_update_store(): - """Ensures CacheUntilUpdateStore is reset between tests.""" - cache = CacheUntilUpdateStore() - CacheUntilUpdateStore().clear() +def cache_energy_management_store(): + """Ensures CacheEnergyManagementStore is reset between tests.""" + cache = CacheEnergyManagementStore() + CacheEnergyManagementStore().clear() assert len(cache) == 0 return cache -class TestCacheUntilUpdateStore: - def test_cache_initialization(self, cache_until_update_store): - """Test that CacheUntilUpdateStore initializes with the correct properties.""" - cache = CacheUntilUpdateStore() +class TestCacheEnergyManagementStore: + def test_cache_initialization(self, cache_energy_management_store): + """Test that CacheEnergyManagementStore initializes with the correct properties.""" + cache = CacheEnergyManagementStore() assert isinstance(cache.cache, cachebox.LRUCache) assert cache.maxsize == 100 assert len(cache) == 0 - def test_singleton_behavior(self, cache_until_update_store): - """Test that CacheUntilUpdateStore is a singleton.""" - cache1 = CacheUntilUpdateStore() - cache2 = CacheUntilUpdateStore() + def test_singleton_behavior(self, cache_energy_management_store): + """Test that CacheEnergyManagementStore is a singleton.""" + cache1 = CacheEnergyManagementStore() + cache2 = CacheEnergyManagementStore() assert cache1 is cache2 - def test_cache_storage(self, cache_until_update_store): + def test_cache_storage(self, cache_energy_management_store): """Test that items can be added and retrieved from the cache.""" - cache = CacheUntilUpdateStore() + cache = CacheEnergyManagementStore() cache["key1"] = "value1" assert cache["key1"] == "value1" assert len(cache) == 1 - def test_cache_getattr_invalid_method(self, cache_until_update_store): + def test_cache_getattr_invalid_method(self, cache_energy_management_store): """Test that accessing an invalid method raises an AttributeError.""" with pytest.raises(AttributeError): - CacheUntilUpdateStore().non_existent_method() # This should raise AttributeError + CacheEnergyManagementStore().non_existent_method() # This should raise AttributeError class TestCacheUntilUpdateDecorators: - def test_cachemethod_until_update(self, cache_until_update_store): - """Test that cachemethod_until_update caches method results.""" + def test_cachemethod_energy_management(self, cache_energy_management_store): + """Test that cachemethod_energy_management caches method results.""" class MyClass: - @cachemethod_until_update + @cachemethod_energy_management def compute(self, value: int) -> int: return value * 2 obj = MyClass() # Call method and assert caching - assert CacheUntilUpdateStore.miss_count == 0 - assert CacheUntilUpdateStore.hit_count == 0 + assert CacheEnergyManagementStore.miss_count == 0 + assert CacheEnergyManagementStore.hit_count == 0 result1 = obj.compute(5) - assert CacheUntilUpdateStore.miss_count == 1 - assert CacheUntilUpdateStore.hit_count == 0 + assert CacheEnergyManagementStore.miss_count == 1 + assert CacheEnergyManagementStore.hit_count == 0 result2 = obj.compute(5) - assert CacheUntilUpdateStore.miss_count == 1 - assert CacheUntilUpdateStore.hit_count == 1 + assert CacheEnergyManagementStore.miss_count == 1 + assert CacheEnergyManagementStore.hit_count == 1 assert result1 == result2 - def test_cache_until_update(self, cache_until_update_store): - """Test that cache_until_update caches function results.""" + def test_cache_energy_management(self, cache_energy_management_store): + """Test that cache_energy_management caches function results.""" - @cache_until_update + @cache_energy_management def compute(value: int) -> int: return value * 3 # Call function and assert caching result1 = compute(4) - assert CacheUntilUpdateStore.last_event == cachebox.EVENT_MISS + assert CacheEnergyManagementStore.last_event == cachebox.EVENT_MISS result2 = compute(4) - assert CacheUntilUpdateStore.last_event == cachebox.EVENT_HIT + assert CacheEnergyManagementStore.last_event == cachebox.EVENT_HIT assert result1 == result2 - def test_cache_with_different_arguments(self, cache_until_update_store): + def test_cache_with_different_arguments(self, cache_energy_management_store): """Test that caching works for different arguments.""" class MyClass: - @cachemethod_until_update + @cachemethod_energy_management def compute(self, value: int) -> int: return value * 2 obj = MyClass() - assert CacheUntilUpdateStore.miss_count == 0 + assert CacheEnergyManagementStore.miss_count == 0 result1 = obj.compute(3) - assert CacheUntilUpdateStore.last_event == cachebox.EVENT_MISS - assert CacheUntilUpdateStore.miss_count == 1 + assert CacheEnergyManagementStore.last_event == cachebox.EVENT_MISS + assert CacheEnergyManagementStore.miss_count == 1 result2 = obj.compute(5) - assert CacheUntilUpdateStore.last_event == cachebox.EVENT_MISS - assert CacheUntilUpdateStore.miss_count == 2 + assert CacheEnergyManagementStore.last_event == cachebox.EVENT_MISS + assert CacheEnergyManagementStore.miss_count == 2 assert result1 == 6 assert result2 == 10 - def test_cache_clearing(self, cache_until_update_store): + def test_cache_clearing(self, cache_energy_management_store): """Test that cache is cleared between EMS update cycles.""" class MyClass: - @cachemethod_until_update + @cachemethod_energy_management def compute(self, value: int) -> int: return value * 2 @@ -131,26 +131,26 @@ class TestCacheUntilUpdateDecorators: obj.compute(5) # Clear cache - CacheUntilUpdateStore().clear() + CacheEnergyManagementStore().clear() with pytest.raises(KeyError): - _ = CacheUntilUpdateStore()[""] + _ = CacheEnergyManagementStore()[""] - def test_decorator_works_for_standalone_function(self, cache_until_update_store): - """Test that cache_until_update works with standalone functions.""" + def test_decorator_works_for_standalone_function(self, cache_energy_management_store): + """Test that cache_energy_management works with standalone functions.""" - @cache_until_update + @cache_energy_management def add(a: int, b: int) -> int: return a + b - assert CacheUntilUpdateStore.miss_count == 0 - assert CacheUntilUpdateStore.hit_count == 0 + assert CacheEnergyManagementStore.miss_count == 0 + assert CacheEnergyManagementStore.hit_count == 0 result1 = add(1, 2) - assert CacheUntilUpdateStore.miss_count == 1 - assert CacheUntilUpdateStore.hit_count == 0 + assert CacheEnergyManagementStore.miss_count == 1 + assert CacheEnergyManagementStore.hit_count == 0 result2 = add(1, 2) - assert CacheUntilUpdateStore.miss_count == 1 - assert CacheUntilUpdateStore.hit_count == 1 + assert CacheEnergyManagementStore.miss_count == 1 + assert CacheEnergyManagementStore.hit_count == 1 assert result1 == result2 diff --git a/tests/test_class_optimize.py b/tests/test_class_optimize.py deleted file mode 100644 index 462f57d..0000000 --- a/tests/test_class_optimize.py +++ /dev/null @@ -1,104 +0,0 @@ -import json -from pathlib import Path -from typing import Any -from unittest.mock import patch - -import pytest - -from akkudoktoreos.config.config import ConfigEOS -from akkudoktoreos.optimization.genetic import ( - OptimizationParameters, - OptimizeResponse, - optimization_problem, -) -from akkudoktoreos.utils.visualize import ( - prepare_visualize, # Import the new prepare_visualize -) - -DIR_TESTDATA = Path(__file__).parent / "testdata" - - -def compare_dict(actual: dict[str, Any], expected: dict[str, Any]): - assert set(actual) == set(expected) - - for key, value in expected.items(): - if isinstance(value, dict): - assert isinstance(actual[key], dict) - compare_dict(actual[key], value) - elif isinstance(value, list): - assert isinstance(actual[key], list) - assert actual[key] == pytest.approx(value) - else: - assert actual[key] == pytest.approx(value) - - -@pytest.mark.parametrize( - "fn_in, fn_out, ngen", - [ - ("optimize_input_1.json", "optimize_result_1.json", 3), - ("optimize_input_2.json", "optimize_result_2.json", 3), - ("optimize_input_2.json", "optimize_result_2_full.json", 400), - ], -) -def test_optimize( - fn_in: str, - fn_out: str, - ngen: int, - config_eos: ConfigEOS, - is_full_run: bool, -): - """Test optimierung_ems.""" - # Assure configuration holds the correct values - config_eos.merge_settings_from_dict( - {"prediction": {"hours": 48}, "optimization": {"hours": 48}} - ) - - # Load input and output data - file = DIR_TESTDATA / fn_in - with file.open("r") as f_in: - input_data = OptimizationParameters(**json.load(f_in)) - - file = DIR_TESTDATA / fn_out - # In case a new test case is added, we don't want to fail here, so the new output is written to disk before - try: - with file.open("r") as f_out: - expected_result = OptimizeResponse(**json.load(f_out)) - except FileNotFoundError: - pass - - opt_class = optimization_problem(fixed_seed=42) - start_hour = 10 - - # Activate with pytest --full-run - if ngen > 10 and not is_full_run: - pytest.skip() - - visualize_filename = str((DIR_TESTDATA / f"new_{fn_out}").with_suffix(".pdf")) - - with patch( - "akkudoktoreos.utils.visualize.prepare_visualize", - side_effect=lambda parameters, results, *args, **kwargs: prepare_visualize( - parameters, results, filename=visualize_filename, **kwargs - ), - ) as prepare_visualize_patch: - # Call the optimization function - ergebnis = opt_class.optimierung_ems( - parameters=input_data, start_hour=start_hour, ngen=ngen - ) - # Write test output to file, so we can take it as new data on intended change - TESTDATA_FILE = DIR_TESTDATA / f"new_{fn_out}" - with TESTDATA_FILE.open("w", encoding="utf-8", newline="\n") as f_out: - f_out.write(ergebnis.model_dump_json(indent=4, exclude_unset=True)) - - assert ergebnis.result.Gesamtbilanz_Euro == pytest.approx( - expected_result.result.Gesamtbilanz_Euro - ) - - # Assert that the output contains all expected entries. - # This does not assert that the optimization always gives the same result! - # Reproducibility and mathematical accuracy should be tested on the level of individual components. - compare_dict(ergebnis.model_dump(), expected_result.model_dump()) - - # The function creates a visualization result PDF as a side-effect. - prepare_visualize_patch.assert_called_once() - assert Path(visualize_filename).exists() diff --git a/tests/test_config.py b/tests/test_config.py index 24da024..acae32b 100644 --- a/tests/test_config.py +++ b/tests/test_config.py @@ -1,11 +1,11 @@ import tempfile from pathlib import Path -from typing import Union +from typing import Any, Optional, Union from unittest.mock import patch import pytest from loguru import logger -from pydantic import ValidationError +from pydantic import IPvAnyAddress, ValidationError from akkudoktoreos.config.config import ConfigEOS, GeneralSettings @@ -20,11 +20,11 @@ def test_fixture_new_config_file(config_default_dirs): """Assure fixture stash_config_file is working.""" config_default_dir_user, config_default_dir_cwd, _, _ = config_default_dirs - config_file_path_user = config_default_dir_user.joinpath(ConfigEOS.CONFIG_FILE_NAME) - config_file_path_cwd = config_default_dir_cwd.joinpath(ConfigEOS.CONFIG_FILE_NAME) + config_file_user = config_default_dir_user.joinpath(ConfigEOS.CONFIG_FILE_NAME) + config_file_cwd = config_default_dir_cwd.joinpath(ConfigEOS.CONFIG_FILE_NAME) - assert not config_file_path_user.exists() - assert not config_file_path_cwd.exists() + assert not config_file_user.exists() + assert not config_file_cwd.exists() def test_config_constants(config_eos): @@ -62,6 +62,38 @@ def test_computed_paths(config_eos): config_eos.reset_settings() +def test_config_from_env(monkeypatch, config_eos): + """Test configuration from env.""" + assert config_eos.server.port == 8503 + assert config_eos.server.eosdash_port is None + + monkeypatch.setenv("EOS_SERVER__PORT", "8553") + monkeypatch.setenv("EOS_SERVER__EOSDASH_PORT", "8555") + + config_eos.reset_settings() + + assert config_eos.server.port == 8553 + assert config_eos.server.eosdash_port == 8555 + + +def test_config_ipaddress(monkeypatch, config_eos): + """Test configuration for IP adresses.""" + assert config_eos.server.host == "127.0.0.1" + + monkeypatch.setenv("EOS_SERVER__HOST", "0.0.0.0") + config_eos.reset_settings() + assert config_eos.server.host == "0.0.0.0" + + monkeypatch.setenv("EOS_SERVER__HOST", "mail.akkudoktor.net") + config_eos.reset_settings() + assert config_eos.server.host == "mail.akkudoktor.net" + + # keep last + monkeypatch.setenv("EOS_SERVER__HOST", "localhost") + config_eos.reset_settings() + assert config_eos.server.host == "localhost" + + def test_singleton_behavior(config_eos, config_default_dirs): """Test that ConfigEOS behaves as a singleton.""" initial_cfg_file = config_eos.general.config_file_path @@ -85,21 +117,34 @@ def test_default_config_path(config_eos, config_default_dirs): def test_config_file_priority(config_default_dirs): - """Test config file priority.""" - from akkudoktoreos.config.config import get_config + """Test config file priority. + + Priority is: + 1. environment variable directory + 2. user configuration directory + 3. current working directory + """ config_default_dir_user, config_default_dir_cwd, _, _ = config_default_dirs + config_file_cwd = Path(config_default_dir_cwd) / ConfigEOS.CONFIG_FILE_NAME + config_file_user = Path(config_default_dir_user) / ConfigEOS.CONFIG_FILE_NAME - config_file = Path(config_default_dir_cwd) / ConfigEOS.CONFIG_FILE_NAME - config_file.write_text("{}") - config_eos = get_config() - assert config_eos.general.config_file_path == config_file + assert not config_file_cwd.exists() + assert not config_file_user.exists() - config_file = Path(config_default_dir_user) / ConfigEOS.CONFIG_FILE_NAME - config_file.parent.mkdir() - config_file.write_text("{}") + # current working directory (prio 3) + config_file_cwd.write_text("{}") + + config_eos = ConfigEOS() config_eos.update() - assert config_eos.general.config_file_path == config_file + assert config_eos.general.config_file_path == config_file_cwd + + # user configuration directory (prio 2) + config_file_user.parent.mkdir() + config_file_user.write_text("{}") + + config_eos.update() + assert config_eos.general.config_file_path == config_file_user @patch("akkudoktoreos.config.config.user_config_dir") @@ -220,6 +265,7 @@ def test_config_common_settings_timezone_none_when_coordinates_missing(): assert config_no_coords.timezone is None + # Test partial assignments and possible side effects @pytest.mark.parametrize( "path, value, expected, exception", @@ -280,13 +326,20 @@ def test_config_common_settings_timezone_none_when_coordinates_missing(): [("general.latitude", 52.52), ("general.longitude", 13.405)], ValueError, ), + # Correct value assignment - preparation for list + ( + "devices/max_electric_vehicles", + 1, + [("devices.max_electric_vehicles", 1), ], + None, + ), # Correct value for list ( - "optimization/ev_available_charge_rates_percent/0", - 0.1, + "devices/electric_vehicles/0/charge_rates", + [0.1, 0.375, 0.5, 0.625, 0.75, 0.875, 1.0], [ ( - "optimization.ev_available_charge_rates_percent", + "devices.electric_vehicles[0].charge_rates", [0.1, 0.375, 0.5, 0.625, 0.75, 0.875, 1.0], ) ], @@ -294,23 +347,23 @@ def test_config_common_settings_timezone_none_when_coordinates_missing(): ), # Invalid value for list ( - "optimization/ev_available_charge_rates_percent/0", + "devices/electric_vehicles/0/charge_rates", "invalid", [ ( - "optimization.ev_available_charge_rates_percent", + "devices.electric_vehicles[0].charge_rates", [0.0, 0.375, 0.5, 0.625, 0.75, 0.875, 1.0], ) ], - TypeError, + ValueError, ), # Invalid index (out of bound) ( - "optimization/ev_available_charge_rates_percent/10", + "devices/electric_vehicles/0/charge_rates/10", 0, [ ( - "optimization.ev_available_charge_rates_percent", + "devices.electric_vehicles[0].charge_rates", [0.0, 0.375, 0.5, 0.625, 0.75, 0.875, 1.0], ) ], @@ -318,11 +371,11 @@ def test_config_common_settings_timezone_none_when_coordinates_missing(): ), # Invalid index (no number) ( - "optimization/ev_available_charge_rates_percent/test", + "devices/electric_vehicles/0/charge_rates/test", 0, [ ( - "optimization.ev_available_charge_rates_percent", + "devices.electric_vehicles[0].charge_rates", [0.0, 0.375, 0.5, 0.625, 0.75, 0.875, 1.0], ) ], @@ -330,11 +383,11 @@ def test_config_common_settings_timezone_none_when_coordinates_missing(): ), # Unset value (set None) ( - "optimization/ev_available_charge_rates_percent", + "devices/electric_vehicles/0/charge_rates", None, [ ( - "optimization.ev_available_charge_rates_percent", + "devices.electric_vehicles[0].charge_rates", None, ) ], @@ -373,18 +426,6 @@ def test_set_nested_key(path, value, expected, exception, config_eos): ("general/latitude", 52.52, None), ("general/latitude/", 52.52, None), ("general/latitude/test", None, KeyError), - ( - "optimization/ev_available_charge_rates_percent/1", - 0.375, - None, - ), - ("optimization/ev_available_charge_rates_percent/10", 0, IndexError), - ("optimization/ev_available_charge_rates_percent/test", 0, IndexError), - ( - "optimization/ev_available_charge_rates_percent", - [0.0, 0.375, 0.5, 0.625, 0.75, 0.875, 1.0], - None, - ), ], ) def test_get_nested_key(path, expected_value, exception, config_eos): @@ -409,7 +450,8 @@ def test_merge_settings_from_dict_invalid(config_eos): def test_merge_settings_partial(config_eos): """Test merging only a subset of settings.""" - partial_settings: dict[str, dict[str, Union[float, None, str]]] = { + + partial_settings: dict[str, Any] = { "general": { "latitude": 51.1657 # Only latitude is updated }, @@ -419,6 +461,8 @@ def test_merge_settings_partial(config_eos): assert config_eos.general.latitude == 51.1657 assert config_eos.general.longitude == 13.405 # Should remain unchanged + #----------------- + partial_settings = { "weather": { "provider": "BrightSky", @@ -428,6 +472,8 @@ def test_merge_settings_partial(config_eos): config_eos.merge_settings_from_dict(partial_settings) assert config_eos.weather.provider == "BrightSky" + #----------------- + partial_settings = { "general": { "latitude": None, @@ -446,6 +492,36 @@ def test_merge_settings_partial(config_eos): assert config_eos.general.latitude is None assert config_eos.weather.provider == "ClearOutside" + #----------------- + + partial_settings = { + "devices": { + "max_electric_vehicles": 1, + "electric_vehicles": [ + { + "charge_rates": [0.0, 0.375, 0.5, 0.625, 0.75, 0.875, 1.0], + } + ], + } + } + + config_eos.merge_settings_from_dict(partial_settings) + assert config_eos.devices.max_electric_vehicles == 1 + assert len(config_eos.devices.electric_vehicles) == 1 + assert config_eos.devices.electric_vehicles[0].charge_rates == [0.0, 0.375, 0.5, 0.625, 0.75, 0.875, 1.0] + + # Assure re-apply generates the same config + config_eos.merge_settings_from_dict(partial_settings) + assert config_eos.devices.max_electric_vehicles == 1 + assert len(config_eos.devices.electric_vehicles) == 1 + assert config_eos.devices.electric_vehicles[0].charge_rates == [0.0, 0.375, 0.5, 0.625, 0.75, 0.875, 1.0] + + # Assure update keeps same values + config_eos.update() + assert config_eos.devices.max_electric_vehicles == 1 + assert len(config_eos.devices.electric_vehicles) == 1 + assert config_eos.devices.electric_vehicles[0].charge_rates == [0.0, 0.375, 0.5, 0.625, 0.75, 0.875, 1.0] + def test_merge_settings_empty(config_eos): """Test merging an empty dictionary does not change settings.""" diff --git a/tests/test_configmigrate.py b/tests/test_configmigrate.py new file mode 100644 index 0000000..e97def3 --- /dev/null +++ b/tests/test_configmigrate.py @@ -0,0 +1,229 @@ +import json +import shutil +import tempfile +from pathlib import Path +from typing import Any + +import pytest + +from akkudoktoreos.config import configmigrate +from akkudoktoreos.config.config import ConfigEOS, SettingsEOSDefaults +from akkudoktoreos.core.version import __version__ + +# Test data directory and known migration pairs +DIR_TESTDATA = Path(__file__).absolute().parent.joinpath("testdata") + +MIGRATION_PAIRS = [ + ( + DIR_TESTDATA / "eos_config_minimal_0_1_0.json", + DIR_TESTDATA / "eos_config_minimal_now.json", + ), + ( + DIR_TESTDATA / "eos_config_andreas_0_1_0.json", + DIR_TESTDATA / "eos_config_andreas_now.json", + ), + # Add more pairs here: + # (DIR_TESTDATA / "old_config_X.json", DIR_TESTDATA / "expected_config_X.json"), +] + + +def _dict_contains(superset: Any, subset: Any, path="") -> list[str]: + """Recursively verify that all key-value pairs from a subset dictionary or list exist in a superset. + + Supports nested dictionaries and lists. Extra keys in superset are allowed. + Numeric values (int/float) are compared with tolerance. + + Args: + superset (Any): The dictionary or list that should contain all items from `subset`. + subset (Any): The expected dictionary or list. + path (str, optional): Current nested path used for error reporting. Defaults to "". + + Returns: + list[str]: A list of strings describing mismatches or missing keys. Empty list if all subset items are present. + """ + errors = [] + + if isinstance(subset, dict) and isinstance(superset, dict): + for key, sub_value in subset.items(): + full_path = f"{path}/{key}" if path else key + if key not in superset: + errors.append(f"Missing key: {full_path}") + continue + errors.extend(_dict_contains(superset[key], sub_value, full_path)) + + elif isinstance(subset, list) and isinstance(superset, list): + for i, elem in enumerate(subset): + if i >= len(superset): + full_path = f"{path}[{i}]" if path else f"[{i}]" + errors.append(f"List too short at {full_path}: expected element {elem}") + continue + errors.extend(_dict_contains(superset[i], elem, f"{path}[{i}]" if path else f"[{i}]")) + + else: + # Compare values (with numeric tolerance) + if isinstance(subset, (int, float)) and isinstance(superset, (int, float)): + if abs(float(subset) - float(superset)) > 1e-6: + errors.append(f"Value mismatch at {path}: expected {subset}, got {superset}") + elif subset != superset: + errors.append(f"Value mismatch at {path}: expected {subset}, got {superset}") + + return errors + + +class TestConfigMigration: + """Tests for migrate_config_file()""" + + @pytest.fixture + def tmp_config_file(self, config_default_dirs) -> Path: + """Create a temporary valid config file with an invalid version.""" + config_default_dir_user, _, _, _ = config_default_dirs + config_file_user = config_default_dir_user.joinpath(ConfigEOS.CONFIG_FILE_NAME) + + # Create a default config object (simulates the latest schema) + default_config = SettingsEOSDefaults() + + # Dump to JSON + config_json = json.loads(default_config.model_dump_json()) + + # Corrupt the version (simulate outdated config) + config_json["general"]["version"] = "0.0.0-old" + + # Write file + with config_file_user.open("w", encoding="utf-8") as f: + json.dump(config_json, f, indent=4) + + return config_file_user + + + def test_migrate_config_file_from_invalid_version(self, tmp_config_file: Path): + """Test that migration updates an outdated config version successfully.""" + backup_file = tmp_config_file.with_suffix(".bak") + + # Run migration + result = configmigrate.migrate_config_file(tmp_config_file, backup_file) + + # Verify success + assert result is True, "Migration should succeed even from invalid version." + + # Verify backup exists + assert backup_file.exists(), "Backup file should be created before migration." + + # Verify version updated + with tmp_config_file.open("r", encoding="utf-8") as f: + migrated_data = json.load(f) + assert migrated_data["general"]["version"] == __version__, \ + "Migrated config should have updated version." + + # Verify it still matches the structure of SettingsEOSDefaults + new_model = SettingsEOSDefaults(**migrated_data) + assert isinstance(new_model, SettingsEOSDefaults) + + def test_migrate_config_file_already_current(self, tmp_path: Path): + """Test that a current config file returns True immediately.""" + config_path = tmp_path / "EOS_current.json" + default = SettingsEOSDefaults() + with config_path.open("w", encoding="utf-8") as f: + f.write(default.model_dump_json(indent=4)) + + backup_file = config_path.with_suffix(".bak") + + result = configmigrate.migrate_config_file(config_path, backup_file) + assert result is True + assert not backup_file.exists(), "No backup should be made if config is already current." + + + @pytest.mark.parametrize("old_file, expected_file", MIGRATION_PAIRS) + def test_migrate_old_version_config(self, tmp_path: Path, old_file: Path, expected_file: Path): + """Ensure migration from old → new schema produces the expected output.""" + # --- Prepare temporary working file based on expected file name --- + working_file = expected_file.with_suffix(".new") + shutil.copy(old_file, working_file) + + # Backup file path (inside tmp_path to avoid touching repo files) + backup_file = tmp_path / f"{old_file.stem}.bak" + + failed = False + try: + assert working_file.exists(), f"Working config file is missing: {working_file}" + + # --- Perform migration --- + result = configmigrate.migrate_config_file(working_file, backup_file) + + # --- Assertions --- + assert result is True, f"Migration failed for {old_file.name}" + + assert configmigrate.mapped_count >= 1, f"No mapped migrations for {old_file.name}" + assert configmigrate.auto_count >= 1, f"No automatic migrations for {old_file.name}" + + assert len(configmigrate.skipped_paths) <= 7, ( + f"Too many skipped paths in {old_file.name}: {configmigrate.skipped_paths}" + ) + + assert backup_file.exists(), f"Backup file not created for {old_file.name}" + + # --- Compare migrated result with expected output --- + new_data = json.loads(working_file.read_text(encoding="utf-8")) + expected_data = json.loads(expected_file.read_text(encoding="utf-8")) + + # Check version + assert new_data["general"]["version"] == __version__, ( + f"Expected version {__version__}, got {new_data['general']['version']}" + ) + + # Recursive subset comparison + errors = _dict_contains(new_data, expected_data) + assert not errors, ( + f"Migrated config for {old_file.name} is missing or mismatched fields:\n" + + "\n".join(errors) + ) + + # --- Compare migrated result with migration map --- + # Ensure all expected mapped fields are actually migrated and correct + missing_migrations = [] + mismatched_values = [] + + for old_path, mapping in configmigrate.MIGRATION_MAP.items(): + if mapping is None: + continue # skip intentionally dropped fields + + # Determine new path (string or tuple) + new_path = mapping[0] if isinstance(mapping, tuple) else mapping + + # Get value from expected data (if present) + expected_value = configmigrate._get_json_nested_value(expected_data, new_path) + if expected_value is None: + continue # new field not present in expected config + + # Check that migration recorded this old path + if old_path.strip("/") not in configmigrate.migrated_source_paths: + missing_migrations.append(f"{old_path} → {new_path}") + continue + + # Verify the migrated value matches the expected one + new_value = configmigrate._get_json_nested_value(new_data, new_path) + if new_value != expected_value: + mismatched_values.append( + f"{old_path} → {new_path}: expected {expected_value!r}, got {new_value!r}" + ) + + assert not missing_migrations, ( + "Some expected migration map entries were not migrated:\n" + + "\n".join(missing_migrations) + ) + assert not mismatched_values, ( + "Migrated values differ from expected results:\n" + + "\n".join(mismatched_values) + ) + + # Validate migrated config with schema + new_model = SettingsEOSDefaults(**new_data) + assert isinstance(new_model, SettingsEOSDefaults) + + except Exception: + # mark failure and re-raise so pytest records the error and the working_file is kept + failed = True + raise + finally: + # Remove the .new working file only if the test passed (failed == False) + if not failed and working_file.exists(): + working_file.unlink(missing_ok=True) diff --git a/tests/test_dataabc.py b/tests/test_dataabc.py index 519c14c..36fe1a6 100644 --- a/tests/test_dataabc.py +++ b/tests/test_dataabc.py @@ -22,7 +22,6 @@ from akkudoktoreos.utils.datetimeutil import compare_datetimes, to_datetime, to_ # Derived classes for testing # --------------------------- - class DerivedConfig(SettingsBaseModel): env_var: Optional[int] = Field(default=None, description="Test config by environment var") instance_field: Optional[str] = Field(default=None, description="Test config by instance field") @@ -35,8 +34,19 @@ class DerivedBase(DataBase): class DerivedRecord(DataRecord): + """Date Record derived from base class DataRecord. + + The derived data record got the + - `data_value` field and the + - `dish_washer_emr`, `solar_power`, `temp` configurable field like data. + """ + data_value: Optional[float] = Field(default=None, description="Data Value") + @classmethod + def configured_data_keys(cls) -> Optional[list[str]]: + return ["dish_washer_emr", "solar_power", "temp"] + class DerivedSequence(DataSequence): # overload @@ -128,6 +138,13 @@ class TestDataRecord: """Helper function to create a test DataRecord.""" return DerivedRecord(date_time=date, data_value=value) + @pytest.fixture + def record(self): + """Fixture to create a sample DerivedDataRecord with some data set.""" + rec = DerivedRecord(date_time=None, data_value=10.0) + rec.configured_data = {"dish_washer_emr": 123.0, "solar_power": 456.0} + return rec + def test_getitem(self): record = self.create_test_record(datetime(2024, 1, 3, tzinfo=timezone.utc), 10.0) assert record["data_value"] == 10.0 @@ -147,7 +164,7 @@ class TestDataRecord: record = self.create_test_record(datetime(2024, 1, 3, tzinfo=timezone.utc), 10.0) record.date_time = None record.data_value = 20.0 - assert len(record) == 2 + assert len(record) == 5 # 2 regular fields + 3 configured data "fields" def test_to_dict(self): record = self.create_test_record(datetime(2024, 1, 3, tzinfo=timezone.utc), 10.0) @@ -167,6 +184,202 @@ class TestDataRecord: record2 = DerivedRecord.from_json(json_str) assert record2.model_dump() == record.model_dump() + def test_record_keys_includes_configured_data_keys(self, record): + """Ensure record_keys includes all configured configured data keys.""" + assert set(record.record_keys()) >= set(record.configured_data_keys()) + + def test_record_keys_writable_includes_configured_data_keys(self, record): + """Ensure record_keys_writable includes all configured configured data keys.""" + assert set(record.record_keys_writable()) >= set(record.configured_data_keys()) + + def test_getitem_existing_field(self, record): + """Test that __getitem__ returns correct value for existing native field.""" + record.date_time = "2024-01-01T00:00:00+00:00" + assert record["date_time"] is not None + + def test_getitem_existing_configured_data(self, record): + """Test that __getitem__ retrieves existing configured data values.""" + assert record["dish_washer_emr"] == 123.0 + assert record["solar_power"] == 456.0 + + def test_getitem_missing_configured_data_returns_none(self, record): + """Test that __getitem__ returns None for missing but known configured data keys.""" + assert record["temp"] is None + + def test_getitem_raises_keyerror(self, record): + """Test that __getitem__ raises KeyError for completely unknown keys.""" + with pytest.raises(KeyError): + _ = record["nonexistent"] + + def test_setitem_field(self, record): + """Test setting a native field using __setitem__.""" + record["date_time"] = "2025-01-01T12:00:00+00:00" + assert str(record.date_time).startswith("2025-01-01") + + def test_setitem_configured_data(self, record): + """Test setting a known configured data key using __setitem__.""" + record["temp"] = 25.5 + assert record.configured_data["temp"] == 25.5 + + def test_setitem_invalid_key_raises(self, record): + """Test that __setitem__ raises KeyError for unknown keys.""" + with pytest.raises(KeyError): + record["unknown_key"] = 123 + + def test_delitem_field(self, record): + """Test deleting a native field using __delitem__.""" + record["date_time"] = "2025-01-01T12:00:00+00:00" + del record["date_time"] + assert record.date_time is None + + def test_delitem_configured_data(self, record): + """Test deleting a known configured data key using __delitem__.""" + del record["solar_power"] + assert "solar_power" not in record.configured_data + + def test_delitem_unknown_raises(self, record): + """Test that __delitem__ raises KeyError for unknown keys.""" + with pytest.raises(KeyError): + del record["nonexistent"] + + def test_attribute_get_existing_field(self, record): + """Test accessing a native field via attribute.""" + record.date_time = "2025-01-01T12:00:00+00:00" + assert record.date_time is not None + + def test_attribute_get_existing_configured_data(self, record): + """Test accessing an existing configured data via attribute.""" + assert record.dish_washer_emr == 123.0 + + def test_attribute_get_missing_configured_data(self, record): + """Test accessing a missing but known configured data returns None.""" + assert record.temp is None + + def test_attribute_get_invalid_raises(self, record): + """Test accessing an unknown attribute raises AttributeError.""" + with pytest.raises(AttributeError): + _ = record.nonexistent + + def test_attribute_set_existing_field(self, record): + """Test setting a native field via attribute.""" + record.date_time = "2025-06-25T12:00:00+00:00" + assert record.date_time is not None + + def test_attribute_set_existing_configured_data(self, record): + """Test setting a known configured data key via attribute.""" + record.temp = 99.9 + assert record.configured_data["temp"] == 99.9 + + def test_attribute_set_invalid_raises(self, record): + """Test setting an unknown attribute raises AttributeError.""" + with pytest.raises(AttributeError): + record.invalid = 123 + + def test_delattr_field(self, record): + """Test deleting a native field via attribute.""" + record.date_time = "2025-06-25T12:00:00+00:00" + del record.date_time + assert record.date_time is None + + def test_delattr_configured_data(self, record): + """Test deleting a known configured data key via attribute.""" + record.temp = 88.0 + del record.temp + assert "temp" not in record.configured_data + + def test_delattr_ignored_missing_configured_data_key(self, record): + """Test deleting a known configured data key that was never set is a no-op.""" + del record.temp + assert "temp" not in record.configured_data + + def test_len_and_iter(self, record): + """Test that __len__ and __iter__ behave as expected.""" + keys = list(iter(record)) + assert set(record.record_keys_writable()) == set(keys) + assert len(record) == len(keys) + + def test_in_operator_includes_configured_data(self, record): + """Test that 'in' operator includes configured data keys.""" + assert "dish_washer_emr" in record + assert "temp" in record # known key, even if not yet set + assert "nonexistent" not in record + + def test_hasattr_behavior(self, record): + """Test that hasattr returns True for fields and known configured dataWs.""" + assert hasattr(record, "date_time") + assert hasattr(record, "dish_washer_emr") + assert hasattr(record, "temp") # allowed, even if not yet set + assert not hasattr(record, "nonexistent") + + def test_model_validate_roundtrip(self, record): + """Test that MeasurementDataRecord can be serialized and revalidated.""" + dumped = record.model_dump() + restored = DerivedRecord.model_validate(dumped) + assert restored.dish_washer_emr == 123.0 + assert restored.solar_power == 456.0 + assert restored.temp is None # not set + + def test_copy_preserves_configured_data(self, record): + """Test that copying preserves configured data values.""" + record.temp = 22.2 + copied = record.model_copy() + assert copied.dish_washer_emr == 123.0 + assert copied.temp == 22.2 + assert copied is not record + + def test_equality_includes_configured_data(self, record): + """Test that equality includes the `configured data` content.""" + other = record.model_copy() + assert record == other + + def test_inequality_differs_with_configured_data(self, record): + """Test that records with different configured datas are not equal.""" + other = record.model_copy(deep=True) + # Modify one configured data value in the copy + other.configured_data["dish_washer_emr"] = 999.9 + assert record != other + + def test_in_operator_for_configured_data_and_fields(self, record): + """Ensure 'in' works for both fields and configured configured data keys.""" + assert "dish_washer_emr" in record + assert "solar_power" in record + assert "date_time" in record # standard field + assert "temp" in record # allowed but not yet set + assert "unknown" not in record + + def test_hasattr_equivalence_to_getattr(self, record): + """hasattr should return True for all valid keys/configured datas.""" + assert hasattr(record, "dish_washer_emr") + assert hasattr(record, "temp") + assert hasattr(record, "date_time") + assert not hasattr(record, "nonexistent") + + def test_dir_includes_configured_data_keys(self, record): + """`dir(record)` should include configured data keys for introspection. + It shall not include the internal 'configured datas' attribute. + """ + keys = dir(record) + assert "configured datas" not in keys + for key in record.configured_data_keys(): + assert key in keys + + def test_init_configured_field_like_data_applies_before_model_init(self): + """Test that keys listed in `_configured_data_keys` are moved to `configured_data` at init time.""" + record = DerivedRecord( + date_time="2024-01-03T00:00:00+00:00", + data_value=42.0, + dish_washer_emr=111.1, + solar_power=222.2, + temp=333.3 # assume `temp` is also a valid configured key + ) + + assert record.data_value == 42.0 + assert record.configured_data == { + "dish_washer_emr": 111.1, + "solar_power": 222.2, + "temp": 333.3, + } + class TestDataSequence: @pytest.fixture @@ -385,6 +598,25 @@ class TestDataSequence: assert array[1] == 0.8 # Forward-filled value assert array[2] == 1.0 + def test_key_to_array_ffill_one_value(self, sequence): + """Test key_to_array with forward filling for missing values and only one value at end available.""" + interval = to_duration("1 hour") + record1 = self.create_test_record(pendulum.datetime(2023, 11, 6, 2), 1.0) + sequence.insert_by_datetime(record1) + + array = sequence.key_to_array( + key="data_value", + start_datetime=pendulum.datetime(2023, 11, 6), + end_datetime=pendulum.datetime(2023, 11, 6, 4), + interval=interval, + fill_method="ffill", + ) + assert len(array) == 4 + assert array[0] == 1.0 # Backward-filled value + assert array[1] == 1.0 # Backward-filled value + assert array[2] == 1.0 + assert array[2] == 1.0 # Forward-filled value + def test_key_to_array_bfill(self, sequence): """Test key_to_array with backward filling for missing values.""" interval = to_duration("1 hour") @@ -543,6 +775,55 @@ class TestDataSequence: assert sequence[0].date_time == sequence2[0].date_time assert sequence[0].data_value == sequence2[0].data_value + def test_key_to_value_exact_match(self, sequence): + """Test key_to_value returns exact match when datetime matches a record.""" + dt = datetime(2023, 11, 5) + record = self.create_test_record(dt, 0.75) + sequence.append(record) + result = sequence.key_to_value("data_value", dt) + assert result == 0.75 + + def test_key_to_value_nearest(self, sequence): + """Test key_to_value returns value closest in time to the given datetime.""" + record1 = self.create_test_record(datetime(2023, 11, 5, 12), 0.6) + record2 = self.create_test_record(datetime(2023, 11, 6, 12), 0.9) + sequence.append(record1) + sequence.append(record2) + dt = datetime(2023, 11, 6, 10) # closer to record2 + result = sequence.key_to_value("data_value", dt) + assert result == 0.9 + + def test_key_to_value_nearest_after(self, sequence): + """Test key_to_value returns value nearest after the given datetime.""" + record1 = self.create_test_record(datetime(2023, 11, 5, 10), 0.7) + record2 = self.create_test_record(datetime(2023, 11, 5, 15), 0.8) + sequence.append(record1) + sequence.append(record2) + dt = datetime(2023, 11, 5, 14) # closer to record2 + result = sequence.key_to_value("data_value", dt) + assert result == 0.8 + + def test_key_to_value_empty_sequence(self, sequence): + """Test key_to_value returns None when sequence is empty.""" + result = sequence.key_to_value("data_value", datetime(2023, 11, 5)) + assert result is None + + def test_key_to_value_missing_key(self, sequence): + """Test key_to_value returns None when key is missing in records.""" + record = self.create_test_record(datetime(2023, 11, 5), None) + sequence.append(record) + result = sequence.key_to_value("data_value", datetime(2023, 11, 5)) + assert result is None + + def test_key_to_value_multiple_records_with_none(self, sequence): + """Test key_to_value skips records with None values.""" + r1 = self.create_test_record(datetime(2023, 11, 5), None) + r2 = self.create_test_record(datetime(2023, 11, 6), 1.0) + sequence.append(r1) + sequence.append(r2) + result = sequence.key_to_value("data_value", datetime(2023, 11, 5, 12)) + assert result == 1.0 + def test_key_to_dict(self, sequence): record1 = self.create_test_record(datetime(2023, 11, 5), 0.8) record2 = self.create_test_record(datetime(2023, 11, 6), 0.9) @@ -694,7 +975,7 @@ class TestDataProvider: ems_eos.set_start_datetime(sample_start_datetime) provider.update_data() - assert provider.start_datetime == sample_start_datetime + assert provider.ems_start_datetime == sample_start_datetime def test_update_method_force_enable(self, provider, monkeypatch): """Test that `update` executes when `force_enable` is True, even if `enabled` is False.""" @@ -857,7 +1138,7 @@ class TestDataContainer: del container_with_providers["non_existent_key"] def test_len(self, container_with_providers): - assert len(container_with_providers) == 3 + assert len(container_with_providers) == 5 def test_repr(self, container_with_providers): representation = repr(container_with_providers) diff --git a/tests/test_datetimeutil.py b/tests/test_datetimeutil.py index 4fffaa4..cc3516a 100644 --- a/tests/test_datetimeutil.py +++ b/tests/test_datetimeutil.py @@ -1,20 +1,1297 @@ -"""Test Module for pendulum.datetimeutil Module.""" +"""Comprehensive test suite for the date/time utility module. +This test suite covers all classes and functions in the datetimeutil module, +including edge cases, error handling, and timezone behavior. +""" + +import datetime +import json import re +from typing import Any +from unittest.mock import MagicMock, patch +import babel import pendulum import pytest from pendulum.tz.timezone import Timezone +from pydantic import ValidationError +from akkudoktoreos.core.pydantic import PydanticBaseModel from akkudoktoreos.utils.datetimeutil import ( MAX_DURATION_STRING_LENGTH, + Date, + DateTime, + Duration, + Time, + TimeWindow, + TimeWindowSequence, + _parse_time_string, compare_datetimes, hours_in_day, to_datetime, to_duration, + to_time, to_timezone, ) +# ---------- +# Time Class +# ---------- + + +class TestTimeParsing: + """Comprehensive tests for string → pendulum.Time conversion and roundtrip correctness.""" + + # ------------------------------- + # VALID FORMATS + # ------------------------------- + @pytest.mark.parametrize( + "input_str, expected", + [ + # 24-hour basic formats + ("00:00", (0, 0, 0)), + ("23:59:59", (23, 59, 59)), + ("14:30", (14, 30, 0)), + ("14:30:45", (14, 30, 45)), + ("14:30:45.123456", (14, 30, 45)), + ("1430", (14, 30, 0)), + ("143045", (14, 30, 45)), + ("930", (9, 30, 0)), + ("14", (14, 0, 0)), + ("14.5", (14, 30, 0)), + ("14.25", (14, 15, 0)), + ("14h30", (14, 30, 0)), + ("14-30", (14, 30, 0)), + ("14 30", (14, 30, 0)), + + # 12-hour AM/PM formats + ("12:00 AM", (0, 0, 0)), + ("12:00 PM", (12, 0, 0)), + ("2:30 PM", (14, 30, 0)), + ("2:30:45 PM", (14, 30, 45)), + ("2PM", (14, 0, 0)), + ("11AM", (11, 0, 0)), + + # Compact & decimal + ("3", (3, 0, 0)), + ("23.75", (23, 45, 0)), + + # Offset-based / ISO-like + ("08:00:00.000000+01:00", (8, 0, 0)), + ("14:30 +05:30", (14, 30, 0)), + ("14:30 -03:00", (14, 30, 0)), + ("22:15 -0800", (22, 15, 0)), + + # Alternative separators + ("14-30", (14, 30, 0)), + ("14 30", (14, 30, 0)), + ("14-30-45", (14, 30, 45)), + ("14 30 45", (14, 30, 45)), + + # Timezones by abbreviation + ("14:30 UTC", (14, 30, 0)), + ("14:30 GMT", (14, 30, 0)), + ("2:30 PM EST", (14, 30, 0)), + ("9:15 CST", (9, 15, 0)), + ("23:59 PST", (23, 59, 0)), + + # Named timezones + ("14h30 Europe/Berlin", (14, 30, 0)), + ("14:30 America/New_York", (14, 30, 0)), + ("08:15 Asia/Tokyo", (8, 15, 0)), + ("23:45 Australia/Sydney", (23, 45, 0)), + ], + ) + def test_parse_time_string_valid(self, input_str, expected): + """Ensure various valid time strings parse correctly.""" + result = _parse_time_string(input_str) + assert isinstance(result, pendulum.Time) + assert (result.hour, result.minute, result.second) == expected[:3] + + # ------------------------------- + # INVALID INPUTS + # ------------------------------- + @pytest.mark.parametrize( + "input_str", + [ + "", # empty + "25:00", # invalid hour + "14:61", # invalid minute + "2:30 XM", # bad AM/PM + "noonish", # nonsense + "2400", # invalid compact + "24.999", # beyond 23.999 + "14:30 Mars/Terra", # invalid tz + ], + ) + def test_parse_time_string_invalid(self, input_str): + """Invalid inputs should raise ValueError.""" + with pytest.raises(ValueError): + _parse_time_string(input_str) + + # ------------------------------- + # TIMEZONE HANDLING + # ------------------------------- + @pytest.mark.parametrize( + "input_str, tz_name", + [ + ("14:30 UTC", "UTC"), + ("14:30 Europe/Berlin", "Europe/Berlin"), + ("2:30 PM PST", "America/Los_Angeles"), + ("08:00:00.000000+01:00", "+01:00"), + ("14:30 +05:30", "+05:30"), + ("22:00 -04:00", "-04:00"), + ], + ) + def test_parse_time_string_with_timezone(self, input_str, tz_name): + """Test timezone-aware parsing results in a Time with tzinfo.""" + t = _parse_time_string(input_str) + assert isinstance(t, pendulum.Time) + assert t.tzinfo is not None + # compare normalized zone name + assert tz_name.split("/")[-1] in str(t.tzinfo) or tz_name in str(t.tzinfo), f"{str(t.tzinfo)} vs. expected {tz_name}" + + @pytest.mark.parametrize( + "time_str", + [ + "08:00:00.000000+01:00", + "14:30 UTC", + "2:30 PM PST", + "14h30 Europe/Berlin", + "23:45 America/New_York", + ], + ) + def test_roundtrip_to_string(self, time_str): + """Test that parsing and serializing preserves hour, minute, offset.""" + t = _parse_time_string(time_str) + s = t.isoformat() + reparsed = _parse_time_string(s) + assert t.hour == reparsed.hour + assert t.minute == reparsed.minute + assert t.second == reparsed.second + assert t.utcoffset() == reparsed.utcoffset() + + def test_microsecond_precision_and_offset(self): + """Ensure microseconds and offset are exact.""" + t = _parse_time_string("08:00:00.000001+01:00") + assert t.microsecond == 1 + assert t.strftime("%z") in ("+0100", "+01:00") + + def test_parse_edge_cases(self): + """Test parsing edge cases.""" + # Test with whitespace + result = _parse_time_string(" 14:30 ") + assert result.hour == 14 + assert result.minute == 30 + + # Test case insensitivity + result = _parse_time_string("2:30 pm") + assert result.hour == 14 + assert result.minute == 30 + + # Test mixed case + result = _parse_time_string("14H30") + assert result.hour == 14 + assert result.minute == 30 + + +class TestTime: + """Test suite for the custom Time class.""" + + def test_time_creation_basic(self): + """Test basic Time object creation.""" + t = Time(14, 30, 45, 123456) + assert t.hour == 14 + assert t.minute == 30 + assert t.second == 45 + assert t.microsecond == 123456 + assert t.tzinfo is None + + def test_time_creation_with_timezone(self): + """Test Time object creation with timezone.""" + berlin_tz = pendulum.timezone("Europe/Berlin") + t = Time(14, 30, 0, tzinfo=berlin_tz) + assert t.hour == 14 + assert t.minute == 30 + assert t.second == 0 + assert t.tzinfo == berlin_tz + + def test_pydantic_validation_valid_time(self): + """Test pydantic validation with valid Time object.""" + berlin_tz = pendulum.timezone("Europe/Berlin") + pend_time = pendulum.time(14, 30, 0).replace(tzinfo=berlin_tz) + + # This should not raise an exception + validated = Time._validate(pend_time) + assert isinstance(validated, Time) + assert validated.hour == 14 + assert validated.minute == 30 + assert validated.tzinfo == berlin_tz + + def test_pydantic_validation_valid_time_in_other_timezone(self, set_other_timezone): + """Test pydantic validation with valid Time object running in different timezone.""" + timezone = set_other_timezone() + + berlin_tz = pendulum.timezone("Europe/Berlin") + pend_time = pendulum.time(14, 30, 0).replace(tzinfo=berlin_tz) + assert isinstance(pend_time.tzinfo, Timezone) + assert pend_time.tzinfo == berlin_tz + + # This should not raise an exception + validated = Time._validate(pend_time) + assert isinstance(validated, Time) + assert validated.hour == 14 + assert validated.minute == 30 + assert validated.tzinfo == berlin_tz + + def test_pydantic_validation_string_input(self): + """Test pydantic validation with string input.""" + time_str = "14:30:45" + validated = Time._validate(time_str) + assert isinstance(validated, Time) + assert validated.hour == 14 + assert validated.minute == 30 + assert validated.second == 45 + + def test_pydantic_validation_none_input(self): + """Test pydantic validation with None input raises ValueError.""" + with pytest.raises(ValueError, match="Time value cannot be None"): + Time._validate(None) + + def test_pydantic_validation_invalid_input(self): + """Test pydantic validation with invalid input.""" + with pytest.raises(ValueError, match="Invalid time value"): + Time._validate("invalid_time") + + def test_serialization_naive_time(self): + """Test serialization of naive Time object.""" + t = Time(14, 30, 45, 123456) + serialized = Time._serialize(t) + assert serialized == "14:30:45.123456" + + def test_serialization_timezone_aware_time(self): + """Test serialization of timezone-aware Time object.""" + berlin_tz = pendulum.timezone("Europe/Berlin") + t = Time(14, 30, 45, 123456, tzinfo=berlin_tz) + serialized = Time._serialize(t) + assert "14:30:45.123456" in serialized + assert "Europe/Berlin" in serialized or "+0" in serialized + + def test_serialization_none_value(self): + """Test serialization of None value.""" + serialized = Time._serialize(None) + assert serialized == "" + + def test_repr_naive_time(self): + """Test __repr__ for naive Time.""" + t = Time(14, 30, 45, 123456) + repr_str = repr(t) + assert "Time(14, 30, 45, 123456)" in repr_str + assert "tzinfo" not in repr_str + + def test_repr_timezone_aware_time(self): + """Test __repr__ for timezone-aware Time.""" + berlin_tz = pendulum.timezone("Europe/Berlin") + t = Time(14, 30, 45, 123456, tzinfo=berlin_tz) + repr_str = repr(t) + assert "Time(14, 30, 45, 123456, tzinfo=" in repr_str + assert "Europe/Berlin" in repr_str + + def test_str_representation(self): + """Test __str__ method.""" + t = Time(14, 30, 45, 123456) + str_repr = str(t) + assert str_repr == "14:30:45.123456" + + def test_equality_naive_times(self): + """Test equality comparison for naive times.""" + t1 = Time(14, 30, 45) + t2 = Time(14, 30, 45) + t3 = Time(14, 30, 46) + + assert t1 == t2 + assert t1 != t3 + + def test_equality_timezone_aware_times(self): + """Test equality comparison for timezone-aware times.""" + berlin_tz = pendulum.timezone("Europe/Berlin") + utc_tz = pendulum.timezone("UTC") + + t1 = Time(14, 30, 0, tzinfo=berlin_tz) + t2 = Time(14, 30, 0, tzinfo=berlin_tz) + t3 = Time(14, 30, 0, tzinfo=utc_tz) + + assert t1 == t2 + + def test_equality_mixed_timezone_naive(self): + """Test equality comparison between timezone-aware and naive times.""" + berlin_tz = pendulum.timezone("Europe/Berlin") + t1 = Time(14, 30, 0, tzinfo=berlin_tz) + t2 = Time(14, 30, 0) # naive + + # Mixed comparison should use direct comparison + assert t1 == t2 + + def test_hash_naive_time(self): + """Test hash function for naive time.""" + t1 = Time(14, 30, 45) + t2 = Time(14, 30, 45) + + assert hash(t1) == hash(t2) + + # Test that times can be used in sets + time_set = {t1, t2} + assert len(time_set) == 1 + + def test_hash_timezone_aware_time(self): + """Test hash function for timezone-aware time.""" + berlin_tz = pendulum.timezone("Europe/Berlin") + t1 = Time(14, 30, 0, tzinfo=berlin_tz) + t2 = Time(14, 30, 0, tzinfo=berlin_tz) + + assert hash(t1) == hash(t2) + + def test_is_naive(self): + """Test is_naive method.""" + t_naive = Time(14, 30, 0) + t_aware = Time(14, 30, 0, tzinfo=pendulum.timezone("UTC")) + + assert t_naive.is_naive() is True + assert t_aware.is_naive() is False + + def test_is_aware(self): + """Test is_aware method.""" + t_naive = Time(14, 30, 0) + t_aware = Time(14, 30, 0, tzinfo=pendulum.timezone("UTC")) + + assert t_naive.is_aware() is False + assert t_aware.is_aware() is True + + def test_replace_timezone(self): + """Test replace_timezone method.""" + t = Time(14, 30, 0) + berlin_tz = pendulum.timezone("Europe/Berlin") + + t_with_tz = t.replace_timezone(berlin_tz) + assert t_with_tz.tzinfo == berlin_tz + assert t_with_tz.hour == 14 # Time should not change + + # Test with string timezone + t_with_str_tz = t.replace_timezone("UTC") + assert t_with_str_tz.tzinfo == pendulum.timezone("UTC") + + def test_replace_timezone_none(self): + """Test replace_timezone with None removes timezone.""" + berlin_tz = pendulum.timezone("Europe/Berlin") + t = Time(14, 30, 0, tzinfo=berlin_tz) + + t_naive = t.replace_timezone(None) + assert t_naive.tzinfo is None + + def test_format_user_friendly_basic(self): + """Test format_user_friendly with basic options.""" + t = Time(14, 30, 45) + + # Without seconds + formatted = t.format_user_friendly(include_seconds=False) + assert formatted == "14:30" + + # With seconds + formatted = t.format_user_friendly(include_seconds=True) + assert formatted == "14:30:45" + + def test_format_user_friendly_with_timezone(self): + """Test format_user_friendly with timezone.""" + berlin_tz = pendulum.timezone("Europe/Berlin") + t = Time(14, 30, 45, tzinfo=berlin_tz) + + # Auto-include timezone + formatted = t.format_user_friendly() + assert "14:30" in formatted + assert any(tz_indicator in formatted for tz_indicator in ["+", "-", "Z"]) + + def test_now_classmethod(self): + """Test now() class method.""" + now = Time.now() + assert isinstance(now, Time) + assert now.tzinfo is not None # Should have timezone info + + # Test with specific timezone + utc_now = Time.now("UTC") + assert isinstance(utc_now, Time) + assert utc_now.tzinfo == pendulum.timezone("UTC") + + def test_parse_classmethod(self): + """Test parse() class method.""" + time_str = "14:30:45" + parsed = Time.parse(time_str) + assert isinstance(parsed, Time) + assert parsed.hour == 14 + assert parsed.minute == 30 + assert parsed.second == 45 + + def test_in_timezone_conversion(self): + """Test in_timezone method for actual timezone conversion.""" + utc_tz = pendulum.timezone("UTC") + berlin_tz = pendulum.timezone("Europe/Berlin") + + # Create UTC time + utc_time = Time(12, 0, 0, tzinfo=utc_tz) + + # Convert to Berlin time + berlin_time = utc_time.in_timezone(berlin_tz) + assert isinstance(berlin_time, Time) + assert berlin_time.tzinfo == berlin_tz + # The actual hour will depend on DST, but it should be different from 12 + # This is a simplified test - you may need to adjust based on actual conversion logic + + def test_in_timezone_naive_time(self): + """Test in_timezone with naive time.""" + t = Time(14, 30, 0) # naive + berlin_tz = pendulum.timezone("Europe/Berlin") + + result = t.in_timezone(berlin_tz) + assert isinstance(result, Time) + # Should assume local timezone and convert + + def test_to_local(self): + """Test to_local method.""" + utc_tz = pendulum.timezone("UTC") + t = Time(12, 0, 0, tzinfo=utc_tz) + + local_time = t.to_local() + assert isinstance(local_time, Time) + assert local_time.tzinfo == pendulum.local_timezone() + + def test_to_utc(self): + """Test to_utc method.""" + berlin_tz = pendulum.timezone("Europe/Berlin") + t = Time(14, 0, 0, tzinfo=berlin_tz) + + utc_time = t.to_utc() + assert isinstance(utc_time, Time) + assert utc_time.tzinfo == pendulum.timezone("UTC") + + def test_create_from_pendulum_time(self): + """Test _create_from_pendulum_time class method.""" + berlin_tz = pendulum.timezone("Europe/Berlin") + pend_time = pendulum.time(14, 30, 45, 123456).replace(tzinfo=berlin_tz) + + custom_time = Time._create_from_pendulum_time(pend_time) + assert isinstance(custom_time, Time) + assert custom_time.hour == 14 + assert custom_time.minute == 30 + assert custom_time.second == 45 + assert custom_time.microsecond == 123456 + assert custom_time.tzinfo == berlin_tz + + +# ------- +# to_time +# ------- + + +class TestToTime: + """Test suite for the to_time function.""" + + def test_to_time_string_input(self): + """Test to_time with string input.""" + result = to_time("14:30:45") + assert isinstance(result, Time) + assert result.hour == 14 + assert result.minute == 30 + assert result.second == 45 + + def test_to_time_time_object_input(self): + """Test to_time with Time object input.""" + t = Time(14, 30, 45) + result = to_time(t) + assert isinstance(result, Time) + assert result.hour == 14 + assert result.minute == 30 + assert result.second == 45 + + def test_to_time_pendulum_time_input(self): + """Test to_time with pendulum.Time input.""" + pend_time = pendulum.time(14, 30, 45) + result = to_time(pend_time) + assert isinstance(result, Time) + assert result.hour == 14 + assert result.minute == 30 + assert result.second == 45 + + def test_to_time_datetime_time_input(self): + """Test to_time with datetime.time input.""" + dt_time = datetime.time(14, 30, 45) + result = to_time(dt_time) + assert isinstance(result, Time) + assert result.hour == 14 + assert result.minute == 30 + assert result.second == 45 + + def test_to_time_datetime_datetime_input(self): + """Test to_time with datetime.datetime input.""" + dt_datetime = datetime.datetime(2023, 10, 15, 14, 30, 45) + result = to_time(dt_datetime, in_timezone = "UTC") + assert isinstance(result, Time) + assert result.hour == 14 + assert result.minute == 30 + assert result.second == 45 + + def test_to_time_integer_input(self): + """Test to_time with integer input (hour only).""" + result = to_time(14) + assert isinstance(result, Time) + assert result.hour == 14 + assert result.minute == 0 + assert result.second == 0 + + def test_to_time_float_input(self): + """Test to_time with float input (decimal hours).""" + result = to_time(14.5) # 14:30 + assert isinstance(result, Time) + assert result.hour == 14 + assert result.minute == 30 + assert result.second == 0 + + def test_to_time_tuple_input(self): + """Test to_time with tuple input.""" + test_cases = [ + ((14,), 14, 0, 0, 0), + ((14, 30), 14, 30, 0, 0), + ((14, 30, 45), 14, 30, 45, 0), + ((14, 30, 45, 123456), 14, 30, 45, 123456), + ] + + for tuple_input, expected_hour, expected_minute, expected_second, expected_microsecond in test_cases: + result = to_time(tuple_input) + assert isinstance(result, Time) + assert result.hour == expected_hour + assert result.minute == expected_minute + assert result.second == expected_second + assert result.microsecond == expected_microsecond + + def test_to_time_with_timezone(self): + """Test to_time with timezone parameter.""" + result = to_time("14:30", in_timezone="Europe/Berlin") + assert isinstance(result, Time) + assert result.hour == 14 + assert result.minute == 30 + assert result.tzinfo == pendulum.timezone("Europe/Berlin") + + def test_to_time_to_naive(self): + """Test to_time with to_naive=True.""" + #result = to_time("14:30", in_timezone="Europe/Berlin", to_naive=True) + result = to_time("14:30", to_naive=True) + #result = to_time("14:30") + assert isinstance(result, Time) + assert result.hour == 14 + assert result.minute == 30 + assert result.tzinfo is None + + def test_to_time_as_string_true(self): + """Test to_time with as_string=True.""" + result = to_time("14:30:45", as_string=True) + assert isinstance(result, str) + assert "14:30:45" in result + + def test_to_time_as_string_format(self): + """Test to_time with custom format string.""" + result = to_time("14:30:45", as_string="HH:mm") + assert isinstance(result, str) + assert result == "14:30" + + def test_to_time_timezone_conversion(self): + """Test to_time with timezone conversion.""" + berlin_tz = pendulum.timezone("Europe/Berlin") + utc_tz = pendulum.timezone("UTC") + + # Create time with Berlin timezone + berlin_time = pendulum.time(14, 30, 0).replace(tzinfo=berlin_tz) + + # Convert to UTC + result = to_time(berlin_time, in_timezone="UTC") + assert isinstance(result, Time) + assert result.tzinfo == utc_tz + # The hour should be different due to timezone conversion + + def test_to_time_invalid_timezone(self): + """Test to_time with invalid timezone.""" + with pytest.raises(ValueError, match="Invalid timezone"): + to_time("14:30", in_timezone="Invalid/Timezone") + + def test_to_time_invalid_input_type(self): + """Test to_time with invalid input type.""" + with pytest.raises(ValueError, match="Unsupported type"): + to_time({"invalid": "input"}) + + def test_to_time_invalid_hour_integer(self): + """Test to_time with invalid hour as integer.""" + with pytest.raises(ValueError, match="Hour must be between 0 and 23"): + to_time(25) + + def test_to_time_invalid_hour_float(self): + """Test to_time with invalid hour as float.""" + with pytest.raises(ValueError, match="Hour must be between 0 and 23"): + to_time(25.5) + + def test_to_time_empty_tuple(self): + """Test to_time with empty tuple.""" + with pytest.raises(ValueError, match="Empty tuple provided"): + to_time(()) + + def test_to_time_pendulum_datetime_input(self): + """Test to_time with pendulum DateTime input.""" + dt = pendulum.datetime(2023, 10, 15, 14, 30, 45) + result = to_time(dt, in_timezone = "UTC") + assert isinstance(result, Time) + assert result.hour == 14 + assert result.minute == 30 + assert result.second == 45 + + def test_to_time_with_timezone_object(self): + """Test to_time with timezone object instead of string.""" + berlin_tz = pendulum.timezone("Europe/Berlin") + result = to_time("14:30", in_timezone=berlin_tz) + assert isinstance(result, Time) + assert result.tzinfo == berlin_tz + + def test_to_time_invalid_timezone_type(self): + """Test to_time with invalid timezone type.""" + with pytest.raises(ValueError, match="Invalid timezone"): + to_time("14:30", in_timezone=123) + + def test_to_time_microseconds_precision(self): + """Test to_time preserves microsecond precision.""" + result = to_time("14:30:45.123456") + assert isinstance(result, Time) + assert result.microsecond == 123456 + + def test_to_time_fallback_parsing(self): + """Test to_time fallback parsing mechanisms.""" + # Test with a format that might not be caught by the main parser + # This tests the fallback to pendulum.parse + result = to_time("14:30:45") + assert isinstance(result, Time) + assert result.hour == 14 + assert result.minute == 30 + assert result.second == 45 + + @patch('akkudoktoreos.utils.datetimeutil.logger.trace') + def test_to_time_logging_on_parse_failures(self, mock_trace): + """Test that parsing failures are logged appropriately.""" + # This test verifies that failed parsing attempts are logged + with pytest.raises(ValueError): + to_time("definitely_invalid_time_format") + + # Verify that trace logs were called for failed parsing attempts + assert mock_trace.called + + def test_to_time_timezone_aware_datetime_input(self): + """Test to_time with timezone-aware datetime input.""" + tz = datetime.timezone.utc + dt = datetime.datetime(2023, 10, 15, 14, 30, 45, tzinfo=tz) + result = to_time(dt) + assert isinstance(result, Time) + assert result.hour == 14 + assert result.minute == 30 + assert result.second == 45 + assert result.tzinfo is not None + + +# ---------------- +# to_time and Time +# ---------------- + + +class TestTimeUtilityIntegration: + """Integration tests for the time utility functions.""" + + def test_time_roundtrip_serialization(self): + """Test that Time objects can be serialized and deserialized.""" + original = Time(14, 30, 45, 123456, tzinfo=pendulum.timezone("Europe/Berlin")) + + # Serialize + serialized = Time._serialize(original) + assert serialized == "14:30:45.123456 Europe/Berlin" + + # Parse back + parsed = Time.parse(serialized) + + assert parsed.hour == original.hour + assert parsed.minute == original.minute + assert parsed.second == original.second + assert parsed.microsecond == original.microsecond + + def test_time_pydantic_integration(self): + """Test Time class integration with Pydantic models.""" + class TestModel(PydanticBaseModel): + test_time: Time + + # Test with string input + model = TestModel(test_time="14:30:45") + assert isinstance(model.test_time, Time) + assert model.test_time.hour == 14 + + + def test_time_class_uses_to_time_logic(self): + """Test that Time class validation uses the same logic as to_time.""" + # Test with various inputs that both should handle identically + test_cases = [ + "14:30", + 14.5, + (14, 30), + datetime.time(14, 30), + pendulum.time(14, 30) + ] + + class TestModel(PydanticBaseModel): + test_time: Time + + for case in test_cases: + # Both should produce the same result + direct_result = to_time(case) + model_result = TestModel(test_time=case).test_time + + assert direct_result.hour == model_result.hour + assert direct_result.minute == model_result.minute + assert direct_result.second == model_result.second + + +# ------------------------------------ +# date and time types used in pydantic +# ------------------------------------ + +class ScheduleModel(PydanticBaseModel): + start_time: Time + run_duration: Duration + scheduled_at: DateTime + run_on: Date + + +class TestPendulumTypes: + + def test_valid_schedule_model(self): + model = ScheduleModel( + start_time="14:30:00", + run_duration=to_duration("PT2H"), + scheduled_at=to_datetime("2025-07-04T09:00:00+02:00"), + run_on=to_datetime("2025-07-04") + ) + + assert isinstance(model.start_time, pendulum.Time) + assert isinstance(model.run_duration, pendulum.Duration) + assert isinstance(model.scheduled_at, pendulum.DateTime) + assert isinstance(model.run_on, pendulum.Date) + + assert model.start_time.hour == 14 + assert model.run_duration.in_hours() == 2 + assert model.scheduled_at.to_date_string() == "2025-07-04" + assert model.run_on.to_date_string() == "2025-07-04" + + def test_json_serialization(self): + model = ScheduleModel( + start_time=pendulum.time(6, 15), + run_duration=pendulum.duration(minutes=45), + scheduled_at=pendulum.datetime(2025, 7, 4, 6, 15, tz="Europe/Berlin"), + run_on=pendulum.date(2025, 7, 4) + ) + + json_data = model.model_dump(mode="json") + assert "06:15:00" in json_data["start_time"] + assert "PT45M" in json_data["run_duration"] + assert "2025-07-04T06:15:00" in json_data["scheduled_at"] + assert "2025-07-04" in json_data["run_on"] + + json_str = model.model_dump_json() + assert '"06:15:00' in json_str + assert "45 minutes" in json_str + assert "2025-07-04 06:15:00" in json_str + assert '"2025-07-04"' in json_str + + def test_invalid_start_time(self): + with pytest.raises(ValidationError): + ScheduleModel( + start_time="invalid", + run_duration="PT1H", + scheduled_at="2025-07-04T09:00:00+02:00", + run_on="2025-07-04" + ) + + def test_invalid_duration(self): + with pytest.raises(ValidationError): + ScheduleModel( + start_time="10:00:00", + run_duration="2 hours", # invalid ISO 8601 duration + scheduled_at="2025-07-04T09:00:00+02:00", + run_on="2025-07-04" + ) + + def test_type_coercion(self): + dt = pendulum.datetime(2025, 7, 4, 12, 0) + model = ScheduleModel( + start_time=pendulum.time(12, 0), + run_duration=pendulum.duration(hours=3), + scheduled_at=dt, + run_on=dt.date() + ) + assert model.scheduled_at.hour == 12 + assert model.run_duration.total_minutes() == 180 + + +# ----------------------------- +# TimeWindow +# ----------------------------- + + +class TestTimeWindow: + """Tests for the TimeWindow model.""" + + def test_datetime_within_and_outside_window(self): + """Test datetime containment logic inside and outside the time window.""" + window = TimeWindow(start_time=Time(6, 0), duration=Duration(hours=3)) + assert window.contains(DateTime(2025, 7, 12, 7, 30)) is True # Inside + assert window.contains(DateTime(2025, 7, 12, 9, 30)) is False # Outside + + def test_contains_with_duration(self): + """Test datetime with duration that does and doesn't fit in the window.""" + window = TimeWindow(start_time=Time(6, 0), duration=Duration(hours=3)) + assert window.contains(DateTime(2025, 7, 12, 6, 30), duration=Duration(minutes=60)) is True + assert window.contains(DateTime(2025, 7, 12, 6, 30), duration=Duration(hours=3)) is False + + def test_day_of_week_filter(self): + """Test time window restricted by day of week.""" + window = TimeWindow(start_time=Time(6, 0), duration=Duration(hours=2), day_of_week=5) # Saturday + assert window.contains(DateTime(2025, 7, 12, 6, 30)) is True # Saturday + assert window.contains(DateTime(2025, 7, 11, 6, 30)) is False # Friday + + def test_day_of_week_as_english_name(self): + """Test time window with English weekday name.""" + window = TimeWindow(start_time=Time(6, 0), duration=Duration(hours=2), day_of_week="monday") + assert window.contains(DateTime(2025, 7, 7, 6, 30)) is True # Monday + assert window.contains(DateTime(2025, 7, 5, 6, 30)) is False # Saturday + + def test_specific_date_filter(self): + """Test time window restricted by exact date.""" + window = TimeWindow(start_time=Time(6, 0), duration=Duration(hours=2), date=Date(2025, 7, 12)) + assert window.contains(DateTime(2025, 7, 12, 6, 30)) is True + assert window.contains(DateTime(2025, 7, 13, 6, 30)) is False + + def test_invalid_field_types_raise_validation(self): + """Test invalid types raise a Pydantic validation error.""" + with pytest.raises(ValidationError): + TimeWindow(start_time="not_a_time", duration="3h") + + @pytest.mark.parametrize("locale, weekday_name, expected_dow", [ + ("de", "Montag", 0), + ("de", "Samstag", 5), + ("es", "lunes", 0), + ("es", "sábado", 5), + ("fr", "lundi", 0), + ("fr", "samedi", 5), + ]) + def test_localized_day_names(self, locale, weekday_name, expected_dow): + """Test that localized weekday names are resolved to correct weekday index.""" + window = TimeWindow(start_time=Time(6, 0), duration=Duration(hours=2), day_of_week=weekday_name, locale=locale) + assert window.day_of_week == expected_dow + + +# ------------------ +# TimeWindowSequence +# ------------------ + + +class TestTimeWindowSequence: + """Test suite for TimeWindowSequence model.""" + + @pytest.fixture + def sample_time_window_1(self): + """Morning window: 9:00 AM - 12:00 PM.""" + return TimeWindow( + start_time=Time(9, 0, 0), + duration=Duration(hours=3) + ) + + @pytest.fixture + def sample_time_window_2(self): + """Afternoon window: 2:00 PM - 5:00 PM.""" + return TimeWindow( + start_time=Time(14, 0, 0), + duration=Duration(hours=3) + ) + + @pytest.fixture + def monday_window(self): + """Monday only window: 10:00 AM - 11:00 AM.""" + return TimeWindow( + start_time=Time(10, 0, 0), + duration=Duration(hours=1), + day_of_week=0 # Monday + ) + + @pytest.fixture + def specific_date_window(self): + """Specific date window: 1:00 PM - 3:00 PM on 2025-01-15.""" + return TimeWindow( + start_time=Time(13, 0, 0), + duration=Duration(hours=2), + date=Date(2025, 1, 15) + ) + + @pytest.fixture + def sample_sequence(self, sample_time_window_1, sample_time_window_2): + """Sequence with morning and afternoon windows.""" + return TimeWindowSequence(windows=[sample_time_window_1, sample_time_window_2]) + + @pytest.fixture + def sample_sequence_json(self, sample_time_window_1, sample_time_window_2): + """Sequence with morning and afternoon windows.""" + seq_json = TimeWindowSequence(windows=[sample_time_window_1, sample_time_window_2]).model_dump() + return seq_json + + @pytest.fixture + def sample_sequence_json_str(self, sample_time_window_1, sample_time_window_2): + """Sequence with morning and afternoon windows.""" + seq_json_str = TimeWindowSequence(windows=[sample_time_window_1, sample_time_window_2]).model_dumps(indent=2) + return seq_json_str + + @pytest.fixture + def reference_date(self): + """Reference date for testing: 2025-01-15 (Wednesday).""" + return pendulum.parse("2025-01-15T08:00:00") + + def test_init_with_none_windows(self): + """Test initialization with None windows creates empty list.""" + sequence = TimeWindowSequence() + assert sequence.windows == [] + assert len(sequence) == 0 + + def test_init_with_explicit_none(self): + """Test initialization with explicit None windows.""" + sequence = TimeWindowSequence(windows=None) + assert sequence.windows == [] + assert len(sequence) == 0 + + def test_init_with_empty_list(self): + """Test initialization with empty list.""" + sequence = TimeWindowSequence(windows=[]) + assert sequence.windows == [] + assert len(sequence) == 0 + + def test_init_with_windows(self, sample_time_window_1, sample_time_window_2): + """Test initialization with windows.""" + sequence = TimeWindowSequence(windows=[sample_time_window_1, sample_time_window_2]) + assert len(sequence) == 2 + assert sequence.windows is not None # make mypy happy + assert sequence.windows[0] == sample_time_window_1 + assert sequence.windows[1] == sample_time_window_2 + + def test_iterator_protocol(self, sample_sequence): + """Test that sequence supports iteration.""" + windows = list(sample_sequence) + assert len(windows) == 2 + assert all(isinstance(window, TimeWindow) for window in windows) + + def test_indexing(self, sample_sequence, sample_time_window_1): + """Test indexing into sequence.""" + assert sample_sequence[0] == sample_time_window_1 + + def test_length(self, sample_sequence): + """Test len() support.""" + assert len(sample_sequence) == 2 + + def test_contains_empty_sequence(self, reference_date): + """Test contains() with empty sequence returns False.""" + sequence = TimeWindowSequence() + assert not sequence.contains(reference_date) + assert not sequence.contains(reference_date, Duration(hours=1)) + + def test_contains_datetime_in_window(self, sample_sequence, reference_date): + """Test contains() finds datetime in one of the windows.""" + # 10:00 AM should be in the morning window (9:00 AM - 12:00 PM) + test_time = reference_date.replace(hour=10, minute=0) + assert sample_sequence.contains(test_time) + + def test_contains_datetime_not_in_any_window(self, sample_sequence, reference_date): + """Test contains() returns False when datetime is not in any window.""" + # 1:00 PM should not be in any window (gap between morning and afternoon) + test_time = reference_date.replace(hour=13, minute=0) + assert not sample_sequence.contains(test_time) + + def test_contains_with_duration_fits(self, sample_sequence, reference_date): + """Test contains() with duration that fits in a window.""" + # 10:00 AM with 1 hour duration should fit in morning window + test_time = reference_date.replace(hour=10, minute=0) + assert sample_sequence.contains(test_time, Duration(hours=1)) + + def test_contains_with_duration_too_long(self, sample_sequence, reference_date): + """Test contains() with duration that doesn't fit in any window.""" + # 11:00 AM with 2 hours duration won't fit in remaining morning window time + test_time = reference_date.replace(hour=11, minute=0) + assert not sample_sequence.contains(test_time, Duration(hours=2)) + + def test_earliest_start_time_empty_sequence(self, reference_date): + """Test earliest_start_time() with empty sequence returns None.""" + sequence = TimeWindowSequence() + assert sequence.earliest_start_time(Duration(hours=1), reference_date) is None + + def test_earliest_start_time_finds_earliest(self, sample_sequence, reference_date): + """Test earliest_start_time() finds the earliest time across all windows.""" + # Should return 9:00 AM (start of morning window) + earliest = sample_sequence.earliest_start_time(Duration(hours=1), reference_date) + expected = reference_date.replace(hour=9, minute=0, second=0, microsecond=0) + assert earliest == expected + + def test_earliest_start_time_duration_too_long(self, sample_sequence, reference_date): + """Test earliest_start_time() with duration longer than any window.""" + # 4 hours won't fit in any 3-hour window + assert sample_sequence.earliest_start_time(Duration(hours=4), reference_date) is None + + def test_latest_start_time_empty_sequence(self, reference_date): + """Test latest_start_time() with empty sequence returns None.""" + sequence = TimeWindowSequence() + assert sequence.latest_start_time(Duration(hours=1), reference_date) is None + + def test_latest_start_time_finds_latest(self, sample_sequence, reference_date): + """Test latest_start_time() finds the latest time across all windows.""" + # Should return 4:00 PM (latest start for 1 hour in afternoon window) + latest = sample_sequence.latest_start_time(Duration(hours=1), reference_date) + expected = reference_date.replace(hour=16, minute=0, second=0, microsecond=0) + assert latest == expected + + def test_can_fit_duration_empty_sequence(self, reference_date): + """Test can_fit_duration() with empty sequence returns False.""" + sequence = TimeWindowSequence() + assert not sequence.can_fit_duration(Duration(hours=1), reference_date) + + def test_can_fit_duration_fits_in_one_window(self, sample_sequence, reference_date): + """Test can_fit_duration() returns True when duration fits in one window.""" + assert sample_sequence.can_fit_duration(Duration(hours=2), reference_date) + + def test_can_fit_duration_too_long(self, sample_sequence, reference_date): + """Test can_fit_duration() returns False when duration is too long.""" + assert not sample_sequence.can_fit_duration(Duration(hours=4), reference_date) + + def test_available_duration_empty_sequence(self, reference_date): + """Test available_duration() with empty sequence returns None.""" + sequence = TimeWindowSequence() + assert sequence.available_duration(reference_date) is None + + def test_available_duration_sums_all_windows(self, sample_sequence, reference_date): + """Test available_duration() sums durations from all applicable windows.""" + # 3 hours + 3 hours = 6 hours total + total = sample_sequence.available_duration(reference_date) + assert total == Duration(hours=6) + + def test_available_duration_with_day_restriction(self, monday_window, reference_date): + """Test available_duration() respects day restrictions.""" + sequence = TimeWindowSequence(windows=[monday_window]) + + # Reference date is Wednesday, so Monday window shouldn't apply + assert sequence.available_duration(reference_date) is None + + # Monday date should apply + monday_date = pendulum.parse("2025-01-13T08:00:00") # Monday + assert sequence.available_duration(monday_date) == Duration(hours=1) + + def test_get_applicable_windows_empty_sequence(self, reference_date): + """Test get_applicable_windows() with empty sequence.""" + sequence = TimeWindowSequence() + assert sequence.get_applicable_windows(reference_date) == [] + + def test_get_applicable_windows_all_apply(self, sample_sequence, reference_date): + """Test get_applicable_windows() returns all windows when they all apply.""" + applicable = sample_sequence.get_applicable_windows(reference_date) + assert len(applicable) == 2 + + def test_get_applicable_windows_with_restrictions(self, monday_window, reference_date): + """Test get_applicable_windows() respects day restrictions.""" + sequence = TimeWindowSequence(windows=[monday_window]) + + # Wednesday - no applicable windows + assert sequence.get_applicable_windows(reference_date) == [] + + # Monday - one applicable window + monday_date = pendulum.parse("2025-01-13T08:00:00") + applicable = sequence.get_applicable_windows(monday_date) + assert len(applicable) == 1 + assert applicable[0] == monday_window + + def test_find_windows_for_duration_empty_sequence(self, reference_date): + """Test find_windows_for_duration() with empty sequence.""" + sequence = TimeWindowSequence() + assert sequence.find_windows_for_duration(Duration(hours=1), reference_date) == [] + + def test_find_windows_for_duration_all_fit(self, sample_sequence, reference_date): + """Test find_windows_for_duration() when duration fits in all windows.""" + fitting = sample_sequence.find_windows_for_duration(Duration(hours=2), reference_date) + assert len(fitting) == 2 + + def test_find_windows_for_duration_some_fit(self, sample_sequence, reference_date): + """Test find_windows_for_duration() when duration fits in some windows.""" + # Add a short window that can't fit 2.5 hours + short_window = TimeWindow(start_time=Time(18, 0, 0), duration=Duration(hours=1)) + sequence = TimeWindowSequence(windows=sample_sequence.windows + [short_window]) + + fitting = sequence.find_windows_for_duration(Duration(hours=2, minutes=30), reference_date) + assert len(fitting) == 2 # Only the first two windows can fit 2.5 hours + + def test_get_all_possible_start_times_empty_sequence(self, reference_date): + """Test get_all_possible_start_times() with empty sequence.""" + sequence = TimeWindowSequence() + assert sequence.get_all_possible_start_times(Duration(hours=1), reference_date) == [] + + def test_get_all_possible_start_times_multiple_windows(self, sample_sequence, reference_date): + """Test get_all_possible_start_times() returns ranges for all fitting windows.""" + ranges = sample_sequence.get_all_possible_start_times(Duration(hours=1), reference_date) + assert len(ranges) == 2 + + # Check morning window range + earliest_morning, latest_morning, morning_window = ranges[0] + assert earliest_morning == reference_date.replace(hour=9, minute=0, second=0, microsecond=0) + assert latest_morning == reference_date.replace(hour=11, minute=0, second=0, microsecond=0) + + # Check afternoon window range + earliest_afternoon, latest_afternoon, afternoon_window = ranges[1] + assert earliest_afternoon == reference_date.replace(hour=14, minute=0, second=0, microsecond=0) + assert latest_afternoon == reference_date.replace(hour=16, minute=0, second=0, microsecond=0) + + def test_add_window(self, sample_time_window_1): + """Test adding a window to the sequence.""" + sequence = TimeWindowSequence() + assert len(sequence) == 0 + + sequence.add_window(sample_time_window_1) + assert len(sequence) == 1 + assert sequence[0] == sample_time_window_1 + + def test_remove_window(self, sample_sequence, sample_time_window_1): + """Test removing a window from the sequence.""" + assert len(sample_sequence) == 2 + + removed = sample_sequence.remove_window(0) + assert removed == sample_time_window_1 + assert len(sample_sequence) == 1 + + def test_remove_window_invalid_index(self, sample_sequence): + """Test removing a window with invalid index raises IndexError.""" + with pytest.raises(IndexError): + sample_sequence.remove_window(10) + + def test_remove_window_from_empty_sequence(self): + """Test removing a window from empty sequence raises IndexError.""" + sequence = TimeWindowSequence() + with pytest.raises(IndexError): + sequence.remove_window(0) + + def test_clear_windows(self, sample_sequence): + """Test clearing all windows from the sequence.""" + assert len(sample_sequence) == 2 + + sample_sequence.clear_windows() + assert len(sample_sequence) == 0 + assert sample_sequence.windows == [] + + def test_sort_windows_by_start_time(self, reference_date): + """Test sorting windows by start time.""" + # Create windows in reverse chronological order + afternoon_window = TimeWindow(start_time=Time(14, 0, 0), duration=Duration(hours=2)) + morning_window = TimeWindow(start_time=Time(9, 0, 0), duration=Duration(hours=2)) + evening_window = TimeWindow(start_time=Time(18, 0, 0), duration=Duration(hours=2)) + + sequence = TimeWindowSequence(windows=[afternoon_window, morning_window, evening_window]) + sequence.sort_windows_by_start_time(reference_date) + + # Should now be sorted: morning, afternoon, evening + assert sequence[0] == morning_window + assert sequence[1] == afternoon_window + assert sequence[2] == evening_window + + def test_sort_windows_with_non_applicable_windows(self, monday_window, reference_date): + """Test sorting windows with some non-applicable windows.""" + daily_window = TimeWindow(start_time=Time(10, 0, 0), duration=Duration(hours=1)) + + sequence = TimeWindowSequence(windows=[monday_window, daily_window]) + sequence.sort_windows_by_start_time(reference_date) # Wednesday + + # Daily window should come first (applicable), Monday window last (not applicable) + assert sequence[0] == daily_window + assert sequence[1] == monday_window + + def test_sort_windows_empty_sequence(self, reference_date): + """Test sorting an empty sequence doesn't raise errors.""" + sequence = TimeWindowSequence() + sequence.sort_windows_by_start_time(reference_date) + assert len(sequence) == 0 + + def test_default_reference_date_handling(self, sample_sequence): + """Test that methods handle default reference date (today) correctly.""" + # These should not raise errors and should return reasonable values + assert isinstance(sample_sequence.can_fit_duration(Duration(hours=1)), bool) + assert sample_sequence.available_duration() is not None + assert isinstance(sample_sequence.get_applicable_windows(), list) + + def test_specific_date_window_functionality(self, specific_date_window): + """Test functionality with specific date restrictions.""" + sequence = TimeWindowSequence(windows=[specific_date_window]) + + # Should work on the specific date + specific_date = pendulum.parse("2025-01-15T12:00:00") + assert sequence.can_fit_duration(Duration(hours=1), specific_date) + + # Should not work on other dates + other_date = pendulum.parse("2025-01-16T12:00:00") + assert not sequence.can_fit_duration(Duration(hours=1), other_date) + + def test_edge_cases_with_zero_duration(self, sample_sequence, reference_date): + """Test edge cases with zero duration.""" + zero_duration = Duration() + + # Should be able to fit zero duration + assert sample_sequence.can_fit_duration(zero_duration, reference_date) + + # Should find start times for zero duration + earliest = sample_sequence.earliest_start_time(zero_duration, reference_date) + assert earliest is not None + + def test_overlapping_windows(self, reference_date): + """Test behavior with overlapping windows.""" + window1 = TimeWindow(start_time=Time(10, 0, 0), duration=Duration(hours=3)) + window2 = TimeWindow(start_time=Time(11, 0, 0), duration=Duration(hours=3)) + + sequence = TimeWindowSequence(windows=[window1, window2]) + + # Should handle overlapping windows correctly + test_time = reference_date.replace(hour=11, minute=30) + assert sequence.contains(test_time) + + # Total duration should be sum of both windows (even though they overlap) + total = sequence.available_duration(reference_date) + assert total == Duration(hours=6) + + def test_sequence_model_dump(self, sample_sequence_json): + """Test that model dump creates the correct json.""" + assert sample_sequence_json == json.loads(""" +{ + "windows": [ + { + "start_time": "09:00:00.000000", + "duration": "3 hours", + "day_of_week": null, + "date": null, + "locale": null + }, + { + "start_time": "14:00:00.000000", + "duration": "3 hours", + "day_of_week": null, + "date": null, + "locale": null + } + ] +}""") + + # ----------------------------- # to_datetime # ----------------------------- diff --git a/tests/test_elecpriceakkudoktor.py b/tests/test_elecpriceakkudoktor.py index 568777f..b01f53e 100644 --- a/tests/test_elecpriceakkudoktor.py +++ b/tests/test_elecpriceakkudoktor.py @@ -128,7 +128,7 @@ def test_update_data(mock_get, provider, sample_akkudoktor_1_json, cache_store): # Assert we get hours prioce values by resampling np_price_array = provider.key_to_array( key="elecprice_marketprice_wh", - start_datetime=provider.start_datetime, + start_datetime=provider.ems_start_datetime, end_datetime=provider.end_datetime, ) assert len(np_price_array) == provider.total_hours @@ -188,7 +188,7 @@ def test_key_to_array_resampling(provider): provider.update_data(force_update=True) array = provider.key_to_array( key="elecprice_marketprice_wh", - start_datetime=provider.start_datetime, + start_datetime=provider.ems_start_datetime, end_datetime=provider.end_datetime, ) assert isinstance(array, np.ndarray) @@ -204,7 +204,7 @@ def test_key_to_array_resampling(provider): def test_akkudoktor_development_forecast_data(provider): """Fetch data from real Akkudoktor server.""" # Preset, as this is usually done by update_data() - provider.start_datetime = to_datetime("2024-10-26 00:00:00") + provider.ems_start_datetime = to_datetime("2024-10-26 00:00:00") akkudoktor_data = provider._request_forecast() diff --git a/tests/test_elecpriceenergycharts.py b/tests/test_elecpriceenergycharts.py index 1f18c32..dacf905 100644 --- a/tests/test_elecpriceenergycharts.py +++ b/tests/test_elecpriceenergycharts.py @@ -125,7 +125,7 @@ def test_update_data(mock_get, provider, sample_energycharts_json, cache_store): # Assert we get hours prioce values by resampling np_price_array = provider.key_to_array( key="elecprice_marketprice_wh", - start_datetime=provider.start_datetime, + start_datetime=provider.ems_start_datetime, end_datetime=provider.end_datetime, ) assert len(np_price_array) == provider.total_hours @@ -182,7 +182,7 @@ def test_key_to_array_resampling(provider): provider.update_data(force_update=True) array = provider.key_to_array( key="elecprice_marketprice_wh", - start_datetime=provider.start_datetime, + start_datetime=provider.ems_start_datetime, end_datetime=provider.end_datetime, ) assert isinstance(array, np.ndarray) @@ -198,7 +198,7 @@ def test_key_to_array_resampling(provider): def test_akkudoktor_development_forecast_data(provider): """Fetch data from real Energy-Charts server.""" # Preset, as this is usually done by update_data() - provider.start_datetime = to_datetime("2024-10-26 00:00:00") + provider.ems_start_datetime = to_datetime("2024-10-26 00:00:00") energy_charts_data = provider._request_forecast() diff --git a/tests/test_elecpriceimport.py b/tests/test_elecpriceimport.py index 420f15e..6f44596 100644 --- a/tests/test_elecpriceimport.py +++ b/tests/test_elecpriceimport.py @@ -19,8 +19,10 @@ def provider(sample_import_1_json, config_eos): "elecprice": { "provider": "ElecPriceImport", "provider_settings": { - "import_file_path": str(FILE_TESTDATA_ELECPRICEIMPORT_1_JSON), - "import_json": json.dumps(sample_import_1_json), + "ElecPriceImport": { + "import_file_path": str(FILE_TESTDATA_ELECPRICEIMPORT_1_JSON), + "import_json": json.dumps(sample_import_1_json), + }, }, } } @@ -55,7 +57,9 @@ def test_invalid_provider(provider, config_eos): "elecprice": { "provider": "", "provider_settings": { - "import_file_path": str(FILE_TESTDATA_ELECPRICEIMPORT_1_JSON), + "ElecPriceImport": { + "import_file_path": str(FILE_TESTDATA_ELECPRICEIMPORT_1_JSON), + }, }, } } @@ -86,20 +90,20 @@ def test_import(provider, sample_import_1_json, start_datetime, from_file, confi ems_eos = get_ems() ems_eos.set_start_datetime(to_datetime(start_datetime, in_timezone="Europe/Berlin")) if from_file: - config_eos.elecprice.provider_settings.import_json = None - assert config_eos.elecprice.provider_settings.import_json is None + config_eos.elecprice.provider_settings.ElecPriceImport.import_json = None + assert config_eos.elecprice.provider_settings.ElecPriceImport.import_json is None else: - config_eos.elecprice.provider_settings.import_file_path = None - assert config_eos.elecprice.provider_settings.import_file_path is None + config_eos.elecprice.provider_settings.ElecPriceImport.import_file_path = None + assert config_eos.elecprice.provider_settings.ElecPriceImport.import_file_path is None provider.clear() # Call the method provider.update_data() # Assert: Verify the result is as expected - assert provider.start_datetime is not None + assert provider.ems_start_datetime is not None assert provider.total_hours is not None - assert compare_datetimes(provider.start_datetime, ems_eos.start_datetime).equal + assert compare_datetimes(provider.ems_start_datetime, ems_eos.start_datetime).equal values = sample_import_1_json["elecprice_marketprice_wh"] value_datetime_mapping = provider.import_datetimes(ems_eos.start_datetime, len(values)) for i, mapping in enumerate(value_datetime_mapping): diff --git a/tests/test_emplan.py b/tests/test_emplan.py new file mode 100644 index 0000000..f225e43 --- /dev/null +++ b/tests/test_emplan.py @@ -0,0 +1,235 @@ +from typing import Optional + +import pytest + +from akkudoktoreos.core.emplan import ( + BaseInstruction, + CommodityQuantity, + DDBCInstruction, + EnergyManagementPlan, + FRBCInstruction, + OMBCInstruction, + PEBCInstruction, + PEBCPowerEnvelope, + PEBCPowerEnvelopeElement, + PPBCEndInterruptionInstruction, + PPBCScheduleInstruction, + PPBCStartInterruptionInstruction, +) +from akkudoktoreos.utils.datetimeutil import Duration, to_datetime, to_duration + + +@pytest.fixture +def fixed_now(): + return to_datetime("2025-06-01T12:00:00") + + +class TestEnergyManagementPlan: + def test_add_instruction_and_time_range(self, fixed_now): + plan = EnergyManagementPlan( + id="plan-123", + generated_at=fixed_now, + instructions=[] + ) + instr1 = OMBCInstruction( + resource_id="dev-1", + execution_time=fixed_now, + operation_mode_id="mymode1", + operation_mode_factor=1.0, + ) + instr2 = OMBCInstruction( + resource_id="dev-2", + execution_time=fixed_now.add(minutes=5), + operation_mode_id="mymode1", + operation_mode_factor=1.0, + ) + plan.add_instruction(instr1) + plan.add_instruction(instr2) + + # Check that valid_from matches the earliest execution_time + assert plan.valid_from == fixed_now + + # instr2 has infinite duration so valid_until must be None + assert plan.valid_until is None + + assert plan.instructions == [instr1, instr2] + + def test_clear(self, fixed_now): + plan = EnergyManagementPlan( + id="plan-123", + generated_at=fixed_now, + instructions=[ + OMBCInstruction( + resource_id="dev-1", + execution_time=fixed_now, + operation_mode_id="mymode1", + operation_mode_factor=1.0, + ) + ], + ) + plan.clear() + assert plan.instructions == [] + assert plan.valid_until is None + assert plan.valid_from is not None + + def test_get_active_instructions(self, fixed_now): + instr1 = OMBCInstruction( + resource_id="dev-1", + execution_time=fixed_now.subtract(minutes=1), + operation_mode_id="mymode1", + operation_mode_factor=1.0, + ) + instr2 = OMBCInstruction( + resource_id="dev-2", + execution_time=fixed_now.add(minutes=1), + operation_mode_id="mymode1", + operation_mode_factor=1.0, + ) + instr3 = OMBCInstruction( + resource_id="dev-3", + execution_time=fixed_now.subtract(minutes=10), + operation_mode_id="mymode1", + operation_mode_factor=1.0, + ) + plan = EnergyManagementPlan( + id="plan-123", + generated_at=fixed_now, + instructions=[instr1, instr2, instr3], + ) + plan._update_time_range() + + active = plan.get_active_instructions(now=fixed_now) + ids = {i.resource_id for i in active} + assert ids == {"dev-1", "dev-3"} + + def test_get_next_instruction(self, fixed_now): + instr1 = OMBCInstruction( + resource_id="dev-1", + execution_time=fixed_now.subtract(minutes=1), + operation_mode_id="mymode1", + operation_mode_factor=1.0, + ) + instr2 = OMBCInstruction( + resource_id="dev-2", + execution_time=fixed_now.add(minutes=10), + operation_mode_id="mymode1", + operation_mode_factor=1.0, + ) + instr3 = OMBCInstruction( + resource_id="dev-3", + execution_time=fixed_now.add(minutes=5), + operation_mode_id="mymode1", + operation_mode_factor=1.0, + ) + plan = EnergyManagementPlan( + id="plan-123", + generated_at=fixed_now, + instructions=[instr1, instr2, instr3], + ) + plan._update_time_range() + + next_instr = plan.get_next_instruction(now=fixed_now) + assert next_instr is not None + assert next_instr.resource_id == "dev-3" + + def test_get_instructions_for_resource(self, fixed_now): + instr1 = OMBCInstruction( + resource_id="dev-1", + execution_time=fixed_now, + operation_mode_id="mymode1", + operation_mode_factor=1.0, + ) + instr2 = OMBCInstruction( + resource_id="dev-2", + execution_time=fixed_now, + operation_mode_id="mymode1", + operation_mode_factor=1.0, + ) + plan = EnergyManagementPlan( + id="plan-123", + generated_at=fixed_now, + instructions=[instr1, instr2], + ) + dev1_instructions = plan.get_instructions_for_resource("dev-1") + assert len(dev1_instructions) == 1 + assert dev1_instructions[0].resource_id == "dev-1" + + + def test_add_various_instructions(self, fixed_now): + plan = EnergyManagementPlan( + id="plan-123", + generated_at=fixed_now, + instructions=[] + ) + + instrs = [ + DDBCInstruction( + id="actuatorA@123", + execution_time=fixed_now, + actuator_id="actuatorA", + operation_mode_id="mode123", + operation_mode_factor=0.5, + ), + FRBCInstruction( + id="actuatorB@456", + execution_time=fixed_now.add(minutes=1), + actuator_id="actuatorB", + operation_mode_id="FRBC_Mode_1", + operation_mode_factor=1.0, + ), + OMBCInstruction( + id="controller@789", + execution_time=fixed_now.add(minutes=2), + operation_mode_id="OMBC_Mode_42", + operation_mode_factor=0.8, + ), + PPBCEndInterruptionInstruction( + id="end_int@001", + execution_time=fixed_now.add(minutes=3), + power_profile_id="profile-123", + sequence_container_id="container-456", + power_sequence_id="seq-789", + ), + PPBCStartInterruptionInstruction( + id="start_int@002", + execution_time=fixed_now.add(minutes=4), + power_profile_id="profile-321", + sequence_container_id="container-654", + power_sequence_id="seq-987", + ), + PPBCScheduleInstruction( + id="schedule@003", + execution_time=fixed_now.add(minutes=5), + power_profile_id="profile-999", + sequence_container_id="container-888", + power_sequence_id="seq-777", + ), + PEBCInstruction( + id="pebc@004", + execution_time=fixed_now.add(minutes=6), + power_constraints_id="pc-123", + power_envelopes=[ + PEBCPowerEnvelope( + id="pebcpe@1234", + commodity_quantity=CommodityQuantity.ELECTRIC_POWER_L1, + power_envelope_elements = [ + PEBCPowerEnvelopeElement( + duration = to_duration(10), + upper_limit = 1010.0, + lower_limit = 990.0, + ), + ], + ), + ], + ), + ] + + for instr in instrs: + plan.add_instruction(instr) + + assert len(plan.instructions) == len(instrs) + # Check that get_instructions_for_device returns the right instructions + assert any( + instr for instr in plan.get_instructions_for_resource("actuatorA") + if isinstance(instr, DDBCInstruction) + ) diff --git a/tests/test_eosdashserver.py b/tests/test_eosdashserver.py index 505f3ec..fbb4564 100644 --- a/tests/test_eosdashserver.py +++ b/tests/test_eosdashserver.py @@ -5,19 +5,16 @@ import requests class TestEOSDash: - def test_eosdash_started(self, server_setup_for_class, is_system_test): - """Test the EOSdash server is started by EOS server.""" - server = server_setup_for_class["server"] - eosdash_server = server_setup_for_class["eosdash_server"] - eos_dir = server_setup_for_class["eos_dir"] - timeout = server_setup_for_class["timeout"] - # Assure EOSdash is up + def _assert_server_alive(self, base_url: str, timeout: int): + """Poll the /eosdash/health endpoint until it's alive or timeout reached.""" startup = False error = "" - for retries in range(int(timeout / 3)): + result = None + + for _ in range(int(timeout / 3)): try: - result = requests.get(f"{eosdash_server}/eosdash/health", timeout=2) + result = requests.get(f"{base_url}/eosdash/health", timeout=2) if result.status_code == HTTPStatus.OK: startup = True break @@ -25,27 +22,19 @@ class TestEOSDash: except Exception as ex: error = str(ex) time.sleep(3) - assert startup, f"Connection to {eosdash_server}/eosdash/health failed: {error}" + + assert startup, f"Connection to {base_url}/eosdash/health failed: {error}" + assert result is not None assert result.json()["status"] == "alive" + def test_eosdash_started(self, server_setup_for_class, is_system_test): + """Test the EOSdash server is started by EOS server.""" + eosdash_server = server_setup_for_class["eosdash_server"] + timeout = server_setup_for_class["timeout"] + self._assert_server_alive(eosdash_server, timeout) + def test_eosdash_proxied_by_eos(self, server_setup_for_class, is_system_test): """Test the EOSdash server proxied by EOS server.""" server = server_setup_for_class["server"] - eos_dir = server_setup_for_class["eos_dir"] timeout = server_setup_for_class["timeout"] - - # Assure EOSdash is up - startup = False - error = "" - for retries in range(int(timeout / 3)): - try: - result = requests.get(f"{server}/eosdash/health", timeout=2) - if result.status_code == HTTPStatus.OK: - startup = True - break - error = f"{result.status_code}, {str(result.content)}" - except Exception as ex: - error = str(ex) - time.sleep(3) - assert startup, f"Connection to {server}/eosdash/health failed: {error}" - assert result.json()["status"] == "alive" + self._assert_server_alive(server, timeout) diff --git a/tests/test_feedintarifffixed.py b/tests/test_feedintarifffixed.py new file mode 100644 index 0000000..a8e24e8 --- /dev/null +++ b/tests/test_feedintarifffixed.py @@ -0,0 +1,62 @@ +import json +from pathlib import Path + +import pytest + +from akkudoktoreos.core.ems import get_ems +from akkudoktoreos.prediction.feedintarifffixed import FeedInTariffFixed +from akkudoktoreos.utils.datetimeutil import compare_datetimes, to_datetime + +DIR_TESTDATA = Path(__file__).absolute().parent.joinpath("testdata") + + +@pytest.fixture +def provider(config_eos): + """Fixture to create a ElecPriceProvider instance.""" + settings = { + "feedintariff": { + "provider": "FeedInTariffFixed", + "provider_settings": { + "FeedInTariffFixed": { + "feed_in_tariff_kwh": 0.078, + }, + }, + } + } + config_eos.merge_settings_from_dict(settings) + assert config_eos.feedintariff.provider == "FeedInTariffFixed" + provider = FeedInTariffFixed() + assert provider.enabled() + return provider + + +# ------------------------------------------------ +# General forecast +# ------------------------------------------------ + + +def test_singleton_instance(provider): + """Test that ElecPriceForecast behaves as a singleton.""" + another_instance = FeedInTariffFixed() + assert provider is another_instance + + +def test_invalid_provider(provider, config_eos): + """Test requesting an unsupported provider.""" + settings = { + "feedintariff": { + "provider": "", + "provider_settings": { + "FeedInTariffFixed": { + "feed_in_tariff_kwh": 0.078, + }, + }, + } + } + with pytest.raises(ValueError, match="not a valid feed in tariff provider"): + config_eos.merge_settings_from_dict(settings) + + +# ------------------------------------------------ +# Fixed feed in tariv values +# ------------------------------------------------ diff --git a/tests/test_geneticoptimize.py b/tests/test_geneticoptimize.py new file mode 100644 index 0000000..4301adc --- /dev/null +++ b/tests/test_geneticoptimize.py @@ -0,0 +1,150 @@ +import json +from pathlib import Path +from typing import Any +from unittest.mock import patch + +import pytest + +from akkudoktoreos.config.config import ConfigEOS +from akkudoktoreos.core.cache import CacheEnergyManagementStore +from akkudoktoreos.core.ems import get_ems +from akkudoktoreos.optimization.genetic.genetic import GeneticOptimization +from akkudoktoreos.optimization.genetic.geneticparams import ( + GeneticOptimizationParameters, +) +from akkudoktoreos.optimization.genetic.geneticsolution import GeneticSolution +from akkudoktoreos.utils.datetimeutil import to_datetime +from akkudoktoreos.utils.visualize import ( + prepare_visualize, # Import the new prepare_visualize +) + +ems_eos = get_ems() + +DIR_TESTDATA = Path(__file__).parent / "testdata" + + +def compare_dict(actual: dict[str, Any], expected: dict[str, Any]): + assert set(actual) == set(expected) + + for key, value in expected.items(): + if isinstance(value, dict): + assert isinstance(actual[key], dict) + compare_dict(actual[key], value) + elif isinstance(value, list): + assert isinstance(actual[key], list) + assert actual[key] == pytest.approx(value) + else: + assert actual[key] == pytest.approx(value) + + +@pytest.mark.parametrize( + "fn_in, fn_out, ngen", + [ + ("optimize_input_1.json", "optimize_result_1.json", 3), + ("optimize_input_2.json", "optimize_result_2.json", 3), + ("optimize_input_2.json", "optimize_result_2_full.json", 400), + ], +) +def test_optimize( + fn_in: str, + fn_out: str, + ngen: int, + config_eos: ConfigEOS, + is_full_run: bool, +): + """Test optimierung_ems.""" + # Test parameters + fixed_start_hour = 10 + fixed_seed = 42 + + # Assure configuration holds the correct values + config_eos.merge_settings_from_dict( + { + "prediction": { + "hours": 48 + }, + "optimization": { + "horizon_hours": 48, + "genetic": { + "individuals": 300, + "generations": 10, + "penalties": { + "ev_soc_miss": 10 + } + } + }, + "devices": { + "max_electric_vehicles": 1, + "electric_vehicles": [ + { + "charge_rates": [0.0, 0.375, 0.5, 0.625, 0.75, 0.875, 1.0], + } + ], + } + } + ) + + # Load input and output data + file = DIR_TESTDATA / fn_in + with file.open("r") as f_in: + input_data = GeneticOptimizationParameters(**json.load(f_in)) + + file = DIR_TESTDATA / fn_out + # In case a new test case is added, we don't want to fail here, so the new output is written + # to disk before + try: + with file.open("r") as f_out: + expected_data = json.load(f_out) + expected_result = GeneticSolution(**expected_data) + except FileNotFoundError: + pass + + # Fake energy management run start datetime + ems_eos.set_start_datetime(to_datetime().set(hour=fixed_start_hour)) + + # Throw away any cached results of the last energy management run. + CacheEnergyManagementStore().clear() + + genetic_optimization = GeneticOptimization(fixed_seed=fixed_seed) + + # Activate with pytest --full-run + if ngen > 10 and not is_full_run: + pytest.skip() + + visualize_filename = str((DIR_TESTDATA / f"new_{fn_out}").with_suffix(".pdf")) + + with patch( + "akkudoktoreos.utils.visualize.prepare_visualize", + side_effect=lambda parameters, results, *args, **kwargs: prepare_visualize( + parameters, results, filename=visualize_filename, **kwargs + ), + ) as prepare_visualize_patch: + # Call the optimization function + genetic_solution = genetic_optimization.optimierung_ems( + parameters=input_data, start_hour=fixed_start_hour, ngen=ngen + ) + # The function creates a visualization result PDF as a side-effect. + prepare_visualize_patch.assert_called_once() + assert Path(visualize_filename).exists() + + # Write test output to file, so we can take it as new data on intended change + TESTDATA_FILE = DIR_TESTDATA / f"new_{fn_out}" + with TESTDATA_FILE.open("w", encoding="utf-8", newline="\n") as f_out: + f_out.write(genetic_solution.model_dump_json(indent=4, exclude_unset=True)) + + assert genetic_solution.result.Gesamtbilanz_Euro == pytest.approx( + expected_result.result.Gesamtbilanz_Euro + ) + + # Assert that the output contains all expected entries. + # This does not assert that the optimization always gives the same result! + # Reproducibility and mathematical accuracy should be tested on the level of individual components. + compare_dict(genetic_solution.model_dump(), expected_result.model_dump()) + + # Check the correct generic optimization solution is created + optimization_solution = genetic_solution.optimization_solution() + # @TODO + + # Check the correct generic energy management plan is created + plan = genetic_solution.energy_management_plan() + # @TODO diff --git a/tests/test_class_ems.py b/tests/test_geneticsimulation.py similarity index 80% rename from tests/test_class_ems.py rename to tests/test_geneticsimulation.py index bbb7c44..57c1778 100644 --- a/tests/test_class_ems.py +++ b/tests/test_geneticsimulation.py @@ -1,32 +1,41 @@ import numpy as np import pytest -from akkudoktoreos.core.ems import ( - EnergyManagement, - EnergyManagementParameters, - SimulationResult, - get_ems, -) -from akkudoktoreos.devices.battery import ( - Battery, +from akkudoktoreos.devices.genetic.battery import Battery +from akkudoktoreos.devices.genetic.homeappliance import HomeAppliance +from akkudoktoreos.devices.genetic.inverter import Inverter +from akkudoktoreos.optimization.genetic.genetic import GeneticSimulation +from akkudoktoreos.optimization.genetic.geneticdevices import ( ElectricVehicleParameters, + HomeApplianceParameters, + InverterParameters, SolarPanelBatteryParameters, ) -from akkudoktoreos.devices.generic import HomeAppliance, HomeApplianceParameters -from akkudoktoreos.devices.inverter import Inverter, InverterParameters +from akkudoktoreos.optimization.genetic.geneticparams import ( + GeneticEnergyManagementParameters, + GeneticOptimizationParameters, +) +from akkudoktoreos.optimization.genetic.geneticsolution import GeneticSimulationResult +from akkudoktoreos.utils.datetimeutil import ( + TimeWindow, + TimeWindowSequence, + to_duration, + to_time, +) start_hour = 1 # Example initialization of necessary components @pytest.fixture -def create_ems_instance(devices_eos, config_eos) -> EnergyManagement: +def genetic_simulation(config_eos) -> GeneticSimulation: """Fixture to create an EnergyManagement instance with given test parameters.""" # Assure configuration holds the correct values config_eos.merge_settings_from_dict( {"prediction": {"hours": 48}, "optimization": {"hours": 24}} ) assert config_eos.prediction.hours == 48 + assert config_eos.optimization.horizon_hours == 24 # Initialize the battery and the inverter akku = Battery( @@ -35,15 +44,15 @@ def create_ems_instance(devices_eos, config_eos) -> EnergyManagement: capacity_wh=5000, initial_soc_percentage=80, min_soc_percentage=10, - ) + ), + prediction_hours = config_eos.prediction.hours, ) akku.reset() - devices_eos.add_device(akku) inverter = Inverter( - InverterParameters(device_id="inverter1", max_power_wh=10000, battery_id=akku.device_id) + InverterParameters(device_id="inverter1", max_power_wh=10000, battery_id=akku.parameters.device_id), + battery = akku, ) - devices_eos.add_device(inverter) # Household device (currently not used, set to None) home_appliance = HomeAppliance( @@ -51,21 +60,21 @@ def create_ems_instance(devices_eos, config_eos) -> EnergyManagement: device_id="dishwasher1", consumption_wh=2000, duration_h=2, + time_windows=None, ), + optimization_hours = config_eos.optimization.horizon_hours, + prediction_hours = config_eos.prediction.hours, ) home_appliance.set_starting_time(2) - devices_eos.add_device(home_appliance) # Example initialization of electric car battery eauto = Battery( ElectricVehicleParameters( device_id="ev1", capacity_wh=26400, initial_soc_percentage=10, min_soc_percentage=10 ), + prediction_hours = config_eos.prediction.hours, ) eauto.set_charge_per_hour(np.full(config_eos.prediction.hours, 1)) - devices_eos.add_device(eauto) - - devices_eos.post_setup() # Parameters based on previous example data pv_prognose_wh = [ @@ -225,39 +234,41 @@ def create_ems_instance(devices_eos, config_eos) -> EnergyManagement: ] # Initialize the energy management system with the respective parameters - ems = get_ems() - ems.set_parameters( - EnergyManagementParameters( + simulation = GeneticSimulation() + simulation.prepare( + GeneticEnergyManagementParameters( pv_prognose_wh=pv_prognose_wh, strompreis_euro_pro_wh=strompreis_euro_pro_wh, einspeiseverguetung_euro_pro_wh=einspeiseverguetung_euro_pro_wh, preis_euro_pro_wh_akku=preis_euro_pro_wh_akku, gesamtlast=gesamtlast, ), + optimization_hours = config_eos.optimization.horizon_hours, + prediction_hours = config_eos.prediction.hours, inverter=inverter, ev=eauto, home_appliance=home_appliance, ) - return ems + return simulation -def test_simulation(create_ems_instance): +def test_simulation(genetic_simulation): """Test the EnergyManagement simulation method.""" - ems = create_ems_instance + simulation = genetic_simulation # Simulate starting from hour 1 (this value can be adjusted) - result = ems.simulate(start_hour=start_hour) + result = simulation.simulate(start_hour=start_hour) # visualisiere_ergebnisse( - # ems.gesamtlast, - # ems.pv_prognose_wh, - # ems.strompreis_euro_pro_wh, + # simulation.gesamtlast, + # simulation.pv_prognose_wh, + # simulation.strompreis_euro_pro_wh, # result, - # ems.akku.discharge_array+ems.akku.charge_array, + # simulation.akku.discharge_array+simulation.akku.charge_array, # None, - # ems.pv_prognose_wh, + # simulation.pv_prognose_wh, # start_hour, # 48, # np.full(48, 0.0), @@ -278,7 +289,7 @@ def test_simulation(create_ems_instance): # Check that the result is a dictionary assert isinstance(result, dict), "Result should be a dictionary." - assert SimulationResult(**result) is not None + assert GeneticSimulationResult(**result) is not None # Check the length of the main arrays assert len(result["Last_Wh_pro_Stunde"]) == 47, ( @@ -341,8 +352,8 @@ def test_simulation(create_ems_instance): ) # Check home appliances - assert sum(ems.home_appliance.get_load_curve()) == 2000, ( - "The sum of 'ems.home_appliance.get_load_curve()' should be 2000." + assert sum(simulation.home_appliance.get_load_curve()) == 2000, ( + "The sum of 'simulation.home_appliance.get_load_curve()' should be 2000." ) assert ( diff --git a/tests/test_class_ems_2.py b/tests/test_geneticsimulation2.py similarity index 62% rename from tests/test_class_ems_2.py rename to tests/test_geneticsimulation2.py index f141fab..a5749d7 100644 --- a/tests/test_class_ems_2.py +++ b/tests/test_geneticsimulation2.py @@ -1,46 +1,58 @@ import numpy as np import pytest -from akkudoktoreos.core.ems import ( - EnergyManagement, - EnergyManagementParameters, - SimulationResult, - get_ems, -) -from akkudoktoreos.devices.battery import ( - Battery, +from akkudoktoreos.devices.genetic.battery import Battery +from akkudoktoreos.devices.genetic.homeappliance import HomeAppliance +from akkudoktoreos.devices.genetic.inverter import Inverter +from akkudoktoreos.optimization.genetic.genetic import GeneticSimulation +from akkudoktoreos.optimization.genetic.geneticdevices import ( ElectricVehicleParameters, + HomeApplianceParameters, + InverterParameters, SolarPanelBatteryParameters, ) -from akkudoktoreos.devices.generic import HomeAppliance, HomeApplianceParameters -from akkudoktoreos.devices.inverter import Inverter, InverterParameters +from akkudoktoreos.optimization.genetic.geneticparams import ( + GeneticEnergyManagementParameters, + GeneticOptimizationParameters, +) +from akkudoktoreos.optimization.genetic.geneticsolution import GeneticSimulationResult +from akkudoktoreos.utils.datetimeutil import ( + TimeWindow, + TimeWindowSequence, + to_duration, + to_time, +) start_hour = 0 # Example initialization of necessary components @pytest.fixture -def create_ems_instance(devices_eos, config_eos) -> EnergyManagement: +def genetic_simulation_2(config_eos) -> GeneticSimulation: """Fixture to create an EnergyManagement instance with given test parameters.""" # Assure configuration holds the correct values config_eos.merge_settings_from_dict( {"prediction": {"hours": 48}, "optimization": {"hours": 24}} ) assert config_eos.prediction.hours == 48 + assert config_eos.optimization.horizon_hours == 24 # Initialize the battery and the inverter akku = Battery( SolarPanelBatteryParameters( - device_id="pv1", capacity_wh=5000, initial_soc_percentage=80, min_soc_percentage=10 - ) + device_id="battery1", + capacity_wh=5000, + initial_soc_percentage=80, + min_soc_percentage=10, + ), + prediction_hours = config_eos.prediction.hours, ) akku.reset() - devices_eos.add_device(akku) inverter = Inverter( - InverterParameters(device_id="iv1", max_power_wh=10000, battery_id=akku.device_id) + InverterParameters(device_id="inverter1", max_power_wh=10000, battery_id=akku.parameters.device_id), + battery = akku, ) - devices_eos.add_device(inverter) # Household device (currently not used, set to None) home_appliance = HomeAppliance( @@ -48,20 +60,20 @@ def create_ems_instance(devices_eos, config_eos) -> EnergyManagement: device_id="dishwasher1", consumption_wh=2000, duration_h=2, - ) + time_windows=None, + ), + optimization_hours = config_eos.optimization.horizon_hours, + prediction_hours = config_eos.prediction.hours, ) home_appliance.set_starting_time(2) - devices_eos.add_device(home_appliance) # Example initialization of electric car battery eauto = Battery( ElectricVehicleParameters( - device_id="ev1", capacity_wh=26400, initial_soc_percentage=100, min_soc_percentage=100 + device_id="ev1", capacity_wh=26400, initial_soc_percentage=10, min_soc_percentage=10 ), + prediction_hours = config_eos.prediction.hours, ) - devices_eos.add_device(eauto) - - devices_eos.post_setup() # Parameters based on previous example data pv_prognose_wh = [0.0] * config_eos.prediction.hours @@ -128,15 +140,17 @@ def create_ems_instance(devices_eos, config_eos) -> EnergyManagement: ] # Initialize the energy management system with the respective parameters - ems = get_ems() - ems.set_parameters( - EnergyManagementParameters( + simulation = GeneticSimulation() + simulation.prepare( + GeneticEnergyManagementParameters( pv_prognose_wh=pv_prognose_wh, strompreis_euro_pro_wh=strompreis_euro_pro_wh, einspeiseverguetung_euro_pro_wh=einspeiseverguetung_euro_pro_wh, preis_euro_pro_wh_akku=preis_euro_pro_wh_akku, gesamtlast=gesamtlast, ), + optimization_hours = config_eos.optimization.horizon_hours, + prediction_hours = config_eos.prediction.hours, inverter=inverter, ev=eauto, home_appliance=home_appliance, @@ -144,30 +158,30 @@ def create_ems_instance(devices_eos, config_eos) -> EnergyManagement: ac = np.full(config_eos.prediction.hours, 0.0) ac[20] = 1 - ems.set_akku_ac_charge_hours(ac) + simulation.set_akku_ac_charge_hours(ac) dc = np.full(config_eos.prediction.hours, 0.0) dc[11] = 1 - ems.set_akku_dc_charge_hours(dc) + simulation.set_akku_dc_charge_hours(dc) - return ems + return simulation -def test_simulation(create_ems_instance): +def test_simulation(genetic_simulation_2): """Test the EnergyManagement simulation method.""" - ems = create_ems_instance + simulation = genetic_simulation_2 # Simulate starting from hour 0 (this value can be adjusted) - result = ems.simulate(start_hour=start_hour) + result = simulation.simulate(start_hour=start_hour) # --- Pls do not remove! --- # visualisiere_ergebnisse( - # ems.gesamtlast, - # ems.pv_prognose_wh, - # ems.strompreis_euro_pro_wh, + # simulation.gesamtlast, + # simulation.pv_prognose_wh, + # simulation.strompreis_euro_pro_wh, # result, - # ems.akku.discharge_array+ems.akku.charge_array, + # simulation.akku.discharge_array+simulation.akku.charge_array, # None, - # ems.pv_prognose_wh, + # simulation.pv_prognose_wh, # start_hour, # 48, # np.full(48, 0.0), @@ -178,7 +192,7 @@ def test_simulation(create_ems_instance): # Assertions to validate results assert result is not None, "Result should not be None" assert isinstance(result, dict), "Result should be a dictionary" - assert SimulationResult(**result) is not None + assert GeneticSimulationResult(**result) is not None assert "Last_Wh_pro_Stunde" in result, "Result should contain 'Last_Wh_pro_Stunde'" """ @@ -253,73 +267,64 @@ def test_simulation(create_ems_instance): print("All tests passed successfully.") -def test_set_parameters(create_ems_instance): +def test_set_parameters(genetic_simulation_2): """Test the set_parameters method of EnergyManagement.""" - ems = create_ems_instance + simulation = genetic_simulation_2 # Check if parameters are set correctly - assert ems.load_energy_array is not None, "load_energy_array should not be None" - assert ems.pv_prediction_wh is not None, "pv_prediction_wh should not be None" - assert ems.elect_price_hourly is not None, "elect_price_hourly should not be None" - assert ems.elect_revenue_per_hour_arr is not None, ( + assert simulation.load_energy_array is not None, "load_energy_array should not be None" + assert simulation.pv_prediction_wh is not None, "pv_prediction_wh should not be None" + assert simulation.elect_price_hourly is not None, "elect_price_hourly should not be None" + assert simulation.elect_revenue_per_hour_arr is not None, ( "elect_revenue_per_hour_arr should not be None" ) -def test_set_akku_discharge_hours(create_ems_instance): +def test_set_akku_discharge_hours(genetic_simulation_2): """Test the set_akku_discharge_hours method of EnergyManagement.""" - ems = create_ems_instance - discharge_hours = np.full(ems.config.prediction.hours, 1.0) - ems.set_akku_discharge_hours(discharge_hours) - assert np.array_equal(ems.battery.discharge_array, discharge_hours), ( + simulation = genetic_simulation_2 + discharge_hours = np.full(simulation.prediction_hours, 1.0) + simulation.set_akku_discharge_hours(discharge_hours) + assert np.array_equal(simulation.battery.discharge_array, discharge_hours), ( "Discharge hours should be set correctly" ) -def test_set_akku_ac_charge_hours(create_ems_instance): +def test_set_akku_ac_charge_hours(genetic_simulation_2): """Test the set_akku_ac_charge_hours method of EnergyManagement.""" - ems = create_ems_instance - ac_charge_hours = np.full(ems.config.prediction.hours, 1.0) - ems.set_akku_ac_charge_hours(ac_charge_hours) - assert np.array_equal(ems.ac_charge_hours, ac_charge_hours), ( + simulation = genetic_simulation_2 + ac_charge_hours = np.full(simulation.prediction_hours, 1.0) + simulation.set_akku_ac_charge_hours(ac_charge_hours) + assert np.array_equal(simulation.ac_charge_hours, ac_charge_hours), ( "AC charge hours should be set correctly" ) -def test_set_akku_dc_charge_hours(create_ems_instance): +def test_set_akku_dc_charge_hours(genetic_simulation_2): """Test the set_akku_dc_charge_hours method of EnergyManagement.""" - ems = create_ems_instance - dc_charge_hours = np.full(ems.config.prediction.hours, 1.0) - ems.set_akku_dc_charge_hours(dc_charge_hours) - assert np.array_equal(ems.dc_charge_hours, dc_charge_hours), ( + simulation = genetic_simulation_2 + dc_charge_hours = np.full(simulation.prediction_hours, 1.0) + simulation.set_akku_dc_charge_hours(dc_charge_hours) + assert np.array_equal(simulation.dc_charge_hours, dc_charge_hours), ( "DC charge hours should be set correctly" ) -def test_set_ev_charge_hours(create_ems_instance): +def test_set_ev_charge_hours(genetic_simulation_2): """Test the set_ev_charge_hours method of EnergyManagement.""" - ems = create_ems_instance - ev_charge_hours = np.full(ems.config.prediction.hours, 1.0) - ems.set_ev_charge_hours(ev_charge_hours) - assert np.array_equal(ems.ev_charge_hours, ev_charge_hours), ( + simulation = genetic_simulation_2 + ev_charge_hours = np.full(simulation.prediction_hours, 1.0) + simulation.set_ev_charge_hours(ev_charge_hours) + assert np.array_equal(simulation.ev_charge_hours, ev_charge_hours), ( "EV charge hours should be set correctly" ) -def test_reset(create_ems_instance): +def test_reset(genetic_simulation_2): """Test the reset method of EnergyManagement.""" - ems = create_ems_instance - ems.reset() - assert ems.ev.current_soc_percentage() == 100, "EV SOC should be reset to initial value" - assert ems.battery.current_soc_percentage() == 80, ( + simulation = genetic_simulation_2 + simulation.reset() + assert simulation.ev.current_soc_percentage() == simulation.ev.parameters.initial_soc_percentage, "EV SOC should be reset to initial value" + assert simulation.battery.current_soc_percentage() == simulation.battery.parameters.initial_soc_percentage, ( "Battery SOC should be reset to initial value" ) - - -def test_simulate_start_now(create_ems_instance): - """Test the simulate_start_now method of EnergyManagement.""" - ems = create_ems_instance - result = ems.simulate_start_now() - assert result is not None, "Result should not be None" - assert isinstance(result, dict), "Result should be a dictionary" - assert "Last_Wh_pro_Stunde" in result, "Result should contain 'Last_Wh_pro_Stunde'" diff --git a/tests/test_heatpump.py b/tests/test_heatpump.py index 153e707..79f53f7 100644 --- a/tests/test_heatpump.py +++ b/tests/test_heatpump.py @@ -1,6 +1,6 @@ import pytest -from akkudoktoreos.devices.heatpump import Heatpump +from akkudoktoreos.devices.genetic.heatpump import Heatpump @pytest.fixture(scope="function") diff --git a/tests/test_inverter.py b/tests/test_inverter.py index 7b91788..ff82d43 100644 --- a/tests/test_inverter.py +++ b/tests/test_inverter.py @@ -2,7 +2,7 @@ from unittest.mock import Mock, patch import pytest -from akkudoktoreos.devices.inverter import Inverter, InverterParameters +from akkudoktoreos.devices.genetic.inverter import Inverter, InverterParameters @pytest.fixture @@ -10,26 +10,24 @@ def mock_battery() -> Mock: mock_battery = Mock() mock_battery.charge_energy = Mock(return_value=(0.0, 0.0)) mock_battery.discharge_energy = Mock(return_value=(0.0, 0.0)) - mock_battery.device_id = "battery1" + mock_battery.parameters.device_id = "battery1" return mock_battery @pytest.fixture -def inverter(mock_battery, devices_eos) -> Inverter: - devices_eos.add_device(mock_battery) +def inverter(mock_battery) -> Inverter: mock_self_consumption_predictor = Mock() mock_self_consumption_predictor.calculate_self_consumption.return_value = 1.0 with patch( - "akkudoktoreos.devices.inverter.get_eos_load_interpolator", + "akkudoktoreos.devices.genetic.inverter.get_eos_load_interpolator", return_value=mock_self_consumption_predictor, ): iv = Inverter( InverterParameters( - device_id="iv1", max_power_wh=500.0, battery_id=mock_battery.device_id + device_id="iv1", max_power_wh=500.0, battery_id=mock_battery.parameters.device_id ), + battery = mock_battery ) - devices_eos.add_device(iv) - devices_eos.post_setup() return iv diff --git a/tests/test_loadakkudoktor.py b/tests/test_loadakkudoktor.py index b5fef2c..4b31ccb 100644 --- a/tests/test_loadakkudoktor.py +++ b/tests/test_loadakkudoktor.py @@ -20,14 +20,18 @@ def provider(config_eos): "load": { "provider": "LoadAkkudoktor", "provider_settings": { - "load_name": "Akkudoktor Profile", - "loadakkudoktor_year_energy": "1000", + "LoadAkkudoktor": { + "loadakkudoktor_year_energy": "1000", + }, }, + }, + "measurement": { + "load_emr_keys": ["load0_mr", "load1_mr"] } } config_eos.merge_settings_from_dict(settings) assert config_eos.load.provider == "LoadAkkudoktor" - assert config_eos.load.provider_settings.loadakkudoktor_year_energy == 1000 + assert config_eos.load.provider_settings.LoadAkkudoktor.loadakkudoktor_year_energy == 1000 return LoadAkkudoktor() diff --git a/tests/test_loadvrm.py b/tests/test_loadvrm.py index eca4523..d7d42de 100644 --- a/tests/test_loadvrm.py +++ b/tests/test_loadvrm.py @@ -19,8 +19,10 @@ def load_vrm_instance(config_eos): "load": { "provider": "LoadVrm", "provider_settings": { - "load_vrm_token": "dummy-token", - "load_vrm_idsite": 12345 + "LoadVrm": { + "load_vrm_token": "dummy-token", + "load_vrm_idsite": 12345, + }, } } } diff --git a/tests/test_measurement.py b/tests/test_measurement.py index 2c5c615..ef8148c 100644 --- a/tests/test_measurement.py +++ b/tests/test_measurement.py @@ -10,217 +10,390 @@ from akkudoktoreos.measurement.measurement import ( ) -@pytest.fixture -def measurement_eos(): - """Fixture to create a Measurement instance.""" - measurement = get_measurement() - measurement.records = [ - MeasurementDataRecord( +class TestMeasurementDataRecord: + """Test suite for the MeasurementDataRecord class. + + Ensuring that both dictionary-like and attribute-style access work correctly for fields and + configured measurements. + """ + + @pytest.fixture + def sample_config(self, config_eos): + """Fixture to configure the measurement keys on the global config.""" + config_eos.measurement.load_emr_keys = ["dish_washer_mr", "temp"] + config_eos.measurement.pv_production_emr_keys = ["solar_power"] + return config_eos + + @pytest.fixture + def record(self, sample_config): + """Fixture to create a sample MeasurementDataRecord with some measurements set.""" + rec = MeasurementDataRecord(date_time=None) + rec.configured_data = {"dish_washer_mr": 123.0, "solar_power": 456.0} + return rec + + def test_record_keys_includes_measurement_keys(self, record): + """Ensure record_keys includes all configured measurement keys.""" + assert set(record.record_keys()) >= set(record.config.measurement.keys) + + def test_record_keys_writable_includes_measurement_keys(self, record): + """Ensure record_keys_writable includes all configured measurement keys.""" + assert set(record.record_keys_writable()) >= set(record.config.measurement.keys) + + def test_getitem_existing_field(self, record): + """Test that __getitem__ returns correct value for existing native field.""" + record.date_time = "2024-01-01T00:00:00+00:00" + assert record["date_time"] is not None + + def test_getitem_existing_measurement(self, record): + """Test that __getitem__ retrieves existing measurement values.""" + assert record["dish_washer_mr"] == 123.0 + assert record["solar_power"] == 456.0 + + def test_getitem_missing_measurement_returns_none(self, record): + """Test that __getitem__ returns None for missing but known measurement keys.""" + assert record["temp"] is None + + def test_getitem_raises_keyerror(self, record): + """Test that __getitem__ raises KeyError for completely unknown keys.""" + with pytest.raises(KeyError): + _ = record["nonexistent"] + + def test_setitem_field(self, record): + """Test setting a native field using __setitem__.""" + record["date_time"] = "2025-01-01T12:00:00+00:00" + assert str(record.date_time).startswith("2025-01-01") + + def test_setitem_measurement(self, record): + """Test setting a known measurement key using __setitem__.""" + record["temp"] = 25.5 + assert record["temp"] == 25.5 + + def test_setitem_invalid_key_raises(self, record): + """Test that __setitem__ raises KeyError for unknown keys.""" + with pytest.raises(KeyError): + record["unknown_key"] = 123 + + def test_delitem_field(self, record): + """Test deleting a native field using __delitem__.""" + record["date_time"] = "2025-01-01T12:00:00+00:00" + del record["date_time"] + assert record.date_time is None + + def test_delitem_measurement(self, record): + """Test deleting a known measurement key using __delitem__.""" + del record["solar_power"] + assert record["solar_power"] is None + + def test_delitem_unknown_raises(self, record): + """Test that __delitem__ raises KeyError for unknown keys.""" + with pytest.raises(KeyError): + del record["nonexistent"] + + def test_attribute_get_existing_field(self, record): + """Test accessing a native field via attribute.""" + record.date_time = "2025-01-01T12:00:00+00:00" + assert record.date_time is not None + + def test_attribute_get_existing_measurement(self, record): + """Test accessing an existing measurement via attribute.""" + assert record.dish_washer_mr == 123.0 + + def test_attribute_get_missing_measurement(self, record): + """Test accessing a missing but known measurement returns None.""" + assert record.temp is None + + def test_attribute_get_invalid_raises(self, record): + """Test accessing an unknown attribute raises AttributeError.""" + with pytest.raises(AttributeError): + _ = record.nonexistent + + def test_attribute_set_existing_field(self, record): + """Test setting a native field via attribute.""" + record.date_time = "2025-06-25T12:00:00+00:00" + assert record.date_time is not None + + def test_attribute_set_existing_measurement(self, record): + """Test setting a known measurement key via attribute.""" + record.temp = 99.9 + assert record["temp"] == 99.9 + + def test_attribute_set_invalid_raises(self, record): + """Test setting an unknown attribute raises AttributeError.""" + with pytest.raises(AttributeError): + record.invalid = 123 + + def test_delattr_field(self, record): + """Test deleting a native field via attribute.""" + record.date_time = "2025-06-25T12:00:00+00:00" + del record.date_time + assert record.date_time is None + + def test_delattr_measurement(self, record): + """Test deleting a known measurement key via attribute.""" + record.temp = 88.0 + del record.temp + assert record.temp is None + + def test_delattr_ignored_missing_measurement_key(self, record): + """Test deleting a known measurement key that was never set is a no-op.""" + del record.temp + assert record.temp is None + + def test_len_and_iter(self, record): + """Test that __len__ and __iter__ behave as expected.""" + keys = list(iter(record)) + assert set(record.record_keys_writable()) == set(keys) + assert len(record) == len(keys) + + def test_in_operator_includes_measurements(self, record): + """Test that 'in' operator includes measurement keys.""" + assert "dish_washer_mr" in record + assert "temp" in record # known key, even if not yet set + assert "nonexistent" not in record + + def test_hasattr_behavior(self, record): + """Test that hasattr returns True for fields and known measurements.""" + assert hasattr(record, "date_time") + assert hasattr(record, "dish_washer_mr") + assert hasattr(record, "temp") # allowed, even if not yet set + assert not hasattr(record, "nonexistent") + + def test_model_validate_roundtrip(self, record): + """Test that MeasurementDataRecord can be serialized and revalidated.""" + dumped = record.model_dump() + restored = MeasurementDataRecord.model_validate(dumped) + assert restored.dish_washer_mr == 123.0 + assert restored.solar_power == 456.0 + assert restored.temp is None # not set + + def test_copy_preserves_measurements(self, record): + """Test that copying preserves measurement values.""" + record.temp = 22.2 + copied = record.model_copy() + assert copied.dish_washer_mr == 123.0 + assert copied.temp == 22.2 + assert copied is not record + + def test_equality_includes_measurements(self, record): + """Test that equality includes the `measurements` content.""" + other = record.model_copy() + assert record == other + + def test_inequality_differs_with_measurements(self, record): + """Test that records with different measurements are not equal.""" + other = record.model_copy(deep=True) + # Modify one measurement value in the copy + other["dish_washer_mr"] = 999.9 + assert record != other + + def test_in_operator_for_measurements_and_fields(self, record): + """Ensure 'in' works for both fields and configured measurement keys.""" + assert "dish_washer_mr" in record + assert "solar_power" in record + assert "date_time" in record # standard field + assert "temp" in record # allowed but not yet set + assert "unknown" not in record + + def test_hasattr_equivalence_to_getattr(self, record): + """hasattr should return True for all valid keys/measurements.""" + assert hasattr(record, "dish_washer_mr") + assert hasattr(record, "temp") + assert hasattr(record, "date_time") + assert not hasattr(record, "nonexistent") + + def test_dir_includes_measurement_keys(self, record): + """`dir(record)` should include measurement keys for introspection. + It shall not include the internal 'measurements' attribute. + """ + keys = dir(record) + assert "measurements" not in keys + for key in record.config.measurement.keys: + assert key in keys + + +class TestMeasurement: + """Test suite for the Measuremen class.""" + + @pytest.fixture + def measurement_eos(self, config_eos): + """Fixture to create a Measurement instance.""" + config_eos.measurement.load_emr_keys = ["load0_mr", "load1_mr", "load2_mr", "load3_mr"] + measurement = get_measurement() + record0 = MeasurementDataRecord( date_time=datetime(2023, 1, 1, hour=0), load0_mr=100, load1_mr=200, - ), - MeasurementDataRecord( - date_time=datetime(2023, 1, 1, hour=1), - load0_mr=150, - load1_mr=250, - ), - MeasurementDataRecord( - date_time=datetime(2023, 1, 1, hour=2), - load0_mr=200, - load1_mr=300, - ), - MeasurementDataRecord( - date_time=datetime(2023, 1, 1, hour=3), - load0_mr=250, - load1_mr=350, - ), - MeasurementDataRecord( - date_time=datetime(2023, 1, 1, hour=4), - load0_mr=300, - load1_mr=400, - ), - MeasurementDataRecord( - date_time=datetime(2023, 1, 1, hour=5), - load0_mr=350, - load1_mr=450, - ), - ] - return measurement + ) + assert record0.load0_mr == 100 + assert record0.load1_mr == 200 + measurement.records = [ + MeasurementDataRecord( + date_time=datetime(2023, 1, 1, hour=0), + load0_mr=100, + load1_mr=200, + ), + MeasurementDataRecord( + date_time=datetime(2023, 1, 1, hour=1), + load0_mr=150, + load1_mr=250, + ), + MeasurementDataRecord( + date_time=datetime(2023, 1, 1, hour=2), + load0_mr=200, + load1_mr=300, + ), + MeasurementDataRecord( + date_time=datetime(2023, 1, 1, hour=3), + load0_mr=250, + load1_mr=350, + ), + MeasurementDataRecord( + date_time=datetime(2023, 1, 1, hour=4), + load0_mr=300, + load1_mr=400, + ), + MeasurementDataRecord( + date_time=datetime(2023, 1, 1, hour=5), + load0_mr=350, + load1_mr=450, + ), + ] + return measurement + def test_interval_count(self, measurement_eos): + """Test interval count calculation.""" + start = datetime(2023, 1, 1, 0) + end = datetime(2023, 1, 1, 3) + interval = duration(hours=1) -def test_interval_count(measurement_eos): - """Test interval count calculation.""" - start = datetime(2023, 1, 1, 0) - end = datetime(2023, 1, 1, 3) - interval = duration(hours=1) + assert measurement_eos._interval_count(start, end, interval) == 3 - assert measurement_eos._interval_count(start, end, interval) == 3 + def test_interval_count_invalid_end_before_start(self, measurement_eos): + """Test interval count raises ValueError when end_datetime is before start_datetime.""" + start = datetime(2023, 1, 1, 3) + end = datetime(2023, 1, 1, 0) + interval = duration(hours=1) + with pytest.raises(ValueError, match="end_datetime must be after start_datetime"): + measurement_eos._interval_count(start, end, interval) -def test_interval_count_invalid_end_before_start(measurement_eos): - """Test interval count raises ValueError when end_datetime is before start_datetime.""" - start = datetime(2023, 1, 1, 3) - end = datetime(2023, 1, 1, 0) - interval = duration(hours=1) + def test_interval_count_invalid_non_positive_interval(self, measurement_eos): + """Test interval count raises ValueError when interval is non-positive.""" + start = datetime(2023, 1, 1, 0) + end = datetime(2023, 1, 1, 3) - with pytest.raises(ValueError, match="end_datetime must be after start_datetime"): - measurement_eos._interval_count(start, end, interval) + with pytest.raises(ValueError, match="interval must be positive"): + measurement_eos._interval_count(start, end, duration(hours=0)) + def test_energy_from_meter_readings_valid_input(self, measurement_eos): + """Test _energy_from_meter_readings with valid inputs and proper alignment of load data.""" + key = "load0_mr" + start_datetime = datetime(2023, 1, 1, 0) + end_datetime = datetime(2023, 1, 1, 5) + interval = duration(hours=1) -def test_interval_count_invalid_non_positive_interval(measurement_eos): - """Test interval count raises ValueError when interval is non-positive.""" - start = datetime(2023, 1, 1, 0) - end = datetime(2023, 1, 1, 3) - - with pytest.raises(ValueError, match="interval must be positive"): - measurement_eos._interval_count(start, end, duration(hours=0)) - - -def test_energy_from_meter_readings_valid_input(measurement_eos): - """Test _energy_from_meter_readings with valid inputs and proper alignment of load data.""" - key = "load0_mr" - start_datetime = datetime(2023, 1, 1, 0) - end_datetime = datetime(2023, 1, 1, 5) - interval = duration(hours=1) - - load_array = measurement_eos._energy_from_meter_readings( - key, start_datetime, end_datetime, interval - ) - - expected_load_array = np.array([50, 50, 50, 50, 50]) # Differences between consecutive readings - np.testing.assert_array_equal(load_array, expected_load_array) - - -def test_energy_from_meter_readings_empty_array(measurement_eos): - """Test _energy_from_meter_readings with no data (empty array).""" - key = "load0_mr" - start_datetime = datetime(2023, 1, 1, 0) - end_datetime = datetime(2023, 1, 1, 5) - interval = duration(hours=1) - - # Use empyt records array - measurement_eos.records = [] - - load_array = measurement_eos._energy_from_meter_readings( - key, start_datetime, end_datetime, interval - ) - - # Expected: an array of zeros with one less than the number of intervals - expected_size = ( - measurement_eos._interval_count(start_datetime, end_datetime + interval, interval) - 1 - ) - expected_load_array = np.zeros(expected_size) - np.testing.assert_array_equal(load_array, expected_load_array) - - -def test_energy_from_meter_readings_misaligned_array(measurement_eos): - """Test _energy_from_meter_readings with misaligned array size.""" - key = "load1_mr" - start_datetime = measurement_eos.min_datetime - end_datetime = measurement_eos.max_datetime - interval = duration(hours=1) - - # Use misaligned array, latest interval set to 2 hours (instead of 1 hour) - measurement_eos.records[-1].date_time = datetime(2023, 1, 1, 6) - - load_array = measurement_eos._energy_from_meter_readings( - key, start_datetime, end_datetime, interval - ) - - expected_load_array = np.array([50, 50, 50, 50, 25]) # Differences between consecutive readings - np.testing.assert_array_equal(load_array, expected_load_array) - - -def test_energy_from_meter_readings_partial_data(measurement_eos, caplog): - """Test _energy_from_meter_readings with partial data (misaligned but empty array).""" - key = "load2_mr" - start_datetime = datetime(2023, 1, 1, 0) - end_datetime = datetime(2023, 1, 1, 5) - interval = duration(hours=1) - - with caplog.at_level("DEBUG"): load_array = measurement_eos._energy_from_meter_readings( key, start_datetime, end_datetime, interval ) - expected_size = ( - measurement_eos._interval_count(start_datetime, end_datetime + interval, interval) - 1 - ) - expected_load_array = np.zeros(expected_size) - np.testing.assert_array_equal(load_array, expected_load_array) + expected_load_array = np.array([50, 50, 50, 50, 50]) # Differences between consecutive readings + np.testing.assert_array_equal(load_array, expected_load_array) + def test_energy_from_meter_readings_empty_array(self, measurement_eos): + """Test _energy_from_meter_readings with no data (empty array).""" + key = "load0_mr" + start_datetime = datetime(2023, 1, 1, 0) + end_datetime = datetime(2023, 1, 1, 5) + interval = duration(hours=1) -def test_energy_from_meter_readings_negative_interval(measurement_eos): - """Test _energy_from_meter_readings with a negative interval.""" - key = "load3_mr" - start_datetime = datetime(2023, 1, 1, 0) - end_datetime = datetime(2023, 1, 1, 5) - interval = duration(hours=-1) + # Use empyt records array + measurement_eos.records = [] - with pytest.raises(ValueError, match="interval must be positive"): - measurement_eos._energy_from_meter_readings(key, start_datetime, end_datetime, interval) - - -def test_load_total(measurement_eos): - """Test total load calculation.""" - start = datetime(2023, 1, 1, 0) - end = datetime(2023, 1, 1, 2) - interval = duration(hours=1) - - result = measurement_eos.load_total(start_datetime=start, end_datetime=end, interval=interval) - - # Expected total load per interval - expected = np.array([100, 100]) # Differences between consecutive meter readings - np.testing.assert_array_equal(result, expected) - - -def test_load_total_no_data(measurement_eos): - """Test total load calculation with no data.""" - measurement_eos.records = [] - start = datetime(2023, 1, 1, 0) - end = datetime(2023, 1, 1, 3) - interval = duration(hours=1) - - result = measurement_eos.load_total(start_datetime=start, end_datetime=end, interval=interval) - expected = np.zeros(3) # No data, so all intervals are zero - np.testing.assert_array_equal(result, expected) - - -def test_name_to_key(measurement_eos): - """Test name_to_key functionality.""" - settings = SettingsEOS( - measurement=MeasurementCommonSettings( - load0_name="Household", - load1_name="Heat Pump", + load_array = measurement_eos._energy_from_meter_readings( + key, start_datetime, end_datetime, interval ) - ) - measurement_eos.config.merge_settings(settings) - assert measurement_eos.name_to_key("Household", "load") == "load0_mr" - assert measurement_eos.name_to_key("Heat Pump", "load") == "load1_mr" - assert measurement_eos.name_to_key("Unknown", "load") is None - - -def test_name_to_key_invalid_topic(measurement_eos): - """Test name_to_key with an invalid topic.""" - settings = SettingsEOS( - MeasurementCommonSettings( - load0_name="Household", - load1_name="Heat Pump", + # Expected: an array of zeros with one less than the number of intervals + expected_size = ( + measurement_eos._interval_count(start_datetime, end_datetime + interval, interval) - 1 ) - ) - measurement_eos.config.merge_settings(settings) + expected_load_array = np.zeros(expected_size) + np.testing.assert_array_equal(load_array, expected_load_array) - assert measurement_eos.name_to_key("Household", "invalid_topic") is None + def test_energy_from_meter_readings_misaligned_array(self, measurement_eos): + """Test _energy_from_meter_readings with misaligned array size.""" + key = "load1_mr" + start_datetime = measurement_eos.min_datetime + end_datetime = measurement_eos.max_datetime + interval = duration(hours=1) + # Use misaligned array, latest interval set to 2 hours (instead of 1 hour) + measurement_eos.records[-1].date_time = datetime(2023, 1, 1, 6) -def test_load_total_partial_intervals(measurement_eos): - """Test total load calculation with partial intervals.""" - start = datetime(2023, 1, 1, 0, 30) # Start in the middle of an interval - end = datetime(2023, 1, 1, 1, 30) # End in the middle of another interval - interval = duration(hours=1) + load_array = measurement_eos._energy_from_meter_readings( + key, start_datetime, end_datetime, interval + ) - result = measurement_eos.load_total(start_datetime=start, end_datetime=end, interval=interval) - expected = np.array([100]) # Only one complete interval covered - np.testing.assert_array_equal(result, expected) + expected_load_array = np.array([50, 50, 50, 50, 25]) # Differences between consecutive readings + np.testing.assert_array_equal(load_array, expected_load_array) + + def test_energy_from_meter_readings_partial_data(self, measurement_eos, caplog): + """Test _energy_from_meter_readings with partial data (misaligned but empty array).""" + key = "load2_mr" + start_datetime = datetime(2023, 1, 1, 0) + end_datetime = datetime(2023, 1, 1, 5) + interval = duration(hours=1) + + with caplog.at_level("DEBUG"): + load_array = measurement_eos._energy_from_meter_readings( + key, start_datetime, end_datetime, interval + ) + + expected_size = ( + measurement_eos._interval_count(start_datetime, end_datetime + interval, interval) - 1 + ) + expected_load_array = np.zeros(expected_size) + np.testing.assert_array_equal(load_array, expected_load_array) + + def test_energy_from_meter_readings_negative_interval(self, measurement_eos): + """Test _energy_from_meter_readings with a negative interval.""" + key = "load3_mr" + start_datetime = datetime(2023, 1, 1, 0) + end_datetime = datetime(2023, 1, 1, 5) + interval = duration(hours=-1) + + with pytest.raises(ValueError, match="interval must be positive"): + measurement_eos._energy_from_meter_readings(key, start_datetime, end_datetime, interval) + + def test_load_total(self, measurement_eos): + """Test total load calculation.""" + start = datetime(2023, 1, 1, 0) + end = datetime(2023, 1, 1, 2) + interval = duration(hours=1) + + result = measurement_eos.load_total(start_datetime=start, end_datetime=end, interval=interval) + + # Expected total load per interval + expected = np.array([100, 100]) # Differences between consecutive meter readings + np.testing.assert_array_equal(result, expected) + + def test_load_total_no_data(self, measurement_eos): + """Test total load calculation with no data.""" + measurement_eos.records = [] + start = datetime(2023, 1, 1, 0) + end = datetime(2023, 1, 1, 3) + interval = duration(hours=1) + + result = measurement_eos.load_total(start_datetime=start, end_datetime=end, interval=interval) + expected = np.zeros(3) # No data, so all intervals are zero + np.testing.assert_array_equal(result, expected) + + def test_load_total_partial_intervals(self, measurement_eos): + """Test total load calculation with partial intervals.""" + start = datetime(2023, 1, 1, 0, 30) # Start in the middle of an interval + end = datetime(2023, 1, 1, 1, 30) # End in the middle of another interval + interval = duration(hours=1) + + result = measurement_eos.load_total(start_datetime=start, end_datetime=end, interval=interval) + expected = np.array([100]) # Only one complete interval covered + np.testing.assert_array_equal(result, expected) diff --git a/tests/test_prediction.py b/tests/test_prediction.py index de5fbbf..8e86ec8 100644 --- a/tests/test_prediction.py +++ b/tests/test_prediction.py @@ -4,6 +4,8 @@ from pydantic import ValidationError from akkudoktoreos.prediction.elecpriceakkudoktor import ElecPriceAkkudoktor from akkudoktoreos.prediction.elecpriceenergycharts import ElecPriceEnergyCharts from akkudoktoreos.prediction.elecpriceimport import ElecPriceImport +from akkudoktoreos.prediction.feedintarifffixed import FeedInTariffFixed +from akkudoktoreos.prediction.feedintariffimport import FeedInTariffImport from akkudoktoreos.prediction.loadakkudoktor import LoadAkkudoktor from akkudoktoreos.prediction.loadimport import LoadImport from akkudoktoreos.prediction.loadvrm import LoadVrm @@ -33,6 +35,8 @@ def forecast_providers(): ElecPriceAkkudoktor(), ElecPriceEnergyCharts(), ElecPriceImport(), + FeedInTariffFixed(), + FeedInTariffImport(), LoadAkkudoktor(), LoadVrm(), LoadImport(), @@ -76,15 +80,17 @@ def test_provider_sequence(prediction): assert isinstance(prediction.providers[0], ElecPriceAkkudoktor) assert isinstance(prediction.providers[1], ElecPriceEnergyCharts) assert isinstance(prediction.providers[2], ElecPriceImport) - assert isinstance(prediction.providers[3], LoadAkkudoktor) - assert isinstance(prediction.providers[4], LoadVrm) - assert isinstance(prediction.providers[5], LoadImport) - assert isinstance(prediction.providers[6], PVForecastAkkudoktor) - assert isinstance(prediction.providers[7], PVForecastVrm) - assert isinstance(prediction.providers[8], PVForecastImport) - assert isinstance(prediction.providers[9], WeatherBrightSky) - assert isinstance(prediction.providers[10], WeatherClearOutside) - assert isinstance(prediction.providers[11], WeatherImport) + assert isinstance(prediction.providers[3], FeedInTariffFixed) + assert isinstance(prediction.providers[4], FeedInTariffImport) + assert isinstance(prediction.providers[5], LoadAkkudoktor) + assert isinstance(prediction.providers[6], LoadVrm) + assert isinstance(prediction.providers[7], LoadImport) + assert isinstance(prediction.providers[8], PVForecastAkkudoktor) + assert isinstance(prediction.providers[9], PVForecastVrm) + assert isinstance(prediction.providers[10], PVForecastImport) + assert isinstance(prediction.providers[11], WeatherBrightSky) + assert isinstance(prediction.providers[12], WeatherClearOutside) + assert isinstance(prediction.providers[13], WeatherImport) def test_provider_by_id(prediction, forecast_providers): @@ -100,6 +106,8 @@ def test_prediction_repr(prediction): assert "ElecPriceAkkudoktor" in result assert "ElecPriceEnergyCharts" in result assert "ElecPriceImport" in result + assert "FeedInTariffFixed" in result + assert "FeedInTariffImport" in result assert "LoadAkkudoktor" in result assert "LoadVrm" in result assert "LoadImport" in result diff --git a/tests/test_predictionabc.py b/tests/test_predictionabc.py index 2f276dc..4bb84f1 100644 --- a/tests/test_predictionabc.py +++ b/tests/test_predictionabc.py @@ -101,7 +101,7 @@ class TestPredictionBase: assert base.config.prediction.hours == 2 def test_config_value_from_field_default(self, base, monkeypatch): - assert base.config.prediction.model_fields["historic_hours"].default == 48 + assert base.config.prediction.__class__.model_fields["historic_hours"].default == 48 assert base.config.prediction.historic_hours == 48 monkeypatch.setenv("EOS_PREDICTION__HISTORIC_HOURS", "128") base.config.reset_settings() @@ -192,7 +192,7 @@ class TestPredictionProvider: assert provider.config.prediction.hours == config_eos.prediction.hours assert provider.config.prediction.historic_hours == 2 - assert provider.start_datetime == sample_start_datetime + assert provider.ems_start_datetime == sample_start_datetime assert provider.end_datetime == sample_start_datetime + to_duration( f"{provider.config.prediction.hours} hours" ) @@ -416,7 +416,7 @@ class TestPredictionContainer: del container_with_providers["non_existent_key"] def test_len(self, container_with_providers): - assert len(container_with_providers) == 3 + assert len(container_with_providers) == 2 def test_repr(self, container_with_providers): representation = repr(container_with_providers) diff --git a/tests/test_pvforecastakkudoktor.py b/tests/test_pvforecastakkudoktor.py index 8831030..f117ca8 100644 --- a/tests/test_pvforecastakkudoktor.py +++ b/tests/test_pvforecastakkudoktor.py @@ -276,7 +276,7 @@ def test_pvforecast_akkudoktor_update_with_sample_forecast( ems_eos = get_ems() ems_eos.set_start_datetime(sample_forecast_start) provider.update_data(force_enable=True, force_update=True) - assert compare_datetimes(provider.start_datetime, sample_forecast_start).equal + assert compare_datetimes(provider.ems_start_datetime, sample_forecast_start).equal assert compare_datetimes(provider[0].date_time, to_datetime(sample_forecast_start)).equal @@ -328,7 +328,7 @@ def test_timezone_behaviour( ems_eos = get_ems() ems_eos.set_start_datetime(other_start_datetime) provider.update_data(force_update=True) - assert compare_datetimes(provider.start_datetime, other_start_datetime).equal + assert compare_datetimes(provider.ems_start_datetime, other_start_datetime).equal # Check wether first record starts at requested sample start time assert compare_datetimes(provider[0].date_time, sample_forecast_start).equal diff --git a/tests/test_pvforecastimport.py b/tests/test_pvforecastimport.py index a7664cc..882e901 100644 --- a/tests/test_pvforecastimport.py +++ b/tests/test_pvforecastimport.py @@ -19,8 +19,10 @@ def provider(sample_import_1_json, config_eos): "pvforecast": { "provider": "PVForecastImport", "provider_settings": { - "import_file_path": str(FILE_TESTDATA_PVFORECASTIMPORT_1_JSON), - "import_json": json.dumps(sample_import_1_json), + "PVForecastImport": { + "import_file_path": str(FILE_TESTDATA_PVFORECASTIMPORT_1_JSON), + "import_json": json.dumps(sample_import_1_json), + }, }, } } @@ -55,7 +57,9 @@ def test_invalid_provider(provider, config_eos): "pvforecast": { "provider": "", "provider_settings": { - "import_file_path": str(FILE_TESTDATA_PVFORECASTIMPORT_1_JSON), + "PVForecastImport": { + "import_file_path": str(FILE_TESTDATA_PVFORECASTIMPORT_1_JSON), + }, }, } } @@ -86,20 +90,20 @@ def test_import(provider, sample_import_1_json, start_datetime, from_file, confi ems_eos = get_ems() ems_eos.set_start_datetime(to_datetime(start_datetime, in_timezone="Europe/Berlin")) if from_file: - config_eos.pvforecast.provider_settings.import_json = None - assert config_eos.pvforecast.provider_settings.import_json is None + config_eos.pvforecast.provider_settings.PVForecastImport.import_json = None + assert config_eos.pvforecast.provider_settings.PVForecastImport.import_json is None else: - config_eos.pvforecast.provider_settings.import_file_path = None - assert config_eos.pvforecast.provider_settings.import_file_path is None + config_eos.pvforecast.provider_settings.PVForecastImport.import_file_path = None + assert config_eos.pvforecast.provider_settings.PVForecastImport.import_file_path is None provider.clear() # Call the method provider.update_data() # Assert: Verify the result is as expected - assert provider.start_datetime is not None + assert provider.ems_start_datetime is not None assert provider.total_hours is not None - assert compare_datetimes(provider.start_datetime, ems_eos.start_datetime).equal + assert compare_datetimes(provider.ems_start_datetime, ems_eos.start_datetime).equal values = sample_import_1_json["pvforecast_ac_power"] value_datetime_mapping = provider.import_datetimes(ems_eos.start_datetime, len(values)) for i, mapping in enumerate(value_datetime_mapping): diff --git a/tests/test_pvforecastvrm.py b/tests/test_pvforecastvrm.py index 4aa3740..7369373 100644 --- a/tests/test_pvforecastvrm.py +++ b/tests/test_pvforecastvrm.py @@ -19,8 +19,10 @@ def pvforecast_instance(config_eos): "pvforecast": { "provider": "PVForecastVrm", "provider_settings": { - "pvforecast_vrm_token": "dummy-token", - "pvforecast_vrm_idsite": 12345 + "PVForecastVrm": { + "pvforecast_vrm_token": "dummy-token", + "pvforecast_vrm_idsite": 12345, + }, } } } diff --git a/tests/test_pydantic.py b/tests/test_pydantic.py index 49f1f30..c905640 100644 --- a/tests/test_pydantic.py +++ b/tests/test_pydantic.py @@ -1,4 +1,4 @@ -from typing import Optional +from typing import Any, Optional import pandas as pd import pendulum @@ -11,12 +11,13 @@ from akkudoktoreos.core.pydantic import ( PydanticDateTimeDataFrame, PydanticDateTimeSeries, PydanticModelNestedValueMixin, + merge_models, ) -from akkudoktoreos.utils.datetimeutil import compare_datetimes, to_datetime +from akkudoktoreos.utils.datetimeutil import DateTime, compare_datetimes, to_datetime class PydanticTestModel(PydanticBaseModel): - datetime_field: pendulum.DateTime = Field( + datetime_field: DateTime = Field( ..., description="A datetime field with pendulum support." ) optional_field: Optional[str] = Field(default=None, description="An optional field.") @@ -33,6 +34,108 @@ class User(PydanticBaseModel): settings: Optional[dict[str, str]] = None +class SampleNestedModel(PydanticBaseModel): + threshold: int + enabled: bool = True + + +class SampleModel(PydanticBaseModel): + name: str + count: int + config: SampleNestedModel + optional: str | None = None + + +class TestMergeModels: + """Test suite for the merge_models utility function with None overriding.""" + + def test_flat_override(self): + """Top-level fields in update_dict override those in source, including None.""" + source = SampleModel(name="Test", count=10, config={"threshold": 5}) + update = {"name": "Updated"} + result = merge_models(source, update) + + assert result["name"] == "Updated" + assert result["count"] == 10 + assert result["config"]["threshold"] == 5 + + def test_flat_override_with_none(self): + """Update with None value should override source value.""" + source = SampleModel(name="Test", count=10, config={"threshold": 5}, optional="keep me") + update = {"optional": None} + result = merge_models(source, update) + + assert result["optional"] is None + + def test_nested_override(self): + """Nested fields in update_dict override nested fields in source, including None.""" + source = SampleModel(name="Test", count=10, config={"threshold": 5, "enabled": True}) + update = {"config": {"threshold": 99, "enabled": False}} + result = merge_models(source, update) + + assert result["config"]["threshold"] == 99 + assert result["config"]["enabled"] is False + + def test_nested_override_with_none(self): + """Nested update with None should override nested source values.""" + source = SampleModel(name="Test", count=10, config={"threshold": 5, "enabled": True}) + update = {"config": {"threshold": None}} + result = merge_models(source, update) + + assert result["config"]["threshold"] is None + assert result["config"]["enabled"] is True # untouched because not in update + + def test_preserve_source_values(self): + """Source values are preserved if not overridden in update_dict.""" + source = SampleModel(name="Source", count=7, config={"threshold": 1}) + update: dict[str, Any] = {} + result = merge_models(source, update) + + assert result["name"] == "Source" + assert result["count"] == 7 + assert result["config"]["threshold"] == 1 + + def test_update_extends_source(self): + """Optional fields in update_dict are added to result.""" + source = SampleModel(name="Test", count=10, config={"threshold": 5}) + update = {"optional": "new value"} + result = merge_models(source, update) + + assert result["optional"] == "new value" + + def test_update_extends_source_with_none(self): + """Optional field with None in update_dict is added and overrides source.""" + source = SampleModel(name="Test", count=10, config={"threshold": 5}, optional="value") + update = {"optional": None} + result = merge_models(source, update) + + assert result["optional"] is None + + def test_deep_merge_behavior(self): + """Nested updates merge with source, overriding only specified subkeys.""" + source = SampleModel(name="Model", count=3, config={"threshold": 1, "enabled": False}) + update = {"config": {"enabled": True}} + result = merge_models(source, update) + + assert result["config"]["enabled"] is True + assert result["config"]["threshold"] == 1 + + def test_override_all(self): + """All fields in update_dict override all fields in source, including None.""" + source = SampleModel(name="Orig", count=1, config={"threshold": 10, "enabled": True}) + update = { + "name": "New", + "count": None, + "config": {"threshold": 50, "enabled": None} + } + result = merge_models(source, update) + + assert result["name"] == "New" + assert result["count"] is None + assert result["config"]["threshold"] == 50 + assert result["config"]["enabled"] is None + + class TestPydanticModelNestedValueMixin: """Umbrella test class to group all test cases for `PydanticModelNestedValueMixin`.""" @@ -242,7 +345,7 @@ class TestPydanticBaseModel: assert model.datetime_field == dt def test_invalid_datetime_string(self): - with pytest.raises(ValidationError, match="Cannot convert 'invalid_datetime' to datetime"): + with pytest.raises(ValueError): PydanticTestModel(datetime_field="invalid_datetime") def test_iso8601_serialization(self): @@ -299,6 +402,7 @@ class TestPydanticDateTimeData: class TestPydanticDateTimeDataFrame: def test_valid_dataframe(self): + """Ensure conversion from and to DataFrame preserves index and values.""" df = pd.DataFrame( { "value": [100, 200], @@ -308,13 +412,101 @@ class TestPydanticDateTimeDataFrame: model = PydanticDateTimeDataFrame.from_dataframe(df) result = model.to_dataframe() - # Check index assert len(result.index) == len(df.index) for i, dt in enumerate(df.index): expected_dt = to_datetime(dt) result_dt = to_datetime(result.index[i]) assert compare_datetimes(result_dt, expected_dt).equal + def test_add_row(self): + """Verify that a new row can be inserted with matching columns.""" + model = PydanticDateTimeDataFrame( + data={"2024-12-21T00:00:00": {"value": 100}}, dtypes={"value": "int64"} + ) + model.add_row("2024-12-22T00:00:00", {"value": 200}) + + # Normalize key the same way the model stores it + key = model._normalize_index("2024-12-22T00:00:00") + + assert key in model.data + assert model.data[key]["value"] == 200 + + def test_add_row_column_mismatch_raises(self): + """Ensure adding a row with mismatched columns raises ValueError.""" + model = PydanticDateTimeDataFrame( + data={"2024-12-21T00:00:00": {"value": 100}}, dtypes={"value": "int64"} + ) + with pytest.raises(ValueError): + model.add_row("2024-12-22T00:00:00", {"wrong": 200}) + + def test_update_row(self): + """Check updating an existing row's values works.""" + model = PydanticDateTimeDataFrame( + data={"2024-12-21T00:00:00": {"value": 100}}, dtypes={"value": "int64"} + ) + model.update_row("2024-12-21T00:00:00", {"value": 999}) + + key = model._normalize_index("2024-12-21T00:00:00") + assert model.data[key]["value"] == 999 + + def test_update_row_missing_raises(self): + """Verify updating a non-existing row raises KeyError.""" + model = PydanticDateTimeDataFrame( + data={"2024-12-21T00:00:00": {"value": 100}}, dtypes={"value": "int64"} + ) + with pytest.raises(KeyError): + model.update_row("2024-12-22T00:00:00", {"value": 999}) + + def test_delete_row(self): + """Ensure rows can be deleted by index.""" + model = PydanticDateTimeDataFrame( + data={"2024-12-21T00:00:00": {"value": 100}}, dtypes={"value": "int64"} + ) + model.delete_row("2024-12-21T00:00:00") + assert "2024-12-21T00:00:00" not in model.data + + def test_set_and_get_value(self): + """Confirm set_value and get_value operate correctly.""" + model = PydanticDateTimeDataFrame( + data={"2024-12-21T00:00:00": {"value": 100}}, dtypes={"value": "int64"} + ) + model.set_value("2024-12-21T00:00:00", "value", 555) + assert model.get_value("2024-12-21T00:00:00", "value") == 555 + + def test_add_column(self): + """Check that a new column can be added with default value.""" + model = PydanticDateTimeDataFrame( + data={"2024-12-21T00:00:00": {"value": 100}}, dtypes={"value": "int64"} + ) + model.add_column("extra", default=0, dtype="int64") + + key = model._normalize_index("2024-12-21T00:00:00") + assert model.data[key]["extra"] == 0 + assert model.dtypes["extra"] == "int64" + + def test_rename_column(self): + """Ensure renaming a column updates all rows and dtypes.""" + model = PydanticDateTimeDataFrame( + data={"2024-12-21T00:00:00": {"value": 100}}, dtypes={"value": "int64"} + ) + model.rename_column("value", "renamed") + + key = model._normalize_index("2024-12-21T00:00:00") + assert "renamed" in model.data[key] + assert "value" not in model.data[key] + assert model.dtypes["renamed"] == "int64" + + def test_drop_column(self): + """Verify dropping a column removes it from both data and dtypes.""" + model = PydanticDateTimeDataFrame( + data={"2024-12-21T00:00:00": {"value": 100, "extra": 1}}, dtypes={"value": "int64", "extra": "int64"} + ) + model.drop_column("extra") + + key = model._normalize_index("2024-12-21T00:00:00") + assert "extra" not in model.data[key] + assert "extra" not in model.dtypes + class TestPydanticDateTimeSeries: def test_valid_series(self): diff --git a/tests/test_server.py b/tests/test_server.py index b5bc5da..8fbe41b 100644 --- a/tests/test_server.py +++ b/tests/test_server.py @@ -8,12 +8,10 @@ from pathlib import Path import psutil import pytest import requests +from conftest import cleanup_eos_eosdash -from akkudoktoreos.server.server import get_default_host - -DIR_TESTDATA = Path(__file__).absolute().parent.joinpath("testdata") - -FILE_TESTDATA_EOSSERVER_CONFIG_1 = DIR_TESTDATA.joinpath("eosserver_config_1.json") +from akkudoktoreos.core.version import __version__ +from akkudoktoreos.server.server import get_default_host, wait_for_port_free class TestServer: @@ -22,7 +20,14 @@ class TestServer: server = server_setup_for_class["server"] eos_dir = server_setup_for_class["eos_dir"] - result = requests.get(f"{server}/v1/config") + # Assure server is running + result = requests.get(f"{server}/v1/health", timeout=2) + assert result.status_code == HTTPStatus.OK + health = result.json() + assert health["status"] == "alive" + assert health["version"] == __version__ + + result = requests.get(f"{server}/v1/config", timeout=2) assert result.status_code == HTTPStatus.OK # Get testing config @@ -37,258 +42,36 @@ class TestServer: assert str(data_folder_path).startswith(eos_dir) assert str(data_ouput_path).startswith(eos_dir) - def test_prediction_brightsky(self, server_setup_for_class, is_system_test): - """Test weather prediction by BrightSky.""" - server = server_setup_for_class["server"] - eos_dir = server_setup_for_class["eos_dir"] - - result = requests.get(f"{server}/v1/config") - assert result.status_code == HTTPStatus.OK - - # Get testing config - config_json = result.json() - config_folder_path = Path(config_json["general"]["config_folder_path"]) - # Assure we are working in test environment - assert str(config_folder_path).startswith(eos_dir) - - result = requests.put(f"{server}/v1/config/weather/provider", json="BrightSky") - assert result.status_code == HTTPStatus.OK - - # Assure prediction is enabled - result = requests.get(f"{server}/v1/prediction/providers?enabled=true") - assert result.status_code == HTTPStatus.OK - providers = result.json() - assert "BrightSky" in providers - - if is_system_test: - result = requests.post(f"{server}/v1/prediction/update/BrightSky") - assert result.status_code == HTTPStatus.OK - - result = requests.get(f"{server}/v1/prediction/series?key=weather_temp_air") - assert result.status_code == HTTPStatus.OK - - data = result.json() - assert len(data["data"]) > 24 - - else: - pass - - def test_prediction_clearoutside(self, server_setup_for_class, is_system_test): - """Test weather prediction by ClearOutside.""" - server = server_setup_for_class["server"] - eos_dir = server_setup_for_class["eos_dir"] - - result = requests.put(f"{server}/v1/config/weather/provider", json="ClearOutside") - assert result.status_code == HTTPStatus.OK - - # Assure prediction is enabled - result = requests.get(f"{server}/v1/prediction/providers?enabled=true") - assert result.status_code == HTTPStatus.OK - providers = result.json() - assert "ClearOutside" in providers - - if is_system_test: - result = requests.post(f"{server}/v1/prediction/update/ClearOutside") - assert result.status_code == HTTPStatus.OK - - result = requests.get(f"{server}/v1/prediction/series?key=weather_temp_air") - assert result.status_code == HTTPStatus.OK - - data = result.json() - assert len(data["data"]) > 24 - - else: - pass - - def test_prediction_pvforecastakkudoktor(self, server_setup_for_class, is_system_test): - """Test PV prediction by PVForecastAkkudoktor.""" - server = server_setup_for_class["server"] - eos_dir = server_setup_for_class["eos_dir"] - - # Reset config - with FILE_TESTDATA_EOSSERVER_CONFIG_1.open("r", encoding="utf-8", newline=None) as fd: - config = json.load(fd) - config["pvforecast"]["provider"] = "PVForecastAkkudoktor" - result = requests.put(f"{server}/v1/config", json=config) - assert result.status_code == HTTPStatus.OK - - # Assure prediction is enabled - result = requests.get(f"{server}/v1/prediction/providers?enabled=true") - assert result.status_code == HTTPStatus.OK - providers = result.json() - assert "PVForecastAkkudoktor" in providers - - if is_system_test: - result = requests.post(f"{server}/v1/prediction/update/PVForecastAkkudoktor") - assert result.status_code == HTTPStatus.OK - - result = requests.get(f"{server}/v1/prediction/series?key=pvforecast_ac_power") - assert result.status_code == HTTPStatus.OK - - data = result.json() - assert len(data["data"]) > 24 - - else: - pass - - def test_prediction_elecpriceakkudoktor(self, server_setup_for_class, is_system_test): - """Test electricity price prediction by ElecPriceImport.""" - server = server_setup_for_class["server"] - eos_dir = server_setup_for_class["eos_dir"] - - # Reset config - with FILE_TESTDATA_EOSSERVER_CONFIG_1.open("r", encoding="utf-8", newline=None) as fd: - config = json.load(fd) - config["elecprice"]["provider"] = "ElecPriceAkkudoktor" - result = requests.put(f"{server}/v1/config", json=config) - assert result.status_code == HTTPStatus.OK - - # Assure prediction is enabled - result = requests.get(f"{server}/v1/prediction/providers?enabled=true") - assert result.status_code == HTTPStatus.OK - providers = result.json() - assert "ElecPriceAkkudoktor" in providers - - if is_system_test: - result = requests.post(f"{server}/v1/prediction/update/ElecPriceAkkudoktor") - assert result.status_code == HTTPStatus.OK - - result = requests.get(f"{server}/v1/prediction/series?key=elecprice_marketprice_wh") - assert result.status_code == HTTPStatus.OK - - data = result.json() - assert len(data["data"]) > 24 - - else: - pass - - def test_prediction_loadakkudoktor(self, server_setup_for_class, is_system_test): - """Test load prediction by LoadAkkudoktor.""" - server = server_setup_for_class["server"] - eos_dir = server_setup_for_class["eos_dir"] - - result = requests.put(f"{server}/v1/config/load/provider", json="LoadAkkudoktor") - assert result.status_code == HTTPStatus.OK - - # Assure prediction is enabled - result = requests.get(f"{server}/v1/prediction/providers?enabled=true") - assert result.status_code == HTTPStatus.OK - providers = result.json() - assert "LoadAkkudoktor" in providers - - if is_system_test: - result = requests.post(f"{server}/v1/prediction/update/LoadAkkudoktor") - assert result.status_code == HTTPStatus.OK - - result = requests.get(f"{server}/v1/prediction/series?key=load_mean") - assert result.status_code == HTTPStatus.OK - - data = result.json() - assert len(data["data"]) > 24 - - else: - pass - - def test_admin_cache(self, server_setup_for_class, is_system_test): - """Test whether cache is reconstructed from cached files.""" - server = server_setup_for_class["server"] - eos_dir = server_setup_for_class["eos_dir"] - - result = requests.get(f"{server}/v1/admin/cache") - assert result.status_code == HTTPStatus.OK - cache = result.json() - - if is_system_test: - # There should be some cache data - assert cache != {} - - # Save cache - result = requests.post(f"{server}/v1/admin/cache/save") - assert result.status_code == HTTPStatus.OK - cache_saved = result.json() - assert cache_saved == cache - - # Clear cache - should clear nothing as all cache files expire in the future - result = requests.post(f"{server}/v1/admin/cache/clear") - assert result.status_code == HTTPStatus.OK - cache_cleared = result.json() - assert cache_cleared == cache - - # Force clear cache - result = requests.post(f"{server}/v1/admin/cache/clear?clear_all=true") - assert result.status_code == HTTPStatus.OK - cache_cleared = result.json() - assert cache_cleared == {} - - # Try to load already deleted cache entries - result = requests.post(f"{server}/v1/admin/cache/load") - assert result.status_code == HTTPStatus.OK - cache_loaded = result.json() - assert cache_loaded == {} - - # Cache should still be empty - result = requests.get(f"{server}/v1/admin/cache") - assert result.status_code == HTTPStatus.OK - cache = result.json() - assert cache == {} - class TestServerStartStop: def test_server_start_eosdash(self, tmpdir): """Test the EOSdash server startup from EOS.""" # Do not use any fixture as this will make pytest the owner of the EOSdash port. host = get_default_host() - if os.name == "nt": - # Windows does not provide SIGKILL - sigkill = signal.SIGTERM # type: ignore[attr-defined,unused-ignore] - else: - sigkill = signal.SIGKILL # type: ignore port = 8503 + eosdash_host = host eosdash_port = 8504 timeout = 120 server = f"http://{host}:{port}" - eosdash_server = f"http://{host}:{eosdash_port}" + eosdash_server = f"http://{eosdash_host}:{eosdash_port}" eos_dir = str(tmpdir) - # Cleanup any EOSdash process left. - try: - result = requests.get(f"{eosdash_server}/eosdash/health", timeout=2) - if result.status_code == HTTPStatus.OK: - pid = result.json()["pid"] - os.kill(pid, sigkill) - time.sleep(1) - result = requests.get(f"{eosdash_server}/eosdash/health", timeout=2) - assert result.status_code != HTTPStatus.OK - except: - pass - - # Wait for EOSdash port to be freed - process_info: list[dict] = [] - for retries in range(int(timeout / 3)): - process_info = [] - pids: list[int] = [] - for conn in psutil.net_connections(kind="inet"): - if conn.laddr.port == eosdash_port: - if conn.pid not in pids: - # Get fresh process info - process = psutil.Process(conn.pid) - pids.append(conn.pid) - process_info.append(process.as_dict(attrs=["pid", "cmdline"])) - if len(process_info) == 0: - break - time.sleep(3) - assert len(process_info) == 0 + # Cleanup any EOS and EOSdash process left. + cleanup_eos_eosdash(host, port, eosdash_host, eosdash_port, timeout) # Import after test setup to prevent creation of config file before test from akkudoktoreos.server.eos import start_eosdash + # Port may be blocked + assert wait_for_port_free(eosdash_port, timeout=120, waiting_app_name="EOSdash") + process = start_eosdash( - host=host, + host=eosdash_host, port=eosdash_port, eos_host=host, eos_port=port, - log_level="debug", + log_level="DEBUG", access_log=False, reload=False, eos_dir=eos_dir, @@ -310,7 +93,9 @@ class TestServerStartStop: time.sleep(3) assert startup, f"Connection to {eosdash_server}/eosdash/health failed: {error}" - assert result.json()["status"] == "alive" + health = result.json() + assert health["status"] == "alive" + assert health["version"] == __version__ # Shutdown eosdash try: @@ -324,6 +109,9 @@ class TestServerStartStop: except: pass + # Cleanup any EOS and EOSdash process left. + cleanup_eos_eosdash(host, port, eosdash_host, eosdash_port, timeout) + @pytest.mark.skipif(os.name == "nt", reason="Server restart not supported on Windows") def test_server_restart(self, server_setup_for_function, is_system_test): """Test server restart.""" @@ -403,7 +191,7 @@ class TestServerStartStop: # Assure EOS is up again startup = False error = "" - for retries in range(int(timeout / 3)): + for retries in range(int(timeout / 5)): try: result = requests.get(f"{server}/v1/health", timeout=2) if result.status_code == HTTPStatus.OK: @@ -412,7 +200,7 @@ class TestServerStartStop: error = f"{result.status_code}, {str(result.content)}" except Exception as ex: error = str(ex) - time.sleep(3) + time.sleep(5) assert startup, f"Connection to {server}/v1/health failed: {error}" assert result.json()["status"] == "alive" @@ -442,3 +230,24 @@ class TestServerStartStop: assert result.status_code == HTTPStatus.OK assert "Stopping EOS.." in result.json()["message"] new_pid = result.json()["pid"] + + +class TestServerWithEnv: + eos_env = { + "EOS_SERVER__EOSDASH_PORT": "8555", + } + + def test_server_setup_for_class(self, server_setup_for_class): + """Ensure server is started with environment passed to configuration.""" + server = server_setup_for_class["server"] + + assert server_setup_for_class["eosdash_port"] == int(self.eos_env["EOS_SERVER__EOSDASH_PORT"]) + + result = requests.get(f"{server}/v1/config") + assert result.status_code == HTTPStatus.OK + + # Get testing config + config_json = result.json() + + # Assure config got configuration from environment + assert config_json["server"]["eosdash_port"] == int(self.eos_env["EOS_SERVER__EOSDASH_PORT"]) diff --git a/tests/test_stringutil.py b/tests/test_stringutil.py new file mode 100644 index 0000000..82603fa --- /dev/null +++ b/tests/test_stringutil.py @@ -0,0 +1,51 @@ +"""Tests for the stringutil module.""" + +import pytest + +from akkudoktoreos.utils.stringutil import str2bool + + +class TestStr2Bool: + """Unit tests for the str2bool function.""" + + @pytest.mark.parametrize( + "input_value", + ["yes", "YES", "y", "Y", "true", "TRUE", "t", "T", "1", "on", "ON"], + ) + def test_truthy_values(self, input_value): + """Test that all accepted truthy string values return True.""" + assert str2bool(input_value) is True + + @pytest.mark.parametrize( + "input_value", + ["no", "NO", "n", "N", "false", "FALSE", "f", "F", "0", "off", "OFF"], + ) + def test_falsy_values(self, input_value): + """Test that all accepted falsy string values return False.""" + assert str2bool(input_value) is False + + def test_bool_input_returns_itself(self): + """Test that passing a boolean returns the same value.""" + assert str2bool(True) is True + assert str2bool(False) is False + + def test_whitespace_is_ignored(self): + """Test that surrounding whitespace does not affect the result.""" + assert str2bool(" yes ") is True + assert str2bool("\tno\n") is False + + def test_invalid_string_raises_value_error(self): + """Test that invalid strings raise a ValueError.""" + with pytest.raises(ValueError, match="Invalid boolean value"): + str2bool("maybe") + with pytest.raises(ValueError): + str2bool("truthish") + + def test_type_error_on_non_string_non_bool(self): + """Test that non-string, non-boolean inputs raise ValueError.""" + with pytest.raises(ValueError, match="Invalid boolean value"): + str2bool(None) + with pytest.raises(ValueError, match="Invalid boolean value"): + str2bool(1.23) + with pytest.raises(ValueError, match="Invalid boolean value"): + str2bool([]) diff --git a/tests/test_system.py b/tests/test_system.py new file mode 100644 index 0000000..868d595 --- /dev/null +++ b/tests/test_system.py @@ -0,0 +1,210 @@ +import json +import os +import signal +import time +from http import HTTPStatus +from pathlib import Path + +import pytest +import requests + +DIR_TESTDATA = Path(__file__).absolute().parent.joinpath("testdata") + +FILE_TESTDATA_EOSSERVER_CONFIG_1 = DIR_TESTDATA.joinpath("eosserver_config_1.json") + + +class TestSystem: + def test_prediction_brightsky(self, server_setup_for_class, is_system_test): + """Test weather prediction by BrightSky.""" + server = server_setup_for_class["server"] + eos_dir = server_setup_for_class["eos_dir"] + + result = requests.get(f"{server}/v1/config", timeout=2) + assert result.status_code == HTTPStatus.OK + + # Get testing config + config_json = result.json() + config_folder_path = Path(config_json["general"]["config_folder_path"]) + # Assure we are working in test environment + assert str(config_folder_path).startswith(eos_dir) + + result = requests.put(f"{server}/v1/config/weather/provider", json="BrightSky") + assert result.status_code == HTTPStatus.OK + + # Assure prediction is enabled + result = requests.get(f"{server}/v1/prediction/providers?enabled=true") + assert result.status_code == HTTPStatus.OK + providers = result.json() + assert "BrightSky" in providers + + if is_system_test: + result = requests.post(f"{server}/v1/prediction/update/BrightSky") + assert result.status_code == HTTPStatus.OK + + result = requests.get(f"{server}/v1/prediction/series?key=weather_temp_air") + assert result.status_code == HTTPStatus.OK + + data = result.json() + assert len(data["data"]) > 24 + + else: + pass + + def test_prediction_clearoutside(self, server_setup_for_class, is_system_test): + """Test weather prediction by ClearOutside.""" + server = server_setup_for_class["server"] + eos_dir = server_setup_for_class["eos_dir"] + + result = requests.put(f"{server}/v1/config/weather/provider", json="ClearOutside") + assert result.status_code == HTTPStatus.OK + + # Assure prediction is enabled + result = requests.get(f"{server}/v1/prediction/providers?enabled=true") + assert result.status_code == HTTPStatus.OK + providers = result.json() + assert "ClearOutside" in providers + + if is_system_test: + result = requests.post(f"{server}/v1/prediction/update/ClearOutside") + assert result.status_code == HTTPStatus.OK, f"Failed: {result.headers} {result.text}" + + result = requests.get(f"{server}/v1/prediction/series?key=weather_temp_air") + assert result.status_code == HTTPStatus.OK + + data = result.json() + assert len(data["data"]) > 24 + + else: + pass + + def test_prediction_pvforecastakkudoktor(self, server_setup_for_class, is_system_test): + """Test PV prediction by PVForecastAkkudoktor.""" + server = server_setup_for_class["server"] + eos_dir = server_setup_for_class["eos_dir"] + + # Reset config + with FILE_TESTDATA_EOSSERVER_CONFIG_1.open("r", encoding="utf-8", newline=None) as fd: + config = json.load(fd) + config["pvforecast"]["provider"] = "PVForecastAkkudoktor" + result = requests.put(f"{server}/v1/config", json=config) + assert result.status_code == HTTPStatus.OK + + # Assure prediction is enabled + result = requests.get(f"{server}/v1/prediction/providers?enabled=true") + assert result.status_code == HTTPStatus.OK + providers = result.json() + assert "PVForecastAkkudoktor" in providers + + if is_system_test: + result = requests.post(f"{server}/v1/prediction/update/PVForecastAkkudoktor") + assert result.status_code == HTTPStatus.OK, f"Failed: {result.headers} {result.text}" + + result = requests.get(f"{server}/v1/prediction/series?key=pvforecast_ac_power") + assert result.status_code == HTTPStatus.OK + + data = result.json() + assert len(data["data"]) > 24 + + else: + pass + + def test_prediction_elecpriceakkudoktor(self, server_setup_for_class, is_system_test): + """Test electricity price prediction by ElecPriceImport.""" + server = server_setup_for_class["server"] + eos_dir = server_setup_for_class["eos_dir"] + + # Reset config + with FILE_TESTDATA_EOSSERVER_CONFIG_1.open("r", encoding="utf-8", newline=None) as fd: + config = json.load(fd) + config["elecprice"]["provider"] = "ElecPriceAkkudoktor" + result = requests.put(f"{server}/v1/config", json=config) + assert result.status_code == HTTPStatus.OK + + # Assure prediction is enabled + result = requests.get(f"{server}/v1/prediction/providers?enabled=true") + assert result.status_code == HTTPStatus.OK + providers = result.json() + assert "ElecPriceAkkudoktor" in providers + + if is_system_test: + result = requests.post(f"{server}/v1/prediction/update/ElecPriceAkkudoktor") + assert result.status_code == HTTPStatus.OK + + result = requests.get(f"{server}/v1/prediction/series?key=elecprice_marketprice_wh") + assert result.status_code == HTTPStatus.OK + + data = result.json() + assert len(data["data"]) > 24 + + else: + pass + + def test_prediction_loadakkudoktor(self, server_setup_for_class, is_system_test): + """Test load prediction by LoadAkkudoktor.""" + server = server_setup_for_class["server"] + eos_dir = server_setup_for_class["eos_dir"] + + result = requests.put(f"{server}/v1/config/load/provider", json="LoadAkkudoktor") + assert result.status_code == HTTPStatus.OK + + # Assure prediction is enabled + result = requests.get(f"{server}/v1/prediction/providers?enabled=true") + assert result.status_code == HTTPStatus.OK + providers = result.json() + assert "LoadAkkudoktor" in providers + + if is_system_test: + result = requests.post(f"{server}/v1/prediction/update/LoadAkkudoktor") + assert result.status_code == HTTPStatus.OK + + result = requests.get(f"{server}/v1/prediction/series?key=load_mean") + assert result.status_code == HTTPStatus.OK + + data = result.json() + assert len(data["data"]) > 24 + + else: + pass + + def test_admin_cache(self, server_setup_for_class, is_system_test): + """Test whether cache is reconstructed from cached files.""" + server = server_setup_for_class["server"] + eos_dir = server_setup_for_class["eos_dir"] + + result = requests.get(f"{server}/v1/admin/cache") + assert result.status_code == HTTPStatus.OK + cache = result.json() + + if is_system_test: + # There should be some cache data + assert cache != {} + + # Save cache + result = requests.post(f"{server}/v1/admin/cache/save") + assert result.status_code == HTTPStatus.OK + cache_saved = result.json() + assert cache_saved == cache + + # Clear expired cache - should clear nothing as all cache files expire in the future + result = requests.post(f"{server}/v1/admin/cache/clear-expired") + assert result.status_code == HTTPStatus.OK + cache_cleared = result.json() + assert cache_cleared == cache + + # Force clear cache + result = requests.post(f"{server}/v1/admin/cache/clear") + assert result.status_code == HTTPStatus.OK + cache_cleared = result.json() + assert cache_cleared == {} + + # Try to load already deleted cache entries + result = requests.post(f"{server}/v1/admin/cache/load") + assert result.status_code == HTTPStatus.OK + cache_loaded = result.json() + assert cache_loaded == {} + + # Cache should still be empty + result = requests.get(f"{server}/v1/admin/cache") + assert result.status_code == HTTPStatus.OK + cache = result.json() + assert cache == {} diff --git a/tests/test_weatherclearoutside.py b/tests/test_weatherclearoutside.py index fe1b97d..7f1292c 100644 --- a/tests/test_weatherclearoutside.py +++ b/tests/test_weatherclearoutside.py @@ -162,7 +162,7 @@ def test_update_data(mock_get, provider, sample_clearout_1_html, sample_clearout # Check for correct prediction time window assert provider.config.prediction.hours == 48 assert provider.config.prediction.historic_hours == 48 - assert compare_datetimes(provider.start_datetime, expected_start).equal + assert compare_datetimes(provider.ems_start_datetime, expected_start).equal assert compare_datetimes(provider.end_datetime, expected_end).equal assert compare_datetimes(provider.keep_datetime, expected_keep).equal diff --git a/tests/test_weatherimport.py b/tests/test_weatherimport.py index 1d66683..6125cd1 100644 --- a/tests/test_weatherimport.py +++ b/tests/test_weatherimport.py @@ -19,8 +19,10 @@ def provider(sample_import_1_json, config_eos): "weather": { "provider": "WeatherImport", "provider_settings": { - "import_file_path": str(FILE_TESTDATA_WEATHERIMPORT_1_JSON), - "import_json": json.dumps(sample_import_1_json), + "WeatherImport": { + "import_file_path": str(FILE_TESTDATA_WEATHERIMPORT_1_JSON), + "import_json": json.dumps(sample_import_1_json), + }, }, } } @@ -55,7 +57,9 @@ def test_invalid_provider(provider, config_eos, monkeypatch): "weather": { "provider": "", "provider_settings": { - "import_file_path": str(FILE_TESTDATA_WEATHERIMPORT_1_JSON), + "WeatherImport": { + "import_file_path": str(FILE_TESTDATA_WEATHERIMPORT_1_JSON), + }, }, } } @@ -86,20 +90,20 @@ def test_import(provider, sample_import_1_json, start_datetime, from_file, confi ems_eos = get_ems() ems_eos.set_start_datetime(to_datetime(start_datetime, in_timezone="Europe/Berlin")) if from_file: - config_eos.weather.provider_settings.import_json = None - assert config_eos.weather.provider_settings.import_json is None + config_eos.weather.provider_settings.WeatherImport.import_json = None + assert config_eos.weather.provider_settings.WeatherImport.import_json is None else: - config_eos.weather.provider_settings.import_file_path = None - assert config_eos.weather.provider_settings.import_file_path is None + config_eos.weather.provider_settings.WeatherImport.import_file_path = None + assert config_eos.weather.provider_settings.WeatherImport.import_file_path is None provider.clear() # Call the method provider.update_data() # Assert: Verify the result is as expected - assert provider.start_datetime is not None + assert provider.ems_start_datetime is not None assert provider.total_hours is not None - assert compare_datetimes(provider.start_datetime, ems_eos.start_datetime).equal + assert compare_datetimes(provider.ems_start_datetime, ems_eos.start_datetime).equal values = sample_import_1_json["weather_temp_air"] value_datetime_mapping = provider.import_datetimes(ems_eos.start_datetime, len(values)) for i, mapping in enumerate(value_datetime_mapping): diff --git a/tests/testdata/eos_config_andreas_0_1_0.json b/tests/testdata/eos_config_andreas_0_1_0.json new file mode 100644 index 0000000..fa9d716 --- /dev/null +++ b/tests/testdata/eos_config_andreas_0_1_0.json @@ -0,0 +1,101 @@ +{ + "general": { + "data_folder_path": null, + "data_output_subpath": "output", + "latitude": 52.5, + "longitude": 13.4 + }, + "cache": { + "subpath": "cache", + "cleanup_interval": 300.0 + }, + "ems": { + "startup_delay": 5.0, + "interval": 300.0 + }, + "logging": { + "level": "INFO" + }, + "devices": { + "batteries": [ + { + "device_id": "pv_akku", + "hours": null, + "capacity_wh": 30000, + "charging_efficiency": 0.88, + "discharging_efficiency": 0.88, + "max_charge_power_w": 5000, + "initial_soc_percentage": 0, + "min_soc_percentage": 0, + "max_soc_percentage": 100 + } + ], + "inverters": [], + "home_appliances": [] + }, + "measurement": { + "load0_name": "Household", + "load1_name": null, + "load2_name": null, + "load3_name": null, + "load4_name": null + }, + "optimization": { + "hours": 48, + "penalty": 10, + "ev_available_charge_rates_percent": [ + 0.0, 37.5, 50.0, 62.5, 75.0, 87.5, 100.0 + ] + }, + "prediction": { + "hours": 48, + "historic_hours": 48 + }, + "elecprice": { + "provider": "ElecPriceAkkudoktor", + "charges_kwh": 0.21, + "provider_settings": null + }, + "load": { + "provider_settings": { + "loadakkudoktor_year_energy": 13000 + } + }, + "pvforecast": { + "provider": "PVForecastAkkudoktor", + "planes": [ + { + "surface_tilt": 87.907, + "surface_azimuth": 175.0, + "userhorizon": [28.0, 34.0, 32.0, 60.0], + "peakpower": 13.110, + "pvtechchoice": "crystSi", + "mountingplace": "free", + "loss": 18.6, + "trackingtype": 0, + "optimal_surface_tilt": false, + "optimalangles": false, + "albedo": 0.25, + "module_model": null, + "inverter_model": null, + "inverter_paco": 15000, + "modules_per_string": 20, + "strings_per_inverter": 2 + } + ], + "provider_settings": null + }, + "weather": { + "provider": "WeatherImport", + "provider_settings": null + }, + "server": { + "host": "0.0.0.0", + "port": 8503, + "verbose": true, + "startup_eosdash": true, + "eosdash_host": "0.0.0.0", + "eosdash_port": 8504 + }, + "utils": {} +} diff --git a/tests/testdata/eos_config_andreas_now.json b/tests/testdata/eos_config_andreas_now.json new file mode 100644 index 0000000..3110997 --- /dev/null +++ b/tests/testdata/eos_config_andreas_now.json @@ -0,0 +1,100 @@ +{ + "general": { + "data_folder_path": null, + "data_output_subpath": "output", + "latitude": 52.5, + "longitude": 13.4 + }, + "cache": { + "subpath": "cache", + "cleanup_interval": 300.0 + }, + "ems": { + "startup_delay": 5.0, + "interval": 300.0 + }, + "logging": { + "console_level": "INFO" + }, + "devices": { + "batteries": [ + { + "device_id": "pv_akku", + "capacity_wh": 30000, + "charging_efficiency": 0.88, + "discharging_efficiency": 0.88, + "max_charge_power_w": 5000, + "min_soc_percentage": 0, + "max_soc_percentage": 100 + } + ], + "electric_vehicles": [ + { + "charge_rates": [0.0, 0.375, 0.5, 0.625, 0.75, 0.875, 1.0] + } + ], + "inverters": [], + "home_appliances": [] + }, + "measurement": { + "load_emr_keys": ["Household"] + }, + "optimization": { + "horizon_hours": 48, + "genetic": { + "penalties": { + "ev_soc_miss": 10 + } + } + }, + "prediction": { + "hours": 48, + "historic_hours": 48 + }, + "elecprice": { + "provider": "ElecPriceAkkudoktor", + "charges_kwh": 0.21 + }, + "load": { + "provider_settings": { + "LoadAkkudoktor": { + "loadakkudoktor_year_energy": 13000 + } + } + }, + "pvforecast": { + "provider": "PVForecastAkkudoktor", + "planes": [ + { + "surface_tilt": 87.907, + "surface_azimuth": 175.0, + "userhorizon": [28.0, 34.0, 32.0, 60.0], + "peakpower": 13.110, + "pvtechchoice": "crystSi", + "mountingplace": "free", + "loss": 18.6, + "trackingtype": 0, + "optimal_surface_tilt": false, + "optimalangles": false, + "albedo": 0.25, + "module_model": null, + "inverter_model": null, + "inverter_paco": 15000, + "modules_per_string": 20, + "strings_per_inverter": 2 + } + ] + }, + "weather": { + "provider": "WeatherImport" + }, + "server": { + "host": "0.0.0.0", + "port": 8503, + "verbose": true, + "startup_eosdash": true, + "eosdash_host": "0.0.0.0", + "eosdash_port": 8504 + }, + "utils": {} +} diff --git a/tests/testdata/eos_config_minimal_0_1_0.json b/tests/testdata/eos_config_minimal_0_1_0.json new file mode 100644 index 0000000..ac16967 --- /dev/null +++ b/tests/testdata/eos_config_minimal_0_1_0.json @@ -0,0 +1,24 @@ +{ + "elecprice": { + "charges_kwh": 0.21, + "provider": "ElecPriceImport" + }, + "prediction": { + "historic_hours": 48, + "hours": 48 + }, + "optimization": { + "hours": 48 + }, + "general": { + "latitude": 52.5, + "longitude": 13.4 + }, + "server": { + "startup_eosdash": true, + "host": "0.0.0.0", + "port": 8503, + "eosdash_host": "0.0.0.0", + "eosdash_port": 8504 + } +} diff --git a/tests/testdata/eos_config_minimal_now.json b/tests/testdata/eos_config_minimal_now.json new file mode 100644 index 0000000..3888a75 --- /dev/null +++ b/tests/testdata/eos_config_minimal_now.json @@ -0,0 +1,24 @@ +{ + "elecprice": { + "charges_kwh": 0.21, + "provider": "ElecPriceImport" + }, + "prediction": { + "historic_hours": 48, + "hours": 48 + }, + "optimization": { + "horizon_hours": 48 + }, + "general": { + "latitude": 52.5, + "longitude": 13.4 + }, + "server": { + "startup_eosdash": true, + "host": "0.0.0.0", + "port": 8503, + "eosdash_host": "0.0.0.0", + "eosdash_port": 8504 + } +} diff --git a/tests/testdata/eosserver_config_1.json b/tests/testdata/eosserver_config_1.json index 8d99f95..f7a0871 100644 --- a/tests/testdata/eosserver_config_1.json +++ b/tests/testdata/eosserver_config_1.json @@ -14,11 +14,13 @@ "load": { "provider": "LoadImport", "provider_settings": { - "loadakkudoktor_year_energy": 20000 + "LoadAkkudoktor": { + "loadakkudoktor_year_energy": 20000 + } } }, "optimization": { - "hours": 48 + "horizon_hours": 48 }, "pvforecast": { "planes": [ diff --git a/tests/testdata/optimize_input_1.json b/tests/testdata/optimize_input_1.json index 6e52328..86e5d04 100644 --- a/tests/testdata/optimize_input_1.json +++ b/tests/testdata/optimize_input_1.json @@ -65,5 +65,5 @@ 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 ] - } - +} + diff --git a/tests/testdata/weatherforecast_brightsky_1.json b/tests/testdata/weatherforecast_brightsky_1.json index a12cf45..b1f5942 100644 --- a/tests/testdata/weatherforecast_brightsky_1.json +++ b/tests/testdata/weatherforecast_brightsky_1.json @@ -21,13 +21,13 @@ "solar": null, "fallback_source_ids": { "cloud_cover": 219419, - "wind_speed": 219419, - "wind_direction": 219419, - "pressure_msl": 219419, "visibility": 219419, + "wind_gust_speed": 219419, + "wind_speed": 219419, "wind_gust_direction": 219419, + "pressure_msl": 219419, "sunshine": 219419, - "wind_gust_speed": 219419 + "wind_direction": 219419 }, "icon": "cloudy" }, @@ -52,13 +52,13 @@ "solar": null, "fallback_source_ids": { "cloud_cover": 219419, - "wind_speed": 219419, - "wind_direction": 219419, - "pressure_msl": 219419, "visibility": 219419, + "wind_gust_speed": 219419, + "wind_speed": 219419, "wind_gust_direction": 219419, + "pressure_msl": 219419, "sunshine": 219419, - "wind_gust_speed": 219419 + "wind_direction": 219419 }, "icon": "cloudy" }, @@ -83,13 +83,13 @@ "solar": null, "fallback_source_ids": { "cloud_cover": 219419, - "wind_speed": 219419, - "wind_direction": 219419, - "pressure_msl": 219419, "visibility": 219419, + "wind_gust_speed": 219419, + "wind_speed": 219419, "wind_gust_direction": 219419, + "pressure_msl": 219419, "sunshine": 219419, - "wind_gust_speed": 219419 + "wind_direction": 219419 }, "icon": "cloudy" }, @@ -114,13 +114,13 @@ "solar": null, "fallback_source_ids": { "cloud_cover": 219419, - "wind_speed": 219419, - "wind_direction": 219419, - "pressure_msl": 219419, "visibility": 219419, + "wind_gust_speed": 219419, + "wind_speed": 219419, "wind_gust_direction": 219419, + "pressure_msl": 219419, "sunshine": 219419, - "wind_gust_speed": 219419 + "wind_direction": 219419 }, "icon": "cloudy" }, @@ -145,13 +145,13 @@ "solar": null, "fallback_source_ids": { "cloud_cover": 219419, - "wind_speed": 219419, - "wind_direction": 219419, - "pressure_msl": 219419, "visibility": 219419, + "wind_gust_speed": 219419, + "wind_speed": 219419, "wind_gust_direction": 219419, + "pressure_msl": 219419, "sunshine": 219419, - "wind_gust_speed": 219419 + "wind_direction": 219419 }, "icon": "cloudy" }, @@ -176,13 +176,13 @@ "solar": null, "fallback_source_ids": { "cloud_cover": 219419, - "wind_speed": 219419, - "wind_direction": 219419, - "pressure_msl": 219419, "visibility": 219419, + "wind_gust_speed": 219419, + "wind_speed": 219419, "wind_gust_direction": 219419, + "pressure_msl": 219419, "sunshine": 219419, - "wind_gust_speed": 219419 + "wind_direction": 219419 }, "icon": "cloudy" }, @@ -207,13 +207,13 @@ "solar": null, "fallback_source_ids": { "cloud_cover": 219419, - "wind_speed": 219419, - "wind_direction": 219419, - "pressure_msl": 219419, "visibility": 219419, + "wind_gust_speed": 219419, + "wind_speed": 219419, "wind_gust_direction": 219419, + "pressure_msl": 219419, "sunshine": 219419, - "wind_gust_speed": 219419 + "wind_direction": 219419 }, "icon": "cloudy" }, @@ -238,13 +238,13 @@ "solar": null, "fallback_source_ids": { "cloud_cover": 219419, - "wind_speed": 219419, - "wind_direction": 219419, - "pressure_msl": 219419, "visibility": 219419, + "wind_gust_speed": 219419, + "wind_speed": 219419, "wind_gust_direction": 219419, + "pressure_msl": 219419, "sunshine": 219419, - "wind_gust_speed": 219419 + "wind_direction": 219419 }, "icon": "cloudy" }, @@ -269,13 +269,13 @@ "solar": null, "fallback_source_ids": { "cloud_cover": 219419, - "wind_speed": 219419, - "wind_direction": 219419, - "pressure_msl": 219419, "visibility": 219419, + "wind_gust_speed": 219419, + "wind_speed": 219419, "wind_gust_direction": 219419, + "pressure_msl": 219419, "sunshine": 219419, - "wind_gust_speed": 219419 + "wind_direction": 219419 }, "icon": "cloudy" }, @@ -300,13 +300,13 @@ "solar": null, "fallback_source_ids": { "cloud_cover": 219419, - "wind_speed": 219419, - "wind_direction": 219419, - "pressure_msl": 219419, "visibility": 219419, + "wind_gust_speed": 219419, + "wind_speed": 219419, "wind_gust_direction": 219419, + "pressure_msl": 219419, "sunshine": 219419, - "wind_gust_speed": 219419 + "wind_direction": 219419 }, "icon": "cloudy" }, @@ -331,13 +331,13 @@ "solar": null, "fallback_source_ids": { "cloud_cover": 219419, - "wind_speed": 219419, - "wind_direction": 219419, - "pressure_msl": 219419, "visibility": 219419, + "wind_gust_speed": 219419, + "wind_speed": 219419, "wind_gust_direction": 219419, + "pressure_msl": 219419, "sunshine": 219419, - "wind_gust_speed": 219419 + "wind_direction": 219419 }, "icon": "cloudy" }, @@ -362,13 +362,13 @@ "solar": null, "fallback_source_ids": { "cloud_cover": 219419, - "wind_speed": 219419, - "wind_direction": 219419, - "pressure_msl": 219419, "visibility": 219419, + "wind_gust_speed": 219419, + "wind_speed": 219419, "wind_gust_direction": 219419, + "pressure_msl": 219419, "sunshine": 219419, - "wind_gust_speed": 219419 + "wind_direction": 219419 }, "icon": "cloudy" }, @@ -393,13 +393,13 @@ "solar": null, "fallback_source_ids": { "cloud_cover": 219419, - "wind_speed": 219419, - "wind_direction": 219419, - "pressure_msl": 219419, "visibility": 219419, + "wind_gust_speed": 219419, + "wind_speed": 219419, "wind_gust_direction": 219419, + "pressure_msl": 219419, "sunshine": 219419, - "wind_gust_speed": 219419 + "wind_direction": 219419 }, "icon": "cloudy" }, @@ -424,13 +424,13 @@ "solar": null, "fallback_source_ids": { "cloud_cover": 219419, - "wind_speed": 219419, - "wind_direction": 219419, - "pressure_msl": 219419, "visibility": 219419, + "wind_gust_speed": 219419, + "wind_speed": 219419, "wind_gust_direction": 219419, + "pressure_msl": 219419, "sunshine": 219419, - "wind_gust_speed": 219419 + "wind_direction": 219419 }, "icon": "cloudy" }, @@ -455,13 +455,13 @@ "solar": null, "fallback_source_ids": { "cloud_cover": 219419, - "wind_speed": 219419, - "wind_direction": 219419, - "pressure_msl": 219419, "visibility": 219419, + "wind_gust_speed": 219419, + "wind_speed": 219419, "wind_gust_direction": 219419, + "pressure_msl": 219419, "sunshine": 219419, - "wind_gust_speed": 219419 + "wind_direction": 219419 }, "icon": "cloudy" }, @@ -486,13 +486,13 @@ "solar": null, "fallback_source_ids": { "cloud_cover": 219419, - "wind_speed": 219419, - "wind_direction": 219419, - "pressure_msl": 219419, "visibility": 219419, + "wind_gust_speed": 219419, + "wind_speed": 219419, "wind_gust_direction": 219419, + "pressure_msl": 219419, "sunshine": 219419, - "wind_gust_speed": 219419 + "wind_direction": 219419 }, "icon": "cloudy" }, @@ -517,13 +517,13 @@ "solar": null, "fallback_source_ids": { "cloud_cover": 219419, - "wind_speed": 219419, - "wind_direction": 219419, - "pressure_msl": 219419, "visibility": 219419, + "wind_gust_speed": 219419, + "wind_speed": 219419, "wind_gust_direction": 219419, + "pressure_msl": 219419, "sunshine": 219419, - "wind_gust_speed": 219419 + "wind_direction": 219419 }, "icon": "cloudy" }, @@ -548,13 +548,13 @@ "solar": null, "fallback_source_ids": { "cloud_cover": 219419, - "wind_speed": 219419, - "wind_direction": 219419, - "pressure_msl": 219419, "visibility": 219419, + "wind_gust_speed": 219419, + "wind_speed": 219419, "wind_gust_direction": 219419, + "pressure_msl": 219419, "sunshine": 219419, - "wind_gust_speed": 219419 + "wind_direction": 219419 }, "icon": "cloudy" }, @@ -579,13 +579,13 @@ "solar": null, "fallback_source_ids": { "cloud_cover": 219419, - "wind_speed": 219419, - "wind_direction": 219419, - "pressure_msl": 219419, "visibility": 219419, + "wind_gust_speed": 219419, + "wind_speed": 219419, "wind_gust_direction": 219419, + "pressure_msl": 219419, "sunshine": 219419, - "wind_gust_speed": 219419 + "wind_direction": 219419 }, "icon": "partly-cloudy-day" }, @@ -610,13 +610,13 @@ "solar": null, "fallback_source_ids": { "cloud_cover": 219419, - "wind_speed": 219419, - "wind_direction": 219419, - "pressure_msl": 219419, "visibility": 219419, + "wind_gust_speed": 219419, + "wind_speed": 219419, "wind_gust_direction": 219419, + "pressure_msl": 219419, "sunshine": 219419, - "wind_gust_speed": 219419 + "wind_direction": 219419 }, "icon": "cloudy" }, @@ -641,13 +641,13 @@ "solar": null, "fallback_source_ids": { "cloud_cover": 219419, - "wind_speed": 219419, - "wind_direction": 219419, - "pressure_msl": 219419, "visibility": 219419, + "wind_gust_speed": 219419, + "wind_speed": 219419, "wind_gust_direction": 219419, + "pressure_msl": 219419, "sunshine": 219419, - "wind_gust_speed": 219419 + "wind_direction": 219419 }, "icon": "cloudy" }, @@ -672,13 +672,13 @@ "solar": null, "fallback_source_ids": { "cloud_cover": 219419, - "wind_speed": 219419, - "wind_direction": 219419, - "pressure_msl": 219419, "visibility": 219419, + "wind_gust_speed": 219419, + "wind_speed": 219419, "wind_gust_direction": 219419, + "pressure_msl": 219419, "sunshine": 219419, - "wind_gust_speed": 219419 + "wind_direction": 219419 }, "icon": "cloudy" }, @@ -703,13 +703,13 @@ "solar": null, "fallback_source_ids": { "cloud_cover": 219419, - "wind_speed": 219419, - "wind_direction": 219419, - "pressure_msl": 219419, "visibility": 219419, + "wind_gust_speed": 219419, + "wind_speed": 219419, "wind_gust_direction": 219419, + "pressure_msl": 219419, "sunshine": 219419, - "wind_gust_speed": 219419 + "wind_direction": 219419 }, "icon": "cloudy" }, @@ -734,13 +734,13 @@ "solar": null, "fallback_source_ids": { "cloud_cover": 219419, - "wind_speed": 219419, - "wind_direction": 219419, - "pressure_msl": 219419, "visibility": 219419, + "wind_gust_speed": 219419, + "wind_speed": 219419, "wind_gust_direction": 219419, + "pressure_msl": 219419, "sunshine": 219419, - "wind_gust_speed": 219419 + "wind_direction": 219419 }, "icon": "cloudy" }, @@ -765,13 +765,13 @@ "solar": null, "fallback_source_ids": { "cloud_cover": 219419, - "wind_speed": 219419, - "wind_direction": 219419, - "pressure_msl": 219419, "visibility": 219419, + "wind_gust_speed": 219419, + "wind_speed": 219419, "wind_gust_direction": 219419, + "pressure_msl": 219419, "sunshine": 219419, - "wind_gust_speed": 219419 + "wind_direction": 219419 }, "icon": "cloudy" }, @@ -796,13 +796,13 @@ "solar": null, "fallback_source_ids": { "cloud_cover": 219419, - "wind_speed": 219419, - "wind_direction": 219419, - "pressure_msl": 219419, "visibility": 219419, + "wind_gust_speed": 219419, + "wind_speed": 219419, "wind_gust_direction": 219419, + "pressure_msl": 219419, "sunshine": 219419, - "wind_gust_speed": 219419 + "wind_direction": 219419 }, "icon": "cloudy" }, @@ -827,13 +827,13 @@ "solar": null, "fallback_source_ids": { "cloud_cover": 219419, - "wind_speed": 219419, - "wind_direction": 219419, - "pressure_msl": 219419, "visibility": 219419, + "wind_gust_speed": 219419, + "wind_speed": 219419, "wind_gust_direction": 219419, + "pressure_msl": 219419, "sunshine": 219419, - "wind_gust_speed": 219419 + "wind_direction": 219419 }, "icon": "cloudy" }, @@ -858,13 +858,13 @@ "solar": null, "fallback_source_ids": { "cloud_cover": 219419, - "wind_speed": 219419, - "wind_direction": 219419, - "pressure_msl": 219419, "visibility": 219419, + "wind_gust_speed": 219419, + "wind_speed": 219419, "wind_gust_direction": 219419, + "pressure_msl": 219419, "sunshine": 219419, - "wind_gust_speed": 219419 + "wind_direction": 219419 }, "icon": "cloudy" }, @@ -889,13 +889,13 @@ "solar": null, "fallback_source_ids": { "cloud_cover": 219419, - "wind_speed": 219419, - "wind_direction": 219419, - "pressure_msl": 219419, "visibility": 219419, + "wind_gust_speed": 219419, + "wind_speed": 219419, "wind_gust_direction": 219419, + "pressure_msl": 219419, "sunshine": 219419, - "wind_gust_speed": 219419 + "wind_direction": 219419 }, "icon": "cloudy" }, @@ -920,13 +920,13 @@ "solar": null, "fallback_source_ids": { "cloud_cover": 219419, - "wind_speed": 219419, - "wind_direction": 219419, - "pressure_msl": 219419, "visibility": 219419, + "wind_gust_speed": 219419, + "wind_speed": 219419, "wind_gust_direction": 219419, + "pressure_msl": 219419, "sunshine": 219419, - "wind_gust_speed": 219419 + "wind_direction": 219419 }, "icon": "cloudy" }, @@ -951,13 +951,13 @@ "solar": null, "fallback_source_ids": { "cloud_cover": 219419, - "wind_speed": 219419, - "wind_direction": 219419, - "pressure_msl": 219419, "visibility": 219419, + "wind_gust_speed": 219419, + "wind_speed": 219419, "wind_gust_direction": 219419, + "pressure_msl": 219419, "sunshine": 219419, - "wind_gust_speed": 219419 + "wind_direction": 219419 }, "icon": "cloudy" }, @@ -982,13 +982,13 @@ "solar": null, "fallback_source_ids": { "cloud_cover": 219419, - "wind_speed": 219419, - "wind_direction": 219419, - "pressure_msl": 219419, "visibility": 219419, + "wind_gust_speed": 219419, + "wind_speed": 219419, "wind_gust_direction": 219419, + "pressure_msl": 219419, "sunshine": 219419, - "wind_gust_speed": 219419 + "wind_direction": 219419 }, "icon": "cloudy" }, @@ -1013,13 +1013,13 @@ "solar": null, "fallback_source_ids": { "cloud_cover": 219419, - "wind_speed": 219419, - "wind_direction": 219419, - "pressure_msl": 219419, "visibility": 219419, + "wind_gust_speed": 219419, + "wind_speed": 219419, "wind_gust_direction": 219419, + "pressure_msl": 219419, "sunshine": 219419, - "wind_gust_speed": 219419 + "wind_direction": 219419 }, "icon": "cloudy" }, @@ -1044,13 +1044,13 @@ "solar": null, "fallback_source_ids": { "cloud_cover": 219419, - "wind_speed": 219419, - "wind_direction": 219419, - "pressure_msl": 219419, "visibility": 219419, + "wind_gust_speed": 219419, + "wind_speed": 219419, "wind_gust_direction": 219419, + "pressure_msl": 219419, "sunshine": 219419, - "wind_gust_speed": 219419 + "wind_direction": 219419 }, "icon": "cloudy" }, @@ -1075,13 +1075,13 @@ "solar": null, "fallback_source_ids": { "cloud_cover": 219419, - "wind_speed": 219419, - "wind_direction": 219419, - "pressure_msl": 219419, "visibility": 219419, + "wind_gust_speed": 219419, + "wind_speed": 219419, "wind_gust_direction": 219419, + "pressure_msl": 219419, "sunshine": 219419, - "wind_gust_speed": 219419 + "wind_direction": 219419 }, "icon": "cloudy" }, @@ -1106,13 +1106,13 @@ "solar": null, "fallback_source_ids": { "cloud_cover": 219419, - "wind_speed": 219419, - "wind_direction": 219419, - "pressure_msl": 219419, "visibility": 219419, + "wind_gust_speed": 219419, + "wind_speed": 219419, "wind_gust_direction": 219419, + "pressure_msl": 219419, "sunshine": 219419, - "wind_gust_speed": 219419 + "wind_direction": 219419 }, "icon": "cloudy" }, @@ -1137,13 +1137,13 @@ "solar": null, "fallback_source_ids": { "cloud_cover": 219419, - "wind_speed": 219419, - "wind_direction": 219419, - "pressure_msl": 219419, "visibility": 219419, + "wind_gust_speed": 219419, + "wind_speed": 219419, "wind_gust_direction": 219419, + "pressure_msl": 219419, "sunshine": 219419, - "wind_gust_speed": 219419 + "wind_direction": 219419 }, "icon": "cloudy" }, @@ -1168,13 +1168,13 @@ "solar": null, "fallback_source_ids": { "cloud_cover": 219419, - "wind_speed": 219419, - "wind_direction": 219419, - "pressure_msl": 219419, "visibility": 219419, + "wind_gust_speed": 219419, + "wind_speed": 219419, "wind_gust_direction": 219419, + "pressure_msl": 219419, "sunshine": 219419, - "wind_gust_speed": 219419 + "wind_direction": 219419 }, "icon": "cloudy" }, @@ -1199,13 +1199,13 @@ "solar": null, "fallback_source_ids": { "cloud_cover": 219419, - "wind_speed": 219419, - "wind_direction": 219419, - "pressure_msl": 219419, "visibility": 219419, + "wind_gust_speed": 219419, + "wind_speed": 219419, "wind_gust_direction": 219419, + "pressure_msl": 219419, "sunshine": 219419, - "wind_gust_speed": 219419 + "wind_direction": 219419 }, "icon": "cloudy" }, @@ -1230,13 +1230,13 @@ "solar": null, "fallback_source_ids": { "cloud_cover": 219419, - "wind_speed": 219419, - "wind_direction": 219419, - "pressure_msl": 219419, "visibility": 219419, + "wind_gust_speed": 219419, + "wind_speed": 219419, "wind_gust_direction": 219419, + "pressure_msl": 219419, "sunshine": 219419, - "wind_gust_speed": 219419 + "wind_direction": 219419 }, "icon": "cloudy" }, @@ -1261,13 +1261,13 @@ "solar": null, "fallback_source_ids": { "cloud_cover": 219419, - "wind_speed": 219419, - "wind_direction": 219419, - "pressure_msl": 219419, "visibility": 219419, + "wind_gust_speed": 219419, + "wind_speed": 219419, "wind_gust_direction": 219419, + "pressure_msl": 219419, "sunshine": 219419, - "wind_gust_speed": 219419 + "wind_direction": 219419 }, "icon": "cloudy" }, @@ -1292,13 +1292,13 @@ "solar": null, "fallback_source_ids": { "cloud_cover": 219419, - "wind_speed": 219419, - "wind_direction": 219419, - "pressure_msl": 219419, "visibility": 219419, + "wind_gust_speed": 219419, + "wind_speed": 219419, "wind_gust_direction": 219419, + "pressure_msl": 219419, "sunshine": 219419, - "wind_gust_speed": 219419 + "wind_direction": 219419 }, "icon": "cloudy" }, @@ -1323,13 +1323,13 @@ "solar": null, "fallback_source_ids": { "cloud_cover": 219419, - "wind_speed": 219419, - "wind_direction": 219419, - "pressure_msl": 219419, "visibility": 219419, + "wind_gust_speed": 219419, + "wind_speed": 219419, "wind_gust_direction": 219419, + "pressure_msl": 219419, "sunshine": 219419, - "wind_gust_speed": 219419 + "wind_direction": 219419 }, "icon": "cloudy" }, @@ -1354,13 +1354,13 @@ "solar": null, "fallback_source_ids": { "cloud_cover": 219419, - "wind_speed": 219419, - "wind_direction": 219419, - "pressure_msl": 219419, "visibility": 219419, + "wind_gust_speed": 219419, + "wind_speed": 219419, "wind_gust_direction": 219419, + "pressure_msl": 219419, "sunshine": 219419, - "wind_gust_speed": 219419 + "wind_direction": 219419 }, "icon": "cloudy" }, @@ -1385,13 +1385,13 @@ "solar": null, "fallback_source_ids": { "cloud_cover": 219419, - "wind_speed": 219419, - "wind_direction": 219419, - "pressure_msl": 219419, "visibility": 219419, + "wind_gust_speed": 219419, + "wind_speed": 219419, "wind_gust_direction": 219419, + "pressure_msl": 219419, "sunshine": 219419, - "wind_gust_speed": 219419 + "wind_direction": 219419 }, "icon": "cloudy" }, @@ -1416,13 +1416,13 @@ "solar": null, "fallback_source_ids": { "cloud_cover": 219419, - "wind_speed": 219419, - "wind_direction": 219419, - "pressure_msl": 219419, "visibility": 219419, + "wind_gust_speed": 219419, + "wind_speed": 219419, "wind_gust_direction": 219419, + "pressure_msl": 219419, "sunshine": 219419, - "wind_gust_speed": 219419 + "wind_direction": 219419 }, "icon": "cloudy" }, @@ -1447,13 +1447,13 @@ "solar": null, "fallback_source_ids": { "cloud_cover": 219419, - "wind_speed": 219419, - "wind_direction": 219419, - "pressure_msl": 219419, "visibility": 219419, + "wind_gust_speed": 219419, + "wind_speed": 219419, "wind_gust_direction": 219419, + "pressure_msl": 219419, "sunshine": 219419, - "wind_gust_speed": 219419 + "wind_direction": 219419 }, "icon": "cloudy" }, @@ -1478,13 +1478,13 @@ "solar": null, "fallback_source_ids": { "cloud_cover": 219419, - "wind_speed": 219419, - "wind_direction": 219419, - "pressure_msl": 219419, "visibility": 219419, + "wind_gust_speed": 219419, + "wind_speed": 219419, "wind_gust_direction": 219419, + "pressure_msl": 219419, "sunshine": 219419, - "wind_gust_speed": 219419 + "wind_direction": 219419 }, "icon": "cloudy" }, @@ -1509,13 +1509,13 @@ "solar": null, "fallback_source_ids": { "cloud_cover": 219419, - "wind_speed": 219419, - "wind_direction": 219419, - "pressure_msl": 219419, "visibility": 219419, + "wind_gust_speed": 219419, + "wind_speed": 219419, "wind_gust_direction": 219419, + "pressure_msl": 219419, "sunshine": 219419, - "wind_gust_speed": 219419 + "wind_direction": 219419 }, "icon": "cloudy" }, @@ -1540,13 +1540,13 @@ "solar": null, "fallback_source_ids": { "cloud_cover": 219419, - "wind_speed": 219419, - "wind_direction": 219419, - "pressure_msl": 219419, "visibility": 219419, + "wind_gust_speed": 219419, + "wind_speed": 219419, "wind_gust_direction": 219419, + "pressure_msl": 219419, "sunshine": 219419, - "wind_gust_speed": 219419 + "wind_direction": 219419 }, "icon": "cloudy" } @@ -1561,8 +1561,8 @@ "height": 216.5, "station_name": "Arnstein-M\u00fcdesheim", "wmo_station_id": "P125", - "first_record": "2010-01-01T00:00:00+00:00", - "last_record": "2025-02-13T23:00:00+00:00", + "first_record": "2010-01-01T01:00:00+01:00", + "last_record": "2025-10-23T01:00:00+02:00", "distance": 7199.0 }, { @@ -1574,8 +1574,8 @@ "height": 281.73, "station_name": "Kissingen, Bad", "wmo_station_id": "10658", - "first_record": "2010-01-01T00:00:00+00:00", - "last_record": "2025-02-14T23:00:00+00:00", + "first_record": "2010-01-01T01:00:00+01:00", + "last_record": "2025-10-23T01:00:00+02:00", "distance": 25569.0 } ] diff --git a/tests/testdata/weatherforecast_brightsky_2.json b/tests/testdata/weatherforecast_brightsky_2.json index bf00e74..ff3f99e 100644 --- a/tests/testdata/weatherforecast_brightsky_2.json +++ b/tests/testdata/weatherforecast_brightsky_2.json @@ -2,6 +2,7 @@ "records": [ { "date_time": "2024-10-26 00:00:00+02:00", + "configured_data": {}, "weather_total_clouds": 100.0, "weather_low_clouds": null, "weather_medium_clouds": null, @@ -23,11 +24,11 @@ "weather_ozone": null, "weather_ghi": 0.0, "weather_dni": 0.0, - "weather_dhi": 0.0, - "start_datetime": "2024-10-26 00:00:00+02:00" + "weather_dhi": 0.0 }, { "date_time": "2024-10-26 01:00:00+02:00", + "configured_data": {}, "weather_total_clouds": 100.0, "weather_low_clouds": null, "weather_medium_clouds": null, @@ -49,11 +50,11 @@ "weather_ozone": null, "weather_ghi": 0.0, "weather_dni": 0.0, - "weather_dhi": 0.0, - "start_datetime": "2024-10-26 00:00:00+02:00" + "weather_dhi": 0.0 }, { "date_time": "2024-10-26 02:00:00+02:00", + "configured_data": {}, "weather_total_clouds": 100.0, "weather_low_clouds": null, "weather_medium_clouds": null, @@ -75,11 +76,11 @@ "weather_ozone": null, "weather_ghi": 0.0, "weather_dni": 0.0, - "weather_dhi": 0.0, - "start_datetime": "2024-10-26 00:00:00+02:00" + "weather_dhi": 0.0 }, { "date_time": "2024-10-26 03:00:00+02:00", + "configured_data": {}, "weather_total_clouds": 100.0, "weather_low_clouds": null, "weather_medium_clouds": null, @@ -101,11 +102,11 @@ "weather_ozone": null, "weather_ghi": 0.0, "weather_dni": 0.0, - "weather_dhi": 0.0, - "start_datetime": "2024-10-26 00:00:00+02:00" + "weather_dhi": 0.0 }, { "date_time": "2024-10-26 04:00:00+02:00", + "configured_data": {}, "weather_total_clouds": 100.0, "weather_low_clouds": null, "weather_medium_clouds": null, @@ -127,11 +128,11 @@ "weather_ozone": null, "weather_ghi": 0.0, "weather_dni": 0.0, - "weather_dhi": 0.0, - "start_datetime": "2024-10-26 00:00:00+02:00" + "weather_dhi": 0.0 }, { "date_time": "2024-10-26 05:00:00+02:00", + "configured_data": {}, "weather_total_clouds": 87.0, "weather_low_clouds": null, "weather_medium_clouds": null, @@ -153,11 +154,11 @@ "weather_ozone": null, "weather_ghi": 0.0, "weather_dni": 0.0, - "weather_dhi": 0.0, - "start_datetime": "2024-10-26 00:00:00+02:00" + "weather_dhi": 0.0 }, { "date_time": "2024-10-26 06:00:00+02:00", + "configured_data": {}, "weather_total_clouds": 100.0, "weather_low_clouds": null, "weather_medium_clouds": null, @@ -179,11 +180,11 @@ "weather_ozone": null, "weather_ghi": 0.0, "weather_dni": 0.0, - "weather_dhi": 0.0, - "start_datetime": "2024-10-26 00:00:00+02:00" + "weather_dhi": 0.0 }, { "date_time": "2024-10-26 07:00:00+02:00", + "configured_data": {}, "weather_total_clouds": 100.0, "weather_low_clouds": null, "weather_medium_clouds": null, @@ -205,11 +206,11 @@ "weather_ozone": null, "weather_ghi": 0.0, "weather_dni": 0.0, - "weather_dhi": 0.0, - "start_datetime": "2024-10-26 00:00:00+02:00" + "weather_dhi": 0.0 }, { "date_time": "2024-10-26 08:00:00+02:00", + "configured_data": {}, "weather_total_clouds": 100.0, "weather_low_clouds": null, "weather_medium_clouds": null, @@ -231,11 +232,11 @@ "weather_ozone": null, "weather_ghi": 0.0, "weather_dni": 0.0, - "weather_dhi": 0.0, - "start_datetime": "2024-10-26 00:00:00+02:00" + "weather_dhi": 0.0 }, { "date_time": "2024-10-26 09:00:00+02:00", + "configured_data": {}, "weather_total_clouds": 100.0, "weather_low_clouds": null, "weather_medium_clouds": null, @@ -257,11 +258,11 @@ "weather_ozone": null, "weather_ghi": 22.22705922303379, "weather_dni": 0.0, - "weather_dhi": 22.22705922303379, - "start_datetime": "2024-10-26 00:00:00+02:00" + "weather_dhi": 22.22705922303379 }, { "date_time": "2024-10-26 10:00:00+02:00", + "configured_data": {}, "weather_total_clouds": 100.0, "weather_low_clouds": null, "weather_medium_clouds": null, @@ -283,11 +284,11 @@ "weather_ozone": null, "weather_ghi": 68.16265202099999, "weather_dni": 0.0, - "weather_dhi": 68.16265202099999, - "start_datetime": "2024-10-26 00:00:00+02:00" + "weather_dhi": 68.16265202099999 }, { "date_time": "2024-10-26 11:00:00+02:00", + "configured_data": {}, "weather_total_clouds": 100.0, "weather_low_clouds": null, "weather_medium_clouds": null, @@ -309,11 +310,11 @@ "weather_ozone": null, "weather_ghi": 108.0100746278567, "weather_dni": 0.0, - "weather_dhi": 108.0100746278567, - "start_datetime": "2024-10-26 00:00:00+02:00" + "weather_dhi": 108.0100746278567 }, { "date_time": "2024-10-26 12:00:00+02:00", + "configured_data": {}, "weather_total_clouds": 100.0, "weather_low_clouds": null, "weather_medium_clouds": null, @@ -335,11 +336,11 @@ "weather_ozone": null, "weather_ghi": 134.2816493853918, "weather_dni": 0.0, - "weather_dhi": 134.2816493853918, - "start_datetime": "2024-10-26 00:00:00+02:00" + "weather_dhi": 134.2816493853918 }, { "date_time": "2024-10-26 13:00:00+02:00", + "configured_data": {}, "weather_total_clouds": 100.0, "weather_low_clouds": null, "weather_medium_clouds": null, @@ -361,11 +362,11 @@ "weather_ozone": null, "weather_ghi": 144.04237088707308, "weather_dni": 0.0, - "weather_dhi": 144.04237088707308, - "start_datetime": "2024-10-26 00:00:00+02:00" + "weather_dhi": 144.04237088707308 }, { "date_time": "2024-10-26 14:00:00+02:00", + "configured_data": {}, "weather_total_clouds": 100.0, "weather_low_clouds": null, "weather_medium_clouds": null, @@ -387,11 +388,11 @@ "weather_ozone": null, "weather_ghi": 136.35519419190516, "weather_dni": 0.0, - "weather_dhi": 136.35519419190516, - "start_datetime": "2024-10-26 00:00:00+02:00" + "weather_dhi": 136.35519419190516 }, { "date_time": "2024-10-26 15:00:00+02:00", + "configured_data": {}, "weather_total_clouds": 100.0, "weather_low_clouds": null, "weather_medium_clouds": null, @@ -413,11 +414,11 @@ "weather_ozone": null, "weather_ghi": 111.94730962791996, "weather_dni": 0.0, - "weather_dhi": 111.94730962791996, - "start_datetime": "2024-10-26 00:00:00+02:00" + "weather_dhi": 111.94730962791996 }, { "date_time": "2024-10-26 16:00:00+02:00", + "configured_data": {}, "weather_total_clouds": 100.0, "weather_low_clouds": null, "weather_medium_clouds": null, @@ -439,11 +440,11 @@ "weather_ozone": null, "weather_ghi": 73.45834328182735, "weather_dni": 0.0, - "weather_dhi": 73.45834328182735, - "start_datetime": "2024-10-26 00:00:00+02:00" + "weather_dhi": 73.45834328182735 }, { "date_time": "2024-10-26 17:00:00+02:00", + "configured_data": {}, "weather_total_clouds": 87.0, "weather_low_clouds": null, "weather_medium_clouds": null, @@ -465,11 +466,11 @@ "weather_ozone": null, "weather_ghi": 34.07062080450064, "weather_dni": 0.0, - "weather_dhi": 34.07062080450064, - "start_datetime": "2024-10-26 00:00:00+02:00" + "weather_dhi": 34.07062080450064 }, { "date_time": "2024-10-26 18:00:00+02:00", + "configured_data": {}, "weather_total_clouds": 62.0, "weather_low_clouds": null, "weather_medium_clouds": null, @@ -491,11 +492,11 @@ "weather_ozone": null, "weather_ghi": 0.11256372587508819, "weather_dni": 0.0, - "weather_dhi": 0.11256372587508819, - "start_datetime": "2024-10-26 00:00:00+02:00" + "weather_dhi": 0.11256372587508819 }, { "date_time": "2024-10-26 19:00:00+02:00", + "configured_data": {}, "weather_total_clouds": 100.0, "weather_low_clouds": null, "weather_medium_clouds": null, @@ -517,11 +518,11 @@ "weather_ozone": null, "weather_ghi": 0.0, "weather_dni": 0.0, - "weather_dhi": 0.0, - "start_datetime": "2024-10-26 00:00:00+02:00" + "weather_dhi": 0.0 }, { "date_time": "2024-10-26 20:00:00+02:00", + "configured_data": {}, "weather_total_clouds": 100.0, "weather_low_clouds": null, "weather_medium_clouds": null, @@ -543,11 +544,11 @@ "weather_ozone": null, "weather_ghi": 0.0, "weather_dni": 0.0, - "weather_dhi": 0.0, - "start_datetime": "2024-10-26 00:00:00+02:00" + "weather_dhi": 0.0 }, { "date_time": "2024-10-26 21:00:00+02:00", + "configured_data": {}, "weather_total_clouds": 87.0, "weather_low_clouds": null, "weather_medium_clouds": null, @@ -569,11 +570,11 @@ "weather_ozone": null, "weather_ghi": 0.0, "weather_dni": 0.0, - "weather_dhi": 0.0, - "start_datetime": "2024-10-26 00:00:00+02:00" + "weather_dhi": 0.0 }, { "date_time": "2024-10-26 22:00:00+02:00", + "configured_data": {}, "weather_total_clouds": 87.0, "weather_low_clouds": null, "weather_medium_clouds": null, @@ -595,11 +596,11 @@ "weather_ozone": null, "weather_ghi": 0.0, "weather_dni": 0.0, - "weather_dhi": 0.0, - "start_datetime": "2024-10-26 00:00:00+02:00" + "weather_dhi": 0.0 }, { "date_time": "2024-10-26 23:00:00+02:00", + "configured_data": {}, "weather_total_clouds": 87.0, "weather_low_clouds": null, "weather_medium_clouds": null, @@ -621,11 +622,11 @@ "weather_ozone": null, "weather_ghi": 0.0, "weather_dni": 0.0, - "weather_dhi": 0.0, - "start_datetime": "2024-10-26 00:00:00+02:00" + "weather_dhi": 0.0 }, { "date_time": "2024-10-27 00:00:00+02:00", + "configured_data": {}, "weather_total_clouds": 100.0, "weather_low_clouds": null, "weather_medium_clouds": null, @@ -647,11 +648,11 @@ "weather_ozone": null, "weather_ghi": 0.0, "weather_dni": 0.0, - "weather_dhi": 0.0, - "start_datetime": "2024-10-26 00:00:00+02:00" + "weather_dhi": 0.0 }, { "date_time": "2024-10-27 01:00:00+02:00", + "configured_data": {}, "weather_total_clouds": 100.0, "weather_low_clouds": null, "weather_medium_clouds": null, @@ -673,11 +674,11 @@ "weather_ozone": null, "weather_ghi": 0.0, "weather_dni": 0.0, - "weather_dhi": 0.0, - "start_datetime": "2024-10-26 00:00:00+02:00" + "weather_dhi": 0.0 }, { "date_time": "2024-10-27 02:00:00+02:00", + "configured_data": {}, "weather_total_clouds": 100.0, "weather_low_clouds": null, "weather_medium_clouds": null, @@ -699,11 +700,11 @@ "weather_ozone": null, "weather_ghi": 0.0, "weather_dni": 0.0, - "weather_dhi": 0.0, - "start_datetime": "2024-10-26 00:00:00+02:00" + "weather_dhi": 0.0 }, { "date_time": "2024-10-27 02:00:00+01:00", + "configured_data": {}, "weather_total_clouds": 100.0, "weather_low_clouds": null, "weather_medium_clouds": null, @@ -725,11 +726,11 @@ "weather_ozone": null, "weather_ghi": null, "weather_dni": null, - "weather_dhi": null, - "start_datetime": "2024-10-26 00:00:00+02:00" + "weather_dhi": null }, { "date_time": "2024-10-27 03:00:00+01:00", + "configured_data": {}, "weather_total_clouds": 100.0, "weather_low_clouds": null, "weather_medium_clouds": null, @@ -751,11 +752,11 @@ "weather_ozone": null, "weather_ghi": 0.0, "weather_dni": 0.0, - "weather_dhi": 0.0, - "start_datetime": "2024-10-26 00:00:00+02:00" + "weather_dhi": 0.0 }, { "date_time": "2024-10-27 04:00:00+01:00", + "configured_data": {}, "weather_total_clouds": 100.0, "weather_low_clouds": null, "weather_medium_clouds": null, @@ -777,11 +778,11 @@ "weather_ozone": null, "weather_ghi": 0.0, "weather_dni": 0.0, - "weather_dhi": 0.0, - "start_datetime": "2024-10-26 00:00:00+02:00" + "weather_dhi": 0.0 }, { "date_time": "2024-10-27 05:00:00+01:00", + "configured_data": {}, "weather_total_clouds": 100.0, "weather_low_clouds": null, "weather_medium_clouds": null, @@ -803,11 +804,11 @@ "weather_ozone": null, "weather_ghi": 0.0, "weather_dni": 0.0, - "weather_dhi": 0.0, - "start_datetime": "2024-10-26 00:00:00+02:00" + "weather_dhi": 0.0 }, { "date_time": "2024-10-27 06:00:00+01:00", + "configured_data": {}, "weather_total_clouds": 100.0, "weather_low_clouds": null, "weather_medium_clouds": null, @@ -829,11 +830,11 @@ "weather_ozone": null, "weather_ghi": 0.0, "weather_dni": 0.0, - "weather_dhi": 0.0, - "start_datetime": "2024-10-26 00:00:00+02:00" + "weather_dhi": 0.0 }, { "date_time": "2024-10-27 07:00:00+01:00", + "configured_data": {}, "weather_total_clouds": 100.0, "weather_low_clouds": null, "weather_medium_clouds": null, @@ -855,11 +856,11 @@ "weather_ozone": null, "weather_ghi": 0.0, "weather_dni": 0.0, - "weather_dhi": 0.0, - "start_datetime": "2024-10-26 00:00:00+02:00" + "weather_dhi": 0.0 }, { "date_time": "2024-10-27 08:00:00+01:00", + "configured_data": {}, "weather_total_clouds": 100.0, "weather_low_clouds": null, "weather_medium_clouds": null, @@ -881,11 +882,11 @@ "weather_ozone": null, "weather_ghi": 20.901591088639343, "weather_dni": 0.0, - "weather_dhi": 20.901591088639343, - "start_datetime": "2024-10-26 00:00:00+02:00" + "weather_dhi": 20.901591088639343 }, { "date_time": "2024-10-27 09:00:00+01:00", + "configured_data": {}, "weather_total_clouds": 100.0, "weather_low_clouds": null, "weather_medium_clouds": null, @@ -907,11 +908,11 @@ "weather_ozone": null, "weather_ghi": 66.41841804602629, "weather_dni": 0.0, - "weather_dhi": 66.41841804602629, - "start_datetime": "2024-10-26 00:00:00+02:00" + "weather_dhi": 66.41841804602629 }, { "date_time": "2024-10-27 10:00:00+01:00", + "configured_data": {}, "weather_total_clouds": 100.0, "weather_low_clouds": null, "weather_medium_clouds": null, @@ -933,11 +934,11 @@ "weather_ozone": null, "weather_ghi": 106.12345605852113, "weather_dni": 0.0, - "weather_dhi": 106.12345605852113, - "start_datetime": "2024-10-26 00:00:00+02:00" + "weather_dhi": 106.12345605852113 }, { "date_time": "2024-10-27 11:00:00+01:00", + "configured_data": {}, "weather_total_clouds": 100.0, "weather_low_clouds": null, "weather_medium_clouds": null, @@ -959,11 +960,11 @@ "weather_ozone": null, "weather_ghi": 132.31929512932624, "weather_dni": 0.0, - "weather_dhi": 132.31929512932624, - "start_datetime": "2024-10-26 00:00:00+02:00" + "weather_dhi": 132.31929512932624 }, { "date_time": "2024-10-27 12:00:00+01:00", + "configured_data": {}, "weather_total_clouds": 100.0, "weather_low_clouds": null, "weather_medium_clouds": null, @@ -985,11 +986,11 @@ "weather_ozone": null, "weather_ghi": 142.03807516868267, "weather_dni": 0.0, - "weather_dhi": 142.03807516868267, - "start_datetime": "2024-10-26 00:00:00+02:00" + "weather_dhi": 142.03807516868267 }, { "date_time": "2024-10-27 13:00:00+01:00", + "configured_data": {}, "weather_total_clouds": 100.0, "weather_low_clouds": null, "weather_medium_clouds": null, @@ -1011,11 +1012,11 @@ "weather_ozone": null, "weather_ghi": 134.33853283469773, "weather_dni": 0.0, - "weather_dhi": 134.33853283469773, - "start_datetime": "2024-10-26 00:00:00+02:00" + "weather_dhi": 134.33853283469773 }, { "date_time": "2024-10-27 14:00:00+01:00", + "configured_data": {}, "weather_total_clouds": 100.0, "weather_low_clouds": null, "weather_medium_clouds": null, @@ -1037,11 +1038,11 @@ "weather_ozone": null, "weather_ghi": 109.95561941571053, "weather_dni": 0.0, - "weather_dhi": 109.95561941571053, - "start_datetime": "2024-10-26 00:00:00+02:00" + "weather_dhi": 109.95561941571053 }, { "date_time": "2024-10-27 15:00:00+01:00", + "configured_data": {}, "weather_total_clouds": 87.0, "weather_low_clouds": null, "weather_medium_clouds": null, @@ -1063,11 +1064,11 @@ "weather_ozone": null, "weather_ghi": 88.84019629738314, "weather_dni": 0.0, - "weather_dhi": 88.84019629738314, - "start_datetime": "2024-10-26 00:00:00+02:00" + "weather_dhi": 88.84019629738314 }, { "date_time": "2024-10-27 16:00:00+01:00", + "configured_data": {}, "weather_total_clouds": 100.0, "weather_low_clouds": null, "weather_medium_clouds": null, @@ -1089,11 +1090,11 @@ "weather_ozone": null, "weather_ghi": 25.90303659201319, "weather_dni": 0.0, - "weather_dhi": 25.90303659201319, - "start_datetime": "2024-10-26 00:00:00+02:00" + "weather_dhi": 25.90303659201319 }, { "date_time": "2024-10-27 17:00:00+01:00", + "configured_data": {}, "weather_total_clouds": 100.0, "weather_low_clouds": null, "weather_medium_clouds": null, @@ -1115,11 +1116,11 @@ "weather_ozone": null, "weather_ghi": 0.027781191847857583, "weather_dni": 0.0, - "weather_dhi": 0.027781191847857583, - "start_datetime": "2024-10-26 00:00:00+02:00" + "weather_dhi": 0.027781191847857583 }, { "date_time": "2024-10-27 18:00:00+01:00", + "configured_data": {}, "weather_total_clouds": 100.0, "weather_low_clouds": null, "weather_medium_clouds": null, @@ -1141,11 +1142,11 @@ "weather_ozone": null, "weather_ghi": 0.0, "weather_dni": 0.0, - "weather_dhi": 0.0, - "start_datetime": "2024-10-26 00:00:00+02:00" + "weather_dhi": 0.0 }, { "date_time": "2024-10-27 19:00:00+01:00", + "configured_data": {}, "weather_total_clouds": 100.0, "weather_low_clouds": null, "weather_medium_clouds": null, @@ -1167,11 +1168,11 @@ "weather_ozone": null, "weather_ghi": 0.0, "weather_dni": 0.0, - "weather_dhi": 0.0, - "start_datetime": "2024-10-26 00:00:00+02:00" + "weather_dhi": 0.0 }, { "date_time": "2024-10-27 20:00:00+01:00", + "configured_data": {}, "weather_total_clouds": 100.0, "weather_low_clouds": null, "weather_medium_clouds": null, @@ -1193,11 +1194,11 @@ "weather_ozone": null, "weather_ghi": 0.0, "weather_dni": 0.0, - "weather_dhi": 0.0, - "start_datetime": "2024-10-26 00:00:00+02:00" + "weather_dhi": 0.0 }, { "date_time": "2024-10-27 21:00:00+01:00", + "configured_data": {}, "weather_total_clouds": 100.0, "weather_low_clouds": null, "weather_medium_clouds": null, @@ -1219,11 +1220,11 @@ "weather_ozone": null, "weather_ghi": 0.0, "weather_dni": 0.0, - "weather_dhi": 0.0, - "start_datetime": "2024-10-26 00:00:00+02:00" + "weather_dhi": 0.0 }, { "date_time": "2024-10-27 22:00:00+01:00", + "configured_data": {}, "weather_total_clouds": 100.0, "weather_low_clouds": null, "weather_medium_clouds": null, @@ -1245,11 +1246,11 @@ "weather_ozone": null, "weather_ghi": 0.0, "weather_dni": 0.0, - "weather_dhi": 0.0, - "start_datetime": "2024-10-26 00:00:00+02:00" + "weather_dhi": 0.0 }, { "date_time": "2024-10-27 23:00:00+01:00", + "configured_data": {}, "weather_total_clouds": 100.0, "weather_low_clouds": null, "weather_medium_clouds": null, @@ -1271,11 +1272,11 @@ "weather_ozone": null, "weather_ghi": 0.0, "weather_dni": 0.0, - "weather_dhi": 0.0, - "start_datetime": "2024-10-26 00:00:00+02:00" + "weather_dhi": 0.0 }, { "date_time": "2024-10-28 00:00:00+01:00", + "configured_data": {}, "weather_total_clouds": 100.0, "weather_low_clouds": null, "weather_medium_clouds": null, @@ -1285,7 +1286,7 @@ "weather_precip_type": null, "weather_precip_prob": null, "weather_precip_amt": 0.0, - "weather_preciptable_water": 2.174201406952952, + "weather_preciptable_water": null, "weather_wind_speed": 11.9, "weather_wind_direction": 180.0, "weather_frost_chance": null, @@ -1297,14 +1298,12 @@ "weather_ozone": null, "weather_ghi": 0.0, "weather_dni": 0.0, - "weather_dhi": 0.0, - "start_datetime": "2024-10-26 00:00:00+02:00" + "weather_dhi": 0.0 } ], - "update_datetime": "2025-02-15T08:48:36.218971+01:00", - "start_datetime": "2024-10-26T00:00:00+02:00", - "min_datetime": "2024-10-26T00:00:00+02:00", - "max_datetime": "2024-10-28T00:00:00+01:00", + "update_datetime": "2025-10-23 17:16:44.461593+02:00", + "min_datetime": "2024-10-26 00:00:00+02:00", + "max_datetime": "2024-10-28 00:00:00+01:00", "record_keys": [ "date_time", "weather_total_clouds", @@ -1328,8 +1327,7 @@ "weather_ozone", "weather_ghi", "weather_dni", - "weather_dhi", - "start_datetime" + "weather_dhi" ], "record_keys_writable": [ "date_time", @@ -1356,8 +1354,8 @@ "weather_dni", "weather_dhi" ], - "end_datetime": "2024-10-28T00:00:00+01:00", - "keep_datetime": "2024-10-24T00:00:00+02:00", + "end_datetime": "2024-10-28 00:00:00+01:00", + "keep_datetime": "2024-10-24 00:00:00+02:00", "total_hours": 49, "keep_hours": 48 } \ No newline at end of file