Commit Graph

100 Commits

Author SHA1 Message Date
Bobby Noelte
b397b5d43e fix: automatic optimization (#596)
This fix implements the long term goal to have the EOS server run optimization (or
energy management) on regular intervals automatically. Thus clients can request
the current energy management plan at any time and it is updated on regular
intervals without interaction by the client.

This fix started out to "only" make automatic optimization (or energy management)
runs working. It turned out there are several endpoints that in some way
update predictions or run the optimization. To lock against such concurrent attempts
the code had to be refactored to allow control of execution. During refactoring it
became clear that some classes and files are named without a proper reference
to their usage. Thus not only refactoring but also renaming became necessary.
The names are still not the best, but I hope they are more intuitive.

The fix includes several bug fixes that are not directly related to the automatic optimization
but are necessary to keep EOS running properly to do the automatic optimization and
to test and document the changes.

This is a breaking change as the configuration structure changed once again and
the server API was also enhanced and streamlined. The server API that is used by
Andreas and Jörg in their videos has not changed.

* fix: automatic optimization

  Allow optimization to automatically run on configured intervals gathering all
  optimization parameters from configuration and predictions. The automatic run
  can be configured to only run prediction updates skipping the optimization.
  Extend documentaion to also cover automatic optimization. Lock automatic runs
  against runs initiated by the /optimize or other endpoints. Provide new
  endpoints to retrieve the energy management plan and the genetic solution
  of the latest automatic optimization run. Offload energy management to thread
  pool executor to keep the app more responsive during the CPU heavy optimization
  run.

* fix: EOS servers recognize environment variables on startup

  Force initialisation of EOS configuration on server startup to assure
  all sources of EOS configuration are properly set up and read. Adapt
  server tests and configuration tests to also test for environment
  variable configuration.

* fix: Remove 0.0.0.0 to localhost translation under Windows

  EOS imposed a 0.0.0.0 to localhost translation under Windows for
  convenience. This caused some trouble in user configurations. Now, as the
  default IP address configuration is 127.0.0.1, the user is responsible
  for to set up the correct Windows compliant IP address.

* fix: allow names for hosts additional to IP addresses

* fix: access pydantic model fields by class

  Access by instance is deprecated.

* fix: down sampling key_to_array

* fix: make cache clear endpoint clear all cache files

  Make /v1/admin/cache/clear clear all cache files. Before it only cleared
  expired cache files by default. Add new endpoint /v1/admin/clear-expired
  to only clear expired cache files.

* fix: timezonefinder returns Europe/Paris instead of Europe/Berlin

  timezonefinder 8.10 got more inaccurate for timezones in europe as there is
  a common timezone. Use new package tzfpy instead which is still returning
  Europe/Berlin if you are in Germany. tzfpy also claims to be faster than
  timezonefinder.

* fix: provider settings configuration

  Provider configuration used to be a union holding the settings for several
  providers. Pydantic union handling does not always find the correct type
  for a provider setting. This led to exceptions in specific configurations.
  Now provider settings are explicit comfiguration items for each possible
  provider. This is a breaking change as the configuration structure was
  changed.

* fix: ClearOutside weather prediction irradiance calculation

  Pvlib needs a pandas time index. Convert time index.

* fix: test config file priority

  Do not use config_eos fixture as this fixture already creates a config file.

* fix: optimization sample request documentation

  Provide all data in documentation of optimization sample request.

* fix: gitlint blocking pip dependency resolution

  Replace gitlint by commitizen. Gitlint is not actively maintained anymore.
  Gitlint dependencies blocked pip from dependency resolution.

* fix: sync pre-commit config to actual dependency requirements

  .pre-commit-config.yaml was out of sync, also requirements-dev.txt.

* fix: missing babel in requirements.txt

  Add babel to requirements.txt

* feat: setup default device configuration for automatic optimization

  In case the parameters for automatic optimization are not fully defined a
  default configuration is setup to allow the automatic energy management
  run. The default configuration may help the user to correctly define
  the device configuration.

* feat: allow configuration of genetic algorithm parameters

  The genetic algorithm parameters for number of individuals, number of
  generations, the seed and penalty function parameters are now avaliable
  as configuration options.

* feat: allow configuration of home appliance time windows

  The time windows a home appliance is allowed to run are now configurable
  by the configuration (for /v1 API) and also by the home appliance parameters
  (for the classic /optimize API). If there is no such configuration the
  time window defaults to optimization hours, which was the standard before
  the change. Documentation on how to configure time windows is added.

* feat: standardize mesaurement keys for battery/ ev SoC measurements

  The standardized measurement keys to report battery SoC to the device
  simulations can now be retrieved from the device configuration as a
  read-only config option.

* feat: feed in tariff prediction

  Add feed in tarif predictions needed for automatic optimization. The feed in
  tariff can be retrieved as fixed feed in tarif or can be imported. Also add
  tests for the different feed in tariff providers. Extend documentation to
  cover the feed in tariff providers.

* feat: add energy management plan based on S2 standard instructions

  EOS can generate an energy management plan as a list of simple instructions.
  May be retrieved by the /v1/energy-management/plan endpoint. The instructions
  loosely follow the S2 energy management standard.

* feat: make measurement keys configurable by EOS configuration.

  The fixed measurement keys are replaced by configurable measurement keys.

* feat: make pendulum DateTime, Date, Duration types usable for pydantic models

  Use pydantic_extra_types.pendulum_dt to get pydantic pendulum types. Types are
  added to the datetimeutil utility. Remove custom made pendulum adaptations
  from EOS pydantic module. Make EOS modules use the pydantic pendulum types
  managed by the datetimeutil module instead of the core pendulum types.

* feat: Add Time, TimeWindow, TimeWindowSequence and to_time to datetimeutil.

  The time windows are are added to support home appliance time window
  configuration. All time classes are also pydantic models. Time is the base
  class for time definition derived from pendulum.Time.

* feat: Extend DataRecord by configurable field like data.

  Configurable field like data was added to support the configuration of
  measurement records.

* feat: Add additional information to health information

  Version information is added to the health endpoints of eos and eosDash.
  The start time of the last optimization and the latest run time of the energy
  management is added to the EOS health information.

* feat: add pydantic merge model tests

* feat: add plan tab to EOSdash

  The plan tab displays the current energy management instructions.

* feat: add predictions tab to EOSdash

  The predictions tab displays the current predictions.

* feat: add cache management to EOSdash admin tab

  The admin tab is extended by a section for cache management. It allows to
  clear the cache.

* feat: add about tab to EOSdash

  The about tab resembles the former hello tab and provides extra information.

* feat: Adapt changelog and prepare for release management

  Release management using commitizen is added. The changelog file is adapted and
  teh changelog and a description for release management is added in the
  documentation.

* feat(doc): Improve install and devlopment documentation

  Provide a more concise installation description in Readme.md and add extra
  installation page and development page to documentation.

* chore: Use memory cache for interpolation instead of dict in inverter

  Decorate calculate_self_consumption() with @cachemethod_until_update to cache
  results in memory during an energy management/ optimization run. Replacement
  of dict type caching in inverter is now possible because all optimization
  runs are properly locked and the memory cache CacheUntilUpdateStore is properly
  cleared at the start of any energy management/ optimization operation.

* chore: refactor genetic

  Refactor the genetic algorithm modules for enhanced module structure and better
  readability. Removed unnecessary and overcomplex devices singleton. Also
  split devices configuration from genetic algorithm parameters to allow further
  development independently from genetic algorithm parameter format. Move
  charge rates configuration for electric vehicles from optimization to devices
  configuration to allow to have different charge rates for different cars in
  the future.

* chore: Rename memory cache to CacheEnergyManagementStore

  The name better resembles the task of the cache to chache function and method
  results for an energy management run. Also the decorator functions are renamed
  accordingly: cachemethod_energy_management, cache_energy_management

* chore: use class properties for config/ems/prediction mixin classes

* chore: skip debug logs from mathplotlib

  Mathplotlib is very noisy in debug mode.

* chore: automatically sync bokeh js to bokeh python package

  bokeh was updated to 3.8.0, make JS CDN automatically follow the package version.

* chore: rename hello.py to about.py

  Make hello.py the adapted EOSdash about page.

* chore: remove demo page from EOSdash

  As no the plan and prediction pages are working without configuration, the demo
  page is no longer necessary

* chore: split test_server.py for system test

  Split test_server.py to create explicit test_system.py for system tests.

* chore: move doc utils to generate_config_md.py

  The doc utils are only used in scripts/generate_config_md.py. Move it there to
  attribute for strong cohesion.

* chore: improve pydantic merge model documentation

* chore: remove pendulum warning from readme

* chore: remove GitHub discussions from contributing documentation

  Github discussions is to be replaced by Akkudoktor.net.

* chore(release): bump version to 0.1.0+dev for development

* build(deps): bump fastapi[standard] from 0.115.14 to 0.117.1

  bump fastapi and make coverage version (for pytest-cov) explicit to avoid pip break.

* build(deps): bump uvicorn from 0.36.0 to 0.37.0

BREAKING CHANGE: EOS configuration changed. V1 API changed.

  - The available_charge_rates_percent configuration is removed from optimization.
    Use the new charge_rate configuration for the electric vehicle
  - Optimization configuration parameter hours renamed to horizon_hours
  - Device configuration now has to provide the number of devices and device
    properties per device.
  - Specific prediction provider configuration to be provided by explicit
    configuration item (no union for all providers).
  - Measurement keys to be provided as a list.
  - New feed in tariff providers have to be configured.
  - /v1/measurement/loadxxx endpoints are removed. Use generic mesaurement endpoints.
  - /v1/admin/cache/clear now clears all cache files. Use
    /v1/admin/cache/clear-expired to only clear all expired cache files.

Signed-off-by: Bobby Noelte <b0661n0e17e@gmail.com>
2025-10-28 02:50:31 +01:00
redmoon2711
8e69caba73 feat(VRM forecast): add load and pv forecast by VRM API (#611)
Some checks failed
docker-build / platform-excludes (push) Has been cancelled
docker-build / build (push) Has been cancelled
docker-build / merge (push) Has been cancelled
pre-commit / pre-commit (push) Has been cancelled
Run Pytest on Pull Request / test (push) Has been cancelled
Add support for fetching forecasts from the VRM API by Victron Energy. Retrieve forecasts for PV generation and total load of a Victron system from the internet. 

Tests for the new modules have been added, and the documentation has been updated accordingly.

Signed-off-by: redmoon2711 <redmoon2711@gmx.de>
2025-07-19 08:55:16 +02:00
redmoon2711
8c56410338 Add new electricity price provider: Energy-Charts #381 (#590)
Some checks failed
docker-build / platform-excludes (push) Has been cancelled
pre-commit / pre-commit (push) Has been cancelled
Run Pytest on Pull Request / test (push) Has been cancelled
docker-build / build (push) Has been cancelled
docker-build / merge (push) Has been cancelled
Close stale pull requests/issues / Find Stale issues and PRs (push) Has been cancelled
* feat(ElecPriceEnergyCharts): Add new electricity price provider: Energy-Charts

* feat(ElecPriceEnergyCharts): update data only if needed

* test(elecpriceforecast): add test for energycharts

* docs(predictions.md): add ElecPriceEnergyCharts Provider

Signed-off-by: redmoon2711 <redmoon2711@gmx.de>
2025-06-23 07:29:33 +02:00
Bobby Noelte
bd38b3c5ef fix: logging, prediction update, multiple bugs (#584)
Some checks failed
docker-build / platform-excludes (push) Has been cancelled
pre-commit / pre-commit (push) Has been cancelled
Run Pytest on Pull Request / test (push) Has been cancelled
docker-build / build (push) Has been cancelled
docker-build / merge (push) Has been cancelled
Close stale pull requests/issues / Find Stale issues and PRs (push) Has been cancelled
* Fix logging configuration issues that made logging stop operation. Switch to Loguru
  logging (from Python logging). Enable console and file logging with different log levels.
  Add logging documentation.

* Fix logging configuration and EOS configuration out of sync. Added tracking support
  for nested value updates of Pydantic models. This used to update the logging configuration
  when the EOS configurationm for logging is changed. Should keep logging config and EOS
  config in sync as long as all changes to the EOS logging configuration are done by
  set_nested_value(), which is the case for the REST API.

* Fix energy management task looping endlessly after the second update when trying to update
  the last_update datetime.

* Fix get_nested_value() to correctly take values from the dicts in a Pydantic model instance.

* Fix usage of model classes instead of model instances in nested value access when evaluation
  the value type that is associated to each key.

* Fix illegal json format in prediction documentation for PVForecastAkkudoktor provider.

* Fix documentation qirks and add EOS Connect to integrations.

* Support deprecated fields in configuration in documentation generation and EOSdash.

* Enhance EOSdash demo to show BrightSky humidity data (that is often missing)

* Update documentation reference to German EOS installation videos.

Signed-off-by: Bobby Noelte <b0661n0e17e@gmail.com>
2025-06-10 22:00:28 +02:00
Bobby Noelte
3421b2303b ci(ruff): add bandit checks (#575)
Added bandit checks to continuous integration.

Updated sources to pass bandit checks:
- replaced asserts
- added timeouts to requests
- added checks for process command execution
- changed to 127.0.0.1 as default IP address for EOS and EOSdash for security reasons

Added a rudimentary check for outdated config files.

BREAKING CHANGE: Default IP address for EOS and EOSdash changed to 127.0.0.1

Signed-off-by: Bobby Noelte <b0661n0e17e@gmail.com>
2025-06-03 08:30:37 +02:00
rompic
e413543222 fix: pvforecast fails when there is only a single plane (#569)
* fix: pvforecast fails when there is only a single plane

* fix: formatting

* fix: formatting

* fix: add type annotations

* add testdata and validation test for single plane

* fix: formatting
2025-05-30 21:55:45 +02:00
rompic
058356d1b8 fix: delete empty inverter from testdata optimize_input_2.json (#568) 2025-05-30 10:40:45 +02:00
Bobby Noelte
3ec36e0932 fix: azimuth setting of pvforecastakkudoktor provider (#567)
EOS now enforces the general azimuth definition as e.g. defined in ISO 19111:
north=0, east=90, south=180, west=270. This is the convention that is and was
in the EOS documentation.

As the PV forecast of akkudoktor.net follows a different convention
(north=+-180, east=-90, south=0, west=90) the values from EOS are now converted
before the request is sent to akkudoktor.net.

BREAKING CHANGE: Azimuth configurations that followed the PVForecastAkkudoktor convention
(north=+-180, east=-90, south=0, west=90) must be converted to the general azimuth definition:
north=0, east=90, south=180, west=270.

Signed-off-by: Bobby Noelte <b0661n0e17e@gmail.com>
2025-05-28 20:42:43 +02:00
Christian Heinrich Hohlfeld
3c12e99970 fix: mitigate ReDoS in to_duration via max input length check (closes #494) (#523) 2025-04-22 00:16:33 +02:00
Dominique Lasserre
00229c39e4 visualize.py: Support variable remuneration Closes #451 (#459) 2025-04-06 14:46:52 +02:00
Bobby Noelte
0bda5ba4cc EOSdash: Improve PV forecast configuration. (#500)
* Allow to configure planes and configuration values of planes separatedly.

Make single configuration values for planes explicitly available for configuration.
Still allows to also configure a plane by a whole plane value struct.

* Enhance admin page by file import and export of the EOS configuration

The actual EOS configuration can now be exported to the EOSdash server.
From there it can be also imported. For security reasons only import and export
from/ to a predefined directory on the EOSdash server is possible.

* Improve handling of nested value pathes in pydantic models.

Added separate methods for nested path access (get_nested_value, set_nested_value).
On value setting the missing fields along the nested path are now added automatically
and initialized with default values. Nested path access was before restricted to the
EOS configuration and is now part of the pydantic base model.

* Makefile

Add new target to run rests as CI does on Github. Improve target docs.

* Datetimeutil tests

Prolong acceptable time difference for comparison of approximately equal times in tests.

Signed-off-by: Bobby Noelte <b0661n0e17e@gmail.com>
2025-04-05 13:08:12 +02:00
Bobby Noelte
7aaf193682 EOSdash: Enable EOS configuration by EOSdash. (#477)
Improve config page to edit actual configuration used by EOS.
Add admin page to save the actual configuration to the configuration file.

Signed-off-by: Bobby Noelte <b0661n0e17e@gmail.com>
2025-03-27 21:53:01 +01:00
Normann
a7d58eed9a pre-commit update and ignore changes (#461)
* pre-commit autoupdate
* type: ignore changes
* [attr-defined,unused-ignore] usage
2025-02-24 10:00:09 +01:00
Bobby Noelte
c8cad0f277 Fix BrightSky weather prediction
- Get weather data with fully specified end_date datetime argument to not miss data.
- Make preciptable water records generation robust against missing temperature
  or humidity values.

Signed-off-by: Bobby Noelte <b0661n0e17e@gmail.com>
2025-02-18 07:04:54 +01:00
Bobby Noelte
ab6a518b5f Improve EOSdash.
Make EOSdash use UI components from MonsterUI to ease further development.

- Add a first menu with some dummy pages and the configuration page.
- Make the configuration scrollable.
- Add markdown component that uses markdown-it-py (same as used by
  the myth-parser for documentation generation).
- Add bokeh (https://docs.bokeh.org/) component for charts
- Added several prediction charts to demo
- Add a footer that displays connection status with EOS server
- Add logo and favicon

Update EOS server:

- Move error message generation to extra module
- Use redirect instead of proxy

Signed-off-by: Bobby Noelte <b0661n0e17e@gmail.com>
2025-02-15 14:09:53 +01:00
Bobby Noelte
80bfe4d0f0 Improve caching. (#431)
* Move the caching module to core.

Add an in memory cache that for caching function and method results
during an energy management run (optimization run). Two decorators
are provided for methods and functions.

* Improve the file cache store by load and save functions.

Make EOS load the cache file store on startup and save it on shutdown.
Add a cyclic task that cleans the cache file store from outdated cache files.

* Improve startup of EOSdash by EOS

Make EOS starting EOSdash adhere to path configuration given in EOS.
The whole environment from EOS is now passed to EOSdash.
Should also prevent test errors due to unwanted/ wrong config file creation.

Both servers now provide a health endpoint that can be used to detect whether
the server is running. This is also used for testing now.

* Improve startup of EOS

EOS now has got an energy management task that runs shortly after startup.
It tries to execute energy management runs with predictions newly fetched
or initialized from cached data on first run.

* Improve shutdown of EOS

EOS has now a shutdown task that shuts EOS down gracefully with some
time delay to allow REST API requests for shutdwon or restart to be fully serviced.

* Improve EMS

Add energy management task for repeated energy management controlled by
startup delay and interval configuration parameters.
Translate EnergieManagementSystem to english EnergyManagement.

* Add administration endpoints

  - endpoints to control caching from REST API.
  - endpoints to control server restart (will not work on Windows) and shutdown from REST API

* Improve doc generation

Use "\n" linenend convention also on Windows when generating doc files.
Replace Windows specific 127.0.0.1 address by standard 0.0.0.0.

* Improve test support (to be able to test caching)

  - Add system test option to pytest for running tests with "real" resources
  - Add new test fixture to start server for test class and test function
  - Make kill signal adapt to Windows/ Linux
  - Use consistently "\n" for lineends when writing text files in  doc test
  - Fix test_logging under Windows
  - Fix conftest config_default_dirs test fixture under Windows

From @Lasall

* Improve Windows support

 - Use 127.0.0.1 as default config host (model defaults) and
   addionally redirect 0.0.0.0 to localhost on Windows (because default
   config file still has 0.0.0.0).
 - Update install/startup instructions as package installation is
   required atm.

Signed-off-by: Bobby Noelte <b0661n0e17e@gmail.com>
2025-02-12 21:35:51 +01:00
Dominique Lasserre
da4994ca39 Remove potentially unexpected config update 2025-02-02 10:09:15 +01:00
Dominique Lasserre
94618f5f66 REST: Allow setting single config value
* /v1/config/{path} supports setting single config value (post body). Lists are
   supported as well by using the index:
    - general/latitude (value: 55.55)
    - optimize/ev_available_charge_rates_percent/0 (value: 42)

   Whole tree can be overriden as well (no merge):
    - optimize/ev_available_charge_rates_percent (value: [42, 43, 44]

 * ConfigEOS: Add set_config_value, get_config_value
2025-02-02 10:09:15 +01:00
Normann
6743d8df4f Info text for intentional errors Closes #416 (#422)
* info before error
* core.logging usage
2025-01-26 22:42:54 +01:00
Normann
480adf8100 Data prefetch ems for feature (#420)
* Pre-fetch data

* maintanance and extend tests

* comment clean up

* nansum usage (to be save)

* Feature/config nested (#421)

* Nested config, devices registry

 * All config now nested.
    - Use default config from model field default values. If providers
      should be enabled by default, non-empty default config file could
      be provided again.
    - Environment variable support with EOS_ prefix and __ between levels,
      e.g. EOS_SERVER__EOS_SERVER_PORT=8503 where all values are case
      insensitive.
      For more information see:
      https://docs.pydantic.dev/latest/concepts/pydantic_settings/#parsing-environment-variable-values
    - Use devices as registry for configured devices. DeviceBase as base
      class with for now just initializion support (in the future expand
      to operations during optimization).
    - Strip down ConfigEOS to the only configuration instance. Reload
      from file or reset to defaults is possible.

 * Fix multi-initialization of derived SingletonMixin classes.

* Documentation: Support nested config

 * Add examples to pydantic models.

* EOSdash: Support nested types

* Rename settings variables (remove prefixes)

* Fix API endpoint

* Fix EOSdash startup (docker)

 * Docker: Copy the same directory structure (src/) to support the
   lifespan startup of EOSdash.
   Use EOS_SERVER_EOSDASH_SESSKEY environment variable to provide
   EOSdash with session key.

* PR review

* PVForecast: planes as nested config (list)

* Update manual documentation for nested config.

 * Add config_file_path, config_folder_path back to general
   (ConfigCommonSettings). Overwrite in docs generation.

* Config: Move lat/long/timezone from prediction to general

* Docs: Add global example documentation.

 * merge_models: Use deecopy to not change input data.

* EOSdash: Sort config by name

* Review comments

* Feature/config nested dependabot req. (#415)

* Bump numpydantic from 1.6.4 to 1.6.7 (#413)

Bumps [numpydantic](https://github.com/p2p-ld/numpydantic) from 1.6.4 to 1.6.7.
- [Release notes](https://github.com/p2p-ld/numpydantic/releases)
- [Changelog](https://github.com/p2p-ld/numpydantic/blob/main/docs/changelog.md)
- [Commits](https://github.com/p2p-ld/numpydantic/compare/v1.6.4...v1.6.7)

---
updated-dependencies:
- dependency-name: numpydantic
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump timezonefinder from 6.5.7 to 6.5.8 (#414)

Bumps [timezonefinder](https://github.com/jannikmi/timezonefinder) from 6.5.7 to 6.5.8.
- [Release notes](https://github.com/jannikmi/timezonefinder/releases)
- [Changelog](https://github.com/jannikmi/timezonefinder/blob/master/CHANGELOG.rst)
- [Commits](https://github.com/jannikmi/timezonefinder/compare/6.5.7...6.5.8)

---
updated-dependencies:
- dependency-name: timezonefinder
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump pydantic from 2.10.5 to 2.10.6 (#412)

Bumps [pydantic](https://github.com/pydantic/pydantic) from 2.10.5 to 2.10.6.
- [Release notes](https://github.com/pydantic/pydantic/releases)
- [Changelog](https://github.com/pydantic/pydantic/blob/main/HISTORY.md)
- [Commits](https://github.com/pydantic/pydantic/compare/v2.10.5...v2.10.6)

---
updated-dependencies:
- dependency-name: pydantic
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump fastapi[standard] from 0.115.6 to 0.115.7 (#411)

Bumps [fastapi[standard]](https://github.com/fastapi/fastapi) from 0.115.6 to 0.115.7.
- [Release notes](https://github.com/fastapi/fastapi/releases)
- [Commits](https://github.com/fastapi/fastapi/compare/0.115.6...0.115.7)

---
updated-dependencies:
- dependency-name: fastapi[standard]
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Readme: Add hint for interfering ports on Synology Closes #408 (#419)

* Pics or it didn't happen (#402)

* inverter added

* png creation

* save svg into cache folder

* mypy

* comment

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: Dominique Lasserre <lasserre.d@gmail.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* inverter, prediction.hours

* self.config.general.data_cache_path

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: Dominique Lasserre <lasserre.d@gmail.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-01-26 19:12:14 +01:00
Dominique Lasserre
26762e5e93 Review comments 2025-01-24 21:14:37 +01:00
Dominique Lasserre
c1dd31528b Config: Move lat/long/timezone from prediction to general 2025-01-24 20:08:53 +01:00
Dominique Lasserre
1658b491d2 Update manual documentation for nested config.
* Add config_file_path, config_folder_path back to general
   (ConfigCommonSettings). Overwrite in docs generation.
2025-01-24 20:08:52 +01:00
Dominique Lasserre
af5e4a753a PVForecast: planes as nested config (list) 2025-01-24 20:08:52 +01:00
Dominique Lasserre
3257dac92b Rename settings variables (remove prefixes) 2025-01-24 20:07:21 +01:00
Dominique Lasserre
d74a56b75a Documentation: Support nested config
* Add examples to pydantic models.
2025-01-24 20:05:48 +01:00
Dominique Lasserre
be26457563 Nested config, devices registry
* All config now nested.
    - Use default config from model field default values. If providers
      should be enabled by default, non-empty default config file could
      be provided again.
    - Environment variable support with EOS_ prefix and __ between levels,
      e.g. EOS_SERVER__EOS_SERVER_PORT=8503 where all values are case
      insensitive.
      For more information see:
      https://docs.pydantic.dev/latest/concepts/pydantic_settings/#parsing-environment-variable-values
    - Use devices as registry for configured devices. DeviceBase as base
      class with for now just initializion support (in the future expand
      to operations during optimization).
    - Strip down ConfigEOS to the only configuration instance. Reload
      from file or reset to defaults is possible.

 * Fix multi-initialization of derived SingletonMixin classes.
2025-01-24 20:05:48 +01:00
Normann
98f1105c38 Probability typo fix (#383)
* Propability->Probability
2025-01-16 18:20:39 +01:00
Andreas
10703bbc1b Battery SoC = 0 > set Discharge randomply to 0/1. For longer periods of storage (#366)
Some checks are pending
docker-build / platform-excludes (push) Waiting to run
docker-build / build (push) Blocked by required conditions
docker-build / merge (push) Blocked by required conditions
* Battery SoC = 0 > hard Discharge Setting Off
2025-01-13 21:47:58 +01:00
Dominique Lasserre
9ad61f66b2 Cleanup: parameters: extra=forbid, optimize: battery, inverter optional (#361)
* Cleanup: parameters: extra=forbid, optimize: battery, inverter optional

 * Don't allow extra fields for parameters/REST-API (at least for now while
   changing API).
 * Allow both battery and inverter to be set optionally (atm optional
   battery not implemented, no API constraints).
 * inverter: Remove default max_power_wh
 * single_test_optimization: Add more cli-parameters

* Workflow docker-build: Don't try to authenticate for PRs

 * Secrets are not available anyway for forks.
2025-01-13 21:44:17 +01:00
Normann
745086c2eb Visualisation footer with current date and time (#364)
* footer with date and version

* ruff

* replace toml module with build in

* using re to extract the string

* optimize re usage

* use of use pendulum in Akkudoktor-EOS

* create_line_chart_date function added

* replace datetime with pendulum

* align ax2 with ax1 and 0 first point

* dynamic ticks

* all charts with dates

* style changes

* mypy fixes

* fix test

* fixed current time
2025-01-13 21:42:52 +01:00
Bobby Noelte
34d8e88771 Rename FastAPI server to EOS. (#355)
Rename FastAPI server to `eos` and FastHTML server to `eosdash`.

Make an user easily identify what server is meant. FastAPI and FastHTML are
implementation details that may confuse the non-technical user.

Signed-off-by: Bobby Noelte <b0661n0e17e@gmail.com>
2025-01-09 16:54:49 +01:00
Normann
05a1ade3a3 needed_prediction_hours fixed 2025-01-09 12:15:00 +01:00
Normann
902a8cbacd test fix 2025-01-09 12:15:00 +01:00
Bobby Noelte
d2f6e9866d Assert text files are in utf8 (#350)
Read and write text files with utf8 encoding.

Signed-off-by: Bobby Noelte <b0661n0e17e@gmail.com>
2025-01-07 19:31:44 +01:00
Dominique Lasserre
ef60f5b9f5 Tests: Ignore config in current test execution directory. Closes #342 (#345) 2025-01-06 23:50:20 +01:00
Bobby Noelte
214768795f Adapt documentation generation to be compliant to ReadTheDocs and Windows. (#341)
Use documentation generation tools that are available for Windows and Linux.
Use python instead of shell scripts to generate documentation.

For ReadTheDocs make generated documentation content static to avoid
running scripts outside of the docs/ path which is the default path for ReadTheDOcs.

Add tests that check if generated content does go out of sync with latest source.

Use tabs to show commands for Windows and Linux to improve user experience.

Signed-off-by: Bobby Noelte <b0661n0e17e@gmail.com>
2025-01-06 16:27:57 +01:00
Bobby Noelte
d4e31d556a Add Documentation 2 (#334)
Add documentation that covers:

- configuration
- prediction

Add Python scripts that support automatic documentation generation for
configuration data defined with pydantic.

Adapt EOS configuration to provide more methods for REST API and
automatic documentation generation.

Adapt REST API to allow for EOS configuration file load and save.
Sort REST API on generation of openapi markdown for docs.

Move logutil to core/logging to allow configuration of logging by standard config.

Make Akkudoktor predictions always start extraction of prediction data at start of day.
Previously extraction started at actual hour. This is to support the code that assumes
prediction data to start at start of day.

Signed-off-by: Bobby Noelte <b0661n0e17e@gmail.com>
2025-01-05 14:41:07 +01:00
Bobby Noelte
1866055478 Add documentation. (#321)
Add documentation that covers:

- Prediction
- Measuremnt
- REST API

Add Python scripts that support automatic documentation generation using the Sphinx
sphinxcontrib.eval extension.

Add automatic update/ test for REST API documentation.

Filter proxy endpoints from REST API documentation.

Signed-off-by: Bobby Noelte <b0661n0e17e@gmail.com>
2025-01-03 00:31:20 +01:00
Dominique Lasserre
4cb6dc7270 Fix ruff (#328) 2025-01-02 18:34:55 +01:00
Andreas
1b597d1e0f Test update 2025-01-02 15:29:01 +01:00
Bobby Noelte
ed79cacf63 Fix PVForecast settings active plane detection. (#320)
Detect active PV planes by pvforecast_surface_tilt and pvforecast_surface_azimuth to be non None. Assure by default these configuration values are None.

Signed-off-by: Bobby Noelte <b0661n0e17e@gmail.com>
2025-01-02 12:40:12 +01:00
Bobby Noelte
c0ea13d0f4 Fix2 electricity price prediction. (#296)
Normalize electricity price prediction to €/Wh.
Provide electricity price prediction by €/kWh for convenience.

Allow to configure electricity price charges by €/kWh.

Also added error page to fastapi rest server to get rid of annoying
unrelated fault messages during testing.

Signed-off-by: Bobby Noelte <b0661n0e17e@gmail.com>
2024-12-30 21:29:50 +01:00
Dominique Lasserre
75987db9e1 Reasonable defaults, isolate tests, EOS_LOGGING_LEVEL, EOS_CONFIG_DIR
* Add EOS_CONFIG_DIR to set config dir (relative path to EOS_DIR or
   absolute path).
    - config_folder_path read-only
    - config_file_path read-only
 * Default values to support app start with empty config:
    - latitude/longitude (Berlin)
    - optimization_ev_available_charge_rates_percent (null, so model
      default value is used)
    - Enable Akkudoktor electricity price forecast (docker-compose).
 * Fix some endpoints (empty data, remove unused params, fix types).
 * cacheutil: Use cache dir. Closes #240
 * Support EOS_LOGGING_LEVEL environment variable to set log level.
 * tests: All tests use separate temporary config
    - Add pytest switch --check-config-side-effect to check user
      config file existence after each test. Will also fail if user config
      existed before test execution (but will only check after the test has
      run).
      Enable flag in github workflow.
    - Globally mock platformdirs in config module. Now no longer required
      to patch individually.
      Function calls to config instance (e.g. merge_settings_from_dict)
      were unaffected previously.
 * Set Berlin as default location (default config/docker-compose).
2024-12-30 13:41:39 +01:00
Dominique Lasserre
267a9bf427 pvforecast: Some values optional. Closes #292 (#293)
* temperature, relativehumidity_2m, windspeed_10m can be null
 * prettify pv_forecast_input_1.json (and add null value)
2024-12-30 13:07:08 +01:00
Dominique Lasserre
b386c4be70 inverter test: Mock SelfConsumptionProbabilityInterpolator 2024-12-30 13:07:08 +01:00
Andreas
86d2173a4d Awattar Price changed to more precise value; if SoC=0 discharge =1 forced. Exp Avg for siple Price Forecast
* Discharge Mask Bug, Tests updated, simple Price Forecast with linear weighting
 * Price Forecast with linear weighting, last value = highest weighting Discharge enforce when soc = 0 -> mask bug
2024-12-30 13:07:08 +01:00
Andreas
54eae5a871 Soc 1 hour shift fixed + some german -> english translations in ems 2024-12-30 13:07:08 +01:00
Dominique Lasserre
053e60aca4 Pythonic: Rename self_consumption_probability.py -> interpolator.py
* Rename class self_consumption_probability_interpolator to PascalCase
   SelfConsumptionProbabilityInterpolator.
2024-12-30 13:07:08 +01:00
Andreas
410a23e375 Self consumption predictor
* Inverter: Self consumption interpolator for better discharge_hour results
 * Small penalty when EV 100% and charge >0
 * Price Forceast (use mean of last 7 days instead of repeat)
 * Price Prediction as JSON simulation output, config fixed electricty fees configurable + MyPy & Ruff
2024-12-30 13:07:08 +01:00