2025-01-03 00:31:20 +01:00
|
|
|
% SPDX-License-Identifier: Apache-2.0
|
2025-03-23 13:27:40 +01:00
|
|
|
(prediction-page)=
|
2025-01-03 00:31:20 +01:00
|
|
|
|
|
|
|
|
# Predictions
|
|
|
|
|
|
|
|
|
|
Predictions, along with simulations and measurements, form the foundation upon which energy
|
|
|
|
|
optimization is executed. In EOS, a standard set of predictions is managed, including:
|
|
|
|
|
|
2025-02-12 14:24:17 +01:00
|
|
|
- Household Load Prediction
|
|
|
|
|
- Electricity Price Prediction
|
fix: automatic optimization (#596)
This fix implements the long term goal to have the EOS server run optimization (or
energy management) on regular intervals automatically. Thus clients can request
the current energy management plan at any time and it is updated on regular
intervals without interaction by the client.
This fix started out to "only" make automatic optimization (or energy management)
runs working. It turned out there are several endpoints that in some way
update predictions or run the optimization. To lock against such concurrent attempts
the code had to be refactored to allow control of execution. During refactoring it
became clear that some classes and files are named without a proper reference
to their usage. Thus not only refactoring but also renaming became necessary.
The names are still not the best, but I hope they are more intuitive.
The fix includes several bug fixes that are not directly related to the automatic optimization
but are necessary to keep EOS running properly to do the automatic optimization and
to test and document the changes.
This is a breaking change as the configuration structure changed once again and
the server API was also enhanced and streamlined. The server API that is used by
Andreas and Jörg in their videos has not changed.
* fix: automatic optimization
Allow optimization to automatically run on configured intervals gathering all
optimization parameters from configuration and predictions. The automatic run
can be configured to only run prediction updates skipping the optimization.
Extend documentaion to also cover automatic optimization. Lock automatic runs
against runs initiated by the /optimize or other endpoints. Provide new
endpoints to retrieve the energy management plan and the genetic solution
of the latest automatic optimization run. Offload energy management to thread
pool executor to keep the app more responsive during the CPU heavy optimization
run.
* fix: EOS servers recognize environment variables on startup
Force initialisation of EOS configuration on server startup to assure
all sources of EOS configuration are properly set up and read. Adapt
server tests and configuration tests to also test for environment
variable configuration.
* fix: Remove 0.0.0.0 to localhost translation under Windows
EOS imposed a 0.0.0.0 to localhost translation under Windows for
convenience. This caused some trouble in user configurations. Now, as the
default IP address configuration is 127.0.0.1, the user is responsible
for to set up the correct Windows compliant IP address.
* fix: allow names for hosts additional to IP addresses
* fix: access pydantic model fields by class
Access by instance is deprecated.
* fix: down sampling key_to_array
* fix: make cache clear endpoint clear all cache files
Make /v1/admin/cache/clear clear all cache files. Before it only cleared
expired cache files by default. Add new endpoint /v1/admin/clear-expired
to only clear expired cache files.
* fix: timezonefinder returns Europe/Paris instead of Europe/Berlin
timezonefinder 8.10 got more inaccurate for timezones in europe as there is
a common timezone. Use new package tzfpy instead which is still returning
Europe/Berlin if you are in Germany. tzfpy also claims to be faster than
timezonefinder.
* fix: provider settings configuration
Provider configuration used to be a union holding the settings for several
providers. Pydantic union handling does not always find the correct type
for a provider setting. This led to exceptions in specific configurations.
Now provider settings are explicit comfiguration items for each possible
provider. This is a breaking change as the configuration structure was
changed.
* fix: ClearOutside weather prediction irradiance calculation
Pvlib needs a pandas time index. Convert time index.
* fix: test config file priority
Do not use config_eos fixture as this fixture already creates a config file.
* fix: optimization sample request documentation
Provide all data in documentation of optimization sample request.
* fix: gitlint blocking pip dependency resolution
Replace gitlint by commitizen. Gitlint is not actively maintained anymore.
Gitlint dependencies blocked pip from dependency resolution.
* fix: sync pre-commit config to actual dependency requirements
.pre-commit-config.yaml was out of sync, also requirements-dev.txt.
* fix: missing babel in requirements.txt
Add babel to requirements.txt
* feat: setup default device configuration for automatic optimization
In case the parameters for automatic optimization are not fully defined a
default configuration is setup to allow the automatic energy management
run. The default configuration may help the user to correctly define
the device configuration.
* feat: allow configuration of genetic algorithm parameters
The genetic algorithm parameters for number of individuals, number of
generations, the seed and penalty function parameters are now avaliable
as configuration options.
* feat: allow configuration of home appliance time windows
The time windows a home appliance is allowed to run are now configurable
by the configuration (for /v1 API) and also by the home appliance parameters
(for the classic /optimize API). If there is no such configuration the
time window defaults to optimization hours, which was the standard before
the change. Documentation on how to configure time windows is added.
* feat: standardize mesaurement keys for battery/ ev SoC measurements
The standardized measurement keys to report battery SoC to the device
simulations can now be retrieved from the device configuration as a
read-only config option.
* feat: feed in tariff prediction
Add feed in tarif predictions needed for automatic optimization. The feed in
tariff can be retrieved as fixed feed in tarif or can be imported. Also add
tests for the different feed in tariff providers. Extend documentation to
cover the feed in tariff providers.
* feat: add energy management plan based on S2 standard instructions
EOS can generate an energy management plan as a list of simple instructions.
May be retrieved by the /v1/energy-management/plan endpoint. The instructions
loosely follow the S2 energy management standard.
* feat: make measurement keys configurable by EOS configuration.
The fixed measurement keys are replaced by configurable measurement keys.
* feat: make pendulum DateTime, Date, Duration types usable for pydantic models
Use pydantic_extra_types.pendulum_dt to get pydantic pendulum types. Types are
added to the datetimeutil utility. Remove custom made pendulum adaptations
from EOS pydantic module. Make EOS modules use the pydantic pendulum types
managed by the datetimeutil module instead of the core pendulum types.
* feat: Add Time, TimeWindow, TimeWindowSequence and to_time to datetimeutil.
The time windows are are added to support home appliance time window
configuration. All time classes are also pydantic models. Time is the base
class for time definition derived from pendulum.Time.
* feat: Extend DataRecord by configurable field like data.
Configurable field like data was added to support the configuration of
measurement records.
* feat: Add additional information to health information
Version information is added to the health endpoints of eos and eosDash.
The start time of the last optimization and the latest run time of the energy
management is added to the EOS health information.
* feat: add pydantic merge model tests
* feat: add plan tab to EOSdash
The plan tab displays the current energy management instructions.
* feat: add predictions tab to EOSdash
The predictions tab displays the current predictions.
* feat: add cache management to EOSdash admin tab
The admin tab is extended by a section for cache management. It allows to
clear the cache.
* feat: add about tab to EOSdash
The about tab resembles the former hello tab and provides extra information.
* feat: Adapt changelog and prepare for release management
Release management using commitizen is added. The changelog file is adapted and
teh changelog and a description for release management is added in the
documentation.
* feat(doc): Improve install and devlopment documentation
Provide a more concise installation description in Readme.md and add extra
installation page and development page to documentation.
* chore: Use memory cache for interpolation instead of dict in inverter
Decorate calculate_self_consumption() with @cachemethod_until_update to cache
results in memory during an energy management/ optimization run. Replacement
of dict type caching in inverter is now possible because all optimization
runs are properly locked and the memory cache CacheUntilUpdateStore is properly
cleared at the start of any energy management/ optimization operation.
* chore: refactor genetic
Refactor the genetic algorithm modules for enhanced module structure and better
readability. Removed unnecessary and overcomplex devices singleton. Also
split devices configuration from genetic algorithm parameters to allow further
development independently from genetic algorithm parameter format. Move
charge rates configuration for electric vehicles from optimization to devices
configuration to allow to have different charge rates for different cars in
the future.
* chore: Rename memory cache to CacheEnergyManagementStore
The name better resembles the task of the cache to chache function and method
results for an energy management run. Also the decorator functions are renamed
accordingly: cachemethod_energy_management, cache_energy_management
* chore: use class properties for config/ems/prediction mixin classes
* chore: skip debug logs from mathplotlib
Mathplotlib is very noisy in debug mode.
* chore: automatically sync bokeh js to bokeh python package
bokeh was updated to 3.8.0, make JS CDN automatically follow the package version.
* chore: rename hello.py to about.py
Make hello.py the adapted EOSdash about page.
* chore: remove demo page from EOSdash
As no the plan and prediction pages are working without configuration, the demo
page is no longer necessary
* chore: split test_server.py for system test
Split test_server.py to create explicit test_system.py for system tests.
* chore: move doc utils to generate_config_md.py
The doc utils are only used in scripts/generate_config_md.py. Move it there to
attribute for strong cohesion.
* chore: improve pydantic merge model documentation
* chore: remove pendulum warning from readme
* chore: remove GitHub discussions from contributing documentation
Github discussions is to be replaced by Akkudoktor.net.
* chore(release): bump version to 0.1.0+dev for development
* build(deps): bump fastapi[standard] from 0.115.14 to 0.117.1
bump fastapi and make coverage version (for pytest-cov) explicit to avoid pip break.
* build(deps): bump uvicorn from 0.36.0 to 0.37.0
BREAKING CHANGE: EOS configuration changed. V1 API changed.
- The available_charge_rates_percent configuration is removed from optimization.
Use the new charge_rate configuration for the electric vehicle
- Optimization configuration parameter hours renamed to horizon_hours
- Device configuration now has to provide the number of devices and device
properties per device.
- Specific prediction provider configuration to be provided by explicit
configuration item (no union for all providers).
- Measurement keys to be provided as a list.
- New feed in tariff providers have to be configured.
- /v1/measurement/loadxxx endpoints are removed. Use generic mesaurement endpoints.
- /v1/admin/cache/clear now clears all cache files. Use
/v1/admin/cache/clear-expired to only clear all expired cache files.
Signed-off-by: Bobby Noelte <b0661n0e17e@gmail.com>
2025-10-28 02:50:31 +01:00
|
|
|
- Feed In Tariff Prediction
|
2025-02-12 14:24:17 +01:00
|
|
|
- PV Power Prediction
|
|
|
|
|
- Weather Prediction
|
2025-01-03 00:31:20 +01:00
|
|
|
|
|
|
|
|
## Storing Predictions
|
|
|
|
|
|
|
|
|
|
EOS stores predictions in a **key-value store**, where the term `prediction key` refers to the
|
fix: automatic optimization (#596)
This fix implements the long term goal to have the EOS server run optimization (or
energy management) on regular intervals automatically. Thus clients can request
the current energy management plan at any time and it is updated on regular
intervals without interaction by the client.
This fix started out to "only" make automatic optimization (or energy management)
runs working. It turned out there are several endpoints that in some way
update predictions or run the optimization. To lock against such concurrent attempts
the code had to be refactored to allow control of execution. During refactoring it
became clear that some classes and files are named without a proper reference
to their usage. Thus not only refactoring but also renaming became necessary.
The names are still not the best, but I hope they are more intuitive.
The fix includes several bug fixes that are not directly related to the automatic optimization
but are necessary to keep EOS running properly to do the automatic optimization and
to test and document the changes.
This is a breaking change as the configuration structure changed once again and
the server API was also enhanced and streamlined. The server API that is used by
Andreas and Jörg in their videos has not changed.
* fix: automatic optimization
Allow optimization to automatically run on configured intervals gathering all
optimization parameters from configuration and predictions. The automatic run
can be configured to only run prediction updates skipping the optimization.
Extend documentaion to also cover automatic optimization. Lock automatic runs
against runs initiated by the /optimize or other endpoints. Provide new
endpoints to retrieve the energy management plan and the genetic solution
of the latest automatic optimization run. Offload energy management to thread
pool executor to keep the app more responsive during the CPU heavy optimization
run.
* fix: EOS servers recognize environment variables on startup
Force initialisation of EOS configuration on server startup to assure
all sources of EOS configuration are properly set up and read. Adapt
server tests and configuration tests to also test for environment
variable configuration.
* fix: Remove 0.0.0.0 to localhost translation under Windows
EOS imposed a 0.0.0.0 to localhost translation under Windows for
convenience. This caused some trouble in user configurations. Now, as the
default IP address configuration is 127.0.0.1, the user is responsible
for to set up the correct Windows compliant IP address.
* fix: allow names for hosts additional to IP addresses
* fix: access pydantic model fields by class
Access by instance is deprecated.
* fix: down sampling key_to_array
* fix: make cache clear endpoint clear all cache files
Make /v1/admin/cache/clear clear all cache files. Before it only cleared
expired cache files by default. Add new endpoint /v1/admin/clear-expired
to only clear expired cache files.
* fix: timezonefinder returns Europe/Paris instead of Europe/Berlin
timezonefinder 8.10 got more inaccurate for timezones in europe as there is
a common timezone. Use new package tzfpy instead which is still returning
Europe/Berlin if you are in Germany. tzfpy also claims to be faster than
timezonefinder.
* fix: provider settings configuration
Provider configuration used to be a union holding the settings for several
providers. Pydantic union handling does not always find the correct type
for a provider setting. This led to exceptions in specific configurations.
Now provider settings are explicit comfiguration items for each possible
provider. This is a breaking change as the configuration structure was
changed.
* fix: ClearOutside weather prediction irradiance calculation
Pvlib needs a pandas time index. Convert time index.
* fix: test config file priority
Do not use config_eos fixture as this fixture already creates a config file.
* fix: optimization sample request documentation
Provide all data in documentation of optimization sample request.
* fix: gitlint blocking pip dependency resolution
Replace gitlint by commitizen. Gitlint is not actively maintained anymore.
Gitlint dependencies blocked pip from dependency resolution.
* fix: sync pre-commit config to actual dependency requirements
.pre-commit-config.yaml was out of sync, also requirements-dev.txt.
* fix: missing babel in requirements.txt
Add babel to requirements.txt
* feat: setup default device configuration for automatic optimization
In case the parameters for automatic optimization are not fully defined a
default configuration is setup to allow the automatic energy management
run. The default configuration may help the user to correctly define
the device configuration.
* feat: allow configuration of genetic algorithm parameters
The genetic algorithm parameters for number of individuals, number of
generations, the seed and penalty function parameters are now avaliable
as configuration options.
* feat: allow configuration of home appliance time windows
The time windows a home appliance is allowed to run are now configurable
by the configuration (for /v1 API) and also by the home appliance parameters
(for the classic /optimize API). If there is no such configuration the
time window defaults to optimization hours, which was the standard before
the change. Documentation on how to configure time windows is added.
* feat: standardize mesaurement keys for battery/ ev SoC measurements
The standardized measurement keys to report battery SoC to the device
simulations can now be retrieved from the device configuration as a
read-only config option.
* feat: feed in tariff prediction
Add feed in tarif predictions needed for automatic optimization. The feed in
tariff can be retrieved as fixed feed in tarif or can be imported. Also add
tests for the different feed in tariff providers. Extend documentation to
cover the feed in tariff providers.
* feat: add energy management plan based on S2 standard instructions
EOS can generate an energy management plan as a list of simple instructions.
May be retrieved by the /v1/energy-management/plan endpoint. The instructions
loosely follow the S2 energy management standard.
* feat: make measurement keys configurable by EOS configuration.
The fixed measurement keys are replaced by configurable measurement keys.
* feat: make pendulum DateTime, Date, Duration types usable for pydantic models
Use pydantic_extra_types.pendulum_dt to get pydantic pendulum types. Types are
added to the datetimeutil utility. Remove custom made pendulum adaptations
from EOS pydantic module. Make EOS modules use the pydantic pendulum types
managed by the datetimeutil module instead of the core pendulum types.
* feat: Add Time, TimeWindow, TimeWindowSequence and to_time to datetimeutil.
The time windows are are added to support home appliance time window
configuration. All time classes are also pydantic models. Time is the base
class for time definition derived from pendulum.Time.
* feat: Extend DataRecord by configurable field like data.
Configurable field like data was added to support the configuration of
measurement records.
* feat: Add additional information to health information
Version information is added to the health endpoints of eos and eosDash.
The start time of the last optimization and the latest run time of the energy
management is added to the EOS health information.
* feat: add pydantic merge model tests
* feat: add plan tab to EOSdash
The plan tab displays the current energy management instructions.
* feat: add predictions tab to EOSdash
The predictions tab displays the current predictions.
* feat: add cache management to EOSdash admin tab
The admin tab is extended by a section for cache management. It allows to
clear the cache.
* feat: add about tab to EOSdash
The about tab resembles the former hello tab and provides extra information.
* feat: Adapt changelog and prepare for release management
Release management using commitizen is added. The changelog file is adapted and
teh changelog and a description for release management is added in the
documentation.
* feat(doc): Improve install and devlopment documentation
Provide a more concise installation description in Readme.md and add extra
installation page and development page to documentation.
* chore: Use memory cache for interpolation instead of dict in inverter
Decorate calculate_self_consumption() with @cachemethod_until_update to cache
results in memory during an energy management/ optimization run. Replacement
of dict type caching in inverter is now possible because all optimization
runs are properly locked and the memory cache CacheUntilUpdateStore is properly
cleared at the start of any energy management/ optimization operation.
* chore: refactor genetic
Refactor the genetic algorithm modules for enhanced module structure and better
readability. Removed unnecessary and overcomplex devices singleton. Also
split devices configuration from genetic algorithm parameters to allow further
development independently from genetic algorithm parameter format. Move
charge rates configuration for electric vehicles from optimization to devices
configuration to allow to have different charge rates for different cars in
the future.
* chore: Rename memory cache to CacheEnergyManagementStore
The name better resembles the task of the cache to chache function and method
results for an energy management run. Also the decorator functions are renamed
accordingly: cachemethod_energy_management, cache_energy_management
* chore: use class properties for config/ems/prediction mixin classes
* chore: skip debug logs from mathplotlib
Mathplotlib is very noisy in debug mode.
* chore: automatically sync bokeh js to bokeh python package
bokeh was updated to 3.8.0, make JS CDN automatically follow the package version.
* chore: rename hello.py to about.py
Make hello.py the adapted EOSdash about page.
* chore: remove demo page from EOSdash
As no the plan and prediction pages are working without configuration, the demo
page is no longer necessary
* chore: split test_server.py for system test
Split test_server.py to create explicit test_system.py for system tests.
* chore: move doc utils to generate_config_md.py
The doc utils are only used in scripts/generate_config_md.py. Move it there to
attribute for strong cohesion.
* chore: improve pydantic merge model documentation
* chore: remove pendulum warning from readme
* chore: remove GitHub discussions from contributing documentation
Github discussions is to be replaced by Akkudoktor.net.
* chore(release): bump version to 0.1.0+dev for development
* build(deps): bump fastapi[standard] from 0.115.14 to 0.117.1
bump fastapi and make coverage version (for pytest-cov) explicit to avoid pip break.
* build(deps): bump uvicorn from 0.36.0 to 0.37.0
BREAKING CHANGE: EOS configuration changed. V1 API changed.
- The available_charge_rates_percent configuration is removed from optimization.
Use the new charge_rate configuration for the electric vehicle
- Optimization configuration parameter hours renamed to horizon_hours
- Device configuration now has to provide the number of devices and device
properties per device.
- Specific prediction provider configuration to be provided by explicit
configuration item (no union for all providers).
- Measurement keys to be provided as a list.
- New feed in tariff providers have to be configured.
- /v1/measurement/loadxxx endpoints are removed. Use generic mesaurement endpoints.
- /v1/admin/cache/clear now clears all cache files. Use
/v1/admin/cache/clear-expired to only clear all expired cache files.
Signed-off-by: Bobby Noelte <b0661n0e17e@gmail.com>
2025-10-28 02:50:31 +01:00
|
|
|
unique key used to retrieve specific prediction data.
|
2025-01-03 00:31:20 +01:00
|
|
|
|
|
|
|
|
## Prediction Providers
|
|
|
|
|
|
|
|
|
|
Most predictions can be sourced from various providers. The specific provider to use is configured
|
2025-01-19 21:47:21 +01:00
|
|
|
in the EOS configuration and can be set by prediction type. For example:
|
2025-01-03 00:31:20 +01:00
|
|
|
|
fix: automatic optimization (#596)
This fix implements the long term goal to have the EOS server run optimization (or
energy management) on regular intervals automatically. Thus clients can request
the current energy management plan at any time and it is updated on regular
intervals without interaction by the client.
This fix started out to "only" make automatic optimization (or energy management)
runs working. It turned out there are several endpoints that in some way
update predictions or run the optimization. To lock against such concurrent attempts
the code had to be refactored to allow control of execution. During refactoring it
became clear that some classes and files are named without a proper reference
to their usage. Thus not only refactoring but also renaming became necessary.
The names are still not the best, but I hope they are more intuitive.
The fix includes several bug fixes that are not directly related to the automatic optimization
but are necessary to keep EOS running properly to do the automatic optimization and
to test and document the changes.
This is a breaking change as the configuration structure changed once again and
the server API was also enhanced and streamlined. The server API that is used by
Andreas and Jörg in their videos has not changed.
* fix: automatic optimization
Allow optimization to automatically run on configured intervals gathering all
optimization parameters from configuration and predictions. The automatic run
can be configured to only run prediction updates skipping the optimization.
Extend documentaion to also cover automatic optimization. Lock automatic runs
against runs initiated by the /optimize or other endpoints. Provide new
endpoints to retrieve the energy management plan and the genetic solution
of the latest automatic optimization run. Offload energy management to thread
pool executor to keep the app more responsive during the CPU heavy optimization
run.
* fix: EOS servers recognize environment variables on startup
Force initialisation of EOS configuration on server startup to assure
all sources of EOS configuration are properly set up and read. Adapt
server tests and configuration tests to also test for environment
variable configuration.
* fix: Remove 0.0.0.0 to localhost translation under Windows
EOS imposed a 0.0.0.0 to localhost translation under Windows for
convenience. This caused some trouble in user configurations. Now, as the
default IP address configuration is 127.0.0.1, the user is responsible
for to set up the correct Windows compliant IP address.
* fix: allow names for hosts additional to IP addresses
* fix: access pydantic model fields by class
Access by instance is deprecated.
* fix: down sampling key_to_array
* fix: make cache clear endpoint clear all cache files
Make /v1/admin/cache/clear clear all cache files. Before it only cleared
expired cache files by default. Add new endpoint /v1/admin/clear-expired
to only clear expired cache files.
* fix: timezonefinder returns Europe/Paris instead of Europe/Berlin
timezonefinder 8.10 got more inaccurate for timezones in europe as there is
a common timezone. Use new package tzfpy instead which is still returning
Europe/Berlin if you are in Germany. tzfpy also claims to be faster than
timezonefinder.
* fix: provider settings configuration
Provider configuration used to be a union holding the settings for several
providers. Pydantic union handling does not always find the correct type
for a provider setting. This led to exceptions in specific configurations.
Now provider settings are explicit comfiguration items for each possible
provider. This is a breaking change as the configuration structure was
changed.
* fix: ClearOutside weather prediction irradiance calculation
Pvlib needs a pandas time index. Convert time index.
* fix: test config file priority
Do not use config_eos fixture as this fixture already creates a config file.
* fix: optimization sample request documentation
Provide all data in documentation of optimization sample request.
* fix: gitlint blocking pip dependency resolution
Replace gitlint by commitizen. Gitlint is not actively maintained anymore.
Gitlint dependencies blocked pip from dependency resolution.
* fix: sync pre-commit config to actual dependency requirements
.pre-commit-config.yaml was out of sync, also requirements-dev.txt.
* fix: missing babel in requirements.txt
Add babel to requirements.txt
* feat: setup default device configuration for automatic optimization
In case the parameters for automatic optimization are not fully defined a
default configuration is setup to allow the automatic energy management
run. The default configuration may help the user to correctly define
the device configuration.
* feat: allow configuration of genetic algorithm parameters
The genetic algorithm parameters for number of individuals, number of
generations, the seed and penalty function parameters are now avaliable
as configuration options.
* feat: allow configuration of home appliance time windows
The time windows a home appliance is allowed to run are now configurable
by the configuration (for /v1 API) and also by the home appliance parameters
(for the classic /optimize API). If there is no such configuration the
time window defaults to optimization hours, which was the standard before
the change. Documentation on how to configure time windows is added.
* feat: standardize mesaurement keys for battery/ ev SoC measurements
The standardized measurement keys to report battery SoC to the device
simulations can now be retrieved from the device configuration as a
read-only config option.
* feat: feed in tariff prediction
Add feed in tarif predictions needed for automatic optimization. The feed in
tariff can be retrieved as fixed feed in tarif or can be imported. Also add
tests for the different feed in tariff providers. Extend documentation to
cover the feed in tariff providers.
* feat: add energy management plan based on S2 standard instructions
EOS can generate an energy management plan as a list of simple instructions.
May be retrieved by the /v1/energy-management/plan endpoint. The instructions
loosely follow the S2 energy management standard.
* feat: make measurement keys configurable by EOS configuration.
The fixed measurement keys are replaced by configurable measurement keys.
* feat: make pendulum DateTime, Date, Duration types usable for pydantic models
Use pydantic_extra_types.pendulum_dt to get pydantic pendulum types. Types are
added to the datetimeutil utility. Remove custom made pendulum adaptations
from EOS pydantic module. Make EOS modules use the pydantic pendulum types
managed by the datetimeutil module instead of the core pendulum types.
* feat: Add Time, TimeWindow, TimeWindowSequence and to_time to datetimeutil.
The time windows are are added to support home appliance time window
configuration. All time classes are also pydantic models. Time is the base
class for time definition derived from pendulum.Time.
* feat: Extend DataRecord by configurable field like data.
Configurable field like data was added to support the configuration of
measurement records.
* feat: Add additional information to health information
Version information is added to the health endpoints of eos and eosDash.
The start time of the last optimization and the latest run time of the energy
management is added to the EOS health information.
* feat: add pydantic merge model tests
* feat: add plan tab to EOSdash
The plan tab displays the current energy management instructions.
* feat: add predictions tab to EOSdash
The predictions tab displays the current predictions.
* feat: add cache management to EOSdash admin tab
The admin tab is extended by a section for cache management. It allows to
clear the cache.
* feat: add about tab to EOSdash
The about tab resembles the former hello tab and provides extra information.
* feat: Adapt changelog and prepare for release management
Release management using commitizen is added. The changelog file is adapted and
teh changelog and a description for release management is added in the
documentation.
* feat(doc): Improve install and devlopment documentation
Provide a more concise installation description in Readme.md and add extra
installation page and development page to documentation.
* chore: Use memory cache for interpolation instead of dict in inverter
Decorate calculate_self_consumption() with @cachemethod_until_update to cache
results in memory during an energy management/ optimization run. Replacement
of dict type caching in inverter is now possible because all optimization
runs are properly locked and the memory cache CacheUntilUpdateStore is properly
cleared at the start of any energy management/ optimization operation.
* chore: refactor genetic
Refactor the genetic algorithm modules for enhanced module structure and better
readability. Removed unnecessary and overcomplex devices singleton. Also
split devices configuration from genetic algorithm parameters to allow further
development independently from genetic algorithm parameter format. Move
charge rates configuration for electric vehicles from optimization to devices
configuration to allow to have different charge rates for different cars in
the future.
* chore: Rename memory cache to CacheEnergyManagementStore
The name better resembles the task of the cache to chache function and method
results for an energy management run. Also the decorator functions are renamed
accordingly: cachemethod_energy_management, cache_energy_management
* chore: use class properties for config/ems/prediction mixin classes
* chore: skip debug logs from mathplotlib
Mathplotlib is very noisy in debug mode.
* chore: automatically sync bokeh js to bokeh python package
bokeh was updated to 3.8.0, make JS CDN automatically follow the package version.
* chore: rename hello.py to about.py
Make hello.py the adapted EOSdash about page.
* chore: remove demo page from EOSdash
As no the plan and prediction pages are working without configuration, the demo
page is no longer necessary
* chore: split test_server.py for system test
Split test_server.py to create explicit test_system.py for system tests.
* chore: move doc utils to generate_config_md.py
The doc utils are only used in scripts/generate_config_md.py. Move it there to
attribute for strong cohesion.
* chore: improve pydantic merge model documentation
* chore: remove pendulum warning from readme
* chore: remove GitHub discussions from contributing documentation
Github discussions is to be replaced by Akkudoktor.net.
* chore(release): bump version to 0.1.0+dev for development
* build(deps): bump fastapi[standard] from 0.115.14 to 0.117.1
bump fastapi and make coverage version (for pytest-cov) explicit to avoid pip break.
* build(deps): bump uvicorn from 0.36.0 to 0.37.0
BREAKING CHANGE: EOS configuration changed. V1 API changed.
- The available_charge_rates_percent configuration is removed from optimization.
Use the new charge_rate configuration for the electric vehicle
- Optimization configuration parameter hours renamed to horizon_hours
- Device configuration now has to provide the number of devices and device
properties per device.
- Specific prediction provider configuration to be provided by explicit
configuration item (no union for all providers).
- Measurement keys to be provided as a list.
- New feed in tariff providers have to be configured.
- /v1/measurement/loadxxx endpoints are removed. Use generic mesaurement endpoints.
- /v1/admin/cache/clear now clears all cache files. Use
/v1/admin/cache/clear-expired to only clear all expired cache files.
Signed-off-by: Bobby Noelte <b0661n0e17e@gmail.com>
2025-10-28 02:50:31 +01:00
|
|
|
```json
|
2025-01-19 21:47:21 +01:00
|
|
|
{
|
|
|
|
|
"weather": {
|
|
|
|
|
"provider": "ClearOutside"
|
|
|
|
|
}
|
|
|
|
|
}
|
2025-01-03 00:31:20 +01:00
|
|
|
```
|
|
|
|
|
|
|
|
|
|
Some providers offer multiple prediction keys. For instance, a weather provider might provide data
|
|
|
|
|
to prediction keys like:
|
|
|
|
|
|
|
|
|
|
- `weather_temp_air` (air temperature)
|
|
|
|
|
- `weather_wind_speed` (wind speed)
|
|
|
|
|
|
|
|
|
|
### Prediction Import Providers
|
|
|
|
|
|
|
|
|
|
The prediction import providers are designed to import prediction data from a file or a JSON
|
|
|
|
|
string. An external entity should update the file or JSON string whenever new prediction data
|
|
|
|
|
becomes available.
|
|
|
|
|
|
|
|
|
|
The prediction data must be provided in one of the following formats:
|
|
|
|
|
|
|
|
|
|
#### 1. DateTimeData
|
|
|
|
|
|
|
|
|
|
A dictionary with the following structure:
|
|
|
|
|
|
fix: automatic optimization (#596)
This fix implements the long term goal to have the EOS server run optimization (or
energy management) on regular intervals automatically. Thus clients can request
the current energy management plan at any time and it is updated on regular
intervals without interaction by the client.
This fix started out to "only" make automatic optimization (or energy management)
runs working. It turned out there are several endpoints that in some way
update predictions or run the optimization. To lock against such concurrent attempts
the code had to be refactored to allow control of execution. During refactoring it
became clear that some classes and files are named without a proper reference
to their usage. Thus not only refactoring but also renaming became necessary.
The names are still not the best, but I hope they are more intuitive.
The fix includes several bug fixes that are not directly related to the automatic optimization
but are necessary to keep EOS running properly to do the automatic optimization and
to test and document the changes.
This is a breaking change as the configuration structure changed once again and
the server API was also enhanced and streamlined. The server API that is used by
Andreas and Jörg in their videos has not changed.
* fix: automatic optimization
Allow optimization to automatically run on configured intervals gathering all
optimization parameters from configuration and predictions. The automatic run
can be configured to only run prediction updates skipping the optimization.
Extend documentaion to also cover automatic optimization. Lock automatic runs
against runs initiated by the /optimize or other endpoints. Provide new
endpoints to retrieve the energy management plan and the genetic solution
of the latest automatic optimization run. Offload energy management to thread
pool executor to keep the app more responsive during the CPU heavy optimization
run.
* fix: EOS servers recognize environment variables on startup
Force initialisation of EOS configuration on server startup to assure
all sources of EOS configuration are properly set up and read. Adapt
server tests and configuration tests to also test for environment
variable configuration.
* fix: Remove 0.0.0.0 to localhost translation under Windows
EOS imposed a 0.0.0.0 to localhost translation under Windows for
convenience. This caused some trouble in user configurations. Now, as the
default IP address configuration is 127.0.0.1, the user is responsible
for to set up the correct Windows compliant IP address.
* fix: allow names for hosts additional to IP addresses
* fix: access pydantic model fields by class
Access by instance is deprecated.
* fix: down sampling key_to_array
* fix: make cache clear endpoint clear all cache files
Make /v1/admin/cache/clear clear all cache files. Before it only cleared
expired cache files by default. Add new endpoint /v1/admin/clear-expired
to only clear expired cache files.
* fix: timezonefinder returns Europe/Paris instead of Europe/Berlin
timezonefinder 8.10 got more inaccurate for timezones in europe as there is
a common timezone. Use new package tzfpy instead which is still returning
Europe/Berlin if you are in Germany. tzfpy also claims to be faster than
timezonefinder.
* fix: provider settings configuration
Provider configuration used to be a union holding the settings for several
providers. Pydantic union handling does not always find the correct type
for a provider setting. This led to exceptions in specific configurations.
Now provider settings are explicit comfiguration items for each possible
provider. This is a breaking change as the configuration structure was
changed.
* fix: ClearOutside weather prediction irradiance calculation
Pvlib needs a pandas time index. Convert time index.
* fix: test config file priority
Do not use config_eos fixture as this fixture already creates a config file.
* fix: optimization sample request documentation
Provide all data in documentation of optimization sample request.
* fix: gitlint blocking pip dependency resolution
Replace gitlint by commitizen. Gitlint is not actively maintained anymore.
Gitlint dependencies blocked pip from dependency resolution.
* fix: sync pre-commit config to actual dependency requirements
.pre-commit-config.yaml was out of sync, also requirements-dev.txt.
* fix: missing babel in requirements.txt
Add babel to requirements.txt
* feat: setup default device configuration for automatic optimization
In case the parameters for automatic optimization are not fully defined a
default configuration is setup to allow the automatic energy management
run. The default configuration may help the user to correctly define
the device configuration.
* feat: allow configuration of genetic algorithm parameters
The genetic algorithm parameters for number of individuals, number of
generations, the seed and penalty function parameters are now avaliable
as configuration options.
* feat: allow configuration of home appliance time windows
The time windows a home appliance is allowed to run are now configurable
by the configuration (for /v1 API) and also by the home appliance parameters
(for the classic /optimize API). If there is no such configuration the
time window defaults to optimization hours, which was the standard before
the change. Documentation on how to configure time windows is added.
* feat: standardize mesaurement keys for battery/ ev SoC measurements
The standardized measurement keys to report battery SoC to the device
simulations can now be retrieved from the device configuration as a
read-only config option.
* feat: feed in tariff prediction
Add feed in tarif predictions needed for automatic optimization. The feed in
tariff can be retrieved as fixed feed in tarif or can be imported. Also add
tests for the different feed in tariff providers. Extend documentation to
cover the feed in tariff providers.
* feat: add energy management plan based on S2 standard instructions
EOS can generate an energy management plan as a list of simple instructions.
May be retrieved by the /v1/energy-management/plan endpoint. The instructions
loosely follow the S2 energy management standard.
* feat: make measurement keys configurable by EOS configuration.
The fixed measurement keys are replaced by configurable measurement keys.
* feat: make pendulum DateTime, Date, Duration types usable for pydantic models
Use pydantic_extra_types.pendulum_dt to get pydantic pendulum types. Types are
added to the datetimeutil utility. Remove custom made pendulum adaptations
from EOS pydantic module. Make EOS modules use the pydantic pendulum types
managed by the datetimeutil module instead of the core pendulum types.
* feat: Add Time, TimeWindow, TimeWindowSequence and to_time to datetimeutil.
The time windows are are added to support home appliance time window
configuration. All time classes are also pydantic models. Time is the base
class for time definition derived from pendulum.Time.
* feat: Extend DataRecord by configurable field like data.
Configurable field like data was added to support the configuration of
measurement records.
* feat: Add additional information to health information
Version information is added to the health endpoints of eos and eosDash.
The start time of the last optimization and the latest run time of the energy
management is added to the EOS health information.
* feat: add pydantic merge model tests
* feat: add plan tab to EOSdash
The plan tab displays the current energy management instructions.
* feat: add predictions tab to EOSdash
The predictions tab displays the current predictions.
* feat: add cache management to EOSdash admin tab
The admin tab is extended by a section for cache management. It allows to
clear the cache.
* feat: add about tab to EOSdash
The about tab resembles the former hello tab and provides extra information.
* feat: Adapt changelog and prepare for release management
Release management using commitizen is added. The changelog file is adapted and
teh changelog and a description for release management is added in the
documentation.
* feat(doc): Improve install and devlopment documentation
Provide a more concise installation description in Readme.md and add extra
installation page and development page to documentation.
* chore: Use memory cache for interpolation instead of dict in inverter
Decorate calculate_self_consumption() with @cachemethod_until_update to cache
results in memory during an energy management/ optimization run. Replacement
of dict type caching in inverter is now possible because all optimization
runs are properly locked and the memory cache CacheUntilUpdateStore is properly
cleared at the start of any energy management/ optimization operation.
* chore: refactor genetic
Refactor the genetic algorithm modules for enhanced module structure and better
readability. Removed unnecessary and overcomplex devices singleton. Also
split devices configuration from genetic algorithm parameters to allow further
development independently from genetic algorithm parameter format. Move
charge rates configuration for electric vehicles from optimization to devices
configuration to allow to have different charge rates for different cars in
the future.
* chore: Rename memory cache to CacheEnergyManagementStore
The name better resembles the task of the cache to chache function and method
results for an energy management run. Also the decorator functions are renamed
accordingly: cachemethod_energy_management, cache_energy_management
* chore: use class properties for config/ems/prediction mixin classes
* chore: skip debug logs from mathplotlib
Mathplotlib is very noisy in debug mode.
* chore: automatically sync bokeh js to bokeh python package
bokeh was updated to 3.8.0, make JS CDN automatically follow the package version.
* chore: rename hello.py to about.py
Make hello.py the adapted EOSdash about page.
* chore: remove demo page from EOSdash
As no the plan and prediction pages are working without configuration, the demo
page is no longer necessary
* chore: split test_server.py for system test
Split test_server.py to create explicit test_system.py for system tests.
* chore: move doc utils to generate_config_md.py
The doc utils are only used in scripts/generate_config_md.py. Move it there to
attribute for strong cohesion.
* chore: improve pydantic merge model documentation
* chore: remove pendulum warning from readme
* chore: remove GitHub discussions from contributing documentation
Github discussions is to be replaced by Akkudoktor.net.
* chore(release): bump version to 0.1.0+dev for development
* build(deps): bump fastapi[standard] from 0.115.14 to 0.117.1
bump fastapi and make coverage version (for pytest-cov) explicit to avoid pip break.
* build(deps): bump uvicorn from 0.36.0 to 0.37.0
BREAKING CHANGE: EOS configuration changed. V1 API changed.
- The available_charge_rates_percent configuration is removed from optimization.
Use the new charge_rate configuration for the electric vehicle
- Optimization configuration parameter hours renamed to horizon_hours
- Device configuration now has to provide the number of devices and device
properties per device.
- Specific prediction provider configuration to be provided by explicit
configuration item (no union for all providers).
- Measurement keys to be provided as a list.
- New feed in tariff providers have to be configured.
- /v1/measurement/loadxxx endpoints are removed. Use generic mesaurement endpoints.
- /v1/admin/cache/clear now clears all cache files. Use
/v1/admin/cache/clear-expired to only clear all expired cache files.
Signed-off-by: Bobby Noelte <b0661n0e17e@gmail.com>
2025-10-28 02:50:31 +01:00
|
|
|
```json
|
2025-01-03 00:31:20 +01:00
|
|
|
{
|
|
|
|
|
"start_datetime": "2024-01-01 00:00:00",
|
|
|
|
|
"interval": "1 Hour",
|
|
|
|
|
"<prediction key>": [value, value, ...],
|
|
|
|
|
"<prediction key>": [value, value, ...],
|
|
|
|
|
...
|
|
|
|
|
}
|
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
#### 2. DateTimeDataFrame
|
|
|
|
|
|
|
|
|
|
A JSON string created from a [pandas](https://pandas.pydata.org/docs/index.html) dataframe with a
|
2025-02-12 14:24:17 +01:00
|
|
|
`DatetimeIndex`. Use
|
|
|
|
|
[pandas.DataFrame.to_json(orient="index")](https://pandas.pydata.org/docs/reference/api/pandas.DataFrame.to_json.html#pandas.DataFrame.to_json).
|
2025-01-03 00:31:20 +01:00
|
|
|
The column name of the data must be the same as the names of the `prediction key`s.
|
|
|
|
|
|
|
|
|
|
#### 3. DateTimeSeries
|
|
|
|
|
|
|
|
|
|
A JSON string created from a [pandas](https://pandas.pydata.org/docs/index.html) series with a
|
2025-02-12 14:24:17 +01:00
|
|
|
`DatetimeIndex`. Use
|
|
|
|
|
[pandas.Series.to_json(orient="index")](https://pandas.pydata.org/docs/reference/api/pandas.Series.to_json.html#pandas.Series.to_json).
|
2025-01-03 00:31:20 +01:00
|
|
|
|
|
|
|
|
## Adjusted Predictions
|
|
|
|
|
|
|
|
|
|
Certain prediction keys include an `_adjusted` suffix, such as `load_total_adjusted`. These
|
|
|
|
|
predictions are adjusted by real data from your system's measurements if given to enhance accuracy.
|
|
|
|
|
|
|
|
|
|
For example, the load prediction provider `LoadAkkudoktor` takes generic load data assembled by
|
|
|
|
|
Akkudoktor.net, maps that to the yearly energy consumption given in the configuration option
|
2025-01-18 14:26:34 +01:00
|
|
|
`loadakkudoktor_year_energy`, and finally adjusts the predicted load by the `loads`
|
2025-01-03 00:31:20 +01:00
|
|
|
of your system.
|
|
|
|
|
|
|
|
|
|
## Prediction Updates
|
|
|
|
|
|
|
|
|
|
Predictions are updated at the start of each energy management run, i.e., when EOS performs
|
|
|
|
|
optimization. Key considerations for updates include:
|
|
|
|
|
|
|
|
|
|
- Predictions sourced from online providers are usually rate-limited to one retrieval per hour.
|
|
|
|
|
- Only predictions with a configured provider are updated.
|
|
|
|
|
- Some providers may not support all generic prediction keys, leading to potential gaps
|
|
|
|
|
in updated predictions even after update.
|
|
|
|
|
|
|
|
|
|
## Accessing Predictions
|
|
|
|
|
|
|
|
|
|
Prediction data can be accessed using the EOS **REST API** via the `/v1/prediction/<...>` endpoints.
|
|
|
|
|
|
2025-01-09 16:52:57 +01:00
|
|
|
In a standard configuration, the [**REST API**](http://localhost:8503/docs) of a running EOS instance
|
|
|
|
|
is available at [http://localhost:8503/docs](http://localhost:8503/docs). This link provides access to
|
2025-01-03 00:31:20 +01:00
|
|
|
the API documentation and allows you to explore available endpoints interactively.
|
|
|
|
|
|
|
|
|
|
To view all available prediction keys, use the **GET** `/v1/prediction/keys` endpoint.
|
|
|
|
|
|
|
|
|
|
If no keys are displayed, or if the ones you need are missing, it indicates that your configuration
|
|
|
|
|
lacks the necessary prediction provider settings. You can configure prediction providers by using
|
|
|
|
|
the **PUT** `/v1/config` endpoint. You may save your configuration to the EOS configuration file.
|
|
|
|
|
|
|
|
|
|
## Electricity Price Prediction
|
|
|
|
|
|
|
|
|
|
Prediction keys:
|
|
|
|
|
|
|
|
|
|
- `elecprice_marketprice_wh`: Electricity market price per Wh (€/Wh).
|
|
|
|
|
- `elecprice_marketprice_kwh`: Electricity market price per kWh (€/kWh).
|
|
|
|
|
|
|
|
|
|
Configuration options:
|
|
|
|
|
|
2025-01-19 21:47:21 +01:00
|
|
|
- `elecprice`: Electricity price configuration.
|
|
|
|
|
|
|
|
|
|
- `provider`: Electricity price provider id of provider to be used.
|
2025-01-03 00:31:20 +01:00
|
|
|
|
2025-01-19 21:47:21 +01:00
|
|
|
- `ElecPriceAkkudoktor`: Retrieves from Akkudoktor.net.
|
2025-06-23 07:29:33 +02:00
|
|
|
- `ElecPriceEnergyCharts`: Retrieves from Energy-Charts.info.
|
2025-01-19 21:47:21 +01:00
|
|
|
- `ElecPriceImport`: Imports from a file or JSON string.
|
2025-01-03 00:31:20 +01:00
|
|
|
|
2025-01-19 21:47:21 +01:00
|
|
|
- `charges_kwh`: Electricity price charges (€/kWh).
|
2025-09-14 23:53:41 +02:00
|
|
|
- `vat_rate`: VAT rate factor applied to electricity price when charges are used (default: 1.19).
|
2025-01-19 21:47:21 +01:00
|
|
|
- `provider_settings.import_file_path`: Path to the file to import electricity price forecast data from.
|
|
|
|
|
- `provider_settings.import_json`: JSON string, dictionary of electricity price forecast value lists.
|
2025-01-03 00:31:20 +01:00
|
|
|
|
2025-01-19 21:47:21 +01:00
|
|
|
### ElecPriceAkkudoktor Provider
|
2025-01-03 00:31:20 +01:00
|
|
|
|
2025-01-19 21:47:21 +01:00
|
|
|
The `ElecPriceAkkudoktor` provider retrieves electricity prices directly from **Akkudoktor.net**,
|
2025-01-03 00:31:20 +01:00
|
|
|
which supplies price data for the next 24 hours. For periods beyond 24 hours, the provider generates
|
|
|
|
|
prices by extrapolating historical price data combined with the most recent actual prices obtained
|
2025-01-18 14:26:34 +01:00
|
|
|
from Akkudoktor.net. Electricity price charges given in the `charges_kwh` configuration
|
2025-01-03 00:31:20 +01:00
|
|
|
option are added.
|
|
|
|
|
|
2025-06-23 07:29:33 +02:00
|
|
|
### ElecPriceEnergyCharts Provider
|
|
|
|
|
|
2025-06-25 08:26:15 +02:00
|
|
|
The `ElecPriceEnergyCharts` provider retrieves day-ahead electricity market prices from
|
|
|
|
|
[Energy-Charts.info](https://www.Energy-Charts.info). It supports both short-term and extended forecasting by combining
|
|
|
|
|
real-time market data with historical price trends.
|
|
|
|
|
|
|
|
|
|
- For the next 24 hours, market prices are fetched directly from Energy-Charts.info.
|
|
|
|
|
- For periods beyond 24 hours, prices are estimated using extrapolation based on historical data and the latest
|
|
|
|
|
available market values.
|
|
|
|
|
|
|
|
|
|
Charges and VAT
|
|
|
|
|
|
|
|
|
|
- If `charges_kwh` configuration option is greater than 0, the electricity price is calculated as:
|
2025-09-14 23:53:41 +02:00
|
|
|
`(market price + charges_kwh) * vat_rate` where `vat_rate` is configurable (default: 1.19 for 19% VAT).
|
2025-06-25 08:26:15 +02:00
|
|
|
- If `charges_kwh` is set to 0, the electricity price is simply: `market_price` (no VAT applied).
|
|
|
|
|
|
|
|
|
|
**Note:** For the most accurate forecasts, it is recommended to set the `historic_hours` parameter to 840.
|
2025-06-23 07:29:33 +02:00
|
|
|
|
2025-01-19 21:47:21 +01:00
|
|
|
### ElecPriceImport Provider
|
2025-01-03 00:31:20 +01:00
|
|
|
|
2025-01-19 21:47:21 +01:00
|
|
|
The `ElecPriceImport` provider is designed to import electricity prices from a file or a JSON
|
2025-01-03 00:31:20 +01:00
|
|
|
string. An external entity should update the file or JSON string whenever new prediction data
|
|
|
|
|
becomes available.
|
|
|
|
|
|
|
|
|
|
The prediction key for the electricity price forecast data is:
|
|
|
|
|
|
|
|
|
|
- `elecprice_marketprice_wh`: Electricity market price per Wh (€/Wh).
|
|
|
|
|
|
|
|
|
|
The electricity proce forecast data must be provided in one of the formats described in
|
2025-02-12 21:35:51 +01:00
|
|
|
<project:#prediction-import-providers>. The data source can be given in the
|
2025-01-18 14:26:34 +01:00
|
|
|
`import_file_path` or `import_json` configuration option.
|
2025-01-03 00:31:20 +01:00
|
|
|
|
2025-02-12 21:35:51 +01:00
|
|
|
The data may additionally or solely be provided by the
|
|
|
|
|
**PUT** `/v1/prediction/import/ElecPriceImport` endpoint.
|
|
|
|
|
|
fix: automatic optimization (#596)
This fix implements the long term goal to have the EOS server run optimization (or
energy management) on regular intervals automatically. Thus clients can request
the current energy management plan at any time and it is updated on regular
intervals without interaction by the client.
This fix started out to "only" make automatic optimization (or energy management)
runs working. It turned out there are several endpoints that in some way
update predictions or run the optimization. To lock against such concurrent attempts
the code had to be refactored to allow control of execution. During refactoring it
became clear that some classes and files are named without a proper reference
to their usage. Thus not only refactoring but also renaming became necessary.
The names are still not the best, but I hope they are more intuitive.
The fix includes several bug fixes that are not directly related to the automatic optimization
but are necessary to keep EOS running properly to do the automatic optimization and
to test and document the changes.
This is a breaking change as the configuration structure changed once again and
the server API was also enhanced and streamlined. The server API that is used by
Andreas and Jörg in their videos has not changed.
* fix: automatic optimization
Allow optimization to automatically run on configured intervals gathering all
optimization parameters from configuration and predictions. The automatic run
can be configured to only run prediction updates skipping the optimization.
Extend documentaion to also cover automatic optimization. Lock automatic runs
against runs initiated by the /optimize or other endpoints. Provide new
endpoints to retrieve the energy management plan and the genetic solution
of the latest automatic optimization run. Offload energy management to thread
pool executor to keep the app more responsive during the CPU heavy optimization
run.
* fix: EOS servers recognize environment variables on startup
Force initialisation of EOS configuration on server startup to assure
all sources of EOS configuration are properly set up and read. Adapt
server tests and configuration tests to also test for environment
variable configuration.
* fix: Remove 0.0.0.0 to localhost translation under Windows
EOS imposed a 0.0.0.0 to localhost translation under Windows for
convenience. This caused some trouble in user configurations. Now, as the
default IP address configuration is 127.0.0.1, the user is responsible
for to set up the correct Windows compliant IP address.
* fix: allow names for hosts additional to IP addresses
* fix: access pydantic model fields by class
Access by instance is deprecated.
* fix: down sampling key_to_array
* fix: make cache clear endpoint clear all cache files
Make /v1/admin/cache/clear clear all cache files. Before it only cleared
expired cache files by default. Add new endpoint /v1/admin/clear-expired
to only clear expired cache files.
* fix: timezonefinder returns Europe/Paris instead of Europe/Berlin
timezonefinder 8.10 got more inaccurate for timezones in europe as there is
a common timezone. Use new package tzfpy instead which is still returning
Europe/Berlin if you are in Germany. tzfpy also claims to be faster than
timezonefinder.
* fix: provider settings configuration
Provider configuration used to be a union holding the settings for several
providers. Pydantic union handling does not always find the correct type
for a provider setting. This led to exceptions in specific configurations.
Now provider settings are explicit comfiguration items for each possible
provider. This is a breaking change as the configuration structure was
changed.
* fix: ClearOutside weather prediction irradiance calculation
Pvlib needs a pandas time index. Convert time index.
* fix: test config file priority
Do not use config_eos fixture as this fixture already creates a config file.
* fix: optimization sample request documentation
Provide all data in documentation of optimization sample request.
* fix: gitlint blocking pip dependency resolution
Replace gitlint by commitizen. Gitlint is not actively maintained anymore.
Gitlint dependencies blocked pip from dependency resolution.
* fix: sync pre-commit config to actual dependency requirements
.pre-commit-config.yaml was out of sync, also requirements-dev.txt.
* fix: missing babel in requirements.txt
Add babel to requirements.txt
* feat: setup default device configuration for automatic optimization
In case the parameters for automatic optimization are not fully defined a
default configuration is setup to allow the automatic energy management
run. The default configuration may help the user to correctly define
the device configuration.
* feat: allow configuration of genetic algorithm parameters
The genetic algorithm parameters for number of individuals, number of
generations, the seed and penalty function parameters are now avaliable
as configuration options.
* feat: allow configuration of home appliance time windows
The time windows a home appliance is allowed to run are now configurable
by the configuration (for /v1 API) and also by the home appliance parameters
(for the classic /optimize API). If there is no such configuration the
time window defaults to optimization hours, which was the standard before
the change. Documentation on how to configure time windows is added.
* feat: standardize mesaurement keys for battery/ ev SoC measurements
The standardized measurement keys to report battery SoC to the device
simulations can now be retrieved from the device configuration as a
read-only config option.
* feat: feed in tariff prediction
Add feed in tarif predictions needed for automatic optimization. The feed in
tariff can be retrieved as fixed feed in tarif or can be imported. Also add
tests for the different feed in tariff providers. Extend documentation to
cover the feed in tariff providers.
* feat: add energy management plan based on S2 standard instructions
EOS can generate an energy management plan as a list of simple instructions.
May be retrieved by the /v1/energy-management/plan endpoint. The instructions
loosely follow the S2 energy management standard.
* feat: make measurement keys configurable by EOS configuration.
The fixed measurement keys are replaced by configurable measurement keys.
* feat: make pendulum DateTime, Date, Duration types usable for pydantic models
Use pydantic_extra_types.pendulum_dt to get pydantic pendulum types. Types are
added to the datetimeutil utility. Remove custom made pendulum adaptations
from EOS pydantic module. Make EOS modules use the pydantic pendulum types
managed by the datetimeutil module instead of the core pendulum types.
* feat: Add Time, TimeWindow, TimeWindowSequence and to_time to datetimeutil.
The time windows are are added to support home appliance time window
configuration. All time classes are also pydantic models. Time is the base
class for time definition derived from pendulum.Time.
* feat: Extend DataRecord by configurable field like data.
Configurable field like data was added to support the configuration of
measurement records.
* feat: Add additional information to health information
Version information is added to the health endpoints of eos and eosDash.
The start time of the last optimization and the latest run time of the energy
management is added to the EOS health information.
* feat: add pydantic merge model tests
* feat: add plan tab to EOSdash
The plan tab displays the current energy management instructions.
* feat: add predictions tab to EOSdash
The predictions tab displays the current predictions.
* feat: add cache management to EOSdash admin tab
The admin tab is extended by a section for cache management. It allows to
clear the cache.
* feat: add about tab to EOSdash
The about tab resembles the former hello tab and provides extra information.
* feat: Adapt changelog and prepare for release management
Release management using commitizen is added. The changelog file is adapted and
teh changelog and a description for release management is added in the
documentation.
* feat(doc): Improve install and devlopment documentation
Provide a more concise installation description in Readme.md and add extra
installation page and development page to documentation.
* chore: Use memory cache for interpolation instead of dict in inverter
Decorate calculate_self_consumption() with @cachemethod_until_update to cache
results in memory during an energy management/ optimization run. Replacement
of dict type caching in inverter is now possible because all optimization
runs are properly locked and the memory cache CacheUntilUpdateStore is properly
cleared at the start of any energy management/ optimization operation.
* chore: refactor genetic
Refactor the genetic algorithm modules for enhanced module structure and better
readability. Removed unnecessary and overcomplex devices singleton. Also
split devices configuration from genetic algorithm parameters to allow further
development independently from genetic algorithm parameter format. Move
charge rates configuration for electric vehicles from optimization to devices
configuration to allow to have different charge rates for different cars in
the future.
* chore: Rename memory cache to CacheEnergyManagementStore
The name better resembles the task of the cache to chache function and method
results for an energy management run. Also the decorator functions are renamed
accordingly: cachemethod_energy_management, cache_energy_management
* chore: use class properties for config/ems/prediction mixin classes
* chore: skip debug logs from mathplotlib
Mathplotlib is very noisy in debug mode.
* chore: automatically sync bokeh js to bokeh python package
bokeh was updated to 3.8.0, make JS CDN automatically follow the package version.
* chore: rename hello.py to about.py
Make hello.py the adapted EOSdash about page.
* chore: remove demo page from EOSdash
As no the plan and prediction pages are working without configuration, the demo
page is no longer necessary
* chore: split test_server.py for system test
Split test_server.py to create explicit test_system.py for system tests.
* chore: move doc utils to generate_config_md.py
The doc utils are only used in scripts/generate_config_md.py. Move it there to
attribute for strong cohesion.
* chore: improve pydantic merge model documentation
* chore: remove pendulum warning from readme
* chore: remove GitHub discussions from contributing documentation
Github discussions is to be replaced by Akkudoktor.net.
* chore(release): bump version to 0.1.0+dev for development
* build(deps): bump fastapi[standard] from 0.115.14 to 0.117.1
bump fastapi and make coverage version (for pytest-cov) explicit to avoid pip break.
* build(deps): bump uvicorn from 0.36.0 to 0.37.0
BREAKING CHANGE: EOS configuration changed. V1 API changed.
- The available_charge_rates_percent configuration is removed from optimization.
Use the new charge_rate configuration for the electric vehicle
- Optimization configuration parameter hours renamed to horizon_hours
- Device configuration now has to provide the number of devices and device
properties per device.
- Specific prediction provider configuration to be provided by explicit
configuration item (no union for all providers).
- Measurement keys to be provided as a list.
- New feed in tariff providers have to be configured.
- /v1/measurement/loadxxx endpoints are removed. Use generic mesaurement endpoints.
- /v1/admin/cache/clear now clears all cache files. Use
/v1/admin/cache/clear-expired to only clear all expired cache files.
Signed-off-by: Bobby Noelte <b0661n0e17e@gmail.com>
2025-10-28 02:50:31 +01:00
|
|
|
## Feed In Tariff Prediction
|
|
|
|
|
|
|
|
|
|
Prediction keys:
|
|
|
|
|
|
|
|
|
|
- `feed_in_tarif_wh`: Feed in tarif per Wh (€/Wh).
|
|
|
|
|
- `feed_in_tarif_kwh`: Feed in tarif per kWh (€/kWh)
|
|
|
|
|
|
|
|
|
|
Configuration options:
|
|
|
|
|
|
|
|
|
|
- `feedintarif`: Feed in tariff configuration.
|
|
|
|
|
|
|
|
|
|
- `provider`: Feed in tariff provider id of provider to be used.
|
|
|
|
|
|
|
|
|
|
- `FeedInTariffFixed`: Provides fixed feed in tariff values.
|
|
|
|
|
- `FeedInTariffImport`: Imports from a file or JSON string.
|
|
|
|
|
|
|
|
|
|
- `provider_settings.feed_in_tariff_kwh`: Fixed feed in tariff (€/kWh).
|
|
|
|
|
- `provider_settings.import_file_path`: Path to the file to import feed in tariff forecast data from.
|
|
|
|
|
- `provider_settings.import_json`: JSON string, dictionary of feed in tariff value lists.
|
|
|
|
|
|
2025-01-03 00:31:20 +01:00
|
|
|
## Load Prediction
|
|
|
|
|
|
|
|
|
|
Prediction keys:
|
|
|
|
|
|
|
|
|
|
- `load_mean`: Predicted load mean value (W).
|
|
|
|
|
- `load_std`: Predicted load standard deviation (W).
|
|
|
|
|
- `load_mean_adjusted`: Predicted load mean value adjusted by load measurement (W).
|
|
|
|
|
|
|
|
|
|
Configuration options:
|
|
|
|
|
|
2025-01-19 21:47:21 +01:00
|
|
|
- `load`: Load configuration.
|
2025-01-03 00:31:20 +01:00
|
|
|
|
2025-01-19 21:47:21 +01:00
|
|
|
- `provider`: Load provider id of provider to be used.
|
2025-01-03 00:31:20 +01:00
|
|
|
|
2025-01-19 21:47:21 +01:00
|
|
|
- `LoadAkkudoktor`: Retrieves from local database.
|
2025-07-19 08:55:16 +02:00
|
|
|
- `LoadVrm`: Retrieves data from the VRM API by Victron Energy.
|
2025-01-19 21:47:21 +01:00
|
|
|
- `LoadImport`: Imports from a file or JSON string.
|
|
|
|
|
|
|
|
|
|
- `provider_settings.loadakkudoktor_year_energy`: Yearly energy consumption (kWh).
|
|
|
|
|
- `provider_settings.loadimport_file_path`: Path to the file to import load forecast data from.
|
|
|
|
|
- `provider_settings.loadimport_json`: JSON string, dictionary of load forecast value lists.
|
2025-01-03 00:31:20 +01:00
|
|
|
|
|
|
|
|
### LoadAkkudoktor Provider
|
|
|
|
|
|
|
|
|
|
The `LoadAkkudoktor` provider retrieves generic load data from a local database and tailors it to
|
|
|
|
|
align with the annual energy consumption specified in the `loadakkudoktor_year_energy` configuration
|
|
|
|
|
option.
|
|
|
|
|
|
2025-07-19 08:55:16 +02:00
|
|
|
### LoadVrm Provider
|
|
|
|
|
|
|
|
|
|
The `LoadVrm` provider retrieves load forecast data from the VRM API by Victron Energy.
|
|
|
|
|
To receive forecasts, the system data must be configured under Dynamic ESS in the VRM portal.
|
|
|
|
|
To query the forecasts, an API token is required, which can also be created in the VRM portal under Preferences.
|
|
|
|
|
This token must be stored in the EOS configuration along with the VRM-Installations-ID.
|
|
|
|
|
|
|
|
|
|
```python
|
|
|
|
|
{
|
|
|
|
|
"load": {
|
|
|
|
|
"provider": "LoadVrm",
|
|
|
|
|
"provider_settings": {
|
|
|
|
|
"load_vrm_token": "dummy-token",
|
|
|
|
|
"load_vrm_idsite": 12345
|
|
|
|
|
}
|
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
The prediction keys for the load forecast data are:
|
|
|
|
|
|
|
|
|
|
- `load_mean`: Predicted load mean value (W).
|
|
|
|
|
|
2025-01-03 00:31:20 +01:00
|
|
|
### LoadImport Provider
|
|
|
|
|
|
|
|
|
|
The `LoadImport` provider is designed to import load forecast data from a file or a JSON
|
|
|
|
|
string. An external entity should update the file or JSON string whenever new prediction data
|
|
|
|
|
becomes available.
|
|
|
|
|
|
|
|
|
|
The prediction keys for the load forecast data are:
|
|
|
|
|
|
|
|
|
|
- `load_mean`: Predicted load mean value (W).
|
|
|
|
|
- `load_std`: Predicted load standard deviation (W).
|
|
|
|
|
- `load_mean_adjusted`: Predicted load mean value adjusted by load measurement (W).
|
|
|
|
|
|
|
|
|
|
The load forecast data must be provided in one of the formats described in
|
2025-02-12 21:35:51 +01:00
|
|
|
<project:#prediction-import-providers>. The data source can be given in the `loadimport_file_path`
|
2025-01-03 00:31:20 +01:00
|
|
|
or `loadimport_json` configuration option.
|
|
|
|
|
|
2025-02-12 21:35:51 +01:00
|
|
|
The data may additionally or solely be provided by the
|
|
|
|
|
**PUT** `/v1/prediction/import/LoadImport` endpoint.
|
|
|
|
|
|
2025-01-03 00:31:20 +01:00
|
|
|
## PV Power Prediction
|
|
|
|
|
|
|
|
|
|
Prediction keys:
|
|
|
|
|
|
2025-01-19 21:47:21 +01:00
|
|
|
- `pvforecast_ac_power`: Total DC power (W).
|
|
|
|
|
- `pvforecast_dc_power`: Total AC power (W).
|
2025-01-03 00:31:20 +01:00
|
|
|
|
|
|
|
|
Configuration options:
|
|
|
|
|
|
2025-01-24 21:14:37 +01:00
|
|
|
- `general`: General configuration.
|
2025-01-19 21:47:21 +01:00
|
|
|
|
|
|
|
|
- `latitude`: Latitude in decimal degrees, between -90 and 90, north is positive (ISO 19115) (°)"
|
|
|
|
|
- `longitude`: Longitude in decimal degrees, within -180 to 180 (°)
|
|
|
|
|
|
|
|
|
|
- `pvforecast`: PV forecast configuration.
|
|
|
|
|
|
|
|
|
|
- `provider`: PVForecast provider id of provider to be used.
|
|
|
|
|
|
|
|
|
|
- `PVForecastAkkudoktor`: Retrieves from Akkudoktor.net.
|
2025-07-19 08:55:16 +02:00
|
|
|
- `PVForecastVrm`: Retrieves data from the VRM API by Victron Energy.
|
2025-01-19 21:47:21 +01:00
|
|
|
- `PVForecastImport`: Imports from a file or JSON string.
|
|
|
|
|
|
|
|
|
|
- `planes[].surface_tilt`: Tilt angle from horizontal plane. Ignored for two-axis tracking.
|
2025-02-12 14:24:17 +01:00
|
|
|
- `planes[].surface_azimuth`: Orientation (azimuth angle) of the (fixed) plane.
|
|
|
|
|
Clockwise from north (north=0, east=90, south=180, west=270).
|
2025-01-19 21:47:21 +01:00
|
|
|
- `planes[].userhorizon`: Elevation of horizon in degrees, at equally spaced azimuth clockwise from north.
|
|
|
|
|
- `planes[].peakpower`: Nominal power of PV system in kW.
|
|
|
|
|
- `planes[].pvtechchoice`: PV technology. One of 'crystSi', 'CIS', 'CdTe', 'Unknown'.
|
2025-02-12 14:24:17 +01:00
|
|
|
- `planes[].mountingplace`: Type of mounting for PV system.
|
|
|
|
|
Options are 'free' for free-standing and 'building' for building-integrated.
|
2025-01-19 21:47:21 +01:00
|
|
|
- `planes[].loss`: Sum of PV system losses in percent
|
2025-02-12 14:24:17 +01:00
|
|
|
- `planes[].trackingtype`: Type of suntracking.
|
|
|
|
|
0=fixed,
|
|
|
|
|
1=single horizontal axis aligned north-south,
|
|
|
|
|
2=two-axis tracking,
|
|
|
|
|
3=vertical axis tracking,
|
|
|
|
|
4=single horizontal axis aligned east-west,
|
|
|
|
|
5=single inclined axis aligned north-south.
|
2025-01-19 21:47:21 +01:00
|
|
|
- `planes[].optimal_surface_tilt`: Calculate the optimum tilt angle. Ignored for two-axis tracking.
|
|
|
|
|
- `planes[].optimalangles`: Calculate the optimum tilt and azimuth angles. Ignored for two-axis tracking.
|
|
|
|
|
- `planes[].albedo`: Proportion of the light hitting the ground that it reflects back.
|
|
|
|
|
- `planes[].module_model`: Model of the PV modules of this plane.
|
|
|
|
|
- `planes[].inverter_model`: Model of the inverter of this plane.
|
|
|
|
|
- `planes[].inverter_paco`: AC power rating of the inverter. [W]
|
|
|
|
|
- `planes[].modules_per_string`: Number of the PV modules of the strings of this plane.
|
|
|
|
|
- `planes[].strings_per_inverter`: Number of the strings of the inverter of this plane.
|
|
|
|
|
- `provider_settings.import_file_path`: Path to the file to import PV forecast data from.
|
|
|
|
|
- `provider_settings.import_json`: JSON string, dictionary of PV forecast value lists.
|
2025-01-03 00:31:20 +01:00
|
|
|
|
2025-03-23 13:27:40 +01:00
|
|
|
---
|
2025-01-05 14:41:07 +01:00
|
|
|
|
2025-02-12 14:24:17 +01:00
|
|
|
Detailed definitions taken from
|
|
|
|
|
[PVGIS](https://joint-research-centre.ec.europa.eu/photovoltaic-geographical-information-system-pvgis/getting-started-pvgis/pvgis-user-manual_en).
|
2025-01-05 14:41:07 +01:00
|
|
|
|
2025-01-19 21:47:21 +01:00
|
|
|
- `pvtechchoice`
|
2025-01-05 14:41:07 +01:00
|
|
|
|
2025-02-12 14:24:17 +01:00
|
|
|
The performance of PV modules depends on the temperature and on the solar irradiance, but the exact
|
|
|
|
|
dependence varies between different types of PV modules. At the moment we can estimate the losses
|
|
|
|
|
due to temperature and irradiance effects for the following types of modules: crystalline silicon
|
|
|
|
|
cells; thin film modules made from CIS or CIGS and thin film modules made from Cadmium Telluride
|
|
|
|
|
(CdTe).
|
2025-01-05 14:41:07 +01:00
|
|
|
|
2025-02-12 14:24:17 +01:00
|
|
|
For other technologies (especially various amorphous technologies), this correction cannot be
|
|
|
|
|
calculated here. If you choose one of the first three options here the calculation of performance
|
|
|
|
|
will take into account the temperature dependence of the performance of the chosen technology. If
|
|
|
|
|
you choose the other option (other/unknown), the calculation will assume a loss of 8% of power due
|
|
|
|
|
to temperature effects (a generic value which has found to be reasonable for temperate climates).
|
2025-01-05 14:41:07 +01:00
|
|
|
|
2025-02-12 14:24:17 +01:00
|
|
|
PV power output also depends on the spectrum of the solar radiation. PVGIS can calculate how the
|
|
|
|
|
variations of the spectrum of sunlight affects the overall energy production from a PV system. At
|
|
|
|
|
the moment this calculation can be done for crystalline silicon and CdTe modules. Note that this
|
|
|
|
|
calculation is not yet available when using the NSRDB solar radiation database.
|
2025-01-05 14:41:07 +01:00
|
|
|
|
2025-01-19 21:47:21 +01:00
|
|
|
- `peakpower`
|
2025-01-05 14:41:07 +01:00
|
|
|
|
2025-02-12 14:24:17 +01:00
|
|
|
This is the power that the manufacturer declares that the PV array can produce under standard test
|
|
|
|
|
conditions (STC), which are a constant 1000W of solar irradiation per square meter in the plane of
|
|
|
|
|
the array, at an array temperature of 25°C. The peak power should be entered in kilowatt-peak (kWp).
|
|
|
|
|
If you do not know the declared peak power of your modules but instead know the area of the modules
|
|
|
|
|
and the declared conversion efficiency (in percent), you can calculate the peak power as
|
2025-03-23 13:27:40 +01:00
|
|
|
power = area \* efficiency / 100.
|
2025-01-05 14:41:07 +01:00
|
|
|
|
2025-02-12 14:24:17 +01:00
|
|
|
Bifacial modules: PVGIS doesn't make specific calculations for bifacial modules at present. Users
|
|
|
|
|
who wish to explore the possible benefits of this technology can input the power value for Bifacial
|
|
|
|
|
Nameplate Irradiance. This can also be can also be estimated from the front side peak power P_STC
|
|
|
|
|
value and the bifaciality factor, φ (if reported in the module data sheet) as:
|
2025-03-23 13:27:40 +01:00
|
|
|
P_BNPI = P_STC \* (1 + φ \* 0.135). NB this bifacial approach is not appropriate for BAPV or BIPV
|
2025-02-12 14:24:17 +01:00
|
|
|
installations or for modules mounting on a N-S axis i.e. facing E-W.
|
2025-01-05 14:41:07 +01:00
|
|
|
|
2025-01-19 21:47:21 +01:00
|
|
|
- `loss`
|
2025-01-05 14:41:07 +01:00
|
|
|
|
2025-02-12 14:24:17 +01:00
|
|
|
The estimated system losses are all the losses in the system, which cause the power actually
|
|
|
|
|
delivered to the electricity grid to be lower than the power produced by the PV modules. There are
|
|
|
|
|
several causes for this loss, such as losses in cables, power inverters, dirt (sometimes snow) on
|
|
|
|
|
the modules and so on. Over the years the modules also tend to lose a bit of their power, so the
|
|
|
|
|
average yearly output over the lifetime of the system will be a few percent lower than the output
|
|
|
|
|
in the first years.
|
2025-01-05 14:41:07 +01:00
|
|
|
|
2025-02-12 14:24:17 +01:00
|
|
|
We have given a default value of 14% for the overall losses. If you have a good idea that your value
|
|
|
|
|
will be different (maybe due to a really high-efficiency inverter) you may reduce this value a little.
|
2025-01-05 14:41:07 +01:00
|
|
|
|
2025-01-19 21:47:21 +01:00
|
|
|
- `mountingplace`
|
2025-01-05 14:41:07 +01:00
|
|
|
|
2025-02-12 14:24:17 +01:00
|
|
|
For fixed (non-tracking) systems, the way the modules are mounted will have an influence on the
|
|
|
|
|
temperature of the module, which in turn affects the efficiency. Experiments have shown that if the
|
|
|
|
|
movement of air behind the modules is restricted, the modules can get considerably hotter
|
|
|
|
|
(up to 15°C at 1000W/m2 of sunlight).
|
2025-01-05 14:41:07 +01:00
|
|
|
|
2025-02-12 14:24:17 +01:00
|
|
|
In PVGIS there are two possibilities: free-standing, meaning that the modules are mounted on a rack
|
|
|
|
|
with air flowing freely behind the modules; and building- integrated, which means that the modules
|
|
|
|
|
are completely built into the structure of the wall or roof of a building, with no air movement
|
|
|
|
|
behind the modules.
|
2025-01-05 14:41:07 +01:00
|
|
|
|
2025-02-12 14:24:17 +01:00
|
|
|
Some types of mounting are in between these two extremes, for instance if the modules are mounted on
|
|
|
|
|
a roof with curved roof tiles, allowing air to move behind the modules. In such cases, the
|
|
|
|
|
performance will be somewhere between the results of the two calculations that are possible here.
|
2025-01-05 14:41:07 +01:00
|
|
|
|
2025-01-19 21:47:21 +01:00
|
|
|
- `userhorizon`
|
2025-01-05 14:41:07 +01:00
|
|
|
|
|
|
|
|
Elevation of horizon in degrees, at equally spaced azimuth clockwise from north. In the user horizon
|
|
|
|
|
data each number represents the horizon height in degrees in a certain compass direction around the
|
|
|
|
|
point of interest. The horizon heights should be given in a clockwise direction starting at North;
|
|
|
|
|
that is, from North, going to East, South, West, and back to North. The values are assumed to
|
|
|
|
|
represent equal angular distance around the horizon. For instance, if you have 36 values, the first
|
|
|
|
|
point is due north, the next is 10 degrees east of north, and so on, until the last point, 10
|
|
|
|
|
degrees west of north.
|
|
|
|
|
|
2025-03-23 13:27:40 +01:00
|
|
|
---
|
2025-01-05 14:41:07 +01:00
|
|
|
|
2025-02-12 14:24:17 +01:00
|
|
|
Most of the configuration options are in line with the
|
|
|
|
|
[PVLib](https://pvlib-python.readthedocs.io/en/stable/_modules/pvlib/iotools/pvgis.html) definition for PVGIS data.
|
2025-01-05 14:41:07 +01:00
|
|
|
|
|
|
|
|
Detailed definitions from **PVLib** for PVGIS data.
|
|
|
|
|
|
2025-01-19 21:47:21 +01:00
|
|
|
- `surface_tilt`:
|
2025-01-05 14:41:07 +01:00
|
|
|
|
|
|
|
|
Tilt angle from horizontal plane.
|
|
|
|
|
|
2025-01-19 21:47:21 +01:00
|
|
|
- `surface_azimuth`
|
2025-01-05 14:41:07 +01:00
|
|
|
|
|
|
|
|
Orientation (azimuth angle) of the (fixed) plane. Clockwise from north (north=0, east=90, south=180,
|
|
|
|
|
west=270). This is offset 180 degrees from the convention used by PVGIS.
|
|
|
|
|
|
2025-03-23 13:27:40 +01:00
|
|
|
---
|
2025-01-05 14:41:07 +01:00
|
|
|
|
2025-01-03 00:31:20 +01:00
|
|
|
### PVForecastAkkudoktor Provider
|
|
|
|
|
|
|
|
|
|
The `PVForecastAkkudoktor` provider retrieves the PV power forecast data directly from
|
|
|
|
|
**Akkudoktor.net**.
|
|
|
|
|
|
2025-01-19 21:47:21 +01:00
|
|
|
The following prediction configuration options of the PV system must be set:
|
2025-01-03 00:31:20 +01:00
|
|
|
|
2025-01-20 22:58:59 +01:00
|
|
|
- `general.latitude`: Latitude in decimal degrees, between -90 and 90, north is positive (ISO 19115) (°)"
|
|
|
|
|
- `general.longitude`: Longitude in decimal degrees, within -180 to 180 (°)
|
2025-01-03 00:31:20 +01:00
|
|
|
|
2025-01-19 21:47:21 +01:00
|
|
|
For each plane of the PV system the following configuration options must be set:
|
2025-01-03 00:31:20 +01:00
|
|
|
|
2025-01-19 21:47:21 +01:00
|
|
|
- `pvforecast.planes[].surface_tilt`: Tilt angle from horizontal plane. Ignored for two-axis tracking.
|
2025-02-12 14:24:17 +01:00
|
|
|
- `pvforecast.planes[].surface_azimuth`: Orientation (azimuth angle) of the (fixed) plane.
|
|
|
|
|
Clockwise from north (north=0, east=90, south=180, west=270).
|
2025-01-19 21:47:21 +01:00
|
|
|
- `pvforecast.planes[].userhorizon`: Elevation of horizon in degrees, at equally spaced azimuth clockwise from north.
|
|
|
|
|
- `pvforecast.planes[].inverter_paco`: AC power rating of the inverter. [W]
|
|
|
|
|
- `pvforecast.planes[].peakpower`: Nominal power of PV system in kW.
|
2025-01-03 00:31:20 +01:00
|
|
|
|
|
|
|
|
Example:
|
|
|
|
|
|
|
|
|
|
```Python
|
|
|
|
|
{
|
2025-01-20 22:58:59 +01:00
|
|
|
"general": {
|
2025-01-19 21:47:21 +01:00
|
|
|
"latitude": 50.1234,
|
|
|
|
|
"longitude": 9.7654,
|
|
|
|
|
},
|
|
|
|
|
"pvforecast": {
|
|
|
|
|
"provider": "PVForecastAkkudoktor",
|
|
|
|
|
"planes": [
|
|
|
|
|
{
|
|
|
|
|
"peakpower": 5.0,
|
|
|
|
|
"surface_azimuth": -10,
|
|
|
|
|
"surface_tilt": 7,
|
|
|
|
|
"userhorizon": [20, 27, 22, 20],
|
2025-06-10 22:00:28 +02:00
|
|
|
"inverter_paco": 10000
|
2025-01-19 21:47:21 +01:00
|
|
|
},
|
|
|
|
|
{
|
|
|
|
|
"peakpower": 4.8,
|
|
|
|
|
"surface_azimuth": -90,
|
|
|
|
|
"surface_tilt": 7,
|
|
|
|
|
"userhorizon": [30, 30, 30, 50],
|
2025-06-10 22:00:28 +02:00
|
|
|
"inverter_paco": 10000
|
2025-01-19 21:47:21 +01:00
|
|
|
},
|
|
|
|
|
{
|
|
|
|
|
"peakpower": 1.4,
|
|
|
|
|
"surface_azimuth": -40,
|
|
|
|
|
"surface_tilt": 60,
|
|
|
|
|
"userhorizon": [60, 30, 0, 30],
|
2025-06-10 22:00:28 +02:00
|
|
|
"inverter_paco": 2000
|
2025-01-19 21:47:21 +01:00
|
|
|
},
|
|
|
|
|
{
|
|
|
|
|
"peakpower": 1.6,
|
|
|
|
|
"surface_azimuth": 5,
|
|
|
|
|
"surface_tilt": 45,
|
|
|
|
|
"userhorizon": [45, 25, 30, 60],
|
2025-06-10 22:00:28 +02:00
|
|
|
"inverter_paco": 1400
|
2025-01-19 21:47:21 +01:00
|
|
|
}
|
|
|
|
|
]
|
|
|
|
|
}
|
2025-01-03 00:31:20 +01:00
|
|
|
}
|
|
|
|
|
```
|
|
|
|
|
|
2025-07-19 08:55:16 +02:00
|
|
|
### PVForecastVrm Provider
|
|
|
|
|
|
|
|
|
|
The `PVForecastVrm` provider retrieves pv power forecast data from the VRM API by Victron Energy.
|
|
|
|
|
To receive forecasts, the system data must be configured under Dynamic ESS in the VRM portal.
|
|
|
|
|
To query the forecasts, an API token is required, which can also be created in the VRM portal under Preferences.
|
|
|
|
|
This token must be stored in the EOS configuration along with the VRM-Installations-ID.
|
|
|
|
|
|
|
|
|
|
```python
|
|
|
|
|
{
|
|
|
|
|
"pvforecast": {
|
|
|
|
|
"provider": "PVForecastVrm",
|
|
|
|
|
"provider_settings": {
|
|
|
|
|
"pvforecast_vrm_token": "dummy-token",
|
|
|
|
|
"pvforecast_vrm_idsite": 12345
|
|
|
|
|
}
|
|
|
|
|
}
|
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
The prediction keys for the PV forecast data are:
|
|
|
|
|
|
|
|
|
|
- `pvforecast_dc_power`: Total DC power (W).
|
|
|
|
|
|
2025-01-03 00:31:20 +01:00
|
|
|
### PVForecastImport Provider
|
|
|
|
|
|
|
|
|
|
The `PVForecastImport` provider is designed to import PV forecast data from a file or a JSON
|
|
|
|
|
string. An external entity should update the file or JSON string whenever new prediction data
|
|
|
|
|
becomes available.
|
|
|
|
|
|
|
|
|
|
The prediction keys for the PV forecast data are:
|
|
|
|
|
|
2025-07-19 08:55:16 +02:00
|
|
|
- `pvforecast_ac_power`: Total AC power (W).
|
|
|
|
|
- `pvforecast_dc_power`: Total DC power (W).
|
2025-01-03 00:31:20 +01:00
|
|
|
|
|
|
|
|
The PV forecast data must be provided in one of the formats described in
|
2025-02-12 21:35:51 +01:00
|
|
|
<project:#prediction-import-providers>. The data source can be given in the
|
2025-01-19 18:12:50 +01:00
|
|
|
`import_file_path` or `import_json` configuration option.
|
2025-01-03 00:31:20 +01:00
|
|
|
|
2025-02-12 21:35:51 +01:00
|
|
|
The data may additionally or solely be provided by the
|
|
|
|
|
**PUT** `/v1/prediction/import/PVForecastImport` endpoint.
|
|
|
|
|
|
2025-01-03 00:31:20 +01:00
|
|
|
## Weather Prediction
|
|
|
|
|
|
|
|
|
|
Prediction keys:
|
|
|
|
|
|
|
|
|
|
- `weather_dew_point`: Dew Point (°C)
|
|
|
|
|
- `weather_dhi`: Diffuse Horizontal Irradiance (W/m2)
|
|
|
|
|
- `weather_dni`: Direct Normal Irradiance (W/m2)
|
|
|
|
|
- `weather_feels_like`: Feels Like (°C)
|
|
|
|
|
- `weather_fog`: Fog (%)
|
|
|
|
|
- `weather_frost_chance`: Chance of Frost
|
|
|
|
|
- `weather_ghi`: Global Horizontal Irradiance (W/m2)
|
|
|
|
|
- `weather_high_clouds`: High Clouds (% Sky Obscured)
|
|
|
|
|
- `weather_low_clouds`: Low Clouds (% Sky Obscured)
|
|
|
|
|
- `weather_medium_clouds`: Medium Clouds (% Sky Obscured)
|
|
|
|
|
- `weather_ozone`: Ozone (du)
|
|
|
|
|
- `weather_precip_amt`: Precipitation Amount (mm)
|
|
|
|
|
- `weather_precip_prob`: Precipitation Probability (%)
|
|
|
|
|
- `weather_preciptable_water`: Precipitable Water (cm)
|
|
|
|
|
- `weather_precip_type`: Precipitation Type
|
|
|
|
|
- `weather_pressure`: Pressure (mb)
|
|
|
|
|
- `weather_relative_humidity`: Relative Humidity (%)
|
|
|
|
|
- `weather_temp_air`: Temperature (°C)
|
|
|
|
|
- `weather_total_clouds`: Total Clouds (% Sky Obscured)
|
|
|
|
|
- `weather_visibility`: Visibility (m)
|
|
|
|
|
- `weather_wind_direction`: "Wind Direction (°)
|
|
|
|
|
- `weather_wind_speed`: Wind Speed (kmph)
|
|
|
|
|
|
|
|
|
|
Configuration options:
|
|
|
|
|
|
2025-01-19 21:47:21 +01:00
|
|
|
- `weather`: General weather configuration.
|
|
|
|
|
|
|
|
|
|
- `provider`: Load provider id of provider to be used.
|
2025-01-03 00:31:20 +01:00
|
|
|
|
2025-02-12 14:24:17 +01:00
|
|
|
- `BrightSky`: Retrieves from [BrightSky](https://api.brightsky.dev).
|
|
|
|
|
- `ClearOutside`: Retrieves from [ClearOutside](https://clearoutside.com/forecast).
|
2025-01-19 21:47:21 +01:00
|
|
|
- `LoadImport`: Imports from a file or JSON string.
|
2025-01-03 00:31:20 +01:00
|
|
|
|
2025-01-19 21:47:21 +01:00
|
|
|
- `provider_settings.import_file_path`: Path to the file to import weatherforecast data from.
|
|
|
|
|
- `provider_settings.import_json`: JSON string, dictionary of weather forecast value lists.
|
2025-01-03 00:31:20 +01:00
|
|
|
|
|
|
|
|
### BrightSky Provider
|
|
|
|
|
|
|
|
|
|
The `BrightSky` provider retrieves the PV power forecast data directly from
|
|
|
|
|
[**BrightSky**](https://api.brightsky.dev).
|
|
|
|
|
|
|
|
|
|
The provider provides forecast data for the following prediction keys:
|
|
|
|
|
|
|
|
|
|
- `weather_dew_point`: Dew Point (°C)
|
|
|
|
|
- `weather_ghi`: Global Horizontal Irradiance (W/m2)
|
|
|
|
|
- `weather_precip_amt`: Precipitation Amount (mm)
|
|
|
|
|
- `weather_precip_prob`: Precipitation Probability (%)
|
|
|
|
|
- `weather_pressure`: Pressure (mb)
|
|
|
|
|
- `weather_relative_humidity`: Relative Humidity (%)
|
|
|
|
|
- `weather_temp_air`: Temperature (°C)
|
|
|
|
|
- `weather_total_clouds`: Total Clouds (% Sky Obscured)
|
|
|
|
|
- `weather_visibility`: Visibility (m)
|
|
|
|
|
- `weather_wind_direction`: "Wind Direction (°)
|
|
|
|
|
- `weather_wind_speed`: Wind Speed (kmph)
|
|
|
|
|
|
|
|
|
|
### ClearOutside Provider
|
|
|
|
|
|
|
|
|
|
The `ClearOutside` provider retrieves the PV power forecast data directly from
|
|
|
|
|
[**ClearOutside**](https://clearoutside.com/forecast).
|
|
|
|
|
|
|
|
|
|
The provider provides forecast data for the following prediction keys:
|
|
|
|
|
|
|
|
|
|
- `weather_dew_point`: Dew Point (°C)
|
|
|
|
|
- `weather_dhi`: Diffuse Horizontal Irradiance (W/m2)
|
|
|
|
|
- `weather_dni`: Direct Normal Irradiance (W/m2)
|
|
|
|
|
- `weather_feels_like`: Feels Like (°C)
|
|
|
|
|
- `weather_fog`: Fog (%)
|
|
|
|
|
- `weather_frost_chance`: Chance of Frost
|
|
|
|
|
- `weather_ghi`: Global Horizontal Irradiance (W/m2)
|
|
|
|
|
- `weather_high_clouds`: High Clouds (% Sky Obscured)
|
|
|
|
|
- `weather_low_clouds`: Low Clouds (% Sky Obscured)
|
|
|
|
|
- `weather_medium_clouds`: Medium Clouds (% Sky Obscured)
|
|
|
|
|
- `weather_ozone`: Ozone (du)
|
|
|
|
|
- `weather_precip_amt`: Precipitation Amount (mm)
|
|
|
|
|
- `weather_precip_prob`: Precipitation Probability (%)
|
|
|
|
|
- `weather_preciptable_water`: Precipitable Water (cm)
|
|
|
|
|
- `weather_precip_type`: Precipitation Type
|
|
|
|
|
- `weather_pressure`: Pressure (mb)
|
|
|
|
|
- `weather_relative_humidity`: Relative Humidity (%)
|
|
|
|
|
- `weather_temp_air`: Temperature (°C)
|
|
|
|
|
- `weather_total_clouds`: Total Clouds (% Sky Obscured)
|
|
|
|
|
- `weather_visibility`: Visibility (m)
|
|
|
|
|
- `weather_wind_direction`: "Wind Direction (°)
|
|
|
|
|
- `weather_wind_speed`: Wind Speed (kmph)
|
|
|
|
|
|
|
|
|
|
### WeatherImport Provider
|
|
|
|
|
|
|
|
|
|
The `WeatherImport` provider is designed to import weather forecast data from a file or a JSON
|
|
|
|
|
string. An external entity should update the file or JSON string whenever new prediction data
|
|
|
|
|
becomes available.
|
|
|
|
|
|
2025-02-12 21:35:51 +01:00
|
|
|
The prediction keys for the weather forecast data are:
|
2025-01-03 00:31:20 +01:00
|
|
|
|
|
|
|
|
- `weather_dew_point`: Dew Point (°C)
|
|
|
|
|
- `weather_dhi`: Diffuse Horizontal Irradiance (W/m2)
|
|
|
|
|
- `weather_dni`: Direct Normal Irradiance (W/m2)
|
|
|
|
|
- `weather_feels_like`: Feels Like (°C)
|
|
|
|
|
- `weather_fog`: Fog (%)
|
|
|
|
|
- `weather_frost_chance`: Chance of Frost
|
|
|
|
|
- `weather_ghi`: Global Horizontal Irradiance (W/m2)
|
|
|
|
|
- `weather_high_clouds`: High Clouds (% Sky Obscured)
|
|
|
|
|
- `weather_low_clouds`: Low Clouds (% Sky Obscured)
|
|
|
|
|
- `weather_medium_clouds`: Medium Clouds (% Sky Obscured)
|
|
|
|
|
- `weather_ozone`: Ozone (du)
|
|
|
|
|
- `weather_precip_amt`: Precipitation Amount (mm)
|
|
|
|
|
- `weather_precip_prob`: Precipitation Probability (%)
|
|
|
|
|
- `weather_preciptable_water`: Precipitable Water (cm)
|
|
|
|
|
- `weather_precip_type`: Precipitation Type
|
|
|
|
|
- `weather_pressure`: Pressure (mb)
|
|
|
|
|
- `weather_relative_humidity`: Relative Humidity (%)
|
|
|
|
|
- `weather_temp_air`: Temperature (°C)
|
|
|
|
|
- `weather_total_clouds`: Total Clouds (% Sky Obscured)
|
|
|
|
|
- `weather_visibility`: Visibility (m)
|
|
|
|
|
- `weather_wind_direction`: "Wind Direction (°)
|
|
|
|
|
- `weather_wind_speed`: Wind Speed (kmph)
|
|
|
|
|
|
|
|
|
|
The PV forecast data must be provided in one of the formats described in
|
2025-02-12 21:35:51 +01:00
|
|
|
<project:#prediction-import-providers>. The data source can be given in the
|
2025-01-19 18:12:50 +01:00
|
|
|
`import_file_path` or `import_json` configuration option.
|
2025-02-12 21:35:51 +01:00
|
|
|
|
|
|
|
|
The data may additionally or solely be provided by the
|
|
|
|
|
**PUT** `/v1/prediction/import/WeatherImport` endpoint.
|