Mirrors/EOS
1
0
Fork 0
mirror of https://github.com/Akkudoktor-EOS/EOS.git synced 2025-10-30 14:26:21 +00:00
Code Issues Actions 4 Packages Projects Releases Wiki Activity

fix: automatic optimization (#596)

Browse Source 
This fix implements the long term goal to have the EOS server run optimization (or
energy management) on regular intervals automatically. Thus clients can request
the current energy management plan at any time and it is updated on regular
intervals without interaction by the client.

This fix started out to "only" make automatic optimization (or energy management)
runs working. It turned out there are several endpoints that in some way
update predictions or run the optimization. To lock against such concurrent attempts
the code had to be refactored to allow control of execution. During refactoring it
became clear that some classes and files are named without a proper reference
to their usage. Thus not only refactoring but also renaming became necessary.
The names are still not the best, but I hope they are more intuitive.

The fix includes several bug fixes that are not directly related to the automatic optimization
but are necessary to keep EOS running properly to do the automatic optimization and
to test and document the changes.

This is a breaking change as the configuration structure changed once again and
the server API was also enhanced and streamlined. The server API that is used by
Andreas and Jörg in their videos has not changed.

* fix: automatic optimization

  Allow optimization to automatically run on configured intervals gathering all
  optimization parameters from configuration and predictions. The automatic run
  can be configured to only run prediction updates skipping the optimization.
  Extend documentaion to also cover automatic optimization. Lock automatic runs
  against runs initiated by the /optimize or other endpoints. Provide new
  endpoints to retrieve the energy management plan and the genetic solution
  of the latest automatic optimization run. Offload energy management to thread
  pool executor to keep the app more responsive during the CPU heavy optimization
  run.

* fix: EOS servers recognize environment variables on startup

  Force initialisation of EOS configuration on server startup to assure
  all sources of EOS configuration are properly set up and read. Adapt
  server tests and configuration tests to also test for environment
  variable configuration.

* fix: Remove 0.0.0.0 to localhost translation under Windows

  EOS imposed a 0.0.0.0 to localhost translation under Windows for
  convenience. This caused some trouble in user configurations. Now, as the
  default IP address configuration is 127.0.0.1, the user is responsible
  for to set up the correct Windows compliant IP address.

* fix: allow names for hosts additional to IP addresses

* fix: access pydantic model fields by class

  Access by instance is deprecated.

* fix: down sampling key_to_array

* fix: make cache clear endpoint clear all cache files

  Make /v1/admin/cache/clear clear all cache files. Before it only cleared
  expired cache files by default. Add new endpoint /v1/admin/clear-expired
  to only clear expired cache files.

* fix: timezonefinder returns Europe/Paris instead of Europe/Berlin

  timezonefinder 8.10 got more inaccurate for timezones in europe as there is
  a common timezone. Use new package tzfpy instead which is still returning
  Europe/Berlin if you are in Germany. tzfpy also claims to be faster than
  timezonefinder.

* fix: provider settings configuration

  Provider configuration used to be a union holding the settings for several
  providers. Pydantic union handling does not always find the correct type
  for a provider setting. This led to exceptions in specific configurations.
  Now provider settings are explicit comfiguration items for each possible
  provider. This is a breaking change as the configuration structure was
  changed.

* fix: ClearOutside weather prediction irradiance calculation

  Pvlib needs a pandas time index. Convert time index.

* fix: test config file priority

  Do not use config_eos fixture as this fixture already creates a config file.

* fix: optimization sample request documentation

  Provide all data in documentation of optimization sample request.

* fix: gitlint blocking pip dependency resolution

  Replace gitlint by commitizen. Gitlint is not actively maintained anymore.
  Gitlint dependencies blocked pip from dependency resolution.

* fix: sync pre-commit config to actual dependency requirements

  .pre-commit-config.yaml was out of sync, also requirements-dev.txt.

* fix: missing babel in requirements.txt

  Add babel to requirements.txt

* feat: setup default device configuration for automatic optimization

  In case the parameters for automatic optimization are not fully defined a
  default configuration is setup to allow the automatic energy management
  run. The default configuration may help the user to correctly define
  the device configuration.

* feat: allow configuration of genetic algorithm parameters

  The genetic algorithm parameters for number of individuals, number of
  generations, the seed and penalty function parameters are now avaliable
  as configuration options.

* feat: allow configuration of home appliance time windows

  The time windows a home appliance is allowed to run are now configurable
  by the configuration (for /v1 API) and also by the home appliance parameters
  (for the classic /optimize API). If there is no such configuration the
  time window defaults to optimization hours, which was the standard before
  the change. Documentation on how to configure time windows is added.

* feat: standardize mesaurement keys for battery/ ev SoC measurements

  The standardized measurement keys to report battery SoC to the device
  simulations can now be retrieved from the device configuration as a
  read-only config option.

* feat: feed in tariff prediction

  Add feed in tarif predictions needed for automatic optimization. The feed in
  tariff can be retrieved as fixed feed in tarif or can be imported. Also add
  tests for the different feed in tariff providers. Extend documentation to
  cover the feed in tariff providers.

* feat: add energy management plan based on S2 standard instructions

  EOS can generate an energy management plan as a list of simple instructions.
  May be retrieved by the /v1/energy-management/plan endpoint. The instructions
  loosely follow the S2 energy management standard.

* feat: make measurement keys configurable by EOS configuration.

  The fixed measurement keys are replaced by configurable measurement keys.

* feat: make pendulum DateTime, Date, Duration types usable for pydantic models

  Use pydantic_extra_types.pendulum_dt to get pydantic pendulum types. Types are
  added to the datetimeutil utility. Remove custom made pendulum adaptations
  from EOS pydantic module. Make EOS modules use the pydantic pendulum types
  managed by the datetimeutil module instead of the core pendulum types.

* feat: Add Time, TimeWindow, TimeWindowSequence and to_time to datetimeutil.

  The time windows are are added to support home appliance time window
  configuration. All time classes are also pydantic models. Time is the base
  class for time definition derived from pendulum.Time.

* feat: Extend DataRecord by configurable field like data.

  Configurable field like data was added to support the configuration of
  measurement records.

* feat: Add additional information to health information

  Version information is added to the health endpoints of eos and eosDash.
  The start time of the last optimization and the latest run time of the energy
  management is added to the EOS health information.

* feat: add pydantic merge model tests

* feat: add plan tab to EOSdash

  The plan tab displays the current energy management instructions.

* feat: add predictions tab to EOSdash

  The predictions tab displays the current predictions.

* feat: add cache management to EOSdash admin tab

  The admin tab is extended by a section for cache management. It allows to
  clear the cache.

* feat: add about tab to EOSdash

  The about tab resembles the former hello tab and provides extra information.

* feat: Adapt changelog and prepare for release management

  Release management using commitizen is added. The changelog file is adapted and
  teh changelog and a description for release management is added in the
  documentation.

* feat(doc): Improve install and devlopment documentation

  Provide a more concise installation description in Readme.md and add extra
  installation page and development page to documentation.

* chore: Use memory cache for interpolation instead of dict in inverter

  Decorate calculate_self_consumption() with @cachemethod_until_update to cache
  results in memory during an energy management/ optimization run. Replacement
  of dict type caching in inverter is now possible because all optimization
  runs are properly locked and the memory cache CacheUntilUpdateStore is properly
  cleared at the start of any energy management/ optimization operation.

* chore: refactor genetic

  Refactor the genetic algorithm modules for enhanced module structure and better
  readability. Removed unnecessary and overcomplex devices singleton. Also
  split devices configuration from genetic algorithm parameters to allow further
  development independently from genetic algorithm parameter format. Move
  charge rates configuration for electric vehicles from optimization to devices
  configuration to allow to have different charge rates for different cars in
  the future.

* chore: Rename memory cache to CacheEnergyManagementStore

  The name better resembles the task of the cache to chache function and method
  results for an energy management run. Also the decorator functions are renamed
  accordingly: cachemethod_energy_management, cache_energy_management

* chore: use class properties for config/ems/prediction mixin classes

* chore: skip debug logs from mathplotlib

  Mathplotlib is very noisy in debug mode.

* chore: automatically sync bokeh js to bokeh python package

  bokeh was updated to 3.8.0, make JS CDN automatically follow the package version.

* chore: rename hello.py to about.py

  Make hello.py the adapted EOSdash about page.

* chore: remove demo page from EOSdash

  As no the plan and prediction pages are working without configuration, the demo
  page is no longer necessary

* chore: split test_server.py for system test

  Split test_server.py to create explicit test_system.py for system tests.

* chore: move doc utils to generate_config_md.py

  The doc utils are only used in scripts/generate_config_md.py. Move it there to
  attribute for strong cohesion.

* chore: improve pydantic merge model documentation

* chore: remove pendulum warning from readme

* chore: remove GitHub discussions from contributing documentation

  Github discussions is to be replaced by Akkudoktor.net.

* chore(release): bump version to 0.1.0+dev for development

* build(deps): bump fastapi[standard] from 0.115.14 to 0.117.1

  bump fastapi and make coverage version (for pytest-cov) explicit to avoid pip break.

* build(deps): bump uvicorn from 0.36.0 to 0.37.0

BREAKING CHANGE: EOS configuration changed. V1 API changed.

  - The available_charge_rates_percent configuration is removed from optimization.
    Use the new charge_rate configuration for the electric vehicle
  - Optimization configuration parameter hours renamed to horizon_hours
  - Device configuration now has to provide the number of devices and device
    properties per device.
  - Specific prediction provider configuration to be provided by explicit
    configuration item (no union for all providers).
  - Measurement keys to be provided as a list.
  - New feed in tariff providers have to be configured.
  - /v1/measurement/loadxxx endpoints are removed. Use generic mesaurement endpoints.
  - /v1/admin/cache/clear now clears all cache files. Use
    /v1/admin/cache/clear-expired to only clear all expired cache files.

Signed-off-by: Bobby Noelte <b0661n0e17e@gmail.com>
This commit is contained in:
Bobby Noelte
2025-10-28 02:50:31 +01:00
committed by GitHub
parent 20a9eb78d8
commit b397b5d43e
146 changed files with 22024 additions and 5339 deletions
Download Patch File Download Diff File Expand all files Collapse all files

150
src/akkudoktoreos/config/config.py
View File

@@ -14,28 +14,28 @@ import shutil
from pathlib import Path
from typing import Any, ClassVar, Optional, Type
import pydantic_settings
from loguru import logger
from platformdirs import user_config_dir, user_data_dir
from pydantic import Field, computed_field
from pydantic_settings import (
BaseSettings,
JsonConfigSettingsSource,
PydanticBaseSettingsSource,
SettingsConfigDict,
)
from pydantic import Field, computed_field, field_validator
# settings
from akkudoktoreos.config.configabc import SettingsBaseModel
from akkudoktoreos.config.configmigrate import migrate_config_file
from akkudoktoreos.core.cachesettings import CacheCommonSettings
from akkudoktoreos.core.coreabc import SingletonMixin
from akkudoktoreos.core.decorators import classproperty
from akkudoktoreos.core.emsettings import EnergyManagementCommonSettings
from akkudoktoreos.core.emsettings import (
EnergyManagementCommonSettings,
)
from akkudoktoreos.core.logsettings import LoggingCommonSettings
from akkudoktoreos.core.pydantic import PydanticModelNestedValueMixin, merge_models
from akkudoktoreos.devices.settings import DevicesCommonSettings
from akkudoktoreos.core.version import __version__
from akkudoktoreos.devices.devices import DevicesCommonSettings
from akkudoktoreos.measurement.measurement import MeasurementCommonSettings
from akkudoktoreos.optimization.optimization import OptimizationCommonSettings
from akkudoktoreos.prediction.elecprice import ElecPriceCommonSettings
from akkudoktoreos.prediction.feedintariff import FeedInTariffCommonSettings
from akkudoktoreos.prediction.load import LoadCommonSettings
from akkudoktoreos.prediction.prediction import PredictionCommonSettings
from akkudoktoreos.prediction.pvforecast import PVForecastCommonSettings
@@ -83,6 +83,10 @@ class GeneralSettings(SettingsBaseModel):
_config_folder_path: ClassVar[Optional[Path]] = None
_config_file_path: ClassVar[Optional[Path]] = None
version: str = Field(
default=__version__, description="Configuration file version. Used to check compatibility."
)
data_folder_path: Optional[Path] = Field(
default=None, description="Path to EOS data directory.", examples=[None, "/home/eos/data"]
)
@@ -131,11 +135,25 @@ class GeneralSettings(SettingsBaseModel):
"""Path to EOS configuration file."""
return self._config_file_path
compatible_versions: ClassVar[list[str]] = [__version__]
class SettingsEOS(BaseSettings, PydanticModelNestedValueMixin):
@field_validator("version")
@classmethod
def check_version(cls, v: str) -> str:
if v not in cls.compatible_versions:
error = (
f"Incompatible configuration version '{v}'. "
f"Expected one of: {', '.join(cls.compatible_versions)}."
)
logger.error(error)
raise ValueError(error)
return v
class SettingsEOS(pydantic_settings.BaseSettings, PydanticModelNestedValueMixin):
"""Settings for all EOS.
Used by updating the configuration with specific settings only.
Only used to update the configuration with specific settings.
"""
general: Optional[GeneralSettings] = Field(
@@ -174,6 +192,10 @@ class SettingsEOS(BaseSettings, PydanticModelNestedValueMixin):
default=None,
description="Electricity Price Settings",
)
feedintariff: Optional[FeedInTariffCommonSettings] = Field(
default=None,
description="Feed In Tariff Settings",
)
load: Optional[LoadCommonSettings] = Field(
default=None,
description="Load Settings",
@@ -195,7 +217,7 @@ class SettingsEOS(BaseSettings, PydanticModelNestedValueMixin):
description="Utilities Settings",
)
model_config = SettingsConfigDict(
model_config = pydantic_settings.SettingsConfigDict(
env_nested_delimiter="__",
nested_model_default_partial_update=True,
env_prefix="EOS_",
@@ -218,12 +240,18 @@ class SettingsEOSDefaults(SettingsEOS):
optimization: OptimizationCommonSettings = OptimizationCommonSettings()
prediction: PredictionCommonSettings = PredictionCommonSettings()
elecprice: ElecPriceCommonSettings = ElecPriceCommonSettings()
feedintariff: FeedInTariffCommonSettings = FeedInTariffCommonSettings()
load: LoadCommonSettings = LoadCommonSettings()
pvforecast: PVForecastCommonSettings = PVForecastCommonSettings()
weather: WeatherCommonSettings = WeatherCommonSettings()
server: ServerCommonSettings = ServerCommonSettings()
utils: UtilsCommonSettings = UtilsCommonSettings()
def __hash__(self) -> int:
# Just for usage in configmigrate, finally overwritten when used by ConfigEOS.
# This is mutable, so pydantic does not set a hash.
return id(self)
class ConfigEOS(SingletonMixin, SettingsEOSDefaults):
"""Singleton configuration handler for the EOS application.
@@ -290,33 +318,34 @@ class ConfigEOS(SingletonMixin, SettingsEOSDefaults):
@classmethod
def settings_customise_sources(
cls,
settings_cls: Type[BaseSettings],
init_settings: PydanticBaseSettingsSource,
env_settings: PydanticBaseSettingsSource,
dotenv_settings: PydanticBaseSettingsSource,
file_secret_settings: PydanticBaseSettingsSource,
) -> tuple[PydanticBaseSettingsSource, ...]:
"""Customizes the order and handling of settings sources for a Pydantic BaseSettings subclass.
settings_cls: Type[pydantic_settings.BaseSettings],
init_settings: pydantic_settings.PydanticBaseSettingsSource,
env_settings: pydantic_settings.PydanticBaseSettingsSource,
dotenv_settings: pydantic_settings.PydanticBaseSettingsSource,
file_secret_settings: pydantic_settings.PydanticBaseSettingsSource,
) -> tuple[pydantic_settings.PydanticBaseSettingsSource, ...]:
"""Customizes the order and handling of settings sources for a pydantic_settings.BaseSettings subclass.
This method determines the sources for application configuration settings, including
environment variables, dotenv files and JSON configuration files.
It ensures that a default configuration file exists and creates one if necessary.
Args:
settings_cls (Type[BaseSettings]): The Pydantic BaseSettings class for which sources are customized.
init_settings (PydanticBaseSettingsSource): The initial settings source, typically passed at runtime.
env_settings (PydanticBaseSettingsSource): Settings sourced from environment variables.
dotenv_settings (PydanticBaseSettingsSource): Settings sourced from a dotenv file.
file_secret_settings (PydanticBaseSettingsSource): Unused (needed for parent class interface).
settings_cls (Type[pydantic_settings.BaseSettings]): The Pydantic BaseSettings class for
which sources are customized.
init_settings (pydantic_settings.PydanticBaseSettingsSource): The initial settings source, typically passed at runtime.
env_settings (pydantic_settings.PydanticBaseSettingsSource): Settings sourced from environment variables.
dotenv_settings (pydantic_settings.PydanticBaseSettingsSource): Settings sourced from a dotenv file.
file_secret_settings (pydantic_settings.PydanticBaseSettingsSource): Unused (needed for parent class interface).
Returns:
tuple[PydanticBaseSettingsSource, ...]: A tuple of settings sources in the order they should be applied.
tuple[pydantic_settings.PydanticBaseSettingsSource, ...]: A tuple of settings sources in the order they should be applied.
Behavior:
1. Checks for the existence of a JSON configuration file in the expected location.
2. If the configuration file does not exist, creates the directory (if needed) and attempts to copy a
default configuration file to the location. If the copy fails, uses the default configuration file directly.
3. Creates a `JsonConfigSettingsSource` for both the configuration file and the default configuration file.
3. Creates a `pydantic_settings.JsonConfigSettingsSource` for both the configuration file and the default configuration file.
4. Updates class attributes `GeneralSettings._config_folder_path` and
`GeneralSettings._config_file_path` to reflect the determined paths.
5. Returns a tuple containing all provided and newly created settings sources in the desired order.
@@ -325,13 +354,7 @@ class ConfigEOS(SingletonMixin, SettingsEOSDefaults):
- This method logs a warning if the default configuration file cannot be copied.
- It ensures that a fallback to the default configuration file is always possible.
"""
setting_sources = [
init_settings,
env_settings,
dotenv_settings,
]
file_settings: Optional[JsonConfigSettingsSource] = None
# Ensure we know and have the config folder path and the config file
config_file, exists = cls._get_config_file_path()
config_dir = config_file.parent
if not exists:
@@ -342,20 +365,38 @@ class ConfigEOS(SingletonMixin, SettingsEOSDefaults):
logger.warning(f"Could not copy default config: {exc}. Using default config...")
config_file = cls.config_default_file_path
config_dir = config_file.parent
try:
file_settings = JsonConfigSettingsSource(settings_cls, json_file=config_file)
setting_sources.append(file_settings)
except Exception as e:
logger.error(
f"Error reading config file '{config_file}' (falling back to default config): {e}"
)
default_settings = JsonConfigSettingsSource(
settings_cls, json_file=cls.config_default_file_path
)
# Remember config_dir and config file
GeneralSettings._config_folder_path = config_dir
GeneralSettings._config_file_path = config_file
# All the settings sources in priority sequence
setting_sources = [
init_settings,
env_settings,
dotenv_settings,
]
# Apend file settings to sources
file_settings: Optional[pydantic_settings.JsonConfigSettingsSource] = None
try:
backup_file = config_file.with_suffix(".bak")
if migrate_config_file(config_file, backup_file):
# If correct version add it as settings source
file_settings = pydantic_settings.JsonConfigSettingsSource(
settings_cls, json_file=config_file
)
setting_sources.append(file_settings)
except Exception as ex:
logger.error(
f"Error reading config file '{config_file}' (falling back to default config): {ex}"
)
# Append default settings to sources
default_settings = pydantic_settings.JsonConfigSettingsSource(
settings_cls, json_file=cls.config_default_file_path
)
setting_sources.append(default_settings)
return tuple(setting_sources)
@classproperty
@@ -374,28 +415,24 @@ class ConfigEOS(SingletonMixin, SettingsEOSDefaults):
Configuration data is loaded from a configuration file or a default one is created if none
exists.
"""
logger.debug("Config init with parameters {} {}", args, kwargs)
# Check for singleton guard
if hasattr(self, "_initialized"):
return
self._setup(self, *args, **kwargs)
def _setup(self, *args: Any, **kwargs: Any) -> None:
"""Re-initialize global settings."""
# Check for config file content/ version type
config_file, exists = self._get_config_file_path()
if exists:
with config_file.open("r", encoding="utf-8", newline=None) as f_config:
config_txt = f_config.read()
if '"directories": {' in config_txt or '"server_eos_host": ' in config_txt:
error_msg = f"Configuration file '{config_file}' is outdated. Please remove or update manually."
logger.error(error_msg)
raise ValueError(error_msg)
# Assure settings base knows EOS configuration
logger.debug("Config setup with parameters {} {}", args, kwargs)
# Assure settings base knows the singleton EOS configuration
SettingsBaseModel.config = self
# (Re-)load settings
# (Re-)load settings - call base class init
SettingsEOSDefaults.__init__(self, *args, **kwargs)
# Init config file and data folder pathes
self._create_initial_config_file()
self._update_data_folder_path()
self._initialized = True
logger.debug("Config setup:\n{}", self)
def merge_settings(self, settings: SettingsEOS) -> None:
"""Merges the provided settings into the global settings for EOS, with optional overwrite.
@@ -488,6 +525,11 @@ class ConfigEOS(SingletonMixin, SettingsEOSDefaults):
def _get_config_file_path(cls) -> tuple[Path, bool]:
"""Find a valid configuration file or return the desired path for a new config file.
Searches:
1. environment variable directory
2. user configuration directory
3. current working directory
Returns:
tuple[Path, bool]: The path to the configuration file and if there is already a config file there
"""

225
src/akkudoktoreos/config/configmigrate.py Normal file
View File

@@ -0,0 +1,225 @@
"""Migrate config file to actual version."""
import json
import shutil
from pathlib import Path
from typing import Any, Callable, Dict, List, Set, Tuple, Union
from loguru import logger
from akkudoktoreos.core.version import __version__
# -----------------------------
# Global migration map constant
# -----------------------------
# key: old JSON path, value: either
# - str (new model path)
# - tuple[str, Callable[[Any], Any]] (new path + transform)
# - None (drop)
MIGRATION_MAP: Dict[str, Union[str, Tuple[str, Callable[[Any], Any]], None]] = {
# 0.1.0 -> now
"devices/batteries/0/initial_soc_percentage": None,
"devices/electric_vehicles/0/initial_soc_percentage": None,
"elecprice/provider_settings/import_file_path": "elecprice/provider_settings/ElecPriceImport/import_file_path",
"elecprice/provider_settings/import_json": "elecprice/provider_settings/ElecPriceImport/import_json",
"load/provider_settings/import_file_path": "load/provider_settings/LoadImport/import_file_path",
"load/provider_settings/import_json": "load/provider_settings/LoadImport/import_json",
"load/provider_settings/loadakkudoktor_year_energy": "load/provider_settings/LoadAkkudoktor/loadakkudoktor_year_energy",
"load/provider_settings/load_vrm_idsite": "load/provider_settings/LoadVrm/load_vrm_idsite",
"load/provider_settings/load_vrm_token": "load/provider_settings/LoadVrm/load_vrm_token",
"logging/level": "logging/console_level",
"logging/root_level": None,
"measurement/load0_name": "measurement/load_emr_keys/0",
"measurement/load1_name": "measurement/load_emr_keys/1",
"measurement/load2_name": "measurement/load_emr_keys/2",
"measurement/load3_name": "measurement/load_emr_keys/3",
"measurement/load4_name": "measurement/load_emr_keys/4",
"optimization/ev_available_charge_rates_percent": (
"devices/electric_vehicles/0/charge_rates",
lambda v: [x / 100 for x in v],
),
"optimization/hours": "optimization/horizon_hours",
"optimization/penalty": ("optimization/genetic/penalties/ev_soc_miss", lambda v: float(v)),
"pvforecast/provider_settings/import_file_path": "pvforecast/provider_settings/PVForecastImport/import_file_path",
"pvforecast/provider_settings/import_json": "pvforecast/provider_settings/PVForecastImport/import_json",
"pvforecast/provider_settings/load_vrm_idsite": "pvforecast/provider_settings/PVForecastVrm/load_vrm_idsite",
"pvforecast/provider_settings/load_vrm_token": "pvforecast/provider_settings/PVForecastVrm/load_vrm_token",
"weather/provider_settings/import_file_path": "weather/provider_settings/WeatherImport/import_file_path",
"weather/provider_settings/import_json": "weather/provider_settings/WeatherImport/import_json",
}
# -----------------------------
# Global migration stats
# -----------------------------
migrated_source_paths: Set[str] = set()
mapped_count: int = 0
auto_count: int = 0
skipped_paths: List[str] = []
def migrate_config_file(config_file: Path, backup_file: Path) -> bool:
"""Migrate configuration file to the current version.
Returns:
bool: True if up-to-date or successfully migrated, False on failure.
"""
global migrated_source_paths, mapped_count, auto_count, skipped_paths
# Reset globals at the start of each migration
migrated_source_paths = set()
mapped_count = 0
auto_count = 0
skipped_paths = []
try:
with config_file.open("r", encoding="utf-8") as f:
config_data: Dict[str, Any] = json.load(f)
except (FileNotFoundError, json.JSONDecodeError) as e:
logger.error(f"Failed to read configuration file '{config_file}': {e}")
return False
match config_data:
case {"general": {"version": v}} if v == __version__:
logger.debug(f"Configuration file '{config_file}' is up to date (v{v}).")
return True
case _:
logger.info(
f"Configuration file '{config_file}' is missing current version info. "
f"Starting migration to v{__version__}..."
)
try:
# Backup existing file - we already know it is existing
try:
config_file.replace(backup_file)
logger.info(f"Backed up old configuration to '{backup_file}'.")
except Exception as e_replace:
try:
shutil.copy(config_file, backup_file)
logger.info(
f"Could not replace; copied old configuration to '{backup_file}' instead."
)
except Exception as e_copy:
logger.warning(
f"Failed to backup existing config (replace: {e_replace}; copy: {e_copy}). Continuing without backup."
)
from akkudoktoreos.config.config import SettingsEOSDefaults
new_config = SettingsEOSDefaults()
# 1) Apply explicit migration map
for old_path, mapping in MIGRATION_MAP.items():
new_path = None
transform = None
if mapping is None:
migrated_source_paths.add(old_path.strip("/"))
logger.debug(f"🗑️ Migration map: dropping '{old_path}'")
continue
if isinstance(mapping, tuple):
new_path, transform = mapping
else:
new_path = mapping
old_value = _get_json_nested_value(config_data, old_path)
if old_value is None:
continue
try:
if transform:
old_value = transform(old_value)
new_config.set_nested_value(new_path, old_value)
migrated_source_paths.add(old_path.strip("/"))
mapped_count += 1
logger.debug(f"✅ Migrated mapped '{old_path}' → '{new_path}' = {old_value!r}")
except Exception as e:
logger.opt(exception=True).warning(
f"Failed mapped migration '{old_path}' -> '{new_path}': {e}", exc_info=True
)
# 2) Automatic migration for remaining fields
auto_count += _migrate_matching_fields(
config_data, new_config, migrated_source_paths, skipped_paths
)
# 3) Ensure version
try:
new_config.set_nested_value("general/version", __version__)
except Exception as e:
logger.warning(f"Could not set version on new configuration model: {e}")
# 4) Write migrated configuration
try:
with config_file.open("w", encoding="utf-8", newline=None) as f_out:
json_str = new_config.model_dump_json(indent=4)
f_out.write(json_str)
except Exception as e_write:
logger.error(f"Failed to write migrated configuration to '{config_file}': {e_write}")
return False
# 5) Log final migration summary
logger.info(
f"Migration summary for '{config_file}': "
f"mapped fields: {mapped_count}, automatically migrated: {auto_count}, skipped: {len(skipped_paths)}"
)
if skipped_paths:
logger.debug(f"Skipped paths: {', '.join(skipped_paths)}")
logger.success(f"Configuration successfully migrated to version {__version__}.")
return True
except Exception as e:
logger.exception(f"Unexpected error during migration: {e}")
return False
def _get_json_nested_value(data: dict, path: str) -> Any:
"""Retrieve a nested value from a JSON-like dict using '/'-separated path."""
current: Any = data
for part in path.strip("/").split("/"):
if isinstance(current, list):
try:
part_idx = int(part)
current = current[part_idx]
except (ValueError, IndexError):
return None
elif isinstance(current, dict):
if part not in current:
return None
current = current[part]
else:
return None
return current
def _migrate_matching_fields(
source: Dict[str, Any],
target_model: Any,
migrated_source_paths: Set[str],
skipped_paths: List[str],
prefix: str = "",
) -> int:
"""Recursively copy matching keys from source dict into target_model using set_nested_value.
Returns:
int: number of fields successfully auto-migrated
"""
count: int = 0
for key, value in source.items():
full_path = f"{prefix}/{key}".strip("/")
if full_path in migrated_source_paths:
continue
if isinstance(value, dict):
count += _migrate_matching_fields(
value, target_model, migrated_source_paths, skipped_paths, full_path
)
else:
try:
target_model.set_nested_value(full_path, value)
count += 1
except Exception:
skipped_paths.append(full_path)
continue
return count

76
src/akkudoktoreos/core/cache.py
View File

@@ -28,12 +28,17 @@ from typing import (
import cachebox
from loguru import logger
from pendulum import DateTime, Duration
from pydantic import Field
from akkudoktoreos.core.coreabc import ConfigMixin, SingletonMixin
from akkudoktoreos.core.pydantic import PydanticBaseModel
from akkudoktoreos.utils.datetimeutil import compare_datetimes, to_datetime, to_duration
from akkudoktoreos.utils.datetimeutil import (
DateTime,
Duration,
compare_datetimes,
to_datetime,
to_duration,
)
# ---------------------------------
# In-Memory Caching Functionality
@@ -43,25 +48,28 @@ from akkudoktoreos.utils.datetimeutil import compare_datetimes, to_datetime, to_
TCallable = TypeVar("TCallable", bound=Callable[..., Any])
def cache_until_update_store_callback(event: int, key: Any, value: Any) -> None:
"""Calback function for CacheUntilUpdateStore."""
CacheUntilUpdateStore.last_event = event
CacheUntilUpdateStore.last_key = key
CacheUntilUpdateStore.last_value = value
def cache_energy_management_store_callback(event: int, key: Any, value: Any) -> None:
"""Calback function for CacheEnergyManagementStore."""
CacheEnergyManagementStore.last_event = event
CacheEnergyManagementStore.last_key = key
CacheEnergyManagementStore.last_value = value
if event == cachebox.EVENT_MISS:
CacheUntilUpdateStore.miss_count += 1
CacheEnergyManagementStore.miss_count += 1
elif event == cachebox.EVENT_HIT:
CacheUntilUpdateStore.hit_count += 1
CacheEnergyManagementStore.hit_count += 1
else:
# unreachable code
raise NotImplementedError
class CacheUntilUpdateStore(SingletonMixin):
class CacheEnergyManagementStore(SingletonMixin):
"""Singleton-based in-memory LRU (Least Recently Used) cache.
This cache is shared across the application to store results of decorated
methods or functions until the next EMS (Energy Management System) update.
methods or functions during energy management runs.
Energy management tasks shall clear the cache at the start of the energy management
task.
The cache uses an LRU eviction strategy, storing up to 100 items, with the oldest
items being evicted once the cache reaches its capacity.
@@ -75,14 +83,14 @@ class CacheUntilUpdateStore(SingletonMixin):
miss_count: ClassVar[int] = 0
def __init__(self, *args: Any, **kwargs: Any) -> None:
"""Initializes the `CacheUntilUpdateStore` instance with default parameters.
"""Initializes the `CacheEnergyManagementStore` instance with default parameters.
The cache uses an LRU eviction strategy with a maximum size of 100 items.
This cache is a singleton, meaning only one instance will exist throughout
the application lifecycle.
Example:
>>> cache = CacheUntilUpdateStore()
>>> cache = CacheEnergyManagementStore()
"""
if hasattr(self, "_initialized"):
return
@@ -128,7 +136,7 @@ class CacheUntilUpdateStore(SingletonMixin):
Example:
>>> value = cache["user_data"]
"""
return CacheUntilUpdateStore.cache[key]
return CacheEnergyManagementStore.cache[key]
def __setitem__(self, key: Any, value: Any) -> None:
"""Stores an item in the cache.
@@ -140,15 +148,15 @@ class CacheUntilUpdateStore(SingletonMixin):
Example:
>>> cache["user_data"] = {"name": "Alice", "age": 30}
"""
CacheUntilUpdateStore.cache[key] = value
CacheEnergyManagementStore.cache[key] = value
def __len__(self) -> int:
"""Returns the number of items in the cache."""
return len(CacheUntilUpdateStore.cache)
return len(CacheEnergyManagementStore.cache)
def __repr__(self) -> str:
"""Provides a string representation of the CacheUntilUpdateStore object."""
return repr(CacheUntilUpdateStore.cache)
"""Provides a string representation of the CacheEnergyManagementStore object."""
return repr(CacheEnergyManagementStore.cache)
def clear(self) -> None:
"""Clears the cache, removing all stored items.
@@ -161,22 +169,22 @@ class CacheUntilUpdateStore(SingletonMixin):
>>> cache.clear()
"""
if hasattr(self.cache, "clear") and callable(getattr(self.cache, "clear")):
CacheUntilUpdateStore.cache.clear()
CacheUntilUpdateStore.last_event = None
CacheUntilUpdateStore.last_key = None
CacheUntilUpdateStore.last_value = None
CacheUntilUpdateStore.miss_count = 0
CacheUntilUpdateStore.hit_count = 0
CacheEnergyManagementStore.cache.clear()
CacheEnergyManagementStore.last_event = None
CacheEnergyManagementStore.last_key = None
CacheEnergyManagementStore.last_value = None
CacheEnergyManagementStore.miss_count = 0
CacheEnergyManagementStore.hit_count = 0
else:
raise AttributeError(f"'{self.cache.__class__.__name__}' object has no method 'clear'")
def cachemethod_until_update(method: TCallable) -> TCallable:
def cachemethod_energy_management(method: TCallable) -> TCallable:
"""Decorator for in memory caching the result of an instance method.
This decorator caches the method's result in `CacheUntilUpdateStore`, ensuring
This decorator caches the method's result in `CacheEnergyManagementStore`, ensuring
that subsequent calls with the same arguments return the cached result until the
next EMS update cycle.
next energy management start.
Args:
method (Callable): The instance method to be decorated.
@@ -186,14 +194,14 @@ def cachemethod_until_update(method: TCallable) -> TCallable:
Example:
>>> class MyClass:
>>> @cachemethod_until_update
>>> @cachemethod_energy_management
>>> def expensive_method(self, param: str) -> str:
>>> # Perform expensive computation
>>> return f"Computed {param}"
"""
@cachebox.cachedmethod(
cache=CacheUntilUpdateStore().cache, callback=cache_until_update_store_callback
cache=CacheEnergyManagementStore().cache, callback=cache_energy_management_store_callback
)
@functools.wraps(method)
def wrapper(self: Any, *args: Any, **kwargs: Any) -> Any:
@@ -203,12 +211,12 @@ def cachemethod_until_update(method: TCallable) -> TCallable:
return wrapper
def cache_until_update(func: TCallable) -> TCallable:
def cache_energy_management(func: TCallable) -> TCallable:
"""Decorator for in memory caching the result of a standalone function.
This decorator caches the function's result in `CacheUntilUpdateStore`, ensuring
This decorator caches the function's result in `CacheEnergyManagementStore`, ensuring
that subsequent calls with the same arguments return the cached result until the
next EMS update cycle.
next energy management start.
Args:
func (Callable): The function to be decorated.
@@ -224,7 +232,7 @@ def cache_until_update(func: TCallable) -> TCallable:
"""
@cachebox.cached(
cache=CacheUntilUpdateStore().cache, callback=cache_until_update_store_callback
cache=CacheEnergyManagementStore().cache, callback=cache_energy_management_store_callback
)
@functools.wraps(func)
def wrapper(*args: Any, **kwargs: Any) -> Any:
@@ -435,7 +443,7 @@ class CacheFileStore(ConfigMixin, SingletonMixin):
)
logger.debug(
f"Search: ttl:{ttl_duration}, until:{until_datetime}, at:{at_datetime}, before:{before_datetime} -> hit: {generated_key == cache_file_key}, item: {cache_item.cache_file.seek(0), cache_item.cache_file.read()}"
f"Search: ttl:{ttl_duration}, until:{until_datetime}, at:{at_datetime}, before:{before_datetime} -> hit: {generated_key == cache_file_key}, item: {cache_item.cache_file.seek(0), cache_item.cache_file.read()[:10]}..."
)
if generated_key == cache_file_key:

87
src/akkudoktoreos/core/coreabc.py
View File

@@ -14,13 +14,13 @@ import threading
from typing import Any, ClassVar, Dict, Optional, Type
from loguru import logger
from pendulum import DateTime
from pydantic import computed_field
from akkudoktoreos.core.decorators import classproperty
from akkudoktoreos.utils.datetimeutil import DateTime
config_eos: Any = None
measurement_eos: Any = None
prediction_eos: Any = None
devices_eos: Any = None
ems_eos: Any = None
@@ -46,9 +46,9 @@ class ConfigMixin:
```
"""
@property
def config(self) -> Any:
"""Convenience method/ attribute to retrieve the EOS configuration data.
@classproperty
def config(cls) -> Any:
"""Convenience class method/ attribute to retrieve the EOS configuration data.
Returns:
ConfigEOS: The configuration.
@@ -86,9 +86,9 @@ class MeasurementMixin:
```
"""
@property
def measurement(self) -> Any:
"""Convenience method/ attribute to retrieve the EOS measurement data.
@classproperty
def measurement(cls) -> Any:
"""Convenience class method/ attribute to retrieve the EOS measurement data.
Returns:
Measurement: The measurement.
@@ -126,9 +126,9 @@ class PredictionMixin:
```
"""
@property
def prediction(self) -> Any:
"""Convenience method/ attribute to retrieve the EOS prediction data.
@classproperty
def prediction(cls) -> Any:
"""Convenience class method/ attribute to retrieve the EOS prediction data.
Returns:
Prediction: The prediction.
@@ -143,46 +143,6 @@ class PredictionMixin:
return prediction_eos
class DevicesMixin:
"""Mixin class for managing EOS devices simulation data.
This class serves as a foundational component for EOS-related classes requiring access
to global devices simulation data. It provides a `devices` property that dynamically retrieves
the devices instance, ensuring up-to-date access to devices simulation results.
Usage:
Subclass this base class to gain access to the `devices` attribute, which retrieves the
global devices instance lazily to avoid import-time circular dependencies.
Attributes:
devices (Devices): Property to access the global EOS devices simulation data.
Example:
```python
class MyOptimizationClass(DevicesMixin):
def analyze_mydevicesimulation(self):
device_simulation_data = self.devices.mydevicesresult
# Perform analysis
```
"""
@property
def devices(self) -> Any:
"""Convenience method/ attribute to retrieve the EOS devices simulation data.
Returns:
Devices: The devices simulation.
"""
# avoid circular dependency at import time
global devices_eos
if devices_eos is None:
from akkudoktoreos.devices.devices import get_devices
devices_eos = get_devices()
return devices_eos
class EnergyManagementSystemMixin:
"""Mixin class for managing EOS energy management system.
@@ -207,9 +167,9 @@ class EnergyManagementSystemMixin:
```
"""
@property
def ems(self) -> Any:
"""Convenience method/ attribute to retrieve the EOS energy management system.
@classproperty
def ems(cls) -> Any:
"""Convenience class method/ attribute to retrieve the EOS energy management system.
Returns:
EnergyManagementSystem: The energy management system.
@@ -231,16 +191,21 @@ class StartMixin(EnergyManagementSystemMixin):
- `start_datetime`: The starting datetime of the current or latest energy management.
"""
# Computed field for start_datetime
@computed_field # type: ignore[prop-decorator]
@property
def start_datetime(self) -> Optional[DateTime]:
"""Returns the start datetime of the current or latest energy management.
@classproperty
def ems_start_datetime(cls) -> Optional[DateTime]:
"""Convenience class method/ attribute to retrieve the start datetime of the current or latest energy management.
Returns:
DateTime: The starting datetime of the current or latest energy management, or None.
"""
return self.ems.start_datetime
# avoid circular dependency at import time
global ems_eos
if ems_eos is None:
from akkudoktoreos.core.ems import get_ems
ems_eos = get_ems()
return ems_eos.start_datetime
class SingletonMixin:

275
src/akkudoktoreos/core/dataabc.py
View File

@@ -14,14 +14,23 @@ from abc import abstractmethod
from collections.abc import MutableMapping, MutableSequence
from itertools import chain
from pathlib import Path
from typing import Any, Dict, Iterator, List, Optional, Tuple, Type, Union, overload
from typing import (
Any,
Dict,
Iterator,
List,
Optional,
Tuple,
Type,
Union,
overload,
)
import numpy as np
import pandas as pd
import pendulum
from loguru import logger
from numpydantic import NDArray, Shape
from pendulum import DateTime, Duration
from pydantic import (
AwareDatetime,
ConfigDict,
@@ -29,6 +38,7 @@ from pydantic import (
ValidationError,
computed_field,
field_validator,
model_validator,
)
from akkudoktoreos.core.coreabc import ConfigMixin, SingletonMixin, StartMixin
@@ -37,7 +47,13 @@ from akkudoktoreos.core.pydantic import (
PydanticDateTimeData,
PydanticDateTimeDataFrame,
)
from akkudoktoreos.utils.datetimeutil import compare_datetimes, to_datetime, to_duration
from akkudoktoreos.utils.datetimeutil import (
DateTime,
Duration,
compare_datetimes,
to_datetime,
to_duration,
)
class DataBase(ConfigMixin, StartMixin, PydanticBaseModel):
@@ -55,6 +71,11 @@ class DataRecord(DataBase, MutableMapping):
Fields can be accessed and mutated both using dictionary-style access (`record['field_name']`)
and attribute-style access (`record.field_name`).
The data record also provides configured field like data. Configuration has to be done by the
derived class. Configuration is a list of key strings, which is usually taken from the EOS
configuration. The internal field for these data `configured_data` is mostly hidden from
dictionary-style and attribute-style access.
Attributes:
date_time (Optional[DateTime]): Aware datetime indicating when the data record applies.
@@ -65,9 +86,42 @@ class DataRecord(DataBase, MutableMapping):
date_time: Optional[DateTime] = Field(default=None, description="DateTime")
configured_data: dict[str, Any] = Field(
default_factory=dict,
description="Configured field like data",
examples=[{"load0_mr": 40421}],
)
# Pydantic v2 model configuration
model_config = ConfigDict(arbitrary_types_allowed=True, populate_by_name=True)
@model_validator(mode="before")
@classmethod
def init_configured_field_like_data(cls, data: Any) -> Any:
"""Extracts configured data keys from the input and assigns them to `configured_data`.
This validator is called before the model is initialized. It filters out any keys from the input
dictionary that are listed in the configured data keys, and moves them into
the `configured_data` field of the model. This enables flexible, key-driven population of
dynamic data while keeping the model schema clean.
Args:
data (Any): The raw input data used to initialize the model.
Returns:
Any: The modified input data dictionary, with configured keys moved to `configured_data`.
"""
if not isinstance(data, dict):
return data
configured_keys: Union[list[str], set] = cls.configured_data_keys() or set()
extracted = {k: data.pop(k) for k in list(data.keys()) if k in configured_keys}
if extracted:
data.setdefault("configured_data", {}).update(extracted)
return data
@field_validator("date_time", mode="before")
@classmethod
def transform_to_datetime(cls, value: Any) -> Optional[DateTime]:
@@ -77,18 +131,39 @@ class DataRecord(DataBase, MutableMapping):
return None
return to_datetime(value)
@classmethod
def configured_data_keys(cls) -> Optional[list[str]]:
"""Return the keys for the configured field like data.
Can be overwritten by derived classes to define specific field like data. Usually provided
by configuration data.
"""
return None
@classmethod
def record_keys(cls) -> List[str]:
"""Returns the keys of all fields in the data record."""
key_list = []
key_list.extend(list(cls.model_fields.keys()))
key_list.extend(list(cls.__pydantic_decorators__.computed_fields.keys()))
# Add also keys that may be added by configuration
key_list.remove("configured_data")
configured_keys = cls.configured_data_keys()
if configured_keys is not None:
key_list.extend(configured_keys)
return key_list
@classmethod
def record_keys_writable(cls) -> List[str]:
"""Returns the keys of all fields in the data record that are writable."""
return list(cls.model_fields.keys())
keys_writable = []
keys_writable.extend(list(cls.model_fields.keys()))
# Add also keys that may be added by configuration
keys_writable.remove("configured_data")
configured_keys = cls.configured_data_keys()
if configured_keys is not None:
keys_writable.extend(configured_keys)
return keys_writable
def _validate_key_writable(self, key: str) -> None:
"""Verify that a specified key exists and is writable in the current record keys.
@@ -104,6 +179,40 @@ class DataRecord(DataBase, MutableMapping):
f"Key '{key}' is not in writable record keys: {self.record_keys_writable()}"
)
def __dir__(self) -> list[str]:
"""Extend the default `dir()` output to include configured field like data keys.
This enables editor auto-completion and interactive introspection, while hiding the internal
`configured_data` dictionary.
This ensures the configured field like data values appear like native fields,
in line with the base model's attribute behavior.
"""
base = super().__dir__()
keys = set(base)
# Expose configured data keys as attributes
configured_keys = self.configured_data_keys()
if configured_keys is not None:
keys.update(configured_keys)
# Explicitly hide the 'configured_data' internal dict
keys.discard("configured_data")
return sorted(keys)
def __eq__(self, other: Any) -> bool:
"""Ensure equality comparison includes the contents of the `configured_data` dict.
Contents of the `configured_data` dict are in addition to the base model fields.
"""
if not isinstance(other, self.__class__):
return NotImplemented
# Compare all fields except `configured_data`
if self.model_dump(exclude={"configured_data"}) != other.model_dump(
exclude={"configured_data"}
):
return False
# Compare `configured_data` explicitly
return self.configured_data == other.configured_data
def __getitem__(self, key: str) -> Any:
"""Retrieve the value of a field by key name.
@@ -116,9 +225,11 @@ class DataRecord(DataBase, MutableMapping):
Raises:
KeyError: If the specified key does not exist.
"""
if key in self.model_fields:
return getattr(self, key)
raise KeyError(f"'{key}' not found in the record fields.")
try:
# Let getattr do the work
return self.__getattr__(key)
except:
raise KeyError(f"'{key}' not found in the record fields.")
def __setitem__(self, key: str, value: Any) -> None:
"""Set the value of a field by key name.
@@ -130,9 +241,10 @@ class DataRecord(DataBase, MutableMapping):
Raises:
KeyError: If the specified key does not exist in the fields.
"""
if key in self.model_fields:
setattr(self, key, value)
else:
try:
# Let setattr do the work
self.__setattr__(key, value)
except:
raise KeyError(f"'{key}' is not a recognized field.")
def __delitem__(self, key: str) -> None:
@@ -144,9 +256,9 @@ class DataRecord(DataBase, MutableMapping):
Raises:
KeyError: If the specified key does not exist in the fields.
"""
if key in self.model_fields:
setattr(self, key, None) # Optional: set to None instead of deleting
else:
try:
self.__delattr__(key)
except:
raise KeyError(f"'{key}' is not a recognized field.")
def __iter__(self) -> Iterator[str]:
@@ -155,7 +267,7 @@ class DataRecord(DataBase, MutableMapping):
Returns:
Iterator[str]: An iterator over field names.
"""
return iter(self.model_fields)
return iter(self.record_keys_writable())
def __len__(self) -> int:
"""Return the number of fields in the data record.
@@ -163,7 +275,7 @@ class DataRecord(DataBase, MutableMapping):
Returns:
int: The number of defined fields.
"""
return len(self.model_fields)
return len(self.record_keys_writable())
def __repr__(self) -> str:
"""Provide a string representation of the data record.
@@ -171,7 +283,7 @@ class DataRecord(DataBase, MutableMapping):
Returns:
str: A string representation showing field names and their values.
"""
field_values = {field: getattr(self, field) for field in self.model_fields}
field_values = {field: getattr(self, field) for field in self.__class__.model_fields}
return f"{self.__class__.__name__}({field_values})"
def __getattr__(self, key: str) -> Any:
@@ -186,8 +298,13 @@ class DataRecord(DataBase, MutableMapping):
Raises:
AttributeError: If the field does not exist.
"""
if key in self.model_fields:
if key in self.__class__.model_fields:
return getattr(self, key)
if key in self.configured_data.keys():
return self.configured_data[key]
configured_keys = self.configured_data_keys()
if configured_keys is not None and key in configured_keys:
return None
raise AttributeError(f"'{self.__class__.__name__}' object has no attribute '{key}'")
def __setattr__(self, key: str, value: Any) -> None:
@@ -200,10 +317,14 @@ class DataRecord(DataBase, MutableMapping):
Raises:
AttributeError: If the attribute/field does not exist.
"""
if key in self.model_fields:
if key in self.__class__.model_fields:
super().__setattr__(key, value)
else:
raise AttributeError(f"'{self.__class__.__name__}' object has no attribute '{key}'")
return
configured_keys = self.configured_data_keys()
if configured_keys is not None and key in configured_keys:
self.configured_data[key] = value
return
raise AttributeError(f"'{self.__class__.__name__}' object has no attribute '{key}'")
def __delattr__(self, key: str) -> None:
"""Delete an attribute by setting it to None if it exists as a field.
@@ -214,10 +335,21 @@ class DataRecord(DataBase, MutableMapping):
Raises:
AttributeError: If the attribute/field does not exist.
"""
if key in self.model_fields:
setattr(self, key, None) # Optional: set to None instead of deleting
else:
super().__delattr__(key)
if key in self.__class__.model_fields:
data: Optional[dict]
if key == "configured_data":
data = dict()
else:
data = None
setattr(self, key, data)
return
if key in self.configured_data:
del self.configured_data[key]
return
configured_keys = self.configured_data_keys()
if configured_keys is not None and key in configured_keys:
return
super().__delattr__(key)
@classmethod
def key_from_description(cls, description: str, threshold: float = 0.8) -> Optional[str]:
@@ -352,10 +484,7 @@ class DataSequence(DataBase, MutableSequence):
@property
def record_keys(self) -> List[str]:
"""Returns the keys of all fields in the data records."""
key_list = []
key_list.extend(list(self.record_class().model_fields.keys()))
key_list.extend(list(self.record_class().__pydantic_decorators__.computed_fields.keys()))
return key_list
return self.record_class().record_keys()
@computed_field # type: ignore[prop-decorator]
@property
@@ -369,7 +498,7 @@ class DataSequence(DataBase, MutableSequence):
Returns:
List[str]: A list of field keys that are writable in the data records.
"""
return list(self.record_class().model_fields.keys())
return self.record_class().record_keys_writable()
@classmethod
def record_class(cls) -> Type:
@@ -707,6 +836,38 @@ class DataSequence(DataBase, MutableSequence):
return filtered_data
def key_to_value(self, key: str, target_datetime: DateTime) -> Optional[float]:
"""Returns the value corresponding to the specified key that is nearest to the given datetime.
Args:
key (str): The key of the attribute in DataRecord to extract.
target_datetime (datetime): The datetime to search nearest to.
Returns:
Optional[float]: The value nearest to the given datetime, or None if no valid records are found.
Raises:
KeyError: If the specified key is not found in any of the DataRecords.
"""
self._validate_key(key)
# Filter out records with None or NaN values for the key
valid_records = [
record
for record in self.records
if record.date_time is not None
and getattr(record, key, None) not in (None, float("nan"))
]
if not valid_records:
return None
# Find the record with datetime nearest to target_datetime
target = to_datetime(target_datetime)
nearest_record = min(valid_records, key=lambda r: abs(r.date_time - target))
return getattr(nearest_record, key, None)
def key_to_lists(
self,
key: str,
@@ -868,6 +1029,11 @@ class DataSequence(DataBase, MutableSequence):
KeyError: If the specified key is not found in any of the DataRecords.
"""
self._validate_key(key)
# General check on fill_method
if fill_method not in ("ffill", "bfill", "linear", "none", None):
raise ValueError(f"Unsupported fill method: {fill_method}")
# Ensure datetime objects are normalized
start_datetime = to_datetime(start_datetime, to_maxtime=False) if start_datetime else None
end_datetime = to_datetime(end_datetime, to_maxtime=False) if end_datetime else None
@@ -880,7 +1046,7 @@ class DataSequence(DataBase, MutableSequence):
values_len = len(values)
if values_len < 1:
# No values, assume at at least one value set to None
# No values, assume at least one value set to None
if start_datetime is not None:
dates.append(start_datetime - interval)
else:
@@ -902,6 +1068,11 @@ class DataSequence(DataBase, MutableSequence):
# Truncate all values before latest value before start_datetime
dates = dates[start_index - 1 :]
values = values[start_index - 1 :]
# We have a start_datetime, align to start datetime
resample_origin = start_datetime
else:
# We do not have a start_datetime, align resample buckets to midnight of first day
resample_origin = "start_day"
if end_datetime is not None:
if compare_datetimes(dates[-1], end_datetime).lt:
@@ -922,7 +1093,7 @@ class DataSequence(DataBase, MutableSequence):
if fill_method is None:
fill_method = "linear"
# Resample the series to the specified interval
resampled = series.resample(interval, origin="start").first()
resampled = series.resample(interval, origin=resample_origin).first()
if fill_method == "linear":
resampled = resampled.interpolate(method="linear")
elif fill_method == "ffill":
@@ -936,7 +1107,7 @@ class DataSequence(DataBase, MutableSequence):
if fill_method is None:
fill_method = "ffill"
# Resample the series to the specified interval
resampled = series.resample(interval, origin="start").first()
resampled = series.resample(interval, origin=resample_origin).first()
if fill_method == "ffill":
resampled = resampled.ffill()
elif fill_method == "bfill":
@@ -944,12 +1115,24 @@ class DataSequence(DataBase, MutableSequence):
elif fill_method != "none":
raise ValueError(f"Unsupported fill method for non-numeric data: {fill_method}")
logger.debug(
"Resampled for '{}' with length {}: {}...{}",
key,
len(resampled),
resampled[:10],
resampled[-10:],
)
# Convert the resampled series to a NumPy array
if start_datetime is not None and len(resampled) > 0:
resampled = resampled.truncate(before=start_datetime)
if end_datetime is not None and len(resampled) > 0:
resampled = resampled.truncate(after=end_datetime.subtract(seconds=1))
array = resampled.values
logger.debug(
"Array for '{}' with length {}: {}...{}", key, len(array), array[:10], array[-10:]
)
return array
def to_dataframe(
@@ -1197,7 +1380,7 @@ class DataImportMixin:
the values. `ìnterval` may be used to define the fixed time interval between two values.
On import `self.update_value(datetime, key, value)` is called which has to be provided.
Also `self.start_datetime` may be necessary as a default in case `start_datetime`is not given.
Also `self.ems_start_datetime` may be necessary as a default in case `start_datetime`is not given.
"""
# Attributes required but defined elsehere.
@@ -1315,7 +1498,7 @@ class DataImportMixin:
raise ValueError(f"Invalid start_datetime in import data: {e}")
if start_datetime is None:
start_datetime = self.start_datetime # type: ignore
start_datetime = self.ems_start_datetime # type: ignore
if "interval" in import_data:
try:
@@ -1406,7 +1589,7 @@ class DataImportMixin:
raise ValueError(f"Invalid datetime index in DataFrame: {e}")
else:
if start_datetime is None:
start_datetime = self.start_datetime # type: ignore
start_datetime = self.ems_start_datetime # type: ignore
has_datetime_index = False
# Filter columns based on key_prefix and record_keys_writable
@@ -1463,7 +1646,7 @@ class DataImportMixin:
If start_datetime and or interval is given in the JSON dict it will be used. Otherwise
the given parameters are used. If None is given start_datetime defaults to
'self.start_datetime' and interval defaults to 1 hour.
'self.ems_start_datetime' and interval defaults to 1 hour.
Args:
json_str (str): The JSON string containing the generic data.
@@ -1538,7 +1721,7 @@ class DataImportMixin:
If start_datetime and or interval is given in the JSON dict it will be used. Otherwise
the given parameters are used. If None is given start_datetime defaults to
'self.start_datetime' and interval defaults to 1 hour.
'self.ems_start_datetime' and interval defaults to 1 hour.
Args:
import_file_path (Path): The path to the JSON file containing the generic data.
@@ -1749,7 +1932,12 @@ class DataContainer(SingletonMixin, DataBase, MutableMapping):
force_update (bool, optional): If True, forces the providers to update the data even if still cached.
"""
for provider in self.providers:
provider.update_data(force_enable=force_enable, force_update=force_update)
try:
provider.update_data(force_enable=force_enable, force_update=force_update)
except Exception as ex:
error = f"Provider {provider.provider_id()} fails on update - enabled={provider.enabled()}, force_enable={force_enable}, force_update={force_update}: {ex}"
logger.error(error)
raise RuntimeError(error)
def key_to_series(
self,
@@ -1854,7 +2042,7 @@ class DataContainer(SingletonMixin, DataBase, MutableMapping):
) -> pd.DataFrame:
"""Retrieve a dataframe indexed by fixed time intervals for specified keys from the data in each DataProvider.
Generates a pandas DataFrame using the NumPy arrays for each specified key, ensuring a common time index..
Generates a pandas DataFrame using the NumPy arrays for each specified key, ensuring a common time index.
Args:
keys (list[str]): A list of field names to retrieve.
@@ -1903,8 +2091,15 @@ class DataContainer(SingletonMixin, DataBase, MutableMapping):
end_datetime.add(seconds=1)
# Create a DatetimeIndex based on start, end, and interval
if start_datetime is None or end_datetime is None:
raise ValueError(
f"Can not determine datetime range. Got '{start_datetime}'..'{end_datetime}'."
)
reference_index = pd.date_range(
start=start_datetime, end=end_datetime, freq=interval, inclusive="left"
start=start_datetime,
end=end_datetime,
freq=interval,
inclusive="left",
)
data = {}

1914
src/akkudoktoreos/core/emplan.py Normal file
View File

File diff suppressed because it is too large Load Diff

641
src/akkudoktoreos/core/ems.py
View File

@@ -1,123 +1,50 @@
import traceback
from typing import Any, ClassVar, Optional
from asyncio import Lock, get_running_loop
from concurrent.futures import ThreadPoolExecutor
from functools import partial
from typing import ClassVar, Optional
import numpy as np
from loguru import logger
from numpydantic import NDArray, Shape
from pendulum import DateTime
from pydantic import ConfigDict, Field, computed_field, field_validator, model_validator
from typing_extensions import Self
from pydantic import computed_field
from akkudoktoreos.core.cache import CacheUntilUpdateStore
from akkudoktoreos.core.cache import CacheEnergyManagementStore
from akkudoktoreos.core.coreabc import ConfigMixin, PredictionMixin, SingletonMixin
from akkudoktoreos.core.pydantic import ParametersBaseModel, PydanticBaseModel
from akkudoktoreos.devices.battery import Battery
from akkudoktoreos.devices.generic import HomeAppliance
from akkudoktoreos.devices.inverter import Inverter
from akkudoktoreos.utils.datetimeutil import compare_datetimes, to_datetime
from akkudoktoreos.utils.utils import NumpyEncoder
from akkudoktoreos.core.emplan import EnergyManagementPlan
from akkudoktoreos.core.emsettings import EnergyManagementMode
from akkudoktoreos.core.pydantic import PydanticBaseModel
from akkudoktoreos.optimization.genetic.genetic import GeneticOptimization
from akkudoktoreos.optimization.genetic.geneticparams import (
GeneticOptimizationParameters,
)
from akkudoktoreos.optimization.genetic.geneticsolution import GeneticSolution
from akkudoktoreos.optimization.optimization import OptimizationSolution
from akkudoktoreos.utils.datetimeutil import DateTime, compare_datetimes, to_datetime
class EnergyManagementParameters(ParametersBaseModel):
pv_prognose_wh: list[float] = Field(
description="An array of floats representing the forecasted photovoltaic output in watts for different time intervals."
)
strompreis_euro_pro_wh: list[float] = Field(
description="An array of floats representing the electricity price in euros per watt-hour for different time intervals."
)
einspeiseverguetung_euro_pro_wh: list[float] | float = Field(
description="A float or array of floats representing the feed-in compensation in euros per watt-hour."
)
preis_euro_pro_wh_akku: float = Field(
description="A float representing the cost of battery energy per watt-hour."
)
gesamtlast: list[float] = Field(
description="An array of floats representing the total load (consumption) in watts for different time intervals."
)
@model_validator(mode="after")
def validate_list_length(self) -> Self:
pv_prognose_length = len(self.pv_prognose_wh)
if (
pv_prognose_length != len(self.strompreis_euro_pro_wh)
or pv_prognose_length != len(self.gesamtlast)
or (
isinstance(self.einspeiseverguetung_euro_pro_wh, list)
and pv_prognose_length != len(self.einspeiseverguetung_euro_pro_wh)
)
):
raise ValueError("Input lists have different lengths")
return self
class SimulationResult(ParametersBaseModel):
"""This object contains the results of the simulation and provides insights into various parameters over the entire forecast period."""
Last_Wh_pro_Stunde: list[Optional[float]] = Field(description="TBD")
EAuto_SoC_pro_Stunde: list[Optional[float]] = Field(
description="The state of charge of the EV for each hour."
)
Einnahmen_Euro_pro_Stunde: list[Optional[float]] = Field(
description="The revenue from grid feed-in or other sources in euros per hour."
)
Gesamt_Verluste: float = Field(
description="The total losses in watt-hours over the entire period."
)
Gesamtbilanz_Euro: float = Field(
description="The total balance of revenues minus costs in euros."
)
Gesamteinnahmen_Euro: float = Field(description="The total revenues in euros.")
Gesamtkosten_Euro: float = Field(description="The total costs in euros.")
Home_appliance_wh_per_hour: list[Optional[float]] = Field(
description="The energy consumption of a household appliance in watt-hours per hour."
)
Kosten_Euro_pro_Stunde: list[Optional[float]] = Field(
description="The costs in euros per hour."
)
Netzbezug_Wh_pro_Stunde: list[Optional[float]] = Field(
description="The grid energy drawn in watt-hours per hour."
)
Netzeinspeisung_Wh_pro_Stunde: list[Optional[float]] = Field(
description="The energy fed into the grid in watt-hours per hour."
)
Verluste_Pro_Stunde: list[Optional[float]] = Field(
description="The losses in watt-hours per hour."
)
akku_soc_pro_stunde: list[Optional[float]] = Field(
description="The state of charge of the battery (not the EV) in percentage per hour."
)
Electricity_price: list[Optional[float]] = Field(
description="Used Electricity Price, including predictions"
)
@field_validator(
"Last_Wh_pro_Stunde",
"Netzeinspeisung_Wh_pro_Stunde",
"akku_soc_pro_stunde",
"Netzbezug_Wh_pro_Stunde",
"Kosten_Euro_pro_Stunde",
"Einnahmen_Euro_pro_Stunde",
"EAuto_SoC_pro_Stunde",
"Verluste_Pro_Stunde",
"Home_appliance_wh_per_hour",
"Electricity_price",
mode="before",
)
def convert_numpy(cls, field: Any) -> Any:
return NumpyEncoder.convert_numpy(field)[0]
# The executor to execute the CPU heavy energy management run
executor = ThreadPoolExecutor(max_workers=1)
class EnergyManagement(SingletonMixin, ConfigMixin, PredictionMixin, PydanticBaseModel):
# Disable validation on assignment to speed up simulation runs.
model_config = ConfigDict(
validate_assignment=False,
)
"""Energy management."""
# Start datetime.
_start_datetime: ClassVar[Optional[DateTime]] = None
# last run datetime. Used by energy management task
_last_datetime: ClassVar[Optional[DateTime]] = None
_last_run_datetime: ClassVar[Optional[DateTime]] = None
# energy management plan of latest energy management run with optimization
_plan: ClassVar[Optional[EnergyManagementPlan]] = None
# opimization solution of the latest energy management run
_optimization_solution: ClassVar[Optional[OptimizationSolution]] = None
# Solution of the genetic algorithm of latest energy management run with optimization
# For classic API
_genetic_solution: ClassVar[Optional[GeneticSolution]] = None
# energy management lock (for energy management run)
_run_lock: ClassVar[Lock] = Lock()
@computed_field # type: ignore[prop-decorator]
@property
@@ -127,9 +54,15 @@ class EnergyManagement(SingletonMixin, ConfigMixin, PredictionMixin, PydanticBas
EnergyManagement.set_start_datetime()
return EnergyManagement._start_datetime
@computed_field # type: ignore[prop-decorator]
@property
def last_run_datetime(self) -> Optional[DateTime]:
"""The datetime the last energy management was run."""
return EnergyManagement._last_run_datetime
@classmethod
def set_start_datetime(cls, start_datetime: Optional[DateTime] = None) -> DateTime:
"""Set the start datetime for the next energy management cycle.
"""Set the start datetime for the next energy management run.
If no datetime is provided, the current datetime is used.
@@ -148,142 +81,208 @@ class EnergyManagement(SingletonMixin, ConfigMixin, PredictionMixin, PydanticBas
cls._start_datetime = start_datetime.set(minute=0, second=0, microsecond=0)
return cls._start_datetime
# -------------------------
# TODO: Take from prediction
# -------------------------
@classmethod
def plan(cls) -> Optional[EnergyManagementPlan]:
"""Get the latest energy management plan.
load_energy_array: Optional[NDArray[Shape["*"], float]] = Field(
default=None,
description="An array of floats representing the total load (consumption) in watts for different time intervals.",
)
pv_prediction_wh: Optional[NDArray[Shape["*"], float]] = Field(
default=None,
description="An array of floats representing the forecasted photovoltaic output in watts for different time intervals.",
)
elect_price_hourly: Optional[NDArray[Shape["*"], float]] = Field(
default=None,
description="An array of floats representing the electricity price in euros per watt-hour for different time intervals.",
)
elect_revenue_per_hour_arr: Optional[NDArray[Shape["*"], float]] = Field(
default=None,
description="An array of floats representing the feed-in compensation in euros per watt-hour.",
)
Returns:
Optional[EnergyManagementPlan]: The latest energy management plan or None.
"""
return cls._plan
# -------------------------
# TODO: Move to devices
# -------------------------
@classmethod
def optimization_solution(cls) -> Optional[OptimizationSolution]:
"""Get the latest optimization solution.
battery: Optional[Battery] = Field(default=None, description="TBD.")
ev: Optional[Battery] = Field(default=None, description="TBD.")
home_appliance: Optional[HomeAppliance] = Field(default=None, description="TBD.")
inverter: Optional[Inverter] = Field(default=None, description="TBD.")
Returns:
Optional[OptimizationSolution]: The latest optimization solution.
"""
return cls._optimization_solution
# -------------------------
# TODO: Move to devices
# -------------------------
@classmethod
def genetic_solution(cls) -> Optional[GeneticSolution]:
"""Get the latest solution of the genetic algorithm.
ac_charge_hours: Optional[NDArray[Shape["*"], float]] = Field(default=None, description="TBD")
dc_charge_hours: Optional[NDArray[Shape["*"], float]] = Field(default=None, description="TBD")
ev_charge_hours: Optional[NDArray[Shape["*"], float]] = Field(default=None, description="TBD")
Returns:
Optional[GeneticSolution]: The latest solution of the genetic algorithm.
"""
return cls._genetic_solution
def __init__(self, *args: Any, **kwargs: Any) -> None:
if hasattr(self, "_initialized"):
return
super().__init__(*args, **kwargs)
def set_parameters(
self,
parameters: EnergyManagementParameters,
ev: Optional[Battery] = None,
home_appliance: Optional[HomeAppliance] = None,
inverter: Optional[Inverter] = None,
) -> None:
self.load_energy_array = np.array(parameters.gesamtlast, float)
self.pv_prediction_wh = np.array(parameters.pv_prognose_wh, float)
self.elect_price_hourly = np.array(parameters.strompreis_euro_pro_wh, float)
self.elect_revenue_per_hour_arr = (
parameters.einspeiseverguetung_euro_pro_wh
if isinstance(parameters.einspeiseverguetung_euro_pro_wh, list)
else np.full(
len(self.load_energy_array), parameters.einspeiseverguetung_euro_pro_wh, float
)
)
if inverter:
self.battery = inverter.battery
else:
self.battery = None
self.ev = ev
self.home_appliance = home_appliance
self.inverter = inverter
self.ac_charge_hours = np.full(self.config.prediction.hours, 0.0)
self.dc_charge_hours = np.full(self.config.prediction.hours, 1.0)
self.ev_charge_hours = np.full(self.config.prediction.hours, 0.0)
def set_akku_discharge_hours(self, ds: np.ndarray) -> None:
if self.battery:
self.battery.set_discharge_per_hour(ds)
def set_akku_ac_charge_hours(self, ds: np.ndarray) -> None:
self.ac_charge_hours = ds
def set_akku_dc_charge_hours(self, ds: np.ndarray) -> None:
self.dc_charge_hours = ds
def set_ev_charge_hours(self, ds: np.ndarray) -> None:
self.ev_charge_hours = ds
def set_home_appliance_start(self, ds: int, global_start_hour: int = 0) -> None:
if self.home_appliance:
self.home_appliance.set_starting_time(ds, global_start_hour=global_start_hour)
def reset(self) -> None:
if self.ev:
self.ev.reset()
if self.battery:
self.battery.reset()
def run(
self,
start_hour: Optional[int] = None,
@classmethod
def _run(
cls,
start_datetime: Optional[DateTime] = None,
mode: Optional[EnergyManagementMode] = None,
genetic_parameters: Optional[GeneticOptimizationParameters] = None,
genetic_individuals: Optional[int] = None,
genetic_seed: Optional[int] = None,
force_enable: Optional[bool] = False,
force_update: Optional[bool] = False,
) -> None:
"""Run energy management.
"""Run the energy management.
Sets `start_datetime` to current hour, updates the configuration and the prediction, and
starts simulation at current hour.
This method initializes the energy management run by setting its
start datetime, updating predictions, and optionally starting
optimization depending on the selected mode or configuration.
Args:
start_hour (int, optional): Hour to take as start time for the energy management. Defaults
to now.
force_enable (bool, optional): If True, forces to update even if disabled. This
is mostly relevant to prediction providers.
force_update (bool, optional): If True, forces to update the data even if still cached.
start_datetime (DateTime, optional): The starting timestamp
of the energy management run. Defaults to the current datetime
if not provided.
mode (EnergyManagementMode, optional): The management mode to use. Must be one of:
- "OPTIMIZATION": Runs the optimization process.
- "PREDICTION": Updates the forecast without optimization.
Defaults to the mode defined in the current configuration.
genetic_parameters (GeneticOptimizationParameters, optional): The
parameter set for the genetic algorithm. If not provided, it will
be constructed based on the current configuration and predictions.
genetic_individuals (int, optional): The number of individuals for the
genetic algorithm. Defaults to the algorithm's internal default (400)
if not specified.
genetic_seed (int, optional): The seed for the genetic algorithm. Defaults
to the algorithm's internal random seed if not specified.
force_enable (bool, optional): If True, bypasses any disabled state
to force the update process. This is mostly applicable to
prediction providers.
force_update (bool, optional): If True, forces data to be refreshed
even if a cached version is still valid.
Returns:
None
"""
# Throw away any cached results of the last run.
CacheUntilUpdateStore().clear()
self.set_start_hour(start_hour=start_hour)
# Ensure there is only one optimization/ energy management run at a time
if mode not in (None, "PREDICTION", "OPTIMIZATION"):
raise ValueError(f"Unknown energy management mode {mode}.")
# Check for run definitions
if self.start_datetime is None:
error_msg = "Start datetime unknown."
logger.error(error_msg)
raise ValueError(error_msg)
if self.config.prediction.hours is None:
error_msg = "Prediction hours unknown."
logger.error(error_msg)
raise ValueError(error_msg)
if self.config.optimization.hours is None:
error_msg = "Optimization hours unknown."
logger.error(error_msg)
raise ValueError(error_msg)
logger.info("Starting energy management run.")
self.prediction.update_data(force_enable=force_enable, force_update=force_update)
# TODO: Create optimisation problem that calls into devices.update_data() for simulations.
# Remember/ set the start datetime of this energy management run.
# None leads
cls.set_start_datetime(start_datetime)
logger.info("Energy management run (crippled version - prediction update only)")
# Throw away any memory cached results of the last energy management run.
CacheEnergyManagementStore().clear()
def manage_energy(self) -> None:
if mode is None:
mode = cls.config.ems.mode
if mode is None or mode == "PREDICTION":
# Update the predictions
cls.prediction.update_data(force_enable=force_enable, force_update=force_update)
logger.info("Energy management run done (predictions updated)")
return
# Prepare optimization parameters
# This also creates default configurations for missing values and updates the predictions
logger.info(
"Starting energy management prediction update and optimzation parameter preparation."
)
if genetic_parameters is None:
genetic_parameters = GeneticOptimizationParameters.prepare()
if not genetic_parameters:
logger.error(
"Energy management run canceled. Could not prepare optimisation parameters."
)
return
# Take values from config if not given
if genetic_individuals is None:
genetic_individuals = cls.config.optimization.genetic.individuals
if genetic_seed is None:
genetic_seed = cls.config.optimization.genetic.seed
if cls._start_datetime is None: # Make mypy happy - already set by us
raise RuntimeError("Start datetime not set.")
logger.info("Starting energy management optimization.")
try:
optimization = GeneticOptimization(
verbose=bool(cls.config.server.verbose),
fixed_seed=genetic_seed,
)
solution = optimization.optimierung_ems(
start_hour=cls._start_datetime.hour,
parameters=genetic_parameters,
ngen=genetic_individuals,
)
except:
logger.exception("Energy management optimization failed.")
return
# Make genetic solution public
cls._genetic_solution = solution
# Make optimization solution public
cls._optimization_solution = solution.optimization_solution()
# Make plan public
cls._plan = solution.energy_management_plan()
logger.debug("Energy management genetic solution:\n{}", cls._genetic_solution)
logger.debug("Energy management optimization solution:\n{}", cls._optimization_solution)
logger.debug("Energy management plan:\n{}", cls._plan)
logger.info("Energy management run done (optimization updated)")
async def run(
self,
start_datetime: Optional[DateTime] = None,
mode: Optional[EnergyManagementMode] = None,
genetic_parameters: Optional[GeneticOptimizationParameters] = None,
genetic_individuals: Optional[int] = None,
genetic_seed: Optional[int] = None,
force_enable: Optional[bool] = False,
force_update: Optional[bool] = False,
) -> None:
"""Run the energy management.
This method initializes the energy management run by setting its
start datetime, updating predictions, and optionally starting
optimization depending on the selected mode or configuration.
Args:
start_datetime (DateTime, optional): The starting timestamp
of the energy management run. Defaults to the current datetime
if not provided.
mode (EnergyManagementMode, optional): The management mode to use. Must be one of:
- "OPTIMIZATION": Runs the optimization process.
- "PREDICTION": Updates the forecast without optimization.
Defaults to the mode defined in the current configuration.
genetic_parameters (GeneticOptimizationParameters, optional): The
parameter set for the genetic algorithm. If not provided, it will
be constructed based on the current configuration and predictions.
genetic_individuals (int, optional): The number of individuals for the
genetic algorithm. Defaults to the algorithm's internal default (400)
if not specified.
genetic_seed (int, optional): The seed for the genetic algorithm. Defaults
to the algorithm's internal random seed if not specified.
force_enable (bool, optional): If True, bypasses any disabled state
to force the update process. This is mostly applicable to
prediction providers.
force_update (bool, optional): If True, forces data to be refreshed
even if a cached version is still valid.
Returns:
None
"""
async with self._run_lock:
loop = get_running_loop()
# Create a partial function with parameters "baked in"
func = partial(
EnergyManagement._run,
start_datetime=start_datetime,
mode=mode,
genetic_parameters=genetic_parameters,
genetic_individuals=genetic_individuals,
genetic_seed=genetic_seed,
force_enable=force_enable,
force_update=force_update,
)
# Run optimization in background thread to avoid blocking event loop
await loop.run_in_executor(executor, func)
async def manage_energy(self) -> None:
"""Repeating task for managing energy.
This task should be executed by the server regularly (e.g., every 10 seconds)
@@ -304,13 +303,13 @@ class EnergyManagement(SingletonMixin, ConfigMixin, PredictionMixin, PydanticBas
current_datetime = to_datetime()
interval = self.config.ems.interval # interval maybe changed in between
if EnergyManagement._last_datetime is None:
if EnergyManagement._last_run_datetime is None:
# Never run before
try:
# Remember energy run datetime.
EnergyManagement._last_datetime = current_datetime
EnergyManagement._last_run_datetime = current_datetime
# Try to run a first energy management. May fail due to config incomplete.
self.run()
await self.run()
except Exception as e:
trace = "".join(traceback.TracebackException.from_exception(e).format())
message = f"EOS init: {e}\n{trace}"
@@ -322,14 +321,14 @@ class EnergyManagement(SingletonMixin, ConfigMixin, PredictionMixin, PydanticBas
return
if (
compare_datetimes(current_datetime, EnergyManagement._last_datetime).time_diff
compare_datetimes(current_datetime, EnergyManagement._last_run_datetime).time_diff
< interval
):
# Wait for next run
return
try:
self.run()
await self.run()
except Exception as e:
trace = "".join(traceback.TracebackException.from_exception(e).format())
message = f"EOS run: {e}\n{trace}"
@@ -337,187 +336,13 @@ class EnergyManagement(SingletonMixin, ConfigMixin, PredictionMixin, PydanticBas
# Remember the energy management run - keep on interval even if we missed some intervals
while (
compare_datetimes(current_datetime, EnergyManagement._last_datetime).time_diff
compare_datetimes(current_datetime, EnergyManagement._last_run_datetime).time_diff
>= interval
):
EnergyManagement._last_datetime = EnergyManagement._last_datetime.add(seconds=interval)
def set_start_hour(self, start_hour: Optional[int] = None) -> None:
"""Sets start datetime to given hour.
Args:
start_hour (int, optional): Hour to take as start time for the energy management. Defaults
to now.
"""
if start_hour is None:
self.set_start_datetime()
else:
start_datetime = to_datetime().set(hour=start_hour, minute=0, second=0, microsecond=0)
self.set_start_datetime(start_datetime)
def simulate_start_now(self) -> dict[str, Any]:
start_hour = to_datetime().now().hour
return self.simulate(start_hour)
def simulate(self, start_hour: int) -> dict[str, Any]:
"""Simulate energy usage and costs for the given start hour.
akku_soc_pro_stunde begin of the hour, initial hour state!
last_wh_pro_stunde integral of last hour (end state)
"""
# Check for simulation integrity
required_attrs = [
"load_energy_array",
"pv_prediction_wh",
"elect_price_hourly",
"ev_charge_hours",
"ac_charge_hours",
"dc_charge_hours",
"elect_revenue_per_hour_arr",
]
missing_data = [
attr.replace("_", " ").title() for attr in required_attrs if getattr(self, attr) is None
]
if missing_data:
logger.error("Mandatory data missing - %s", ", ".join(missing_data))
raise ValueError(f"Mandatory data missing: {', '.join(missing_data)}")
# Pre-fetch data
load_energy_array = np.array(self.load_energy_array)
pv_prediction_wh = np.array(self.pv_prediction_wh)
elect_price_hourly = np.array(self.elect_price_hourly)
ev_charge_hours = np.array(self.ev_charge_hours)
ac_charge_hours = np.array(self.ac_charge_hours)
dc_charge_hours = np.array(self.dc_charge_hours)
elect_revenue_per_hour_arr = np.array(self.elect_revenue_per_hour_arr)
# Fetch objects
battery = self.battery
if battery is None:
raise ValueError(f"battery not set: {battery}")
ev = self.ev
home_appliance = self.home_appliance
inverter = self.inverter
if not (len(load_energy_array) == len(pv_prediction_wh) == len(elect_price_hourly)):
error_msg = f"Array sizes do not match: Load Curve = {len(load_energy_array)}, PV Forecast = {len(pv_prediction_wh)}, Electricity Price = {len(elect_price_hourly)}"
logger.error(error_msg)
raise ValueError(error_msg)
end_hour = len(load_energy_array)
total_hours = end_hour - start_hour
# Pre-allocate arrays for the results, optimized for speed
loads_energy_per_hour = np.full((total_hours), np.nan)
feedin_energy_per_hour = np.full((total_hours), np.nan)
consumption_energy_per_hour = np.full((total_hours), np.nan)
costs_per_hour = np.full((total_hours), np.nan)
revenue_per_hour = np.full((total_hours), np.nan)
soc_per_hour = np.full((total_hours), np.nan)
soc_ev_per_hour = np.full((total_hours), np.nan)
losses_wh_per_hour = np.full((total_hours), np.nan)
home_appliance_wh_per_hour = np.full((total_hours), np.nan)
electricity_price_per_hour = np.full((total_hours), np.nan)
# Set initial state
soc_per_hour[0] = battery.current_soc_percentage()
if ev:
soc_ev_per_hour[0] = ev.current_soc_percentage()
for hour in range(start_hour, end_hour):
hour_idx = hour - start_hour
# save begin states
soc_per_hour[hour_idx] = battery.current_soc_percentage()
if ev:
soc_ev_per_hour[hour_idx] = ev.current_soc_percentage()
# Accumulate loads and PV generation
consumption = load_energy_array[hour]
losses_wh_per_hour[hour_idx] = 0.0
# Home appliances
if home_appliance:
ha_load = home_appliance.get_load_for_hour(hour)
consumption += ha_load
home_appliance_wh_per_hour[hour_idx] = ha_load
# E-Auto handling
if ev and ev_charge_hours[hour] > 0:
loaded_energy_ev, verluste_eauto = ev.charge_energy(
None, hour, relative_power=ev_charge_hours[hour]
)
consumption += loaded_energy_ev
losses_wh_per_hour[hour_idx] += verluste_eauto
# Process inverter logic
energy_feedin_grid_actual = energy_consumption_grid_actual = losses = eigenverbrauch = (
0.0
EnergyManagement._last_run_datetime = EnergyManagement._last_run_datetime.add(
seconds=interval
)
hour_ac_charge = ac_charge_hours[hour]
hour_dc_charge = dc_charge_hours[hour]
hourly_electricity_price = elect_price_hourly[hour]
hourly_energy_revenue = elect_revenue_per_hour_arr[hour]
battery.set_charge_allowed_for_hour(hour_dc_charge, hour)
if inverter:
energy_produced = pv_prediction_wh[hour]
(
energy_feedin_grid_actual,
energy_consumption_grid_actual,
losses,
eigenverbrauch,
) = inverter.process_energy(energy_produced, consumption, hour)
# AC PV Battery Charge
if hour_ac_charge > 0.0:
battery.set_charge_allowed_for_hour(1, hour)
battery_charged_energy_actual, battery_losses_actual = battery.charge_energy(
None, hour, relative_power=hour_ac_charge
)
total_battery_energy = battery_charged_energy_actual + battery_losses_actual
consumption += total_battery_energy
energy_consumption_grid_actual += total_battery_energy
losses_wh_per_hour[hour_idx] += battery_losses_actual
# Update hourly arrays
feedin_energy_per_hour[hour_idx] = energy_feedin_grid_actual
consumption_energy_per_hour[hour_idx] = energy_consumption_grid_actual
losses_wh_per_hour[hour_idx] += losses
loads_energy_per_hour[hour_idx] = consumption
electricity_price_per_hour[hour_idx] = hourly_electricity_price
# Financial calculations
costs_per_hour[hour_idx] = energy_consumption_grid_actual * hourly_electricity_price
revenue_per_hour[hour_idx] = energy_feedin_grid_actual * hourly_energy_revenue
total_cost = np.nansum(costs_per_hour)
total_losses = np.nansum(losses_wh_per_hour)
total_revenue = np.nansum(revenue_per_hour)
# Prepare output dictionary
return {
"Last_Wh_pro_Stunde": loads_energy_per_hour,
"Netzeinspeisung_Wh_pro_Stunde": feedin_energy_per_hour,
"Netzbezug_Wh_pro_Stunde": consumption_energy_per_hour,
"Kosten_Euro_pro_Stunde": costs_per_hour,
"akku_soc_pro_stunde": soc_per_hour,
"Einnahmen_Euro_pro_Stunde": revenue_per_hour,
"Gesamtbilanz_Euro": total_cost - total_revenue,
"EAuto_SoC_pro_Stunde": soc_ev_per_hour,
"Gesamteinnahmen_Euro": total_revenue,
"Gesamtkosten_Euro": total_cost,
"Verluste_Pro_Stunde": losses_wh_per_hour,
"Gesamt_Verluste": total_losses,
"Home_appliance_wh_per_hour": home_appliance_wh_per_hour,
"Electricity_price": electricity_price_per_hour,
}
# Initialize the Energy Management System, it is a singleton.
ems = EnergyManagement()

14
src/akkudoktoreos/core/emsettings.py
View File

@@ -3,6 +3,7 @@
Kept in an extra module to avoid cyclic dependencies on package import.
"""
from enum import Enum
from typing import Optional
from pydantic import Field
@@ -10,6 +11,13 @@ from pydantic import Field
from akkudoktoreos.config.configabc import SettingsBaseModel
class EnergyManagementMode(str, Enum):
"""Energy management mode."""
PREDICTION = "PREDICTION"
OPTIMIZATION = "OPTIMIZATION"
class EnergyManagementCommonSettings(SettingsBaseModel):
"""Energy Management Configuration."""
@@ -24,3 +32,9 @@ class EnergyManagementCommonSettings(SettingsBaseModel):
description="Intervall in seconds between EOS energy management runs.",
examples=["300"],
)
mode: Optional[EnergyManagementMode] = Field(
default=None,
description="Energy management mode [OPTIMIZATION | PREDICTION].",
examples=["OPTIMIZATION", "PREDICTION"],
)

4
src/akkudoktoreos/core/logging.py
View File

@@ -42,6 +42,10 @@ class InterceptHandler(pylogging.Handler):
Args:
record (logging.LogRecord): A record object containing log message and metadata.
"""
# Skip DEBUG logs from matplotlib - very noisy
if record.name.startswith("matplotlib") and record.levelno <= pylogging.DEBUG:
return
try:
level = logger.level(record.levelname).name
except AttributeError:

5
src/akkudoktoreos/core/logsettings.py
View File

@@ -15,11 +15,6 @@ from akkudoktoreos.core.logabc import LOGGING_LEVELS
class LoggingCommonSettings(SettingsBaseModel):
"""Logging Configuration."""
level: Optional[str] = Field(
default=None,
deprecated="This is deprecated. Use console_level and file_level instead.",
)
console_level: Optional[str] = Field(
default=None,
description="Logging level when logging to console.",

312
src/akkudoktoreos/core/pydantic.py
View File

@@ -14,7 +14,6 @@ Key Features:
import inspect
import json
import re
import uuid
import weakref
from copy import deepcopy
@@ -32,23 +31,20 @@ from typing import (
from zoneinfo import ZoneInfo
import pandas as pd
import pendulum
from loguru import logger
from pandas.api.types import is_datetime64_any_dtype
from pydantic import (
AwareDatetime,
BaseModel,
ConfigDict,
Field,
PrivateAttr,
RootModel,
TypeAdapter,
ValidationError,
ValidationInfo,
field_validator,
)
from akkudoktoreos.utils.datetimeutil import to_datetime, to_duration
from akkudoktoreos.utils.datetimeutil import DateTime, to_datetime, to_duration
# Global weakref dictionary to hold external state per model instance
# Used as a workaround for PrivateAttr not working in e.g. Mixin Classes
@@ -56,49 +52,40 @@ _model_private_state: "weakref.WeakKeyDictionary[Union[PydanticBaseModel, Pydant
def merge_models(source: BaseModel, update_dict: dict[str, Any]) -> dict[str, Any]:
def deep_update(source_dict: dict[str, Any], update_dict: dict[str, Any]) -> dict[str, Any]:
for key, value in source_dict.items():
if isinstance(value, dict) and isinstance(update_dict.get(key), dict):
update_dict[key] = deep_update(update_dict[key], value)
else:
update_dict[key] = value
return update_dict
"""Merge a Pydantic model instance with an update dictionary.
Values in update_dict (including None) override source values.
Nested dictionaries are merged recursively.
Lists in update_dict replace source lists entirely.
Args:
source (BaseModel): Pydantic model instance serving as the source.
update_dict (dict[str, Any]): Dictionary with updates to apply.
Returns:
dict[str, Any]: Merged dictionary representing combined model data.
"""
def deep_merge(source_data: Any, update_data: Any) -> Any:
if isinstance(source_data, dict) and isinstance(update_data, dict):
merged = dict(source_data)
for key, update_value in update_data.items():
if key in merged:
merged[key] = deep_merge(merged[key], update_value)
else:
merged[key] = update_value
return merged
# If both are lists, replace source list with update list
if isinstance(source_data, list) and isinstance(update_data, list):
return update_data
# For other types or if update_data is None, override source_data
return update_data
source_dict = source.model_dump(exclude_unset=True)
merged_dict = deep_update(source_dict, deepcopy(update_dict))
return merged_dict
class PydanticTypeAdapterDateTime(TypeAdapter[pendulum.DateTime]):
"""Custom type adapter for Pendulum DateTime fields."""
@classmethod
def serialize(cls, value: Any) -> str:
"""Convert pendulum.DateTime to ISO 8601 string."""
if isinstance(value, pendulum.DateTime):
return value.to_iso8601_string()
raise ValueError(f"Expected pendulum.DateTime, got {type(value)}")
@classmethod
def deserialize(cls, value: Any) -> pendulum.DateTime:
"""Convert ISO 8601 string to pendulum.DateTime."""
if isinstance(value, str) and cls.is_iso8601(value):
try:
return pendulum.parse(value)
except pendulum.parsing.exceptions.ParserError as e:
raise ValueError(f"Invalid date format: {value}") from e
elif isinstance(value, pendulum.DateTime):
return value
raise ValueError(f"Expected ISO 8601 string or pendulum.DateTime, got {type(value)}")
@staticmethod
def is_iso8601(value: str) -> bool:
"""Check if the string is a valid ISO 8601 date string."""
iso8601_pattern = (
r"^(\d{4}-\d{2}-\d{2}[T ]\d{2}:\d{2}:\d{2}(?:\.\d{1,3})?(?:Z|[+-]\d{2}:\d{2})?)$"
)
return bool(re.match(iso8601_pattern, value))
merged_result = deep_merge(source_dict, deepcopy(update_dict))
return merged_result
class PydanticModelNestedValueMixin:
@@ -653,67 +640,9 @@ class PydanticBaseModel(PydanticModelNestedValueMixin, BaseModel):
"""
return hash(self._uuid)
@field_validator("*", mode="before")
def validate_and_convert_pendulum(cls, value: Any, info: ValidationInfo) -> Any:
"""Validator to convert fields of type `pendulum.DateTime`.
Converts fields to proper `pendulum.DateTime` objects, ensuring correct input types.
This method is invoked for every field before the field value is set. If the field's type
is `pendulum.DateTime`, it tries to convert string or timestamp values to `pendulum.DateTime`
objects. If the value cannot be converted, a validation error is raised.
Args:
value: The value to be assigned to the field.
info: Validation information for the field.
Returns:
The converted value, if successful.
Raises:
ValidationError: If the value cannot be converted to `pendulum.DateTime`.
"""
# Get the field name and expected type
field_name = info.field_name
expected_type = cls.model_fields[field_name].annotation
# Convert
if expected_type is pendulum.DateTime or expected_type is AwareDatetime:
try:
value = to_datetime(value)
except Exception as e:
raise ValueError(f"Cannot convert {value!r} to datetime: {e}")
return value
# Override Pydantic’s serialization for all DateTime fields
def model_dump(
self, *args: Any, include_computed_fields: bool = True, **kwargs: Any
) -> dict[str, Any]:
"""Custom dump method to handle serialization for DateTime fields."""
result = super().model_dump(*args, **kwargs)
if not include_computed_fields:
for computed_field_name in self.model_computed_fields:
result.pop(computed_field_name, None)
for key, value in result.items():
if isinstance(value, pendulum.DateTime):
result[key] = PydanticTypeAdapterDateTime.serialize(value)
return result
@classmethod
def model_construct(
cls, _fields_set: set[str] | None = None, **values: Any
) -> "PydanticBaseModel":
"""Custom constructor to handle deserialization for DateTime fields."""
for key, value in values.items():
if isinstance(value, str) and PydanticTypeAdapterDateTime.is_iso8601(value):
values[key] = PydanticTypeAdapterDateTime.deserialize(value)
return super().model_construct(_fields_set, **values)
def reset_to_defaults(self) -> "PydanticBaseModel":
"""Resets the fields to their default values."""
for field_name, field_info in self.model_fields.items():
for field_name, field_info in self.__class__.model_fields.items():
if field_info.default_factory is not None: # Handle fields with default_factory
default_value = field_info.default_factory()
else:
@@ -725,6 +654,19 @@ class PydanticBaseModel(PydanticModelNestedValueMixin, BaseModel):
pass
return self
# Override Pydantic’s serialization to include computed fields by default
def model_dump(
self, *args: Any, include_computed_fields: bool = True, **kwargs: Any
) -> dict[str, Any]:
"""Custom dump method to serialize computed fields by default."""
result = super().model_dump(*args, **kwargs)
if not include_computed_fields:
for computed_field_name in self.__class__.model_computed_fields:
result.pop(computed_field_name, None)
return result
def to_dict(self) -> dict:
"""Convert this PredictionRecord instance to a dictionary representation.
@@ -910,16 +852,27 @@ class PydanticDateTimeDataFrame(PydanticBaseModel):
if not v:
return v
valid_dtypes = {"int64", "float64", "bool", "datetime64[ns]", "object", "string"}
invalid_dtypes = set(v.values()) - valid_dtypes
if invalid_dtypes:
raise ValueError(f"Unsupported dtypes: {invalid_dtypes}")
# Allowed exact dtypes
valid_base_dtypes = {"int64", "float64", "bool", "object", "string"}
def is_valid_dtype(dtype: str) -> bool:
# Allow timezone-aware or naive datetime64
if dtype.startswith("datetime64[ns"):
return True
return dtype in valid_base_dtypes
invalid_dtypes = [dtype for dtype in v.values() if not is_valid_dtype(dtype)]
if invalid_dtypes:
raise ValueError(f"Unsupported dtypes: {set(invalid_dtypes)}")
# Cross-check with data column existence
data = info.data.get("data", {})
if data:
columns = set(next(iter(data.values())).keys())
if not all(col in columns for col in v.keys()):
raise ValueError("dtype columns must exist in data columns")
missing_columns = set(v.keys()) - columns
if missing_columns:
raise ValueError(f"dtype columns must exist in data columns: {missing_columns}")
return v
def to_dataframe(self) -> pd.DataFrame:
@@ -927,7 +880,8 @@ class PydanticDateTimeDataFrame(PydanticBaseModel):
df = pd.DataFrame.from_dict(self.data, orient="index")
# Convert index to datetime
index = pd.Index([to_datetime(dt, in_timezone=self.tz) for dt in df.index])
# index = pd.Index([to_datetime(dt, in_timezone=self.tz) for dt in df.index])
index = [to_datetime(dt, in_timezone=self.tz) for dt in df.index]
df.index = index
# Check if 'date_time' column exists, if not, create it
@@ -943,8 +897,8 @@ class PydanticDateTimeDataFrame(PydanticBaseModel):
# Apply dtypes
for col, dtype in self.dtypes.items():
if dtype == "datetime64[ns]":
df[col] = pd.to_datetime(to_datetime(df[col], in_timezone=self.tz))
if dtype.startswith("datetime64[ns"):
df[col] = pd.to_datetime(df[col], utc=True)
elif dtype in dtype_mapping.keys():
df[col] = df[col].astype(dtype_mapping[dtype])
else:
@@ -969,6 +923,132 @@ class PydanticDateTimeDataFrame(PydanticBaseModel):
datetime_columns=datetime_columns,
)
# --- Direct Manipulation Methods ---
def _normalize_index(self, index: str | DateTime) -> str:
"""Normalize index into timezone-aware datetime string.
Args:
index (str | DateTime): A datetime-like value.
Returns:
str: Normalized datetime string based on model timezone.
"""
return to_datetime(index, as_string=True, in_timezone=self.tz)
def add_row(self, index: str | DateTime, row: Dict[str, Any]) -> None:
"""Add a new row to the dataset.
Args:
index (str | DateTime): Timestamp of the new row.
row (Dict[str, Any]): Dictionary of column values. Must match existing columns.
Raises:
ValueError: If row does not contain the exact same columns as existing rows.
"""
idx = self._normalize_index(index)
if self.data:
existing_cols = set(next(iter(self.data.values())).keys())
if set(row.keys()) != existing_cols:
raise ValueError(f"Row must have exactly these columns: {existing_cols}")
self.data[idx] = row
def update_row(self, index: str | DateTime, updates: Dict[str, Any]) -> None:
"""Update values for an existing row.
Args:
index (str | DateTime): Timestamp of the row to modify.
updates (Dict[str, Any]): Key/value pairs of columns to update.
Raises:
KeyError: If row or column does not exist.
"""
idx = self._normalize_index(index)
if idx not in self.data:
raise KeyError(f"Row {idx} not found")
for col, value in updates.items():
if col not in self.data[idx]:
raise KeyError(f"Column {col} does not exist")
self.data[idx][col] = value
def delete_row(self, index: str | DateTime) -> None:
"""Delete a row from the dataset.
Args:
index (str | DateTime): Timestamp of the row to delete.
"""
idx = self._normalize_index(index)
if idx in self.data:
del self.data[idx]
def set_value(self, index: str | DateTime, column: str, value: Any) -> None:
"""Set a single cell value.
Args:
index (str | datetime): Timestamp of the row.
column (str): Column name.
value (Any): New value.
"""
self.update_row(index, {column: value})
def get_value(self, index: str | DateTime, column: str) -> Any:
"""Retrieve a single cell value.
Args:
index (str | DateTime): Timestamp of the row.
column (str): Column name.
Returns:
Any: Value stored at the given location.
"""
idx = self._normalize_index(index)
return self.data[idx][column]
def add_column(self, name: str, default: Any = None, dtype: Optional[str] = None) -> None:
"""Add a new column to all rows.
Args:
name (str): Name of the column to add.
default (Any, optional): Default value for all rows. Defaults to None.
dtype (Optional[str], optional): Declared data type. Defaults to None.
"""
for row in self.data.values():
row[name] = default
if dtype:
self.dtypes[name] = dtype
def rename_column(self, old: str, new: str) -> None:
"""Rename a column across all rows.
Args:
old (str): Existing column name.
new (str): New column name.
Raises:
KeyError: If column does not exist.
"""
for row in self.data.values():
if old not in row:
raise KeyError(f"Column {old} does not exist")
row[new] = row.pop(old)
if old in self.dtypes:
self.dtypes[new] = self.dtypes.pop(old)
if old in self.datetime_columns:
self.datetime_columns = [new if c == old else c for c in self.datetime_columns]
def drop_column(self, name: str) -> None:
"""Remove a column from all rows.
Args:
name (str): Column to remove.
"""
for row in self.data.values():
if name in row:
del row[name]
self.dtypes.pop(name, None)
self.datetime_columns = [c for c in self.datetime_columns if c != name]
class PydanticDateTimeSeries(PydanticBaseModel):
"""Pydantic model for validating pandas Series with datetime index in JSON format.
@@ -1068,10 +1148,6 @@ class PydanticDateTimeSeries(PydanticBaseModel):
)
class ParametersBaseModel(PydanticBaseModel):
model_config = ConfigDict(extra="forbid")
def set_private_attr(
model: Union[PydanticBaseModel, PydanticModelNestedValueMixin], key: str, value: Any
) -> None:

5
src/akkudoktoreos/core/version.py Normal file
View File

@@ -0,0 +1,5 @@
"""Version information for akkudoktoreos."""
# For development add `+dev` to previous release
# For release omit `+dev`.
__version__ = "0.1.0+dev"

3
src/akkudoktoreos/data/default.config.json
View File

@@ -1,2 +1,5 @@
{
"general": {
"version": "0.1.0+dev"
}
}

405
src/akkudoktoreos/devices/devices.py
View File

@@ -1,49 +1,376 @@
from typing import Optional
"""General configuration settings for simulated devices for optimization."""
from akkudoktoreos.core.coreabc import SingletonMixin
from akkudoktoreos.devices.battery import Battery
from akkudoktoreos.devices.devicesabc import DevicesBase
from akkudoktoreos.devices.generic import HomeAppliance
from akkudoktoreos.devices.inverter import Inverter
from akkudoktoreos.devices.settings import DevicesCommonSettings
import json
from typing import Any, Optional, TextIO, cast
from loguru import logger
from pydantic import Field, computed_field, model_validator
from akkudoktoreos.config.configabc import SettingsBaseModel
from akkudoktoreos.core.cache import CacheFileStore
from akkudoktoreos.core.coreabc import ConfigMixin, SingletonMixin
from akkudoktoreos.core.emplan import ResourceStatus
from akkudoktoreos.core.pydantic import ConfigDict, PydanticBaseModel
from akkudoktoreos.devices.devicesabc import DevicesBaseSettings
from akkudoktoreos.utils.datetimeutil import DateTime, TimeWindowSequence, to_datetime
class Devices(SingletonMixin, DevicesBase):
def __init__(self, settings: Optional[DevicesCommonSettings] = None):
if hasattr(self, "_initialized"):
return
super().__init__()
if settings is None:
settings = self.config.devices
if settings is None:
return
class BatteriesCommonSettings(DevicesBaseSettings):
"""Battery devices base settings."""
# initialize devices
if settings.batteries is not None:
for battery_params in settings.batteries:
self.add_device(Battery(battery_params))
if settings.inverters is not None:
for inverter_params in settings.inverters:
self.add_device(Inverter(inverter_params))
if settings.home_appliances is not None:
for home_appliance_params in settings.home_appliances:
self.add_device(HomeAppliance(home_appliance_params))
capacity_wh: int = Field(
default=8000,
gt=0,
description="Capacity [Wh].",
examples=[8000],
)
self.post_setup()
charging_efficiency: float = Field(
default=0.88,
gt=0,
le=1,
description="Charging efficiency [0.01 ... 1.00].",
examples=[0.88],
)
def post_setup(self) -> None:
for device in self.devices.values():
device.post_setup()
discharging_efficiency: float = Field(
default=0.88,
gt=0,
le=1,
description="Discharge efficiency [0.01 ... 1.00].",
examples=[0.88],
)
levelized_cost_of_storage_kwh: float = Field(
default=0.0,
description="Levelized cost of storage (LCOS), the average lifetime cost of delivering one kWh [€/kWh].",
examples=[0.12],
)
max_charge_power_w: Optional[float] = Field(
default=5000,
gt=0,
description="Maximum charging power [W].",
examples=[5000],
)
min_charge_power_w: Optional[float] = Field(
default=50,
gt=0,
description="Minimum charging power [W].",
examples=[50],
)
charge_rates: Optional[list[float]] = Field(
default=None,
description="Charge rates as factor of maximum charging power [0.00 ... 1.00]. None denotes all charge rates are available.",
examples=[[0.0, 0.25, 0.5, 0.75, 1.0], None],
)
min_soc_percentage: int = Field(
default=0,
ge=0,
le=100,
description="Minimum state of charge (SOC) as percentage of capacity [%].",
examples=[10],
)
max_soc_percentage: int = Field(
default=100,
ge=0,
le=100,
description="Maximum state of charge (SOC) as percentage of capacity [%].",
examples=[100],
)
@computed_field # type: ignore[prop-decorator]
@property
def measurement_key_soc_factor(self) -> str:
"""Measurement key for the battery state of charge (SoC) as factor of total capacity [0.0 ... 1.0]."""
return f"{self.device_id}-soc-factor"
@computed_field # type: ignore[prop-decorator]
@property
def measurement_key_power_l1_w(self) -> str:
"""Measurement key for the L1 power the battery is charged or discharged with [W]."""
return f"{self.device_id}-power-l1-w"
@computed_field # type: ignore[prop-decorator]
@property
def measurement_key_power_l2_w(self) -> str:
"""Measurement key for the L2 power the battery is charged or discharged with [W]."""
return f"{self.device_id}-power-l2-w"
@computed_field # type: ignore[prop-decorator]
@property
def measurement_key_power_l3_w(self) -> str:
"""Measurement key for the L3 power the battery is charged or discharged with [W]."""
return f"{self.device_id}-power-l3-w"
@computed_field # type: ignore[prop-decorator]
@property
def measurement_key_power_3_phase_sym_w(self) -> str:
"""Measurement key for the symmetric 3 phase power the battery is charged or discharged with [W]."""
return f"{self.device_id}-power-3-phase-sym-w"
@computed_field # type: ignore[prop-decorator]
@property
def measurement_keys(self) -> Optional[list[str]]:
"""Measurement keys for the battery stati that are measurements.
Battery SoC, power.
"""
keys: list[str] = [
self.measurement_key_soc_factor,
self.measurement_key_power_l1_w,
self.measurement_key_power_l2_w,
self.measurement_key_power_l3_w,
self.measurement_key_power_3_phase_sym_w,
]
return keys
# Initialize the Devices simulation, it is a singleton.
devices: Optional[Devices] = None
class InverterCommonSettings(DevicesBaseSettings):
"""Inverter devices base settings."""
max_power_w: Optional[float] = Field(
default=None,
gt=0,
description="Maximum power [W].",
examples=[10000],
)
battery_id: Optional[str] = Field(
default=None,
description="ID of battery controlled by this inverter.",
examples=[None, "battery1"],
)
@computed_field # type: ignore[prop-decorator]
@property
def measurement_keys(self) -> Optional[list[str]]:
"""Measurement keys for the inverter stati that are measurements."""
keys: list[str] = []
return keys
def get_devices() -> Devices:
global devices
# Fix circular import at runtime
if devices is None:
devices = Devices()
"""Gets the EOS Devices simulation."""
return devices
class HomeApplianceCommonSettings(DevicesBaseSettings):
"""Home Appliance devices base settings."""
consumption_wh: int = Field(
gt=0,
description="Energy consumption [Wh].",
examples=[2000],
)
duration_h: int = Field(
gt=0,
le=24,
description="Usage duration in hours [0 ... 24].",
examples=[1],
)
time_windows: Optional[TimeWindowSequence] = Field(
default=None,
description="Sequence of allowed time windows. Defaults to optimization general time window.",
examples=[
{
"windows": [
{"start_time": "10:00", "duration": "2 hours"},
],
},
],
)
@computed_field # type: ignore[prop-decorator]
@property
def measurement_keys(self) -> Optional[list[str]]:
"""Measurement keys for the home appliance stati that are measurements."""
keys: list[str] = []
return keys
class DevicesCommonSettings(SettingsBaseModel):
"""Base configuration for devices simulation settings."""
batteries: Optional[list[BatteriesCommonSettings]] = Field(
default=None,
description="List of battery devices",
examples=[[{"device_id": "battery1", "capacity_wh": 8000}]],
)
max_batteries: Optional[int] = Field(
default=None,
ge=0,
description="Maximum number of batteries that can be set",
examples=[1, 2],
)
electric_vehicles: Optional[list[BatteriesCommonSettings]] = Field(
default=None,
description="List of electric vehicle devices",
examples=[[{"device_id": "battery1", "capacity_wh": 8000}]],
)
max_electric_vehicles: Optional[int] = Field(
default=None,
ge=0,
description="Maximum number of electric vehicles that can be set",
examples=[1, 2],
)
inverters: Optional[list[InverterCommonSettings]] = Field(
default=None, description="List of inverters", examples=[[]]
)
max_inverters: Optional[int] = Field(
default=None,
ge=0,
description="Maximum number of inverters that can be set",
examples=[1, 2],
)
home_appliances: Optional[list[HomeApplianceCommonSettings]] = Field(
default=None, description="List of home appliances", examples=[[]]
)
max_home_appliances: Optional[int] = Field(
default=None,
ge=0,
description="Maximum number of home_appliances that can be set",
examples=[1, 2],
)
@computed_field # type: ignore[prop-decorator]
@property
def measurement_keys(self) -> Optional[list[str]]:
"""Return the measurement keys for the resource/ device stati that are measurements."""
keys: list[str] = []
if self.max_batteries and self.batteries:
for battery in self.batteries:
keys.extend(battery.measurement_keys)
if self.max_electric_vehicles and self.electric_vehicles:
for electric_vehicle in self.electric_vehicles:
keys.extend(electric_vehicle.measurement_keys)
return keys
# Type used for indexing: (resource_id, optional actuator_id)
class ResourceKey(PydanticBaseModel):
"""Key identifying a resource and optionally an actuator."""
resource_id: str
actuator_id: Optional[str] = None
model_config = ConfigDict(frozen=True)
def __hash__(self) -> int:
"""Returns a stable hash based on the resource_id and actuator_id.
Returns:
int: Hash value derived from the resource_id and actuator_id.
"""
return hash(self.resource_id + self.actuator_id if self.actuator_id else "")
def as_tuple(self) -> tuple[str, Optional[str]]:
"""Return the key as a tuple for internal dictionary indexing."""
return (self.resource_id, self.actuator_id)
def __eq__(self, other: Any) -> bool:
if not isinstance(other, ResourceKey):
return NotImplemented
return self.resource_id == other.resource_id and self.actuator_id == other.actuator_id
class ResourceRegistry(SingletonMixin, ConfigMixin, PydanticBaseModel):
"""Registry for collecting and retrieving device status reports for simulations.
Maintains the latest and optionally historical status reports for each resource.
"""
keep_history: bool = False
history_size: int = 100
latest: dict[ResourceKey, ResourceStatus] = Field(
default_factory=dict,
description="Latest resource status that was reported per resource key.",
example=[],
)
history: dict[ResourceKey, list[tuple[DateTime, ResourceStatus]]] = Field(
default_factory=dict,
description="History of resource stati that were reported per resource key.",
example=[],
)
@model_validator(mode="after")
def _enforce_history_limits(self) -> "ResourceRegistry":
"""Ensure history list lengths respect the history_size limit."""
if self.keep_history:
for key, records in self.history.items():
if len(records) > self.history_size:
self.history[key] = records[-self.history_size :]
return self
def update_status(self, key: ResourceKey, status: ResourceStatus) -> None:
"""Update the latest status and optionally store in history.
Args:
key (ResourceKey): Identifier for the resource.
status (ResourceStatus): Status report to store.
"""
self.latest[key] = status
if self.keep_history:
timestamp = getattr(status, "transition_timestamp", None) or to_datetime()
self.history.setdefault(key, []).append((timestamp, status))
if len(self.history[key]) > self.history_size:
self.history[key] = self.history[key][-self.history_size :]
def status_latest(self, key: ResourceKey) -> Optional[ResourceStatus]:
"""Retrieve the most recent status for a resource."""
return self.latest.get(key)
def status_history(self, key: ResourceKey) -> list[tuple[DateTime, ResourceStatus]]:
"""Retrieve historical status reports for a resource."""
if not self.keep_history:
raise RuntimeError("History tracking is disabled.")
return self.history.get(key, [])
def status_exists(self, key: ResourceKey) -> bool:
"""Check if a status report exists for the given resource.
Args:
key (ResourceKey): Identifier for the resource.
"""
return key in self.latest
def save(self) -> None:
"""Save the registry to file."""
# Make explicit cast to make mypy happy
cache_file = cast(
TextIO, CacheFileStore().create(key="resource_registry", mode="w+", suffix=".json")
)
cache_file.seek(0)
cache_file.write(self.model_dump_json(indent=4))
cache_file.truncate() # Important to remove leftover data!
def load(self) -> None:
"""Load registry state from file and update the current instance."""
cache_file = CacheFileStore().get(key="resource_registry")
if cache_file:
try:
cache_file.seek(0)
data = json.load(cache_file)
loaded = self.__class__.model_validate(data)
self.keep_history = loaded.keep_history
self.history_size = loaded.history_size
self.latest = loaded.latest
self.history = loaded.history
except Exception as e:
logger.error("Can not load resource registry: {}", e)
def get_resource_registry() -> ResourceRegistry:
"""Gets the EOS resource registry."""
return ResourceRegistry()

266
src/akkudoktoreos/devices/devicesabc.py
View File

@@ -1,181 +1,131 @@
"""Abstract and base classes for devices."""
from enum import Enum
from typing import Optional, Type
from enum import StrEnum
from loguru import logger
from pendulum import DateTime
from pydantic import Field, computed_field
from pydantic import Field
from akkudoktoreos.core.coreabc import (
ConfigMixin,
DevicesMixin,
EnergyManagementSystemMixin,
PredictionMixin,
)
from akkudoktoreos.core.pydantic import ParametersBaseModel
from akkudoktoreos.utils.datetimeutil import to_duration
from akkudoktoreos.config.configabc import SettingsBaseModel
class DeviceParameters(ParametersBaseModel):
device_id: str = Field(description="ID of device", examples="device1")
hours: Optional[int] = Field(
default=None,
gt=0,
description="Number of prediction hours. Defaults to global config prediction hours.",
examples=[None],
class DevicesBaseSettings(SettingsBaseModel):
"""Base devices setting."""
device_id: str = Field(
default="<unknown>",
description="ID of device",
examples=["battery1", "ev1", "inverter1", "dishwasher"],
)
class DeviceOptimizeResult(ParametersBaseModel):
device_id: str = Field(description="ID of device", examples=["device1"])
hours: int = Field(gt=0, description="Number of hours in the simulation.", examples=[24])
class BatteryOperationMode(StrEnum):
"""Battery Operation Mode.
Enumerates the operating modes of a battery in a home energy
management simulation. These modes require no direct awareness
of electricity prices or carbon intensity — higher-level
controllers or optimizers decide when to switch modes.
class DeviceState(Enum):
UNINITIALIZED = 0
PREPARED = 1
INITIALIZED = 2
Modes
-----
- IDLE:
No charging or discharging.
- SELF_CONSUMPTION:
Charge from local surplus and discharge to meet local demand.
class DevicesStartEndMixin(ConfigMixin, EnergyManagementSystemMixin):
"""A mixin to manage start, end datetimes for devices data.
- NON_EXPORT:
Charge from on-site or local surplus with the goal of
minimizing or preventing energy export to the external grid.
Discharging to the grid is not allowed.
The starting datetime for devices data generation is provided by the energy management
system. Device data cannot be computed if this value is `None`.
- PEAK_SHAVING:
Discharge during local demand peaks to reduce grid draw.
- GRID_SUPPORT_EXPORT:
Discharge to support the upstream grid when commanded.
- GRID_SUPPORT_IMPORT:
Charge from the grid when instructed to absorb excess supply.
- FREQUENCY_REGULATION:
Perform fast bidirectional power adjustments based on grid
frequency deviations.
- RAMP_RATE_CONTROL:
Smooth changes in local net load or generation.
- RESERVE_BACKUP:
Maintain a minimum state of charge for emergency use.
- OUTAGE_SUPPLY:
Discharge to power critical loads during a grid outage.
- FORCED_CHARGE:
Override all other logic and charge regardless of conditions.
- FORCED_DISCHARGE:
Override all other logic and discharge regardless of conditions.
- FAULT:
Battery is unavailable due to fault or error state.
"""
# Computed field for end_datetime and keep_datetime
@computed_field # type: ignore[prop-decorator]
@property
def end_datetime(self) -> Optional[DateTime]:
"""Compute the end datetime based on the `start_datetime` and `hours`.
Ajusts the calculated end time if DST transitions occur within the prediction window.
Returns:
Optional[DateTime]: The calculated end datetime, or `None` if inputs are missing.
"""
if self.ems.start_datetime and self.config.prediction.hours:
end_datetime = self.ems.start_datetime + to_duration(
f"{self.config.prediction.hours} hours"
)
dst_change = end_datetime.offset_hours - self.ems.start_datetime.offset_hours
logger.debug(
f"Pre: {self.ems.start_datetime}..{end_datetime}: DST change: {dst_change}"
)
if dst_change < 0:
end_datetime = end_datetime + to_duration(f"{abs(int(dst_change))} hours")
elif dst_change > 0:
end_datetime = end_datetime - to_duration(f"{abs(int(dst_change))} hours")
logger.debug(
f"Pst: {self.ems.start_datetime}..{end_datetime}: DST change: {dst_change}"
)
return end_datetime
return None
@computed_field # type: ignore[prop-decorator]
@property
def total_hours(self) -> Optional[int]:
"""Compute the hours from `start_datetime` to `end_datetime`.
Returns:
Optional[pendulum.period]: The duration hours, or `None` if either datetime is unavailable.
"""
end_dt = self.end_datetime
if end_dt is None:
return None
duration = end_dt - self.ems.start_datetime
return int(duration.total_hours())
IDLE = "IDLE"
SELF_CONSUMPTION = "SELF_CONSUMPTION"
NON_EXPORT = "NON_EXPORT"
PEAK_SHAVING = "PEAK_SHAVING"
GRID_SUPPORT_EXPORT = "GRID_SUPPORT_EXPORT"
GRID_SUPPORT_IMPORT = "GRID_SUPPORT_IMPORT"
FREQUENCY_REGULATION = "FREQUENCY_REGULATION"
RAMP_RATE_CONTROL = "RAMP_RATE_CONTROL"
RESERVE_BACKUP = "RESERVE_BACKUP"
OUTAGE_SUPPLY = "OUTAGE_SUPPLY"
FORCED_CHARGE = "FORCED_CHARGE"
FORCED_DISCHARGE = "FORCED_DISCHARGE"
FAULT = "FAULT"
class DeviceBase(DevicesStartEndMixin, PredictionMixin, DevicesMixin):
"""Base class for device simulations.
class ApplianceOperationMode(StrEnum):
"""Appliance operation modes.
Enables access to EOS configuration data (attribute `config`), EOS prediction data (attribute
`prediction`) and EOS device registry (attribute `devices`).
Modes
-----
- OFF:
Stop or prevent any active operation of the appliance.
Behavior:
- Several initialization phases (setup, post_setup):
- setup: Initialize class attributes from DeviceParameters (pydantic input validation)
- post_setup: Set connections between devices
- NotImplemented:
- hooks during optimization
- RUN:
Start or continue normal operation of the appliance.
Notes:
- This class is base to concrete devices like battery, inverter, etc. that are used in optimization.
- Not a pydantic model for a low footprint during optimization.
- DEFER:
Postpone operation to a later time window based on
scheduling or optimization criteria.
- PAUSE:
Temporarily suspend an ongoing operation, keeping the
option to resume later.
- RESUME:
Continue an operation that was previously paused or
deferred.
- LIMIT_POWER:
Run the appliance under reduced power constraints,
for example in response to load-management or
demand-response signals.
- FORCED_RUN:
Start or maintain operation even if constraints or
optimization strategies would otherwise delay or limit it.
- FAULT:
Appliance is unavailable due to fault or error state.
"""
def __init__(self, parameters: Optional[DeviceParameters] = None):
self.device_id: str = "<invalid>"
self.parameters: Optional[DeviceParameters] = None
self.hours = -1
if self.total_hours is not None:
self.hours = self.total_hours
self.initialized = DeviceState.UNINITIALIZED
if parameters is not None:
self.setup(parameters)
def setup(self, parameters: DeviceParameters) -> None:
if self.initialized != DeviceState.UNINITIALIZED:
return
self.parameters = parameters
self.device_id = self.parameters.device_id
if self.parameters.hours is not None:
self.hours = self.parameters.hours
if self.hours < 0:
raise ValueError("hours is unset")
self._setup()
self.initialized = DeviceState.PREPARED
def post_setup(self) -> None:
if self.initialized.value >= DeviceState.INITIALIZED.value:
return
self._post_setup()
self.initialized = DeviceState.INITIALIZED
def _setup(self) -> None:
"""Implement custom setup in derived device classes."""
pass
def _post_setup(self) -> None:
"""Implement custom setup in derived device classes that is run when all devices are initialized."""
pass
class DevicesBase(DevicesStartEndMixin, PredictionMixin):
"""Base class for handling device data.
Enables access to EOS configuration data (attribute `config`) and EOS prediction data (attribute
`prediction`).
"""
def __init__(self) -> None:
super().__init__()
self.devices: dict[str, "DeviceBase"] = dict()
def get_device_by_id(self, device_id: str) -> Optional["DeviceBase"]:
return self.devices.get(device_id)
def add_device(self, device: Optional["DeviceBase"]) -> None:
if device is None:
return
if device.device_id in self.devices:
raise ValueError(f"{device.device_id} already registered")
self.devices[device.device_id] = device
def remove_device(self, device: Type["DeviceBase"] | str) -> bool:
if isinstance(device, DeviceBase):
device = device.device_id
return self.devices.pop(device, None) is not None # type: ignore[arg-type]
def reset(self) -> None:
self.devices = dict()
OFF = "OFF"
RUN = "RUN"
DEFER = "DEFER"
PAUSE = "PAUSE"
RESUME = "RESUME"
LIMIT_POWER = "LIMIT_POWER"
FORCED_RUN = "FORCED_RUN"
FAULT = "FAULT"

79
src/akkudoktoreos/devices/generic.py
View File

@@ -1,79 +0,0 @@
from typing import Optional
import numpy as np
from pydantic import Field
from akkudoktoreos.devices.devicesabc import DeviceBase, DeviceParameters
class HomeApplianceParameters(DeviceParameters):
"""Home Appliance Device Simulation Configuration."""
device_id: str = Field(description="ID of home appliance", examples=["dishwasher"])
consumption_wh: int = Field(
gt=0,
description="An integer representing the energy consumption of a household device in watt-hours.",
examples=[2000],
)
duration_h: int = Field(
gt=0,
description="An integer representing the usage duration of a household device in hours.",
examples=[3],
)
class HomeAppliance(DeviceBase):
def __init__(
self,
parameters: Optional[HomeApplianceParameters] = None,
):
self.parameters: Optional[HomeApplianceParameters] = None
super().__init__(parameters)
def _setup(self) -> None:
if self.parameters is None:
raise ValueError(f"Parameters not set: {self.parameters}")
self.load_curve = np.zeros(self.hours) # Initialize the load curve with zeros
self.duration_h = self.parameters.duration_h
self.consumption_wh = self.parameters.consumption_wh
def set_starting_time(self, start_hour: int, global_start_hour: int = 0) -> None:
"""Sets the start time of the device and generates the corresponding load curve.
:param start_hour: The hour at which the device should start.
"""
self.reset_load_curve()
# Check if the duration of use is within the available time frame
if start_hour + self.duration_h > self.hours:
raise ValueError("The duration of use exceeds the available time frame.")
if start_hour < global_start_hour:
raise ValueError("The start time is earlier than the available time frame.")
# Calculate power per hour based on total consumption and duration
power_per_hour = self.consumption_wh / self.duration_h # Convert to watt-hours
# Set the power for the duration of use in the load curve array
self.load_curve[start_hour : start_hour + self.duration_h] = power_per_hour
def reset_load_curve(self) -> None:
"""Resets the load curve."""
self.load_curve = np.zeros(self.hours)
def get_load_curve(self) -> np.ndarray:
"""Returns the current load curve."""
return self.load_curve
def get_load_for_hour(self, hour: int) -> float:
"""Returns the load for a specific hour.
:param hour: The hour for which the load is queried.
:return: The load in watts for the specified hour.
"""
if hour < 0 or hour >= self.hours:
raise ValueError("The specified hour is outside the available time frame.")
return self.load_curve[hour]
def get_latest_starting_point(self) -> int:
"""Returns the latest possible start time at which the device can still run completely."""
return self.hours - self.duration_h

0
src/akkudoktoreos/devices/genetic/__init__.py Normal file
View File

148
src/akkudoktoreos/devices/battery.py → src/akkudoktoreos/devices/genetic/battery.py
View File

@@ -1,125 +1,23 @@
from typing import Any, Optional
import numpy as np
from pydantic import Field, field_validator
from akkudoktoreos.devices.devicesabc import (
DeviceBase,
DeviceOptimizeResult,
DeviceParameters,
from akkudoktoreos.optimization.genetic.geneticdevices import (
BaseBatteryParameters,
SolarPanelBatteryParameters,
)
from akkudoktoreos.utils.utils import NumpyEncoder
def max_charging_power_field(description: Optional[str] = None) -> float:
if description is None:
description = "Maximum charging power in watts."
return Field(
default=5000,
gt=0,
description=description,
)
def initial_soc_percentage_field(description: str) -> int:
return Field(default=0, ge=0, le=100, description=description, examples=[42])
def discharging_efficiency_field(default_value: float) -> float:
return Field(
default=default_value,
gt=0,
le=1,
description="A float representing the discharge efficiency of the battery.",
)
class BaseBatteryParameters(DeviceParameters):
"""Battery Device Simulation Configuration."""
device_id: str = Field(description="ID of battery", examples=["battery1"])
capacity_wh: int = Field(
gt=0,
description="An integer representing the capacity of the battery in watt-hours.",
examples=[8000],
)
charging_efficiency: float = Field(
default=0.88,
gt=0,
le=1,
description="A float representing the charging efficiency of the battery.",
)
discharging_efficiency: float = discharging_efficiency_field(0.88)
max_charge_power_w: Optional[float] = max_charging_power_field()
initial_soc_percentage: int = initial_soc_percentage_field(
"An integer representing the state of charge of the battery at the **start** of the current hour (not the current state)."
)
min_soc_percentage: int = Field(
default=0,
ge=0,
le=100,
description="An integer representing the minimum state of charge (SOC) of the battery in percentage.",
examples=[10],
)
max_soc_percentage: int = Field(
default=100,
ge=0,
le=100,
description="An integer representing the maximum state of charge (SOC) of the battery in percentage.",
)
class SolarPanelBatteryParameters(BaseBatteryParameters):
max_charge_power_w: Optional[float] = max_charging_power_field()
class ElectricVehicleParameters(BaseBatteryParameters):
"""Battery Electric Vehicle Device Simulation Configuration."""
device_id: str = Field(description="ID of electric vehicle", examples=["ev1"])
discharging_efficiency: float = discharging_efficiency_field(1.0)
initial_soc_percentage: int = initial_soc_percentage_field(
"An integer representing the current state of charge (SOC) of the battery in percentage."
)
class ElectricVehicleResult(DeviceOptimizeResult):
"""Result class containing information related to the electric vehicle's charging and discharging behavior."""
device_id: str = Field(description="ID of electric vehicle", examples=["ev1"])
charge_array: list[float] = Field(
description="Hourly charging status (0 for no charging, 1 for charging)."
)
discharge_array: list[int] = Field(
description="Hourly discharging status (0 for no discharging, 1 for discharging)."
)
discharging_efficiency: float = Field(description="The discharge efficiency as a float..")
capacity_wh: int = Field(description="Capacity of the EV’s battery in watt-hours.")
charging_efficiency: float = Field(description="Charging efficiency as a float..")
max_charge_power_w: int = Field(description="Maximum charging power in watts.")
soc_wh: float = Field(
description="State of charge of the battery in watt-hours at the start of the simulation."
)
initial_soc_percentage: int = Field(
description="State of charge at the start of the simulation in percentage."
)
@field_validator("discharge_array", "charge_array", mode="before")
def convert_numpy(cls, field: Any) -> Any:
return NumpyEncoder.convert_numpy(field)[0]
class Battery(DeviceBase):
class Battery:
"""Represents a battery device with methods to simulate energy charging and discharging."""
def __init__(self, parameters: Optional[BaseBatteryParameters] = None):
self.parameters: Optional[BaseBatteryParameters] = None
super().__init__(parameters)
def __init__(self, parameters: BaseBatteryParameters, prediction_hours: int):
self.parameters = parameters
self.prediction_hours = prediction_hours
self._setup()
def _setup(self) -> None:
"""Sets up the battery parameters based on configuration or provided parameters."""
if self.parameters is None:
raise ValueError(f"Parameters not set: {self.parameters}")
self.capacity_wh = self.parameters.capacity_wh
self.initial_soc_percentage = self.parameters.initial_soc_percentage
self.charging_efficiency = self.parameters.charging_efficiency
@@ -138,8 +36,8 @@ class Battery(DeviceBase):
self.max_charge_power_w = self.parameters.max_charge_power_w
else:
self.max_charge_power_w = self.capacity_wh # TODO this should not be equal capacity_wh
self.discharge_array = np.full(self.hours, 1)
self.charge_array = np.full(self.hours, 1)
self.discharge_array = np.full(self.prediction_hours, 1)
self.charge_array = np.full(self.prediction_hours, 1)
self.soc_wh = (self.initial_soc_percentage / 100) * self.capacity_wh
self.min_soc_wh = (self.min_soc_percentage / 100) * self.capacity_wh
self.max_soc_wh = (self.max_soc_percentage / 100) * self.capacity_wh
@@ -147,11 +45,11 @@ class Battery(DeviceBase):
def to_dict(self) -> dict[str, Any]:
"""Converts the object to a dictionary representation."""
return {
"device_id": self.device_id,
"device_id": self.parameters.device_id,
"capacity_wh": self.capacity_wh,
"initial_soc_percentage": self.initial_soc_percentage,
"soc_wh": self.soc_wh,
"hours": self.hours,
"hours": self.prediction_hours,
"discharge_array": self.discharge_array,
"charge_array": self.charge_array,
"charging_efficiency": self.charging_efficiency,
@@ -163,25 +61,31 @@ class Battery(DeviceBase):
"""Resets the battery state to its initial values."""
self.soc_wh = (self.initial_soc_percentage / 100) * self.capacity_wh
self.soc_wh = min(max(self.soc_wh, self.min_soc_wh), self.max_soc_wh)
self.discharge_array = np.full(self.hours, 1)
self.charge_array = np.full(self.hours, 1)
self.discharge_array = np.full(self.prediction_hours, 1)
self.charge_array = np.full(self.prediction_hours, 1)
def set_discharge_per_hour(self, discharge_array: np.ndarray) -> None:
"""Sets the discharge values for each hour."""
if len(discharge_array) != self.hours:
raise ValueError(f"Discharge array must have exactly {self.hours} elements.")
if len(discharge_array) != self.prediction_hours:
raise ValueError(
f"Discharge array must have exactly {self.prediction_hours} elements. Got {len(discharge_array)} elements."
)
self.discharge_array = np.array(discharge_array)
def set_charge_per_hour(self, charge_array: np.ndarray) -> None:
"""Sets the charge values for each hour."""
if len(charge_array) != self.hours:
raise ValueError(f"Charge array must have exactly {self.hours} elements.")
if len(charge_array) != self.prediction_hours:
raise ValueError(
f"Charge array must have exactly {self.prediction_hours} elements. Got {len(charge_array)} elements."
)
self.charge_array = np.array(charge_array)
def set_charge_allowed_for_hour(self, charge: float, hour: int) -> None:
"""Sets the charge for a specific hour."""
if hour >= self.hours:
raise ValueError(f"Hour {hour} is out of range. Must be less than {self.hours}.")
if hour >= self.prediction_hours:
raise ValueError(
f"Hour {hour} is out of range. Must be less than {self.prediction_hours}."
)
self.charge_array[hour] = charge
def current_soc_percentage(self) -> float:

0
src/akkudoktoreos/devices/heatpump.py → src/akkudoktoreos/devices/genetic/heatpump.py
View File

112
src/akkudoktoreos/devices/genetic/homeappliance.py Normal file
View File

@@ -0,0 +1,112 @@
from typing import Optional
import numpy as np
from akkudoktoreos.optimization.genetic.geneticdevices import HomeApplianceParameters
from akkudoktoreos.utils.datetimeutil import (
TimeWindow,
TimeWindowSequence,
to_datetime,
to_duration,
to_time,
)
class HomeAppliance:
def __init__(
self,
parameters: HomeApplianceParameters,
optimization_hours: int,
prediction_hours: int,
):
self.parameters: HomeApplianceParameters = parameters
self.prediction_hours = prediction_hours
self._setup()
def _setup(self) -> None:
"""Sets up the home appliance parameters based provided parameters."""
self.load_curve = np.zeros(self.prediction_hours) # Initialize the load curve with zeros
self.duration_h = self.parameters.duration_h
self.consumption_wh = self.parameters.consumption_wh
self.appliance_start: Optional[int] = None
# setup possible start times
if self.parameters.time_windows is None:
self.parameters.time_windows = TimeWindowSequence(
windows=[
TimeWindow(
start_time=to_time("00:00"),
duration=to_duration(f"{self.prediction_hours} hours"),
),
]
)
start_datetime = to_datetime().set(hour=0, minute=0, second=0)
duration = to_duration(f"{self.duration_h} hours")
self.start_allowed: list[bool] = []
for hour in range(0, self.prediction_hours):
self.start_allowed.append(
self.parameters.time_windows.contains(
start_datetime.add(hours=hour), duration=duration
)
)
start_earliest = self.parameters.time_windows.earliest_start_time(duration, start_datetime)
if start_earliest:
self.start_earliest = start_earliest.hour
else:
self.start_earliest = 0
start_latest = self.parameters.time_windows.latest_start_time(duration, start_datetime)
if start_latest:
self.start_latest = start_latest.hour
else:
self.start_latest = 23
def set_starting_time(self, start_hour: int, global_start_hour: int = 0) -> None:
"""Sets the start time of the device and generates the corresponding load curve.
:param start_hour: The hour at which the device should start.
"""
self.reset_load_curve()
# Check if the duration of use is within the available time windows
if not self.start_allowed[start_hour]:
# No available time window to start home appliance
# Use the earliest one
start_hour = self.start_earliest
# Check if it is possibility to start the appliance
if start_hour < global_start_hour:
# Start is before current time
# Use the latest one
start_hour = self.start_latest
# Calculate power per hour based on total consumption and duration
power_per_hour = self.consumption_wh / self.duration_h # Convert to watt-hours
# Set the power for the duration of use in the load curve array
self.load_curve[start_hour : start_hour + self.duration_h] = power_per_hour
# Set the selected start hour
self.appliance_start = start_hour
def reset_load_curve(self) -> None:
"""Resets the load curve."""
self.load_curve = np.zeros(self.prediction_hours)
def get_load_curve(self) -> np.ndarray:
"""Returns the current load curve."""
return self.load_curve
def get_load_for_hour(self, hour: int) -> float:
"""Returns the load for a specific hour.
:param hour: The hour for which the load is queried.
:return: The load in watts for the specified hour.
"""
if hour < 0 or hour >= self.prediction_hours:
raise ValueError(
f"The specified hour {hour} is outside the available time frame {self.prediction_hours}."
)
return self.load_curve[hour]
def get_appliance_start(self) -> Optional[int]:
return self.appliance_start

94
src/akkudoktoreos/devices/inverter.py → src/akkudoktoreos/devices/genetic/inverter.py
View File

@@ -1,65 +1,32 @@
from typing import Optional
from loguru import logger
from pydantic import Field
from akkudoktoreos.devices.devicesabc import DeviceBase, DeviceParameters
from akkudoktoreos.devices.genetic.battery import Battery
from akkudoktoreos.optimization.genetic.geneticdevices import InverterParameters
from akkudoktoreos.prediction.interpolator import get_eos_load_interpolator
class InverterParameters(DeviceParameters):
"""Inverter Device Simulation Configuration."""
device_id: str = Field(description="ID of inverter", examples=["inverter1"])
max_power_wh: float = Field(gt=0, examples=[10000])
battery_id: Optional[str] = Field(
default=None, description="ID of battery", examples=[None, "battery1"]
)
class Inverter(DeviceBase):
class Inverter:
def __init__(
self,
parameters: Optional[InverterParameters] = None,
parameters: InverterParameters,
battery: Optional[Battery] = None,
):
self.parameters: Optional[InverterParameters] = None
super().__init__(parameters)
self.scr_lookup: dict = {}
def _calculate_scr(self, consumption: float, generation: float) -> float:
"""Check if the consumption and production is in the lookup table. If not, calculate and store the value."""
if consumption not in self.scr_lookup:
self.scr_lookup[consumption] = {}
if generation not in self.scr_lookup[consumption]:
scr = self.self_consumption_predictor.calculate_self_consumption(
consumption, generation
)
self.scr_lookup[consumption][generation] = scr
return scr
return self.scr_lookup[consumption][generation]
self.parameters: InverterParameters = parameters
self.battery: Optional[Battery] = battery
self._setup()
def _setup(self) -> None:
if self.parameters is None:
raise ValueError(f"Parameters not set: {self.parameters}")
if self.parameters.battery_id is None:
# For the moment raise exception
# TODO: Make battery configurable by config
error_msg = "Battery for PV inverter is mandatory."
if self.battery and self.parameters.battery_id != self.battery.parameters.device_id:
error_msg = f"Battery ID mismatch - {self.parameters.battery_id} is configured; got {self.battery.parameters.device_id}."
logger.error(error_msg)
raise NotImplementedError(error_msg)
raise ValueError(error_msg)
self.self_consumption_predictor = get_eos_load_interpolator()
self.max_power_wh = (
self.parameters.max_power_wh
) # Maximum power that the inverter can handle
def _post_setup(self) -> None:
if self.parameters is None:
raise ValueError(f"Parameters not set: {self.parameters}")
self.battery = self.devices.get_device_by_id(self.parameters.battery_id)
def process_energy(
self, generation: float, consumption: float, hour: int
) -> tuple[float, float, float, float]:
@@ -76,8 +43,10 @@ class Inverter(DeviceBase):
grid_import = -remaining_power # Negative indicates feeding into the grid
self_consumption = self.max_power_wh
else:
# Calculate scr with lookup table
scr = self._calculate_scr(consumption, generation)
# Calculate scr using cached results per energy management/optimization run
scr = self.self_consumption_predictor.calculate_self_consumption(
consumption, generation
)
# Remaining power after consumption
remaining_power = (generation - consumption) * scr # EVQ
@@ -86,11 +55,12 @@ class Inverter(DeviceBase):
if remaining_load_evq > 0:
# Akku muss den Restverbrauch decken
from_battery, discharge_losses = self.battery.discharge_energy(
remaining_load_evq, hour
)
remaining_load_evq -= from_battery # Restverbrauch nach Akkuentladung
losses += discharge_losses
if self.battery:
from_battery, discharge_losses = self.battery.discharge_energy(
remaining_load_evq, hour
)
remaining_load_evq -= from_battery # Restverbrauch nach Akkuentladung
losses += discharge_losses
# Wenn der Akku den Restverbrauch nicht vollständig decken kann, wird der Rest ins Netz gezogen
if remaining_load_evq > 0:
@@ -101,10 +71,13 @@ class Inverter(DeviceBase):
if remaining_power > 0:
# Load battery with excess energy
charged_energie, charge_losses = self.battery.charge_energy(
remaining_power, hour
)
remaining_surplus = remaining_power - (charged_energie + charge_losses)
if self.battery:
charged_energie, charge_losses = self.battery.charge_energy(
remaining_power, hour
)
remaining_surplus = remaining_power - (charged_energie + charge_losses)
else:
remaining_surplus = remaining_power
# Feed-in to the grid based on remaining capacity
if remaining_surplus > self.max_power_wh - consumption:
@@ -124,10 +97,13 @@ class Inverter(DeviceBase):
available_ac_power = max(self.max_power_wh - generation, 0)
# Discharge battery to cover shortfall, if possible
battery_discharge, discharge_losses = self.battery.discharge_energy(
min(shortfall, available_ac_power), hour
)
losses += discharge_losses
if self.battery:
battery_discharge, discharge_losses = self.battery.discharge_energy(
min(shortfall, available_ac_power), hour
)
losses += discharge_losses
else:
battery_discharge = 0
# Draw remaining required power from the grid (discharge_losses are already substraved in the battery)
grid_import = shortfall - battery_discharge

24
src/akkudoktoreos/devices/settings.py
View File

@@ -1,24 +0,0 @@
from typing import Optional
from pydantic import Field
from akkudoktoreos.config.configabc import SettingsBaseModel
from akkudoktoreos.devices.battery import BaseBatteryParameters
from akkudoktoreos.devices.generic import HomeApplianceParameters
from akkudoktoreos.devices.inverter import InverterParameters
class DevicesCommonSettings(SettingsBaseModel):
"""Base configuration for devices simulation settings."""
batteries: Optional[list[BaseBatteryParameters]] = Field(
default=None,
description="List of battery/ev devices",
examples=[[{"device_id": "battery1", "capacity_wh": 8000}]],
)
inverters: Optional[list[InverterParameters]] = Field(
default=None, description="List of inverters", examples=[[]]
)
home_appliances: Optional[list[HomeApplianceParameters]] = Field(
default=None, description="List of home appliances", examples=[[]]
)

164
src/akkudoktoreos/measurement/measurement.py
View File

@@ -6,90 +6,69 @@ data records for measurements.
The measurements can be added programmatically or imported from a file or JSON string.
"""
from typing import Any, ClassVar, List, Optional
from typing import Any, Optional
import numpy as np
from loguru import logger
from numpydantic import NDArray, Shape
from pendulum import DateTime, Duration
from pydantic import Field, computed_field
from akkudoktoreos.config.configabc import SettingsBaseModel
from akkudoktoreos.core.coreabc import SingletonMixin
from akkudoktoreos.core.dataabc import DataImportMixin, DataRecord, DataSequence
from akkudoktoreos.utils.datetimeutil import to_duration
from akkudoktoreos.utils.datetimeutil import DateTime, Duration, to_duration
class MeasurementCommonSettings(SettingsBaseModel):
"""Measurement Configuration."""
load0_name: Optional[str] = Field(
default=None, description="Name of the load0 source", examples=["Household", "Heat Pump"]
load_emr_keys: Optional[list[str]] = Field(
default=None,
description="The keys of the measurements that are energy meter readings of a load [kWh].",
examples=[["load0_emr"]],
)
load1_name: Optional[str] = Field(
default=None, description="Name of the load1 source", examples=[None]
grid_export_emr_keys: Optional[list[str]] = Field(
default=None,
description="The keys of the measurements that are energy meter readings of energy export to grid [kWh].",
examples=[["grid_export_emr"]],
)
load2_name: Optional[str] = Field(
default=None, description="Name of the load2 source", examples=[None]
grid_import_emr_keys: Optional[list[str]] = Field(
default=None,
description="The keys of the measurements that are energy meter readings of energy import from grid [kWh].",
examples=[["grid_import_emr"]],
)
load3_name: Optional[str] = Field(
default=None, description="Name of the load3 source", examples=[None]
)
load4_name: Optional[str] = Field(
default=None, description="Name of the load4 source", examples=[None]
pv_production_emr_keys: Optional[list[str]] = Field(
default=None,
description="The keys of the measurements that are PV production energy meter readings [kWh].",
examples=[["pv1_emr"]],
)
## Computed fields
@computed_field # type: ignore[prop-decorator]
@property
def keys(self) -> list[str]:
"""The keys of the measurements that can be stored."""
key_list = []
for key in self.__class__.model_fields.keys():
if key.endswith("_keys") and (value := getattr(self, key)):
key_list.extend(value)
return sorted(set(key_list))
class MeasurementDataRecord(DataRecord):
"""Represents a measurement data record containing various measurements at a specific datetime.
"""Represents a measurement data record containing various measurements at a specific datetime."""
Attributes:
date_time (Optional[DateTime]): The datetime of the record.
"""
# Single loads, to be aggregated to total load
load0_mr: Optional[float] = Field(
default=None, ge=0, description="Load0 meter reading [kWh]", examples=[40421]
)
load1_mr: Optional[float] = Field(
default=None, ge=0, description="Load1 meter reading [kWh]", examples=[None]
)
load2_mr: Optional[float] = Field(
default=None, ge=0, description="Load2 meter reading [kWh]", examples=[None]
)
load3_mr: Optional[float] = Field(
default=None, ge=0, description="Load3 meter reading [kWh]", examples=[None]
)
load4_mr: Optional[float] = Field(
default=None, ge=0, description="Load4 meter reading [kWh]", examples=[None]
)
max_loads: ClassVar[int] = 5 # Maximum number of loads that can be set
grid_export_mr: Optional[float] = Field(
default=None, ge=0, description="Export to grid meter reading [kWh]", examples=[1000]
)
grid_import_mr: Optional[float] = Field(
default=None, ge=0, description="Import from grid meter reading [kWh]", examples=[1000]
)
# Computed fields
@computed_field # type: ignore[prop-decorator]
@property
def loads(self) -> List[str]:
"""Compute a list of active loads."""
active_loads = []
# Loop through loadx
for i in range(self.max_loads):
load_attr = f"load{i}_mr"
# Check if either attribute is set and add to active loads
if getattr(self, load_attr, None):
active_loads.append(load_attr)
return active_loads
@classmethod
def configured_data_keys(cls) -> Optional[list[str]]:
"""Return the keys for the configured field like data."""
keys = cls.config.measurement.keys
# Add measurment keys that are needed/ handled by the resource/ device simulations.
if cls.config.devices.measurement_keys:
keys.extend(cls.config.devices.measurement_keys)
return keys
class Measurement(SingletonMixin, DataImportMixin, DataSequence):
@@ -98,14 +77,10 @@ class Measurement(SingletonMixin, DataImportMixin, DataSequence):
Measurements can be provided programmatically or read from JSON string or file.
"""
records: List[MeasurementDataRecord] = Field(
default_factory=list, description="List of measurement data records"
records: list[MeasurementDataRecord] = Field(
default_factory=list, description="list of measurement data records"
)
topics: ClassVar[List[str]] = [
"load",
]
def __init__(self, *args: Any, **kwargs: Any) -> None:
if hasattr(self, "_initialized"):
return
@@ -141,34 +116,6 @@ class Measurement(SingletonMixin, DataImportMixin, DataSequence):
# Return ceiling of division to include partial intervals
return int(np.ceil(diff_seconds / interval_seconds))
def name_to_key(self, name: str, topic: str) -> Optional[str]:
"""Provides measurement key for given name and topic."""
topic = topic.lower()
if topic not in self.topics:
return None
topic_keys = [
key for key in self.config.measurement.model_fields.keys() if key.startswith(topic)
]
key = None
if topic == "load":
for config_key in topic_keys:
if (
config_key.endswith("_name")
and getattr(self.config.measurement, config_key) == name
):
key = topic + config_key[len(topic) : len(topic) + 1] + "_mr"
break
if key is not None and key not in self.record_keys:
# Should never happen
error_msg = f"Key '{key}' not available."
logger.error(error_msg)
raise KeyError(error_msg)
return key
def _energy_from_meter_readings(
self,
key: str,
@@ -253,17 +200,20 @@ class Measurement(SingletonMixin, DataImportMixin, DataSequence):
end_datetime = self[-1].date_time
size = self._interval_count(start_datetime, end_datetime, interval)
load_total_array = np.zeros(size)
# Loop through load<x>_mr
for i in range(self.record_class().max_loads):
key = f"load{i}_mr"
# Calculate load per interval
load_array = self._energy_from_meter_readings(
key=key, start_datetime=start_datetime, end_datetime=end_datetime, interval=interval
)
# Add calculated load to total load
load_total_array += load_array
debug_msg = f"Total load '{key}' calculation: {load_total_array}"
logger.debug(debug_msg)
# Loop through all loads
if isinstance(self.config.measurement.load_emr_keys, list):
for key in self.config.measurement.load_emr_keys:
# Calculate load per interval
load_array = self._energy_from_meter_readings(
key=key,
start_datetime=start_datetime,
end_datetime=end_datetime,
interval=interval,
)
# Add calculated load to total load
load_total_array += load_array
debug_msg = f"Total load '{key}' calculation: {load_total_array}"
logger.debug(debug_msg)
return load_total_array

545
src/akkudoktoreos/optimization/genetic.py → src/akkudoktoreos/optimization/genetic/genetic.py
View File

@@ -1,3 +1,5 @@
"""Genetic algorithm."""
import random
import time
from typing import Any, Optional
@@ -5,101 +7,302 @@ from typing import Any, Optional
import numpy as np
from deap import algorithms, base, creator, tools
from loguru import logger
from pydantic import Field, field_validator, model_validator
from typing_extensions import Self
from numpydantic import NDArray, Shape
from pydantic import ConfigDict, Field
from akkudoktoreos.core.coreabc import (
ConfigMixin,
DevicesMixin,
EnergyManagementSystemMixin,
from akkudoktoreos.core.pydantic import PydanticBaseModel
from akkudoktoreos.devices.genetic.battery import Battery
from akkudoktoreos.devices.genetic.homeappliance import HomeAppliance
from akkudoktoreos.devices.genetic.inverter import Inverter
from akkudoktoreos.optimization.genetic.geneticparams import (
GeneticEnergyManagementParameters,
GeneticOptimizationParameters,
)
from akkudoktoreos.core.ems import EnergyManagementParameters, SimulationResult
from akkudoktoreos.core.pydantic import ParametersBaseModel
from akkudoktoreos.devices.battery import (
Battery,
ElectricVehicleParameters,
ElectricVehicleResult,
SolarPanelBatteryParameters,
from akkudoktoreos.optimization.genetic.geneticsolution import (
GeneticSimulationResult,
GeneticSolution,
)
from akkudoktoreos.devices.generic import HomeAppliance, HomeApplianceParameters
from akkudoktoreos.devices.inverter import Inverter, InverterParameters
from akkudoktoreos.utils.utils import NumpyEncoder
from akkudoktoreos.optimization.optimizationabc import OptimizationBase
class OptimizationParameters(ParametersBaseModel):
ems: EnergyManagementParameters
pv_akku: Optional[SolarPanelBatteryParameters]
inverter: Optional[InverterParameters]
eauto: Optional[ElectricVehicleParameters]
dishwasher: Optional[HomeApplianceParameters] = None
temperature_forecast: Optional[list[Optional[float]]] = Field(
class GeneticSimulation(PydanticBaseModel):
"""Device simulation for GENETIC optimization algorithm."""
# Disable validation on assignment to speed up simulation runs.
model_config = ConfigDict(
validate_assignment=False,
)
start_hour: int = Field(
default=0, ge=0, le=23, description="Starting hour on day for optimizations."
)
optimization_hours: Optional[int] = Field(
default=24, ge=0, description="Number of hours into the future for optimizations."
)
prediction_hours: Optional[int] = Field(
default=48, ge=0, description="Number of hours into the future for predictions"
)
load_energy_array: Optional[NDArray[Shape["*"], float]] = Field(
default=None,
description="An array of floats representing the temperature forecast in degrees Celsius for different time intervals.",
description="An array of floats representing the total load (consumption) in watts for different time intervals.",
)
start_solution: Optional[list[float]] = Field(
default=None, description="Can be `null` or contain a previous solution (if available)."
)
@model_validator(mode="after")
def validate_list_length(self) -> Self:
arr_length = len(self.ems.pv_prognose_wh)
if self.temperature_forecast is not None and arr_length != len(self.temperature_forecast):
raise ValueError("Input lists have different lengths")
return self
@field_validator("start_solution")
def validate_start_solution(
cls, start_solution: Optional[list[float]]
) -> Optional[list[float]]:
if start_solution is not None and len(start_solution) < 2:
raise ValueError("Requires at least two values.")
return start_solution
class OptimizeResponse(ParametersBaseModel):
"""**Note**: The first value of "Last_Wh_per_hour", "Netzeinspeisung_Wh_per_hour", and "Netzbezug_Wh_per_hour", will be set to null in the JSON output and represented as NaN or None in the corresponding classes' data returns. This approach is adopted to ensure that the current hour's processing remains unchanged."""
ac_charge: list[float] = Field(
description="Array with AC charging values as relative power (0-1), other values set to 0."
)
dc_charge: list[float] = Field(
description="Array with DC charging values as relative power (0-1), other values set to 0."
)
discharge_allowed: list[int] = Field(
description="Array with discharge values (1 for discharge, 0 otherwise)."
)
eautocharge_hours_float: Optional[list[float]] = Field(description="TBD")
result: SimulationResult
eauto_obj: Optional[ElectricVehicleResult]
start_solution: Optional[list[float]] = Field(
pv_prediction_wh: Optional[NDArray[Shape["*"], float]] = Field(
default=None,
description="An array of binary values (0 or 1) representing a possible starting solution for the simulation.",
description="An array of floats representing the forecasted photovoltaic output in watts for different time intervals.",
)
washingstart: Optional[int] = Field(
elect_price_hourly: Optional[NDArray[Shape["*"], float]] = Field(
default=None,
description="Can be `null` or contain an object representing the start of washing (if applicable).",
description="An array of floats representing the electricity price in euros per watt-hour for different time intervals.",
)
elect_revenue_per_hour_arr: Optional[NDArray[Shape["*"], float]] = Field(
default=None,
description="An array of floats representing the feed-in compensation in euros per watt-hour.",
)
@field_validator(
"ac_charge",
"dc_charge",
"discharge_allowed",
mode="before",
)
def convert_numpy(cls, field: Any) -> Any:
return NumpyEncoder.convert_numpy(field)[0]
battery: Optional[Battery] = Field(default=None, description="TBD.")
ev: Optional[Battery] = Field(default=None, description="TBD.")
home_appliance: Optional[HomeAppliance] = Field(default=None, description="TBD.")
inverter: Optional[Inverter] = Field(default=None, description="TBD.")
@field_validator(
"eauto_obj",
mode="before",
)
def convert_eauto(cls, field: Any) -> Any:
if isinstance(field, Battery):
return ElectricVehicleResult(**field.to_dict())
return field
ac_charge_hours: Optional[NDArray[Shape["*"], float]] = Field(default=None, description="TBD")
dc_charge_hours: Optional[NDArray[Shape["*"], float]] = Field(default=None, description="TBD")
ev_charge_hours: Optional[NDArray[Shape["*"], float]] = Field(default=None, description="TBD")
def prepare(
self,
parameters: GeneticEnergyManagementParameters,
optimization_hours: int,
prediction_hours: int,
ev: Optional[Battery] = None,
home_appliance: Optional[HomeAppliance] = None,
inverter: Optional[Inverter] = None,
) -> None:
self.optimization_hours = optimization_hours
self.prediction_hours = prediction_hours
self.load_energy_array = np.array(parameters.gesamtlast, float)
self.pv_prediction_wh = np.array(parameters.pv_prognose_wh, float)
self.elect_price_hourly = np.array(parameters.strompreis_euro_pro_wh, float)
self.elect_revenue_per_hour_arr = (
parameters.einspeiseverguetung_euro_pro_wh
if isinstance(parameters.einspeiseverguetung_euro_pro_wh, list)
else np.full(
len(self.load_energy_array), parameters.einspeiseverguetung_euro_pro_wh, float
)
)
if inverter:
self.battery = inverter.battery
else:
self.battery = None
self.ev = ev
self.home_appliance = home_appliance
self.inverter = inverter
self.ac_charge_hours = np.full(self.prediction_hours, 0.0)
self.dc_charge_hours = np.full(self.prediction_hours, 1.0)
self.ev_charge_hours = np.full(self.prediction_hours, 0.0)
"""Prepare simulation runs."""
self.load_energy_array = np.array(parameters.gesamtlast, float)
self.pv_prediction_wh = np.array(parameters.pv_prognose_wh, float)
self.elect_price_hourly = np.array(parameters.strompreis_euro_pro_wh, float)
self.elect_revenue_per_hour_arr = (
parameters.einspeiseverguetung_euro_pro_wh
if isinstance(parameters.einspeiseverguetung_euro_pro_wh, list)
else np.full(
len(self.load_energy_array), parameters.einspeiseverguetung_euro_pro_wh, float
)
)
def set_akku_discharge_hours(self, ds: np.ndarray) -> None:
if self.battery:
self.battery.set_discharge_per_hour(ds)
def set_akku_ac_charge_hours(self, ds: np.ndarray) -> None:
self.ac_charge_hours = ds
def set_akku_dc_charge_hours(self, ds: np.ndarray) -> None:
self.dc_charge_hours = ds
def set_ev_charge_hours(self, ds: np.ndarray) -> None:
self.ev_charge_hours = ds
def set_home_appliance_start(self, ds: int, global_start_hour: int = 0) -> None:
if self.home_appliance:
self.home_appliance.set_starting_time(ds, global_start_hour=global_start_hour)
def reset(self) -> None:
if self.ev:
self.ev.reset()
if self.battery:
self.battery.reset()
def simulate(self, start_hour: int) -> dict[str, Any]:
"""Simulate energy usage and costs for the given start hour.
akku_soc_pro_stunde begin of the hour, initial hour state!
last_wh_pro_stunde integral of last hour (end state)
"""
# Remember start hour
self.start_hour = start_hour
# Check for simulation integrity
required_attrs = [
"load_energy_array",
"pv_prediction_wh",
"elect_price_hourly",
"ev_charge_hours",
"ac_charge_hours",
"dc_charge_hours",
"elect_revenue_per_hour_arr",
]
missing_data = [
attr.replace("_", " ").title() for attr in required_attrs if getattr(self, attr) is None
]
if missing_data:
logger.error("Mandatory data missing - %s", ", ".join(missing_data))
raise ValueError(f"Mandatory data missing: {', '.join(missing_data)}")
# Pre-fetch data
load_energy_array = np.array(self.load_energy_array)
pv_prediction_wh = np.array(self.pv_prediction_wh)
elect_price_hourly = np.array(self.elect_price_hourly)
ev_charge_hours = np.array(self.ev_charge_hours)
ac_charge_hours = np.array(self.ac_charge_hours)
dc_charge_hours = np.array(self.dc_charge_hours)
elect_revenue_per_hour_arr = np.array(self.elect_revenue_per_hour_arr)
# Fetch objects
battery = self.battery
ev = self.ev
home_appliance = self.home_appliance
inverter = self.inverter
if not (len(load_energy_array) == len(pv_prediction_wh) == len(elect_price_hourly)):
error_msg = f"Array sizes do not match: Load Curve = {len(load_energy_array)}, PV Forecast = {len(pv_prediction_wh)}, Electricity Price = {len(elect_price_hourly)}"
logger.error(error_msg)
raise ValueError(error_msg)
end_hour = len(load_energy_array)
total_hours = end_hour - start_hour
# Pre-allocate arrays for the results, optimized for speed
loads_energy_per_hour = np.full((total_hours), np.nan)
feedin_energy_per_hour = np.full((total_hours), np.nan)
consumption_energy_per_hour = np.full((total_hours), np.nan)
costs_per_hour = np.full((total_hours), np.nan)
revenue_per_hour = np.full((total_hours), np.nan)
soc_per_hour = np.full((total_hours), np.nan)
soc_ev_per_hour = np.full((total_hours), np.nan)
losses_wh_per_hour = np.full((total_hours), np.nan)
home_appliance_wh_per_hour = np.full((total_hours), np.nan)
electricity_price_per_hour = np.full((total_hours), np.nan)
# Set initial state
if battery:
soc_per_hour[0] = battery.current_soc_percentage()
if ev:
soc_ev_per_hour[0] = ev.current_soc_percentage()
for hour in range(start_hour, end_hour):
hour_idx = hour - start_hour
# save begin states
if battery:
soc_per_hour[hour_idx] = battery.current_soc_percentage()
if ev:
soc_ev_per_hour[hour_idx] = ev.current_soc_percentage()
# Accumulate loads and PV generation
consumption = load_energy_array[hour]
losses_wh_per_hour[hour_idx] = 0.0
# Home appliances
if home_appliance:
ha_load = home_appliance.get_load_for_hour(hour)
consumption += ha_load
home_appliance_wh_per_hour[hour_idx] = ha_load
# E-Auto handling
if ev and ev_charge_hours[hour] > 0:
loaded_energy_ev, verluste_eauto = ev.charge_energy(
None, hour, relative_power=ev_charge_hours[hour]
)
consumption += loaded_energy_ev
losses_wh_per_hour[hour_idx] += verluste_eauto
# Process inverter logic
energy_feedin_grid_actual = energy_consumption_grid_actual = losses = eigenverbrauch = (
0.0
)
hour_ac_charge = ac_charge_hours[hour]
hour_dc_charge = dc_charge_hours[hour]
hourly_electricity_price = elect_price_hourly[hour]
hourly_energy_revenue = elect_revenue_per_hour_arr[hour]
if battery:
battery.set_charge_allowed_for_hour(hour_dc_charge, hour)
if inverter:
energy_produced = pv_prediction_wh[hour]
(
energy_feedin_grid_actual,
energy_consumption_grid_actual,
losses,
eigenverbrauch,
) = inverter.process_energy(energy_produced, consumption, hour)
# AC PV Battery Charge
if battery and hour_ac_charge > 0.0:
battery.set_charge_allowed_for_hour(1, hour)
battery_charged_energy_actual, battery_losses_actual = battery.charge_energy(
None, hour, relative_power=hour_ac_charge
)
total_battery_energy = battery_charged_energy_actual + battery_losses_actual
consumption += total_battery_energy
energy_consumption_grid_actual += total_battery_energy
losses_wh_per_hour[hour_idx] += battery_losses_actual
# Update hourly arrays
feedin_energy_per_hour[hour_idx] = energy_feedin_grid_actual
consumption_energy_per_hour[hour_idx] = energy_consumption_grid_actual
losses_wh_per_hour[hour_idx] += losses
loads_energy_per_hour[hour_idx] = consumption
electricity_price_per_hour[hour_idx] = hourly_electricity_price
# Financial calculations
costs_per_hour[hour_idx] = energy_consumption_grid_actual * hourly_electricity_price
revenue_per_hour[hour_idx] = energy_feedin_grid_actual * hourly_energy_revenue
total_cost = np.nansum(costs_per_hour)
total_losses = np.nansum(losses_wh_per_hour)
total_revenue = np.nansum(revenue_per_hour)
# Prepare output dictionary
return {
"Last_Wh_pro_Stunde": loads_energy_per_hour,
"Netzeinspeisung_Wh_pro_Stunde": feedin_energy_per_hour,
"Netzbezug_Wh_pro_Stunde": consumption_energy_per_hour,
"Kosten_Euro_pro_Stunde": costs_per_hour,
"akku_soc_pro_stunde": soc_per_hour,
"Einnahmen_Euro_pro_Stunde": revenue_per_hour,
"Gesamtbilanz_Euro": total_cost - total_revenue,
"EAuto_SoC_pro_Stunde": soc_ev_per_hour,
"Gesamteinnahmen_Euro": total_revenue,
"Gesamtkosten_Euro": total_cost,
"Verluste_Pro_Stunde": losses_wh_per_hour,
"Gesamt_Verluste": total_losses,
"Home_appliance_wh_per_hour": home_appliance_wh_per_hour,
"Electricity_price": electricity_price_per_hour,
}
class optimization_problem(ConfigMixin, DevicesMixin, EnergyManagementSystemMixin):
class GeneticOptimization(OptimizationBase):
"""GENETIC algorithm to solve energy optimization."""
def __init__(
self,
verbose: bool = False,
@@ -107,8 +310,10 @@ class optimization_problem(ConfigMixin, DevicesMixin, EnergyManagementSystemMixi
):
"""Initialize the optimization problem with the required parameters."""
self.opti_param: dict[str, Any] = {}
self.fixed_eauto_hours = self.config.prediction.hours - self.config.optimization.hours
self.possible_charge_values = self.config.optimization.ev_available_charge_rates_percent
self.fixed_eauto_hours = (
self.config.prediction.hours - self.config.optimization.horizon_hours
)
self.ev_possible_charge_values: list[float] = [1.0]
self.verbose = verbose
self.fix_seed = fixed_seed
self.optimize_ev = True
@@ -122,12 +327,15 @@ class optimization_problem(ConfigMixin, DevicesMixin, EnergyManagementSystemMixi
self.fix_seed = random.randint(1, 100000000000) # noqa: S311
random.seed(self.fix_seed)
# Create Simulation
self.simulation = GeneticSimulation()
def decode_charge_discharge(
self, discharge_hours_bin: np.ndarray
) -> tuple[np.ndarray, np.ndarray, np.ndarray]:
"""Decode the input array into ac_charge, dc_charge, and discharge arrays."""
discharge_hours_bin_np = np.array(discharge_hours_bin)
len_ac = len(self.possible_charge_values)
len_ac = len(self.ev_possible_charge_values)
# Categorization:
# Idle: 0 .. len_ac-1
@@ -159,7 +367,7 @@ class optimization_problem(ConfigMixin, DevicesMixin, EnergyManagementSystemMixi
discharge[discharge_mask] = 1 # Set Discharge states to 1
ac_charge = np.zeros_like(discharge_hours_bin_np, dtype=float)
ac_charge[ac_mask] = [self.possible_charge_values[i] for i in ac_indices]
ac_charge[ac_mask] = [self.ev_possible_charge_values[i] for i in ac_indices]
# Idle is just 0, already default.
@@ -168,7 +376,7 @@ class optimization_problem(ConfigMixin, DevicesMixin, EnergyManagementSystemMixi
def mutate(self, individual: list[int]) -> tuple[list[int]]:
"""Custom mutation function for the individual."""
# Calculate the number of states
len_ac = len(self.possible_charge_values)
len_ac = len(self.ev_possible_charge_values)
if self.optimize_dc_charge:
total_states = 3 * len_ac + 2
else:
@@ -303,7 +511,7 @@ class optimization_problem(ConfigMixin, DevicesMixin, EnergyManagementSystemMixi
creator.create("Individual", list, fitness=creator.FitnessMin)
self.toolbox = base.Toolbox()
len_ac = len(self.possible_charge_values)
len_ac = len(self.ev_possible_charge_values)
# Total number of states without DC:
# Idle: len_ac states
@@ -362,38 +570,39 @@ class optimization_problem(ConfigMixin, DevicesMixin, EnergyManagementSystemMixi
This is an internal function.
"""
self.ems.reset()
self.simulation.reset()
discharge_hours_bin, eautocharge_hours_index, washingstart_int = self.split_individual(
individual
)
if self.opti_param.get("home_appliance", 0) > 0:
self.ems.set_home_appliance_start(
if self.opti_param.get("home_appliance", 0) > 0 and washingstart_int:
self.simulation.set_home_appliance_start(
washingstart_int, global_start_hour=self.ems.start_datetime.hour
)
ac, dc, discharge = self.decode_charge_discharge(discharge_hours_bin)
self.ems.set_akku_discharge_hours(discharge)
self.simulation.set_akku_discharge_hours(discharge)
# Set DC charge hours only if DC optimization is enabled
if self.optimize_dc_charge:
self.ems.set_akku_dc_charge_hours(dc)
self.ems.set_akku_ac_charge_hours(ac)
self.simulation.set_akku_dc_charge_hours(dc)
self.simulation.set_akku_ac_charge_hours(ac)
if eautocharge_hours_index is not None:
eautocharge_hours_float = np.array(
[self.possible_charge_values[i] for i in eautocharge_hours_index],
[self.ev_possible_charge_values[i] for i in eautocharge_hours_index],
float,
)
self.ems.set_ev_charge_hours(eautocharge_hours_float)
self.simulation.set_ev_charge_hours(eautocharge_hours_float)
else:
self.ems.set_ev_charge_hours(np.full(self.config.prediction.hours, 0))
self.simulation.set_ev_charge_hours(np.full(self.config.prediction.hours, 0))
return self.ems.simulate(self.ems.start_datetime.hour)
# Do the simulation and return result.
return self.simulation.simulate(self.ems.start_datetime.hour)
def evaluate(
self,
individual: list[int],
parameters: OptimizationParameters,
parameters: GeneticOptimizationParameters,
start_hour: int,
worst_case: bool,
) -> tuple[float]:
@@ -456,7 +665,8 @@ class optimization_problem(ConfigMixin, DevicesMixin, EnergyManagementSystemMixi
# # len_ac + 2
# # ) # Activate discharge for these hours
# # When Battery SoC then set the Discharge randomly to 0 or 1. otherwise it's very unlikely to get a state where a battery can store energy for a longer time
# # When Battery SoC then set the Discharge randomly to 0 or 1. otherwise it's very
# # unlikely to get a state where a battery can store energy for a longer time
# # Find hours where battery SoC is 0
# zero_soc_mask = battery_soc_per_hour_tail == 0
# # discharge_hours_bin_tail[zero_soc_mask] = (
@@ -478,43 +688,67 @@ class optimization_problem(ConfigMixin, DevicesMixin, EnergyManagementSystemMixi
individual.extra_data = ( # type: ignore[attr-defined]
o["Gesamtbilanz_Euro"],
o["Gesamt_Verluste"],
parameters.eauto.min_soc_percentage - self.ems.ev.current_soc_percentage()
if parameters.eauto and self.ems.ev
parameters.eauto.min_soc_percentage - self.simulation.ev.current_soc_percentage()
if parameters.eauto and self.simulation.ev
else 0,
)
# Adjust total balance with battery value and penalties for unmet SOC
restwert_akku = (
self.ems.battery.current_energy_content() * parameters.ems.preis_euro_pro_wh_akku
)
gesamtbilanz += -restwert_akku
if self.simulation.battery:
restwert_akku = (
self.simulation.battery.current_energy_content()
* parameters.ems.preis_euro_pro_wh_akku
)
gesamtbilanz += -restwert_akku
if self.optimize_ev:
try:
penalty = self.config.optimization.genetic.penalties["ev_soc_miss"]
except:
# Use default
penalty = 10
logger.error(
"Penalty function parameter `ev_soc_miss` not configured, using {}.", penalty
)
gesamtbilanz += max(
0,
(
parameters.eauto.min_soc_percentage - self.ems.ev.current_soc_percentage()
if parameters.eauto and self.ems.ev
parameters.eauto.min_soc_percentage
- self.simulation.ev.current_soc_percentage()
if parameters.eauto and self.simulation.ev
else 0
)
* self.config.optimization.penalty,
* penalty,
)
return (gesamtbilanz,)
def optimize(
self, start_solution: Optional[list[float]] = None, ngen: int = 200
self,
start_solution: Optional[list[float]] = None,
ngen: int = 200,
) -> tuple[Any, dict[str, list[Any]]]:
"""Run the optimization process using a genetic algorithm."""
population = self.toolbox.population(n=300)
"""Run the optimization process using a genetic algorithm.
@TODO: optimize() ngen default (200) is different from optimierung_ems() ngen default (400).
"""
# Set the number of inviduals in a generation
try:
individuals = self.config.optimization.genetic.individuals
if individuals is None:
raise
except:
individuals = 300
logger.error("Individuals not configured. Using {}.", individuals)
population = self.toolbox.population(n=individuals)
hof = tools.HallOfFame(1)
stats = tools.Statistics(lambda ind: ind.fitness.values)
stats.register("min", np.min)
stats.register("avg", np.mean)
stats.register("max", np.max)
if self.verbose:
print("Start optimize:", start_solution)
logger.debug("Start optimize: {}", start_solution)
# Insert the start solution into the population if provided
if start_solution is not None:
@@ -555,39 +789,63 @@ class optimization_problem(ConfigMixin, DevicesMixin, EnergyManagementSystemMixi
def optimierung_ems(
self,
parameters: OptimizationParameters,
parameters: GeneticOptimizationParameters,
start_hour: Optional[int] = None,
worst_case: bool = False,
ngen: int = 400,
) -> OptimizeResponse:
ngen: Optional[int] = None,
) -> GeneticSolution:
"""Perform EMS (Energy Management System) optimization and visualize results."""
if start_hour is None:
start_hour = self.ems.start_datetime.hour
# Start hour has to be in sync with energy management
if start_hour != self.ems.start_datetime.hour:
raise ValueError(
f"Start hour not synced. EMS {self.ems.start_datetime.hour} vs. GENETIC {start_hour}."
)
# Set the number of generations
generations = ngen
if generations is None:
try:
generations = self.config.optimization.genetic.generations
except:
generations = 400
logger.error("Generations not configured. Using {}.", generations)
einspeiseverguetung_euro_pro_wh = np.full(
self.config.prediction.hours, parameters.ems.einspeiseverguetung_euro_pro_wh
)
# TODO: Refactor device setup phase out
self.devices.reset()
self.simulation.reset()
# Initialize PV and EV batteries
akku: Optional[Battery] = None
if parameters.pv_akku:
akku = Battery(parameters.pv_akku)
self.devices.add_device(akku)
akku = Battery(
parameters.pv_akku,
prediction_hours=self.config.prediction.hours,
)
akku.set_charge_per_hour(np.full(self.config.prediction.hours, 1))
eauto: Optional[Battery] = None
if parameters.eauto:
eauto = Battery(
parameters.eauto,
prediction_hours=self.config.prediction.hours,
)
self.devices.add_device(eauto)
eauto.set_charge_per_hour(np.full(self.config.prediction.hours, 1))
self.optimize_ev = (
parameters.eauto.min_soc_percentage - parameters.eauto.initial_soc_percentage >= 0
)
try:
charge_rates = self.config.devices.electric_vehicles[0].charge_rates
if charge_rates is None:
raise
except:
error_msg = "No charge rates provided for electric vehicle."
logger.exception(error_msg)
raise ValueError(error_msg)
self.ev_possible_charge_values = charge_rates
else:
self.optimize_ev = False
@@ -595,29 +853,30 @@ class optimization_problem(ConfigMixin, DevicesMixin, EnergyManagementSystemMixi
dishwasher = (
HomeAppliance(
parameters=parameters.dishwasher,
optimization_hours=self.config.optimization.horizon_hours,
prediction_hours=self.config.prediction.hours,
)
if parameters.dishwasher is not None
else None
)
self.devices.add_device(dishwasher)
# Initialize the inverter and energy management system
inverter: Optional[Inverter] = None
if parameters.inverter:
inverter = Inverter(
parameters.inverter,
battery=akku,
)
self.devices.add_device(inverter)
self.devices.post_setup()
self.ems.set_parameters(
parameters.ems,
inverter=inverter,
# Prepare device simulation
self.simulation.prepare(
parameters=parameters.ems,
optimization_hours=self.config.optimization.horizon_hours,
prediction_hours=self.config.prediction.hours,
inverter=inverter, # battery is part of inverter
ev=eauto,
home_appliance=dishwasher,
)
self.ems.set_start_hour(start_hour)
# Setup the DEAP environment and optimization process
self.setup_deap_environment({"home_appliance": 1 if dishwasher else 0}, start_hour)
@@ -626,20 +885,24 @@ class optimization_problem(ConfigMixin, DevicesMixin, EnergyManagementSystemMixi
lambda ind: self.evaluate(ind, parameters, start_hour, worst_case),
)
if self.verbose:
start_time = time.time()
start_solution, extra_data = self.optimize(parameters.start_solution, ngen=ngen)
start_time = time.time()
start_solution, extra_data = self.optimize(parameters.start_solution, ngen=generations)
elapsed_time = time.time() - start_time
logger.debug(f"Time evaluate inner: {elapsed_time:.4f} sec.")
if self.verbose:
elapsed_time = time.time() - start_time
print(f"Time evaluate inner: {elapsed_time:.4f} sec.")
# Perform final evaluation on the best solution
o = self.evaluate_inner(start_solution)
simulation_result = self.evaluate_inner(start_solution)
# Prepare results
discharge_hours_bin, eautocharge_hours_index, washingstart_int = self.split_individual(
start_solution
)
# home appliance may have choosen a different appliance start hour
if self.simulation.home_appliance:
washingstart_int = self.simulation.home_appliance.get_appliance_start()
eautocharge_hours_float = (
[self.possible_charge_values[i] for i in eautocharge_hours_index]
[self.ev_possible_charge_values[i] for i in eautocharge_hours_index]
if eautocharge_hours_index is not None
else None
)
@@ -651,8 +914,8 @@ class optimization_problem(ConfigMixin, DevicesMixin, EnergyManagementSystemMixi
"dc_charge": dc_charge.tolist(),
"discharge_allowed": discharge.tolist(),
"eautocharge_hours_float": eautocharge_hours_float,
"result": o,
"eauto_obj": self.ems.ev.to_dict(),
"result": simulation_result,
"eauto_obj": self.simulation.ev.to_dict() if self.simulation.ev else None,
"start_solution": start_solution,
"spuelstart": washingstart_int,
"extra_data": extra_data,
@@ -663,14 +926,14 @@ class optimization_problem(ConfigMixin, DevicesMixin, EnergyManagementSystemMixi
prepare_visualize(parameters, visualize, start_hour=start_hour)
return OptimizeResponse(
return GeneticSolution(
**{
"ac_charge": ac_charge,
"dc_charge": dc_charge,
"discharge_allowed": discharge,
"eautocharge_hours_float": eautocharge_hours_float,
"result": SimulationResult(**o),
"eauto_obj": self.ems.ev,
"result": GeneticSimulationResult(**simulation_result),
"eauto_obj": self.simulation.ev,
"start_solution": start_solution,
"washingstart": washingstart_int,
}

11
src/akkudoktoreos/optimization/genetic/geneticabc.py Normal file
View File

@@ -0,0 +1,11 @@
"""Genetic optimization algorithm abstract and base classes."""
from pydantic import ConfigDict
from akkudoktoreos.core.pydantic import PydanticBaseModel
class GeneticParametersBaseModel(PydanticBaseModel):
"""Pydantic base model for parameters for the GENETIC algorithm."""
model_config = ConfigDict(extra="forbid")

127
src/akkudoktoreos/optimization/genetic/geneticdevices.py Normal file
View File

@@ -0,0 +1,127 @@
"""Genetic optimization algorithm device interfaces/ parameters."""
from typing import Optional
from pydantic import Field
from akkudoktoreos.optimization.genetic.geneticabc import GeneticParametersBaseModel
from akkudoktoreos.utils.datetimeutil import TimeWindowSequence
class DeviceParameters(GeneticParametersBaseModel):
device_id: str = Field(description="ID of device", examples="device1")
hours: Optional[int] = Field(
default=None,
gt=0,
description="Number of prediction hours. Defaults to global config prediction hours.",
examples=[None],
)
def max_charging_power_field(description: Optional[str] = None) -> float:
if description is None:
description = "Maximum charging power in watts."
return Field(
default=5000,
gt=0,
description=description,
)
def initial_soc_percentage_field(description: str) -> int:
return Field(default=0, ge=0, le=100, description=description, examples=[42])
def discharging_efficiency_field(default_value: float) -> float:
return Field(
default=default_value,
gt=0,
le=1,
description="A float representing the discharge efficiency of the battery.",
)
class BaseBatteryParameters(DeviceParameters):
"""Battery Device Simulation Configuration."""
device_id: str = Field(description="ID of battery", examples=["battery1"])
capacity_wh: int = Field(
gt=0,
description="An integer representing the capacity of the battery in watt-hours.",
examples=[8000],
)
charging_efficiency: float = Field(
default=0.88,
gt=0,
le=1,
description="A float representing the charging efficiency of the battery.",
)
discharging_efficiency: float = discharging_efficiency_field(0.88)
max_charge_power_w: Optional[float] = max_charging_power_field()
initial_soc_percentage: int = initial_soc_percentage_field(
"An integer representing the state of charge of the battery at the **start** of the current hour (not the current state)."
)
min_soc_percentage: int = Field(
default=0,
ge=0,
le=100,
description="An integer representing the minimum state of charge (SOC) of the battery in percentage.",
examples=[10],
)
max_soc_percentage: int = Field(
default=100,
ge=0,
le=100,
description="An integer representing the maximum state of charge (SOC) of the battery in percentage.",
)
class SolarPanelBatteryParameters(BaseBatteryParameters):
"""PV battery device simulation configuration."""
max_charge_power_w: Optional[float] = max_charging_power_field()
class ElectricVehicleParameters(BaseBatteryParameters):
"""Battery Electric Vehicle Device Simulation Configuration."""
device_id: str = Field(description="ID of electric vehicle", examples=["ev1"])
discharging_efficiency: float = discharging_efficiency_field(1.0)
initial_soc_percentage: int = initial_soc_percentage_field(
"An integer representing the current state of charge (SOC) of the battery in percentage."
)
class HomeApplianceParameters(DeviceParameters):
"""Home Appliance Device Simulation Configuration."""
device_id: str = Field(description="ID of home appliance", examples=["dishwasher"])
consumption_wh: int = Field(
gt=0,
description="An integer representing the energy consumption of a household device in watt-hours.",
examples=[2000],
)
duration_h: int = Field(
gt=0,
description="An integer representing the usage duration of a household device in hours.",
examples=[3],
)
time_windows: Optional[TimeWindowSequence] = Field(
default=None,
description="List of allowed time windows. Defaults to optimization general time window.",
examples=[
[
{"start_time": "10:00", "duration": "2 hours"},
],
],
)
class InverterParameters(DeviceParameters):
"""Inverter Device Simulation Configuration."""
device_id: str = Field(description="ID of inverter", examples=["inverter1"])
max_power_wh: float = Field(gt=0, examples=[10000])
battery_id: Optional[str] = Field(
default=None, description="ID of battery", examples=[None, "battery1"]
)

630
src/akkudoktoreos/optimization/genetic/geneticparams.py Normal file
View File

@@ -0,0 +1,630 @@
"""GENETIC algorithm paramters.
This module defines the Pydantic-based configuration and input parameter models
used in the energy optimization routines, including photovoltaic forecasts,
electricity pricing, and system component parameters.
It also provides a method to assemble these parameters from predictions,
forecasts, and fallback defaults, preparing them for optimization runs.
"""
from typing import Optional, Union
from loguru import logger
from pydantic import Field, field_validator, model_validator
from typing_extensions import Self
from akkudoktoreos.core.coreabc import (
ConfigMixin,
MeasurementMixin,
PredictionMixin,
)
from akkudoktoreos.optimization.genetic.geneticabc import GeneticParametersBaseModel
from akkudoktoreos.optimization.genetic.geneticdevices import (
ElectricVehicleParameters,
HomeApplianceParameters,
InverterParameters,
SolarPanelBatteryParameters,
)
from akkudoktoreos.utils.datetimeutil import to_duration
# Do not import directly from akkudoktoreos.core.coreabc
# EnergyManagementSystemMixin - Creates circular dependency with ems.py
# StartMixin - Creates circular dependency with ems.py
class GeneticEnergyManagementParameters(GeneticParametersBaseModel):
"""Encapsulates energy-related forecasts and costs used in GENETIC optimization."""
pv_prognose_wh: list[float] = Field(
description="An array of floats representing the forecasted photovoltaic output in watts for different time intervals."
)
strompreis_euro_pro_wh: list[float] = Field(
description="An array of floats representing the electricity price in euros per watt-hour for different time intervals."
)
einspeiseverguetung_euro_pro_wh: Union[list[float], float] = Field(
description="A float or array of floats representing the feed-in compensation in euros per watt-hour."
)
preis_euro_pro_wh_akku: float = Field(
description="A float representing the cost of battery energy per watt-hour."
)
gesamtlast: list[float] = Field(
description="An array of floats representing the total load (consumption) in watts for different time intervals."
)
@model_validator(mode="after")
def validate_list_length(self) -> Self:
"""Validate that all input lists are of the same length.
Raises:
ValueError: If input list lengths differ.
"""
pv_prognose_length = len(self.pv_prognose_wh)
if (
pv_prognose_length != len(self.strompreis_euro_pro_wh)
or pv_prognose_length != len(self.gesamtlast)
or (
isinstance(self.einspeiseverguetung_euro_pro_wh, list)
and pv_prognose_length != len(self.einspeiseverguetung_euro_pro_wh)
)
):
raise ValueError("Input lists have different lengths")
return self
class GeneticOptimizationParameters(
ConfigMixin,
MeasurementMixin,
PredictionMixin,
# EnergyManagementSystemMixin, # Creates circular dependency with ems.py
# StartMixin, # Creates circular dependency with ems.py
GeneticParametersBaseModel,
):
"""Main parameter class for running the genetic energy optimization.
Collects all model and configuration parameters necessary to run the
optimization process, such as forecasts, pricing, battery and appliance models.
"""
ems: GeneticEnergyManagementParameters
pv_akku: Optional[SolarPanelBatteryParameters]
inverter: Optional[InverterParameters]
eauto: Optional[ElectricVehicleParameters]
dishwasher: Optional[HomeApplianceParameters] = None
temperature_forecast: Optional[list[Optional[float]]] = Field(
default=None,
description="An array of floats representing the temperature forecast in degrees Celsius for different time intervals.",
)
start_solution: Optional[list[float]] = Field(
default=None, description="Can be `null` or contain a previous solution (if available)."
)
@model_validator(mode="after")
def validate_list_length(self) -> Self:
"""Ensure that temperature forecast list matches the PV forecast length.
Raises:
ValueError: If list lengths mismatch.
"""
arr_length = len(self.ems.pv_prognose_wh)
if self.temperature_forecast is not None and arr_length != len(self.temperature_forecast):
raise ValueError("Input lists have different lengths")
return self
@field_validator("start_solution")
def validate_start_solution(
cls, start_solution: Optional[list[float]]
) -> Optional[list[float]]:
"""Validate that the starting solution has at least two elements.
Args:
start_solution (list[float]): Optional list of solution values.
Returns:
list[float]: Validated list.
Raises:
ValueError: If the solution is too short.
"""
if start_solution is not None and len(start_solution) < 2:
raise ValueError("Requires at least two values.")
return start_solution
@classmethod
def prepare(cls) -> "Optional[GeneticOptimizationParameters]":
"""Prepare optimization parameters from config, forecast and measurement data.
Fills in values needed for optimization from available configuration, predictions and
measurements. If some data is missing, default or demo values are used.
Parameters start by definition of the genetic algorithm at hour 0 of the actual date
(not at start datetime of energy management run)
Returns:
GeneticOptimizationParameters: The fully prepared optimization parameters.
Raises:
ValueError: If required configuration values like start time are missing.
"""
# Avoid circular dependency
from akkudoktoreos.core.ems import get_ems
ems = get_ems()
# The optimization paramters
oparams: "Optional[GeneticOptimizationParameters]" = None
# Check for run definitions
if ems.start_datetime is None:
error_msg = "Start datetime unknown."
logger.error(error_msg)
raise ValueError(error_msg)
# Check for general predictions conditions
if cls.config.general.latitude is None:
default_latitude = 52.52
logger.error(f"Latitude unknown - defaulting to {default_latitude}.")
cls.config.general.latitude = default_latitude
if cls.config.general.longitude is None:
default_longitude = 13.405
logger.error(f"Longitude unknown - defaulting to {default_longitude}.")
cls.config.general.longitude = default_longitude
if cls.config.prediction.hours is None:
logger.error("Prediction hours unknown - defaulting to 48 hours.")
cls.config.prediction.hours = 48
if cls.config.prediction.historic_hours is None:
logger.error("Prediction historic hours unknown - defaulting to 24 hours.")
cls.config.prediction.historic_hours = 24
# Check optimization definitions
if cls.config.optimization.horizon_hours is None:
logger.error("Optimization horizon unknown - defaulting to 24 hours.")
cls.config.optimization.horizon_hours = 24
if cls.config.optimization.interval is None:
logger.error("Optimization interval unknown - defaulting to 3600 seconds.")
cls.config.optimization.interval = 3600
if cls.config.optimization.interval != 3600:
logger.error(
"Optimization interval '{}' seconds not supported - forced to 3600 seconds."
)
cls.config.optimization.interval = 3600
# Check genetic algorithm definitions
if cls.config.optimization.genetic is None:
logger.error(
"Genetic optimization configuration not configured - defaulting to demo config."
)
cls.config.optimization.genetic = {
"individuals": 300,
"generations": 400,
"seed": None,
"penalties": {
"ev_soc_miss": 10,
},
}
if cls.config.optimization.genetic.individuals is None:
logger.error("Genetic individuals unknown - defaulting to 300.")
cls.config.optimization.genetic.individuals = 300
if cls.config.optimization.genetic.generations is None:
logger.error("Genetic generations unknown - defaulting to 400.")
cls.config.optimization.genetic.generations = 400
if cls.config.optimization.genetic.penalties is None:
logger.error("Genetic penalties unknown - defaulting to demo config.")
cls.config.optimization.genetic.penalties = {"ev_soc_miss": 10}
if "ev_soc_miss" not in cls.config.optimization.genetic.penalties:
logger.error("ev_soc_miss penalty function parameter unknown - defaulting to 100.")
cls.config.optimization.genetic.penalties["ev_soc_miss"] = 10
# Add forecast and device data
interval = to_duration(cls.config.optimization.interval)
power_to_energy_per_interval_factor = cls.config.optimization.interval / 3600
parameter_start_datetime = ems.start_datetime.set(hour=0, second=0, microsecond=0)
parameter_end_datetime = parameter_start_datetime.add(hours=cls.config.prediction.hours)
max_retries = 10
for attempt in range(1, max_retries + 1):
# Collect all the data for optimisation, but do not exceed max retries
if attempt > max_retries:
error_msg = f"Maximum retries {max_retries} for parameter collection exceeded. Parameter preparation attempt {attempt}."
logger.error(error_msg)
raise ValueError(error_msg)
# Assure predictions are uptodate
cls.prediction.update_data()
try:
pvforecast_ac_power = (
cls.prediction.key_to_array(
key="pvforecast_ac_power",
start_datetime=parameter_start_datetime,
end_datetime=parameter_end_datetime,
interval=interval,
fill_method="linear",
)
* power_to_energy_per_interval_factor
).tolist()
except:
logger.exception(
"No PV forecast data available - defaulting to demo data. Parameter preparation attempt {}.",
attempt,
)
cls.config.merge_settings_from_dict(
{
"pvforecast": {
"provider": "PVForecastAkkudoktor",
"planes": [
{
"peakpower": 5.0,
"surface_azimuth": 170,
"surface_tilt": 7,
"userhorizon": [20, 27, 22, 20],
"inverter_paco": 10000,
},
{
"peakpower": 4.8,
"surface_azimuth": 90,
"surface_tilt": 7,
"userhorizon": [30, 30, 30, 50],
"inverter_paco": 10000,
},
{
"peakpower": 1.4,
"surface_azimuth": 140,
"surface_tilt": 60,
"userhorizon": [60, 30, 0, 30],
"inverter_paco": 2000,
},
{
"peakpower": 1.6,
"surface_azimuth": 185,
"surface_tilt": 45,
"userhorizon": [45, 25, 30, 60],
"inverter_paco": 1400,
},
],
},
}
)
# Retry
continue
try:
elecprice_marketprice_wh = cls.prediction.key_to_array(
key="elecprice_marketprice_wh",
start_datetime=parameter_start_datetime,
end_datetime=parameter_end_datetime,
interval=interval,
fill_method="ffill",
).tolist()
except:
logger.exception(
"No Electricity Marketprice forecast data available - defaulting to demo data. Parameter preparation attempt {}.",
attempt,
)
cls.config.elecprice.provider = "ElecPriceAkkudoktor"
# Retry
continue
try:
load_mean_adjusted = cls.prediction.key_to_array(
key="load_mean_adjusted",
start_datetime=parameter_start_datetime,
end_datetime=parameter_end_datetime,
interval=interval,
fill_method="ffill",
).tolist()
except:
logger.exception(
"No Load forecast data available - defaulting to demo data. Parameter preparation attempt {}.",
attempt,
)
cls.config.merge_settings_from_dict(
{
"load": {
"provider": "LoadAkkudoktor",
"provider_settings": {
"LoadAkkudoktor": {
"loadakkudoktor_year_energy": "1000",
},
},
},
}
)
# Retry
continue
try:
feed_in_tariff_wh = cls.prediction.key_to_array(
key="feed_in_tariff_wh",
start_datetime=parameter_start_datetime,
end_datetime=parameter_end_datetime,
interval=interval,
fill_method="ffill",
).tolist()
except:
logger.exception(
"No feed in tariff forecast data available - defaulting to demo data. Parameter preparation attempt {}.",
attempt,
)
cls.config.merge_settings_from_dict(
{
"feedintariff": {
"provider": "FeedInTariffFixed",
"provider_settings": {
"FeedInTariffFixed": {
"feed_in_tariff_kwh": 0.078,
},
},
},
}
)
# Retry
continue
try:
weather_temp_air = cls.prediction.key_to_array(
key="weather_temp_air",
start_datetime=parameter_start_datetime,
end_datetime=parameter_end_datetime,
interval=interval,
fill_method="ffill",
).tolist()
except:
logger.exception(
"No weather forecast data available - defaulting to demo data. Parameter preparation attempt {}.",
attempt,
)
cls.config.weather.provider = "BrightSky"
# Retry
continue
# Add device data
# Batteries
# ---------
if cls.config.devices.max_batteries is None:
logger.error("Number of battery devices not configured - defaulting to 1.")
cls.config.devices.max_batteries = 1
if cls.config.devices.max_batteries == 0:
battery_params = None
battery_lcos_kwh = 0
else:
if cls.config.devices.batteries is None:
logger.error("No battery device data available - defaulting to demo data.")
cls.config.devices.batteries = [{"device_id": "battery1", "capacity_wh": 8000}]
try:
battery_config = cls.config.devices.batteries[0]
battery_params = SolarPanelBatteryParameters(
device_id=battery_config.device_id,
capacity_wh=battery_config.capacity_wh,
charging_efficiency=battery_config.charging_efficiency,
discharging_efficiency=battery_config.discharging_efficiency,
max_charge_power_w=battery_config.max_charge_power_w,
min_soc_percentage=battery_config.min_soc_percentage,
max_soc_percentage=battery_config.max_soc_percentage,
)
except:
logger.exception(
"No battery device data available - defaulting to demo data. Parameter preparation attempt {}.",
attempt,
)
cls.config.devices.batteries = [{"device_id": "battery1", "capacity_wh": 8000}]
# Retry
continue
# Levelized cost of ownership
if battery_config.levelized_cost_of_storage_kwh is None:
logger.error(
"No battery device LCOS data available - defaulting to 0 €/kWh. Parameter preparation attempt {}.",
attempt,
)
battery_config.levelized_cost_of_storage_kwh = 0
battery_lcos_kwh = battery_config.levelized_cost_of_storage_kwh
# Initial SOC
try:
initial_soc_factor = cls.measurement.key_to_value(
key=battery_config.measurement_key_soc_factor,
target_datetime=ems.start_datetime,
)
if initial_soc_factor > 1.0 or initial_soc_factor < 0.0:
logger.error(
f"Invalid battery initial SoC factor {initial_soc_factor} - defaulting to 0.0."
)
initial_soc_factor = 0.0
# genetic parameter is 0..100 as int
initial_soc_percentage = int(initial_soc_factor * 100)
except:
initial_soc_percentage = None
if initial_soc_percentage is None:
logger.error(
f"No battery device SoC data (measurement key = '{battery_config.measurement_key_soc_factor}') available - defaulting to 0."
)
initial_soc_percentage = 0
battery_params.initial_soc_percentage = initial_soc_percentage
# Electric Vehicles
# -----------------
if cls.config.devices.max_electric_vehicles is None:
logger.error("Number of electric_vehicle devices not configured - defaulting to 1.")
cls.config.devices.max_electric_vehicles = 1
if cls.config.devices.max_electric_vehicles == 0:
electric_vehicle_params = None
else:
if cls.config.devices.electric_vehicles is None:
logger.error(
"No electric vehicle device data available - defaulting to demo data."
)
cls.config.devices.max_electric_vehicles = 1
cls.config.devices.electric_vehicles = [
{
"device_id": "ev11",
"capacity_wh": 50000,
"charge_rates": [0.0, 0.375, 0.5, 0.625, 0.75, 0.875, 1.0],
"min_soc_percentage": 70,
}
]
try:
electric_vehicle_config = cls.config.devices.electric_vehicles[0]
electric_vehicle_params = ElectricVehicleParameters(
device_id=electric_vehicle_config.device_id,
capacity_wh=electric_vehicle_config.capacity_wh,
charging_efficiency=electric_vehicle_config.charging_efficiency,
discharging_efficiency=electric_vehicle_config.discharging_efficiency,
max_charge_power_w=electric_vehicle_config.max_charge_power_w,
min_soc_percentage=electric_vehicle_config.min_soc_percentage,
max_soc_percentage=electric_vehicle_config.max_soc_percentage,
)
except:
logger.exception(
"No electric_vehicle device data available - defaulting to demo data. Parameter preparation attempt {}.",
attempt,
)
cls.config.devices.max_electric_vehicles = 1
cls.config.devices.electric_vehicles = [
{
"device_id": "ev12",
"capacity_wh": 50000,
"charge_rates": [0.0, 0.375, 0.5, 0.625, 0.75, 0.875, 1.0],
"min_soc_percentage": 70,
}
]
# Retry
continue
# Initial SOC
try:
initial_soc_factor = cls.measurement.key_to_value(
key=electric_vehicle_config.measurement_key_soc_factor,
target_datetime=ems.start_datetime,
)
if initial_soc_factor > 1.0 or initial_soc_factor < 0.0:
logger.error(
f"Invalid electric vehicle initial SoC factor {initial_soc_factor} - defaulting to 0.0."
)
initial_soc_factor = 0.0
# genetic parameter is 0..100 as int
initial_soc_percentage = int(initial_soc_factor * 100)
except:
initial_soc_percentage = None
if initial_soc_percentage is None:
logger.error(
f"No electric vehicle device SoC data (measurement key = '{electric_vehicle_config.measurement_key_soc_factor}') available - defaulting to 0."
)
initial_soc_percentage = 0
electric_vehicle_params.initial_soc_percentage = initial_soc_percentage
# Inverters
# ---------
if cls.config.devices.max_inverters is None:
logger.error("Number of inverter devices not configured - defaulting to 1.")
cls.config.devices.max_inverters = 1
if cls.config.devices.max_inverters == 0:
inverter_params = None
else:
if cls.config.devices.inverters is None:
logger.error("No inverter device data available - defaulting to demo data.")
cls.config.devices.inverters = [
{
"device_id": "inverter1",
"max_power_w": 10000,
"battery_id": battery_config.device_id,
}
]
try:
inverter_config = cls.config.devices.inverters[0]
inverter_params = InverterParameters(
device_id=inverter_config.device_id,
max_power_wh=inverter_config.max_power_w,
battery_id=inverter_config.battery_id,
)
except:
logger.exception(
"No inverter device data available - defaulting to demo data. Parameter preparation attempt {}.",
attempt,
)
cls.config.devices.inverters = [
{
"device_id": "inverter1",
"max_power_w": 10000,
"battery_id": battery_config.device_id,
}
]
# Retry
continue
# Home Appliances
# ---------------
if cls.config.devices.max_home_appliances is None:
logger.error("Number of home appliance devices not configured - defaulting to 1.")
cls.config.devices.max_home_appliances = 1
if cls.config.devices.max_home_appliances == 0:
home_appliance_params = None
else:
home_appliance_params = None
if cls.config.devices.home_appliances is None:
logger.error(
"No home appliance device data available - defaulting to demo data."
)
cls.config.devices.home_appliances = [
{
"device_id": "dishwasher1",
"consumption_wh": 2000,
"duration_h": 3.0,
"time_windows": {
"windows": [
{
"start_time": "08:00",
"duration": "5 hours",
},
{
"start_time": "15:00",
"duration": "3 hours",
},
],
},
}
]
try:
home_appliance_config = cls.config.devices.home_appliances[0]
home_appliance_params = HomeApplianceParameters(
device_id=home_appliance_config.device_id,
consumption_wh=home_appliance_config.consumption_wh,
duration_h=home_appliance_config.duration_h,
time_windows=home_appliance_config.time_windows,
)
except:
logger.exception(
"No home appliance device data available - defaulting to demo data. Parameter preparation attempt {}.",
attempt,
)
cls.config.devices.home_appliances = [
{
"device_id": "dishwasher1",
"consumption_wh": 2000,
"duration_h": 3.0,
"time_windows": None,
}
]
# Retry
continue
# We got all parameter data
try:
oparams = GeneticOptimizationParameters(
ems=GeneticEnergyManagementParameters(
pv_prognose_wh=pvforecast_ac_power,
strompreis_euro_pro_wh=elecprice_marketprice_wh,
einspeiseverguetung_euro_pro_wh=feed_in_tariff_wh,
gesamtlast=load_mean_adjusted,
preis_euro_pro_wh_akku=battery_lcos_kwh / 1000,
),
temperature_forecast=weather_temp_air,
pv_akku=battery_params,
eauto=electric_vehicle_params,
inverter=inverter_params,
dishwasher=home_appliance_params,
)
except:
logger.exception(
"Can not prepare optimization parameters - will retry. Parameter preparation attempt {}.",
attempt,
)
oparams = None
# Retry
continue
# Parameters prepared
break
return oparams

480
src/akkudoktoreos/optimization/genetic/geneticsolution.py Normal file
View File

@@ -0,0 +1,480 @@
"""Genetic algorithm optimisation solution."""
from typing import Any, Optional
import pandas as pd
from loguru import logger
from pydantic import Field, field_validator
from akkudoktoreos.config.config import get_config
from akkudoktoreos.core.emplan import (
DDBCInstruction,
EnergyManagementPlan,
FRBCInstruction,
)
from akkudoktoreos.core.pydantic import PydanticDateTimeDataFrame
from akkudoktoreos.devices.devicesabc import (
ApplianceOperationMode,
BatteryOperationMode,
)
from akkudoktoreos.devices.genetic.battery import Battery
from akkudoktoreos.optimization.genetic.geneticdevices import GeneticParametersBaseModel
from akkudoktoreos.optimization.optimization import OptimizationSolution
from akkudoktoreos.prediction.prediction import get_prediction
from akkudoktoreos.utils.datetimeutil import to_datetime, to_duration
from akkudoktoreos.utils.utils import NumpyEncoder
class DeviceOptimizeResult(GeneticParametersBaseModel):
device_id: str = Field(description="ID of device", examples=["device1"])
hours: int = Field(gt=0, description="Number of hours in the simulation.", examples=[24])
class ElectricVehicleResult(DeviceOptimizeResult):
"""Result class containing information related to the electric vehicle's charging and discharging behavior."""
device_id: str = Field(description="ID of electric vehicle", examples=["ev1"])
charge_array: list[float] = Field(
description="Hourly charging status (0 for no charging, 1 for charging)."
)
discharge_array: list[int] = Field(
description="Hourly discharging status (0 for no discharging, 1 for discharging)."
)
discharging_efficiency: float = Field(description="The discharge efficiency as a float..")
capacity_wh: int = Field(description="Capacity of the EV’s battery in watt-hours.")
charging_efficiency: float = Field(description="Charging efficiency as a float..")
max_charge_power_w: int = Field(description="Maximum charging power in watts.")
soc_wh: float = Field(
description="State of charge of the battery in watt-hours at the start of the simulation."
)
initial_soc_percentage: int = Field(
description="State of charge at the start of the simulation in percentage."
)
@field_validator("discharge_array", "charge_array", mode="before")
def convert_numpy(cls, field: Any) -> Any:
return NumpyEncoder.convert_numpy(field)[0]
class GeneticSimulationResult(GeneticParametersBaseModel):
"""This object contains the results of the simulation and provides insights into various parameters over the entire forecast period."""
Last_Wh_pro_Stunde: list[float] = Field(description="TBD")
EAuto_SoC_pro_Stunde: list[float] = Field(
description="The state of charge of the EV for each hour."
)
Einnahmen_Euro_pro_Stunde: list[float] = Field(
description="The revenue from grid feed-in or other sources in euros per hour."
)
Gesamt_Verluste: float = Field(
description="The total losses in watt-hours over the entire period."
)
Gesamtbilanz_Euro: float = Field(
description="The total balance of revenues minus costs in euros."
)
Gesamteinnahmen_Euro: float = Field(description="The total revenues in euros.")
Gesamtkosten_Euro: float = Field(description="The total costs in euros.")
Home_appliance_wh_per_hour: list[Optional[float]] = Field(
description="The energy consumption of a household appliance in watt-hours per hour."
)
Kosten_Euro_pro_Stunde: list[float] = Field(description="The costs in euros per hour.")
Netzbezug_Wh_pro_Stunde: list[float] = Field(
description="The grid energy drawn in watt-hours per hour."
)
Netzeinspeisung_Wh_pro_Stunde: list[float] = Field(
description="The energy fed into the grid in watt-hours per hour."
)
Verluste_Pro_Stunde: list[float] = Field(description="The losses in watt-hours per hour.")
akku_soc_pro_stunde: list[float] = Field(
description="The state of charge of the battery (not the EV) in percentage per hour."
)
Electricity_price: list[float] = Field(
description="Used Electricity Price, including predictions"
)
@field_validator(
"Last_Wh_pro_Stunde",
"Netzeinspeisung_Wh_pro_Stunde",
"akku_soc_pro_stunde",
"Netzbezug_Wh_pro_Stunde",
"Kosten_Euro_pro_Stunde",
"Einnahmen_Euro_pro_Stunde",
"EAuto_SoC_pro_Stunde",
"Verluste_Pro_Stunde",
"Home_appliance_wh_per_hour",
"Electricity_price",
mode="before",
)
def convert_numpy(cls, field: Any) -> Any:
return NumpyEncoder.convert_numpy(field)[0]
class GeneticSolution(GeneticParametersBaseModel):
"""**Note**: The first value of "Last_Wh_per_hour", "Netzeinspeisung_Wh_per_hour", and "Netzbezug_Wh_per_hour", will be set to null in the JSON output and represented as NaN or None in the corresponding classes' data returns. This approach is adopted to ensure that the current hour's processing remains unchanged."""
ac_charge: list[float] = Field(
description="Array with AC charging values as relative power (0.0-1.0), other values set to 0."
)
dc_charge: list[float] = Field(
description="Array with DC charging values as relative power (0-1), other values set to 0."
)
discharge_allowed: list[int] = Field(
description="Array with discharge values (1 for discharge, 0 otherwise)."
)
eautocharge_hours_float: Optional[list[float]] = Field(description="TBD")
result: GeneticSimulationResult
eauto_obj: Optional[ElectricVehicleResult]
start_solution: Optional[list[float]] = Field(
default=None,
description="An array of binary values (0 or 1) representing a possible starting solution for the simulation.",
)
washingstart: Optional[int] = Field(
default=None,
description="Can be `null` or contain an object representing the start of washing (if applicable).",
)
@field_validator(
"ac_charge",
"dc_charge",
"discharge_allowed",
mode="before",
)
def convert_numpy(cls, field: Any) -> Any:
return NumpyEncoder.convert_numpy(field)[0]
@field_validator(
"eauto_obj",
mode="before",
)
def convert_eauto(cls, field: Any) -> Any:
if isinstance(field, Battery):
return ElectricVehicleResult(**field.to_dict())
return field
def _battery_operation_from_solution(
self,
ac_charge: float,
dc_charge: float,
discharge_allowed: bool,
) -> tuple[BatteryOperationMode, float]:
"""Maps low-level solution to a representative operation mode and factor.
Args:
ac_charge (float): Allowed AC-side charging power (relative units).
dc_charge (float): Allowed DC-side charging power (relative units).
discharge_allowed (bool): Whether discharging is permitted.
Returns:
tuple[BatteryOperationMode, float]:
A tuple containing:
- `BatteryOperationMode`: the representative high-level operation mode.
- `float`: the operation factor corresponding to the active signal.
Notes:
- The mapping prioritizes AC charge > DC charge > discharge.
- Multiple strategies can produce the same low-level signals; this function
returns a representative mode based on a defined priority order.
"""
# (0,0,0) → Nothing allowed
if ac_charge <= 0.0 and dc_charge <= 0.0 and not discharge_allowed:
return BatteryOperationMode.IDLE, 1.0
# (0,0,1) → Discharge only
if ac_charge <= 0.0 and dc_charge <= 0.0 and discharge_allowed:
return BatteryOperationMode.PEAK_SHAVING, 1.0
# (ac>0,0,0) → AC charge only
if ac_charge > 0.0 and dc_charge <= 0.0 and not discharge_allowed:
return BatteryOperationMode.GRID_SUPPORT_IMPORT, ac_charge
# (0,dc>0,0) → DC charge only
if ac_charge <= 0.0 and dc_charge > 0.0 and not discharge_allowed:
return BatteryOperationMode.NON_EXPORT, dc_charge
# (ac>0,dc>0,0) → Both charge paths, no discharge
if ac_charge > 0.0 and dc_charge > 0.0 and not discharge_allowed:
return BatteryOperationMode.FORCED_CHARGE, ac_charge
# (ac>0,0,1) → AC charge + discharge - does not make sense
if ac_charge > 0.0 and dc_charge <= 0.0 and discharge_allowed:
raise ValueError(
f"Illegal state: ac_charge: {ac_charge} and discharge_allowed: {discharge_allowed}"
)
# (0,dc>0,1) → DC charge + discharge
if ac_charge <= 0.0 and dc_charge > 0.0 and discharge_allowed:
return BatteryOperationMode.SELF_CONSUMPTION, dc_charge
# (ac>0,dc>0,1) → Fully flexible - does not make sense
if ac_charge > 0.0 and dc_charge > 0.0 and discharge_allowed:
raise ValueError(
f"Illegal state: ac_charge: {ac_charge} and discharge_allowed: {discharge_allowed}"
)
# Fallback → safe idle
return BatteryOperationMode.IDLE, 1.0
def optimization_solution(self) -> OptimizationSolution:
"""Provide the genetic solution as a general optimization solution.
The battery modes are controlled by the grid control triggers:
- ac_charge: charge from grid
- discharge_allowed: discharge to grid
The following battery modes are supported:
- SELF_CONSUMPTION: ac_charge == 0 and discharge_allowed == 0
- GRID_SUPPORT_EXPORT: ac_charge == 0 and discharge_allowed == 1
- GRID_SUPPORT_IMPORT: ac_charge > 0 and discharge_allowed == 0 or 1
"""
from akkudoktoreos.core.ems import get_ems
config = get_config()
start_datetime = get_ems().start_datetime
interval_hours = 1
# --- Create index based on list length and interval ---
n_points = len(self.result.Kosten_Euro_pro_Stunde)
time_index = pd.date_range(
start=start_datetime,
periods=n_points,
freq=f"{interval_hours}h",
)
end_datetime = start_datetime.add(hours=n_points)
# Fill data into dataframe with correct column names
# - load_energy_wh: Load of all energy consumers in wh"
# - grid_energy_wh: Grid energy feed in (negative) or consumption (positive) in wh"
# - pv_prediction_energy_wh: PV energy prediction (positive) in wh"
# - elec_price_prediction_amt_kwh: Electricity price prediction in money per kwh"
# - costs_amt: Costs in money amount"
# - revenue_amt: Revenue in money amount"
# - losses_energy_wh: Energy losses in wh"
# - <device-id>_<operation>_op_mode: Operation mode of the device (1.0 when active)."
# - <device-id>_<operation>_op_factor: Operation mode factor of the device."
# - <device-id>_soc_factor: State of charge of a battery/ electric vehicle device as factor of total capacity."
# - <device-id>_energy_wh: Energy consumption (positive) of a device in wh."
data = pd.DataFrame(
{
"date_time": time_index,
"load_energy_wh": self.result.Last_Wh_pro_Stunde,
"grid_feedin_energy_wh": self.result.Netzeinspeisung_Wh_pro_Stunde,
"grid_consumption_energy_wh": self.result.Netzbezug_Wh_pro_Stunde,
"elec_price_prediction_amt_kwh": [v * 1000 for v in self.result.Electricity_price],
"costs_amt": self.result.Kosten_Euro_pro_Stunde,
"revenue_amt": self.result.Einnahmen_Euro_pro_Stunde,
"losses_energy_wh": self.result.Verluste_Pro_Stunde,
},
index=time_index,
)
# Add battery data
data["battery1_soc_factor"] = [v / 100 for v in self.result.akku_soc_pro_stunde]
operation: dict[str, list[float]] = {}
for hour, rate in enumerate(self.ac_charge):
if hour >= n_points:
break
operation_mode, operation_mode_factor = self._battery_operation_from_solution(
self.ac_charge[hour], self.dc_charge[hour], bool(self.discharge_allowed[hour])
)
for mode in BatteryOperationMode:
mode_key = f"battery1_{mode.lower()}_op_mode"
factor_key = f"battery1_{mode.lower()}_op_factor"
if mode_key not in operation.keys():
operation[mode_key] = []
operation[factor_key] = []
if mode == operation_mode:
operation[mode_key].append(1.0)
operation[factor_key].append(operation_mode_factor)
else:
operation[mode_key].append(0.0)
operation[factor_key].append(0.0)
for key in operation.keys():
data[key] = operation[key]
# Add EV battery data
if self.eauto_obj:
if self.eautocharge_hours_float is None:
# Electric vehicle is full enough. No load times.
data[f"{self.eauto_obj.device_id}_soc_factor"] = [
self.eauto_obj.initial_soc_percentage / 100.0
] * n_points
# operation modes
operation_mode = BatteryOperationMode.IDLE
for mode in BatteryOperationMode:
mode_key = f"{self.eauto_obj.device_id}_{mode.lower()}_op_mode"
factor_key = f"{self.eauto_obj.device_id}_{mode.lower()}_op_factor"
if mode == operation_mode:
data[mode_key] = [1.0] * n_points
data[factor_key] = [1.0] * n_points
else:
data[mode_key] = [0.0] * n_points
data[factor_key] = [0.0] * n_points
else:
data[f"{self.eauto_obj.device_id}_soc_factor"] = [
v / 100 for v in self.result.EAuto_SoC_pro_Stunde
]
operation = {}
for hour, rate in enumerate(self.eautocharge_hours_float):
if hour >= n_points:
break
operation_mode, operation_mode_factor = self._battery_operation_from_solution(
rate, 0.0, False
)
for mode in BatteryOperationMode:
mode_key = f"{self.eauto_obj.device_id}_{mode.lower()}_op_mode"
factor_key = f"{self.eauto_obj.device_id}_{mode.lower()}_op_factor"
if mode_key not in operation.keys():
operation[mode_key] = []
operation[factor_key] = []
if mode == operation_mode:
operation[mode_key].append(1.0)
operation[factor_key].append(operation_mode_factor)
else:
operation[mode_key].append(0.0)
operation[factor_key].append(0.0)
for key in operation.keys():
data[key] = operation[key]
# Add home appliance data
if self.washingstart:
data["homeappliance1_energy_wh"] = self.result.Home_appliance_wh_per_hour
# Add important predictions that are not already available from the GenericSolution
prediction = get_prediction()
power_to_energy_per_interval_factor = 1.0
if "pvforecast_ac_power" in prediction.record_keys:
data["pv_prediction_energy_wh"] = (
prediction.key_to_array(
key="pvforecast_ac_power",
start_datetime=start_datetime,
end_datetime=end_datetime,
interval=to_duration(f"{interval_hours} hours"),
fill_method="linear",
)
* power_to_energy_per_interval_factor
).tolist()
if "weather_temp_air" in prediction.record_keys:
data["weather_temp_air"] = (
prediction.key_to_array(
key="weather_temp_air",
start_datetime=start_datetime,
end_datetime=end_datetime,
interval=to_duration(f"{interval_hours} hours"),
fill_method="linear",
)
).tolist()
solution = OptimizationSolution(
id=f"optimization-genetic@{to_datetime(as_string=True)}",
generated_at=to_datetime(),
comment="Optimization solution derived from GeneticSolution.",
valid_from=start_datetime,
valid_until=start_datetime.add(hours=config.optimization.horizon_hours),
total_losses_energy_wh=self.result.Gesamt_Verluste,
total_revenues_amt=self.result.Gesamteinnahmen_Euro,
total_costs_amt=self.result.Gesamtkosten_Euro,
data=PydanticDateTimeDataFrame.from_dataframe(data),
)
return solution
def energy_management_plan(self) -> EnergyManagementPlan:
"""Provide the genetic solution as an energy management plan."""
from akkudoktoreos.core.ems import get_ems
start_datetime = get_ems().start_datetime
plan = EnergyManagementPlan(
id=f"plan-genetic@{to_datetime(as_string=True)}",
generated_at=to_datetime(),
instructions=[],
comment="Energy management plan derived from GeneticSolution.",
)
# Add battery instructions (fill rate based control)
last_operation_mode: Optional[str] = None
last_operation_mode_factor: Optional[float] = None
resource_id = "battery1"
logger.debug("BAT: {} - {}", resource_id, self.ac_charge)
for hour, rate in enumerate(self.ac_charge):
operation_mode, operation_mode_factor = self._battery_operation_from_solution(
self.ac_charge[hour], self.dc_charge[hour], bool(self.discharge_allowed[hour])
)
if (
operation_mode == last_operation_mode
and operation_mode_factor == last_operation_mode_factor
):
# Skip, we already added the instruction
continue
last_operation_mode = operation_mode
last_operation_mode_factor = operation_mode_factor
execution_time = start_datetime.add(hours=hour)
plan.add_instruction(
FRBCInstruction(
resource_id=resource_id,
execution_time=execution_time,
actuator_id=resource_id,
operation_mode_id=operation_mode,
operation_mode_factor=operation_mode_factor,
)
)
# Add EV battery instructions (fill rate based control)
if self.eauto_obj:
resource_id = self.eauto_obj.device_id
if self.eautocharge_hours_float is None:
# Electric vehicle is full enough. No load times.
logger.debug("EV: {} - SoC >= min, no optimization", resource_id)
plan.add_instruction(
FRBCInstruction(
resource_id=resource_id,
execution_time=start_datetime,
actuator_id=resource_id,
operation_mode_id=BatteryOperationMode.IDLE,
operation_mode_factor=1.0,
)
)
else:
last_operation_mode = None
last_operation_mode_factor = None
logger.debug("EV: {} - {}", resource_id, self.eauto_obj.charge_array)
for hour, rate in enumerate(self.eautocharge_hours_float):
operation_mode, operation_mode_factor = self._battery_operation_from_solution(
rate, 0.0, False
)
if (
operation_mode == last_operation_mode
and operation_mode_factor == last_operation_mode_factor
):
# Skip, we already added the instruction
continue
last_operation_mode = operation_mode
last_operation_mode_factor = operation_mode_factor
execution_time = start_datetime.add(hours=hour)
plan.add_instruction(
FRBCInstruction(
resource_id=resource_id,
execution_time=execution_time,
actuator_id=resource_id,
operation_mode_id=operation_mode,
operation_mode_factor=operation_mode_factor,
)
)
# Add home appliance instructions (demand driven based control)
if self.washingstart:
resource_id = "homeappliance1"
operation_mode = ApplianceOperationMode.RUN # type: ignore[assignment]
operation_mode_factor = 1.0
execution_time = start_datetime.add(hours=self.washingstart)
plan.add_instruction(
DDBCInstruction(
resource_id=resource_id,
execution_time=execution_time,
actuator_id=resource_id,
operation_mode_id=operation_mode,
operation_mode_factor=operation_mode_factor,
)
)
return plan

126
src/akkudoktoreos/optimization/optimization.py
View File

@@ -1,37 +1,111 @@
from typing import List, Optional
from typing import Optional, Union
from pydantic import Field
from akkudoktoreos.config.configabc import SettingsBaseModel
from akkudoktoreos.core.pydantic import PydanticBaseModel, PydanticDateTimeDataFrame
from akkudoktoreos.utils.datetimeutil import DateTime
class GeneticCommonSettings(SettingsBaseModel):
"""General Genetic Optimization Algorithm Configuration."""
individuals: Optional[int] = Field(
default=300,
ge=10,
description="Number of individuals (solutions) to generate for the (initial) generation [>= 10]. Defaults to 300.",
examples=[300],
)
generations: Optional[int] = Field(
default=400,
ge=10,
description="Number of generations to evaluate the optimal solution [>= 10]. Defaults to 400.",
examples=[400],
)
seed: Optional[int] = Field(
default=None,
ge=0,
description="Fixed seed for genetic algorithm. Defaults to 'None' which means random seed.",
examples=[None],
)
penalties: Optional[dict[str, Union[float, int, str]]] = Field(
default=None,
description="A dictionary of penalty function parameters consisting of a penalty function parameter name and the associated value.",
examples=[
{"ev_soc_miss": 10},
],
)
class OptimizationCommonSettings(SettingsBaseModel):
"""General Optimization Configuration.
"""General Optimization Configuration."""
Attributes:
hours (int): Number of hours for optimizations.
"""
hours: Optional[int] = Field(
default=48, ge=0, description="Number of hours into the future for optimizations."
horizon_hours: Optional[int] = Field(
default=24,
ge=0,
description="The general time window within which the energy optimization goal shall be achieved [h]. Defaults to 24 hours.",
examples=[24],
)
penalty: Optional[int] = Field(default=10, description="Penalty factor used in optimization.")
ev_available_charge_rates_percent: Optional[List[float]] = Field(
default=[
0.0,
6.0 / 16.0,
# 7.0 / 16.0,
8.0 / 16.0,
# 9.0 / 16.0,
10.0 / 16.0,
# 11.0 / 16.0,
12.0 / 16.0,
# 13.0 / 16.0,
14.0 / 16.0,
# 15.0 / 16.0,
1.0,
],
description="Charge rates available for the EV in percent of maximum charge.",
interval: Optional[int] = Field(
default=3600,
ge=15 * 60,
le=60 * 60,
description="The optimization interval [sec].",
examples=[60 * 60, 15 * 60],
)
genetic: Optional[GeneticCommonSettings] = Field(
default=None,
description="Genetic optimization algorithm configuration.",
examples=[{"individuals": 400, "seed": None, "penalties": {"ev_soc_miss": 10}}],
)
class OptimizationSolution(PydanticBaseModel):
"""General Optimization Solution."""
id: str = Field(..., description="Unique ID for the optimization solution.")
generated_at: DateTime = Field(..., description="Timestamp when the solution was generated.")
comment: Optional[str] = Field(
default=None, description="Optional comment or annotation for the solution."
)
valid_from: Optional[DateTime] = Field(
default=None, description="Start time of the optimization solution."
)
valid_until: Optional[DateTime] = Field(
default=None,
description="End time of the optimization solution.",
)
total_losses_energy_wh: float = Field(
description="The total losses in watt-hours over the entire period."
)
total_revenues_amt: float = Field(description="The total revenues [money amount].")
total_costs_amt: float = Field(description="The total costs [money amount].")
data: PydanticDateTimeDataFrame = Field(
description=(
"Datetime data frame with time series optimization data per optimization interval:"
"- load_energy_wh: Load of all energy consumers in wh"
"- grid_energy_wh: Grid energy feed in (negative) or consumption (positive) in wh"
"- pv_prediction_energy_wh: PV energy prediction (positive) in wh"
"- elec_price_prediction_amt_kwh: Electricity price prediction in money per kwh"
"- costs_amt: Costs in money amount"
"- revenue_amt: Revenue in money amount"
"- losses_energy_wh: Energy losses in wh"
"- <device-id>_operation_mode_id: Operation mode id of the device."
"- <device-id>_operation_mode_factor: Operation mode factor of the device."
"- <device-id>_soc_factor: State of charge of a battery/ electric vehicle device as factor of total capacity."
"- <device-id>_energy_wh: Energy consumption (positive) of a device in wh."
)
)

9
src/akkudoktoreos/optimization/optimizationabc.py
View File

@@ -2,11 +2,14 @@
from pydantic import ConfigDict
from akkudoktoreos.core.coreabc import ConfigMixin, PredictionMixin
from akkudoktoreos.core.pydantic import PydanticBaseModel
from akkudoktoreos.core.coreabc import (
ConfigMixin,
EnergyManagementSystemMixin,
PredictionMixin,
)
class OptimizationBase(ConfigMixin, PredictionMixin, PydanticBaseModel):
class OptimizationBase(ConfigMixin, PredictionMixin, EnergyManagementSystemMixin):
"""Base class for handling optimization data.
Enables access to EOS configuration data (attribute `config`) and EOS prediction data (attribute

24
src/akkudoktoreos/prediction/elecprice.py
View File

@@ -17,6 +17,14 @@ elecprice_providers = [
]
class ElecPriceCommonProviderSettings(SettingsBaseModel):
"""Electricity Price Prediction Provider Configuration."""
ElecPriceImport: Optional[ElecPriceImportCommonSettings] = Field(
default=None, description="ElecPriceImport settings", examples=[None]
)
class ElecPriceCommonSettings(SettingsBaseModel):
"""Electricity Price Prediction Configuration."""
@@ -26,7 +34,10 @@ class ElecPriceCommonSettings(SettingsBaseModel):
examples=["ElecPriceAkkudoktor"],
)
charges_kwh: Optional[float] = Field(
default=None, ge=0, description="Electricity price charges (€/kWh).", examples=[0.21]
default=None,
ge=0,
description="Electricity price charges [€/kWh]. Will be added to variable market price.",
examples=[0.21],
)
vat_rate: Optional[float] = Field(
default=1.19,
@@ -35,8 +46,15 @@ class ElecPriceCommonSettings(SettingsBaseModel):
examples=[1.19],
)
provider_settings: Optional[ElecPriceImportCommonSettings] = Field(
default=None, description="Provider settings", examples=[None]
provider_settings: ElecPriceCommonProviderSettings = Field(
default_factory=ElecPriceCommonProviderSettings,
description="Provider settings",
examples=[
# Example 1: Empty/default settings (all providers None)
{
"ElecPriceImport": None,
},
],
)
# Validators

16
src/akkudoktoreos/prediction/elecpriceakkudoktor.py
View File

@@ -102,10 +102,10 @@ class ElecPriceAkkudoktor(ElecPriceProvider):
- add the file cache again.
"""
source = "https://api.akkudoktor.net"
if not self.start_datetime:
raise ValueError(f"Start DateTime not set: {self.start_datetime}")
if not self.ems_start_datetime:
raise ValueError(f"Start DateTime not set: {self.ems_start_datetime}")
# Try to take data from 5 weeks back for prediction
date = to_datetime(self.start_datetime - to_duration("35 days"), as_string="YYYY-MM-DD")
date = to_datetime(self.ems_start_datetime - to_duration("35 days"), as_string="YYYY-MM-DD")
last_date = to_datetime(self.end_datetime, as_string="YYYY-MM-DD")
url = f"{source}/prices?start={date}&end={last_date}&tz={self.config.general.timezone}"
response = requests.get(url, timeout=10)
@@ -147,8 +147,8 @@ class ElecPriceAkkudoktor(ElecPriceProvider):
"""
# Get Akkudoktor electricity price data
akkudoktor_data = self._request_forecast(force_update=force_update) # type: ignore
if not self.start_datetime:
raise ValueError(f"Start DateTime not set: {self.start_datetime}")
if not self.ems_start_datetime:
raise ValueError(f"Start DateTime not set: {self.ems_start_datetime}")
# Assumption that all lists are the same length and are ordered chronologically
# in ascending order and have the same timestamps.
@@ -186,13 +186,13 @@ class ElecPriceAkkudoktor(ElecPriceProvider):
# some of our data is already in the future, so we need to predict less. If we got less data we increase the prediction hours
needed_hours = int(
self.config.prediction.hours
- ((highest_orig_datetime - self.start_datetime).total_seconds() // 3600)
- ((highest_orig_datetime - self.ems_start_datetime).total_seconds() // 3600)
)
if needed_hours <= 0:
logger.warning(
f"No prediction needed. needed_hours={needed_hours}, hours={self.config.prediction.hours},highest_orig_datetime {highest_orig_datetime}, start_datetime {self.start_datetime}"
) # this might keep data longer than self.start_datetime + self.config.prediction.hours in the records
f"No prediction needed. needed_hours={needed_hours}, hours={self.config.prediction.hours},highest_orig_datetime {highest_orig_datetime}, start_datetime {self.ems_start_datetime}"
) # this might keep data longer than self.ems_start_datetime + self.config.prediction.hours in the records
return
if amount_datasets > 800: # we do the full ets with seasons of 1 week

18
src/akkudoktoreos/prediction/elecpriceenergycharts.py
View File

@@ -91,7 +91,7 @@ class ElecPriceEnergyCharts(ElecPriceProvider):
if start_date is None:
# Try to take data from 5 weeks back for prediction
start_date = to_datetime(
self.start_datetime - to_duration("35 days"), as_string="YYYY-MM-DD"
self.ems_start_datetime - to_duration("35 days"), as_string="YYYY-MM-DD"
)
last_date = to_datetime(self.end_datetime, as_string="YYYY-MM-DD")
@@ -172,17 +172,17 @@ class ElecPriceEnergyCharts(ElecPriceProvider):
hours_ahead = 23 if now.time() < pd.Timestamp("14:00").time() else 47
end = midnight + pd.Timedelta(hours=hours_ahead)
if not self.start_datetime:
raise ValueError(f"Start DateTime not set: {self.start_datetime}")
if not self.ems_start_datetime:
raise ValueError(f"Start DateTime not set: {self.ems_start_datetime}")
# Determine if update is needed and how many days
past_days = 35
if self.highest_orig_datetime:
history_series = self.key_to_series(
key="elecprice_marketprice_wh", start_datetime=self.start_datetime
key="elecprice_marketprice_wh", start_datetime=self.ems_start_datetime
)
# If history lower, then start_datetime
if history_series.index.min() <= self.start_datetime:
if history_series.index.min() <= self.ems_start_datetime:
past_days = 0
needs_update = end > self.highest_orig_datetime
@@ -195,7 +195,7 @@ class ElecPriceEnergyCharts(ElecPriceProvider):
)
# Set start_date try to take data from 5 weeks back for prediction
start_date = to_datetime(
self.start_datetime - to_duration(f"{past_days} days"), as_string="YYYY-MM-DD"
self.ems_start_datetime - to_duration(f"{past_days} days"), as_string="YYYY-MM-DD"
)
# Get Energy-Charts electricity price data
energy_charts_data = self._request_forecast(
@@ -227,13 +227,13 @@ class ElecPriceEnergyCharts(ElecPriceProvider):
# some of our data is already in the future, so we need to predict less. If we got less data we increase the prediction hours
needed_hours = int(
self.config.prediction.hours
- ((self.highest_orig_datetime - self.start_datetime).total_seconds() // 3600)
- ((self.highest_orig_datetime - self.ems_start_datetime).total_seconds() // 3600)
)
if needed_hours <= 0:
logger.warning(
f"No prediction needed. needed_hours={needed_hours}, hours={self.config.prediction.hours},highest_orig_datetime {self.highest_orig_datetime}, start_datetime {self.start_datetime}"
) # this might keep data longer than self.start_datetime + self.config.prediction.hours in the records
f"No prediction needed. needed_hours={needed_hours}, hours={self.config.prediction.hours},highest_orig_datetime {self.highest_orig_datetime}, start_datetime {self.ems_start_datetime}"
) # this might keep data longer than self.ems_start_datetime + self.config.prediction.hours in the records
return
if amount_datasets > 800: # we do the full ets with seasons of 1 week

11
src/akkudoktoreos/prediction/elecpriceimport.py
View File

@@ -61,15 +61,16 @@ class ElecPriceImport(ElecPriceProvider, PredictionImportProvider):
return "ElecPriceImport"
def _update_data(self, force_update: Optional[bool] = False) -> None:
if self.config.elecprice.provider_settings is None:
if self.config.elecprice.provider_settings.ElecPriceImport is None:
logger.debug(f"{self.provider_id()} data update without provider settings.")
return
if self.config.elecprice.provider_settings.import_file_path:
if self.config.elecprice.provider_settings.ElecPriceImport.import_file_path:
self.import_from_file(
self.config.elecprice.provider_settings.import_file_path,
self.config.elecprice.provider_settings.ElecPriceImport.import_file_path,
key_prefix="elecprice",
)
if self.config.elecprice.provider_settings.import_json:
if self.config.elecprice.provider_settings.ElecPriceImport.import_json:
self.import_from_json(
self.config.elecprice.provider_settings.import_json, key_prefix="elecprice"
self.config.elecprice.provider_settings.ElecPriceImport.import_json,
key_prefix="elecprice",
)

61
src/akkudoktoreos/prediction/feedintariff.py Normal file
View File

@@ -0,0 +1,61 @@
from typing import Optional
from pydantic import Field, field_validator
from akkudoktoreos.config.configabc import SettingsBaseModel
from akkudoktoreos.prediction.feedintariffabc import FeedInTariffProvider
from akkudoktoreos.prediction.feedintarifffixed import FeedInTariffFixedCommonSettings
from akkudoktoreos.prediction.feedintariffimport import FeedInTariffImportCommonSettings
from akkudoktoreos.prediction.prediction import get_prediction
prediction_eos = get_prediction()
# Valid feedintariff providers
feedintariff_providers = [
provider.provider_id()
for provider in prediction_eos.providers
if isinstance(provider, FeedInTariffProvider)
]
class FeedInTariffCommonProviderSettings(SettingsBaseModel):
"""Feed In Tariff Prediction Provider Configuration."""
FeedInTariffFixed: Optional[FeedInTariffFixedCommonSettings] = Field(
default=None, description="FeedInTariffFixed settings", examples=[None]
)
FeedInTariffImport: Optional[FeedInTariffImportCommonSettings] = Field(
default=None, description="FeedInTariffImport settings", examples=[None]
)
class FeedInTariffCommonSettings(SettingsBaseModel):
"""Feed In Tariff Prediction Configuration."""
provider: Optional[str] = Field(
default=None,
description="Feed in tariff provider id of provider to be used.",
examples=["FeedInTariffFixed", "FeedInTarifImport"],
)
provider_settings: FeedInTariffCommonProviderSettings = Field(
default_factory=FeedInTariffCommonProviderSettings,
description="Provider settings",
examples=[
# Example 1: Empty/default settings (all providers None)
{
"FeedInTariffFixed": None,
"FeedInTariffImport": None,
},
],
)
# Validators
@field_validator("provider", mode="after")
@classmethod
def validate_provider(cls, value: Optional[str]) -> Optional[str]:
if value is None or value in feedintariff_providers:
return value
raise ValueError(
f"Provider '{value}' is not a valid feed in tariff provider: {feedintariff_providers}."
)

58
src/akkudoktoreos/prediction/feedintariffabc.py Normal file
View File

@@ -0,0 +1,58 @@
"""Abstract and base classes for feed in tariff predictions.
Notes:
- Ensure appropriate API keys or configurations are set up if required by external data sources.
"""
from abc import abstractmethod
from typing import List, Optional
from pydantic import Field, computed_field
from akkudoktoreos.prediction.predictionabc import PredictionProvider, PredictionRecord
class FeedInTariffDataRecord(PredictionRecord):
"""Represents a feed in tariff data record containing various price attributes at a specific datetime.
Attributes:
date_time (Optional[AwareDatetime]): The datetime of the record.
"""
feed_in_tariff_wh: Optional[float] = Field(None, description="Feed in tariff per Wh (€/Wh)")
# Computed fields
@computed_field # type: ignore[prop-decorator]
@property
def feed_in_tariff_kwh(self) -> Optional[float]:
"""Feed in tariff per kWh (€/kWh).
Convenience attribute calculated from `feed_in_tariff_wh`.
"""
if self.feed_in_tariff_wh is None:
return None
return self.feed_in_tariff_wh * 1000.0
class FeedInTariffProvider(PredictionProvider):
"""Abstract base class for feed in tariff providers.
FeedInTariffProvider is a thread-safe singleton, ensuring only one instance of this class is created.
Configuration variables:
feed in tariff_provider (str): Prediction provider for feed in tarif.
"""
# overload
records: List[FeedInTariffDataRecord] = Field(
default_factory=list, description="List of FeedInTariffDataRecord records"
)
@classmethod
@abstractmethod
def provider_id(cls) -> str:
return "FeedInTariffProvider"
def enabled(self) -> bool:
return self.provider_id() == self.config.feedintariff.provider

48
src/akkudoktoreos/prediction/feedintarifffixed.py Normal file
View File

@@ -0,0 +1,48 @@
"""Provides feed in tariff data."""
from typing import Optional
from loguru import logger
from pydantic import Field
from akkudoktoreos.config.configabc import SettingsBaseModel
from akkudoktoreos.prediction.feedintariffabc import FeedInTariffProvider
from akkudoktoreos.utils.datetimeutil import to_datetime
class FeedInTariffFixedCommonSettings(SettingsBaseModel):
"""Common settings for elecprice fixed price."""
feed_in_tariff_kwh: Optional[float] = Field(
default=None,
ge=0,
description="Electricity price feed in tariff [€/kWH].",
examples=[0.078],
)
class FeedInTariffFixed(FeedInTariffProvider):
"""Fixed price feed in tariff data.
FeedInTariffFixed is a singleton-based class that retrieves elecprice data.
"""
@classmethod
def provider_id(cls) -> str:
"""Return the unique identifier for the FeedInTariffFixed provider."""
return "FeedInTariffFixed"
def _update_data(self, force_update: Optional[bool] = False) -> None:
error_msg = "Feed in tariff not provided"
try:
feed_in_tariff = (
self.config.feedintariff.provider_settings.FeedInTariffFixed.feed_in_tariff_kwh
)
except:
logger.exception(error_msg)
raise ValueError(error_msg)
if feed_in_tariff is None:
logger.error(error_msg)
raise ValueError(error_msg)
feed_in_tariff_wh = feed_in_tariff / 1000
self.update_value(to_datetime(), "feed_in_tariff_wh", feed_in_tariff_wh)

76
src/akkudoktoreos/prediction/feedintariffimport.py Normal file
View File

@@ -0,0 +1,76 @@
"""Retrieves feed in tariff forecast data from an import file.
This module provides classes and mappings to manage feed in tariff data obtained from
an import file. The data is mapped to the `FeedInTariffDataRecord` format, enabling consistent
access to forecasted and historical feed in tariff attributes.
"""
from pathlib import Path
from typing import Optional, Union
from loguru import logger
from pydantic import Field, field_validator
from akkudoktoreos.config.configabc import SettingsBaseModel
from akkudoktoreos.prediction.feedintariffabc import FeedInTariffProvider
from akkudoktoreos.prediction.predictionabc import PredictionImportProvider
class FeedInTariffImportCommonSettings(SettingsBaseModel):
"""Common settings for feed in tariff data import from file or JSON string."""
import_file_path: Optional[Union[str, Path]] = Field(
default=None,
description="Path to the file to import feed in tariff data from.",
examples=[None, "/path/to/feedintariff.json"],
)
import_json: Optional[str] = Field(
default=None,
description="JSON string, dictionary of feed in tariff forecast value lists.",
examples=['{"fead_in_tariff_wh": [0.000078, 0.000078, 0.000023]}'],
)
# Validators
@field_validator("import_file_path", mode="after")
@classmethod
def validate_feedintariffimport_file_path(
cls, value: Optional[Union[str, Path]]
) -> Optional[Path]:
if value is None:
return None
if isinstance(value, str):
value = Path(value)
"""Ensure file is available."""
value.resolve()
if not value.is_file():
raise ValueError(f"Import file path '{value}' is not a file.")
return value
class FeedInTariffImport(FeedInTariffProvider, PredictionImportProvider):
"""Fetch Feed In Tariff data from import file or JSON string.
FeedInTariffImport is a singleton-based class that retrieves fedd in tariff forecast data
from a file or JSON string and maps it to `FeedInTariffDataRecord` fields. It manages the forecast
over a range of hours into the future and retains historical data.
"""
@classmethod
def provider_id(cls) -> str:
"""Return the unique identifier for the FeedInTariffImport provider."""
return "FeedInTariffImport"
def _update_data(self, force_update: Optional[bool] = False) -> None:
if self.config.feedintariff.provider_settings.FeedInTariffImport is None:
logger.debug(f"{self.provider_id()} data update without provider settings.")
return
if self.config.feedintariff.provider_settings.FeedInTariffImport.import_file_path:
self.import_from_file(
self.config.provider_settings.FeedInTariffImport.import_file_path,
key_prefix="feedintariff",
)
if self.config.feedintariff.provider_settings.FeedInTariffImport.import_json:
self.import_from_json(
self.config.feedintariff.provider_settings.FeedInTariffImport.import_json,
key_prefix="feedintariff",
)

19
src/akkudoktoreos/prediction/interpolator.py
View File

@@ -1,11 +1,11 @@
#!/usr/bin/env python
import pickle
from functools import lru_cache
from pathlib import Path
import numpy as np
from scipy.interpolate import RegularGridInterpolator
from akkudoktoreos.core.cache import cachemethod_energy_management
from akkudoktoreos.core.coreabc import SingletonMixin
@@ -16,8 +16,7 @@ class SelfConsumptionProbabilityInterpolator:
with open(self.filepath, "rb") as file:
self.interpolator: RegularGridInterpolator = pickle.load(file) # noqa: S301
@lru_cache(maxsize=128)
def generate_points(
def _generate_points(
self, load_1h_power: float, pv_power: float
) -> tuple[np.ndarray, np.ndarray]:
"""Generate the grid points for interpolation."""
@@ -25,8 +24,20 @@ class SelfConsumptionProbabilityInterpolator:
points = np.array([np.full_like(partial_loads, load_1h_power), partial_loads]).T
return points, partial_loads
@cachemethod_energy_management
def calculate_self_consumption(self, load_1h_power: float, pv_power: float) -> float:
points, partial_loads = self.generate_points(load_1h_power, pv_power)
"""Calculate the PV self-consumption rate using RegularGridInterpolator.
The results are cached until the start of the next energy management run/ optimization.
Args:
- last_1h_power: 1h power levels (W).
- pv_power: Current PV power output (W).
Returns:
- Self-consumption rate as a float.
"""
points, partial_loads = self._generate_points(load_1h_power, pv_power)
probabilities = self.interpolator(points)
return probabilities.sum()

31
src/akkudoktoreos/prediction/load.py
View File

@@ -1,6 +1,6 @@
"""Load forecast module for load predictions."""
from typing import Optional, Union
from typing import Optional
from pydantic import Field, field_validator
@@ -21,6 +21,20 @@ load_providers = [
]
class LoadCommonProviderSettings(SettingsBaseModel):
"""Load Prediction Provider Configuration."""
LoadAkkudoktor: Optional[LoadAkkudoktorCommonSettings] = Field(
default=None, description="LoadAkkudoktor settings", examples=[None]
)
LoadVrm: Optional[LoadVrmCommonSettings] = Field(
default=None, description="LoadVrm settings", examples=[None]
)
LoadImport: Optional[LoadImportCommonSettings] = Field(
default=None, description="LoadImport settings", examples=[None]
)
class LoadCommonSettings(SettingsBaseModel):
"""Load Prediction Configuration."""
@@ -30,9 +44,18 @@ class LoadCommonSettings(SettingsBaseModel):
examples=["LoadAkkudoktor"],
)
provider_settings: Optional[
Union[LoadAkkudoktorCommonSettings, LoadVrmCommonSettings, LoadImportCommonSettings]
] = Field(default=None, description="Provider settings", examples=[None])
provider_settings: LoadCommonProviderSettings = Field(
default_factory=LoadCommonProviderSettings,
description="Provider settings",
examples=[
# Example 1: Empty/default settings (all providers None)
{
"LoadAkkudoktor": None,
"LoadVrm": None,
"LoadImport": None,
},
],
)
# Validators
@field_validator("provider", mode="after")

8
src/akkudoktoreos/prediction/loadakkudoktor.py
View File

@@ -90,7 +90,9 @@ class LoadAkkudoktor(LoadProvider):
)
# Calculate values in W by relative profile data and yearly consumption given in kWh
data_year_energy = (
profile_data * self.config.load.provider_settings.loadakkudoktor_year_energy * 1000
profile_data
* self.config.load.provider_settings.LoadAkkudoktor.loadakkudoktor_year_energy
* 1000
)
except FileNotFoundError:
error_msg = f"Error: File {load_file} not found."
@@ -108,8 +110,8 @@ class LoadAkkudoktor(LoadProvider):
weekday_adjust, weekend_adjust = self._calculate_adjustment(data_year_energy)
# We provide prediction starting at start of day, to be compatible to old system.
# End date for prediction is prediction hours from now.
date = self.start_datetime.start_of("day")
end_date = self.start_datetime.add(hours=self.config.prediction.hours)
date = self.ems_start_datetime.start_of("day")
end_date = self.ems_start_datetime.add(hours=self.config.prediction.hours)
while compare_datetimes(date, end_date).lt:
# Extract mean (index 0) and standard deviation (index 1) for the given day and hour
# Day indexing starts at 0, -1 because of that

14
src/akkudoktoreos/prediction/loadimport.py
View File

@@ -60,10 +60,14 @@ class LoadImport(LoadProvider, PredictionImportProvider):
return "LoadImport"
def _update_data(self, force_update: Optional[bool] = False) -> None:
if self.config.load.provider_settings is None:
if self.config.load.provider_settings.LoadImport is None:
logger.debug(f"{self.provider_id()} data update without provider settings.")
return
if self.config.load.provider_settings.import_file_path:
self.import_from_file(self.config.provider_settings.import_file_path, key_prefix="load")
if self.config.load.provider_settings.import_json:
self.import_from_json(self.config.load.provider_settings.import_json, key_prefix="load")
if self.config.load.provider_settings.LoadImport.import_file_path:
self.import_from_file(
self.config.provider_settings.LoadImport.import_file_path, key_prefix="load"
)
if self.config.load.provider_settings.LoadImport.import_json:
self.import_from_json(
self.config.load.provider_settings.LoadImport.import_json, key_prefix="load"
)

11
src/akkudoktoreos/prediction/loadvrm.py
View File

@@ -4,13 +4,12 @@ from typing import Any, Optional, Union
import requests
from loguru import logger
from pendulum import DateTime
from pydantic import Field, ValidationError
from akkudoktoreos.config.configabc import SettingsBaseModel
from akkudoktoreos.core.pydantic import PydanticBaseModel
from akkudoktoreos.prediction.loadabc import LoadProvider
from akkudoktoreos.utils.datetimeutil import to_datetime
from akkudoktoreos.utils.datetimeutil import DateTime, to_datetime
class VrmForecastRecords(PydanticBaseModel):
@@ -57,8 +56,8 @@ class LoadVrm(LoadProvider):
def _request_forecast(self, start_ts: int, end_ts: int) -> VrmForecastResponse:
"""Fetch forecast data from Victron VRM API."""
base_url = "https://vrmapi.victronenergy.com/v2/installations"
installation_id = self.config.load.provider_settings.load_vrm_idsite
api_token = self.config.load.provider_settings.load_vrm_token
installation_id = self.config.load.provider_settings.LoadVrm.load_vrm_idsite
api_token = self.config.load.provider_settings.LoadVrm.load_vrm_token
url = f"{base_url}/{installation_id}/stats?type=forecast&start={start_ts}&end={end_ts}&interval=hours"
headers = {"X-Authorization": f"Token {api_token}", "Content-Type": "application/json"}
@@ -80,8 +79,8 @@ class LoadVrm(LoadProvider):
def _update_data(self, force_update: Optional[bool] = False) -> None:
"""Fetch and store VRM load forecast as load_mean and related values."""
start_date = self.start_datetime.start_of("day")
end_date = self.start_datetime.add(hours=self.config.prediction.hours)
start_date = self.ems_start_datetime.start_of("day")
end_date = self.ems_start_datetime.add(hours=self.config.prediction.hours)
start_ts = int(start_date.timestamp())
end_ts = int(end_date.timestamp())

9
src/akkudoktoreos/prediction/prediction.py
View File

@@ -34,6 +34,8 @@ from akkudoktoreos.config.configabc import SettingsBaseModel
from akkudoktoreos.prediction.elecpriceakkudoktor import ElecPriceAkkudoktor
from akkudoktoreos.prediction.elecpriceenergycharts import ElecPriceEnergyCharts
from akkudoktoreos.prediction.elecpriceimport import ElecPriceImport
from akkudoktoreos.prediction.feedintarifffixed import FeedInTariffFixed
from akkudoktoreos.prediction.feedintariffimport import FeedInTariffImport
from akkudoktoreos.prediction.loadakkudoktor import LoadAkkudoktor
from akkudoktoreos.prediction.loadimport import LoadImport
from akkudoktoreos.prediction.loadvrm import LoadVrm
@@ -67,6 +69,7 @@ class PredictionCommonSettings(SettingsBaseModel):
hours: Optional[int] = Field(
default=48, ge=0, description="Number of hours into the future for predictions"
)
historic_hours: Optional[int] = Field(
default=48,
ge=0,
@@ -88,6 +91,8 @@ class Prediction(PredictionContainer):
ElecPriceAkkudoktor,
ElecPriceEnergyCharts,
ElecPriceImport,
FeedInTariffFixed,
FeedInTariffImport,
LoadAkkudoktor,
LoadVrm,
LoadImport,
@@ -105,6 +110,8 @@ class Prediction(PredictionContainer):
elecprice_akkudoktor = ElecPriceAkkudoktor()
elecprice_energy_charts = ElecPriceEnergyCharts()
elecprice_import = ElecPriceImport()
feedintariff_fixed = FeedInTariffFixed()
feedintariff_import = FeedInTariffImport()
load_akkudoktor = LoadAkkudoktor()
load_vrm = LoadVrm()
load_import = LoadImport()
@@ -125,6 +132,8 @@ def get_prediction() -> Prediction:
elecprice_akkudoktor,
elecprice_energy_charts,
elecprice_import,
feedintariff_fixed,
feedintariff_import,
load_akkudoktor,
load_vrm,
load_import,

25
src/akkudoktoreos/prediction/predictionabc.py
View File

@@ -11,7 +11,6 @@ and manipulation of configuration and prediction data in a clear, scalable, and
from typing import List, Optional
from loguru import logger
from pendulum import DateTime
from pydantic import Field, computed_field
from akkudoktoreos.core.coreabc import MeasurementMixin
@@ -23,7 +22,7 @@ from akkudoktoreos.core.dataabc import (
DataRecord,
DataSequence,
)
from akkudoktoreos.utils.datetimeutil import to_duration
from akkudoktoreos.utils.datetimeutil import DateTime, to_duration
class PredictionBase(DataBase, MeasurementMixin):
@@ -119,17 +118,21 @@ class PredictionStartEndKeepMixin(PredictionBase):
Returns:
Optional[DateTime]: The calculated end datetime, or `None` if inputs are missing.
"""
if self.start_datetime and self.config.prediction.hours:
end_datetime = self.start_datetime + to_duration(
if self.ems_start_datetime and self.config.prediction.hours:
end_datetime = self.ems_start_datetime + to_duration(
f"{self.config.prediction.hours} hours"
)
dst_change = end_datetime.offset_hours - self.start_datetime.offset_hours
logger.debug(f"Pre: {self.start_datetime}..{end_datetime}: DST change: {dst_change}")
dst_change = end_datetime.offset_hours - self.ems_start_datetime.offset_hours
logger.debug(
f"Pre: {self.ems_start_datetime}..{end_datetime}: DST change: {dst_change}"
)
if dst_change < 0:
end_datetime = end_datetime + to_duration(f"{abs(int(dst_change))} hours")
elif dst_change > 0:
end_datetime = end_datetime - to_duration(f"{abs(int(dst_change))} hours")
logger.debug(f"Pst: {self.start_datetime}..{end_datetime}: DST change: {dst_change}")
logger.debug(
f"Pst: {self.ems_start_datetime}..{end_datetime}: DST change: {dst_change}"
)
return end_datetime
return None
@@ -141,7 +144,7 @@ class PredictionStartEndKeepMixin(PredictionBase):
Returns:
Optional[DateTime]: The calculated retention cutoff datetime, or `None` if inputs are missing.
"""
if self.start_datetime is None:
if self.ems_start_datetime is None:
return None
historic_hours = self.historic_hours_min()
if (
@@ -149,7 +152,7 @@ class PredictionStartEndKeepMixin(PredictionBase):
and self.config.prediction.historic_hours > historic_hours
):
historic_hours = int(self.config.prediction.historic_hours)
return self.start_datetime - to_duration(f"{historic_hours} hours")
return self.ems_start_datetime - to_duration(f"{historic_hours} hours")
@computed_field # type: ignore[prop-decorator]
@property
@@ -162,7 +165,7 @@ class PredictionStartEndKeepMixin(PredictionBase):
end_dt = self.end_datetime
if end_dt is None:
return None
duration = end_dt - self.start_datetime
duration = end_dt - self.ems_start_datetime
return int(duration.total_hours())
@computed_field # type: ignore[prop-decorator]
@@ -176,7 +179,7 @@ class PredictionStartEndKeepMixin(PredictionBase):
keep_dt = self.keep_datetime
if keep_dt is None:
return None
duration = self.start_datetime - keep_dt
duration = self.ems_start_datetime - keep_dt
return int(duration.total_hours())

72
src/akkudoktoreos/prediction/pvforecast.py
View File

@@ -1,6 +1,6 @@
"""PV forecast module for PV power predictions."""
from typing import Any, List, Optional, Self, Union
from typing import Any, List, Optional, Self
from pydantic import Field, computed_field, field_validator, model_validator
@@ -8,8 +8,7 @@ from akkudoktoreos.config.configabc import SettingsBaseModel
from akkudoktoreos.prediction.prediction import get_prediction
from akkudoktoreos.prediction.pvforecastabc import PVForecastProvider
from akkudoktoreos.prediction.pvforecastimport import PVForecastImportCommonSettings
from akkudoktoreos.prediction.pvforecastvrm import PVforecastVrmCommonSettings
from akkudoktoreos.utils.docs import get_model_structure_from_examples
from akkudoktoreos.prediction.pvforecastvrm import PVForecastVrmCommonSettings
prediction_eos = get_prediction()
@@ -121,6 +120,17 @@ class PVForecastPlaneSetting(SettingsBaseModel):
return pvtechchoice
class PVForecastCommonProviderSettings(SettingsBaseModel):
"""PV Forecast Provider Configuration."""
PVForecastImport: Optional[PVForecastImportCommonSettings] = Field(
default=None, description="PVForecastImport settings", examples=[None]
)
PVForecastVrm: Optional[PVForecastVrmCommonSettings] = Field(
default=None, description="PVForecastVrm settings", examples=[None]
)
class PVForecastCommonSettings(SettingsBaseModel):
"""PV Forecast Configuration."""
@@ -135,20 +145,68 @@ class PVForecastCommonSettings(SettingsBaseModel):
examples=["PVForecastAkkudoktor"],
)
provider_settings: Optional[
Union[PVForecastImportCommonSettings, PVforecastVrmCommonSettings]
] = Field(default=None, description="Provider settings", examples=[None])
provider_settings: PVForecastCommonProviderSettings = Field(
default_factory=PVForecastCommonProviderSettings,
description="Provider settings",
examples=[
# Example 1: Empty/default settings (all providers None)
{
"PVForecastImport": None,
"PVForecastVrm": None,
},
],
)
planes: Optional[list[PVForecastPlaneSetting]] = Field(
default=None,
description="Plane configuration.",
examples=[get_model_structure_from_examples(PVForecastPlaneSetting, True)],
examples=[
[
{
"surface_tilt": 10.0,
"surface_azimuth": 180.0,
"userhorizon": [10.0, 20.0, 30.0],
"peakpower": 5.0,
"pvtechchoice": "crystSi",
"mountingplace": "free",
"loss": 14.0,
"trackingtype": 0,
"optimal_surface_tilt": False,
"optimalangles": False,
"albedo": None,
"module_model": None,
"inverter_model": None,
"inverter_paco": 6000,
"modules_per_string": 20,
"strings_per_inverter": 2,
},
{
"surface_tilt": 20.0,
"surface_azimuth": 90.0,
"userhorizon": [5.0, 15.0, 25.0],
"peakpower": 3.5,
"pvtechchoice": "crystSi",
"mountingplace": "free",
"loss": 14.0,
"trackingtype": 1,
"optimal_surface_tilt": False,
"optimalangles": False,
"albedo": None,
"module_model": None,
"inverter_model": None,
"inverter_paco": 4000,
"modules_per_string": 20,
"strings_per_inverter": 2,
},
]
],
)
max_planes: Optional[int] = Field(
default=0,
ge=0,
description="Maximum number of planes that can be set",
examples=[1, 2],
)
# Validators

8
src/akkudoktoreos/prediction/pvforecastakkudoktor.py
View File

@@ -330,8 +330,8 @@ class PVForecastAkkudoktor(PVForecastProvider):
logger.error(f"Akkudoktor schema change: {error_msg}")
raise ValueError(error_msg)
if not self.start_datetime:
raise ValueError(f"Start DateTime not set: {self.start_datetime}")
if not self.ems_start_datetime:
raise ValueError(f"Start DateTime not set: {self.ems_start_datetime}")
# Iterate over forecast data points
for forecast_values in zip(*akkudoktor_data.values):
@@ -339,7 +339,7 @@ class PVForecastAkkudoktor(PVForecastProvider):
dt = to_datetime(original_datetime, in_timezone=self.config.general.timezone)
# Skip outdated forecast data
if compare_datetimes(dt, self.start_datetime.start_of("day")).lt:
if compare_datetimes(dt, self.ems_start_datetime.start_of("day")).lt:
continue
sum_dc_power = sum(values.dcPower for values in forecast_values)
@@ -357,7 +357,7 @@ class PVForecastAkkudoktor(PVForecastProvider):
if len(self) < self.config.prediction.hours:
raise ValueError(
f"The forecast must cover at least {self.config.prediction.hours} hours, "
f"but only {len(self)} hours starting from {self.start_datetime} "
f"but only {len(self)} hours starting from {self.ems_start_datetime} "
f"were predicted."
)

10
src/akkudoktoreos/prediction/pvforecastimport.py
View File

@@ -61,16 +61,16 @@ class PVForecastImport(PVForecastProvider, PredictionImportProvider):
return "PVForecastImport"
def _update_data(self, force_update: Optional[bool] = False) -> None:
if self.config.pvforecast.provider_settings is None:
if self.config.pvforecast.provider_settings.PVForecastImport is None:
logger.debug(f"{self.provider_id()} data update without provider settings.")
return
if self.config.pvforecast.provider_settings.import_file_path is not None:
if self.config.pvforecast.provider_settings.PVForecastImport.import_file_path is not None:
self.import_from_file(
self.config.pvforecast.provider_settings.import_file_path,
self.config.pvforecast.provider_settings.PVForecastImport.import_file_path,
key_prefix="pvforecast",
)
if self.config.pvforecast.provider_settings.import_json is not None:
if self.config.pvforecast.provider_settings.PVForecastImport.import_json is not None:
self.import_from_json(
self.config.pvforecast.provider_settings.import_json,
self.config.pvforecast.provider_settings.PVForecastImport.import_json,
key_prefix="pvforecast",
)

13
src/akkudoktoreos/prediction/pvforecastvrm.py
View File

@@ -4,13 +4,12 @@ from typing import Any, Optional, Union
import requests
from loguru import logger
from pendulum import DateTime
from pydantic import Field, ValidationError
from akkudoktoreos.config.configabc import SettingsBaseModel
from akkudoktoreos.core.pydantic import PydanticBaseModel
from akkudoktoreos.prediction.pvforecastabc import PVForecastProvider
from akkudoktoreos.utils.datetimeutil import to_datetime
from akkudoktoreos.utils.datetimeutil import DateTime, to_datetime
class VrmForecastRecords(PydanticBaseModel):
@@ -24,7 +23,7 @@ class VrmForecastResponse(PydanticBaseModel):
totals: dict
class PVforecastVrmCommonSettings(SettingsBaseModel):
class PVForecastVrmCommonSettings(SettingsBaseModel):
"""Common settings for VRM API."""
pvforecast_vrm_token: str = Field(
@@ -60,8 +59,8 @@ class PVForecastVrm(PVForecastProvider):
def _request_forecast(self, start_ts: int, end_ts: int) -> VrmForecastResponse:
"""Fetch forecast data from Victron VRM API."""
source = "https://vrmapi.victronenergy.com/v2/installations"
id_site = self.config.pvforecast.provider_settings.pvforecast_vrm_idsite
api_token = self.config.pvforecast.provider_settings.pvforecast_vrm_token
id_site = self.config.pvforecast.provider_settings.PVForecastVrm.pvforecast_vrm_idsite
api_token = self.config.pvforecast.provider_settings.PVForecastVrm.pvforecast_vrm_token
headers = {"X-Authorization": f"Token {api_token}", "Content-Type": "application/json"}
url = f"{source}/{id_site}/stats?type=forecast&start={start_ts}&end={end_ts}&interval=hours"
logger.debug(f"Requesting VRM forecast: {url}")
@@ -82,8 +81,8 @@ class PVForecastVrm(PVForecastProvider):
def _update_data(self, force_update: Optional[bool] = False) -> None:
"""Update forecast data in the PVForecastDataRecord format."""
start_date = self.start_datetime.start_of("day")
end_date = self.start_datetime.add(hours=self.config.prediction.hours)
start_date = self.ems_start_datetime.start_of("day")
end_date = self.ems_start_datetime.add(hours=self.config.prediction.hours)
start_ts = int(start_date.timestamp())
end_ts = int(end_date.timestamp())

19
src/akkudoktoreos/prediction/weather.py
View File

@@ -19,6 +19,14 @@ weather_providers = [
]
class WeatherCommonProviderSettings(SettingsBaseModel):
"""Weather Forecast Provider Configuration."""
WeatherImport: Optional[WeatherImportCommonSettings] = Field(
default=None, description="WeatherImport settings", examples=[None]
)
class WeatherCommonSettings(SettingsBaseModel):
"""Weather Forecast Configuration."""
@@ -28,8 +36,15 @@ class WeatherCommonSettings(SettingsBaseModel):
examples=["WeatherImport"],
)
provider_settings: Optional[WeatherImportCommonSettings] = Field(
default=None, description="Provider settings", examples=[None]
provider_settings: WeatherCommonProviderSettings = Field(
default_factory=WeatherCommonProviderSettings,
description="Provider settings",
examples=[
# Example 1: Empty/default settings (all providers None)
{
"WeatherImport": None,
},
],
)
# Validators

11
src/akkudoktoreos/prediction/weatherbrightsky.py
View File

@@ -94,7 +94,7 @@ class WeatherBrightSky(WeatherProvider):
ValueError: If the API response does not include expected `weather` data.
"""
source = "https://api.brightsky.dev"
date = to_datetime(self.start_datetime, as_string=True)
date = to_datetime(self.ems_start_datetime, as_string=True)
last_date = to_datetime(self.end_datetime, as_string=True)
response = requests.get(
f"{source}/weather?lat={self.config.general.latitude}&lon={self.config.general.longitude}&date={date}&last_date={last_date}&tz={self.config.general.timezone}",
@@ -223,7 +223,7 @@ class WeatherBrightSky(WeatherProvider):
assert key # noqa: S101
temperature = self.key_to_array(
key=key,
start_datetime=self.start_datetime,
start_datetime=self.ems_start_datetime,
end_datetime=self.end_datetime,
interval=to_duration("1 hour"),
)
@@ -236,7 +236,7 @@ class WeatherBrightSky(WeatherProvider):
assert key # noqa: S101
humidity = self.key_to_array(
key=key,
start_datetime=self.start_datetime,
start_datetime=self.ems_start_datetime,
end_datetime=self.end_datetime,
interval=to_duration("1 hour"),
)
@@ -250,7 +250,10 @@ class WeatherBrightSky(WeatherProvider):
data=data,
index=pd.DatetimeIndex(
pd.date_range(
start=self.start_datetime, end=self.end_datetime, freq="1h", inclusive="left"
start=self.ems_start_datetime,
end=self.end_datetime,
freq="1h",
inclusive="left",
)
),
)

3
src/akkudoktoreos/prediction/weatherclearoutside.py
View File

@@ -301,7 +301,8 @@ class WeatherClearOutside(WeatherProvider):
# Converting the cloud cover into Irradiance (GHI, DNI, DHI)
cloud_cover = pd.Series(
data=clearout_data["Total Clouds (% Sky Obscured)"], index=clearout_data["DateTime"]
data=clearout_data["Total Clouds (% Sky Obscured)"],
index=pd.to_datetime(clearout_data["DateTime"]),
)
ghi, dni, dhi = self.estimate_irradiance_from_cloud_cover(
self.config.general.latitude, self.config.general.longitude, cloud_cover

12
src/akkudoktoreos/prediction/weatherimport.py
View File

@@ -61,14 +61,16 @@ class WeatherImport(WeatherProvider, PredictionImportProvider):
return "WeatherImport"
def _update_data(self, force_update: Optional[bool] = False) -> None:
if self.config.weather.provider_settings is None:
if self.config.weather.provider_settings.WeatherImport is None:
logger.debug(f"{self.provider_id()} data update without provider settings.")
return
if self.config.weather.provider_settings.import_file_path:
if self.config.weather.provider_settings.WeatherImport.import_file_path:
self.import_from_file(
self.config.weather.provider_settings.import_file_path, key_prefix="weather"
self.config.weather.provider_settings.WeatherImport.import_file_path,
key_prefix="weather",
)
if self.config.weather.provider_settings.import_json:
if self.config.weather.provider_settings.WeatherImport.import_json:
self.import_from_json(
self.config.weather.provider_settings.import_json, key_prefix="weather"
self.config.weather.provider_settings.WeatherImport.import_json,
key_prefix="weather",
)

14
src/akkudoktoreos/server/dash/hello.py → src/akkudoktoreos/server/dash/about.py
View File

@@ -2,9 +2,10 @@ from typing import Any
from fasthtml.common import Div
from akkudoktoreos.core.version import __version__
from akkudoktoreos.server.dash.markdown import Markdown
hello_md = """![Logo](/eosdash/assets/logo.png)
about_md = f"""![Logo](/eosdash/assets/logo.png)
# Akkudoktor EOSdash
@@ -17,8 +18,15 @@ electricity price data, this system enables forecasting and optimization of ener
over a specified period.
Documentation can be found at [Akkudoktor-EOS](https://akkudoktor-eos.readthedocs.io/en/latest/).
## Version Information
**Current Version:** {__version__}
**License:** Apache License
"""
def Hello(**kwargs: Any) -> Div:
return Markdown(hello_md, **kwargs)
def About(**kwargs: Any) -> Div:
return Markdown(about_md, **kwargs)

96
src/akkudoktoreos/server/dash/admin.py
View File

@@ -56,6 +56,101 @@ def AdminButton(*c: Any, cls: Optional[Union[str, tuple]] = None, **kwargs: Any)
return Button(*c, submit=False, **kwargs)
def AdminCache(
eos_host: str, eos_port: Union[str, int], data: Optional[dict], config: Optional[dict[str, Any]]
) -> tuple[str, Union[Card, list[Card]]]:
"""Creates a cache management card.
Args:
eos_host (str): The hostname of the EOS server.
eos_port (Union[str, int]): The port of the EOS server.
data (Optional[dict]): Incoming data containing action and category for processing.
Returns:
tuple[str, Union[Card, list[Card]]]: A tuple containing the cache category label and the `Card` UI component.
"""
server = f"http://{eos_host}:{eos_port}"
eos_hostname = "EOS server"
eosdash_hostname = "EOSdash server"
category = "cache"
if data and data.get("category", None) == category:
# This data is for us
if data["action"] == "clear":
# Clear all cache files
try:
result = requests.post(f"{server}/v1/admin/cache/clear", timeout=10)
result.raise_for_status()
status = Success(f"Cleared all cache files on '{eos_hostname}'")
except requests.exceptions.HTTPError as e:
detail = result.json()["detail"]
status = Error(f"Can not clear all cache files on '{eos_hostname}': {e}, {detail}")
except Exception as e:
status = Error(f"Can not clear all cache files on '{eos_hostname}': {e}")
elif data["action"] == "clear-expired":
# Clear expired cache files
try:
result = requests.post(f"{server}/v1/admin/cache/clear-expired", timeout=10)
result.raise_for_status()
status = Success(f"Cleared expired cache files on '{eos_hostname}'")
except requests.exceptions.HTTPError as e:
detail = result.json()["detail"]
status = Error(
f"Can not clear expired cache files on '{eos_hostname}': {e}, {detail}"
)
except Exception as e:
status = Error(f"Can not clear expired cache files on '{eos_hostname}': {e}")
return (
category,
[
Card(
Details(
Summary(
Grid(
DivHStacked(
UkIcon(icon="play"),
AdminButton(
"Clear all",
hx_post="/eosdash/admin",
hx_target="#page-content",
hx_swap="innerHTML",
hx_vals='{"category": "cache", "action": "clear"}',
),
P(f"cache files on '{eos_hostname}'"),
),
),
cls="list-none",
),
P(f"Clear all cache files on '{eos_hostname}'."),
),
),
Card(
Details(
Summary(
Grid(
DivHStacked(
UkIcon(icon="play"),
AdminButton(
"Clear expired",
hx_post="/eosdash/admin",
hx_target="#page-content",
hx_swap="innerHTML",
hx_vals='{"category": "cache", "action": "clear-expired"}',
),
P(f"cache files on '{eos_hostname}'"),
),
),
cls="list-none",
),
P(f"Clear expired cache files on '{eos_hostname}'."),
),
),
],
)
def AdminConfig(
eos_host: str, eos_port: Union[str, int], data: Optional[dict], config: Optional[dict[str, Any]]
) -> tuple[str, Union[Card, list[Card]]]:
@@ -282,6 +377,7 @@ def Admin(eos_host: str, eos_port: Union[str, int], data: Optional[dict] = None)
rows = []
last_category = ""
for category, admin in [
AdminCache(eos_host, eos_port, data, config),
AdminConfig(eos_host, eos_port, data, config),
]:
if category != last_category:

68
src/akkudoktoreos/server/dash/bokeh.py
View File

@@ -1,33 +1,87 @@
# Module taken from https://github.com/koaning/fh-altair
# MIT license
from typing import Optional
import bokeh
from bokeh.embed import components
from bokeh.models import Plot
from monsterui.franken import H4, Card, NotStr, Script
bokeh_version = bokeh.__version__
BokehJS = [
Script(src="https://cdn.bokeh.org/bokeh/release/bokeh-3.7.0.min.js", crossorigin="anonymous"),
Script(
src="https://cdn.bokeh.org/bokeh/release/bokeh-widgets-3.7.0.min.js",
src=f"https://cdn.bokeh.org/bokeh/release/bokeh-{bokeh_version}.min.js",
crossorigin="anonymous",
),
Script(
src="https://cdn.bokeh.org/bokeh/release/bokeh-tables-3.7.0.min.js", crossorigin="anonymous"
src=f"https://cdn.bokeh.org/bokeh/release/bokeh-widgets-{bokeh_version}.min.js",
crossorigin="anonymous",
),
Script(
src="https://cdn.bokeh.org/bokeh/release/bokeh-gl-3.7.0.min.js", crossorigin="anonymous"
src=f"https://cdn.bokeh.org/bokeh/release/bokeh-tables-{bokeh_version}.min.js",
crossorigin="anonymous",
),
Script(
src="https://cdn.bokeh.org/bokeh/release/bokeh-mathjax-3.7.0.min.js",
src=f"https://cdn.bokeh.org/bokeh/release/bokeh-gl-{bokeh_version}.min.js",
crossorigin="anonymous",
),
Script(
src=f"https://cdn.bokeh.org/bokeh/release/bokeh-mathjax-{bokeh_version}.min.js",
crossorigin="anonymous",
),
]
def bokey_apply_theme_to_plot(plot: Plot, dark: bool) -> None:
"""Apply a dark or light theme to a Bokeh plot.
This function modifies the appearance of a Bokeh `Plot` object in-place,
adjusting background, border, title, axis, and grid colors based on the
`dark` parameter.
Args:
plot (Plot): The Bokeh plot to style.
dark (bool): Whether to apply the dark theme (`True`) or light theme (`False`).
Notes:
- This only affects the plot passed in; it does not change other plots
in the same document.
"""
if dark:
plot.background_fill_color = "#1e1e1e"
plot.border_fill_color = "#1e1e1e"
plot.title.text_color = "white"
for ax in plot.xaxis + plot.yaxis:
ax.axis_line_color = "white"
ax.major_tick_line_color = "white"
ax.major_label_text_color = "white"
ax.axis_label_text_color = "white"
# Grid lines
for grid in plot.renderers:
if hasattr(grid, "grid_line_color"):
grid.grid_line_color = "#333"
for grid in plot.xgrid + plot.ygrid:
grid.grid_line_color = "#333"
else:
plot.background_fill_color = "white"
plot.border_fill_color = "white"
plot.title.text_color = "black"
for ax in plot.xaxis + plot.yaxis:
ax.axis_line_color = "black"
ax.major_tick_line_color = "black"
ax.major_label_text_color = "black"
ax.axis_label_text_color = "black"
# Grid lines
for grid in plot.renderers:
if hasattr(grid, "grid_line_color"):
grid.grid_line_color = "#ddd"
for grid in plot.xgrid + plot.ygrid:
grid.grid_line_color = "#ddd"
def Bokeh(plot: Plot, header: Optional[str] = None) -> Card:
"""Converts an Bokeh plot to a FastHTML FT component."""
"""Convert a Bokeh plot to a FastHTML FT component."""
script, div = components(plot)
if header:
header = H4(header, cls="mt-2")

14
src/akkudoktoreos/server/dash/components.py
View File

@@ -1,15 +1,13 @@
from typing import Any, Optional, Union
from fasthtml.common import H1, Div, Li
from fasthtml.common import H1, Button, Div, Li
from monsterui.daisy import (
Alert,
AlertT,
)
from monsterui.foundations import stringify
from monsterui.franken import (
from monsterui.franken import ( # Button, Does not pass hx_vals
H3,
Button,
ButtonT,
Card,
Container,
ContainerT,
@@ -246,7 +244,8 @@ def DashboardTrigger(*c: Any, cls: Optional[Union[str, tuple]] = None, **kwargs:
Returns:
Button: A styled `Button` component.
"""
new_cls = f"{ButtonT.primary}"
# new_cls = f"{ButtonT.primary} uk-border-rounded uk-padding-small"
new_cls = "uk-btn uk-btn-primary uk-border-rounded uk-padding-medium"
if cls:
new_cls += f" {stringify(cls)}"
kwargs["cls"] = new_cls
@@ -270,6 +269,7 @@ def DashboardTabs(dashboard_items: dict[str, str]) -> Card:
hx_get=f"{path}",
hx_target="#page-content",
hx_swap="innerHTML",
hx_vals='js:{ "dark": window.matchMedia("(prefers-color-scheme: dark)").matches }',
),
)
for menu, path in dashboard_items.items()
@@ -286,7 +286,9 @@ def DashboardContent(content: Any) -> Card:
Returns:
Card: A styled `Card` element containing the content.
"""
return Card(ScrollArea(Container(content, id="page-content"), cls="h-[75vh] w-full rounded-md"))
return Card(
ScrollArea(Container(content, id="page-content"), cls="h-[75vh] w-full rounded-md"),
)
def Page(

86
src/akkudoktoreos/server/dash/data/democonfig.json
View File

@@ -1,86 +0,0 @@
{
"elecprice": {
"charges_kwh": 0.21,
"provider": "ElecPriceAkkudoktor"
},
"general": {
"latitude": 52.5,
"longitude": 13.4
},
"prediction": {
"historic_hours": 48,
"hours": 48
},
"load": {
"provider": "LoadAkkudoktor",
"provider_settings": {
"loadakkudoktor_year_energy": 20000
}
},
"optimization": {
"hours": 48
},
"pvforecast": {
"planes": [
{
"peakpower": 5.0,
"surface_azimuth": 170,
"surface_tilt": 7,
"userhorizon": [
20,
27,
22,
20
],
"inverter_paco": 10000
},
{
"peakpower": 4.8,
"surface_azimuth": 90,
"surface_tilt": 7,
"userhorizon": [
30,
30,
30,
50
],
"inverter_paco": 10000
},
{
"peakpower": 1.4,
"surface_azimuth": 140,
"surface_tilt": 60,
"userhorizon": [
60,
30,
0,
30
],
"inverter_paco": 2000
},
{
"peakpower": 1.6,
"surface_azimuth": 185,
"surface_tilt": 45,
"userhorizon": [
45,
25,
30,
60
],
"inverter_paco": 1400
}
],
"provider": "PVForecastAkkudoktor"
},
"server": {
"startup_eosdash": true,
"host": "127.0.0.1",
"port": 8503,
"eosdash_host": "127.0.0.1",
"eosdash_port": 8504
},
"weather": {
"provider": "BrightSky"
}
}

13
src/akkudoktoreos/server/dash/eosstatus.py Normal file
View File

@@ -0,0 +1,13 @@
# EOS server status information
from typing import Optional
from akkudoktoreos.config.config import SettingsEOS
from akkudoktoreos.core.emplan import EnergyManagementPlan
from akkudoktoreos.optimization.optimization import OptimizationSolution
# The latest information from the EOS server
eos_health: Optional[dict] = None
eos_solution: Optional[OptimizationSolution] = None
eos_plan: Optional[EnergyManagementPlan] = None
eos_config: Optional[SettingsEOS] = None

6
src/akkudoktoreos/server/dash/footer.py
View File

@@ -6,6 +6,7 @@ from monsterui.daisy import Loading, LoadingT
from monsterui.franken import A, ButtonT, DivFullySpaced, P
from requests.exceptions import RequestException
import akkudoktoreos.server.dash.eosstatus as eosstatus
from akkudoktoreos.config.config import get_config
config_eos = get_config()
@@ -21,12 +22,15 @@ def get_alive(eos_host: str, eos_port: Union[str, int]) -> str:
Returns:
str: Alive data.
"""
global eos_health
result = requests.Response()
try:
result = requests.get(f"http://{eos_host}:{eos_port}/v1/health", timeout=10)
if result.status_code == 200:
alive = result.json()["status"]
eosstatus.eos_health = result.json()
alive = eosstatus.eos_health["status"]
else:
eosstatus.eos_health = None
alive = f"Server responded with status code: {result.status_code}"
except RequestException as e:
warning_msg = f"{e}"

496
src/akkudoktoreos/server/dash/plan.py Normal file
View File

@@ -0,0 +1,496 @@
from typing import Optional, Union
import pandas as pd
import requests
from bokeh.models import ColumnDataSource, LinearAxis, Range1d
from bokeh.plotting import figure
from loguru import logger
from monsterui.franken import (
Card,
Details,
Div,
DivLAligned,
Grid,
LabelCheckboxX,
P,
Summary,
UkIcon,
)
import akkudoktoreos.server.dash.eosstatus as eosstatus
from akkudoktoreos.config.config import SettingsEOS
from akkudoktoreos.core.emplan import (
DDBCInstruction,
EnergyManagementInstruction,
EnergyManagementPlan,
FRBCInstruction,
)
from akkudoktoreos.optimization.optimization import OptimizationSolution
from akkudoktoreos.server.dash.bokeh import Bokeh, bokey_apply_theme_to_plot
from akkudoktoreos.server.dash.components import Error
from akkudoktoreos.utils.datetimeutil import compare_datetimes, to_datetime
# bar width for 1 hour bars (time given in millseconds)
BAR_WIDTH_1HOUR = 1000 * 60 * 60
# Current state of solution displayed
solution_visible: dict[str, bool] = {
"pv_prediction_energy_wh": True,
"elec_price_prediction_amt_kwh": True,
}
solution_color: dict[str, str] = {}
def validate_source(source: ColumnDataSource, x_col: str = "date_time") -> None:
data = source.data
# 1. Source has data at all
if not data:
raise ValueError("ColumnDataSource has no data.")
# 2. x_col must be present
if x_col not in data:
raise ValueError(f"Missing expected x-axis column '{x_col}' in source.")
# 3. All columns must have equal length
lengths = {len(v) for v in data.values()}
if len(lengths) != 1:
raise ValueError(f"ColumnDataSource columns have mismatched lengths: {lengths}")
# 4. Must have at least one non-x column
y_columns = [c for c in data.keys() if c != x_col]
if not y_columns:
raise ValueError("No y-value columns found for plotting (only x-axis present).")
# 5. Each y-column must have at least one valid value
for col in y_columns:
values = [v for v in data[col] if v is not None]
if not values:
raise ValueError(f"Column '{col}' contains only None/NaN or is empty.")
def SolutionCard(solution: OptimizationSolution, config: SettingsEOS, data: Optional[dict]) -> Grid:
"""Creates a optimization solution card.
Args:
data (Optional[dict]): Incoming data containing action and category for processing.
"""
category = "solution"
dark = False
if data and data.get("category", None) == category:
# This data is for us
if data.get("action", None) == "visible":
renderer = data.get("renderer", None)
if renderer:
solution_visible[renderer] = bool(data.get(f"{renderer}-visible", False))
if data and data.get("dark", None) == "true":
dark = True
df = solution.data.to_dataframe()
if df.empty or len(df.columns) <= 1:
raise ValueError(f"DataFrame is empty or missing plottable columns: {list(df.columns)}")
if "date_time" not in df.columns:
raise ValueError(f"DataFrame is missing column 'date_time': {list(df.columns)}")
# Remove time offset from UTC to get naive local time and make bokey plot in local time
dst_offsets = df.index.map(lambda x: x.dst().total_seconds() / 3600)
if config.general is None or config.general.timezone is None:
date_time_tz = "Europe/Berlin"
else:
date_time_tz = config.general.timezone
df["date_time"] = pd.to_datetime(df["date_time"], utc=True).dt.tz_convert(date_time_tz)
# There is a special case if we have daylight saving time change in the time series
if dst_offsets.nunique() > 1:
date_time_tz += " + DST change"
source = ColumnDataSource(df)
validate_source(source)
# Calculate minimum and maximum Range
energy_wh_min = 0.0
energy_wh_max = 0.0
amt_kwh_min = 0.0
amt_kwh_max = 0.0
amt_min = 0.0
amt_max = 0.0
soc_factor_min = 0.0
soc_factor_max = 1.0
for col in df.columns:
if col.endswith("energy_wh"):
energy_wh_min = min(energy_wh_min, float(df[col].min()))
energy_wh_max = max(energy_wh_max, float(df[col].max()))
elif col.endswith("amt_kwh"):
amt_kwh_min = min(amt_kwh_min, float(df[col].min()))
amt_kwh_max = max(amt_kwh_max, float(df[col].max()))
elif col.endswith("amt"):
amt_min = min(amt_min, float(df[col].min()))
amt_max = max(amt_max, float(df[col].max()))
else:
continue
# Adjust to similar y-axis 0-point
# First get the maximum factor for the min value related the maximum value
min_max_factor = max(
(energy_wh_min * -1.0) / energy_wh_max,
(amt_kwh_min * -1.0) / amt_kwh_max,
(amt_min * -1.0) / amt_max,
(soc_factor_min * -1.0) / soc_factor_max,
)
# Adapt the min values to have the same relative min/max factor on all y-axis
energy_wh_min = min_max_factor * energy_wh_max * -1.0
amt_kwh_min = min_max_factor * amt_kwh_max * -1.0
amt_min = min_max_factor * amt_max * -1.0
soc_factor_min = min_max_factor * soc_factor_max * -1.0
# add 5% to min and max values for better display
energy_wh_range_orig = energy_wh_max - energy_wh_min
energy_wh_max += 0.05 * energy_wh_range_orig
energy_wh_min -= 0.05 * energy_wh_range_orig
amt_kwh_range_orig = amt_kwh_max - amt_kwh_min
amt_kwh_max += 0.05 * amt_kwh_range_orig
amt_kwh_min -= 0.05 * amt_kwh_range_orig
amt_range_orig = amt_max - amt_min
amt_max += 0.05 * amt_range_orig
amt_min -= 0.05 * amt_range_orig
soc_factor_range_orig = soc_factor_max - soc_factor_min
soc_factor_max += 0.05 * soc_factor_range_orig
soc_factor_min -= 0.05 * soc_factor_range_orig
if eosstatus.eos_health is not None:
last_run_datetime = eosstatus.eos_health["energy-management"]["last_run_datetime"]
start_datetime = eosstatus.eos_health["energy-management"]["start_datetime"]
else:
last_run_datetime = "unknown"
start_datetime = "unknown"
plot = figure(
title=f"Optimization Solution - last run: {last_run_datetime}",
x_axis_type="datetime",
x_axis_label=f"Datetime [localtime {date_time_tz}] - start: {start_datetime}",
y_axis_label="Energy [Wh]",
sizing_mode="stretch_width",
y_range=Range1d(energy_wh_min, energy_wh_max),
height=400,
)
plot.extra_y_ranges = {
"factor": Range1d(soc_factor_min, soc_factor_max), # y2
"amt_kwh": Range1d(amt_kwh_min, amt_kwh_max), # y3
"amt": Range1d(amt_min, amt_max), # y4
}
# y2 axis
y2_axis = LinearAxis(y_range_name="factor", axis_label="Factor [0.0..1.0]")
plot.add_layout(y2_axis, "left")
# y3 axis
y3_axis = LinearAxis(y_range_name="amt_kwh", axis_label="Electricty Price [currency/kWh]")
y3_axis.axis_label_text_color = "red"
plot.add_layout(y3_axis, "right")
# y4 axis
y4_axis = LinearAxis(y_range_name="amt", axis_label="Amount [currency]")
plot.add_layout(y4_axis, "right")
plot.toolbar.autohide = True
# Create line renderers for each column
renderers = {}
colors = ["black", "blue", "cyan", "green", "orange", "pink", "purple"]
for i, col in enumerate(sorted(df.columns)):
# Exclude some columns that are currently not used or are covered by others
excludes = [
"date_time",
"_op_mode",
"_fault_",
"_forced_discharge_",
"_outage_supply_",
"_reserve_backup_",
"_ramp_rate_control_",
"_frequency_regulation_",
"_grid_support_export_",
"_peak_shaving_",
]
# excludes = ["date_time"]
if any(exclude in col for exclude in excludes):
continue
if col in solution_visible:
visible = solution_visible[col]
else:
visible = False
solution_visible[col] = visible
if col in solution_color:
color = solution_color[col]
elif col == "pv_prediction_energy_wh":
color = "yellow"
solution_color[col] = color
elif col == "elec_price_prediction_amt_kwh":
color = "red"
solution_color[col] = color
else:
color = colors[i % len(colors)]
solution_color[col] = color
if visible:
if col == "pv_prediction_energy_wh":
r = plot.vbar(
x="date_time",
top=col,
source=source,
width=BAR_WIDTH_1HOUR * 0.8,
legend_label=col,
color=color,
level="underlay",
)
elif col.endswith("energy_wh"):
r = plot.step(
x="date_time",
y=col,
mode="before",
source=source,
legend_label=col,
color=color,
)
elif col.endswith("factor"):
r = plot.step(
x="date_time",
y=col,
mode="before",
source=source,
legend_label=col,
color=color,
y_range_name="factor",
)
elif col.endswith("mode"):
r = plot.step(
x="date_time",
y=col,
mode="before",
source=source,
legend_label=col,
color=color,
y_range_name="factor",
)
elif col.endswith("amt_kwh"):
r = plot.step(
x="date_time",
y=col,
mode="before",
source=source,
legend_label=col,
color=color,
y_range_name="amt_kwh",
)
elif col.endswith("amt"):
r = plot.step(
x="date_time",
y=col,
mode="before",
source=source,
legend_label=col,
color=color,
y_range_name="amt",
)
else:
raise ValueError(f"Unexpected column name: {col}")
else:
r = None
renderers[col] = r
plot.legend.visible = False # no legend at plot
bokey_apply_theme_to_plot(plot, dark)
# --- CheckboxGroup to toggle datasets ---
Checkbox = Grid(
*[
LabelCheckboxX(
label=renderer,
id=f"{renderer}-visible",
name=f"{renderer}-visible",
value="true",
checked=solution_visible[renderer],
hx_post="/eosdash/plan",
hx_target="#page-content",
hx_swap="innerHTML",
hx_vals='js:{ "category": "solution", "action": "visible", "renderer": '
+ '"'
+ f"{renderer}"
+ '", '
+ '"dark": window.matchMedia("(prefers-color-scheme: dark)").matches '
+ "}",
lbl_cls=f"text-{solution_color[renderer]}-500",
)
for renderer in list(renderers.keys())
],
cols=2,
)
return Grid(
Bokeh(plot),
Card(
Checkbox,
),
cls="w-full space-y-3 space-x-3",
)
def InstructionCard(
instruction: EnergyManagementInstruction, config: SettingsEOS, data: Optional[dict]
) -> Card:
"""Creates a styled instruction card for displaying instruction details.
This function generates a instruction card that is displayed in the UI with
various sections such as instruction name, type, description, default value,
current value, and error details. It supports both read-only and editable modes.
Args:
instruction (EnergyManagementInstruction): The instruction.
data (Optional[dict]): Incoming data containing action and category for processing.
Returns:
Card: A styled Card component containing the instruction details.
"""
if instruction.id is None:
return Error("Instruction without id encountered. Can not handle")
idx = instruction.id.find("@")
resource_id = instruction.id[:idx] if idx != -1 else instruction.id
execution_time = to_datetime(instruction.execution_time, as_string=True)
description = instruction.type
summary = None
# Search an icon that fits to device_id
if (
config.devices
and config.devices.batteries
and any(
battery_config.device_id == resource_id for battery_config in config.devices.batteries
)
):
# This is a battery
if instruction.operation_mode_id in ("CHARGE",):
icon = "battery-charging"
else:
icon = "battery"
elif (
config.devices
and config.devices.electric_vehicles
and any(
electric_vehicle_config.device_id == resource_id
for electric_vehicle_config in config.devices.electric_vehicles
)
):
# This is a car battery
icon = "car"
elif (
config.devices
and config.devices.home_appliances
and any(
home_appliance.device_id == resource_id
for home_appliance in config.devices.home_appliances
)
):
# This is a home appliance
icon = "washing-machine"
else:
icon = "play"
if isinstance(instruction, (DDBCInstruction, FRBCInstruction)):
summary = f"{instruction.operation_mode_id}"
summary_detail = f"{instruction.operation_mode_factor}"
return Card(
Details(
Summary(
Grid(
Grid(
DivLAligned(
UkIcon(icon=icon),
P(execution_time),
),
DivLAligned(
P(resource_id),
),
),
P(summary),
P(summary_detail),
),
cls="list-none",
),
Grid(
P(description),
P("TBD"),
),
),
cls="w-full",
)
def Plan(eos_host: str, eos_port: Union[str, int], data: Optional[dict] = None) -> Div:
"""Generates the plan dashboard layout.
Args:
eos_host (str): The hostname of the EOS server.
eos_port (Union[str, int]): The port of the EOS server.
data (Optional[dict], optional): Incoming data to trigger plan actions. Defaults to None.
Returns:
Div: A `Div` component containing the assembled admin interface.
"""
server = f"http://{eos_host}:{eos_port}"
print("Plan: ", data)
if (
eosstatus.eos_config is None
or eosstatus.eos_solution is None
or eosstatus.eos_plan is None
or eosstatus.eos_health is None
or compare_datetimes(
to_datetime(eosstatus.eos_plan.generated_at),
to_datetime(eosstatus.eos_health["energy-management"]["last_run_datetime"]),
).lt
):
# Get current configuration from server
try:
result = requests.get(f"{server}/v1/config", timeout=10)
result.raise_for_status()
except requests.exceptions.HTTPError as err:
detail = result.json()["detail"]
return Error(f"Can not retrieve configuration from {server}: {err}, {detail}")
eosstatus.eos_config = SettingsEOS(**result.json())
# Get the optimization solution
try:
result = requests.get(
f"{server}/v1/energy-management/optimization/solution", timeout=10
)
result.raise_for_status()
solution_json = result.json()
except requests.exceptions.HTTPError as e:
detail = result.json()["detail"]
warning_msg = f"Can not retrieve optimization solution from {server}: {e}, {detail}"
logger.warning(warning_msg)
return Error(warning_msg)
except Exception as e:
warning_msg = f"Can not retrieve optimization solution from {server}: {e}"
logger.warning(warning_msg)
return Error(warning_msg)
eosstatus.eos_solution = OptimizationSolution(**solution_json)
# Get the plan
try:
result = requests.get(f"{server}/v1/energy-management/plan", timeout=10)
result.raise_for_status()
plan_json = result.json()
except requests.exceptions.HTTPError as e:
detail = result.json()["detail"]
warning_msg = f"Can not retrieve plan from {server}: {e}, {detail}"
logger.warning(warning_msg)
return Error(warning_msg)
except Exception as e:
warning_msg = f"Can not retrieve plan from {server}: {e}"
logger.warning(warning_msg)
return Error(warning_msg)
eosstatus.eos_plan = EnergyManagementPlan(**plan_json, data=data)
rows = [
SolutionCard(eosstatus.eos_solution, eosstatus.eos_config, data=data),
]
for instruction in eosstatus.eos_plan.instructions:
rows.append(InstructionCard(instruction, eosstatus.eos_config, data=data))
return Div(*rows, cls="space-y-4")
# return Div(f"Plan:\n{json.dumps(plan_json, indent=4)}")

115
src/akkudoktoreos/server/dash/demo.py → src/akkudoktoreos/server/dash/prediction.py
View File

@@ -1,6 +1,4 @@
import json
from pathlib import Path
from typing import Union
from typing import Optional, Union
import pandas as pd
import requests
@@ -9,30 +7,25 @@ from bokeh.plotting import figure
from monsterui.franken import FT, Grid, P
from akkudoktoreos.core.pydantic import PydanticDateTimeDataFrame
from akkudoktoreos.server.dash.bokeh import Bokeh
DIR_DEMODATA = Path(__file__).absolute().parent.joinpath("data")
FILE_DEMOCONFIG = DIR_DEMODATA.joinpath("democonfig.json")
if not FILE_DEMOCONFIG.exists():
raise ValueError(f"File does not exist: {FILE_DEMOCONFIG}")
from akkudoktoreos.server.dash.bokeh import Bokeh, bokey_apply_theme_to_plot
from akkudoktoreos.server.dash.components import Error
# bar width for 1 hour bars (time given in millseconds)
BAR_WIDTH_1HOUR = 1000 * 60 * 60
def DemoPVForecast(predictions: pd.DataFrame, config: dict) -> FT:
def PVForecast(predictions: pd.DataFrame, config: dict, date_time_tz: str, dark: bool) -> FT:
source = ColumnDataSource(predictions)
provider = config["pvforecast"]["provider"]
plot = figure(
x_axis_type="datetime",
title=f"PV Power Prediction ({provider})",
x_axis_label="Datetime",
x_axis_label=f"Datetime [localtime {date_time_tz}]",
y_axis_label="Power [W]",
sizing_mode="stretch_width",
height=400,
)
plot.vbar(
x="date_time",
top="pvforecast_ac_power",
@@ -41,11 +34,15 @@ def DemoPVForecast(predictions: pd.DataFrame, config: dict) -> FT:
legend_label="AC Power",
color="lightblue",
)
plot.toolbar.autohide = True
bokey_apply_theme_to_plot(plot, dark)
return Bokeh(plot)
def DemoElectricityPriceForecast(predictions: pd.DataFrame, config: dict) -> FT:
def ElectricityPriceForecast(
predictions: pd.DataFrame, config: dict, date_time_tz: str, dark: bool
) -> FT:
source = ColumnDataSource(predictions)
provider = config["elecprice"]["provider"]
@@ -56,7 +53,7 @@ def DemoElectricityPriceForecast(predictions: pd.DataFrame, config: dict) -> FT:
predictions["elecprice_marketprice_kwh"].max() + 0.1,
),
title=f"Electricity Price Prediction ({provider})",
x_axis_label="Datetime",
x_axis_label=f"Datetime [localtime {date_time_tz}]",
y_axis_label="Price [€/kWh]",
sizing_mode="stretch_width",
height=400,
@@ -69,18 +66,22 @@ def DemoElectricityPriceForecast(predictions: pd.DataFrame, config: dict) -> FT:
legend_label="Market Price",
color="lightblue",
)
plot.toolbar.autohide = True
bokey_apply_theme_to_plot(plot, dark)
return Bokeh(plot)
def DemoWeatherTempAirHumidity(predictions: pd.DataFrame, config: dict) -> FT:
def WeatherTempAirHumidityForecast(
predictions: pd.DataFrame, config: dict, date_time_tz: str, dark: bool
) -> FT:
source = ColumnDataSource(predictions)
provider = config["weather"]["provider"]
plot = figure(
x_axis_type="datetime",
title=f"Air Temperature and Humidity Prediction ({provider})",
x_axis_label="Datetime",
x_axis_label=f"Datetime [localtime {date_time_tz}]",
y_axis_label="Temperature [°C]",
sizing_mode="stretch_width",
height=400,
@@ -94,7 +95,6 @@ def DemoWeatherTempAirHumidity(predictions: pd.DataFrame, config: dict) -> FT:
plot.line(
"date_time", "weather_temp_air", source=source, legend_label="Air Temperature", color="blue"
)
plot.line(
"date_time",
"weather_relative_humidity",
@@ -103,18 +103,22 @@ def DemoWeatherTempAirHumidity(predictions: pd.DataFrame, config: dict) -> FT:
color="green",
y_range_name="humidity",
)
plot.toolbar.autohide = True
bokey_apply_theme_to_plot(plot, dark)
return Bokeh(plot)
def DemoWeatherIrradiance(predictions: pd.DataFrame, config: dict) -> FT:
def WeatherIrradianceForecast(
predictions: pd.DataFrame, config: dict, date_time_tz: str, dark: bool
) -> FT:
source = ColumnDataSource(predictions)
provider = config["weather"]["provider"]
plot = figure(
x_axis_type="datetime",
title=f"Irradiance Prediction ({provider})",
x_axis_label="Datetime",
x_axis_label=f"Datetime [localtime {date_time_tz}]",
y_axis_label="Irradiance [W/m2]",
sizing_mode="stretch_width",
height=400,
@@ -140,21 +144,25 @@ def DemoWeatherIrradiance(predictions: pd.DataFrame, config: dict) -> FT:
legend_label="Diffuse Horizontal Irradiance",
color="blue",
)
plot.toolbar.autohide = True
bokey_apply_theme_to_plot(plot, dark)
return Bokeh(plot)
def DemoLoad(predictions: pd.DataFrame, config: dict) -> FT:
def LoadForecast(predictions: pd.DataFrame, config: dict, date_time_tz: str, dark: bool) -> FT:
source = ColumnDataSource(predictions)
provider = config["load"]["provider"]
if provider == "LoadAkkudoktor":
year_energy = config["load"]["provider_settings"]["loadakkudoktor_year_energy"]
year_energy = config["load"]["provider_settings"]["LoadAkkudoktor"][
"loadakkudoktor_year_energy"
]
provider = f"{provider}, {year_energy} kWh"
plot = figure(
x_axis_type="datetime",
title=f"Load Prediction ({provider})",
x_axis_label="Datetime",
x_axis_type="datetime",
x_axis_label=f"Datetime [localtime {date_time_tz}]",
y_axis_label="Load [W]",
sizing_mode="stretch_width",
height=400,
@@ -189,13 +197,19 @@ def DemoLoad(predictions: pd.DataFrame, config: dict) -> FT:
color="green",
y_range_name="stddev",
)
plot.toolbar.autohide = True
bokey_apply_theme_to_plot(plot, dark)
return Bokeh(plot)
def Demo(eos_host: str, eos_port: Union[str, int]) -> str:
def Prediction(eos_host: str, eos_port: Union[str, int], data: Optional[dict] = None) -> str:
server = f"http://{eos_host}:{eos_port}"
dark = False
if data and data.get("dark", None) == "true":
dark = True
# Get current configuration from server
try:
result = requests.get(f"{server}/v1/config", timeout=10)
@@ -208,34 +222,6 @@ def Demo(eos_host: str, eos_port: Union[str, int]) -> str:
)
config = result.json()
# Set demo configuration
with FILE_DEMOCONFIG.open("r", encoding="utf-8") as fd:
democonfig = json.load(fd)
try:
result = requests.put(f"{server}/v1/config", json=democonfig, timeout=10)
result.raise_for_status()
except requests.exceptions.HTTPError as err:
detail = result.json()["detail"]
# Try to reset to original config
requests.put(f"{server}/v1/config", json=config, timeout=10)
return P(
f"Can not set demo configuration on {server}: {err}, {detail}",
cls="text-center",
)
# Update all predictions
try:
result = requests.post(f"{server}/v1/prediction/update", timeout=10)
result.raise_for_status()
except requests.exceptions.HTTPError as err:
detail = result.json()["detail"]
# Try to reset to original config
requests.put(f"{server}/v1/config", json=config, timeout=10)
return P(
f"Can not update predictions on {server}: {err}, {detail}",
cls="text-center",
)
# Get Forecasts
try:
params = {
@@ -257,24 +243,19 @@ def Demo(eos_host: str, eos_port: Union[str, int]) -> str:
predictions = PydanticDateTimeDataFrame(**result.json()).to_dataframe()
except requests.exceptions.HTTPError as err:
detail = result.json()["detail"]
return P(
f"Can not retrieve predictions from {server}: {err}, {detail}",
cls="text-center",
)
return Error(f"Can not retrieve predictions from {server}: {err}, {detail}")
except Exception as err:
return P(
f"Can not retrieve predictions from {server}: {err}",
cls="text-center",
)
return Error(f"Can not retrieve predictions from {server}: {err}")
# Reset to original config
requests.put(f"{server}/v1/config", json=config, timeout=10)
# Remove time offset from UTC to get naive local time and make bokeh plot in local time
date_time_tz = predictions["date_time"].dt.tz
predictions["date_time"] = pd.to_datetime(predictions["date_time"]).dt.tz_localize(None)
return Grid(
DemoPVForecast(predictions, democonfig),
DemoElectricityPriceForecast(predictions, democonfig),
DemoWeatherTempAirHumidity(predictions, democonfig),
DemoWeatherIrradiance(predictions, democonfig),
DemoLoad(predictions, democonfig),
PVForecast(predictions, config, date_time_tz, dark),
ElectricityPriceForecast(predictions, config, date_time_tz, dark),
WeatherTempAirHumidityForecast(predictions, config, date_time_tz, dark),
WeatherIrradianceForecast(predictions, config, date_time_tz, dark),
LoadForecast(predictions, config, date_time_tz, dark),
cols_max=2,
)

747
src/akkudoktoreos/server/eos.py
View File

File diff suppressed because it is too large Load Diff

342
src/akkudoktoreos/server/eosdash.py
View File

@@ -3,7 +3,6 @@ import os
import sys
import traceback
from pathlib import Path
from typing import Optional
import psutil
import uvicorn
@@ -12,33 +11,171 @@ from loguru import logger
from monsterui.core import FastHTML, Theme
from akkudoktoreos.config.config import get_config
from akkudoktoreos.core.logabc import LOGGING_LEVELS
from akkudoktoreos.core.logging import track_logging_config
from akkudoktoreos.core.version import __version__
from akkudoktoreos.server.dash.about import About
# Pages
from akkudoktoreos.server.dash.admin import Admin
from akkudoktoreos.server.dash.bokeh import BokehJS
from akkudoktoreos.server.dash.components import Page
from akkudoktoreos.server.dash.configuration import ConfigKeyUpdate, Configuration
from akkudoktoreos.server.dash.demo import Demo
from akkudoktoreos.server.dash.footer import Footer
from akkudoktoreos.server.dash.hello import Hello
from akkudoktoreos.server.dash.plan import Plan
from akkudoktoreos.server.dash.prediction import Prediction
from akkudoktoreos.server.server import get_default_host, wait_for_port_free
from akkudoktoreos.utils.stringutil import str2bool
config_eos = get_config()
# ------------------------------------
# Logging configuration at import time
# ------------------------------------
logger.remove()
track_logging_config(config_eos, "logging", None, None)
config_eos.track_nested_value("/logging", track_logging_config)
# ----------------------------
# Safe argparse at import time
# ----------------------------
parser = argparse.ArgumentParser(description="Start EOSdash server.")
parser.add_argument(
"--host",
type=str,
help="Host for the EOSdash server (default: value from config)",
)
parser.add_argument(
"--port",
type=int,
help="Port for the EOSdash server (default: value from config)",
)
parser.add_argument(
"--eos-host",
type=str,
help="Host of the EOS server (default: value from config)",
)
parser.add_argument(
"--eos-port",
type=int,
help="Port of the EOS server (default: value from config)",
)
parser.add_argument(
"--log_level",
type=str,
default="INFO",
help='Log level for the server. Options: "critical", "error", "warning", "info", "debug", "trace" (default: "INFO")',
)
parser.add_argument(
"--access_log",
type=str2bool,
default=False,
help="Enable or disable access logging. Options: True or False (default: False)",
)
parser.add_argument(
"--reload",
type=str2bool,
default=False,
help="Enable or disable auto-reload. Useful for development. Options: True or False (default: False)",
)
# Command line arguments
args: argparse.Namespace
args_unknown: list[str]
args, args_unknown = parser.parse_known_args()
# -----------------------------
# Prepare config at import time
# -----------------------------
# Set EOS config to actual environment variable & config file content
config_eos.reset_settings()
# Setup parameters from args, config_eos and default
# Remember parameters in config
config_eosdash = {}
# Setup EOS logging level - first to have the other logging messages logged
# - log level
if args and args.log_level is not None:
config_eosdash["log_level"] = args.log_level.upper()
else:
config_eosdash["log_level"] = "info"
# Ensure log_level from command line is in config settings
if config_eosdash["log_level"] in LOGGING_LEVELS:
# Setup console logging level using nested value
# - triggers logging configuration by track_logging_config
config_eos.set_nested_value("logging/console_level", config_eosdash["log_level"])
logger.debug(
f"logging/console_level configuration set by argument to {config_eosdash['log_level']}"
)
# Setup EOS server host
if args and args.eos_host:
config_eosdash["eos_host"] = args.eos_host
elif config_eos.server.host:
config_eosdash["eos_host"] = str(config_eos.server.host)
else:
config_eosdash["eos_host"] = get_default_host()
# Setup EOS server port
if args and args.eos_port:
config_eosdash["eos_port"] = args.eos_port
elif config_eos.server.port:
config_eosdash["eos_port"] = config_eos.server.port
else:
config_eosdash["eos_port"] = 8503
# - EOSdash host
if args and args.host:
config_eosdash["eosdash_host"] = args.host
elif config_eos.server.eosdash_host:
config_eosdash["eosdash_host"] = str(config_eos.server.eosdash_host)
else:
config_eosdash["eosdash_host"] = get_default_host()
# - EOS port
if args and args.port:
config_eosdash["eosdash_port"] = args.port
elif config_eos.server.eosdash_port:
config_eosdash["eosdash_port"] = config_eos.server.eosdash_port
else:
config_eosdash["eosdash_port"] = 8504
# - access log
if args and args.access_log:
config_eosdash["access_log"] = args.access_log
else:
config_eosdash["access_log"] = False
# - reload
if args is None and args.reload is None:
config_eosdash["reload"] = False
else:
config_eosdash["reload"] = args.reload
# ---------------------
# Prepare FastHTML app
# ---------------------
# The favicon for EOSdash
favicon_filepath = Path(__file__).parent.joinpath("dash/assets/favicon/favicon.ico")
if not favicon_filepath.exists():
raise ValueError(f"Does not exist {favicon_filepath}")
# Command line arguments
args: Optional[argparse.Namespace] = None
# Get frankenui and tailwind headers via CDN using Theme.green.headers()
# Add Bokeh headers
# Get frankenui and tailwind headers via CDN using Theme.green.headers()
hdrs = (
*BokehJS,
Theme.green.headers(highlightjs=True),
BokehJS,
)
# The EOSdash application
@@ -52,24 +189,14 @@ app: FastHTML = FastHTML(
def eos_server() -> tuple[str, int]:
"""Retrieves the EOS server host and port configuration.
If `args` is provided, it uses the `eos_host` and `eos_port` from `args`.
Otherwise, it falls back to the values from `config_eos.server`.
Takes values from `config_eos.server` or default.
Returns:
tuple[str, int]: A tuple containing:
- `eos_host` (str): The EOS server hostname or IP.
- `eos_port` (int): The EOS server port.
"""
if args is None:
eos_host = str(config_eos.server.host)
eos_port = config_eos.server.port
else:
eos_host = args.eos_host
eos_port = args.eos_port
eos_host = eos_host if eos_host else get_default_host()
eos_port = eos_port if eos_port else 8503
return eos_host, eos_port
return config_eosdash["eos_host"], config_eosdash["eos_port"]
@app.get("/favicon.ico")
@@ -88,12 +215,13 @@ def get_eosdash(): # type: ignore
return Page(
None,
{
"EOSdash": "/eosdash/hello",
"Plan": "/eosdash/plan",
"Prediction": "/eosdash/prediction",
"Config": "/eosdash/configuration",
"Demo": "/eosdash/demo",
"Admin": "/eosdash/admin",
"About": "/eosdash/about",
},
Hello(),
About(),
Footer(*eos_server()),
"/eosdash/footer",
)
@@ -109,14 +237,14 @@ def get_eosdash_footer(): # type: ignore
return Footer(*eos_server())
@app.get("/eosdash/hello")
def get_eosdash_hello(): # type: ignore
"""Serves the EOSdash Hello page.
@app.get("/eosdash/about")
def get_eosdash_about(): # type: ignore
"""Serves the EOSdash About page.
Returns:
Hello: The Hello page component.
About: The About page component.
"""
return Hello()
return About()
@app.get("/eosdash/admin")
@@ -131,6 +259,13 @@ def get_eosdash_admin(): # type: ignore
@app.post("/eosdash/admin")
def post_eosdash_admin(data: dict): # type: ignore
"""Provide control data to the Admin page.
This endpoint is called from within the Admin page on user actions.
Returns:
Admin: The Admin page component.
"""
return Admin(*eos_server(), data)
@@ -149,14 +284,36 @@ def put_eosdash_configuration(data: dict): # type: ignore
return ConfigKeyUpdate(*eos_server(), data["key"], data["value"])
@app.get("/eosdash/demo")
def get_eosdash_demo(): # type: ignore
"""Serves the EOSdash Demo page.
@app.get("/eosdash/plan")
def get_eosdash_plan(data: dict): # type: ignore
"""Serves the EOSdash Plan page.
Returns:
Demo: The Demo page component.
Plan: The Plan page component.
"""
return Demo(*eos_server())
return Plan(*eos_server(), data)
@app.post("/eosdash/plan")
def post_eosdash_plan(data: dict): # type: ignore
"""Provide control data to the Plan page.
This endpoint is called from within the Plan page on user actions.
Returns:
Plan: The Plan page component.
"""
return Plan(*eos_server(), data)
@app.get("/eosdash/prediction")
def get_eosdash_prediction(data: dict): # type: ignore
"""Serves the EOSdash Prediction page.
Returns:
Prediction: The Prediction page component.
"""
return Prediction(*eos_server(), data)
@app.get("/eosdash/health")
@@ -166,6 +323,7 @@ def get_eosdash_health(): # type: ignore
{
"status": "alive",
"pid": psutil.Process().pid,
"version": __version__,
}
)
@@ -190,74 +348,22 @@ def run_eosdash() -> None:
Returns:
None
"""
# Setup parameters from args, config_eos and default
# Remember parameters that are also in config
# - EOS host
if args and args.eos_host:
eos_host = args.eos_host
elif config_eos.server.host:
eos_host = config_eos.server.host
else:
eos_host = get_default_host()
config_eos.server.host = eos_host
# - EOS port
if args and args.eos_port:
eos_port = args.eos_port
elif config_eos.server.port:
eos_port = config_eos.server.port
else:
eos_port = 8503
config_eos.server.port = eos_port
# - EOSdash host
if args and args.host:
eosdash_host = args.host
elif config_eos.server.eosdash.host:
eosdash_host = config_eos.server.eosdash_host
else:
eosdash_host = get_default_host()
config_eos.server.eosdash_host = eosdash_host
# - EOS port
if args and args.port:
eosdash_port = args.port
elif config_eos.server.eosdash_port:
eosdash_port = config_eos.server.eosdash_port
else:
eosdash_port = 8504
config_eos.server.eosdash_port = eosdash_port
# - log level
if args and args.log_level:
log_level = args.log_level
else:
log_level = "info"
# - access log
if args and args.access_log:
access_log = args.access_log
else:
access_log = False
# - reload
if args and args.reload:
reload = args.reload
else:
reload = False
# Make hostname Windows friendly
if eosdash_host == "0.0.0.0" and os.name == "nt": # noqa: S104
eosdash_host = "localhost"
# Wait for EOSdash port to be free - e.g. in case of restart
wait_for_port_free(eosdash_port, timeout=120, waiting_app_name="EOSdash")
wait_for_port_free(config_eosdash["eosdash_port"], timeout=120, waiting_app_name="EOSdash")
try:
uvicorn.run(
"akkudoktoreos.server.eosdash:app",
host=eosdash_host,
port=eosdash_port,
log_level=log_level.lower(),
access_log=access_log,
reload=reload,
host=config_eosdash["eosdash_host"],
port=config_eosdash["eosdash_port"],
log_level=config_eosdash["log_level"].lower(),
access_log=config_eosdash["access_log"],
reload=config_eosdash["reload"],
)
except Exception as e:
logger.error(f"Could not bind to host {eosdash_host}:{eosdash_port}. Error: {e}")
logger.error(
f"Could not bind to host {config_eosdash['eosdash_host']}:{config_eosdash['eosdash_port']}. Error: {e}"
)
raise e
@@ -278,54 +384,6 @@ def main() -> None:
--access_log (bool): Enable or disable access log. Options: True or False (default: False).
--reload (bool): Enable or disable auto-reload. Useful for development. Options: True or False (default: False).
"""
parser = argparse.ArgumentParser(description="Start EOSdash server.")
parser.add_argument(
"--host",
type=str,
default=str(config_eos.server.eosdash_host),
help="Host for the EOSdash server (default: value from config)",
)
parser.add_argument(
"--port",
type=int,
default=config_eos.server.eosdash_port,
help="Port for the EOSdash server (default: value from config)",
)
parser.add_argument(
"--eos-host",
type=str,
default=str(config_eos.server.host),
help="Host of the EOS server (default: value from config)",
)
parser.add_argument(
"--eos-port",
type=int,
default=config_eos.server.port,
help="Port of the EOS server (default: value from config)",
)
parser.add_argument(
"--log_level",
type=str,
default="info",
help='Log level for the server. Options: "critical", "error", "warning", "info", "debug", "trace" (default: "info")',
)
parser.add_argument(
"--access_log",
type=bool,
default=False,
help="Enable or disable access log. Options: True or False (default: False)",
)
parser.add_argument(
"--reload",
type=bool,
default=False,
help="Enable or disable auto-reload. Useful for development. Options: True or False (default: False)",
)
global args
args = parser.parse_args()
try:
run_eosdash()
except Exception as ex:

83
src/akkudoktoreos/server/server.py
View File

@@ -2,12 +2,13 @@
import ipaddress
import re
import socket
import time
from typing import Optional, Union
from typing import Optional
import psutil
from loguru import logger
from pydantic import Field, IPvAnyAddress, field_validator
from pydantic import Field, field_validator
from akkudoktoreos.config.configabc import SettingsBaseModel
@@ -17,7 +18,29 @@ def get_default_host() -> str:
return "127.0.0.1"
def is_valid_ip_or_hostname(value: str) -> bool:
def get_host_ip() -> str:
"""IP address of the host machine.
This function determines the IP address used to communicate with the outside world
(e.g., for internet access), without sending any actual data. It does so by
opening a UDP socket connection to a public IP address (Google DNS).
Returns:
str: The local IP address as a string. Returns '127.0.0.1' if unable to determine.
Example:
>>> get_host_ip()
'192.168.1.42'
"""
try:
with socket.socket(socket.AF_INET, socket.SOCK_DGRAM) as s:
s.connect(("8.8.8.8", 80))
return s.getsockname()[0]
except Exception:
return "127.0.0.1"
def validate_ip_or_hostname(value: str) -> str:
"""Validate whether a string is a valid IP address (IPv4 or IPv6) or hostname.
This function first attempts to interpret the input as an IP address using the
@@ -29,23 +52,30 @@ def is_valid_ip_or_hostname(value: str) -> bool:
value (str): The input string to validate.
Returns:
bool: True if the input is a valid IP address or hostname, False otherwise.
IP address: Valid IP address or hostname.
"""
try:
ipaddress.ip_address(value)
return True
return value
except ValueError:
pass
if len(value) > 253:
return False
raise ValueError(f"Not a valid hostname: {value}")
hostname_regex = re.compile(
r"^(?=.{1,253}$)(?!-)[A-Z\d-]{1,63}(?<!-)"
r"(?:\.(?!-)[A-Z\d-]{1,63}(?<!-))*\.?$",
re.IGNORECASE,
)
return bool(hostname_regex.fullmatch(value))
if not bool(hostname_regex.fullmatch(value)):
raise ValueError(f"Not a valid hostname: {value}")
ip = socket.gethostbyname(value)
if ip is None:
raise ValueError(f"Unknown host: {value}")
return value
def wait_for_port_free(port: int, timeout: int = 0, waiting_app_name: str = "App") -> bool:
@@ -121,28 +151,39 @@ def wait_for_port_free(port: int, timeout: int = 0, waiting_app_name: str = "App
class ServerCommonSettings(SettingsBaseModel):
"""Server Configuration."""
host: Optional[IPvAnyAddress] = Field(
default=get_default_host(), description="EOS server IP address."
host: Optional[str] = Field(
default=get_default_host(),
description="EOS server IP address. Defaults to 127.0.0.1.",
examples=["127.0.0.1", "localhost"],
)
port: Optional[int] = Field(
default=8503,
description="EOS server IP port number. Defaults to 8503.",
examples=[
8503,
],
)
port: Optional[int] = Field(default=8503, description="EOS server IP port number.")
verbose: Optional[bool] = Field(default=False, description="Enable debug output")
startup_eosdash: Optional[bool] = Field(
default=True, description="EOS server to start EOSdash server."
default=True, description="EOS server to start EOSdash server. Defaults to True."
)
eosdash_host: Optional[IPvAnyAddress] = Field(
default=get_default_host(), description="EOSdash server IP address."
eosdash_host: Optional[str] = Field(
default=None,
description="EOSdash server IP address. Defaults to EOS server IP address.",
examples=["127.0.0.1", "localhost"],
)
eosdash_port: Optional[int] = Field(
default=None,
description="EOSdash server IP port number. Defaults to EOS server IP port number + 1.",
examples=[
8504,
],
)
eosdash_port: Optional[int] = Field(default=8504, description="EOSdash server IP port number.")
@field_validator("host", "eosdash_host", mode="before")
def validate_server_host(
cls, value: Optional[Union[str, IPvAnyAddress]]
) -> Optional[Union[str, IPvAnyAddress]]:
def validate_server_host(cls, value: Optional[str]) -> Optional[str]:
if isinstance(value, str):
if not is_valid_ip_or_hostname(value):
raise ValueError(f"Invalid host: {value}")
if value.lower() in ("localhost", "loopback"):
value = "127.0.0.1"
value = validate_ip_or_hostname(value)
return value
@field_validator("port", "eosdash_port")

1467
src/akkudoktoreos/utils/datetimeutil.py
View File

File diff suppressed because it is too large Load Diff

44
src/akkudoktoreos/utils/docs.py
View File

@@ -1,44 +0,0 @@
from typing import Any
from pydantic.fields import FieldInfo
from akkudoktoreos.core.pydantic import PydanticBaseModel
def get_example_or_default(field_name: str, field_info: FieldInfo, example_ix: int) -> Any:
"""Generate a default value for a field, considering constraints."""
if field_info.examples is not None:
try:
return field_info.examples[example_ix]
except IndexError:
return field_info.examples[-1]
if field_info.default is not None:
return field_info.default
raise NotImplementedError(f"No default or example provided '{field_name}': {field_info}")
def get_model_structure_from_examples(
model_class: type[PydanticBaseModel], multiple: bool
) -> list[dict[str, Any]]:
"""Create a model instance with default or example values, respecting constraints."""
example_max_length = 1
# Get first field with examples (non-default) to get example_max_length
if multiple:
for _, field_info in model_class.model_fields.items():
if field_info.examples is not None:
example_max_length = len(field_info.examples)
break
example_data: list[dict[str, Any]] = [{} for _ in range(example_max_length)]
for field_name, field_info in model_class.model_fields.items():
if field_info.deprecated:
continue
for example_ix in range(example_max_length):
example_data[example_ix][field_name] = get_example_or_default(
field_name, field_info, example_ix
)
return example_data

53
src/akkudoktoreos/utils/stringutil.py Normal file
View File

@@ -0,0 +1,53 @@
"""Utility module for string-to-boolean conversion."""
from typing import Any
def str2bool(value: Any) -> bool:
"""Convert a string or boolean value to a boolean.
This function normalizes common textual representations of truthy and
falsy values (case-insensitive). It also accepts an existing boolean
and returns it unchanged.
Accepted truthy values:
- "yes", "y", "true", "t", "1", "on"
Accepted falsy values:
- "no", "n", "false", "f", "0", "off"
Args:
value (Union[str, bool]): The input value to convert. Can be a string
(e.g., "true", "no", "on") or a boolean.
Returns:
bool: The corresponding boolean value.
Raises:
ValueError: If the input cannot be interpreted as a boolean.
Examples:
>>> str2bool("yes")
True
>>> str2bool("OFF")
False
>>> str2bool(True)
True
>>> str2bool("n")
False
>>> str2bool("maybe")
Traceback (most recent call last):
...
ValueError: Invalid boolean value: maybe
"""
if isinstance(value, bool):
return value
if isinstance(value, str):
val = value.strip().lower()
if val in ("yes", "y", "true", "t", "1", "on"):
return True
if val in ("no", "n", "false", "f", "0", "off"):
return False
raise ValueError(f"Invalid boolean value: {value}")

12
src/akkudoktoreos/utils/visualize.py
View File

@@ -12,9 +12,9 @@ import pendulum
from matplotlib.backends.backend_pdf import PdfPages
from akkudoktoreos.core.coreabc import ConfigMixin
from akkudoktoreos.core.ems import EnergyManagement
from akkudoktoreos.optimization.genetic import OptimizationParameters
from akkudoktoreos.utils.datetimeutil import to_datetime
from akkudoktoreos.core.ems import get_ems
from akkudoktoreos.optimization.genetic.genetic import GeneticOptimizationParameters
from akkudoktoreos.utils.datetimeutil import DateTime, to_datetime
matplotlib.use(
"Agg"
@@ -137,7 +137,7 @@ class VisualizationReport(ConfigMixin):
def create_line_chart_date(
self,
start_date: pendulum.DateTime,
start_date: DateTime,
y_list: list[Union[np.ndarray, list[Optional[float]], list[float]]],
ylabel: str,
xlabel: Optional[str] = None,
@@ -436,7 +436,7 @@ class VisualizationReport(ConfigMixin):
def prepare_visualize(
parameters: OptimizationParameters,
parameters: GeneticOptimizationParameters,
results: dict,
filename: str = "visualization_results.pdf",
start_hour: int = 0,
@@ -444,7 +444,7 @@ def prepare_visualize(
global debug_visualize
report = VisualizationReport(filename)
next_full_hour_date = EnergyManagement.set_start_datetime()
next_full_hour_date = get_ems().start_datetime
# Group 1:
report.create_line_chart_date(
next_full_hour_date,