Commit Graph
3 Commits
Author SHA1 Message Date
8224c64654 feat: integrate device and runtime configuration foundation on main (#1328)
* feat: adapt configuration for multi optimization algorithms

Decouple configuration from optimization algorithm parameters. Add to_[algorithm]_param() methods
to the configuration that derive optimization algorithm specific parameters from the configuration.
Add x-scope tags to the configuration options that describe for which specific algorithms the
configuration option is for.

The whole device settings are restructured. There are now general settings for the device classes
with the afore mentioned to_[algorithm]_param() methods. The general device settings got their own
directory `devices/settings`. By this the parameter class also does not have to be a pydantic model
which can be used for future optimization/ simulations speed up.

Also the parameter class for a device is now part of the device module. This better decouples and
also is the natural place for parameters of a device.

Besides this feature there are also fixes and improvements:

* feat: extend home appliance time window settings and simulation

  Home appliance can now be configured for multiple runs with per-cycle allowed time windows. The
  number of remaining cycles to plan is determined at runtime by reading the
  ``cycles_completed_measurement_key`` from the measurement store.

* feat: specialiced CycleTimeWindowSequence for time window sequences

  Sequence of time windows associated to cycles.

  This model specializes ``ValueTimeWindowSequence`` so that the ``value``
  field of each ``ValueTimeWindow`` encodes the **cycle index** (0-based
  integer) the window belongs to.

  Typical use: an appliance that must run ``n`` times per day, each run
  constrained to a distinct time window.  Assign ``value=0`` to windows
  for the first cycle, ``value=1`` for the second, and so on.  Multiple
  windows may share the same cycle index (their allowed regions are unioned).
  Windows with ``value=None`` are silently ignored by all cycle-aware methods.

* fix: Make test_configmigrate also regard the _ANY_SENTENIEL in key values

* chore: Make devices configurations a map instead of a list

  This makes config paths stable regardless of declaration order and lets each device settings
  class build its own config path from ``self.device_id`` without needing an external index.
  Tests are adapted likewise.

  Devices configurations are automatically migrated from lists to maps.

* chore: rename levelized_cost_of_storage_kwh to levelized_cost_of_storage_amt kwh

  This better fits in the naming scheme and also makes clear the costs are money.

Signed-off-by: Bobby Noelte <b0661n0e17e@gmail.com>

* fix: runtime config update ignored by config file

Runtime settings were handed back to pydantic-settings as init settings,
which rank below the config file and the environment. Any key already
present in EOS.config.json or in the environment silently discarded the
update, so a bulk PUT /v1/config returned 200 without applying anything,
while the granular PUT /v1/config/{path} endpoint kept working.

Add a dedicated runtime settings source ranked directly below the command
line arguments and record granular updates there as well, so both
endpoints share one store that survives re-evaluation of the settings
sources. Environment variables keep precedence over the config file for
all keys that were not set at runtime.

Also repairs revert_settings() and update(), which passed their data
through the same init settings.

Closes #1303

* fix: env vars ignored on first config build

ConfigEOS.__init__ passed self as first positional argument to _setup,
which forwards it to pydantic_settings.BaseSettings.__init__. Its first
positional parameter is _case_sensitive, so the environment source
matched the upper case variable names against the lower case field names
and returned nothing. Environment settings only took effect after the
next configuration setup.

* docs: changelog for config priority fixes

* fix(config): preserve device identities and storage costs during migration

* fix(devices): preserve charge-rate typing and public import compatibility

* ruff format fix

* fix(config): satisfy typed device conversion and migration contracts

* docs(config): refresh validated configuration prerequisite schemas

---------

Signed-off-by: Bobby Noelte <b0661n0e17e@gmail.com>
Co-authored-by: Bobby Noelte <b0661n0e17e@gmail.com>
Co-authored-by: r0b2g1t <r0b2g1t@users.noreply.github.com>
2026-09-17 17:51:40 +02:00
Bobby NoelteandGitHub 1905682113 chore: adapt pdf visualization (#1205)
Change PDF visualization to be created on demand and per optimization algorithm. The PDF
for the GENETIC0 optimization is provided by the /visualization_results.pdf endpoint.
There is no change in the interface.

By this the optimization algorithm is offloaded from the PDF generation which spares some
time.

To cope with several users may call the /visualization_results.pdf endpoint at the same
time the PDF is generated on the fly without any intermediate file taking the stored
GENETIC0 solution as an input. SVG picture generation is removed as this would again
create intermediate files. Chart pictures can easily be taken from the PDF.

To allow on demand creation of the optimization results visualization the optimisation
solution stored is extended by several new attributes. To keep the deprecated
/optimize endpoint compatible the optimization solution is stripped to the legacy
content before returned. Due to the extension of the solution the optimization tests were
adapted to cover the extended content.

The optimization tests are adapted to test the generated visualization report by
the pypdf reader. Pypdf is added to the development dependencies.

Besides the adaptation several fixes and improvements are added:

* feat: extend /v1/prediction/series endpoint by resampling and filling

  Add parameters for resampling and filling. Add the processing parameter
  to control wether raw data or resampled data shall be returned.

* feat: extend /v1/measurement/series endpoint by resampling and filling

  Add parameters for resampling and filling: Add the processing parameter
  to control wether raw data or resampled data shall be returned.

* feat: standardize and improve API error response

  Use FASTApi exception handlers to provide a standardized API exception handling.
  All exceptions are logged.

  Exception traces are only returned if the new logging configuration parameter
  logging.api_logging_level is set to "DEBUG" or "TRACE". Avoids unwanted leackage
  of server internals on exceptions.

* fix: align to intervall when resampling

  Ensure resampling is aligned to interval also when the buckets are shifted due to the
  align_to_intervall parameter is set.

* chore: make dropna mandatory and default to True

* chore: refactor key_to_xxx data management methods

  Make key_to_series the central method for data resampling and fill.
  Add a new key_to_raw_series to retrieve the data as it is stored
  (without resampling and filling).

  Users of key_to_series were mostly moved to key_to_raw_series as this resembles
  the former interface. Especially in predictions and tests this was done.

* chore: create test data sub-directory for each optimization algorithm

  To prevent cluttering the test data directory and ease test data management for
  optimization algorithms each algorithm got it's own sub-directory. The current
  test data was moved to these sub-directories.

* chore: update version

Signed-off-by: Bobby Noelte <b0661n0e17e@gmail.com>
2026-08-01 12:45:19 +02:00
Bobby NoelteandGitHub e23bb7b497 chore: prepare for update of genetic algorithm (#1190)
Bump Version / Bump Version Workflow (push) Has been cancelled
CodeQL Advanced / Analyze (actions) (push) Has been cancelled
CodeQL Advanced / Analyze (python) (push) Has been cancelled
docker-build / platform-excludes (push) Has been cancelled
docker-build / build (push) Has been cancelled
docker-build / merge (push) Has been cancelled
pre-commit / pre-commit (push) Has been cancelled
Run Pytest on Pull Request / test (push) Has been cancelled
Andreas will update the genetic algorithm for 15-minutes optimization
intervals.

Copy the current GENETIC optimization algorithm to GENETIC0 to enable
to keep the algorithm with the current functionality. Also copy resources
like the load interpolator to the GENETIC0 algorithm to keep them despite
possible later changes to the interpolator.

Make the deprecated legacy /optimize endpoint use the GENETIC0 optimization
algorithm to in-fact behave the same way even if there will later be changes
to the GENETIC algorithm by Andreas. Add a new REST endpoint to provide
the unprocessed optimisation results of the GENETIC and GENETIC0 algorithm
in case one wants to use them as done with the deprecated /optimize endpoint.

Adapt the optimization configuration to have distinct configurations for the
GENETIC and the GENETIC0 algorithm.

Create a copy of the current tests for the GENETIC algorithm to be used
for the GENETIC0 algorithm. This avoids the tests for the GENETIC0
algorithm to be influenced by later changes by Andreas.

Signed-off-by: Bobby Noelte <b0661n0e17e@gmail.com>
2026-07-29 12:56:08 +02:00