2025-01-12 14:33:02 +01:00
|
|
|
import json
|
2024-12-24 13:10:31 +01:00
|
|
|
import os
|
2025-01-12 14:33:02 +01:00
|
|
|
import textwrap
|
2024-12-24 13:10:31 +01:00
|
|
|
from collections.abc import Sequence
|
|
|
|
|
from typing import Callable, Optional, Union
|
|
|
|
|
|
2025-01-14 13:27:51 +01:00
|
|
|
import matplotlib
|
2025-01-13 21:42:52 +01:00
|
|
|
import matplotlib.dates as mdates
|
2024-12-24 13:10:31 +01:00
|
|
|
import matplotlib.pyplot as plt
|
|
|
|
|
import numpy as np
|
2025-01-13 21:42:52 +01:00
|
|
|
import pendulum
|
2024-12-24 13:10:31 +01:00
|
|
|
from matplotlib.backends.backend_pdf import PdfPages
|
|
|
|
|
|
Add database support for measurements and historic prediction data. (#848)
The database supports backend selection, compression, incremental data load,
automatic data saving to storage, automatic vaccum and compaction.
Make SQLite3 and LMDB database backends available.
Update tests for new interface conventions regarding data sequences,
data containers, data providers. This includes the measurements provider and
the prediction providers.
Add database documentation.
The fix includes several bug fixes that are not directly related to the database
implementation but are necessary to keep EOS running properly and to test and
document the changes.
* fix: config eos test setup
Make the config_eos fixture generate a new instance of the config_eos singleton.
Use correct env names to setup data folder path.
* fix: startup with no config
Make cache and measurements complain about missing data path configuration but
do not bail out.
* fix: soc data preparation and usage for genetic optimization.
Search for soc measurments 48 hours around the optimization start time.
Only clamp soc to maximum in battery device simulation.
* fix: dashboard bailout on zero value solution display
Do not use zero values to calculate the chart values adjustment for display.
* fix: openapi generation script
Make the script also replace data_folder_path and data_output_path to hide
real (test) environment pathes.
* feat: add make repeated task function
make_repeated_task allows to wrap a function to be repeated cyclically.
* chore: removed index based data sequence access
Index based data sequence access does not make sense as the sequence can be backed
by the database. The sequence is now purely time series data.
* chore: refactor eos startup to avoid module import startup
Avoid module import initialisation expecially of the EOS configuration.
Config mutation, singleton initialization, logging setup, argparse parsing,
background task definitions depending on config and environment-dependent behavior
is now done at function startup.
* chore: introduce retention manager
A single long-running background task that owns the scheduling of all periodic
server-maintenance jobs (cache cleanup, DB autosave, …)
* chore: canonicalize timezone name for UTC
Timezone names that are semantically identical to UTC are canonicalized to UTC.
* chore: extend config file migration for default value handling
Extend the config file migration handling values None or nonexisting values
that will invoke a default value generation in the new config file. Also
adapt test to handle this situation.
* chore: extend datetime util test cases
* chore: make version test check for untracked files
Check for files that are not tracked by git. Version calculation will be
wrong if these files will not be commited.
* chore: bump pandas to 3.0.0
Pandas 3.0 now performs inference on the appropriate resolution (a.k.a. unit)
for the output dtype which may become datetime64[us] (before it was ns). Also
numeric dtype detection is now more strict which needs a different detection for
numerics.
* chore: bump pydantic-settings to 2.12.0
pydantic-settings 2.12.0 under pytest creates a different behaviour. The tests
were adapted and a workaround was introduced. Also ConfigEOS was adapted
to allow for fine grain initialization control to be able to switch
off certain settings such as file settings during test.
* chore: remove sci learn kit from dependencies
The sci learn kit is not strictly necessary as long as we have scipy.
* chore: add documentation mode guarding for sphinx autosummary
Sphinx autosummary excecutes functions. Prevent exceptions in case of pure doc
mode.
* chore: adapt docker-build CI workflow to stricter GitHub handling
Signed-off-by: Bobby Noelte <b0661n0e17e@gmail.com>
2026-02-22 14:12:42 +01:00
|
|
|
from akkudoktoreos.core.coreabc import ConfigMixin, get_ems
|
fix: automatic optimization (#596)
This fix implements the long term goal to have the EOS server run optimization (or
energy management) on regular intervals automatically. Thus clients can request
the current energy management plan at any time and it is updated on regular
intervals without interaction by the client.
This fix started out to "only" make automatic optimization (or energy management)
runs working. It turned out there are several endpoints that in some way
update predictions or run the optimization. To lock against such concurrent attempts
the code had to be refactored to allow control of execution. During refactoring it
became clear that some classes and files are named without a proper reference
to their usage. Thus not only refactoring but also renaming became necessary.
The names are still not the best, but I hope they are more intuitive.
The fix includes several bug fixes that are not directly related to the automatic optimization
but are necessary to keep EOS running properly to do the automatic optimization and
to test and document the changes.
This is a breaking change as the configuration structure changed once again and
the server API was also enhanced and streamlined. The server API that is used by
Andreas and Jörg in their videos has not changed.
* fix: automatic optimization
Allow optimization to automatically run on configured intervals gathering all
optimization parameters from configuration and predictions. The automatic run
can be configured to only run prediction updates skipping the optimization.
Extend documentaion to also cover automatic optimization. Lock automatic runs
against runs initiated by the /optimize or other endpoints. Provide new
endpoints to retrieve the energy management plan and the genetic solution
of the latest automatic optimization run. Offload energy management to thread
pool executor to keep the app more responsive during the CPU heavy optimization
run.
* fix: EOS servers recognize environment variables on startup
Force initialisation of EOS configuration on server startup to assure
all sources of EOS configuration are properly set up and read. Adapt
server tests and configuration tests to also test for environment
variable configuration.
* fix: Remove 0.0.0.0 to localhost translation under Windows
EOS imposed a 0.0.0.0 to localhost translation under Windows for
convenience. This caused some trouble in user configurations. Now, as the
default IP address configuration is 127.0.0.1, the user is responsible
for to set up the correct Windows compliant IP address.
* fix: allow names for hosts additional to IP addresses
* fix: access pydantic model fields by class
Access by instance is deprecated.
* fix: down sampling key_to_array
* fix: make cache clear endpoint clear all cache files
Make /v1/admin/cache/clear clear all cache files. Before it only cleared
expired cache files by default. Add new endpoint /v1/admin/clear-expired
to only clear expired cache files.
* fix: timezonefinder returns Europe/Paris instead of Europe/Berlin
timezonefinder 8.10 got more inaccurate for timezones in europe as there is
a common timezone. Use new package tzfpy instead which is still returning
Europe/Berlin if you are in Germany. tzfpy also claims to be faster than
timezonefinder.
* fix: provider settings configuration
Provider configuration used to be a union holding the settings for several
providers. Pydantic union handling does not always find the correct type
for a provider setting. This led to exceptions in specific configurations.
Now provider settings are explicit comfiguration items for each possible
provider. This is a breaking change as the configuration structure was
changed.
* fix: ClearOutside weather prediction irradiance calculation
Pvlib needs a pandas time index. Convert time index.
* fix: test config file priority
Do not use config_eos fixture as this fixture already creates a config file.
* fix: optimization sample request documentation
Provide all data in documentation of optimization sample request.
* fix: gitlint blocking pip dependency resolution
Replace gitlint by commitizen. Gitlint is not actively maintained anymore.
Gitlint dependencies blocked pip from dependency resolution.
* fix: sync pre-commit config to actual dependency requirements
.pre-commit-config.yaml was out of sync, also requirements-dev.txt.
* fix: missing babel in requirements.txt
Add babel to requirements.txt
* feat: setup default device configuration for automatic optimization
In case the parameters for automatic optimization are not fully defined a
default configuration is setup to allow the automatic energy management
run. The default configuration may help the user to correctly define
the device configuration.
* feat: allow configuration of genetic algorithm parameters
The genetic algorithm parameters for number of individuals, number of
generations, the seed and penalty function parameters are now avaliable
as configuration options.
* feat: allow configuration of home appliance time windows
The time windows a home appliance is allowed to run are now configurable
by the configuration (for /v1 API) and also by the home appliance parameters
(for the classic /optimize API). If there is no such configuration the
time window defaults to optimization hours, which was the standard before
the change. Documentation on how to configure time windows is added.
* feat: standardize mesaurement keys for battery/ ev SoC measurements
The standardized measurement keys to report battery SoC to the device
simulations can now be retrieved from the device configuration as a
read-only config option.
* feat: feed in tariff prediction
Add feed in tarif predictions needed for automatic optimization. The feed in
tariff can be retrieved as fixed feed in tarif or can be imported. Also add
tests for the different feed in tariff providers. Extend documentation to
cover the feed in tariff providers.
* feat: add energy management plan based on S2 standard instructions
EOS can generate an energy management plan as a list of simple instructions.
May be retrieved by the /v1/energy-management/plan endpoint. The instructions
loosely follow the S2 energy management standard.
* feat: make measurement keys configurable by EOS configuration.
The fixed measurement keys are replaced by configurable measurement keys.
* feat: make pendulum DateTime, Date, Duration types usable for pydantic models
Use pydantic_extra_types.pendulum_dt to get pydantic pendulum types. Types are
added to the datetimeutil utility. Remove custom made pendulum adaptations
from EOS pydantic module. Make EOS modules use the pydantic pendulum types
managed by the datetimeutil module instead of the core pendulum types.
* feat: Add Time, TimeWindow, TimeWindowSequence and to_time to datetimeutil.
The time windows are are added to support home appliance time window
configuration. All time classes are also pydantic models. Time is the base
class for time definition derived from pendulum.Time.
* feat: Extend DataRecord by configurable field like data.
Configurable field like data was added to support the configuration of
measurement records.
* feat: Add additional information to health information
Version information is added to the health endpoints of eos and eosDash.
The start time of the last optimization and the latest run time of the energy
management is added to the EOS health information.
* feat: add pydantic merge model tests
* feat: add plan tab to EOSdash
The plan tab displays the current energy management instructions.
* feat: add predictions tab to EOSdash
The predictions tab displays the current predictions.
* feat: add cache management to EOSdash admin tab
The admin tab is extended by a section for cache management. It allows to
clear the cache.
* feat: add about tab to EOSdash
The about tab resembles the former hello tab and provides extra information.
* feat: Adapt changelog and prepare for release management
Release management using commitizen is added. The changelog file is adapted and
teh changelog and a description for release management is added in the
documentation.
* feat(doc): Improve install and devlopment documentation
Provide a more concise installation description in Readme.md and add extra
installation page and development page to documentation.
* chore: Use memory cache for interpolation instead of dict in inverter
Decorate calculate_self_consumption() with @cachemethod_until_update to cache
results in memory during an energy management/ optimization run. Replacement
of dict type caching in inverter is now possible because all optimization
runs are properly locked and the memory cache CacheUntilUpdateStore is properly
cleared at the start of any energy management/ optimization operation.
* chore: refactor genetic
Refactor the genetic algorithm modules for enhanced module structure and better
readability. Removed unnecessary and overcomplex devices singleton. Also
split devices configuration from genetic algorithm parameters to allow further
development independently from genetic algorithm parameter format. Move
charge rates configuration for electric vehicles from optimization to devices
configuration to allow to have different charge rates for different cars in
the future.
* chore: Rename memory cache to CacheEnergyManagementStore
The name better resembles the task of the cache to chache function and method
results for an energy management run. Also the decorator functions are renamed
accordingly: cachemethod_energy_management, cache_energy_management
* chore: use class properties for config/ems/prediction mixin classes
* chore: skip debug logs from mathplotlib
Mathplotlib is very noisy in debug mode.
* chore: automatically sync bokeh js to bokeh python package
bokeh was updated to 3.8.0, make JS CDN automatically follow the package version.
* chore: rename hello.py to about.py
Make hello.py the adapted EOSdash about page.
* chore: remove demo page from EOSdash
As no the plan and prediction pages are working without configuration, the demo
page is no longer necessary
* chore: split test_server.py for system test
Split test_server.py to create explicit test_system.py for system tests.
* chore: move doc utils to generate_config_md.py
The doc utils are only used in scripts/generate_config_md.py. Move it there to
attribute for strong cohesion.
* chore: improve pydantic merge model documentation
* chore: remove pendulum warning from readme
* chore: remove GitHub discussions from contributing documentation
Github discussions is to be replaced by Akkudoktor.net.
* chore(release): bump version to 0.1.0+dev for development
* build(deps): bump fastapi[standard] from 0.115.14 to 0.117.1
bump fastapi and make coverage version (for pytest-cov) explicit to avoid pip break.
* build(deps): bump uvicorn from 0.36.0 to 0.37.0
BREAKING CHANGE: EOS configuration changed. V1 API changed.
- The available_charge_rates_percent configuration is removed from optimization.
Use the new charge_rate configuration for the electric vehicle
- Optimization configuration parameter hours renamed to horizon_hours
- Device configuration now has to provide the number of devices and device
properties per device.
- Specific prediction provider configuration to be provided by explicit
configuration item (no union for all providers).
- Measurement keys to be provided as a list.
- New feed in tariff providers have to be configured.
- /v1/measurement/loadxxx endpoints are removed. Use generic mesaurement endpoints.
- /v1/admin/cache/clear now clears all cache files. Use
/v1/admin/cache/clear-expired to only clear all expired cache files.
Signed-off-by: Bobby Noelte <b0661n0e17e@gmail.com>
2025-10-28 02:50:31 +01:00
|
|
|
from akkudoktoreos.optimization.genetic.genetic import GeneticOptimizationParameters
|
|
|
|
|
from akkudoktoreos.utils.datetimeutil import DateTime, to_datetime
|
2024-12-24 13:10:31 +01:00
|
|
|
|
2025-01-14 13:27:51 +01:00
|
|
|
matplotlib.use(
|
|
|
|
|
"Agg"
|
|
|
|
|
) # non-interactive backend that can only write to files, backend needed to stay in main thread.
|
2025-01-12 14:33:02 +01:00
|
|
|
|
2024-12-24 13:10:31 +01:00
|
|
|
|
2025-06-10 22:00:28 +02:00
|
|
|
debug_visualize: bool = False
|
|
|
|
|
|
|
|
|
|
|
2024-12-24 13:10:31 +01:00
|
|
|
class VisualizationReport(ConfigMixin):
|
2025-01-26 18:27:09 +01:00
|
|
|
def __init__(
|
|
|
|
|
self,
|
|
|
|
|
filename: str = "visualization_results.pdf",
|
|
|
|
|
version: str = "0.0.1",
|
|
|
|
|
create_img: bool = True,
|
|
|
|
|
) -> None:
|
2024-12-24 13:10:31 +01:00
|
|
|
# Initialize the report with a given filename and empty groups
|
|
|
|
|
self.filename = filename
|
|
|
|
|
self.groups: list[list[Callable[[], None]]] = [] # Store groups of charts
|
|
|
|
|
self.current_group: list[
|
|
|
|
|
Callable[[], None]
|
|
|
|
|
] = [] # Store current group of charts being created
|
|
|
|
|
self.pdf_pages = PdfPages(filename, metadata={}) # Initialize PdfPages without metadata
|
2025-01-13 21:42:52 +01:00
|
|
|
self.version = version # overwrite version as test for constant output of pdf for test
|
2025-01-14 13:16:18 +01:00
|
|
|
self.current_time = to_datetime(
|
2025-01-20 22:58:59 +01:00
|
|
|
as_string="YYYY-MM-DD HH:mm:ss", in_timezone=self.config.general.timezone
|
2025-01-14 13:16:18 +01:00
|
|
|
)
|
2025-01-26 18:27:09 +01:00
|
|
|
self.create_img = create_img
|
2024-12-24 13:10:31 +01:00
|
|
|
|
2025-01-26 18:27:09 +01:00
|
|
|
def add_chart_to_group(self, chart_func: Callable[[], None], title: str | None) -> None:
|
|
|
|
|
"""Add a chart function to the current group and save it as a PNG and SVG."""
|
2024-12-24 13:10:31 +01:00
|
|
|
self.current_group.append(chart_func)
|
2025-01-26 18:27:09 +01:00
|
|
|
if self.create_img and title:
|
2025-02-12 21:35:51 +01:00
|
|
|
server_output_dir = self.config.cache.path()
|
2025-01-26 18:27:09 +01:00
|
|
|
server_output_dir.mkdir(parents=True, exist_ok=True)
|
|
|
|
|
fig, ax = plt.subplots()
|
|
|
|
|
chart_func()
|
|
|
|
|
plt.tight_layout() # Adjust the layout to ensure titles are not cut off
|
|
|
|
|
sanitized_title = "".join(c if c.isalnum() else "_" for c in title)
|
|
|
|
|
chart_filename_base = os.path.join(server_output_dir, f"chart_{sanitized_title}")
|
|
|
|
|
fig.savefig(f"{chart_filename_base}.svg")
|
|
|
|
|
plt.close(fig)
|
2024-12-24 13:10:31 +01:00
|
|
|
|
|
|
|
|
def finalize_group(self) -> None:
|
|
|
|
|
"""Finalize the current group and prepare for a new group."""
|
|
|
|
|
if self.current_group: # Check if current group has charts
|
|
|
|
|
self.groups.append(self.current_group) # Add current group to groups
|
|
|
|
|
else:
|
|
|
|
|
print("Finalizing an empty group!") # Warn if group is empty
|
|
|
|
|
self.current_group = [] # Reset current group for new charts
|
|
|
|
|
|
|
|
|
|
def _initialize_pdf(self) -> None:
|
|
|
|
|
"""Create the output directory if it doesn't exist and initialize the PDF."""
|
2025-01-12 05:19:37 +01:00
|
|
|
output_dir = self.config.general.data_output_path
|
2024-12-24 13:10:31 +01:00
|
|
|
|
|
|
|
|
# If self.filename is already a valid path, use it; otherwise, combine it with output_dir
|
|
|
|
|
if os.path.isabs(self.filename):
|
|
|
|
|
output_file = self.filename
|
|
|
|
|
else:
|
|
|
|
|
output_dir.mkdir(parents=True, exist_ok=True)
|
|
|
|
|
output_file = os.path.join(output_dir, self.filename)
|
|
|
|
|
|
|
|
|
|
self.pdf_pages = PdfPages(
|
|
|
|
|
output_file, metadata={}
|
|
|
|
|
) # Re-initialize PdfPages without metadata
|
|
|
|
|
|
|
|
|
|
def _save_group_to_pdf(self, group: list[Callable[[], None]]) -> None:
|
|
|
|
|
"""Save a group of charts to the PDF."""
|
|
|
|
|
fig_count = len(group) # Number of charts in the group
|
2025-01-12 14:33:02 +01:00
|
|
|
|
2024-12-24 13:10:31 +01:00
|
|
|
if fig_count == 0:
|
2025-01-12 14:33:02 +01:00
|
|
|
print("Attempted to save an empty group to PDF!")
|
|
|
|
|
return
|
2024-12-24 13:10:31 +01:00
|
|
|
|
2025-01-12 14:33:02 +01:00
|
|
|
# Check for special charts before creating layout
|
|
|
|
|
special_keywords = {"add_text_page", "add_json_page"}
|
|
|
|
|
for chart_func in group:
|
|
|
|
|
if any(keyword in chart_func.__qualname__ for keyword in special_keywords):
|
|
|
|
|
chart_func() # Special chart functions handle their own rendering
|
|
|
|
|
return
|
2024-12-24 13:10:31 +01:00
|
|
|
|
2025-01-12 14:33:02 +01:00
|
|
|
# Create layout only if no special charts are detected
|
|
|
|
|
if fig_count == 3:
|
|
|
|
|
fig = plt.figure(figsize=(14, 10))
|
|
|
|
|
ax1 = fig.add_subplot(2, 1, 1)
|
|
|
|
|
ax2 = fig.add_subplot(2, 2, 3)
|
|
|
|
|
ax3 = fig.add_subplot(2, 2, 4)
|
2024-12-24 13:10:31 +01:00
|
|
|
axs = [ax1, ax2, ax3]
|
|
|
|
|
else:
|
2025-01-12 14:33:02 +01:00
|
|
|
cols = 2 if fig_count > 1 else 1
|
|
|
|
|
rows = (fig_count + 1) // 2
|
|
|
|
|
fig, axs = plt.subplots(rows, cols, figsize=(14, 7 * rows))
|
2024-12-24 13:10:31 +01:00
|
|
|
axs = list(np.array(axs).reshape(-1))
|
|
|
|
|
|
2025-01-13 21:42:52 +01:00
|
|
|
# Add footer text with current time to each page
|
|
|
|
|
if self.version == "test":
|
|
|
|
|
current_time = "test"
|
|
|
|
|
else:
|
|
|
|
|
current_time = self.current_time
|
|
|
|
|
fig.text(
|
|
|
|
|
0.5,
|
|
|
|
|
0.02,
|
|
|
|
|
f"Generated on: {current_time} with version: {self.version}",
|
|
|
|
|
ha="center",
|
|
|
|
|
va="center",
|
|
|
|
|
fontsize=10,
|
|
|
|
|
)
|
|
|
|
|
|
2025-01-12 14:33:02 +01:00
|
|
|
# Render each chart in its corresponding axis
|
2024-12-24 13:10:31 +01:00
|
|
|
for idx, chart_func in enumerate(group):
|
2025-01-12 14:33:02 +01:00
|
|
|
plt.sca(axs[idx]) # Set current axis
|
|
|
|
|
chart_func() # Render the chart
|
2024-12-24 13:10:31 +01:00
|
|
|
|
2025-01-12 14:33:02 +01:00
|
|
|
# Save the figure to the PDF and clean up
|
2024-12-24 13:10:31 +01:00
|
|
|
for idx in range(fig_count, len(axs)):
|
2025-01-12 14:33:02 +01:00
|
|
|
axs[idx].set_visible(False)
|
2024-12-24 13:10:31 +01:00
|
|
|
|
2025-01-12 14:33:02 +01:00
|
|
|
self.pdf_pages.savefig(fig) # Save the figure to the PDF
|
|
|
|
|
plt.close(fig)
|
2024-12-24 13:10:31 +01:00
|
|
|
|
2025-01-13 21:42:52 +01:00
|
|
|
def create_line_chart_date(
|
|
|
|
|
self,
|
fix: automatic optimization (#596)
This fix implements the long term goal to have the EOS server run optimization (or
energy management) on regular intervals automatically. Thus clients can request
the current energy management plan at any time and it is updated on regular
intervals without interaction by the client.
This fix started out to "only" make automatic optimization (or energy management)
runs working. It turned out there are several endpoints that in some way
update predictions or run the optimization. To lock against such concurrent attempts
the code had to be refactored to allow control of execution. During refactoring it
became clear that some classes and files are named without a proper reference
to their usage. Thus not only refactoring but also renaming became necessary.
The names are still not the best, but I hope they are more intuitive.
The fix includes several bug fixes that are not directly related to the automatic optimization
but are necessary to keep EOS running properly to do the automatic optimization and
to test and document the changes.
This is a breaking change as the configuration structure changed once again and
the server API was also enhanced and streamlined. The server API that is used by
Andreas and Jörg in their videos has not changed.
* fix: automatic optimization
Allow optimization to automatically run on configured intervals gathering all
optimization parameters from configuration and predictions. The automatic run
can be configured to only run prediction updates skipping the optimization.
Extend documentaion to also cover automatic optimization. Lock automatic runs
against runs initiated by the /optimize or other endpoints. Provide new
endpoints to retrieve the energy management plan and the genetic solution
of the latest automatic optimization run. Offload energy management to thread
pool executor to keep the app more responsive during the CPU heavy optimization
run.
* fix: EOS servers recognize environment variables on startup
Force initialisation of EOS configuration on server startup to assure
all sources of EOS configuration are properly set up and read. Adapt
server tests and configuration tests to also test for environment
variable configuration.
* fix: Remove 0.0.0.0 to localhost translation under Windows
EOS imposed a 0.0.0.0 to localhost translation under Windows for
convenience. This caused some trouble in user configurations. Now, as the
default IP address configuration is 127.0.0.1, the user is responsible
for to set up the correct Windows compliant IP address.
* fix: allow names for hosts additional to IP addresses
* fix: access pydantic model fields by class
Access by instance is deprecated.
* fix: down sampling key_to_array
* fix: make cache clear endpoint clear all cache files
Make /v1/admin/cache/clear clear all cache files. Before it only cleared
expired cache files by default. Add new endpoint /v1/admin/clear-expired
to only clear expired cache files.
* fix: timezonefinder returns Europe/Paris instead of Europe/Berlin
timezonefinder 8.10 got more inaccurate for timezones in europe as there is
a common timezone. Use new package tzfpy instead which is still returning
Europe/Berlin if you are in Germany. tzfpy also claims to be faster than
timezonefinder.
* fix: provider settings configuration
Provider configuration used to be a union holding the settings for several
providers. Pydantic union handling does not always find the correct type
for a provider setting. This led to exceptions in specific configurations.
Now provider settings are explicit comfiguration items for each possible
provider. This is a breaking change as the configuration structure was
changed.
* fix: ClearOutside weather prediction irradiance calculation
Pvlib needs a pandas time index. Convert time index.
* fix: test config file priority
Do not use config_eos fixture as this fixture already creates a config file.
* fix: optimization sample request documentation
Provide all data in documentation of optimization sample request.
* fix: gitlint blocking pip dependency resolution
Replace gitlint by commitizen. Gitlint is not actively maintained anymore.
Gitlint dependencies blocked pip from dependency resolution.
* fix: sync pre-commit config to actual dependency requirements
.pre-commit-config.yaml was out of sync, also requirements-dev.txt.
* fix: missing babel in requirements.txt
Add babel to requirements.txt
* feat: setup default device configuration for automatic optimization
In case the parameters for automatic optimization are not fully defined a
default configuration is setup to allow the automatic energy management
run. The default configuration may help the user to correctly define
the device configuration.
* feat: allow configuration of genetic algorithm parameters
The genetic algorithm parameters for number of individuals, number of
generations, the seed and penalty function parameters are now avaliable
as configuration options.
* feat: allow configuration of home appliance time windows
The time windows a home appliance is allowed to run are now configurable
by the configuration (for /v1 API) and also by the home appliance parameters
(for the classic /optimize API). If there is no such configuration the
time window defaults to optimization hours, which was the standard before
the change. Documentation on how to configure time windows is added.
* feat: standardize mesaurement keys for battery/ ev SoC measurements
The standardized measurement keys to report battery SoC to the device
simulations can now be retrieved from the device configuration as a
read-only config option.
* feat: feed in tariff prediction
Add feed in tarif predictions needed for automatic optimization. The feed in
tariff can be retrieved as fixed feed in tarif or can be imported. Also add
tests for the different feed in tariff providers. Extend documentation to
cover the feed in tariff providers.
* feat: add energy management plan based on S2 standard instructions
EOS can generate an energy management plan as a list of simple instructions.
May be retrieved by the /v1/energy-management/plan endpoint. The instructions
loosely follow the S2 energy management standard.
* feat: make measurement keys configurable by EOS configuration.
The fixed measurement keys are replaced by configurable measurement keys.
* feat: make pendulum DateTime, Date, Duration types usable for pydantic models
Use pydantic_extra_types.pendulum_dt to get pydantic pendulum types. Types are
added to the datetimeutil utility. Remove custom made pendulum adaptations
from EOS pydantic module. Make EOS modules use the pydantic pendulum types
managed by the datetimeutil module instead of the core pendulum types.
* feat: Add Time, TimeWindow, TimeWindowSequence and to_time to datetimeutil.
The time windows are are added to support home appliance time window
configuration. All time classes are also pydantic models. Time is the base
class for time definition derived from pendulum.Time.
* feat: Extend DataRecord by configurable field like data.
Configurable field like data was added to support the configuration of
measurement records.
* feat: Add additional information to health information
Version information is added to the health endpoints of eos and eosDash.
The start time of the last optimization and the latest run time of the energy
management is added to the EOS health information.
* feat: add pydantic merge model tests
* feat: add plan tab to EOSdash
The plan tab displays the current energy management instructions.
* feat: add predictions tab to EOSdash
The predictions tab displays the current predictions.
* feat: add cache management to EOSdash admin tab
The admin tab is extended by a section for cache management. It allows to
clear the cache.
* feat: add about tab to EOSdash
The about tab resembles the former hello tab and provides extra information.
* feat: Adapt changelog and prepare for release management
Release management using commitizen is added. The changelog file is adapted and
teh changelog and a description for release management is added in the
documentation.
* feat(doc): Improve install and devlopment documentation
Provide a more concise installation description in Readme.md and add extra
installation page and development page to documentation.
* chore: Use memory cache for interpolation instead of dict in inverter
Decorate calculate_self_consumption() with @cachemethod_until_update to cache
results in memory during an energy management/ optimization run. Replacement
of dict type caching in inverter is now possible because all optimization
runs are properly locked and the memory cache CacheUntilUpdateStore is properly
cleared at the start of any energy management/ optimization operation.
* chore: refactor genetic
Refactor the genetic algorithm modules for enhanced module structure and better
readability. Removed unnecessary and overcomplex devices singleton. Also
split devices configuration from genetic algorithm parameters to allow further
development independently from genetic algorithm parameter format. Move
charge rates configuration for electric vehicles from optimization to devices
configuration to allow to have different charge rates for different cars in
the future.
* chore: Rename memory cache to CacheEnergyManagementStore
The name better resembles the task of the cache to chache function and method
results for an energy management run. Also the decorator functions are renamed
accordingly: cachemethod_energy_management, cache_energy_management
* chore: use class properties for config/ems/prediction mixin classes
* chore: skip debug logs from mathplotlib
Mathplotlib is very noisy in debug mode.
* chore: automatically sync bokeh js to bokeh python package
bokeh was updated to 3.8.0, make JS CDN automatically follow the package version.
* chore: rename hello.py to about.py
Make hello.py the adapted EOSdash about page.
* chore: remove demo page from EOSdash
As no the plan and prediction pages are working without configuration, the demo
page is no longer necessary
* chore: split test_server.py for system test
Split test_server.py to create explicit test_system.py for system tests.
* chore: move doc utils to generate_config_md.py
The doc utils are only used in scripts/generate_config_md.py. Move it there to
attribute for strong cohesion.
* chore: improve pydantic merge model documentation
* chore: remove pendulum warning from readme
* chore: remove GitHub discussions from contributing documentation
Github discussions is to be replaced by Akkudoktor.net.
* chore(release): bump version to 0.1.0+dev for development
* build(deps): bump fastapi[standard] from 0.115.14 to 0.117.1
bump fastapi and make coverage version (for pytest-cov) explicit to avoid pip break.
* build(deps): bump uvicorn from 0.36.0 to 0.37.0
BREAKING CHANGE: EOS configuration changed. V1 API changed.
- The available_charge_rates_percent configuration is removed from optimization.
Use the new charge_rate configuration for the electric vehicle
- Optimization configuration parameter hours renamed to horizon_hours
- Device configuration now has to provide the number of devices and device
properties per device.
- Specific prediction provider configuration to be provided by explicit
configuration item (no union for all providers).
- Measurement keys to be provided as a list.
- New feed in tariff providers have to be configured.
- /v1/measurement/loadxxx endpoints are removed. Use generic mesaurement endpoints.
- /v1/admin/cache/clear now clears all cache files. Use
/v1/admin/cache/clear-expired to only clear all expired cache files.
Signed-off-by: Bobby Noelte <b0661n0e17e@gmail.com>
2025-10-28 02:50:31 +01:00
|
|
|
start_date: DateTime,
|
2025-01-13 21:42:52 +01:00
|
|
|
y_list: list[Union[np.ndarray, list[Optional[float]], list[float]]],
|
|
|
|
|
ylabel: str,
|
|
|
|
|
xlabel: Optional[str] = None,
|
|
|
|
|
title: Optional[str] = None,
|
|
|
|
|
labels: Optional[list[str]] = None,
|
|
|
|
|
markers: Optional[list[str]] = None,
|
|
|
|
|
line_styles: Optional[list[str]] = None,
|
|
|
|
|
x2label: Optional[Union[str, None]] = "Hours Since Start",
|
|
|
|
|
) -> None:
|
|
|
|
|
"""Create a line chart and add it to the current group."""
|
|
|
|
|
|
|
|
|
|
def chart() -> None:
|
|
|
|
|
timestamps = [
|
|
|
|
|
start_date.add(hours=i) for i in range(len(y_list[0]))
|
|
|
|
|
] # 840 timestamps at 1-hour intervals
|
|
|
|
|
|
|
|
|
|
for idx, y_data in enumerate(y_list):
|
|
|
|
|
label = labels[idx] if labels else None # Chart label
|
|
|
|
|
marker = markers[idx] if markers and idx < len(markers) else "o" # Marker style
|
|
|
|
|
line_style = line_styles[idx] if line_styles and idx < len(line_styles) else "-"
|
|
|
|
|
plt.plot(
|
|
|
|
|
timestamps, y_data, label=label, marker=marker, linestyle=line_style
|
|
|
|
|
) # Plot line
|
|
|
|
|
|
|
|
|
|
# Format the time axis
|
|
|
|
|
plt.gca().xaxis.set_major_formatter(
|
2025-02-08 00:47:21 +01:00
|
|
|
mdates.DateFormatter("%Y-%m-%d", tz=self.config.general.timezone)
|
2025-01-13 21:42:52 +01:00
|
|
|
) # Show date and time
|
|
|
|
|
plt.gca().xaxis.set_major_locator(
|
2025-02-08 00:47:21 +01:00
|
|
|
mdates.DayLocator(interval=1, tz=self.config.general.timezone)
|
2025-01-13 21:42:52 +01:00
|
|
|
) # Major ticks every day
|
2025-02-08 00:47:21 +01:00
|
|
|
plt.gca().xaxis.set_minor_locator(
|
|
|
|
|
mdates.HourLocator(interval=2, tz=self.config.general.timezone)
|
|
|
|
|
)
|
2025-01-13 21:42:52 +01:00
|
|
|
# Minor ticks every 6 hours
|
2025-02-08 00:47:21 +01:00
|
|
|
plt.gca().xaxis.set_minor_formatter(
|
|
|
|
|
mdates.DateFormatter("%H", tz=self.config.general.timezone)
|
|
|
|
|
)
|
2025-01-13 21:42:52 +01:00
|
|
|
# plt.gcf().autofmt_xdate(rotation=45, which="major")
|
|
|
|
|
# Auto-format the x-axis for readability
|
|
|
|
|
|
|
|
|
|
# Move major tick labels further down to avoid collision with minor tick labels
|
|
|
|
|
for plt_label in plt.gca().get_xticklabels(which="major"):
|
|
|
|
|
plt_label.set_y(-0.04)
|
|
|
|
|
|
|
|
|
|
# Add labels, title, and legend
|
|
|
|
|
if xlabel:
|
|
|
|
|
plt.xlabel(xlabel)
|
|
|
|
|
plt.ylabel(ylabel)
|
|
|
|
|
if title:
|
|
|
|
|
plt.title(title)
|
|
|
|
|
if labels:
|
|
|
|
|
plt.legend()
|
|
|
|
|
plt.grid(True)
|
|
|
|
|
|
|
|
|
|
# Add vertical line for the current date if within the axis range
|
2025-01-20 22:58:59 +01:00
|
|
|
current_time = pendulum.now(self.config.general.timezone)
|
2025-01-13 21:42:52 +01:00
|
|
|
if timestamps[0].subtract(hours=2) <= current_time <= timestamps[-1]:
|
|
|
|
|
plt.axvline(current_time, color="r", linestyle="--", label="Now")
|
|
|
|
|
plt.text(current_time, plt.ylim()[1], "Now", color="r", ha="center", va="bottom")
|
|
|
|
|
|
|
|
|
|
# Add a second x-axis on top
|
|
|
|
|
ax1 = plt.gca()
|
|
|
|
|
ax2 = ax1.twiny()
|
|
|
|
|
ax2.set_xlim(ax1.get_xlim()) # Align the second axis with the first
|
|
|
|
|
|
|
|
|
|
# Generate integer hour labels
|
|
|
|
|
hours_since_start = [(t - timestamps[0]).total_seconds() / 3600 for t in timestamps]
|
|
|
|
|
# ax2.set_xticks(timestamps[::48]) # Set ticks every 12 hours
|
|
|
|
|
# ax2.set_xticklabels([f"{int(h)}" for h in hours_since_start[::48]])
|
2025-02-08 00:47:21 +01:00
|
|
|
# ax2.set_xticks(timestamps[:: len(timestamps) // 24]) # Select 10 evenly spaced ticks
|
|
|
|
|
ax2.set_xticks(timestamps[:: len(timestamps) // 12]) # Select 10 evenly spaced ticks
|
|
|
|
|
# ax2.set_xticklabels([f"{int(h)}" for h in hours_since_start[:: len(timestamps) // 24]])
|
|
|
|
|
ax2.set_xticklabels([f"{int(h)}" for h in hours_since_start[:: len(timestamps) // 12]])
|
2025-01-13 21:42:52 +01:00
|
|
|
if x2label:
|
|
|
|
|
ax2.set_xlabel(x2label)
|
|
|
|
|
|
|
|
|
|
# Ensure ax1 and ax2 are aligned
|
|
|
|
|
# assert ax1.get_xlim() == ax2.get_xlim(), "ax1 and ax2 are not aligned"
|
|
|
|
|
|
2025-01-26 18:27:09 +01:00
|
|
|
self.add_chart_to_group(chart, title) # Add chart function to current group
|
2025-01-13 21:42:52 +01:00
|
|
|
|
2024-12-24 13:10:31 +01:00
|
|
|
def create_line_chart(
|
|
|
|
|
self,
|
|
|
|
|
start_hour: Optional[int],
|
2024-12-29 16:43:37 +01:00
|
|
|
y_list: list[Union[np.ndarray, list[Optional[float]], list[float]]],
|
2024-12-24 13:10:31 +01:00
|
|
|
title: str,
|
|
|
|
|
xlabel: str,
|
|
|
|
|
ylabel: str,
|
|
|
|
|
labels: Optional[list[str]] = None,
|
|
|
|
|
markers: Optional[list[str]] = None,
|
|
|
|
|
line_styles: Optional[list[str]] = None,
|
|
|
|
|
) -> None:
|
|
|
|
|
"""Create a line chart and add it to the current group."""
|
|
|
|
|
|
|
|
|
|
def chart() -> None:
|
|
|
|
|
nonlocal start_hour # Allow modifying `x` within the nested function
|
|
|
|
|
if start_hour is None:
|
|
|
|
|
start_hour = 0
|
|
|
|
|
first_element = y_list[0]
|
|
|
|
|
x: np.ndarray
|
|
|
|
|
# Case 1: y_list contains np.ndarray elements
|
|
|
|
|
if isinstance(first_element, np.ndarray):
|
|
|
|
|
x = np.arange(
|
|
|
|
|
start_hour, start_hour + len(first_element)
|
|
|
|
|
) # Start at x and extend by ndarray length
|
|
|
|
|
# Case 2: y_list contains float elements (1D list)
|
|
|
|
|
elif isinstance(first_element, float):
|
|
|
|
|
x = np.arange(
|
|
|
|
|
start_hour, start_hour + len(y_list)
|
|
|
|
|
) # Start at x and extend by list length
|
|
|
|
|
# Case 3: y_list is a nested list of floats
|
|
|
|
|
elif isinstance(first_element, list) and all(
|
|
|
|
|
isinstance(i, float) for i in first_element
|
|
|
|
|
):
|
|
|
|
|
max_len = max(len(sublist) for sublist in y_list)
|
|
|
|
|
x = np.arange(
|
|
|
|
|
start_hour, start_hour + max_len
|
|
|
|
|
) # Start at x and extend by max sublist length
|
|
|
|
|
else:
|
|
|
|
|
print(f"Unsupported y_list structure: {type(y_list)}, {y_list}")
|
|
|
|
|
raise TypeError(
|
|
|
|
|
"y_list elements must be np.ndarray, float, or a nested list of floats"
|
|
|
|
|
)
|
|
|
|
|
|
|
|
|
|
for idx, y_data in enumerate(y_list):
|
|
|
|
|
label = labels[idx] if labels else None # Chart label
|
|
|
|
|
marker = markers[idx] if markers and idx < len(markers) else "o" # Marker style
|
|
|
|
|
line_style = (
|
|
|
|
|
line_styles[idx] if line_styles and idx < len(line_styles) else "-"
|
|
|
|
|
) # Line style
|
|
|
|
|
plt.plot(x, y_data, label=label, marker=marker, linestyle=line_style) # Plot line
|
2025-01-13 21:42:52 +01:00
|
|
|
|
2024-12-24 13:10:31 +01:00
|
|
|
plt.title(title) # Set title
|
|
|
|
|
plt.xlabel(xlabel) # Set x-axis label
|
|
|
|
|
plt.ylabel(ylabel) # Set y-axis label
|
|
|
|
|
if labels:
|
|
|
|
|
plt.legend() # Show legend if labels are provided
|
|
|
|
|
plt.grid(True) # Show grid
|
|
|
|
|
plt.xlim(x[0] - 0.5, x[-1] + 0.5) # Adjust x-limits
|
|
|
|
|
|
2025-01-26 18:27:09 +01:00
|
|
|
self.add_chart_to_group(chart, title) # Add chart function to current group
|
2024-12-24 13:10:31 +01:00
|
|
|
|
|
|
|
|
def create_scatter_plot(
|
|
|
|
|
self,
|
|
|
|
|
x: np.ndarray,
|
|
|
|
|
y: np.ndarray,
|
|
|
|
|
title: str,
|
|
|
|
|
xlabel: str,
|
|
|
|
|
ylabel: str,
|
|
|
|
|
c: Optional[np.ndarray] = None,
|
|
|
|
|
) -> None:
|
|
|
|
|
"""Create a scatter plot and add it to the current group."""
|
|
|
|
|
|
|
|
|
|
def chart() -> None:
|
|
|
|
|
scatter = plt.scatter(x, y, c=c, cmap="viridis") # Create scatter plot
|
|
|
|
|
plt.title(title) # Set title
|
|
|
|
|
plt.xlabel(xlabel) # Set x-axis label
|
|
|
|
|
plt.ylabel(ylabel) # Set y-axis label
|
|
|
|
|
if c is not None:
|
|
|
|
|
plt.colorbar(scatter, label="Constraint") # Add colorbar if color data is provided
|
|
|
|
|
plt.grid(True) # Show grid
|
|
|
|
|
|
2025-01-26 18:27:09 +01:00
|
|
|
self.add_chart_to_group(chart, title) # Add chart function to current group
|
2024-12-24 13:10:31 +01:00
|
|
|
|
|
|
|
|
def create_bar_chart(
|
|
|
|
|
self,
|
|
|
|
|
labels: list[str],
|
|
|
|
|
values_list: Sequence[Union[int, float, list[Union[int, float]]]],
|
|
|
|
|
title: str,
|
|
|
|
|
ylabel: str,
|
|
|
|
|
xlabels: Optional[list[str]] = None,
|
|
|
|
|
label_names: Optional[list[str]] = None,
|
|
|
|
|
colors: Optional[list[str]] = None,
|
|
|
|
|
bar_width: float = 0.35,
|
|
|
|
|
bottom: Optional[int] = None,
|
|
|
|
|
) -> None:
|
|
|
|
|
"""Create a bar chart and add it to the current group."""
|
|
|
|
|
|
|
|
|
|
def chart() -> None:
|
|
|
|
|
num_groups = len(values_list) # Number of data groups
|
|
|
|
|
num_bars = len(labels) # Number of bars (categories)
|
|
|
|
|
# Calculate the positions for each bar group on the x-axis
|
|
|
|
|
x = np.arange(num_bars) # x positions for bars
|
|
|
|
|
offset = np.linspace(
|
|
|
|
|
-bar_width * (num_groups - 1) / 2, bar_width * (num_groups - 1) / 2, num_groups
|
|
|
|
|
) # Bar offsets
|
|
|
|
|
for i, values in enumerate(values_list):
|
|
|
|
|
bottom_use = None
|
|
|
|
|
if bottom == i + 1: # Set bottom if specified
|
|
|
|
|
bottom_use = 1
|
|
|
|
|
color = colors[i] if colors and i < len(colors) else None # Bar color
|
|
|
|
|
label_name = label_names[i] if label_names else None # Bar label
|
|
|
|
|
plt.bar(
|
|
|
|
|
x + offset[i],
|
|
|
|
|
values,
|
|
|
|
|
bar_width,
|
|
|
|
|
label=label_name,
|
|
|
|
|
color=color,
|
|
|
|
|
zorder=2,
|
|
|
|
|
alpha=0.6,
|
|
|
|
|
bottom=bottom_use,
|
|
|
|
|
) # Create bar
|
|
|
|
|
if xlabels:
|
|
|
|
|
plt.xticks(x, labels) # Add custom labels to the x-axis
|
|
|
|
|
plt.title(title) # Set title
|
|
|
|
|
plt.ylabel(ylabel) # Set y-axis label
|
|
|
|
|
|
|
|
|
|
if colors and label_names:
|
|
|
|
|
plt.legend() # Show legend if colors are provided
|
|
|
|
|
plt.grid(True, zorder=0) # Show grid in the background
|
|
|
|
|
plt.xlim(-0.5, len(labels) - 0.5) # Set x-axis limits
|
|
|
|
|
|
2025-01-26 18:27:09 +01:00
|
|
|
self.add_chart_to_group(chart, title) # Add chart function to current group
|
2024-12-24 13:10:31 +01:00
|
|
|
|
|
|
|
|
def create_violin_plot(
|
|
|
|
|
self, data_list: list[np.ndarray], labels: list[str], title: str, xlabel: str, ylabel: str
|
|
|
|
|
) -> None:
|
|
|
|
|
"""Create a violin plot and add it to the current group."""
|
|
|
|
|
|
|
|
|
|
def chart() -> None:
|
|
|
|
|
plt.violinplot(data_list, showmeans=True, showmedians=True) # Create violin plot
|
|
|
|
|
plt.xticks(np.arange(1, len(labels) + 1), labels) # Set x-ticks and labels
|
|
|
|
|
plt.title(title) # Set title
|
|
|
|
|
plt.xlabel(xlabel) # Set x-axis label
|
|
|
|
|
plt.ylabel(ylabel) # Set y-axis label
|
|
|
|
|
plt.grid(True) # Show grid
|
|
|
|
|
|
2025-01-26 18:27:09 +01:00
|
|
|
self.add_chart_to_group(chart, title) # Add chart function to current group
|
2024-12-24 13:10:31 +01:00
|
|
|
|
2025-01-12 14:33:02 +01:00
|
|
|
def add_text_page(self, text: str, title: Optional[str] = None, fontsize: int = 12) -> None:
|
|
|
|
|
"""Add a page with text content to the PDF."""
|
|
|
|
|
|
|
|
|
|
def chart() -> None:
|
|
|
|
|
fig = plt.figure(figsize=(8.5, 11)) # Create a standard page size
|
|
|
|
|
plt.axis("off") # Turn off axes for a clean page
|
|
|
|
|
wrapped_text = textwrap.fill(text, width=80) # Wrap text to fit the page width
|
|
|
|
|
y = 0.95 # Start at the top of the page
|
|
|
|
|
|
|
|
|
|
if title:
|
|
|
|
|
plt.text(0.5, y, title, ha="center", va="top", fontsize=fontsize + 4, weight="bold")
|
|
|
|
|
y -= 0.05 # Add space after the title
|
|
|
|
|
|
|
|
|
|
plt.text(0.5, y, wrapped_text, ha="center", va="top", fontsize=fontsize, wrap=True)
|
|
|
|
|
self.pdf_pages.savefig(fig) # Save the figure as a page in the PDF
|
|
|
|
|
plt.close(fig) # Close the figure to free up memory
|
|
|
|
|
|
2025-01-26 18:27:09 +01:00
|
|
|
self.add_chart_to_group(chart, title) # Treat the text page as a "chart" in the group
|
2025-01-12 14:33:02 +01:00
|
|
|
|
|
|
|
|
def add_json_page(
|
|
|
|
|
self, json_obj: dict, title: Optional[str] = None, fontsize: int = 12
|
|
|
|
|
) -> None:
|
|
|
|
|
"""Add a page with a formatted JSON object to the PDF.
|
|
|
|
|
|
|
|
|
|
Args:
|
|
|
|
|
json_obj (dict): The JSON object to display.
|
|
|
|
|
title (Optional[str]): An optional title for the page.
|
|
|
|
|
fontsize (int): The font size for the JSON text.
|
|
|
|
|
"""
|
|
|
|
|
|
|
|
|
|
def chart() -> None:
|
|
|
|
|
# Convert JSON object to a formatted string
|
|
|
|
|
json_str = json.dumps(json_obj, indent=4)
|
|
|
|
|
|
|
|
|
|
fig = plt.figure(figsize=(8.5, 11)) # Standard page size
|
|
|
|
|
plt.axis("off") # Turn off axes for a clean page
|
|
|
|
|
|
|
|
|
|
y = 0.95 # Start at the top of the page
|
|
|
|
|
if title:
|
|
|
|
|
plt.text(0.5, y, title, ha="center", va="top", fontsize=fontsize + 4, weight="bold")
|
|
|
|
|
y -= 0.05 # Add space after the title
|
|
|
|
|
|
|
|
|
|
# Split the JSON string into lines and render them
|
|
|
|
|
lines = json_str.splitlines()
|
|
|
|
|
for line in lines:
|
|
|
|
|
plt.text(0.05, y, line, ha="left", va="top", fontsize=fontsize, family="monospace")
|
|
|
|
|
y -= 0.02 # Move down for the next line
|
|
|
|
|
|
|
|
|
|
# Stop if the text exceeds the page
|
|
|
|
|
if y < 0.05:
|
|
|
|
|
break
|
|
|
|
|
|
|
|
|
|
self.pdf_pages.savefig(fig) # Save the figure as a page in the PDF
|
|
|
|
|
plt.close(fig) # Close the figure to free up memory
|
|
|
|
|
|
2025-01-26 18:27:09 +01:00
|
|
|
self.add_chart_to_group(chart, title) # Treat the JSON page as a "chart" in the group
|
2025-01-12 14:33:02 +01:00
|
|
|
|
2024-12-24 13:10:31 +01:00
|
|
|
def generate_pdf(self) -> None:
|
|
|
|
|
"""Generate the PDF report with all the added chart groups."""
|
|
|
|
|
self._initialize_pdf() # Initialize the PDF
|
|
|
|
|
|
|
|
|
|
for group in self.groups:
|
|
|
|
|
self._save_group_to_pdf(group) # Save each group to the PDF
|
|
|
|
|
|
|
|
|
|
self.pdf_pages.close() # Close the PDF to finalize the report
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
def prepare_visualize(
|
fix: automatic optimization (#596)
This fix implements the long term goal to have the EOS server run optimization (or
energy management) on regular intervals automatically. Thus clients can request
the current energy management plan at any time and it is updated on regular
intervals without interaction by the client.
This fix started out to "only" make automatic optimization (or energy management)
runs working. It turned out there are several endpoints that in some way
update predictions or run the optimization. To lock against such concurrent attempts
the code had to be refactored to allow control of execution. During refactoring it
became clear that some classes and files are named without a proper reference
to their usage. Thus not only refactoring but also renaming became necessary.
The names are still not the best, but I hope they are more intuitive.
The fix includes several bug fixes that are not directly related to the automatic optimization
but are necessary to keep EOS running properly to do the automatic optimization and
to test and document the changes.
This is a breaking change as the configuration structure changed once again and
the server API was also enhanced and streamlined. The server API that is used by
Andreas and Jörg in their videos has not changed.
* fix: automatic optimization
Allow optimization to automatically run on configured intervals gathering all
optimization parameters from configuration and predictions. The automatic run
can be configured to only run prediction updates skipping the optimization.
Extend documentaion to also cover automatic optimization. Lock automatic runs
against runs initiated by the /optimize or other endpoints. Provide new
endpoints to retrieve the energy management plan and the genetic solution
of the latest automatic optimization run. Offload energy management to thread
pool executor to keep the app more responsive during the CPU heavy optimization
run.
* fix: EOS servers recognize environment variables on startup
Force initialisation of EOS configuration on server startup to assure
all sources of EOS configuration are properly set up and read. Adapt
server tests and configuration tests to also test for environment
variable configuration.
* fix: Remove 0.0.0.0 to localhost translation under Windows
EOS imposed a 0.0.0.0 to localhost translation under Windows for
convenience. This caused some trouble in user configurations. Now, as the
default IP address configuration is 127.0.0.1, the user is responsible
for to set up the correct Windows compliant IP address.
* fix: allow names for hosts additional to IP addresses
* fix: access pydantic model fields by class
Access by instance is deprecated.
* fix: down sampling key_to_array
* fix: make cache clear endpoint clear all cache files
Make /v1/admin/cache/clear clear all cache files. Before it only cleared
expired cache files by default. Add new endpoint /v1/admin/clear-expired
to only clear expired cache files.
* fix: timezonefinder returns Europe/Paris instead of Europe/Berlin
timezonefinder 8.10 got more inaccurate for timezones in europe as there is
a common timezone. Use new package tzfpy instead which is still returning
Europe/Berlin if you are in Germany. tzfpy also claims to be faster than
timezonefinder.
* fix: provider settings configuration
Provider configuration used to be a union holding the settings for several
providers. Pydantic union handling does not always find the correct type
for a provider setting. This led to exceptions in specific configurations.
Now provider settings are explicit comfiguration items for each possible
provider. This is a breaking change as the configuration structure was
changed.
* fix: ClearOutside weather prediction irradiance calculation
Pvlib needs a pandas time index. Convert time index.
* fix: test config file priority
Do not use config_eos fixture as this fixture already creates a config file.
* fix: optimization sample request documentation
Provide all data in documentation of optimization sample request.
* fix: gitlint blocking pip dependency resolution
Replace gitlint by commitizen. Gitlint is not actively maintained anymore.
Gitlint dependencies blocked pip from dependency resolution.
* fix: sync pre-commit config to actual dependency requirements
.pre-commit-config.yaml was out of sync, also requirements-dev.txt.
* fix: missing babel in requirements.txt
Add babel to requirements.txt
* feat: setup default device configuration for automatic optimization
In case the parameters for automatic optimization are not fully defined a
default configuration is setup to allow the automatic energy management
run. The default configuration may help the user to correctly define
the device configuration.
* feat: allow configuration of genetic algorithm parameters
The genetic algorithm parameters for number of individuals, number of
generations, the seed and penalty function parameters are now avaliable
as configuration options.
* feat: allow configuration of home appliance time windows
The time windows a home appliance is allowed to run are now configurable
by the configuration (for /v1 API) and also by the home appliance parameters
(for the classic /optimize API). If there is no such configuration the
time window defaults to optimization hours, which was the standard before
the change. Documentation on how to configure time windows is added.
* feat: standardize mesaurement keys for battery/ ev SoC measurements
The standardized measurement keys to report battery SoC to the device
simulations can now be retrieved from the device configuration as a
read-only config option.
* feat: feed in tariff prediction
Add feed in tarif predictions needed for automatic optimization. The feed in
tariff can be retrieved as fixed feed in tarif or can be imported. Also add
tests for the different feed in tariff providers. Extend documentation to
cover the feed in tariff providers.
* feat: add energy management plan based on S2 standard instructions
EOS can generate an energy management plan as a list of simple instructions.
May be retrieved by the /v1/energy-management/plan endpoint. The instructions
loosely follow the S2 energy management standard.
* feat: make measurement keys configurable by EOS configuration.
The fixed measurement keys are replaced by configurable measurement keys.
* feat: make pendulum DateTime, Date, Duration types usable for pydantic models
Use pydantic_extra_types.pendulum_dt to get pydantic pendulum types. Types are
added to the datetimeutil utility. Remove custom made pendulum adaptations
from EOS pydantic module. Make EOS modules use the pydantic pendulum types
managed by the datetimeutil module instead of the core pendulum types.
* feat: Add Time, TimeWindow, TimeWindowSequence and to_time to datetimeutil.
The time windows are are added to support home appliance time window
configuration. All time classes are also pydantic models. Time is the base
class for time definition derived from pendulum.Time.
* feat: Extend DataRecord by configurable field like data.
Configurable field like data was added to support the configuration of
measurement records.
* feat: Add additional information to health information
Version information is added to the health endpoints of eos and eosDash.
The start time of the last optimization and the latest run time of the energy
management is added to the EOS health information.
* feat: add pydantic merge model tests
* feat: add plan tab to EOSdash
The plan tab displays the current energy management instructions.
* feat: add predictions tab to EOSdash
The predictions tab displays the current predictions.
* feat: add cache management to EOSdash admin tab
The admin tab is extended by a section for cache management. It allows to
clear the cache.
* feat: add about tab to EOSdash
The about tab resembles the former hello tab and provides extra information.
* feat: Adapt changelog and prepare for release management
Release management using commitizen is added. The changelog file is adapted and
teh changelog and a description for release management is added in the
documentation.
* feat(doc): Improve install and devlopment documentation
Provide a more concise installation description in Readme.md and add extra
installation page and development page to documentation.
* chore: Use memory cache for interpolation instead of dict in inverter
Decorate calculate_self_consumption() with @cachemethod_until_update to cache
results in memory during an energy management/ optimization run. Replacement
of dict type caching in inverter is now possible because all optimization
runs are properly locked and the memory cache CacheUntilUpdateStore is properly
cleared at the start of any energy management/ optimization operation.
* chore: refactor genetic
Refactor the genetic algorithm modules for enhanced module structure and better
readability. Removed unnecessary and overcomplex devices singleton. Also
split devices configuration from genetic algorithm parameters to allow further
development independently from genetic algorithm parameter format. Move
charge rates configuration for electric vehicles from optimization to devices
configuration to allow to have different charge rates for different cars in
the future.
* chore: Rename memory cache to CacheEnergyManagementStore
The name better resembles the task of the cache to chache function and method
results for an energy management run. Also the decorator functions are renamed
accordingly: cachemethod_energy_management, cache_energy_management
* chore: use class properties for config/ems/prediction mixin classes
* chore: skip debug logs from mathplotlib
Mathplotlib is very noisy in debug mode.
* chore: automatically sync bokeh js to bokeh python package
bokeh was updated to 3.8.0, make JS CDN automatically follow the package version.
* chore: rename hello.py to about.py
Make hello.py the adapted EOSdash about page.
* chore: remove demo page from EOSdash
As no the plan and prediction pages are working without configuration, the demo
page is no longer necessary
* chore: split test_server.py for system test
Split test_server.py to create explicit test_system.py for system tests.
* chore: move doc utils to generate_config_md.py
The doc utils are only used in scripts/generate_config_md.py. Move it there to
attribute for strong cohesion.
* chore: improve pydantic merge model documentation
* chore: remove pendulum warning from readme
* chore: remove GitHub discussions from contributing documentation
Github discussions is to be replaced by Akkudoktor.net.
* chore(release): bump version to 0.1.0+dev for development
* build(deps): bump fastapi[standard] from 0.115.14 to 0.117.1
bump fastapi and make coverage version (for pytest-cov) explicit to avoid pip break.
* build(deps): bump uvicorn from 0.36.0 to 0.37.0
BREAKING CHANGE: EOS configuration changed. V1 API changed.
- The available_charge_rates_percent configuration is removed from optimization.
Use the new charge_rate configuration for the electric vehicle
- Optimization configuration parameter hours renamed to horizon_hours
- Device configuration now has to provide the number of devices and device
properties per device.
- Specific prediction provider configuration to be provided by explicit
configuration item (no union for all providers).
- Measurement keys to be provided as a list.
- New feed in tariff providers have to be configured.
- /v1/measurement/loadxxx endpoints are removed. Use generic mesaurement endpoints.
- /v1/admin/cache/clear now clears all cache files. Use
/v1/admin/cache/clear-expired to only clear all expired cache files.
Signed-off-by: Bobby Noelte <b0661n0e17e@gmail.com>
2025-10-28 02:50:31 +01:00
|
|
|
parameters: GeneticOptimizationParameters,
|
2024-12-24 13:10:31 +01:00
|
|
|
results: dict,
|
2024-12-27 10:46:36 +01:00
|
|
|
filename: str = "visualization_results.pdf",
|
2025-02-08 00:47:21 +01:00
|
|
|
start_hour: int = 0,
|
2024-12-24 13:10:31 +01:00
|
|
|
) -> None:
|
2025-06-10 22:00:28 +02:00
|
|
|
global debug_visualize
|
|
|
|
|
|
2024-12-24 13:10:31 +01:00
|
|
|
report = VisualizationReport(filename)
|
fix: automatic optimization (#596)
This fix implements the long term goal to have the EOS server run optimization (or
energy management) on regular intervals automatically. Thus clients can request
the current energy management plan at any time and it is updated on regular
intervals without interaction by the client.
This fix started out to "only" make automatic optimization (or energy management)
runs working. It turned out there are several endpoints that in some way
update predictions or run the optimization. To lock against such concurrent attempts
the code had to be refactored to allow control of execution. During refactoring it
became clear that some classes and files are named without a proper reference
to their usage. Thus not only refactoring but also renaming became necessary.
The names are still not the best, but I hope they are more intuitive.
The fix includes several bug fixes that are not directly related to the automatic optimization
but are necessary to keep EOS running properly to do the automatic optimization and
to test and document the changes.
This is a breaking change as the configuration structure changed once again and
the server API was also enhanced and streamlined. The server API that is used by
Andreas and Jörg in their videos has not changed.
* fix: automatic optimization
Allow optimization to automatically run on configured intervals gathering all
optimization parameters from configuration and predictions. The automatic run
can be configured to only run prediction updates skipping the optimization.
Extend documentaion to also cover automatic optimization. Lock automatic runs
against runs initiated by the /optimize or other endpoints. Provide new
endpoints to retrieve the energy management plan and the genetic solution
of the latest automatic optimization run. Offload energy management to thread
pool executor to keep the app more responsive during the CPU heavy optimization
run.
* fix: EOS servers recognize environment variables on startup
Force initialisation of EOS configuration on server startup to assure
all sources of EOS configuration are properly set up and read. Adapt
server tests and configuration tests to also test for environment
variable configuration.
* fix: Remove 0.0.0.0 to localhost translation under Windows
EOS imposed a 0.0.0.0 to localhost translation under Windows for
convenience. This caused some trouble in user configurations. Now, as the
default IP address configuration is 127.0.0.1, the user is responsible
for to set up the correct Windows compliant IP address.
* fix: allow names for hosts additional to IP addresses
* fix: access pydantic model fields by class
Access by instance is deprecated.
* fix: down sampling key_to_array
* fix: make cache clear endpoint clear all cache files
Make /v1/admin/cache/clear clear all cache files. Before it only cleared
expired cache files by default. Add new endpoint /v1/admin/clear-expired
to only clear expired cache files.
* fix: timezonefinder returns Europe/Paris instead of Europe/Berlin
timezonefinder 8.10 got more inaccurate for timezones in europe as there is
a common timezone. Use new package tzfpy instead which is still returning
Europe/Berlin if you are in Germany. tzfpy also claims to be faster than
timezonefinder.
* fix: provider settings configuration
Provider configuration used to be a union holding the settings for several
providers. Pydantic union handling does not always find the correct type
for a provider setting. This led to exceptions in specific configurations.
Now provider settings are explicit comfiguration items for each possible
provider. This is a breaking change as the configuration structure was
changed.
* fix: ClearOutside weather prediction irradiance calculation
Pvlib needs a pandas time index. Convert time index.
* fix: test config file priority
Do not use config_eos fixture as this fixture already creates a config file.
* fix: optimization sample request documentation
Provide all data in documentation of optimization sample request.
* fix: gitlint blocking pip dependency resolution
Replace gitlint by commitizen. Gitlint is not actively maintained anymore.
Gitlint dependencies blocked pip from dependency resolution.
* fix: sync pre-commit config to actual dependency requirements
.pre-commit-config.yaml was out of sync, also requirements-dev.txt.
* fix: missing babel in requirements.txt
Add babel to requirements.txt
* feat: setup default device configuration for automatic optimization
In case the parameters for automatic optimization are not fully defined a
default configuration is setup to allow the automatic energy management
run. The default configuration may help the user to correctly define
the device configuration.
* feat: allow configuration of genetic algorithm parameters
The genetic algorithm parameters for number of individuals, number of
generations, the seed and penalty function parameters are now avaliable
as configuration options.
* feat: allow configuration of home appliance time windows
The time windows a home appliance is allowed to run are now configurable
by the configuration (for /v1 API) and also by the home appliance parameters
(for the classic /optimize API). If there is no such configuration the
time window defaults to optimization hours, which was the standard before
the change. Documentation on how to configure time windows is added.
* feat: standardize mesaurement keys for battery/ ev SoC measurements
The standardized measurement keys to report battery SoC to the device
simulations can now be retrieved from the device configuration as a
read-only config option.
* feat: feed in tariff prediction
Add feed in tarif predictions needed for automatic optimization. The feed in
tariff can be retrieved as fixed feed in tarif or can be imported. Also add
tests for the different feed in tariff providers. Extend documentation to
cover the feed in tariff providers.
* feat: add energy management plan based on S2 standard instructions
EOS can generate an energy management plan as a list of simple instructions.
May be retrieved by the /v1/energy-management/plan endpoint. The instructions
loosely follow the S2 energy management standard.
* feat: make measurement keys configurable by EOS configuration.
The fixed measurement keys are replaced by configurable measurement keys.
* feat: make pendulum DateTime, Date, Duration types usable for pydantic models
Use pydantic_extra_types.pendulum_dt to get pydantic pendulum types. Types are
added to the datetimeutil utility. Remove custom made pendulum adaptations
from EOS pydantic module. Make EOS modules use the pydantic pendulum types
managed by the datetimeutil module instead of the core pendulum types.
* feat: Add Time, TimeWindow, TimeWindowSequence and to_time to datetimeutil.
The time windows are are added to support home appliance time window
configuration. All time classes are also pydantic models. Time is the base
class for time definition derived from pendulum.Time.
* feat: Extend DataRecord by configurable field like data.
Configurable field like data was added to support the configuration of
measurement records.
* feat: Add additional information to health information
Version information is added to the health endpoints of eos and eosDash.
The start time of the last optimization and the latest run time of the energy
management is added to the EOS health information.
* feat: add pydantic merge model tests
* feat: add plan tab to EOSdash
The plan tab displays the current energy management instructions.
* feat: add predictions tab to EOSdash
The predictions tab displays the current predictions.
* feat: add cache management to EOSdash admin tab
The admin tab is extended by a section for cache management. It allows to
clear the cache.
* feat: add about tab to EOSdash
The about tab resembles the former hello tab and provides extra information.
* feat: Adapt changelog and prepare for release management
Release management using commitizen is added. The changelog file is adapted and
teh changelog and a description for release management is added in the
documentation.
* feat(doc): Improve install and devlopment documentation
Provide a more concise installation description in Readme.md and add extra
installation page and development page to documentation.
* chore: Use memory cache for interpolation instead of dict in inverter
Decorate calculate_self_consumption() with @cachemethod_until_update to cache
results in memory during an energy management/ optimization run. Replacement
of dict type caching in inverter is now possible because all optimization
runs are properly locked and the memory cache CacheUntilUpdateStore is properly
cleared at the start of any energy management/ optimization operation.
* chore: refactor genetic
Refactor the genetic algorithm modules for enhanced module structure and better
readability. Removed unnecessary and overcomplex devices singleton. Also
split devices configuration from genetic algorithm parameters to allow further
development independently from genetic algorithm parameter format. Move
charge rates configuration for electric vehicles from optimization to devices
configuration to allow to have different charge rates for different cars in
the future.
* chore: Rename memory cache to CacheEnergyManagementStore
The name better resembles the task of the cache to chache function and method
results for an energy management run. Also the decorator functions are renamed
accordingly: cachemethod_energy_management, cache_energy_management
* chore: use class properties for config/ems/prediction mixin classes
* chore: skip debug logs from mathplotlib
Mathplotlib is very noisy in debug mode.
* chore: automatically sync bokeh js to bokeh python package
bokeh was updated to 3.8.0, make JS CDN automatically follow the package version.
* chore: rename hello.py to about.py
Make hello.py the adapted EOSdash about page.
* chore: remove demo page from EOSdash
As no the plan and prediction pages are working without configuration, the demo
page is no longer necessary
* chore: split test_server.py for system test
Split test_server.py to create explicit test_system.py for system tests.
* chore: move doc utils to generate_config_md.py
The doc utils are only used in scripts/generate_config_md.py. Move it there to
attribute for strong cohesion.
* chore: improve pydantic merge model documentation
* chore: remove pendulum warning from readme
* chore: remove GitHub discussions from contributing documentation
Github discussions is to be replaced by Akkudoktor.net.
* chore(release): bump version to 0.1.0+dev for development
* build(deps): bump fastapi[standard] from 0.115.14 to 0.117.1
bump fastapi and make coverage version (for pytest-cov) explicit to avoid pip break.
* build(deps): bump uvicorn from 0.36.0 to 0.37.0
BREAKING CHANGE: EOS configuration changed. V1 API changed.
- The available_charge_rates_percent configuration is removed from optimization.
Use the new charge_rate configuration for the electric vehicle
- Optimization configuration parameter hours renamed to horizon_hours
- Device configuration now has to provide the number of devices and device
properties per device.
- Specific prediction provider configuration to be provided by explicit
configuration item (no union for all providers).
- Measurement keys to be provided as a list.
- New feed in tariff providers have to be configured.
- /v1/measurement/loadxxx endpoints are removed. Use generic mesaurement endpoints.
- /v1/admin/cache/clear now clears all cache files. Use
/v1/admin/cache/clear-expired to only clear all expired cache files.
Signed-off-by: Bobby Noelte <b0661n0e17e@gmail.com>
2025-10-28 02:50:31 +01:00
|
|
|
next_full_hour_date = get_ems().start_datetime
|
2024-12-24 13:10:31 +01:00
|
|
|
# Group 1:
|
2025-01-13 21:42:52 +01:00
|
|
|
report.create_line_chart_date(
|
2025-02-08 00:47:21 +01:00
|
|
|
next_full_hour_date,
|
2025-01-13 21:42:52 +01:00
|
|
|
[
|
2025-02-08 00:47:21 +01:00
|
|
|
parameters.ems.gesamtlast[start_hour:],
|
2025-01-13 21:42:52 +01:00
|
|
|
],
|
2024-12-24 13:10:31 +01:00
|
|
|
title="Load Profile",
|
2025-01-13 21:42:52 +01:00
|
|
|
# xlabel="Hours", # not enough space
|
2024-12-24 13:10:31 +01:00
|
|
|
ylabel="Load (Wh)",
|
|
|
|
|
labels=["Total Load (Wh)"],
|
|
|
|
|
)
|
2025-01-13 21:42:52 +01:00
|
|
|
report.create_line_chart_date(
|
2025-02-08 00:47:21 +01:00
|
|
|
next_full_hour_date,
|
2025-01-13 21:42:52 +01:00
|
|
|
[
|
2025-02-08 00:47:21 +01:00
|
|
|
parameters.ems.pv_prognose_wh[start_hour:],
|
2025-01-13 21:42:52 +01:00
|
|
|
],
|
2024-12-24 13:10:31 +01:00
|
|
|
title="PV Forecast",
|
2025-01-13 21:42:52 +01:00
|
|
|
# xlabel="Hours", # not enough space
|
2024-12-24 13:10:31 +01:00
|
|
|
ylabel="PV Generation (Wh)",
|
|
|
|
|
)
|
|
|
|
|
|
2025-01-13 21:42:52 +01:00
|
|
|
report.create_line_chart_date(
|
2025-02-08 00:47:21 +01:00
|
|
|
next_full_hour_date,
|
|
|
|
|
[
|
|
|
|
|
np.full(
|
|
|
|
|
len(parameters.ems.gesamtlast) - start_hour,
|
2025-02-16 11:06:31 +01:00
|
|
|
parameters.ems.einspeiseverguetung_euro_pro_wh[start_hour:]
|
|
|
|
|
if isinstance(parameters.ems.einspeiseverguetung_euro_pro_wh, list)
|
|
|
|
|
else parameters.ems.einspeiseverguetung_euro_pro_wh,
|
2025-02-08 00:47:21 +01:00
|
|
|
)
|
|
|
|
|
],
|
2024-12-24 13:10:31 +01:00
|
|
|
title="Remuneration",
|
2025-01-13 21:42:52 +01:00
|
|
|
# xlabel="Hours", # not enough space
|
2024-12-24 13:10:31 +01:00
|
|
|
ylabel="€/Wh",
|
2025-01-13 21:42:52 +01:00
|
|
|
x2label=None, # not enough space
|
2024-12-24 13:10:31 +01:00
|
|
|
)
|
|
|
|
|
if parameters.temperature_forecast:
|
2025-01-13 21:42:52 +01:00
|
|
|
report.create_line_chart_date(
|
2025-02-08 00:47:21 +01:00
|
|
|
next_full_hour_date,
|
2025-01-13 21:42:52 +01:00
|
|
|
[
|
2025-02-08 00:47:21 +01:00
|
|
|
parameters.temperature_forecast[start_hour:],
|
2025-01-13 21:42:52 +01:00
|
|
|
],
|
2024-12-24 13:10:31 +01:00
|
|
|
title="Temperature Forecast",
|
2025-01-13 21:42:52 +01:00
|
|
|
# xlabel="Hours", # not enough space
|
2024-12-24 13:10:31 +01:00
|
|
|
ylabel="°C",
|
2025-01-13 21:42:52 +01:00
|
|
|
x2label=None, # not enough space
|
2024-12-24 13:10:31 +01:00
|
|
|
)
|
|
|
|
|
report.finalize_group()
|
|
|
|
|
|
|
|
|
|
# Group 2:
|
2025-01-13 21:42:52 +01:00
|
|
|
report.create_line_chart_date(
|
|
|
|
|
next_full_hour_date, # start_date
|
2024-12-24 13:10:31 +01:00
|
|
|
[
|
|
|
|
|
results["result"]["Last_Wh_pro_Stunde"],
|
|
|
|
|
results["result"]["Home_appliance_wh_per_hour"],
|
|
|
|
|
results["result"]["Netzeinspeisung_Wh_pro_Stunde"],
|
|
|
|
|
results["result"]["Netzbezug_Wh_pro_Stunde"],
|
|
|
|
|
results["result"]["Verluste_Pro_Stunde"],
|
|
|
|
|
],
|
|
|
|
|
title="Energy Flow per Hour",
|
2025-01-13 21:42:52 +01:00
|
|
|
# xlabel="Date", # not enough space
|
2024-12-24 13:10:31 +01:00
|
|
|
ylabel="Energy (Wh)",
|
|
|
|
|
labels=[
|
|
|
|
|
"Load (Wh)",
|
|
|
|
|
"Household Device (Wh)",
|
|
|
|
|
"Grid Feed-in (Wh)",
|
|
|
|
|
"Grid Consumption (Wh)",
|
|
|
|
|
"Losses (Wh)",
|
|
|
|
|
],
|
|
|
|
|
markers=["o", "o", "x", "^", "^"],
|
|
|
|
|
line_styles=["-", "--", ":", "-.", "-"],
|
|
|
|
|
)
|
|
|
|
|
report.finalize_group()
|
|
|
|
|
|
|
|
|
|
# Group 3:
|
2025-01-13 21:42:52 +01:00
|
|
|
report.create_line_chart_date(
|
|
|
|
|
next_full_hour_date, # start_date
|
2024-12-24 13:10:31 +01:00
|
|
|
[results["result"]["akku_soc_pro_stunde"], results["result"]["EAuto_SoC_pro_Stunde"]],
|
|
|
|
|
title="Battery SOC",
|
2025-01-13 21:42:52 +01:00
|
|
|
# xlabel="Date", # not enough space
|
2024-12-24 13:10:31 +01:00
|
|
|
ylabel="%",
|
|
|
|
|
labels=[
|
|
|
|
|
"Battery SOC (%)",
|
|
|
|
|
"Electric Vehicle SOC (%)",
|
|
|
|
|
],
|
|
|
|
|
markers=["o", "x"],
|
|
|
|
|
)
|
2025-01-13 21:42:52 +01:00
|
|
|
report.create_line_chart_date(
|
|
|
|
|
next_full_hour_date, # start_date
|
2025-02-08 00:47:21 +01:00
|
|
|
[parameters.ems.strompreis_euro_pro_wh[start_hour:]],
|
|
|
|
|
# title="Electricity Price", # not enough space
|
2025-01-13 21:42:52 +01:00
|
|
|
# xlabel="Date", # not enough space
|
|
|
|
|
ylabel="Electricity Price (€/Wh)",
|
|
|
|
|
x2label=None, # not enough space
|
2024-12-24 13:10:31 +01:00
|
|
|
)
|
|
|
|
|
|
2025-02-08 00:47:21 +01:00
|
|
|
labels = list(
|
|
|
|
|
item
|
|
|
|
|
for sublist in zip(
|
|
|
|
|
list(str(i) for i in range(0, 23, 2)), list(str(" ") for i in range(0, 23, 2))
|
|
|
|
|
)
|
|
|
|
|
for item in sublist
|
|
|
|
|
)
|
|
|
|
|
labels = labels[start_hour:] + labels
|
|
|
|
|
|
2024-12-24 13:10:31 +01:00
|
|
|
report.create_bar_chart(
|
2025-02-08 00:47:21 +01:00
|
|
|
labels,
|
|
|
|
|
[
|
|
|
|
|
results["ac_charge"][start_hour:],
|
|
|
|
|
results["dc_charge"][start_hour:],
|
|
|
|
|
results["discharge_allowed"][start_hour:],
|
|
|
|
|
],
|
2024-12-24 13:10:31 +01:00
|
|
|
title="AC/DC Charging and Discharge Overview",
|
|
|
|
|
ylabel="Relative Power (0-1) / Discharge (0 or 1)",
|
|
|
|
|
label_names=["AC Charging (relative)", "DC Charging (relative)", "Discharge Allowed"],
|
|
|
|
|
colors=["blue", "green", "red"],
|
|
|
|
|
bottom=3,
|
2025-02-08 00:47:21 +01:00
|
|
|
xlabels=labels,
|
2024-12-24 13:10:31 +01:00
|
|
|
)
|
|
|
|
|
report.finalize_group()
|
|
|
|
|
|
|
|
|
|
# Group 4:
|
|
|
|
|
|
2025-01-13 21:42:52 +01:00
|
|
|
report.create_line_chart_date(
|
|
|
|
|
next_full_hour_date, # start_date
|
2024-12-24 13:10:31 +01:00
|
|
|
[
|
|
|
|
|
results["result"]["Kosten_Euro_pro_Stunde"],
|
|
|
|
|
results["result"]["Einnahmen_Euro_pro_Stunde"],
|
|
|
|
|
],
|
|
|
|
|
title="Financial Balance per Hour",
|
2025-01-13 21:42:52 +01:00
|
|
|
# xlabel="Date", # not enough space
|
2024-12-24 13:10:31 +01:00
|
|
|
ylabel="Euro",
|
|
|
|
|
labels=["Costs", "Revenue"],
|
|
|
|
|
)
|
|
|
|
|
|
|
|
|
|
extra_data = results["extra_data"]
|
|
|
|
|
report.create_scatter_plot(
|
|
|
|
|
extra_data["verluste"],
|
|
|
|
|
extra_data["bilanz"],
|
2025-01-26 18:27:09 +01:00
|
|
|
title="Scatter Plot",
|
2024-12-24 13:10:31 +01:00
|
|
|
xlabel="losses",
|
|
|
|
|
ylabel="balance",
|
|
|
|
|
c=extra_data["nebenbedingung"],
|
|
|
|
|
)
|
|
|
|
|
|
|
|
|
|
values_list = [
|
|
|
|
|
[
|
|
|
|
|
results["result"]["Gesamtkosten_Euro"],
|
|
|
|
|
results["result"]["Gesamteinnahmen_Euro"],
|
|
|
|
|
results["result"]["Gesamtbilanz_Euro"],
|
|
|
|
|
]
|
|
|
|
|
]
|
|
|
|
|
labels = ["Total Costs [€]", "Total Revenue [€]", "Total Balance [€]"]
|
|
|
|
|
|
|
|
|
|
report.create_bar_chart(
|
|
|
|
|
labels=labels,
|
|
|
|
|
values_list=values_list,
|
|
|
|
|
title="Financial Overview",
|
|
|
|
|
ylabel="Euro",
|
|
|
|
|
xlabels=["Total Costs [€]", "Total Revenue [€]", "Total Balance [€]"],
|
|
|
|
|
)
|
|
|
|
|
|
|
|
|
|
report.finalize_group()
|
|
|
|
|
|
|
|
|
|
# Group 1: Scatter plot of losses vs balance with color-coded constraints
|
|
|
|
|
f1 = np.array(extra_data["verluste"]) # Losses
|
|
|
|
|
f2 = np.array(extra_data["bilanz"]) # Balance
|
|
|
|
|
n1 = np.array(extra_data["nebenbedingung"]) # Constraints
|
|
|
|
|
|
|
|
|
|
# Filter data where 'nebenbedingung' < 0.01
|
|
|
|
|
filtered_indices = n1 < 0.01
|
|
|
|
|
filtered_losses = f1[filtered_indices]
|
|
|
|
|
filtered_balance = f2[filtered_indices]
|
|
|
|
|
|
|
|
|
|
# Group 2: Violin plot for filtered losses
|
|
|
|
|
if filtered_losses.size > 0:
|
|
|
|
|
report.create_violin_plot(
|
|
|
|
|
data_list=[filtered_losses], # Data for filtered losses
|
|
|
|
|
labels=["Filtered Losses"], # Label for the violin plot
|
|
|
|
|
title="Violin Plot for Filtered Losses (Constraint < 0.01)",
|
|
|
|
|
xlabel="Losses",
|
|
|
|
|
ylabel="Values",
|
|
|
|
|
)
|
|
|
|
|
else:
|
|
|
|
|
print("No data available for filtered losses violin plot (Constraint < 0.01)")
|
|
|
|
|
|
|
|
|
|
# Group 3: Violin plot for filtered balance
|
|
|
|
|
if filtered_balance.size > 0:
|
|
|
|
|
report.create_violin_plot(
|
|
|
|
|
data_list=[filtered_balance], # Data for filtered balance
|
|
|
|
|
labels=["Filtered Balance"], # Label for the violin plot
|
|
|
|
|
title="Violin Plot for Filtered Balance (Constraint < 0.01)",
|
|
|
|
|
xlabel="Balance",
|
|
|
|
|
ylabel="Values",
|
|
|
|
|
)
|
|
|
|
|
else:
|
|
|
|
|
print("No data available for filtered balance violin plot (Constraint < 0.01)")
|
|
|
|
|
|
|
|
|
|
if filtered_balance.size > 0 or filtered_losses.size > 0:
|
|
|
|
|
report.finalize_group()
|
2025-06-10 22:00:28 +02:00
|
|
|
if debug_visualize or results["fixed_seed"]:
|
2025-01-12 14:33:02 +01:00
|
|
|
report.create_line_chart(
|
|
|
|
|
0,
|
|
|
|
|
[
|
|
|
|
|
results["fitness_history"]["avg"],
|
|
|
|
|
results["fitness_history"]["max"],
|
|
|
|
|
results["fitness_history"]["min"],
|
|
|
|
|
],
|
|
|
|
|
title=f"DEBUG: Generation Fitness for seed {results['fixed_seed']}",
|
|
|
|
|
xlabel="Generation",
|
|
|
|
|
ylabel="Fitness",
|
|
|
|
|
labels=[
|
|
|
|
|
"avg",
|
|
|
|
|
"max",
|
|
|
|
|
"min",
|
|
|
|
|
],
|
|
|
|
|
markers=[".", ".", "."],
|
|
|
|
|
)
|
|
|
|
|
report.finalize_group()
|
2024-12-24 13:10:31 +01:00
|
|
|
# Generate the PDF report
|
|
|
|
|
report.generate_pdf()
|
|
|
|
|
|
|
|
|
|
|
2024-12-29 18:42:49 +01:00
|
|
|
def generate_example_report(filename: str = "example_report.pdf") -> None:
|
|
|
|
|
"""Generate example visualization report."""
|
2025-06-10 22:00:28 +02:00
|
|
|
global debug_visualize
|
|
|
|
|
|
2025-01-13 21:42:52 +01:00
|
|
|
report = VisualizationReport(filename, "test")
|
2024-12-24 13:10:31 +01:00
|
|
|
x_hours = 0 # Define x-axis start values (e.g., hours)
|
|
|
|
|
|
|
|
|
|
# Group 1: Adding charts to be displayed on the same page
|
|
|
|
|
report.create_line_chart(
|
|
|
|
|
x_hours,
|
|
|
|
|
[np.array([10, 20, 30, 40])],
|
|
|
|
|
title="Load Profile",
|
|
|
|
|
xlabel="Hours",
|
|
|
|
|
ylabel="Load (Wh)",
|
|
|
|
|
)
|
|
|
|
|
report.create_line_chart(
|
|
|
|
|
x_hours,
|
|
|
|
|
[np.array([5, 15, 25, 35])],
|
|
|
|
|
title="PV Forecast",
|
|
|
|
|
xlabel="Hours",
|
|
|
|
|
ylabel="PV Generation (Wh)",
|
|
|
|
|
)
|
|
|
|
|
report.create_line_chart(
|
|
|
|
|
x_hours,
|
|
|
|
|
[np.array([5, 15, 25, 35])],
|
|
|
|
|
title="PV Forecast",
|
|
|
|
|
xlabel="Hours",
|
|
|
|
|
ylabel="PV Generation (Wh)",
|
|
|
|
|
)
|
|
|
|
|
# Note: If there are only 3 charts per page, the first is as wide as the page
|
|
|
|
|
|
|
|
|
|
report.finalize_group() # Finalize the first group of charts
|
|
|
|
|
|
|
|
|
|
# Group 2: Adding more charts to be displayed on another page
|
|
|
|
|
report.create_line_chart(
|
|
|
|
|
x_hours,
|
|
|
|
|
[np.array([0.2, 0.25, 0.3, 0.35])],
|
|
|
|
|
title="Electricity Price",
|
|
|
|
|
xlabel="Hours",
|
|
|
|
|
ylabel="Price (€/Wh)",
|
|
|
|
|
)
|
|
|
|
|
report.create_bar_chart(
|
|
|
|
|
["Costs", "Revenue", "Balance"],
|
|
|
|
|
[[500.0], [600.0], [100.0]],
|
|
|
|
|
title="Financial Overview",
|
|
|
|
|
ylabel="Euro",
|
|
|
|
|
label_names=["AC Charging (relative)", "DC Charging (relative)", "Discharge Allowed"],
|
|
|
|
|
colors=["red", "green", "blue"],
|
|
|
|
|
)
|
|
|
|
|
report.create_scatter_plot(
|
|
|
|
|
np.array([5, 6, 7, 8]),
|
|
|
|
|
np.array([100, 200, 150, 250]),
|
|
|
|
|
title="Scatter Plot",
|
|
|
|
|
xlabel="Losses",
|
|
|
|
|
ylabel="Balance",
|
|
|
|
|
c=np.array([0.1, 0.2, 0.3, 0.4]),
|
|
|
|
|
)
|
|
|
|
|
report.finalize_group() # Finalize the second group of charts
|
|
|
|
|
|
|
|
|
|
# Group 3: Adding a violin plot
|
|
|
|
|
data = [np.random.normal(0, std, 100) for std in range(1, 5)] # Example data for violin plot
|
|
|
|
|
report.create_violin_plot(
|
|
|
|
|
data,
|
|
|
|
|
labels=["Group 1", "Group 2", "Group 3", "Group 4"],
|
|
|
|
|
title="Violin Plot",
|
|
|
|
|
xlabel="Groups",
|
|
|
|
|
ylabel="Values",
|
|
|
|
|
)
|
|
|
|
|
data = [np.random.normal(0, 1, 100)] # Example data for violin plot
|
|
|
|
|
report.create_violin_plot(
|
|
|
|
|
data, labels=["Group 1"], title="Violin Plot", xlabel="Group", ylabel="Values"
|
|
|
|
|
)
|
|
|
|
|
|
|
|
|
|
report.finalize_group() # Finalize the third group of charts
|
|
|
|
|
|
2025-06-10 22:00:28 +02:00
|
|
|
debug_visualize = True # set level for example report
|
2025-01-12 14:33:02 +01:00
|
|
|
|
2025-06-10 22:00:28 +02:00
|
|
|
if debug_visualize:
|
2025-01-12 14:33:02 +01:00
|
|
|
report.create_line_chart(
|
|
|
|
|
x_hours,
|
|
|
|
|
[np.array([0.2, 0.25, 0.3, 0.35])],
|
|
|
|
|
title="DEBUG",
|
|
|
|
|
xlabel="DEBUG",
|
|
|
|
|
ylabel="DEBUG",
|
|
|
|
|
)
|
|
|
|
|
report.finalize_group() # Finalize the third group of charts
|
|
|
|
|
|
|
|
|
|
report.add_text_page(
|
|
|
|
|
text=" Bisher passierte folgendes:"
|
|
|
|
|
"Am Anfang wurde das Universum erschaffen."
|
|
|
|
|
"Das machte viele Leute sehr wütend und wurde allent-"
|
|
|
|
|
"halben als Schritt in die falsche Richtung angesehen...",
|
|
|
|
|
title="Don't Panic!",
|
|
|
|
|
fontsize=14,
|
|
|
|
|
)
|
|
|
|
|
report.finalize_group()
|
|
|
|
|
|
|
|
|
|
sample_json = {
|
|
|
|
|
"name": "Visualization Report",
|
|
|
|
|
"version": 1.0,
|
|
|
|
|
"charts": [
|
|
|
|
|
{"type": "line", "data_points": 50},
|
|
|
|
|
{"type": "bar", "categories": 10},
|
|
|
|
|
],
|
|
|
|
|
"metadata": {"author": "AI Assistant", "date": "2025-01-11"},
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
report.add_json_page(json_obj=sample_json, title="Formatted JSON Data", fontsize=10)
|
|
|
|
|
report.finalize_group()
|
|
|
|
|
|
2025-01-13 21:42:52 +01:00
|
|
|
report.create_line_chart_date(
|
|
|
|
|
pendulum.now().subtract(hours=0),
|
|
|
|
|
[list(np.random.random(840))],
|
|
|
|
|
title="test",
|
|
|
|
|
xlabel="test",
|
|
|
|
|
ylabel="test",
|
|
|
|
|
)
|
|
|
|
|
report.finalize_group()
|
2024-12-24 13:10:31 +01:00
|
|
|
# Generate the PDF report
|
|
|
|
|
report.generate_pdf()
|
2024-12-29 18:42:49 +01:00
|
|
|
|
|
|
|
|
|
|
|
|
|
if __name__ == "__main__":
|
|
|
|
|
generate_example_report()
|