Mirrors/EOS
1
0
Fork 0
mirror of https://github.com/Akkudoktor-EOS/EOS.git synced 2025-11-02 07:46:20 +00:00
Code Issues Actions 1 Packages Projects Releases Wiki Activity

fix: automatic optimization (#596)

Browse Source 
This fix implements the long term goal to have the EOS server run optimization (or
energy management) on regular intervals automatically. Thus clients can request
the current energy management plan at any time and it is updated on regular
intervals without interaction by the client.

This fix started out to "only" make automatic optimization (or energy management)
runs working. It turned out there are several endpoints that in some way
update predictions or run the optimization. To lock against such concurrent attempts
the code had to be refactored to allow control of execution. During refactoring it
became clear that some classes and files are named without a proper reference
to their usage. Thus not only refactoring but also renaming became necessary.
The names are still not the best, but I hope they are more intuitive.

The fix includes several bug fixes that are not directly related to the automatic optimization
but are necessary to keep EOS running properly to do the automatic optimization and
to test and document the changes.

This is a breaking change as the configuration structure changed once again and
the server API was also enhanced and streamlined. The server API that is used by
Andreas and Jörg in their videos has not changed.

* fix: automatic optimization

  Allow optimization to automatically run on configured intervals gathering all
  optimization parameters from configuration and predictions. The automatic run
  can be configured to only run prediction updates skipping the optimization.
  Extend documentaion to also cover automatic optimization. Lock automatic runs
  against runs initiated by the /optimize or other endpoints. Provide new
  endpoints to retrieve the energy management plan and the genetic solution
  of the latest automatic optimization run. Offload energy management to thread
  pool executor to keep the app more responsive during the CPU heavy optimization
  run.

* fix: EOS servers recognize environment variables on startup

  Force initialisation of EOS configuration on server startup to assure
  all sources of EOS configuration are properly set up and read. Adapt
  server tests and configuration tests to also test for environment
  variable configuration.

* fix: Remove 0.0.0.0 to localhost translation under Windows

  EOS imposed a 0.0.0.0 to localhost translation under Windows for
  convenience. This caused some trouble in user configurations. Now, as the
  default IP address configuration is 127.0.0.1, the user is responsible
  for to set up the correct Windows compliant IP address.

* fix: allow names for hosts additional to IP addresses

* fix: access pydantic model fields by class

  Access by instance is deprecated.

* fix: down sampling key_to_array

* fix: make cache clear endpoint clear all cache files

  Make /v1/admin/cache/clear clear all cache files. Before it only cleared
  expired cache files by default. Add new endpoint /v1/admin/clear-expired
  to only clear expired cache files.

* fix: timezonefinder returns Europe/Paris instead of Europe/Berlin

  timezonefinder 8.10 got more inaccurate for timezones in europe as there is
  a common timezone. Use new package tzfpy instead which is still returning
  Europe/Berlin if you are in Germany. tzfpy also claims to be faster than
  timezonefinder.

* fix: provider settings configuration

  Provider configuration used to be a union holding the settings for several
  providers. Pydantic union handling does not always find the correct type
  for a provider setting. This led to exceptions in specific configurations.
  Now provider settings are explicit comfiguration items for each possible
  provider. This is a breaking change as the configuration structure was
  changed.

* fix: ClearOutside weather prediction irradiance calculation

  Pvlib needs a pandas time index. Convert time index.

* fix: test config file priority

  Do not use config_eos fixture as this fixture already creates a config file.

* fix: optimization sample request documentation

  Provide all data in documentation of optimization sample request.

* fix: gitlint blocking pip dependency resolution

  Replace gitlint by commitizen. Gitlint is not actively maintained anymore.
  Gitlint dependencies blocked pip from dependency resolution.

* fix: sync pre-commit config to actual dependency requirements

  .pre-commit-config.yaml was out of sync, also requirements-dev.txt.

* fix: missing babel in requirements.txt

  Add babel to requirements.txt

* feat: setup default device configuration for automatic optimization

  In case the parameters for automatic optimization are not fully defined a
  default configuration is setup to allow the automatic energy management
  run. The default configuration may help the user to correctly define
  the device configuration.

* feat: allow configuration of genetic algorithm parameters

  The genetic algorithm parameters for number of individuals, number of
  generations, the seed and penalty function parameters are now avaliable
  as configuration options.

* feat: allow configuration of home appliance time windows

  The time windows a home appliance is allowed to run are now configurable
  by the configuration (for /v1 API) and also by the home appliance parameters
  (for the classic /optimize API). If there is no such configuration the
  time window defaults to optimization hours, which was the standard before
  the change. Documentation on how to configure time windows is added.

* feat: standardize mesaurement keys for battery/ ev SoC measurements

  The standardized measurement keys to report battery SoC to the device
  simulations can now be retrieved from the device configuration as a
  read-only config option.

* feat: feed in tariff prediction

  Add feed in tarif predictions needed for automatic optimization. The feed in
  tariff can be retrieved as fixed feed in tarif or can be imported. Also add
  tests for the different feed in tariff providers. Extend documentation to
  cover the feed in tariff providers.

* feat: add energy management plan based on S2 standard instructions

  EOS can generate an energy management plan as a list of simple instructions.
  May be retrieved by the /v1/energy-management/plan endpoint. The instructions
  loosely follow the S2 energy management standard.

* feat: make measurement keys configurable by EOS configuration.

  The fixed measurement keys are replaced by configurable measurement keys.

* feat: make pendulum DateTime, Date, Duration types usable for pydantic models

  Use pydantic_extra_types.pendulum_dt to get pydantic pendulum types. Types are
  added to the datetimeutil utility. Remove custom made pendulum adaptations
  from EOS pydantic module. Make EOS modules use the pydantic pendulum types
  managed by the datetimeutil module instead of the core pendulum types.

* feat: Add Time, TimeWindow, TimeWindowSequence and to_time to datetimeutil.

  The time windows are are added to support home appliance time window
  configuration. All time classes are also pydantic models. Time is the base
  class for time definition derived from pendulum.Time.

* feat: Extend DataRecord by configurable field like data.

  Configurable field like data was added to support the configuration of
  measurement records.

* feat: Add additional information to health information

  Version information is added to the health endpoints of eos and eosDash.
  The start time of the last optimization and the latest run time of the energy
  management is added to the EOS health information.

* feat: add pydantic merge model tests

* feat: add plan tab to EOSdash

  The plan tab displays the current energy management instructions.

* feat: add predictions tab to EOSdash

  The predictions tab displays the current predictions.

* feat: add cache management to EOSdash admin tab

  The admin tab is extended by a section for cache management. It allows to
  clear the cache.

* feat: add about tab to EOSdash

  The about tab resembles the former hello tab and provides extra information.

* feat: Adapt changelog and prepare for release management

  Release management using commitizen is added. The changelog file is adapted and
  teh changelog and a description for release management is added in the
  documentation.

* feat(doc): Improve install and devlopment documentation

  Provide a more concise installation description in Readme.md and add extra
  installation page and development page to documentation.

* chore: Use memory cache for interpolation instead of dict in inverter

  Decorate calculate_self_consumption() with @cachemethod_until_update to cache
  results in memory during an energy management/ optimization run. Replacement
  of dict type caching in inverter is now possible because all optimization
  runs are properly locked and the memory cache CacheUntilUpdateStore is properly
  cleared at the start of any energy management/ optimization operation.

* chore: refactor genetic

  Refactor the genetic algorithm modules for enhanced module structure and better
  readability. Removed unnecessary and overcomplex devices singleton. Also
  split devices configuration from genetic algorithm parameters to allow further
  development independently from genetic algorithm parameter format. Move
  charge rates configuration for electric vehicles from optimization to devices
  configuration to allow to have different charge rates for different cars in
  the future.

* chore: Rename memory cache to CacheEnergyManagementStore

  The name better resembles the task of the cache to chache function and method
  results for an energy management run. Also the decorator functions are renamed
  accordingly: cachemethod_energy_management, cache_energy_management

* chore: use class properties for config/ems/prediction mixin classes

* chore: skip debug logs from mathplotlib

  Mathplotlib is very noisy in debug mode.

* chore: automatically sync bokeh js to bokeh python package

  bokeh was updated to 3.8.0, make JS CDN automatically follow the package version.

* chore: rename hello.py to about.py

  Make hello.py the adapted EOSdash about page.

* chore: remove demo page from EOSdash

  As no the plan and prediction pages are working without configuration, the demo
  page is no longer necessary

* chore: split test_server.py for system test

  Split test_server.py to create explicit test_system.py for system tests.

* chore: move doc utils to generate_config_md.py

  The doc utils are only used in scripts/generate_config_md.py. Move it there to
  attribute for strong cohesion.

* chore: improve pydantic merge model documentation

* chore: remove pendulum warning from readme

* chore: remove GitHub discussions from contributing documentation

  Github discussions is to be replaced by Akkudoktor.net.

* chore(release): bump version to 0.1.0+dev for development

* build(deps): bump fastapi[standard] from 0.115.14 to 0.117.1

  bump fastapi and make coverage version (for pytest-cov) explicit to avoid pip break.

* build(deps): bump uvicorn from 0.36.0 to 0.37.0

BREAKING CHANGE: EOS configuration changed. V1 API changed.

  - The available_charge_rates_percent configuration is removed from optimization.
    Use the new charge_rate configuration for the electric vehicle
  - Optimization configuration parameter hours renamed to horizon_hours
  - Device configuration now has to provide the number of devices and device
    properties per device.
  - Specific prediction provider configuration to be provided by explicit
    configuration item (no union for all providers).
  - Measurement keys to be provided as a list.
  - New feed in tariff providers have to be configured.
  - /v1/measurement/loadxxx endpoints are removed. Use generic mesaurement endpoints.
  - /v1/admin/cache/clear now clears all cache files. Use
    /v1/admin/cache/clear-expired to only clear all expired cache files.

Signed-off-by: Bobby Noelte <b0661n0e17e@gmail.com>
This commit is contained in:
Bobby Noelte
2025-10-28 02:50:31 +01:00
committed by GitHub
parent 20a9eb78d8
commit b397b5d43e
146 changed files with 22024 additions and 5339 deletions
Download Patch File Download Diff File Expand all files Collapse all files

14
src/akkudoktoreos/server/dash/hello.py → src/akkudoktoreos/server/dash/about.py
View File

@@ -2,9 +2,10 @@ from typing import Any
from fasthtml.common import Div
from akkudoktoreos.core.version import __version__
from akkudoktoreos.server.dash.markdown import Markdown
hello_md = """![Logo](/eosdash/assets/logo.png)
about_md = f"""![Logo](/eosdash/assets/logo.png)
# Akkudoktor EOSdash
@@ -17,8 +18,15 @@ electricity price data, this system enables forecasting and optimization of ener
over a specified period.
Documentation can be found at [Akkudoktor-EOS](https://akkudoktor-eos.readthedocs.io/en/latest/).
## Version Information
**Current Version:** {__version__}
**License:** Apache License
"""
def Hello(**kwargs: Any) -> Div:
return Markdown(hello_md, **kwargs)
def About(**kwargs: Any) -> Div:
return Markdown(about_md, **kwargs)

96
src/akkudoktoreos/server/dash/admin.py
View File

@@ -56,6 +56,101 @@ def AdminButton(*c: Any, cls: Optional[Union[str, tuple]] = None, **kwargs: Any)
return Button(*c, submit=False, **kwargs)
def AdminCache(
eos_host: str, eos_port: Union[str, int], data: Optional[dict], config: Optional[dict[str, Any]]
) -> tuple[str, Union[Card, list[Card]]]:
"""Creates a cache management card.
Args:
eos_host (str): The hostname of the EOS server.
eos_port (Union[str, int]): The port of the EOS server.
data (Optional[dict]): Incoming data containing action and category for processing.
Returns:
tuple[str, Union[Card, list[Card]]]: A tuple containing the cache category label and the `Card` UI component.
"""
server = f"http://{eos_host}:{eos_port}"
eos_hostname = "EOS server"
eosdash_hostname = "EOSdash server"
category = "cache"
if data and data.get("category", None) == category:
# This data is for us
if data["action"] == "clear":
# Clear all cache files
try:
result = requests.post(f"{server}/v1/admin/cache/clear", timeout=10)
result.raise_for_status()
status = Success(f"Cleared all cache files on '{eos_hostname}'")
except requests.exceptions.HTTPError as e:
detail = result.json()["detail"]
status = Error(f"Can not clear all cache files on '{eos_hostname}': {e}, {detail}")
except Exception as e:
status = Error(f"Can not clear all cache files on '{eos_hostname}': {e}")
elif data["action"] == "clear-expired":
# Clear expired cache files
try:
result = requests.post(f"{server}/v1/admin/cache/clear-expired", timeout=10)
result.raise_for_status()
status = Success(f"Cleared expired cache files on '{eos_hostname}'")
except requests.exceptions.HTTPError as e:
detail = result.json()["detail"]
status = Error(
f"Can not clear expired cache files on '{eos_hostname}': {e}, {detail}"
)
except Exception as e:
status = Error(f"Can not clear expired cache files on '{eos_hostname}': {e}")
return (
category,
[
Card(
Details(
Summary(
Grid(
DivHStacked(
UkIcon(icon="play"),
AdminButton(
"Clear all",
hx_post="/eosdash/admin",
hx_target="#page-content",
hx_swap="innerHTML",
hx_vals='{"category": "cache", "action": "clear"}',
),
P(f"cache files on '{eos_hostname}'"),
),
),
cls="list-none",
),
P(f"Clear all cache files on '{eos_hostname}'."),
),
),
Card(
Details(
Summary(
Grid(
DivHStacked(
UkIcon(icon="play"),
AdminButton(
"Clear expired",
hx_post="/eosdash/admin",
hx_target="#page-content",
hx_swap="innerHTML",
hx_vals='{"category": "cache", "action": "clear-expired"}',
),
P(f"cache files on '{eos_hostname}'"),
),
),
cls="list-none",
),
P(f"Clear expired cache files on '{eos_hostname}'."),
),
),
],
)
def AdminConfig(
eos_host: str, eos_port: Union[str, int], data: Optional[dict], config: Optional[dict[str, Any]]
) -> tuple[str, Union[Card, list[Card]]]:
@@ -282,6 +377,7 @@ def Admin(eos_host: str, eos_port: Union[str, int], data: Optional[dict] = None)
rows = []
last_category = ""
for category, admin in [
AdminCache(eos_host, eos_port, data, config),
AdminConfig(eos_host, eos_port, data, config),
]:
if category != last_category:

68
src/akkudoktoreos/server/dash/bokeh.py
View File

@@ -1,33 +1,87 @@
# Module taken from https://github.com/koaning/fh-altair
# MIT license
from typing import Optional
import bokeh
from bokeh.embed import components
from bokeh.models import Plot
from monsterui.franken import H4, Card, NotStr, Script
bokeh_version = bokeh.__version__
BokehJS = [
Script(src="https://cdn.bokeh.org/bokeh/release/bokeh-3.7.0.min.js", crossorigin="anonymous"),
Script(
src="https://cdn.bokeh.org/bokeh/release/bokeh-widgets-3.7.0.min.js",
src=f"https://cdn.bokeh.org/bokeh/release/bokeh-{bokeh_version}.min.js",
crossorigin="anonymous",
),
Script(
src="https://cdn.bokeh.org/bokeh/release/bokeh-tables-3.7.0.min.js", crossorigin="anonymous"
src=f"https://cdn.bokeh.org/bokeh/release/bokeh-widgets-{bokeh_version}.min.js",
crossorigin="anonymous",
),
Script(
src="https://cdn.bokeh.org/bokeh/release/bokeh-gl-3.7.0.min.js", crossorigin="anonymous"
src=f"https://cdn.bokeh.org/bokeh/release/bokeh-tables-{bokeh_version}.min.js",
crossorigin="anonymous",
),
Script(
src="https://cdn.bokeh.org/bokeh/release/bokeh-mathjax-3.7.0.min.js",
src=f"https://cdn.bokeh.org/bokeh/release/bokeh-gl-{bokeh_version}.min.js",
crossorigin="anonymous",
),
Script(
src=f"https://cdn.bokeh.org/bokeh/release/bokeh-mathjax-{bokeh_version}.min.js",
crossorigin="anonymous",
),
]
def bokey_apply_theme_to_plot(plot: Plot, dark: bool) -> None:
"""Apply a dark or light theme to a Bokeh plot.
This function modifies the appearance of a Bokeh `Plot` object in-place,
adjusting background, border, title, axis, and grid colors based on the
`dark` parameter.
Args:
plot (Plot): The Bokeh plot to style.
dark (bool): Whether to apply the dark theme (`True`) or light theme (`False`).
Notes:
- This only affects the plot passed in; it does not change other plots
in the same document.
"""
if dark:
plot.background_fill_color = "#1e1e1e"
plot.border_fill_color = "#1e1e1e"
plot.title.text_color = "white"
for ax in plot.xaxis + plot.yaxis:
ax.axis_line_color = "white"
ax.major_tick_line_color = "white"
ax.major_label_text_color = "white"
ax.axis_label_text_color = "white"
# Grid lines
for grid in plot.renderers:
if hasattr(grid, "grid_line_color"):
grid.grid_line_color = "#333"
for grid in plot.xgrid + plot.ygrid:
grid.grid_line_color = "#333"
else:
plot.background_fill_color = "white"
plot.border_fill_color = "white"
plot.title.text_color = "black"
for ax in plot.xaxis + plot.yaxis:
ax.axis_line_color = "black"
ax.major_tick_line_color = "black"
ax.major_label_text_color = "black"
ax.axis_label_text_color = "black"
# Grid lines
for grid in plot.renderers:
if hasattr(grid, "grid_line_color"):
grid.grid_line_color = "#ddd"
for grid in plot.xgrid + plot.ygrid:
grid.grid_line_color = "#ddd"
def Bokeh(plot: Plot, header: Optional[str] = None) -> Card:
"""Converts an Bokeh plot to a FastHTML FT component."""
"""Convert a Bokeh plot to a FastHTML FT component."""
script, div = components(plot)
if header:
header = H4(header, cls="mt-2")

14
src/akkudoktoreos/server/dash/components.py
View File

@@ -1,15 +1,13 @@
from typing import Any, Optional, Union
from fasthtml.common import H1, Div, Li
from fasthtml.common import H1, Button, Div, Li
from monsterui.daisy import (
Alert,
AlertT,
)
from monsterui.foundations import stringify
from monsterui.franken import (
from monsterui.franken import ( # Button, Does not pass hx_vals
H3,
Button,
ButtonT,
Card,
Container,
ContainerT,
@@ -246,7 +244,8 @@ def DashboardTrigger(*c: Any, cls: Optional[Union[str, tuple]] = None, **kwargs:
Returns:
Button: A styled `Button` component.
"""
new_cls = f"{ButtonT.primary}"
# new_cls = f"{ButtonT.primary} uk-border-rounded uk-padding-small"
new_cls = "uk-btn uk-btn-primary uk-border-rounded uk-padding-medium"
if cls:
new_cls += f" {stringify(cls)}"
kwargs["cls"] = new_cls
@@ -270,6 +269,7 @@ def DashboardTabs(dashboard_items: dict[str, str]) -> Card:
hx_get=f"{path}",
hx_target="#page-content",
hx_swap="innerHTML",
hx_vals='js:{ "dark": window.matchMedia("(prefers-color-scheme: dark)").matches }',
),
)
for menu, path in dashboard_items.items()
@@ -286,7 +286,9 @@ def DashboardContent(content: Any) -> Card:
Returns:
Card: A styled `Card` element containing the content.
"""
return Card(ScrollArea(Container(content, id="page-content"), cls="h-[75vh] w-full rounded-md"))
return Card(
ScrollArea(Container(content, id="page-content"), cls="h-[75vh] w-full rounded-md"),
)
def Page(

86
src/akkudoktoreos/server/dash/data/democonfig.json
View File

@@ -1,86 +0,0 @@
{
"elecprice": {
"charges_kwh": 0.21,
"provider": "ElecPriceAkkudoktor"
},
"general": {
"latitude": 52.5,
"longitude": 13.4
},
"prediction": {
"historic_hours": 48,
"hours": 48
},
"load": {
"provider": "LoadAkkudoktor",
"provider_settings": {
"loadakkudoktor_year_energy": 20000
}
},
"optimization": {
"hours": 48
},
"pvforecast": {
"planes": [
{
"peakpower": 5.0,
"surface_azimuth": 170,
"surface_tilt": 7,
"userhorizon": [
20,
27,
22,
20
],
"inverter_paco": 10000
},
{
"peakpower": 4.8,
"surface_azimuth": 90,
"surface_tilt": 7,
"userhorizon": [
30,
30,
30,
50
],
"inverter_paco": 10000
},
{
"peakpower": 1.4,
"surface_azimuth": 140,
"surface_tilt": 60,
"userhorizon": [
60,
30,
0,
30
],
"inverter_paco": 2000
},
{
"peakpower": 1.6,
"surface_azimuth": 185,
"surface_tilt": 45,
"userhorizon": [
45,
25,
30,
60
],
"inverter_paco": 1400
}
],
"provider": "PVForecastAkkudoktor"
},
"server": {
"startup_eosdash": true,
"host": "127.0.0.1",
"port": 8503,
"eosdash_host": "127.0.0.1",
"eosdash_port": 8504
},
"weather": {
"provider": "BrightSky"
}
}

13
src/akkudoktoreos/server/dash/eosstatus.py Normal file
View File

@@ -0,0 +1,13 @@
# EOS server status information
from typing import Optional
from akkudoktoreos.config.config import SettingsEOS
from akkudoktoreos.core.emplan import EnergyManagementPlan
from akkudoktoreos.optimization.optimization import OptimizationSolution
# The latest information from the EOS server
eos_health: Optional[dict] = None
eos_solution: Optional[OptimizationSolution] = None
eos_plan: Optional[EnergyManagementPlan] = None
eos_config: Optional[SettingsEOS] = None

6
src/akkudoktoreos/server/dash/footer.py
View File

@@ -6,6 +6,7 @@ from monsterui.daisy import Loading, LoadingT
from monsterui.franken import A, ButtonT, DivFullySpaced, P
from requests.exceptions import RequestException
import akkudoktoreos.server.dash.eosstatus as eosstatus
from akkudoktoreos.config.config import get_config
config_eos = get_config()
@@ -21,12 +22,15 @@ def get_alive(eos_host: str, eos_port: Union[str, int]) -> str:
Returns:
str: Alive data.
"""
global eos_health
result = requests.Response()
try:
result = requests.get(f"http://{eos_host}:{eos_port}/v1/health", timeout=10)
if result.status_code == 200:
alive = result.json()["status"]
eosstatus.eos_health = result.json()
alive = eosstatus.eos_health["status"]
else:
eosstatus.eos_health = None
alive = f"Server responded with status code: {result.status_code}"
except RequestException as e:
warning_msg = f"{e}"

496
src/akkudoktoreos/server/dash/plan.py Normal file
View File

@@ -0,0 +1,496 @@
from typing import Optional, Union
import pandas as pd
import requests
from bokeh.models import ColumnDataSource, LinearAxis, Range1d
from bokeh.plotting import figure
from loguru import logger
from monsterui.franken import (
Card,
Details,
Div,
DivLAligned,
Grid,
LabelCheckboxX,
P,
Summary,
UkIcon,
)
import akkudoktoreos.server.dash.eosstatus as eosstatus
from akkudoktoreos.config.config import SettingsEOS
from akkudoktoreos.core.emplan import (
DDBCInstruction,
EnergyManagementInstruction,
EnergyManagementPlan,
FRBCInstruction,
)
from akkudoktoreos.optimization.optimization import OptimizationSolution
from akkudoktoreos.server.dash.bokeh import Bokeh, bokey_apply_theme_to_plot
from akkudoktoreos.server.dash.components import Error
from akkudoktoreos.utils.datetimeutil import compare_datetimes, to_datetime
# bar width for 1 hour bars (time given in millseconds)
BAR_WIDTH_1HOUR = 1000 * 60 * 60
# Current state of solution displayed
solution_visible: dict[str, bool] = {
"pv_prediction_energy_wh": True,
"elec_price_prediction_amt_kwh": True,
}
solution_color: dict[str, str] = {}
def validate_source(source: ColumnDataSource, x_col: str = "date_time") -> None:
data = source.data
# 1. Source has data at all
if not data:
raise ValueError("ColumnDataSource has no data.")
# 2. x_col must be present
if x_col not in data:
raise ValueError(f"Missing expected x-axis column '{x_col}' in source.")
# 3. All columns must have equal length
lengths = {len(v) for v in data.values()}
if len(lengths) != 1:
raise ValueError(f"ColumnDataSource columns have mismatched lengths: {lengths}")
# 4. Must have at least one non-x column
y_columns = [c for c in data.keys() if c != x_col]
if not y_columns:
raise ValueError("No y-value columns found for plotting (only x-axis present).")
# 5. Each y-column must have at least one valid value
for col in y_columns:
values = [v for v in data[col] if v is not None]
if not values:
raise ValueError(f"Column '{col}' contains only None/NaN or is empty.")
def SolutionCard(solution: OptimizationSolution, config: SettingsEOS, data: Optional[dict]) -> Grid:
"""Creates a optimization solution card.
Args:
data (Optional[dict]): Incoming data containing action and category for processing.
"""
category = "solution"
dark = False
if data and data.get("category", None) == category:
# This data is for us
if data.get("action", None) == "visible":
renderer = data.get("renderer", None)
if renderer:
solution_visible[renderer] = bool(data.get(f"{renderer}-visible", False))
if data and data.get("dark", None) == "true":
dark = True
df = solution.data.to_dataframe()
if df.empty or len(df.columns) <= 1:
raise ValueError(f"DataFrame is empty or missing plottable columns: {list(df.columns)}")
if "date_time" not in df.columns:
raise ValueError(f"DataFrame is missing column 'date_time': {list(df.columns)}")
# Remove time offset from UTC to get naive local time and make bokey plot in local time
dst_offsets = df.index.map(lambda x: x.dst().total_seconds() / 3600)
if config.general is None or config.general.timezone is None:
date_time_tz = "Europe/Berlin"
else:
date_time_tz = config.general.timezone
df["date_time"] = pd.to_datetime(df["date_time"], utc=True).dt.tz_convert(date_time_tz)
# There is a special case if we have daylight saving time change in the time series
if dst_offsets.nunique() > 1:
date_time_tz += " + DST change"
source = ColumnDataSource(df)
validate_source(source)
# Calculate minimum and maximum Range
energy_wh_min = 0.0
energy_wh_max = 0.0
amt_kwh_min = 0.0
amt_kwh_max = 0.0
amt_min = 0.0
amt_max = 0.0
soc_factor_min = 0.0
soc_factor_max = 1.0
for col in df.columns:
if col.endswith("energy_wh"):
energy_wh_min = min(energy_wh_min, float(df[col].min()))
energy_wh_max = max(energy_wh_max, float(df[col].max()))
elif col.endswith("amt_kwh"):
amt_kwh_min = min(amt_kwh_min, float(df[col].min()))
amt_kwh_max = max(amt_kwh_max, float(df[col].max()))
elif col.endswith("amt"):
amt_min = min(amt_min, float(df[col].min()))
amt_max = max(amt_max, float(df[col].max()))
else:
continue
# Adjust to similar y-axis 0-point
# First get the maximum factor for the min value related the maximum value
min_max_factor = max(
(energy_wh_min * -1.0) / energy_wh_max,
(amt_kwh_min * -1.0) / amt_kwh_max,
(amt_min * -1.0) / amt_max,
(soc_factor_min * -1.0) / soc_factor_max,
)
# Adapt the min values to have the same relative min/max factor on all y-axis
energy_wh_min = min_max_factor * energy_wh_max * -1.0
amt_kwh_min = min_max_factor * amt_kwh_max * -1.0
amt_min = min_max_factor * amt_max * -1.0
soc_factor_min = min_max_factor * soc_factor_max * -1.0
# add 5% to min and max values for better display
energy_wh_range_orig = energy_wh_max - energy_wh_min
energy_wh_max += 0.05 * energy_wh_range_orig
energy_wh_min -= 0.05 * energy_wh_range_orig
amt_kwh_range_orig = amt_kwh_max - amt_kwh_min
amt_kwh_max += 0.05 * amt_kwh_range_orig
amt_kwh_min -= 0.05 * amt_kwh_range_orig
amt_range_orig = amt_max - amt_min
amt_max += 0.05 * amt_range_orig
amt_min -= 0.05 * amt_range_orig
soc_factor_range_orig = soc_factor_max - soc_factor_min
soc_factor_max += 0.05 * soc_factor_range_orig
soc_factor_min -= 0.05 * soc_factor_range_orig
if eosstatus.eos_health is not None:
last_run_datetime = eosstatus.eos_health["energy-management"]["last_run_datetime"]
start_datetime = eosstatus.eos_health["energy-management"]["start_datetime"]
else:
last_run_datetime = "unknown"
start_datetime = "unknown"
plot = figure(
title=f"Optimization Solution - last run: {last_run_datetime}",
x_axis_type="datetime",
x_axis_label=f"Datetime [localtime {date_time_tz}] - start: {start_datetime}",
y_axis_label="Energy [Wh]",
sizing_mode="stretch_width",
y_range=Range1d(energy_wh_min, energy_wh_max),
height=400,
)
plot.extra_y_ranges = {
"factor": Range1d(soc_factor_min, soc_factor_max), # y2
"amt_kwh": Range1d(amt_kwh_min, amt_kwh_max), # y3
"amt": Range1d(amt_min, amt_max), # y4
}
# y2 axis
y2_axis = LinearAxis(y_range_name="factor", axis_label="Factor [0.0..1.0]")
plot.add_layout(y2_axis, "left")
# y3 axis
y3_axis = LinearAxis(y_range_name="amt_kwh", axis_label="Electricty Price [currency/kWh]")
y3_axis.axis_label_text_color = "red"
plot.add_layout(y3_axis, "right")
# y4 axis
y4_axis = LinearAxis(y_range_name="amt", axis_label="Amount [currency]")
plot.add_layout(y4_axis, "right")
plot.toolbar.autohide = True
# Create line renderers for each column
renderers = {}
colors = ["black", "blue", "cyan", "green", "orange", "pink", "purple"]
for i, col in enumerate(sorted(df.columns)):
# Exclude some columns that are currently not used or are covered by others
excludes = [
"date_time",
"_op_mode",
"_fault_",
"_forced_discharge_",
"_outage_supply_",
"_reserve_backup_",
"_ramp_rate_control_",
"_frequency_regulation_",
"_grid_support_export_",
"_peak_shaving_",
]
# excludes = ["date_time"]
if any(exclude in col for exclude in excludes):
continue
if col in solution_visible:
visible = solution_visible[col]
else:
visible = False
solution_visible[col] = visible
if col in solution_color:
color = solution_color[col]
elif col == "pv_prediction_energy_wh":
color = "yellow"
solution_color[col] = color
elif col == "elec_price_prediction_amt_kwh":
color = "red"
solution_color[col] = color
else:
color = colors[i % len(colors)]
solution_color[col] = color
if visible:
if col == "pv_prediction_energy_wh":
r = plot.vbar(
x="date_time",
top=col,
source=source,
width=BAR_WIDTH_1HOUR * 0.8,
legend_label=col,
color=color,
level="underlay",
)
elif col.endswith("energy_wh"):
r = plot.step(
x="date_time",
y=col,
mode="before",
source=source,
legend_label=col,
color=color,
)
elif col.endswith("factor"):
r = plot.step(
x="date_time",
y=col,
mode="before",
source=source,
legend_label=col,
color=color,
y_range_name="factor",
)
elif col.endswith("mode"):
r = plot.step(
x="date_time",
y=col,
mode="before",
source=source,
legend_label=col,
color=color,
y_range_name="factor",
)
elif col.endswith("amt_kwh"):
r = plot.step(
x="date_time",
y=col,
mode="before",
source=source,
legend_label=col,
color=color,
y_range_name="amt_kwh",
)
elif col.endswith("amt"):
r = plot.step(
x="date_time",
y=col,
mode="before",
source=source,
legend_label=col,
color=color,
y_range_name="amt",
)
else:
raise ValueError(f"Unexpected column name: {col}")
else:
r = None
renderers[col] = r
plot.legend.visible = False # no legend at plot
bokey_apply_theme_to_plot(plot, dark)
# --- CheckboxGroup to toggle datasets ---
Checkbox = Grid(
*[
LabelCheckboxX(
label=renderer,
id=f"{renderer}-visible",
name=f"{renderer}-visible",
value="true",
checked=solution_visible[renderer],
hx_post="/eosdash/plan",
hx_target="#page-content",
hx_swap="innerHTML",
hx_vals='js:{ "category": "solution", "action": "visible", "renderer": '
+ '"'
+ f"{renderer}"
+ '", '
+ '"dark": window.matchMedia("(prefers-color-scheme: dark)").matches '
+ "}",
lbl_cls=f"text-{solution_color[renderer]}-500",
)
for renderer in list(renderers.keys())
],
cols=2,
)
return Grid(
Bokeh(plot),
Card(
Checkbox,
),
cls="w-full space-y-3 space-x-3",
)
def InstructionCard(
instruction: EnergyManagementInstruction, config: SettingsEOS, data: Optional[dict]
) -> Card:
"""Creates a styled instruction card for displaying instruction details.
This function generates a instruction card that is displayed in the UI with
various sections such as instruction name, type, description, default value,
current value, and error details. It supports both read-only and editable modes.
Args:
instruction (EnergyManagementInstruction): The instruction.
data (Optional[dict]): Incoming data containing action and category for processing.
Returns:
Card: A styled Card component containing the instruction details.
"""
if instruction.id is None:
return Error("Instruction without id encountered. Can not handle")
idx = instruction.id.find("@")
resource_id = instruction.id[:idx] if idx != -1 else instruction.id
execution_time = to_datetime(instruction.execution_time, as_string=True)
description = instruction.type
summary = None
# Search an icon that fits to device_id
if (
config.devices
and config.devices.batteries
and any(
battery_config.device_id == resource_id for battery_config in config.devices.batteries
)
):
# This is a battery
if instruction.operation_mode_id in ("CHARGE",):
icon = "battery-charging"
else:
icon = "battery"
elif (
config.devices
and config.devices.electric_vehicles
and any(
electric_vehicle_config.device_id == resource_id
for electric_vehicle_config in config.devices.electric_vehicles
)
):
# This is a car battery
icon = "car"
elif (
config.devices
and config.devices.home_appliances
and any(
home_appliance.device_id == resource_id
for home_appliance in config.devices.home_appliances
)
):
# This is a home appliance
icon = "washing-machine"
else:
icon = "play"
if isinstance(instruction, (DDBCInstruction, FRBCInstruction)):
summary = f"{instruction.operation_mode_id}"
summary_detail = f"{instruction.operation_mode_factor}"
return Card(
Details(
Summary(
Grid(
Grid(
DivLAligned(
UkIcon(icon=icon),
P(execution_time),
),
DivLAligned(
P(resource_id),
),
),
P(summary),
P(summary_detail),
),
cls="list-none",
),
Grid(
P(description),
P("TBD"),
),
),
cls="w-full",
)
def Plan(eos_host: str, eos_port: Union[str, int], data: Optional[dict] = None) -> Div:
"""Generates the plan dashboard layout.
Args:
eos_host (str): The hostname of the EOS server.
eos_port (Union[str, int]): The port of the EOS server.
data (Optional[dict], optional): Incoming data to trigger plan actions. Defaults to None.
Returns:
Div: A `Div` component containing the assembled admin interface.
"""
server = f"http://{eos_host}:{eos_port}"
print("Plan: ", data)
if (
eosstatus.eos_config is None
or eosstatus.eos_solution is None
or eosstatus.eos_plan is None
or eosstatus.eos_health is None
or compare_datetimes(
to_datetime(eosstatus.eos_plan.generated_at),
to_datetime(eosstatus.eos_health["energy-management"]["last_run_datetime"]),
).lt
):
# Get current configuration from server
try:
result = requests.get(f"{server}/v1/config", timeout=10)
result.raise_for_status()
except requests.exceptions.HTTPError as err:
detail = result.json()["detail"]
return Error(f"Can not retrieve configuration from {server}: {err}, {detail}")
eosstatus.eos_config = SettingsEOS(**result.json())
# Get the optimization solution
try:
result = requests.get(
f"{server}/v1/energy-management/optimization/solution", timeout=10
)
result.raise_for_status()
solution_json = result.json()
except requests.exceptions.HTTPError as e:
detail = result.json()["detail"]
warning_msg = f"Can not retrieve optimization solution from {server}: {e}, {detail}"
logger.warning(warning_msg)
return Error(warning_msg)
except Exception as e:
warning_msg = f"Can not retrieve optimization solution from {server}: {e}"
logger.warning(warning_msg)
return Error(warning_msg)
eosstatus.eos_solution = OptimizationSolution(**solution_json)
# Get the plan
try:
result = requests.get(f"{server}/v1/energy-management/plan", timeout=10)
result.raise_for_status()
plan_json = result.json()
except requests.exceptions.HTTPError as e:
detail = result.json()["detail"]
warning_msg = f"Can not retrieve plan from {server}: {e}, {detail}"
logger.warning(warning_msg)
return Error(warning_msg)
except Exception as e:
warning_msg = f"Can not retrieve plan from {server}: {e}"
logger.warning(warning_msg)
return Error(warning_msg)
eosstatus.eos_plan = EnergyManagementPlan(**plan_json, data=data)
rows = [
SolutionCard(eosstatus.eos_solution, eosstatus.eos_config, data=data),
]
for instruction in eosstatus.eos_plan.instructions:
rows.append(InstructionCard(instruction, eosstatus.eos_config, data=data))
return Div(*rows, cls="space-y-4")
# return Div(f"Plan:\n{json.dumps(plan_json, indent=4)}")

115
src/akkudoktoreos/server/dash/demo.py → src/akkudoktoreos/server/dash/prediction.py
View File

@@ -1,6 +1,4 @@
import json
from pathlib import Path
from typing import Union
from typing import Optional, Union
import pandas as pd
import requests
@@ -9,30 +7,25 @@ from bokeh.plotting import figure
from monsterui.franken import FT, Grid, P
from akkudoktoreos.core.pydantic import PydanticDateTimeDataFrame
from akkudoktoreos.server.dash.bokeh import Bokeh
DIR_DEMODATA = Path(__file__).absolute().parent.joinpath("data")
FILE_DEMOCONFIG = DIR_DEMODATA.joinpath("democonfig.json")
if not FILE_DEMOCONFIG.exists():
raise ValueError(f"File does not exist: {FILE_DEMOCONFIG}")
from akkudoktoreos.server.dash.bokeh import Bokeh, bokey_apply_theme_to_plot
from akkudoktoreos.server.dash.components import Error
# bar width for 1 hour bars (time given in millseconds)
BAR_WIDTH_1HOUR = 1000 * 60 * 60
def DemoPVForecast(predictions: pd.DataFrame, config: dict) -> FT:
def PVForecast(predictions: pd.DataFrame, config: dict, date_time_tz: str, dark: bool) -> FT:
source = ColumnDataSource(predictions)
provider = config["pvforecast"]["provider"]
plot = figure(
x_axis_type="datetime",
title=f"PV Power Prediction ({provider})",
x_axis_label="Datetime",
x_axis_label=f"Datetime [localtime {date_time_tz}]",
y_axis_label="Power [W]",
sizing_mode="stretch_width",
height=400,
)
plot.vbar(
x="date_time",
top="pvforecast_ac_power",
@@ -41,11 +34,15 @@ def DemoPVForecast(predictions: pd.DataFrame, config: dict) -> FT:
legend_label="AC Power",
color="lightblue",
)
plot.toolbar.autohide = True
bokey_apply_theme_to_plot(plot, dark)
return Bokeh(plot)
def DemoElectricityPriceForecast(predictions: pd.DataFrame, config: dict) -> FT:
def ElectricityPriceForecast(
predictions: pd.DataFrame, config: dict, date_time_tz: str, dark: bool
) -> FT:
source = ColumnDataSource(predictions)
provider = config["elecprice"]["provider"]
@@ -56,7 +53,7 @@ def DemoElectricityPriceForecast(predictions: pd.DataFrame, config: dict) -> FT:
predictions["elecprice_marketprice_kwh"].max() + 0.1,
),
title=f"Electricity Price Prediction ({provider})",
x_axis_label="Datetime",
x_axis_label=f"Datetime [localtime {date_time_tz}]",
y_axis_label="Price [€/kWh]",
sizing_mode="stretch_width",
height=400,
@@ -69,18 +66,22 @@ def DemoElectricityPriceForecast(predictions: pd.DataFrame, config: dict) -> FT:
legend_label="Market Price",
color="lightblue",
)
plot.toolbar.autohide = True
bokey_apply_theme_to_plot(plot, dark)
return Bokeh(plot)
def DemoWeatherTempAirHumidity(predictions: pd.DataFrame, config: dict) -> FT:
def WeatherTempAirHumidityForecast(
predictions: pd.DataFrame, config: dict, date_time_tz: str, dark: bool
) -> FT:
source = ColumnDataSource(predictions)
provider = config["weather"]["provider"]
plot = figure(
x_axis_type="datetime",
title=f"Air Temperature and Humidity Prediction ({provider})",
x_axis_label="Datetime",
x_axis_label=f"Datetime [localtime {date_time_tz}]",
y_axis_label="Temperature [°C]",
sizing_mode="stretch_width",
height=400,
@@ -94,7 +95,6 @@ def DemoWeatherTempAirHumidity(predictions: pd.DataFrame, config: dict) -> FT:
plot.line(
"date_time", "weather_temp_air", source=source, legend_label="Air Temperature", color="blue"
)
plot.line(
"date_time",
"weather_relative_humidity",
@@ -103,18 +103,22 @@ def DemoWeatherTempAirHumidity(predictions: pd.DataFrame, config: dict) -> FT:
color="green",
y_range_name="humidity",
)
plot.toolbar.autohide = True
bokey_apply_theme_to_plot(plot, dark)
return Bokeh(plot)
def DemoWeatherIrradiance(predictions: pd.DataFrame, config: dict) -> FT:
def WeatherIrradianceForecast(
predictions: pd.DataFrame, config: dict, date_time_tz: str, dark: bool
) -> FT:
source = ColumnDataSource(predictions)
provider = config["weather"]["provider"]
plot = figure(
x_axis_type="datetime",
title=f"Irradiance Prediction ({provider})",
x_axis_label="Datetime",
x_axis_label=f"Datetime [localtime {date_time_tz}]",
y_axis_label="Irradiance [W/m2]",
sizing_mode="stretch_width",
height=400,
@@ -140,21 +144,25 @@ def DemoWeatherIrradiance(predictions: pd.DataFrame, config: dict) -> FT:
legend_label="Diffuse Horizontal Irradiance",
color="blue",
)
plot.toolbar.autohide = True
bokey_apply_theme_to_plot(plot, dark)
return Bokeh(plot)
def DemoLoad(predictions: pd.DataFrame, config: dict) -> FT:
def LoadForecast(predictions: pd.DataFrame, config: dict, date_time_tz: str, dark: bool) -> FT:
source = ColumnDataSource(predictions)
provider = config["load"]["provider"]
if provider == "LoadAkkudoktor":
year_energy = config["load"]["provider_settings"]["loadakkudoktor_year_energy"]
year_energy = config["load"]["provider_settings"]["LoadAkkudoktor"][
"loadakkudoktor_year_energy"
]
provider = f"{provider}, {year_energy} kWh"
plot = figure(
x_axis_type="datetime",
title=f"Load Prediction ({provider})",
x_axis_label="Datetime",
x_axis_type="datetime",
x_axis_label=f"Datetime [localtime {date_time_tz}]",
y_axis_label="Load [W]",
sizing_mode="stretch_width",
height=400,
@@ -189,13 +197,19 @@ def DemoLoad(predictions: pd.DataFrame, config: dict) -> FT:
color="green",
y_range_name="stddev",
)
plot.toolbar.autohide = True
bokey_apply_theme_to_plot(plot, dark)
return Bokeh(plot)
def Demo(eos_host: str, eos_port: Union[str, int]) -> str:
def Prediction(eos_host: str, eos_port: Union[str, int], data: Optional[dict] = None) -> str:
server = f"http://{eos_host}:{eos_port}"
dark = False
if data and data.get("dark", None) == "true":
dark = True
# Get current configuration from server
try:
result = requests.get(f"{server}/v1/config", timeout=10)
@@ -208,34 +222,6 @@ def Demo(eos_host: str, eos_port: Union[str, int]) -> str:
)
config = result.json()
# Set demo configuration
with FILE_DEMOCONFIG.open("r", encoding="utf-8") as fd:
democonfig = json.load(fd)
try:
result = requests.put(f"{server}/v1/config", json=democonfig, timeout=10)
result.raise_for_status()
except requests.exceptions.HTTPError as err:
detail = result.json()["detail"]
# Try to reset to original config
requests.put(f"{server}/v1/config", json=config, timeout=10)
return P(
f"Can not set demo configuration on {server}: {err}, {detail}",
cls="text-center",
)
# Update all predictions
try:
result = requests.post(f"{server}/v1/prediction/update", timeout=10)
result.raise_for_status()
except requests.exceptions.HTTPError as err:
detail = result.json()["detail"]
# Try to reset to original config
requests.put(f"{server}/v1/config", json=config, timeout=10)
return P(
f"Can not update predictions on {server}: {err}, {detail}",
cls="text-center",
)
# Get Forecasts
try:
params = {
@@ -257,24 +243,19 @@ def Demo(eos_host: str, eos_port: Union[str, int]) -> str:
predictions = PydanticDateTimeDataFrame(**result.json()).to_dataframe()
except requests.exceptions.HTTPError as err:
detail = result.json()["detail"]
return P(
f"Can not retrieve predictions from {server}: {err}, {detail}",
cls="text-center",
)
return Error(f"Can not retrieve predictions from {server}: {err}, {detail}")
except Exception as err:
return P(
f"Can not retrieve predictions from {server}: {err}",
cls="text-center",
)
return Error(f"Can not retrieve predictions from {server}: {err}")
# Reset to original config
requests.put(f"{server}/v1/config", json=config, timeout=10)
# Remove time offset from UTC to get naive local time and make bokeh plot in local time
date_time_tz = predictions["date_time"].dt.tz
predictions["date_time"] = pd.to_datetime(predictions["date_time"]).dt.tz_localize(None)
return Grid(
DemoPVForecast(predictions, democonfig),
DemoElectricityPriceForecast(predictions, democonfig),
DemoWeatherTempAirHumidity(predictions, democonfig),
DemoWeatherIrradiance(predictions, democonfig),
DemoLoad(predictions, democonfig),
PVForecast(predictions, config, date_time_tz, dark),
ElectricityPriceForecast(predictions, config, date_time_tz, dark),
WeatherTempAirHumidityForecast(predictions, config, date_time_tz, dark),
WeatherIrradianceForecast(predictions, config, date_time_tz, dark),
LoadForecast(predictions, config, date_time_tz, dark),
cols_max=2,
)

747
src/akkudoktoreos/server/eos.py
View File

File diff suppressed because it is too large Load Diff

342
src/akkudoktoreos/server/eosdash.py
View File

@@ -3,7 +3,6 @@ import os
import sys
import traceback
from pathlib import Path
from typing import Optional
import psutil
import uvicorn
@@ -12,33 +11,171 @@ from loguru import logger
from monsterui.core import FastHTML, Theme
from akkudoktoreos.config.config import get_config
from akkudoktoreos.core.logabc import LOGGING_LEVELS
from akkudoktoreos.core.logging import track_logging_config
from akkudoktoreos.core.version import __version__
from akkudoktoreos.server.dash.about import About
# Pages
from akkudoktoreos.server.dash.admin import Admin
from akkudoktoreos.server.dash.bokeh import BokehJS
from akkudoktoreos.server.dash.components import Page
from akkudoktoreos.server.dash.configuration import ConfigKeyUpdate, Configuration
from akkudoktoreos.server.dash.demo import Demo
from akkudoktoreos.server.dash.footer import Footer
from akkudoktoreos.server.dash.hello import Hello
from akkudoktoreos.server.dash.plan import Plan
from akkudoktoreos.server.dash.prediction import Prediction
from akkudoktoreos.server.server import get_default_host, wait_for_port_free
from akkudoktoreos.utils.stringutil import str2bool
config_eos = get_config()
# ------------------------------------
# Logging configuration at import time
# ------------------------------------
logger.remove()
track_logging_config(config_eos, "logging", None, None)
config_eos.track_nested_value("/logging", track_logging_config)
# ----------------------------
# Safe argparse at import time
# ----------------------------
parser = argparse.ArgumentParser(description="Start EOSdash server.")
parser.add_argument(
"--host",
type=str,
help="Host for the EOSdash server (default: value from config)",
)
parser.add_argument(
"--port",
type=int,
help="Port for the EOSdash server (default: value from config)",
)
parser.add_argument(
"--eos-host",
type=str,
help="Host of the EOS server (default: value from config)",
)
parser.add_argument(
"--eos-port",
type=int,
help="Port of the EOS server (default: value from config)",
)
parser.add_argument(
"--log_level",
type=str,
default="INFO",
help='Log level for the server. Options: "critical", "error", "warning", "info", "debug", "trace" (default: "INFO")',
)
parser.add_argument(
"--access_log",
type=str2bool,
default=False,
help="Enable or disable access logging. Options: True or False (default: False)",
)
parser.add_argument(
"--reload",
type=str2bool,
default=False,
help="Enable or disable auto-reload. Useful for development. Options: True or False (default: False)",
)
# Command line arguments
args: argparse.Namespace
args_unknown: list[str]
args, args_unknown = parser.parse_known_args()
# -----------------------------
# Prepare config at import time
# -----------------------------
# Set EOS config to actual environment variable & config file content
config_eos.reset_settings()
# Setup parameters from args, config_eos and default
# Remember parameters in config
config_eosdash = {}
# Setup EOS logging level - first to have the other logging messages logged
# - log level
if args and args.log_level is not None:
config_eosdash["log_level"] = args.log_level.upper()
else:
config_eosdash["log_level"] = "info"
# Ensure log_level from command line is in config settings
if config_eosdash["log_level"] in LOGGING_LEVELS:
# Setup console logging level using nested value
# - triggers logging configuration by track_logging_config
config_eos.set_nested_value("logging/console_level", config_eosdash["log_level"])
logger.debug(
f"logging/console_level configuration set by argument to {config_eosdash['log_level']}"
)
# Setup EOS server host
if args and args.eos_host:
config_eosdash["eos_host"] = args.eos_host
elif config_eos.server.host:
config_eosdash["eos_host"] = str(config_eos.server.host)
else:
config_eosdash["eos_host"] = get_default_host()
# Setup EOS server port
if args and args.eos_port:
config_eosdash["eos_port"] = args.eos_port
elif config_eos.server.port:
config_eosdash["eos_port"] = config_eos.server.port
else:
config_eosdash["eos_port"] = 8503
# - EOSdash host
if args and args.host:
config_eosdash["eosdash_host"] = args.host
elif config_eos.server.eosdash_host:
config_eosdash["eosdash_host"] = str(config_eos.server.eosdash_host)
else:
config_eosdash["eosdash_host"] = get_default_host()
# - EOS port
if args and args.port:
config_eosdash["eosdash_port"] = args.port
elif config_eos.server.eosdash_port:
config_eosdash["eosdash_port"] = config_eos.server.eosdash_port
else:
config_eosdash["eosdash_port"] = 8504
# - access log
if args and args.access_log:
config_eosdash["access_log"] = args.access_log
else:
config_eosdash["access_log"] = False
# - reload
if args is None and args.reload is None:
config_eosdash["reload"] = False
else:
config_eosdash["reload"] = args.reload
# ---------------------
# Prepare FastHTML app
# ---------------------
# The favicon for EOSdash
favicon_filepath = Path(__file__).parent.joinpath("dash/assets/favicon/favicon.ico")
if not favicon_filepath.exists():
raise ValueError(f"Does not exist {favicon_filepath}")
# Command line arguments
args: Optional[argparse.Namespace] = None
# Get frankenui and tailwind headers via CDN using Theme.green.headers()
# Add Bokeh headers
# Get frankenui and tailwind headers via CDN using Theme.green.headers()
hdrs = (
*BokehJS,
Theme.green.headers(highlightjs=True),
BokehJS,
)
# The EOSdash application
@@ -52,24 +189,14 @@ app: FastHTML = FastHTML(
def eos_server() -> tuple[str, int]:
"""Retrieves the EOS server host and port configuration.
If `args` is provided, it uses the `eos_host` and `eos_port` from `args`.
Otherwise, it falls back to the values from `config_eos.server`.
Takes values from `config_eos.server` or default.
Returns:
tuple[str, int]: A tuple containing:
- `eos_host` (str): The EOS server hostname or IP.
- `eos_port` (int): The EOS server port.
"""
if args is None:
eos_host = str(config_eos.server.host)
eos_port = config_eos.server.port
else:
eos_host = args.eos_host
eos_port = args.eos_port
eos_host = eos_host if eos_host else get_default_host()
eos_port = eos_port if eos_port else 8503
return eos_host, eos_port
return config_eosdash["eos_host"], config_eosdash["eos_port"]
@app.get("/favicon.ico")
@@ -88,12 +215,13 @@ def get_eosdash(): # type: ignore
return Page(
None,
{
"EOSdash": "/eosdash/hello",
"Plan": "/eosdash/plan",
"Prediction": "/eosdash/prediction",
"Config": "/eosdash/configuration",
"Demo": "/eosdash/demo",
"Admin": "/eosdash/admin",
"About": "/eosdash/about",
},
Hello(),
About(),
Footer(*eos_server()),
"/eosdash/footer",
)
@@ -109,14 +237,14 @@ def get_eosdash_footer(): # type: ignore
return Footer(*eos_server())
@app.get("/eosdash/hello")
def get_eosdash_hello(): # type: ignore
"""Serves the EOSdash Hello page.
@app.get("/eosdash/about")
def get_eosdash_about(): # type: ignore
"""Serves the EOSdash About page.
Returns:
Hello: The Hello page component.
About: The About page component.
"""
return Hello()
return About()
@app.get("/eosdash/admin")
@@ -131,6 +259,13 @@ def get_eosdash_admin(): # type: ignore
@app.post("/eosdash/admin")
def post_eosdash_admin(data: dict): # type: ignore
"""Provide control data to the Admin page.
This endpoint is called from within the Admin page on user actions.
Returns:
Admin: The Admin page component.
"""
return Admin(*eos_server(), data)
@@ -149,14 +284,36 @@ def put_eosdash_configuration(data: dict): # type: ignore
return ConfigKeyUpdate(*eos_server(), data["key"], data["value"])
@app.get("/eosdash/demo")
def get_eosdash_demo(): # type: ignore
"""Serves the EOSdash Demo page.
@app.get("/eosdash/plan")
def get_eosdash_plan(data: dict): # type: ignore
"""Serves the EOSdash Plan page.
Returns:
Demo: The Demo page component.
Plan: The Plan page component.
"""
return Demo(*eos_server())
return Plan(*eos_server(), data)
@app.post("/eosdash/plan")
def post_eosdash_plan(data: dict): # type: ignore
"""Provide control data to the Plan page.
This endpoint is called from within the Plan page on user actions.
Returns:
Plan: The Plan page component.
"""
return Plan(*eos_server(), data)
@app.get("/eosdash/prediction")
def get_eosdash_prediction(data: dict): # type: ignore
"""Serves the EOSdash Prediction page.
Returns:
Prediction: The Prediction page component.
"""
return Prediction(*eos_server(), data)
@app.get("/eosdash/health")
@@ -166,6 +323,7 @@ def get_eosdash_health(): # type: ignore
{
"status": "alive",
"pid": psutil.Process().pid,
"version": __version__,
}
)
@@ -190,74 +348,22 @@ def run_eosdash() -> None:
Returns:
None
"""
# Setup parameters from args, config_eos and default
# Remember parameters that are also in config
# - EOS host
if args and args.eos_host:
eos_host = args.eos_host
elif config_eos.server.host:
eos_host = config_eos.server.host
else:
eos_host = get_default_host()
config_eos.server.host = eos_host
# - EOS port
if args and args.eos_port:
eos_port = args.eos_port
elif config_eos.server.port:
eos_port = config_eos.server.port
else:
eos_port = 8503
config_eos.server.port = eos_port
# - EOSdash host
if args and args.host:
eosdash_host = args.host
elif config_eos.server.eosdash.host:
eosdash_host = config_eos.server.eosdash_host
else:
eosdash_host = get_default_host()
config_eos.server.eosdash_host = eosdash_host
# - EOS port
if args and args.port:
eosdash_port = args.port
elif config_eos.server.eosdash_port:
eosdash_port = config_eos.server.eosdash_port
else:
eosdash_port = 8504
config_eos.server.eosdash_port = eosdash_port
# - log level
if args and args.log_level:
log_level = args.log_level
else:
log_level = "info"
# - access log
if args and args.access_log:
access_log = args.access_log
else:
access_log = False
# - reload
if args and args.reload:
reload = args.reload
else:
reload = False
# Make hostname Windows friendly
if eosdash_host == "0.0.0.0" and os.name == "nt": # noqa: S104
eosdash_host = "localhost"
# Wait for EOSdash port to be free - e.g. in case of restart
wait_for_port_free(eosdash_port, timeout=120, waiting_app_name="EOSdash")
wait_for_port_free(config_eosdash["eosdash_port"], timeout=120, waiting_app_name="EOSdash")
try:
uvicorn.run(
"akkudoktoreos.server.eosdash:app",
host=eosdash_host,
port=eosdash_port,
log_level=log_level.lower(),
access_log=access_log,
reload=reload,
host=config_eosdash["eosdash_host"],
port=config_eosdash["eosdash_port"],
log_level=config_eosdash["log_level"].lower(),
access_log=config_eosdash["access_log"],
reload=config_eosdash["reload"],
)
except Exception as e:
logger.error(f"Could not bind to host {eosdash_host}:{eosdash_port}. Error: {e}")
logger.error(
f"Could not bind to host {config_eosdash['eosdash_host']}:{config_eosdash['eosdash_port']}. Error: {e}"
)
raise e
@@ -278,54 +384,6 @@ def main() -> None:
--access_log (bool): Enable or disable access log. Options: True or False (default: False).
--reload (bool): Enable or disable auto-reload. Useful for development. Options: True or False (default: False).
"""
parser = argparse.ArgumentParser(description="Start EOSdash server.")
parser.add_argument(
"--host",
type=str,
default=str(config_eos.server.eosdash_host),
help="Host for the EOSdash server (default: value from config)",
)
parser.add_argument(
"--port",
type=int,
default=config_eos.server.eosdash_port,
help="Port for the EOSdash server (default: value from config)",
)
parser.add_argument(
"--eos-host",
type=str,
default=str(config_eos.server.host),
help="Host of the EOS server (default: value from config)",
)
parser.add_argument(
"--eos-port",
type=int,
default=config_eos.server.port,
help="Port of the EOS server (default: value from config)",
)
parser.add_argument(
"--log_level",
type=str,
default="info",
help='Log level for the server. Options: "critical", "error", "warning", "info", "debug", "trace" (default: "info")',
)
parser.add_argument(
"--access_log",
type=bool,
default=False,
help="Enable or disable access log. Options: True or False (default: False)",
)
parser.add_argument(
"--reload",
type=bool,
default=False,
help="Enable or disable auto-reload. Useful for development. Options: True or False (default: False)",
)
global args
args = parser.parse_args()
try:
run_eosdash()
except Exception as ex:

83
src/akkudoktoreos/server/server.py
View File

@@ -2,12 +2,13 @@
import ipaddress
import re
import socket
import time
from typing import Optional, Union
from typing import Optional
import psutil
from loguru import logger
from pydantic import Field, IPvAnyAddress, field_validator
from pydantic import Field, field_validator
from akkudoktoreos.config.configabc import SettingsBaseModel
@@ -17,7 +18,29 @@ def get_default_host() -> str:
return "127.0.0.1"
def is_valid_ip_or_hostname(value: str) -> bool:
def get_host_ip() -> str:
"""IP address of the host machine.
This function determines the IP address used to communicate with the outside world
(e.g., for internet access), without sending any actual data. It does so by
opening a UDP socket connection to a public IP address (Google DNS).
Returns:
str: The local IP address as a string. Returns '127.0.0.1' if unable to determine.
Example:
>>> get_host_ip()
'192.168.1.42'
"""
try:
with socket.socket(socket.AF_INET, socket.SOCK_DGRAM) as s:
s.connect(("8.8.8.8", 80))
return s.getsockname()[0]
except Exception:
return "127.0.0.1"
def validate_ip_or_hostname(value: str) -> str:
"""Validate whether a string is a valid IP address (IPv4 or IPv6) or hostname.
This function first attempts to interpret the input as an IP address using the
@@ -29,23 +52,30 @@ def is_valid_ip_or_hostname(value: str) -> bool:
value (str): The input string to validate.
Returns:
bool: True if the input is a valid IP address or hostname, False otherwise.
IP address: Valid IP address or hostname.
"""
try:
ipaddress.ip_address(value)
return True
return value
except ValueError:
pass
if len(value) > 253:
return False
raise ValueError(f"Not a valid hostname: {value}")
hostname_regex = re.compile(
r"^(?=.{1,253}$)(?!-)[A-Z\d-]{1,63}(?<!-)"
r"(?:\.(?!-)[A-Z\d-]{1,63}(?<!-))*\.?$",
re.IGNORECASE,
)
return bool(hostname_regex.fullmatch(value))
if not bool(hostname_regex.fullmatch(value)):
raise ValueError(f"Not a valid hostname: {value}")
ip = socket.gethostbyname(value)
if ip is None:
raise ValueError(f"Unknown host: {value}")
return value
def wait_for_port_free(port: int, timeout: int = 0, waiting_app_name: str = "App") -> bool:
@@ -121,28 +151,39 @@ def wait_for_port_free(port: int, timeout: int = 0, waiting_app_name: str = "App
class ServerCommonSettings(SettingsBaseModel):
"""Server Configuration."""
host: Optional[IPvAnyAddress] = Field(
default=get_default_host(), description="EOS server IP address."
host: Optional[str] = Field(
default=get_default_host(),
description="EOS server IP address. Defaults to 127.0.0.1.",
examples=["127.0.0.1", "localhost"],
)
port: Optional[int] = Field(
default=8503,
description="EOS server IP port number. Defaults to 8503.",
examples=[
8503,
],
)
port: Optional[int] = Field(default=8503, description="EOS server IP port number.")
verbose: Optional[bool] = Field(default=False, description="Enable debug output")
startup_eosdash: Optional[bool] = Field(
default=True, description="EOS server to start EOSdash server."
default=True, description="EOS server to start EOSdash server. Defaults to True."
)
eosdash_host: Optional[IPvAnyAddress] = Field(
default=get_default_host(), description="EOSdash server IP address."
eosdash_host: Optional[str] = Field(
default=None,
description="EOSdash server IP address. Defaults to EOS server IP address.",
examples=["127.0.0.1", "localhost"],
)
eosdash_port: Optional[int] = Field(
default=None,
description="EOSdash server IP port number. Defaults to EOS server IP port number + 1.",
examples=[
8504,
],
)
eosdash_port: Optional[int] = Field(default=8504, description="EOSdash server IP port number.")
@field_validator("host", "eosdash_host", mode="before")
def validate_server_host(
cls, value: Optional[Union[str, IPvAnyAddress]]
) -> Optional[Union[str, IPvAnyAddress]]:
def validate_server_host(cls, value: Optional[str]) -> Optional[str]:
if isinstance(value, str):
if not is_valid_ip_or_hostname(value):
raise ValueError(f"Invalid host: {value}")
if value.lower() in ("localhost", "loopback"):
value = "127.0.0.1"
value = validate_ip_or_hostname(value)
return value
@field_validator("port", "eosdash_port")