mirror of
https://github.com/Akkudoktor-EOS/EOS.git
synced 2026-08-31 04:36:37 +00:00
ba76087db9af87c1dd1c6998c7a559bf83c6f968
17
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
  NormannandGitHub
|
6849b731b0 |
chore(deps): combine Dependabot updates (#1250)
### Significant changes / EOS impact The dependency bumps are mostly patch/tooling updates, but a few are worth explicit review: - **`pydantic-settings` 2.14.2 → 2.15.0 — highest runtime impact.** EOS directly subclasses `pydantic_settings.BaseSettings` for `SettingsEOS` / `ConfigEOS`, with `case_sensitive` left at its default. In 2.15.0, `case_sensitive` now also applies to init kwargs and config-file sources, so top-level setting keys are now matched case-insensitively by default where those sources previously did not behave that way. This can change how differently-cased config keys are accepted/resolved. The release also adds warnings for unresolved forward references and changes strict non-JSON env-value failures to `ValidationError`. **Recommended:** exercise JSON config loading, init/update paths, env overrides, and config round-trips. - **`tzfpy` 1.3.2 → 1.3.3 — runtime correctness change.** EOS uses `tzfpy.get_tz()` in `to_timezone()` and exposes the result through `GeneralSettings.timezone`. Queries exactly on timezone polygon borders now resolve instead of returning an empty result, and the `America/Argentina/Ushuaia` boundary is corrected. This can intentionally change timezone output for users on/near affected boundaries. - **`cachebox` 6.2.2 → 6.2.5 — runtime cache correctness/safety.** EOS uses `cachebox.LRUCache` and `cachebox.cached` for the energy-management cache. The update fixes iterator lifetime safety, LRU iterator invalidation when reads promote entries, and a potential `setdefault_with` locking issue. EOS does not appear to rely on those edge cases directly, so this is expected to be low-risk and mostly corrective; existing cache tests are the relevant regression coverage. - **`GitPython` 3.1.58 → 3.1.59 — dev/tooling security hardening.** This release blocks file-reading Git options, separate git-directory use during clone, and hardens config parsing. It appears to be a dev/docs dependency rather than EOS runtime code, but CI/tooling that intentionally passes unusual Git options could be affected. - **`pre-commit` 4.6.1 → 4.6.2 — dev-only bug fix.** Fixes a regression in Node-language hooks using npm 11.x build scripts. - **`mypy` 2.3.1, `commitizen` 4.17.1, and `types-PyYaml` stub update** are tooling/type-checking changes with no expected EOS runtime behavior change. Overall, the main compatibility focus should be **configuration handling (`pydantic-settings`)**, followed by **timezone edge cases (`tzfpy`)**. The remaining updates are primarily correctness, security, or developer-tooling fixes. |
||
|
9eb3c7e483 |
build(deps): refresh dependencies and clean up test warnings (#1243)
Consolidates the currently applicable dependency updates into one PR, including the closed Dependabot backlog such as #1241, plus dependency surfaces that were not covered by the repository's previous pip-only Dependabot configuration. Cleanup of test warnings. Most importent: * GitPython + pypdf security hardening. * Uvicorn WebSocket close/backpressure/header fixes for server/dashboard reliability. * FastAPI dependency-memory/OpenAPI improvements for the API process. * Bokeh WebSocket/resource-leak/prefix fixes for EOSdash and proxied deployments. * cachebox cancellation/lock cleanup fixes for long-running/concurrent work. * pandas 3.0.5 avoiding the yanked 3.0.4 datetime/segfault build. * Ruff security-lint and pydocstyle correctness fixes, plus faster release builds via PGO. * platformdirs malformed-XDG and duplicate-directory fixes for deployment portability. * CI action modernization, regenerated uv.lock, and expanded Dependabot coverage. Runtime dependencies cachebox: 6.1.2 → 6.2.2 fastapi: 0.139.2 → 0.141.1 python-fasthtml: 0.14.9 → 0.14.11 MonsterUI: 1.0.46 → 1.0.47 bokeh: 3.9.1 → 3.9.2 uvicorn: 0.51.0 → 0.52.4 (build(deps): bump uvicorn from 0.51.0 to 0.52.3 #1241, refreshed to latest patch) pandas: 3.0.3 → 3.0.5 platformdirs: 4.11.0 → 4.11.3 Development/test dependencies pandas-stubs: 3.0.3.260530 → 3.0.5.260730 types-PyYAML: 6.0.12.20260518 → 6.0.12.20260724 GitPython: 3.1.53 → 3.1.58 (security/fix releases) coverage: 7.15.2 → 7.15.4 pypdf: 6.14.2 → 6.16.1 (includes security fixes) Pre-commit/tooling ruff-pre-commit: v0.15.21 → v0.16.3 synchronize pandas-stubs, types-docutils, and types-PyYAML pins with pyproject.toml CI / repository dependencies Python 3.13.9 → 3.13.15 in CI, Docker, .env, and local Docker Make targets actions/checkout → v7 in pytest, pre-commit, CodeQL, and release workflows actions/setup-python → v7 in pytest, pre-commit, and release workflows actions/upload-artifact → v7 in pytest workflow actions/stale: v9.1.0 → v11.0.0 (SHA-pinned) regenerate uv.lock from the final dependency pins so locked/frozen installs match pyproject.toml Future update coverage Expand Dependabot from pip-only to also monitor: GitHub Actions Docker The existing open docutils 0.23 update (#1085) is intentionally excluded because it has separate compatibility/ignore handling and should remain isolated. docker-build.yml was audited and is already using the newer action generations, so no changes were needed there. --------- Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> |
||
 Bobby NoelteandGitHub
|
eb9e966de9 |
fix: move data management to async (#1015)
FAstAPI is an async framework. Data may be imported and exported, load and save, set and get asynchronously. Prevent interleaving data operations to corrupt the data. In the previous design sync and async data access was intermixed leading to data corruption. The basic data classes DataSequence and DataContainer and the derived classes like Provider and Measurement now are async. Data access is protected by several async locks. To support the async design of the data classes the database interface became async. The energy management is also adapted to the new async design. Optimization is still off-loaded to another thread, but the prepration for the optimization and the post optimization actions now follow the async design. Adapter operations are now also protected by async locks. Tests were adapted to the async design and new tests were created. Besides this major fix several other improvements and fixes are included in this PR. * fix: key_to_dict/list/array only regard data records with key value set. Before the exclusion of no value data records was only done if the dropna flag was set. * fix: test for visual result pdf generation Due to updates in the library the generated charts text was a little bit different. Adapt the test to create the comaprison pdf in the test data durectory and update the reference pdf. * chore: Remove MutableMapping from DataSequence and DataContainer. Mutable Mapping does not fit to the now async design. * chore: Add NoDB database backend This backend implements the full database backend interface but performs no actual persistence. It is intended for configurations where database persistence is disabled (`provider=None`). * chore: Improve measurement data import testing with real world scenarios. Added two new endpoints to support testing. * chore: Add mermaid to supported documentation tools * chore: Add documentation about async design * chore: Add documentation about generic data handling Covers the basics of measurement and prediction time series data handling. * chore: Add empty lines around markdown lists. * chore: sync pre-commit config to updated package versions Signed-off-by: Bobby Noelte <b0661n0e17e@gmail.com> |
||
|
|
6f28022ed0 |
fix: optimization fail after restart (#1007)
Bump Version / Bump Version Workflow (push) Has been cancelled
CodeQL Advanced / Analyze (actions) (push) Has been cancelled
CodeQL Advanced / Analyze (python) (push) Has been cancelled
docker-build / platform-excludes (push) Has been cancelled
pre-commit / pre-commit (push) Has been cancelled
Run Pytest on Pull Request / test (push) Has been cancelled
docker-build / build (push) Has been cancelled
docker-build / merge (push) Has been cancelled
Close stale pull requests/issues / Find Stale issues and PRs (push) Has been cancelled
Fix documentation for the loadforecast_power_w key. Fix documentation to explain the usage of import file/ JSON string to primarily initialise prediction data. Fix code scanning alert no. 6: URL redirection from remote source Enable to automatically save the configuration to the configuration file by default, which is a widespread user expectation. Make the genetic parameters non optional for better pydantic compliance. Update: - bump pytest to 9.0.3 - bump pillow to 12.2.0 - bump platformdirs to 4.9.6 - bump typespyyaml to 6.0.12.20260408 - bump tzfpy to 1.2.0 - bump pydantic to 2.13.0 - bump types-requests to 2.33.0.20260408 Signed-off-by: Bobby Noelte <b0661n0e17e@gmail.com> Co-authored-by: Normann <github@koldrack.com> |
||
|
Bobby NoelteandGitHub
|
2da724ecf4 |
chore: improve openmeteo test (#974)
Make openmeteo test robust against timing issues. Also update: - types-docutils==0.22.3.20260322 - pytest-cov==7.1.0 - ruff-pre-commit v0.15.7 - uvlock Signed-off-by: Bobby Noelte <b0661n0e17e@gmail.com> |
||
|
Bobby NoelteandGitHub
|
d446274129 |
fix: Adapt versioning scheme to Home Assistant and switch to uv (#896)
Bump Version / Bump Version Workflow (push) Has been cancelled
docker-build / platform-excludes (push) Has been cancelled
docker-build / build (push) Has been cancelled
docker-build / merge (push) Has been cancelled
pre-commit / pre-commit (push) Has been cancelled
Run Pytest on Pull Request / test (push) Has been cancelled
Home Assistant expects versioning always increases numbers. Add a date component to the development version to comply with this expectation. The scheme is now 0.0.0.dev<date><hash>. Use uv for creating and managing the virtual environment for developement. This enourmously speeds up dependency updates. For this change dependency requirements are now solely handled in pyproject.toml. requirements.tx and requirements-dev.txt are deleted. Signed-off-by: Bobby Noelte <b0661n0e17e@gmail.com> |
||
|
Bobby NoelteandGitHub
|
976a2c8405 |
chore: automate development version and release generation (#772)
Bump Version / Bump Version Workflow (push) Has been cancelled
docker-build / platform-excludes (push) Has been cancelled
docker-build / build (push) Has been cancelled
docker-build / merge (push) Has been cancelled
pre-commit / pre-commit (push) Has been cancelled
Run Pytest on Pull Request / test (push) Has been cancelled
This change introduces a GitHub Action to automate release creation, including proper tagging and automatic addition of a development marker to the version. A hash is also appended to development versions to make their state easier to distinguish. Tests and release documentation have been updated to reflect the revised release workflow. Several files now retrieve the current version dynamically. The test --full-run option has been rename to --finalize to make clear it is to do commit finalization testing. Signed-off-by: Bobby Noelte <b0661n0e17e@gmail.com> |
||
|
Bobby NoelteandGitHub
|
7bf9dd723e |
chore: improve doc generation and test (#762)
docker-build / platform-excludes (push) Has been cancelled
pre-commit / pre-commit (push) Has been cancelled
Run Pytest on Pull Request / test (push) Has been cancelled
docker-build / build (push) Has been cancelled
docker-build / merge (push) Has been cancelled
Close stale pull requests/issues / Find Stale issues and PRs (push) Has been cancelled
Improve documentation generation and add tests for documentation. Extend sphinx by todo directive. The configuration table is now split into several tables. The test is adapted accordingly. There is a new test that checks the docstrings to be compliant to the RST format as used by sphinx to create the documentation. We can not use Markdown in docstrings. The docstrings are adapted accordingly. An additional test checks that the documentation can be build with sphinx. This test takes very long is only enabled in full run (aka. ci) mode. Signed-off-by: Bobby Noelte <b0661n0e17e@gmail.com> |
||
|
Bobby NoelteandGitHub
|
b397b5d43e |
fix: automatic optimization (#596)
This fix implements the long term goal to have the EOS server run optimization (or
energy management) on regular intervals automatically. Thus clients can request
the current energy management plan at any time and it is updated on regular
intervals without interaction by the client.
This fix started out to "only" make automatic optimization (or energy management)
runs working. It turned out there are several endpoints that in some way
update predictions or run the optimization. To lock against such concurrent attempts
the code had to be refactored to allow control of execution. During refactoring it
became clear that some classes and files are named without a proper reference
to their usage. Thus not only refactoring but also renaming became necessary.
The names are still not the best, but I hope they are more intuitive.
The fix includes several bug fixes that are not directly related to the automatic optimization
but are necessary to keep EOS running properly to do the automatic optimization and
to test and document the changes.
This is a breaking change as the configuration structure changed once again and
the server API was also enhanced and streamlined. The server API that is used by
Andreas and Jörg in their videos has not changed.
* fix: automatic optimization
Allow optimization to automatically run on configured intervals gathering all
optimization parameters from configuration and predictions. The automatic run
can be configured to only run prediction updates skipping the optimization.
Extend documentaion to also cover automatic optimization. Lock automatic runs
against runs initiated by the /optimize or other endpoints. Provide new
endpoints to retrieve the energy management plan and the genetic solution
of the latest automatic optimization run. Offload energy management to thread
pool executor to keep the app more responsive during the CPU heavy optimization
run.
* fix: EOS servers recognize environment variables on startup
Force initialisation of EOS configuration on server startup to assure
all sources of EOS configuration are properly set up and read. Adapt
server tests and configuration tests to also test for environment
variable configuration.
* fix: Remove 0.0.0.0 to localhost translation under Windows
EOS imposed a 0.0.0.0 to localhost translation under Windows for
convenience. This caused some trouble in user configurations. Now, as the
default IP address configuration is 127.0.0.1, the user is responsible
for to set up the correct Windows compliant IP address.
* fix: allow names for hosts additional to IP addresses
* fix: access pydantic model fields by class
Access by instance is deprecated.
* fix: down sampling key_to_array
* fix: make cache clear endpoint clear all cache files
Make /v1/admin/cache/clear clear all cache files. Before it only cleared
expired cache files by default. Add new endpoint /v1/admin/clear-expired
to only clear expired cache files.
* fix: timezonefinder returns Europe/Paris instead of Europe/Berlin
timezonefinder 8.10 got more inaccurate for timezones in europe as there is
a common timezone. Use new package tzfpy instead which is still returning
Europe/Berlin if you are in Germany. tzfpy also claims to be faster than
timezonefinder.
* fix: provider settings configuration
Provider configuration used to be a union holding the settings for several
providers. Pydantic union handling does not always find the correct type
for a provider setting. This led to exceptions in specific configurations.
Now provider settings are explicit comfiguration items for each possible
provider. This is a breaking change as the configuration structure was
changed.
* fix: ClearOutside weather prediction irradiance calculation
Pvlib needs a pandas time index. Convert time index.
* fix: test config file priority
Do not use config_eos fixture as this fixture already creates a config file.
* fix: optimization sample request documentation
Provide all data in documentation of optimization sample request.
* fix: gitlint blocking pip dependency resolution
Replace gitlint by commitizen. Gitlint is not actively maintained anymore.
Gitlint dependencies blocked pip from dependency resolution.
* fix: sync pre-commit config to actual dependency requirements
.pre-commit-config.yaml was out of sync, also requirements-dev.txt.
* fix: missing babel in requirements.txt
Add babel to requirements.txt
* feat: setup default device configuration for automatic optimization
In case the parameters for automatic optimization are not fully defined a
default configuration is setup to allow the automatic energy management
run. The default configuration may help the user to correctly define
the device configuration.
* feat: allow configuration of genetic algorithm parameters
The genetic algorithm parameters for number of individuals, number of
generations, the seed and penalty function parameters are now avaliable
as configuration options.
* feat: allow configuration of home appliance time windows
The time windows a home appliance is allowed to run are now configurable
by the configuration (for /v1 API) and also by the home appliance parameters
(for the classic /optimize API). If there is no such configuration the
time window defaults to optimization hours, which was the standard before
the change. Documentation on how to configure time windows is added.
* feat: standardize mesaurement keys for battery/ ev SoC measurements
The standardized measurement keys to report battery SoC to the device
simulations can now be retrieved from the device configuration as a
read-only config option.
* feat: feed in tariff prediction
Add feed in tarif predictions needed for automatic optimization. The feed in
tariff can be retrieved as fixed feed in tarif or can be imported. Also add
tests for the different feed in tariff providers. Extend documentation to
cover the feed in tariff providers.
* feat: add energy management plan based on S2 standard instructions
EOS can generate an energy management plan as a list of simple instructions.
May be retrieved by the /v1/energy-management/plan endpoint. The instructions
loosely follow the S2 energy management standard.
* feat: make measurement keys configurable by EOS configuration.
The fixed measurement keys are replaced by configurable measurement keys.
* feat: make pendulum DateTime, Date, Duration types usable for pydantic models
Use pydantic_extra_types.pendulum_dt to get pydantic pendulum types. Types are
added to the datetimeutil utility. Remove custom made pendulum adaptations
from EOS pydantic module. Make EOS modules use the pydantic pendulum types
managed by the datetimeutil module instead of the core pendulum types.
* feat: Add Time, TimeWindow, TimeWindowSequence and to_time to datetimeutil.
The time windows are are added to support home appliance time window
configuration. All time classes are also pydantic models. Time is the base
class for time definition derived from pendulum.Time.
* feat: Extend DataRecord by configurable field like data.
Configurable field like data was added to support the configuration of
measurement records.
* feat: Add additional information to health information
Version information is added to the health endpoints of eos and eosDash.
The start time of the last optimization and the latest run time of the energy
management is added to the EOS health information.
* feat: add pydantic merge model tests
* feat: add plan tab to EOSdash
The plan tab displays the current energy management instructions.
* feat: add predictions tab to EOSdash
The predictions tab displays the current predictions.
* feat: add cache management to EOSdash admin tab
The admin tab is extended by a section for cache management. It allows to
clear the cache.
* feat: add about tab to EOSdash
The about tab resembles the former hello tab and provides extra information.
* feat: Adapt changelog and prepare for release management
Release management using commitizen is added. The changelog file is adapted and
teh changelog and a description for release management is added in the
documentation.
* feat(doc): Improve install and devlopment documentation
Provide a more concise installation description in Readme.md and add extra
installation page and development page to documentation.
* chore: Use memory cache for interpolation instead of dict in inverter
Decorate calculate_self_consumption() with @cachemethod_until_update to cache
results in memory during an energy management/ optimization run. Replacement
of dict type caching in inverter is now possible because all optimization
runs are properly locked and the memory cache CacheUntilUpdateStore is properly
cleared at the start of any energy management/ optimization operation.
* chore: refactor genetic
Refactor the genetic algorithm modules for enhanced module structure and better
readability. Removed unnecessary and overcomplex devices singleton. Also
split devices configuration from genetic algorithm parameters to allow further
development independently from genetic algorithm parameter format. Move
charge rates configuration for electric vehicles from optimization to devices
configuration to allow to have different charge rates for different cars in
the future.
* chore: Rename memory cache to CacheEnergyManagementStore
The name better resembles the task of the cache to chache function and method
results for an energy management run. Also the decorator functions are renamed
accordingly: cachemethod_energy_management, cache_energy_management
* chore: use class properties for config/ems/prediction mixin classes
* chore: skip debug logs from mathplotlib
Mathplotlib is very noisy in debug mode.
* chore: automatically sync bokeh js to bokeh python package
bokeh was updated to 3.8.0, make JS CDN automatically follow the package version.
* chore: rename hello.py to about.py
Make hello.py the adapted EOSdash about page.
* chore: remove demo page from EOSdash
As no the plan and prediction pages are working without configuration, the demo
page is no longer necessary
* chore: split test_server.py for system test
Split test_server.py to create explicit test_system.py for system tests.
* chore: move doc utils to generate_config_md.py
The doc utils are only used in scripts/generate_config_md.py. Move it there to
attribute for strong cohesion.
* chore: improve pydantic merge model documentation
* chore: remove pendulum warning from readme
* chore: remove GitHub discussions from contributing documentation
Github discussions is to be replaced by Akkudoktor.net.
* chore(release): bump version to 0.1.0+dev for development
* build(deps): bump fastapi[standard] from 0.115.14 to 0.117.1
bump fastapi and make coverage version (for pytest-cov) explicit to avoid pip break.
* build(deps): bump uvicorn from 0.36.0 to 0.37.0
BREAKING CHANGE: EOS configuration changed. V1 API changed.
- The available_charge_rates_percent configuration is removed from optimization.
Use the new charge_rate configuration for the electric vehicle
- Optimization configuration parameter hours renamed to horizon_hours
- Device configuration now has to provide the number of devices and device
properties per device.
- Specific prediction provider configuration to be provided by explicit
configuration item (no union for all providers).
- Measurement keys to be provided as a list.
- New feed in tariff providers have to be configured.
- /v1/measurement/loadxxx endpoints are removed. Use generic mesaurement endpoints.
- /v1/admin/cache/clear now clears all cache files. Use
/v1/admin/cache/clear-expired to only clear all expired cache files.
Signed-off-by: Bobby Noelte <b0661n0e17e@gmail.com>
|
||
|
Bobby NoelteandGitHub
|
9626bfb32b |
ci: lint commit messages (#510)
Lint commit messages using gitlint in pre-commit. Gitlint enforces rules that are configured by .gitlint. The checks enforce the [`Conventional Commits`](https://www.conventionalcommits.org) commit message style. Signed-off-by: Bobby Noelte <b0661n0e17e@gmail.com> |
||
|
NormannandGitHub
|
a7d58eed9a |
pre-commit update and ignore changes (#461)
* pre-commit autoupdate * type: ignore changes * [attr-defined,unused-ignore] usage |
||
|
Bobby Noelte
|
1020a46435 |
Add Markdown linter
Add Markdown linter (pymarkdown) to pre-commit. Adapt current markdown files to fulfill linter rules. Signed-off-by: Bobby Noelte <b0661n0e17e@gmail.com> |
||
 Dominique LasserreandGitHub
|
6db7e96bc1 |
pre-commit: Fix mypy hook (#233)
* Install missing deps. In the future we probably just want to reuse the dev environment (e.g. poetry shell) to have same result for hook and explicit execution in venv. |
||
|
Dominique LasserreandGitHub
|
1163ddb4ac |
Mypy (#217)
* Mypy: Initial support * Add to pre-commit (currently installs own deps, could maybe changed to poetry venv in the future to reuse environment and don't need duplicated types deps). * Add type hints. * Mypy: Add missing annotations |
||
 Jürgen EckelandGitHub
|
2a163569bc |
renamed haushaltsgeräte to home appliance (#196)
* * rename Haushaltsgeraete to home appliance * renamed strafe to penalty (optimization problem) Signed-off-by: Jürgen Eckel <juergen.eckel@gmail.com> * removed penalty renaming Signed-off-by: Jürgen Eckel <juergen.eckel@gmail.com> * renamed one variable Signed-off-by: Jürgen Eckel <juergen.eckel@gmail.com> * * renamed variable names and methods of the home appliance class * renamed missed method names * fixed renamed variable * renamed object * adjusted to latest repo changes * renamed file to class_home_applianc.py * renamed method --------- Signed-off-by: Jürgen Eckel <juergen.eckel@gmail.com> Signed-off-by: Jürgen Eckel <juergen@riddleandcode.com> |
||
 
|
de9fe2ddc9 |
Add settings and extension recommendations (#139)
- Added recommended settings for the Pylance and Pyright extensions - Enabled pytest and debugging support in configuration - Reformatted code with Prettier --------- Co-authored-by: Chris <git@nootch.de> |
||
  Michael OsthegeandAndreas
|
bbaaacaca0 | Add pre-commit configuration |