* Fix logging configuration issues that made logging stop operation. Switch to Loguru
logging (from Python logging). Enable console and file logging with different log levels.
Add logging documentation.
* Fix logging configuration and EOS configuration out of sync. Added tracking support
for nested value updates of Pydantic models. This used to update the logging configuration
when the EOS configurationm for logging is changed. Should keep logging config and EOS
config in sync as long as all changes to the EOS logging configuration are done by
set_nested_value(), which is the case for the REST API.
* Fix energy management task looping endlessly after the second update when trying to update
the last_update datetime.
* Fix get_nested_value() to correctly take values from the dicts in a Pydantic model instance.
* Fix usage of model classes instead of model instances in nested value access when evaluation
the value type that is associated to each key.
* Fix illegal json format in prediction documentation for PVForecastAkkudoktor provider.
* Fix documentation qirks and add EOS Connect to integrations.
* Support deprecated fields in configuration in documentation generation and EOSdash.
* Enhance EOSdash demo to show BrightSky humidity data (that is often missing)
* Update documentation reference to German EOS installation videos.
Signed-off-by: Bobby Noelte <b0661n0e17e@gmail.com>
* visualize: fix timestamps on diagrams
* set start time in all graphs to the same beginning hour
---------
Co-authored-by: Normann <github@koldrack.com>
* Move the caching module to core.
Add an in memory cache that for caching function and method results
during an energy management run (optimization run). Two decorators
are provided for methods and functions.
* Improve the file cache store by load and save functions.
Make EOS load the cache file store on startup and save it on shutdown.
Add a cyclic task that cleans the cache file store from outdated cache files.
* Improve startup of EOSdash by EOS
Make EOS starting EOSdash adhere to path configuration given in EOS.
The whole environment from EOS is now passed to EOSdash.
Should also prevent test errors due to unwanted/ wrong config file creation.
Both servers now provide a health endpoint that can be used to detect whether
the server is running. This is also used for testing now.
* Improve startup of EOS
EOS now has got an energy management task that runs shortly after startup.
It tries to execute energy management runs with predictions newly fetched
or initialized from cached data on first run.
* Improve shutdown of EOS
EOS has now a shutdown task that shuts EOS down gracefully with some
time delay to allow REST API requests for shutdwon or restart to be fully serviced.
* Improve EMS
Add energy management task for repeated energy management controlled by
startup delay and interval configuration parameters.
Translate EnergieManagementSystem to english EnergyManagement.
* Add administration endpoints
- endpoints to control caching from REST API.
- endpoints to control server restart (will not work on Windows) and shutdown from REST API
* Improve doc generation
Use "\n" linenend convention also on Windows when generating doc files.
Replace Windows specific 127.0.0.1 address by standard 0.0.0.0.
* Improve test support (to be able to test caching)
- Add system test option to pytest for running tests with "real" resources
- Add new test fixture to start server for test class and test function
- Make kill signal adapt to Windows/ Linux
- Use consistently "\n" for lineends when writing text files in doc test
- Fix test_logging under Windows
- Fix conftest config_default_dirs test fixture under Windows
From @Lasall
* Improve Windows support
- Use 127.0.0.1 as default config host (model defaults) and
addionally redirect 0.0.0.0 to localhost on Windows (because default
config file still has 0.0.0.0).
- Update install/startup instructions as package installation is
required atm.
Signed-off-by: Bobby Noelte <b0661n0e17e@gmail.com>
* Pre-fetch data
* maintanance and extend tests
* comment clean up
* nansum usage (to be save)
* Feature/config nested (#421)
* Nested config, devices registry
* All config now nested.
- Use default config from model field default values. If providers
should be enabled by default, non-empty default config file could
be provided again.
- Environment variable support with EOS_ prefix and __ between levels,
e.g. EOS_SERVER__EOS_SERVER_PORT=8503 where all values are case
insensitive.
For more information see:
https://docs.pydantic.dev/latest/concepts/pydantic_settings/#parsing-environment-variable-values
- Use devices as registry for configured devices. DeviceBase as base
class with for now just initializion support (in the future expand
to operations during optimization).
- Strip down ConfigEOS to the only configuration instance. Reload
from file or reset to defaults is possible.
* Fix multi-initialization of derived SingletonMixin classes.
* Documentation: Support nested config
* Add examples to pydantic models.
* EOSdash: Support nested types
* Rename settings variables (remove prefixes)
* Fix API endpoint
* Fix EOSdash startup (docker)
* Docker: Copy the same directory structure (src/) to support the
lifespan startup of EOSdash.
Use EOS_SERVER_EOSDASH_SESSKEY environment variable to provide
EOSdash with session key.
* PR review
* PVForecast: planes as nested config (list)
* Update manual documentation for nested config.
* Add config_file_path, config_folder_path back to general
(ConfigCommonSettings). Overwrite in docs generation.
* Config: Move lat/long/timezone from prediction to general
* Docs: Add global example documentation.
* merge_models: Use deecopy to not change input data.
* EOSdash: Sort config by name
* Review comments
* Feature/config nested dependabot req. (#415)
* Bump numpydantic from 1.6.4 to 1.6.7 (#413)
Bumps [numpydantic](https://github.com/p2p-ld/numpydantic) from 1.6.4 to 1.6.7.
- [Release notes](https://github.com/p2p-ld/numpydantic/releases)
- [Changelog](https://github.com/p2p-ld/numpydantic/blob/main/docs/changelog.md)
- [Commits](https://github.com/p2p-ld/numpydantic/compare/v1.6.4...v1.6.7)
---
updated-dependencies:
- dependency-name: numpydantic
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* Bump timezonefinder from 6.5.7 to 6.5.8 (#414)
Bumps [timezonefinder](https://github.com/jannikmi/timezonefinder) from 6.5.7 to 6.5.8.
- [Release notes](https://github.com/jannikmi/timezonefinder/releases)
- [Changelog](https://github.com/jannikmi/timezonefinder/blob/master/CHANGELOG.rst)
- [Commits](https://github.com/jannikmi/timezonefinder/compare/6.5.7...6.5.8)
---
updated-dependencies:
- dependency-name: timezonefinder
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* Bump pydantic from 2.10.5 to 2.10.6 (#412)
Bumps [pydantic](https://github.com/pydantic/pydantic) from 2.10.5 to 2.10.6.
- [Release notes](https://github.com/pydantic/pydantic/releases)
- [Changelog](https://github.com/pydantic/pydantic/blob/main/HISTORY.md)
- [Commits](https://github.com/pydantic/pydantic/compare/v2.10.5...v2.10.6)
---
updated-dependencies:
- dependency-name: pydantic
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* Bump fastapi[standard] from 0.115.6 to 0.115.7 (#411)
Bumps [fastapi[standard]](https://github.com/fastapi/fastapi) from 0.115.6 to 0.115.7.
- [Release notes](https://github.com/fastapi/fastapi/releases)
- [Commits](https://github.com/fastapi/fastapi/compare/0.115.6...0.115.7)
---
updated-dependencies:
- dependency-name: fastapi[standard]
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
---------
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* Readme: Add hint for interfering ports on Synology Closes#408 (#419)
* Pics or it didn't happen (#402)
* inverter added
* png creation
* save svg into cache folder
* mypy
* comment
---------
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: Dominique Lasserre <lasserre.d@gmail.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* inverter, prediction.hours
* self.config.general.data_cache_path
---------
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: Dominique Lasserre <lasserre.d@gmail.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* All config now nested.
- Use default config from model field default values. If providers
should be enabled by default, non-empty default config file could
be provided again.
- Environment variable support with EOS_ prefix and __ between levels,
e.g. EOS_SERVER__EOS_SERVER_PORT=8503 where all values are case
insensitive.
For more information see:
https://docs.pydantic.dev/latest/concepts/pydantic_settings/#parsing-environment-variable-values
- Use devices as registry for configured devices. DeviceBase as base
class with for now just initializion support (in the future expand
to operations during optimization).
- Strip down ConfigEOS to the only configuration instance. Reload
from file or reset to defaults is possible.
* Fix multi-initialization of derived SingletonMixin classes.
* footer with date and version
* ruff
* replace toml module with build in
* using re to extract the string
* optimize re usage
* use of use pendulum in Akkudoktor-EOS
* create_line_chart_date function added
* replace datetime with pendulum
* align ax2 with ax1 and 0 first point
* dynamic ticks
* all charts with dates
* style changes
* mypy fixes
* fix test
* fixed current time
measurement:
- Add new measurement class to hold real world measurements.
- Handles load meter readings, grid import and export meter readings.
- Aggregates load meter readings aka. measurements to total load.
- Can import measurements from files, pandas datetime series,
pandas datetime dataframes, simple daetime arrays and
programmatically.
- Maybe expanded to other measurement values.
- Should be used for load prediction adaptions by real world
measurements.
core/coreabc:
- Add mixin class to access measurements
core/pydantic:
- Add pydantic models for pandas datetime series and dataframes.
- Add pydantic models for simple datetime array
core/dataabc:
- Provide DataImport mixin class for generic import handling.
Imports from JSON string and files. Imports from pandas datetime dataframes
and simple datetime arrays. Signature of import method changed to
allow import datetimes to be given programmatically and by data content.
- Use pydantic models for datetime series, dataframes, arrays
- Validate generic imports by pydantic models
- Provide new attributes min_datetime and max_datetime for DataSequence.
- Add parameter dropna to drop NAN/ None values when creating lists, pandas series
or numpy array from DataSequence.
config/config:
- Add common settings for the measurement module.
predictions/elecpriceakkudoktor:
- Use mean values of last 7 days to fill prediction values not provided by
akkudoktor.net (only provides 24 values).
prediction/loadabc:
- Extend the generic prediction keys by 'load_total_adjusted' for load predictions
that adjust the predicted total load by measured load values.
prediction/loadakkudoktor:
- Extend the Akkudoktor load prediction by load adjustment using measured load
values.
prediction/load_aggregator:
- Module removed. Load aggregation is now handled by the measurement module.
prediction/load_corrector:
- Module removed. Load correction (aka. adjustment of load prediction by
measured load energy) is handled by the LoadAkkudoktor prediction and
the generic 'load_mean_adjusted' prediction key.
prediction/load_forecast:
- Module removed. Functionality now completely handled by the LoadAkkudoktor
prediction.
utils/cacheutil:
- Use pydantic.
- Fix potential bug in ttl (time to live) duration handling.
utils/datetimeutil:
- Added missing handling of pendulum.DateTime and pendulum.Duration instances
as input. Handled before as datetime.datetime and datetime.timedelta.
utils/visualize:
- Move main to generate_example_report() for better testing support.
server/server:
- Added new configuration option server_fastapi_startup_server_fasthtml
to make startup of FastHTML server by FastAPI server conditional.
server/fastapi_server:
- Add APIs for measurements
- Improve APIs to provide or take pandas datetime series and
datetime dataframes controlled by pydantic model.
- Improve APIs to provide or take simple datetime data arrays
controlled by pydantic model.
- Move fastAPI server API to v1 for new APIs.
- Update pre v1 endpoints to use new prediction and measurement capabilities.
- Only start FastHTML server if 'server_fastapi_startup_server_fasthtml'
config option is set.
tests:
- Adapt import tests to changed import method signature
- Adapt server test to use the v1 API
- Extend the dataabc test to test for array generation from data
with several data interval scenarios.
- Extend the datetimeutil test to also test for correct handling
of to_datetime() providing now().
- Adapt LoadAkkudoktor test for new adjustment calculation.
- Adapt visualization test to use example report function instead of visualize.py
run as process.
- Removed test_load_aggregator. Functionality is now tested in test_measurement.
- Added tests for measurement module
docs:
- Remove sphinxcontrib-openapi as it prevents build of documentation.
"site-packages/sphinxcontrib/openapi/openapi31.py", line 305, in _get_type_from_schema
for t in schema["anyOf"]: KeyError: 'anyOf'"
Signed-off-by: Bobby Noelte <b0661n0e17e@gmail.com>
