New users have to find the settings EOS needs among all configuration values
on the Config page, without a hint which ones matter or what is still missing.
The new Setup page takes the user through the configuration step by step.
Each step explains a few settings, gives an example, reports what is still to
do and shows the configuration cards of exactly these settings. The cards are
the ones of the Config page; the code that creates a card and sends an update
moved out of Configuration() into config_card() and config_update().
The assistant keeps no state: the step travels with the requests and the
status is derived from the EOS configuration, which is read and written
through the EOS server only.
First steps: location, horizons and energy management, save the configuration.
Signed-off-by: SchaafAlexander <86607129+SchaafAlexander@users.noreply.github.com>
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Assisted-by: Claude Code (claude-fable-5-2)
The isolated pre-commit mypy hook previously omitted runtime type information that
make mypy used, hiding errors involving dependencies such as Pydantic and Pendulum.
Makefile, pre-commit and CI now run the same full-project typing policy in the
development environment defined by uv.lock.
- Use uv run --locked --exact --extra dev and the same mypy arguments for Makefile
and the local hook. Check all of src and tests, including on configuration-only
changes.
- Pin Python 3.13 for local development and the pre-commit CI job, and install the
locked pre-commit version in CI.
- Disable incremental analysis because existing Pendulum cache state changes mypy 2.3.1
diagnostics. Document the policy, the performance tradeoff and the existing typing debt.
- Add a regression test that exercises Makefile, the hook and the CI command in a
temporary project, accepting valid dependency types and detecting deliberate
Pydantic/Pendulum assignment errors.
Resolve the newly detected mypy diagnostics.
- Enable the numpydantic and Pydantic mypy plugins, retaining strict Pydantic
constructor typing with init_typed = true. Validate raw/coercible payloads through model_validate.
- Propagate concrete record, provider and time-window types through generic collections,
factories and lookup methods. Preserve runtime field inspection and generated time-window
documentation.
- Align Pendulum annotations with actual factory/arithmetic results while retaining Pydantic
validation adapters at runtime. Correct optional values, array boundaries, REST handlers
and plotting interfaces.
- Add pinned scipy-stubs and types-psutil, update uv.lock, and supply the plugins' dependencies.
- Add runtime regression coverage for validated path defaults, normalized time-series metadata,
generic field inspection, invalid timestamps and unsupported provider imports.
Runtime and compatibility details:
- Validate path defaults as Path objects while retaining raw string defaults needed by
migration serialization with exclude_defaults.
- Normalize feed-in tariff lists and default charge rates to NumPy arrays; reject missing
timestamps/uninitialized values explicitly. Importing into a provider without import support
returns HTTP 400.
- Public JSON schemas and OpenAPI structure match main (excluding the generated version).
Signed-off-by: dr-dimitry
Signed-off-by: dr-dimitry
Signed-off-by: Bobby Noelte <b0661n0e17e@gmail.com>
Co-authored-by: dr-dimitri <87113560+dr-dimitri@users.noreply.github.com>
Co-authored-by: Normann <github@koldrack.com>
Add new provider class for electricity fees providers.
Add the generic providers:
- ElecFeeFixed
- ElecFeeImport
The providers provide predictions for:
- elecfee_consumption_amt_wh:
Total fixed fee for consumed energy per Wh [amount/Wh]. This is the accumulation of all
fixed per-Wh fees payable on "consumed energy - such as network charge, concession fee,
and electricity charge - into a single amount.
- elecfee_consumption_percent_amt:
Total fixed surcharge on consumed energy, given as a percentage of the monetary amount
already charged for that energy [%]. This is the accumulation of all percentage-based
surcharges payable on top of the consumed-energy fee - such as VAT - into a single
percentage. This is a percentage of the fee amount, not a per-Wh rate.
- elecfee_feedin_amt_wh:
Total fixed deduction from feed-in energy per Wh [amount/Wh]. This is the accumulation of
all fixed per-Wh charges deducted from feed-in energy - such as metering fees or
grid-operator handling "charges - into a single amount. Applied after the percentage-based
deduction, i.e. it reduces the price by a flat amount per Wh rather than by a share of the
raw price.
- elecfee_feedin_percent_amt:
Total percentage deducted from the raw feed-in price (spot price) [%]. This is the
accumulation of all percentage-based deductions payable on the feed-in tariff - such as a
marketing or balancing fee retained by the aggregator - into a single percentage. It is
applied as `raw_price * (100 - percent) / 100`, i.e. it scales down the raw price rather
than adding a surcharge to it.
A new _apply_fee() method is added to the base class for ElecPrice and FeedInTariff to be used to
add the fees in a consistent way. Fees are taken from the active ElecFee provider and applied
to the raw prices given to the _apply_fee() method.
The optional application of fees is added to:
- ElecPriceAkkudoktor
- ElecPriceFixed
- ElecPriceEnergyCharts
- ElecPriceSMARD
- FeedInTariffEnergyCharts
- FeedInTariffFixed
- FeedInTariffSMARD
The import providers ElecPriceImport and FeedInTariffImport do not apply fees by intentention.
The following providers currently do not handle fees defined by ElecFee:
- ElecPriceTibber
- FeedInTariffAkkudoktor
- FeedInTariffDvhubOnline
- FeedInTariffTibber
The tests for this feature are either added or existing tests are extended.
The documentation was extended for the electricity fee provider settings.
Besides this feature further improvements are added:
* feat: add SMARD quarter-hour electricty price and feed-in tariff provider
* feat: to_series method for TimeWindows and ValueTimeWindows
Additional to to_array the time window sequence can now also produce a pandas series.
Test have been extended to cover the series generation.
* feat: use time windows in fixed feedin tariff provider
Feedin tariff can now be configured by time windows - not a single value.
* feat: EOSdash select for PVLib inverters and modules
Provide PVLib inverter and module names in config selection.
* feat: EOSdash lazy select for big option sets
Add a new form for lazy selection of big option sets. Filtering and
generation of the option set is done server-side.
* fix: use raw data for ETS/ median prediction
Use to raw time series data for ETS/ median prediction to avoid interference
by e.g. dynamic grid charges.
* fix: EOSdash config drops by type only on details resolve
Drop configuration by type and path. Prevents dropping of configuration items
with same type and level but different path.
* fix: EOSdash configuration section closes on update
Open section if searching or if last update touched this category — including
updates on deeply nested sub-fields.
* chore: make elecfeefixed, elecpricefixed and feedintarifffixed warn about no windows and default to 0
Missining configuration creates default 0 value and a warning instead of an exception.
* fix: test setup for providers
Reset db state on each test run.
* chore: improve config option naming for elecpricefixed.
* chore: adapt elecpricefixed test to changed time_windows naming
* chore: factorized common price provider helpers to priceabc.py
Factorized common price provider helpers to priceabc.py. Add tests for these helpers.
Reduce/ change testing of elecpriceabc.py and feedintariffabc.py to cover
only specifics. Rest of testing is already covered by test_priceabc.py.
* chore: update version
Signed-off-by: Bobby Noelte <b0661n0e17e@gmail.com>
Add a PV forecast provider that calculates the forecast using a PVLib system model
and weather forecast from the EOS weather forecast provider.
Additional module and inverter models can be easily added as the database
is build from PVLib and SAM databases and a bundled csv file.
The module model and inververt model names are provided by new endpoints
to be used in configuration.
The provider is based on the fantastic work of EMHASS. See
https://github.com/davidusb-geek/emhass/blob/master/src/emhass/forecast.py
A short description of the provider is added to the documentation.
Besides the new features there are the fixes and improvements:
* feat: improve EOSdash config page
* fix: kex_to_series for start_datetime
Make key_to_series always start the series at start_datetime.
* fix: default provider for GENETIC and GENETIC0 optimization
To make the default less dependent on internet servers (with API changes and
availability issues) the default for PVForecast is set to PVForecastPVLib
and for ElecPrice to ElecPriceFixed. The default weather provider is changed
to OpenMeteo.
* fix: EOSdash display resampled prediction values
Make EOSdash display resampled prediction values where resampling fits to
the prediction value type. Use bar width that fits to 15 minutes value samples.
* chore: add a UI hints system to EOSdash
The UI hints system eases the definition of forms for configuration items.
There are also forms for items in maps and lists. These forms allow to add and delete
items to/ from maps and lists. The forms ensure that all required fields of newly
added items are filled.
* chore: Create an enum for valid optimization algorithms
* chore. Make config also provide the available energy management modes.
Used for configuration hints.
* chore: Randomize default device id in configuration
Signed-off-by: Bobby Noelte <b0661n0e17e@gmail.com>
Bare `except:` catches SystemExit and KeyboardInterrupt, which can:
- Mask Ctrl+C handling during long optimization runs
- Hide critical system signals
- Make debugging harder
Replacement by `except Exception:` preserves the same error handling while respecting
system-level exceptions.
Co-authored-by: Milo @KeloYuan
Signed-off-by: Bobby Noelte <b0661n0e17e@gmail.com>
When running as Home Assistant add-on the only viable usage is running with
cyclic optimization. Make this the default to als propvide a better experience
for first time users. The optimization will start with demo data, which also
helps to configure Akkudoktor-EOS to the personal usage.
The disabling of the automatic energy management is now an explicit mode
"DISABLED".
Signed-off-by: Bobby Noelte <b0661n0e17e@gmail.com>
Add a fixed electricity prediction that supports prices per time window.
The time windows may flexible be defined by day or date.
The prediction documentation is updated to also cover the ElecPriceFixed
provider.
The feature includes several changes that are not directly related to the
electricity price prediction implementation but are necessary to keep
EOS running properly and to test and document the changes.
* feat: add value time windows
Add time windows with an associated float value.
* feat: harden eos measurements endpoints error detection and reporting
Cover more errors that may be raised during endpoint access. Report the
errors including trace information to ease debugging.
* feat: extend server configuration to cover all arguments
Make the argument controlled options also available in server configuration.
* fix: eos config configuration by cli arguments
Move the command line argument handling to config eos so that it is
excuted whenever eos config is rebuild or reset.
* chore: extend measurement endpoint system test
* chore: refactor time windows
Move time windows to configabc as they are only used in configurations.
Also move all tests to test_configabc.
* chore: provide config update errors in eosdash with summarized error text
If there is an update error provide the error text as a summary. On click
provide the full error text.
* chore: force eosdash ip address and port in makefile dev run
Ensure eosdash ip address and port are correctly set for development runs.
Signed-off-by: Bobby Noelte <b0661n0e17e@gmail.com>
Add documentation for home assistant adapter.
Make adapter correctly set the measurement keys for PV production.
Add adapter configuration for grid import and export measurements.
Signed-off-by: Bobby Noelte <b0661n0e17e@gmail.com>
Adapters for Home Assistant and NodeRED integration are added.
Akkudoktor-EOS can now be run as Home Assistant add-on and standalone.
As Home Assistant add-on EOS uses ingress to fully integrate the EOSdash dashboard
in Home Assistant.
The fix includes several bug fixes that are not directly related to the adapter
implementation but are necessary to keep EOS running properly and to test and
document the changes.
* fix: development version scheme
The development versioning scheme is adaptet to fit to docker and
home assistant expectations. The new scheme is x.y.z and x.y.z.dev<hash>.
Hash is only digits as expected by home assistant. Development version
is appended by .dev as expected by docker.
* fix: use mean value in interval on resampling for array
When downsampling data use the mean value of all values within the new
sampling interval.
* fix: default battery ev soc and appliance wh
Make the genetic simulation return default values for the
battery SoC, electric vehicle SoC and appliance load if these
assets are not used.
* fix: import json string
Strip outer quotes from JSON strings on import to be compliant to json.loads()
expectation.
* fix: default interval definition for import data
Default interval must be defined in lowercase human definition to
be accepted by pendulum.
* fix: clearoutside schema change
* feat: add adapters for integrations
Adapters for Home Assistant and NodeRED integration are added.
Akkudoktor-EOS can now be run as Home Assistant add-on and standalone.
As Home Assistant add-on EOS uses ingress to fully integrate the EOSdash dashboard
in Home Assistant.
* feat: allow eos to be started with root permissions and drop priviledges
Home assistant starts all add-ons with root permissions. Eos now drops
root permissions if an applicable user is defined by paramter --run_as_user.
The docker image defines the user eos to be used.
* feat: make eos supervise and monitor EOSdash
Eos now not only starts EOSdash but also monitors EOSdash during runtime
and restarts EOSdash on fault. EOSdash logging is captured by EOS
and forwarded to the EOS log to provide better visibility.
* feat: add duration to string conversion
Make to_duration to also return the duration as string on request.
* chore: Use info logging to report missing optimization parameters
In parameter preparation for automatic optimization an error was logged for missing paramters.
Log is now down using the info level.
* chore: make EOSdash use the EOS data directory for file import/ export
EOSdash use the EOS data directory for file import/ export by default.
This allows to use the configuration import/ export function also
within docker images.
* chore: improve EOSdash config tab display
Improve display of JSON code and add more forms for config value update.
* chore: make docker image file system layout similar to home assistant
Only use /data directory for persistent data. This is handled as a
docker volume. The /data volume is mapped to ~/.local/share/net.akkudoktor.eos
if using docker compose.
* chore: add home assistant add-on development environment
Add VSCode devcontainer and task definition for home assistant add-on
development.
* chore: improve documentation
Pydantic deprecates using extra keyword arguments on Field.
Used json_schema_extra instead.
Deprecated in Pydantic V2.0 to be removed in V3.0.
Signed-off-by: Bobby Noelte <b0661n0e17e@gmail.com>
* Fix logging configuration issues that made logging stop operation. Switch to Loguru
logging (from Python logging). Enable console and file logging with different log levels.
Add logging documentation.
* Fix logging configuration and EOS configuration out of sync. Added tracking support
for nested value updates of Pydantic models. This used to update the logging configuration
when the EOS configurationm for logging is changed. Should keep logging config and EOS
config in sync as long as all changes to the EOS logging configuration are done by
set_nested_value(), which is the case for the REST API.
* Fix energy management task looping endlessly after the second update when trying to update
the last_update datetime.
* Fix get_nested_value() to correctly take values from the dicts in a Pydantic model instance.
* Fix usage of model classes instead of model instances in nested value access when evaluation
the value type that is associated to each key.
* Fix illegal json format in prediction documentation for PVForecastAkkudoktor provider.
* Fix documentation qirks and add EOS Connect to integrations.
* Support deprecated fields in configuration in documentation generation and EOSdash.
* Enhance EOSdash demo to show BrightSky humidity data (that is often missing)
* Update documentation reference to German EOS installation videos.
Signed-off-by: Bobby Noelte <b0661n0e17e@gmail.com>
Added bandit checks to continuous integration.
Updated sources to pass bandit checks:
- replaced asserts
- added timeouts to requests
- added checks for process command execution
- changed to 127.0.0.1 as default IP address for EOS and EOSdash for security reasons
Added a rudimentary check for outdated config files.
BREAKING CHANGE: Default IP address for EOS and EOSdash changed to 127.0.0.1
Signed-off-by: Bobby Noelte <b0661n0e17e@gmail.com>
* Allow to configure planes and configuration values of planes separatedly.
Make single configuration values for planes explicitly available for configuration.
Still allows to also configure a plane by a whole plane value struct.
* Enhance admin page by file import and export of the EOS configuration
The actual EOS configuration can now be exported to the EOSdash server.
From there it can be also imported. For security reasons only import and export
from/ to a predefined directory on the EOSdash server is possible.
* Improve handling of nested value pathes in pydantic models.
Added separate methods for nested path access (get_nested_value, set_nested_value).
On value setting the missing fields along the nested path are now added automatically
and initialized with default values. Nested path access was before restricted to the
EOS configuration and is now part of the pydantic base model.
* Makefile
Add new target to run rests as CI does on Github. Improve target docs.
* Datetimeutil tests
Prolong acceptable time difference for comparison of approximately equal times in tests.
Signed-off-by: Bobby Noelte <b0661n0e17e@gmail.com>
Improve config page to edit actual configuration used by EOS.
Add admin page to save the actual configuration to the configuration file.
Signed-off-by: Bobby Noelte <b0661n0e17e@gmail.com>
Make EOSdash use UI components from MonsterUI to ease further development.
- Add a first menu with some dummy pages and the configuration page.
- Make the configuration scrollable.
- Add markdown component that uses markdown-it-py (same as used by
the myth-parser for documentation generation).
- Add bokeh (https://docs.bokeh.org/) component for charts
- Added several prediction charts to demo
- Add a footer that displays connection status with EOS server
- Add logo and favicon
Update EOS server:
- Move error message generation to extra module
- Use redirect instead of proxy
Signed-off-by: Bobby Noelte <b0661n0e17e@gmail.com>
