EOS/CHANGELOG.md

Changelog

All notable changes to the akkudoktoreos project will be documented in this file.

The format is based on Keep a Changelog.

Unreleased

Energy planning update: read before upgrading

This update brings the complete GENETIC energy planner to the current EOS interfaces. It can plan in 15-minute steps, consider battery export, schedule EV charging before departure and fit flexible household loads into their allowed time windows. New measurement tools make missing data, energy use and battery-capacity estimates easier to inspect. The Akkudoktor PV provider also gains an optional local, calibratable model.

Existing installations need an integration check before upgrading. Device settings now use stable IDs instead of list positions. The new POST /v1/optimize takes hardware and schedules from EOS configuration; its request body supplies only current observations, forecasts and a previous plan. The legacy POST /optimize remains available with GENETIC0; changing that URL alone does not migrate an integration to the new planner.

Automatic configuration migration handles supported old settings, but cannot rewrite Home Assistant automations, Node-RED flows or custom scripts. Back up configuration and stored data, then check device IDs, tariff units, fresh battery measurements and the resulting plans. Read the upgrade guide for the compatibility changes and a short test procedure. These changes are not yet a tagged release.

Added

• Complete GENETIC planning at 15- or 60-minute intervals, timestamp-aligned reuse of previous plans, explicitly enabled battery export, EV deadlines and flexible consumers.
• Forecast continuation and AUTO/FIXED valuation of energy left in the battery, separate from battery wear costs. Continuation helps choose the plan; it does not add commands beyond the configured control horizon.
• Optional ems.notify_url: EOS sends an HTTP POST with a small JSON event after every completed optimization, so a client can fetch the new plan right away instead of polling.
• Configuration-driven POST /v1/optimize and a PDF report of the stored GENETIC result.
• Measurement sample, energy, household-balance and battery-capacity APIs with data-quality and coverage information. A capacity estimate does not overwrite configured capacity.
• Optional local, calibratable forecasting within PVForecastAkkudoktor; the remote backend remains the default.
• Optional load-dependent inverter efficiency for GENETIC: dc_to_ac_efficiency_curve models the lower DC-to-AC efficiency at low load. Without it the constant dc_to_ac_efficiency applies unchanged.
• Optional size limit for the GENETIC fitness cache: optimization.genetic.fitness_cache_max_entries (null = unbounded, the default; 0 = off; N = at most N keys). Bounding it lowers peak memory of a run by up to ~150 MB on small devices without changing the plan.
• Optional general.timezone_override: an IANA timezone name used as general.timezone instead of the lookup from latitude/longitude. Unset (default), the lookup is unchanged.
• GET /v1/health reports started_at, the start time of the server process, so a client notices a restart also in a container (where the pid is always 1).
• New PV forecast providers giving operators more cloud forecast sources to choose from in addition to Akkudoktor, VRM and Import:
  • PVForecastPVNode — native 15-minute forecasts from the pvnode.com API.
  • PVForecastForecastSolar — forecasts from the free Forecast.Solar API.
  • PVForecastSolcast — forecasts from the Solcast rooftop-site API.
• Parallel fitness evaluation for GENETIC: optimization.genetic.workers (unset = automatic: at most two worker processes, one CPU core and a container CPU limit stay free; 1 = one process as before) and optimization.genetic.pin_workers (workers pinned to the last cores, default on). For a fixed seed the plan is identical to a single-process run.

Changed / compatibility

• Device collections are maps keyed by device ID. Old list-based configuration migrates, but integrations using paths such as devices.batteries.0 must use the actual device ID. GENETIC and GENETIC0 have separate algorithm settings.
• GENETIC rejects missing, stale or invalid battery state and incomplete control forecasts instead of inventing usable inputs. Unsupported device counts and inconsistent battery links also fail explicitly. The default measurement age limit is five minutes.
• Imported sale prices, including zero and negative prices, are preserved. Corrected battery/inverter limits, losses and wear costs can change the chosen plan and its costs.
• The electricity-fee framework replaces old elecprice.charges_kwh / vat_rate fields. Those old fields are dropped during migration: configure and verify fees explicitly.
• Quarter-hour clients must use the returned timestamps and interval, not assume 24 hourly values. Runtime forecast energy is Wh per slot and prices are currency per Wh.
• Runtime configuration changes now take priority over environment and file values for explicitly updated keys; command-line settings retain higher priority.
• Lower idle memory: matplotlib (PDF reports), statsmodels (price fallback forecast), pvlib, bs4/lxml, scipy (self-consumption table), the GENETIC0 optimizer and the timezone lookup data are imported on first use instead of at server start. Results are unchanged.
• New setting optimization.self_consumption_interpolator (scipy by default, numpy). With numpy the self-consumption probability tables are evaluated without SciPy, so a GENETIC run no longer loads SciPy (about 35 MB less memory). Results are bit-identical to SciPy's Linux builds. The tables are read without SciPy in both cases.
• Lower idle memory: matplotlib (PDF reports), statsmodels (price fallback forecast), pvlib, bs4/lxml, scipy (self-consumption table), the GENETIC0 optimizer and the timezone lookup data are imported on first use instead of at server start. Results are unchanged.
• Faster GENETIC runs with unchanged plans (about 60 % less run time on a Raspberry Pi 4): table lookups, the appliance starts and the battery state decoding are worked out once per run instead of once per candidate, the slot simulation and the battery bookkeeping work on plain numbers instead of single NumPy elements, the terminal value report is built for the final solution only, offspring are created without deepcopy, and identical genomes of a generation are evaluated once also when the fitness cache is off.

Fixed

• Failed optimization requests no longer return a previous successful result as if it belonged to the failed run; successful results and plans are published together.
• Restored JSON measurements retain their timestamps and values.
• Configuration updates made at runtime, e.g. by PUT /v1/config, are no longer discarded when the same configuration key is set in the EOS configuration file or in the environment (#1303).
• Environment variables are applied to the configuration on server startup instead of taking effect only after the first configuration change.
• GENETIC reads its horizon, interval and prediction settings once at the start of a run. A configuration change during a run (for example a client adapting the control horizon) no longer fails the run with an out-of-bounds error; it applies to the next run.

0.3.0 (2026-03-17)

Akkudoktor-EOS can now be run as Home Assistant add-on and standalone. As Home Assistant add-on EOS uses ingress to fully integrate the EOSdash dashboard in Home Assistant.

Adapters for Home Assistant and NodeRed integration are added. These adapters provide a simplified interface to these HEMS besides the standard REST interface.

The prediction and measurement data can now be backed by a database. The database allows to keep historic prediction data and measurement data for long time without keeping it in memory. The database supports backend selection, compression, incremental data load, automatic data saving to storage, automatic vacuum and compaction. Two database backends are integrated and can be configured, LMDB and SQLight3.

New prediction providers allow to access OpenMeteo weather data and to define fixed electricity prices for configurable time windows.

An anoying bug in the genetic algorithm that created unfeasable battery charge and discharge amounts is now hopefully fixed.

In addition, bugs were fixed and new features were added.

Feat

• add inverter AC/DC efficiency and break-even penalty
• add database support for measurements and historic prediction data. The prediction and measurement data can now be backed by a database. The database allows to keep historic prediction data and measurement data for long time without keeping it in memory. Two database backends are integrated and can be configured, LMDB and SQLight3.
• add adapters for integrations Adapters for Home Assistant and NodeRED integration are added. Akkudoktor-EOS can now be run as Home Assistant add-on and standalone. As Home Assistant add-on EOS uses ingress to fully integrate the EOSdash dashboard in Home Assistant.
• add make repeated task function make_repeated_task allows to wrap a function to be repeated cyclically.
• allow eos to be started with root permissions and drop priviledges Home assistant starts all add-ons with root permissions. Eos now drops root permissions if an applicable user is defined by paramter --run_as_user. The docker image defines the user eos to be used.
• make home assistant add-on run optimization by default When running as Home Assistant add-on the only viable usage is running with cyclic optimization. Make this the default to als propvide a better experience for first time users. The optimization will start with demo data, which also helps to configure Akkudoktor-EOS to the personal usage.
• make eos supervise and monitor EOSdash Eos now not only starts EOSdash but also monitors EOSdash during runtime and restarts EOSdash on fault. EOSdash logging is captured by EOS and forwarded to the EOS log to provide better visibility.
• add openmeteo weather provider
• add fixed electricity prediction with time window support
• add duration to string conversion Make to_duration to also return the duration as string on request.

Fixed

• genetic optimizer charge rates and soc accuracy
• energy charts bidding zone in request
• prevent exception when load prediction data is missing
• eosdash startup Ensure that EOSdash is only started after EOS configuration is available.
• config eos test setup Make the config_eos fixture generate a new instance of the config_eos singleton. Use correct env names to setup data folder path.
• startup with no config Make cache and measurements complain about missing data path configuration but do not bail out.
• soc data preparation and usage for genetic optimization. Search for soc measurments 48 hours around the optimization start time. Only clamp soc to maximum in battery device simulation.
• dashboard bailout on zero value solution display Do not use zero values to calculate the chart values adjustment for display.
• openapi generation script Make the script also replace data_folder_path and data_output_path to hide real (test) environment pathes.
• development version scheme The development versioning scheme is adaptet to fit to docker and home assistant expectations. The new scheme is x.y.z and x.y.z.dev'date''hash'. Hash is only digits as expected by home assistant. Development version is appended by .dev as expected by docker.
• use mean value in interval on resampling for array When downsampling data use the mean value of all values within the new sampling interval.
• default battery ev soc and appliance wh Make the genetic simulation return default values for the battery SoC, electric vehicle SoC and appliance load if these assets are not used.
• import json string Strip outer quotes from JSON strings on import to be compliant to json.loads() expectation.
• default interval definition for import data Default interval must be defined in lowercase human definition to be accepted by pendulum.
• clearoutside schema change

Chore

• removed index based data sequence access Index based data sequence access does not make sense as the sequence can be backed by the database. The sequence is now purely time series data.
• refactor eos startup to avoid module import startup Avoid module import initialisation expecially of the EOS configuration. Config mutation, singleton initialization, logging setup, argparse parsing, background task definitions depending on config and environment-dependent behavior is now done at function startup.
• introduce retention manager A single long-running background task that owns the scheduling of all periodic server-maintenance jobs (cache cleanup, DB autosave, …)
• guard against visualization errors in genetic optimization
• improve provider update error handling and add VRM provider settings validation
• canonicalize timezone name for UTC Timezone names that are semantically identical to UTC are canonicalized to UTC.
• extend config file migration for default value handling
• extend datetime util test cases
• make version test check for untracked files Check for files that are not tracked by git. Version calculation will be wrong if these files will not be commited.
• bump pandas to 3.0.0 Pandas 3.0 now performs inference on the appropriate resolution (a.k.a. unit) for the output dtype which may become datetime64[us] (before it was ns). Also numeric dtype detection is now more strict which needs a different detection for numerics.
• bump pydantic-settings to 2.12.0 pydantic-settings 2.12.0 under pytest creates a different behaviour. The tests were adapted and a workaround was introduced. Also ConfigEOS was adapted to allow for fine grain initialization control to be able to switch off certain settings such as file settings during test.
• remove sci learn kit from dependencies The sci learn kit is not strictly necessary as long as we have scipy.
• add documentation mode guarding for sphinx autosummary Sphinx autosummary excecutes functions. Prevent exceptions in case of pure doc mode.
• adapt docker-build CI workflow to stricter GitHub handling
• add CodeQL analysis workflow to CI
• Use info logging to report missing optimization parameters In parameter preparation for automatic optimization an error was logged for missing paramters. Log is now down using the info level.
• make EOSdash use the EOS data directory for file import/ export EOSdash use the EOS data directory for file import/ export by default. This allows to use the configuration import/ export function also within docker images.
• improve EOSdash config tab display Improve display of JSON code and add more forms for config value update.
• make docker image file system layout similar to home assistant Only use /data directory for persistent data. This is handled as a docker volume. The /data volume is mapped to ~/.local/share/net.akkudoktor.eos if using docker compose.
• add home assistant add-on development environment Add VSCode devcontainer and task definition for home assistant add-on development.
• Use uv to manage the virtual environment for development. This enormously increases dependency updates.
• improve documentation

0.2.0 (2025-11-09)

The most important new feature is automatic optimization. EOS can now independently perform optimization at regular intervals. This is based on the configured system parameters and forecasts, and also uses supplied measurement data, such as the current battery SoC. The result is an energy-management plan as well as the optimization output. The existing optimization interface using POST /optimize remains available and can still be used as before.

In addition, bugs were fixed and new features were added:

• Automatic optimization creates a default configuration if none is provided. This is intended to make it easier to create a custom configuration by adapting the default.
• The parameters of the genetic optimization algorithm (number of generations, etc.) are now configurable.
• For home appliances, start windows can now be specified (experimental).
• Configuration files from previous versions are converted to the current format on first launch.
• There are now measurement keys that are permanently assigned to a specific device simulation. This simplifies providing measurement values for device simulations (e.g. battery SoC).
• The infrastructure and first applications for feed-in tariff forecasting (currently only fixed tariffs) are now integrated.
• EOSdash has been expanded with new tabs for displaying the energy-management plan and predictions.
• The documentation has been updated and expanded in many places.

Feat

• Energy-management plan generation based on S2 standard instructions

• Feed-in-tariff prediction support (incl. tests & docs)

• LoadAkkudoktorAdjusted load prediction variant

• Standardized measurement keys for battery/EV SoC

• Measurement keys configurable via EOS configuration

• Setup default device configuration for automatic optimization

• Health endpoints show version + last optimization timestamps

• Configuration of genetic algorithm parameters

• Configuration options for home-appliance time windows

• Mitigation of legacy configuration

• Config backup enhancements:

  • Timestamp-based backup IDs
  • API to list backups
  • API to revert to a specific backup
  • EOSdash Admin tab integration

• Pendulum date types via pydantic_extra_types.pendulum_dt

• Time, TimeWindow, TimeWindowSequence, and to_time helpers in datetimeutil

• Extended DataRecord with configurable field-like semantics

• EOSdash: Solution view now displays genetic optimization results and aggregated totals

• EOSdash UI:

  • Plan tab
  • Predictions tab
  • Cache management in Admin tab
  • About tab

• Pydantic merge model tests

• Developer profiling entry in Makefile

• Changelog & docs updated for commitizen release flow

• Developer documentation updated

• Improved install & development documentation

Changed

• Battery simulation

  • Performance improvements
  • Charge + start times now reflect realistic simulation

• Appliance simulation:

  • Time windows may roll over to next day

• Revised load prediction by splitting original LoadAkkudoktor into:

  • LoadAkkudoktor
  • LoadAkkudoktorAdjusted

Fixed

• Correct URL/path for Akkudoktor forum in README

• Automatic optimization:

  • Reuses previous start solution
  • Interval execution + locking + new endpoints
  • Properly loads required data
  • EV charge-rate migration for proper availability

• Genetic common settings consistently available

• Config markdown generation

• Recognize environment variables on EOS server startup

• Remove 0.0.0.0 → localhost translation on Windows

• Allow hostnames as well as IPs

• Access Pydantic model fields via class instead of instance

• Down-sampling in key_to_array

• /v1/admin/cache/clear clears all cache files; added /clear-expired

• Use tzfpy instead of timezonefinder for more accurate EU timezones

• Explicit provider settings in config instead of union

• ClearOutside weather prediction irradiance calculation

• Test config file priority without config_eos fixture

• Complete optimization sample-request documentation

• Replace gitlint with commitizen

• Synchronize pre-commit config with real dependencies

• Add missing babel to requirements

• Fix documentation, tests, and implementation around optimization + predictions

Chore

• Use memory cache for inverter interpolation
• Refactor genetic modules (split config, remove device singleton)
• Rename memory cache to CacheEnergyManagementStore
• Use class properties for config/EMS/prediction mixins
• Skip matplotlib debug logs
• Auto-sync Bokeh JS CDN version
• Rename hello.py → about.py in EOSdash
• Remove EOSdash demo page
• Split server test from system test
• Move doc utils to generate_config_md.py
• Improve documentation for pydantic merge models
• Remove pendulum warning from README
• Drop GitHub Discussions from contributing docs
• Rename or reorganize files / classes during refactors

BREAKING CHANGES

EOS configuration + v1 API have changed:

• available_charge_rates_percent removed → replaced by charge_rate
• Optimization parameter hours → renamed to horizon_hours
• Device config must explicitly list devices + properties
• Prediction providers now explicit (instead of union)
• Measurement keys provided as lists
• Feed-in-tariff providers must be explicitly configured
• /v1/measurement/loadxxx endpoints removed → use generic measurement endpoints
• /v1/admin/cache/clear now clears *all- cache files; /v1/admin/cache/clear-expired only clears expired entries

v0.1.0 (2025-09-30)

Feat

• added Changelog for 0.0.0 and 0.1.0

v0.0.0 (2025-09-30)

This version represents one year of development of EOS (Energy Optimization System). From this point forward, release management will be introduced.

Feat

Core Features

• energy Management System (EMS) with battery optimization
• PV (Photovoltaic) forecast integration with multiple providers
• load prediction and forecasting capabilities
• electricity price integration
• VRM API integration for load and PV forecasting
• battery State of Charge (SoC) prediction and optimization
• inverter class with AC/DC charging logic
• electric vehicle (EV) charging optimization with configurable currents
• home appliance scheduling optimization
• horizon validation for shading calculations

API & Server

• migration from Flask to FastAPI
• RESTful API with comprehensive endpoints
• EOSdash web interface for configuration and visualization
• Docker support with multi-architecture builds
• web-based visualization with interactive charts
• OpenAPI/Swagger documentation
• configurable server settings (port, host)

Configuration & Data Management

• JSON-based configuration system with nested support
• configuration validation with Pydantic
• device registry for managing multiple devices
• persistent caching for predictions and prices
• manual prediction updates
• timezone support with automatic detection
• configurable VAT rates for electricity prices

Optimization

• DEAP-based genetic algorithm optimization
• multi-objective optimization (cost, battery usage, self-consumption)
• 48-hour prediction and optimization window
• AC/DC charging decision optimization
• discharge hour optimization
• start solution enforcement
• fitness visualization with violin plots
• self-consumption probability interpolator

Testing & Quality

• comprehensive test suite with pytest
• unit tests for major components (EMS, battery, inverter, load, optimization)
• integration tests for server endpoints
• pre-commit hooks for code quality
• type checking with mypy
• code formatting with ruff and isort
• markdown linting

Documentation

• conceptual documentation
• API documentation with Sphinx
• ReadTheDocs integration
• Docker setup instructions
• contributing guidelines
• English README translation

Providers & Integrations

• PVForecast.Akkudoktor provider
• BrightSky weather provider
• ClearOutside weather provider
• electricity price provider

Refactor

• optimized Inverter class for improved SCR calculation performance
• improved caching mechanisms for better performance
• enhanced visualization with proper timestamp handling
• updated dependency management with automatic Dependabot updates
• restructured code into logical submodules
• package directory structure reorganization
• improved error handling and logging
• Windows compatibility improvements

Fix

• cross-site scripting (XSS) vulnerabilities
• ReDoS vulnerability in duration parsing
• timezone and daylight saving time handling
• BrightSky provider with None humidity data
• negative values in load mean adjusted calculations
• SoC calculation bugs
• AC charge efficiency in price calculations
• optimization timing bugs
• Docker BuildKit compatibility
• float value handling in user horizon configuration
• circular runtime import issues
• load simulation data return issues
• multiple optimization-related bugs

Build

• Python version requirement updated to 3.10+
• added Bandit security checks
• improved credential management with environment variables

Dependencies

Major dependencies included in this release:

• FastAPI 0.115.14
• Pydantic 2.11.9
• NumPy 2.3.3
• Pandas 2.3.2
• Scikit-learn 1.7.2
• Uvicorn 0.36.0
• Bokeh 3.8.0
• Matplotlib 3.10.6
• PVLib 0.13.1
• Python-FastHTML 0.12.29

Notes

Development Notes

This version encompasses all development from the initial commit (February 16, 2024) through September 29, 2025. The project evolved from a basic energy optimization concept to a comprehensive energy management system with:

• 698+ commits
• multiple contributor involvement
• continuous integration/deployment setup
• automated dependency updates
• comprehensive testing infrastructure

Migration Notes

As this is the initial versioned release, no migration is required. Future releases will include migration guides as needed.