3 Commits
Author SHA1 Message Date
Bobby NoelteandGitHub ba76087db9 feat: electricity fee provider framework and generic providers (#1235)
Add new provider class for electricity fees providers.

Add the generic providers:
- ElecFeeFixed
- ElecFeeImport

The providers provide predictions for:

- elecfee_consumption_amt_wh:
  Total fixed fee for consumed energy per Wh [amount/Wh]. This is the accumulation of all
  fixed per-Wh fees payable on "consumed energy - such as network charge, concession fee,
  and electricity charge - into a single amount.
- elecfee_consumption_percent_amt:
  Total fixed surcharge on consumed energy, given as a percentage of the monetary amount
  already charged for that energy [%]. This is the accumulation of all percentage-based
  surcharges payable on top of the consumed-energy fee - such as VAT - into a single
  percentage. This is a percentage of the fee amount, not a per-Wh rate.
- elecfee_feedin_amt_wh:
  Total fixed deduction from feed-in energy per Wh [amount/Wh]. This is the accumulation of
  all fixed per-Wh charges deducted from feed-in energy - such as metering fees or
  grid-operator handling "charges - into a single amount. Applied after the percentage-based
  deduction, i.e. it reduces the price by a flat amount per Wh rather than by a share of the
  raw price.
- elecfee_feedin_percent_amt:
  Total percentage deducted from the raw feed-in price (spot price) [%]. This is the
  accumulation of all percentage-based deductions payable on the feed-in tariff - such as a
  marketing or balancing fee retained by the aggregator - into a single percentage. It is
  applied as `raw_price * (100 - percent) / 100`, i.e. it scales down the raw price rather
  than adding a surcharge to it.

A new _apply_fee() method is added to the base class for ElecPrice and FeedInTariff to be used to
add the fees in a consistent way. Fees are taken from the active ElecFee provider and applied
to the raw prices given to the _apply_fee() method.

The optional application of fees is added to:

- ElecPriceAkkudoktor
- ElecPriceFixed
- ElecPriceEnergyCharts
- ElecPriceSMARD
- FeedInTariffEnergyCharts
- FeedInTariffFixed
- FeedInTariffSMARD

The import providers ElecPriceImport and FeedInTariffImport do not apply fees by intentention.

The following providers currently do not handle fees defined by ElecFee:

- ElecPriceTibber
- FeedInTariffAkkudoktor
- FeedInTariffDvhubOnline
- FeedInTariffTibber

The tests for this feature are either added or existing tests are extended.

The documentation was extended for the electricity fee provider settings.

Besides this feature further improvements are added:

* feat: add SMARD quarter-hour electricty price and feed-in tariff provider

* feat: to_series method for TimeWindows and ValueTimeWindows

  Additional to to_array the time window sequence can now also produce a pandas series.
  Test have been extended to cover the series generation.

* feat: use time windows in fixed feedin tariff provider

  Feedin tariff can now be configured by time windows - not a single value.

* feat: EOSdash select for PVLib inverters and modules

  Provide PVLib inverter and module names in config selection.

* feat: EOSdash lazy select for big option sets

  Add a new form for lazy selection of big option sets. Filtering and
  generation of the option set is done server-side.

* fix: use raw data for ETS/ median prediction

  Use to raw time series data for ETS/ median prediction to avoid interference
  by e.g. dynamic grid charges.

* fix: EOSdash config drops by type only on details resolve

  Drop configuration by type and path. Prevents dropping of configuration items
  with same type and level but different path.

* fix: EOSdash configuration section closes on update

  Open section if searching or if last update touched this category — including
  updates on deeply nested sub-fields.

* chore: make elecfeefixed, elecpricefixed and feedintarifffixed warn about no windows and default to 0

  Missining configuration creates default 0 value and a warning instead of an exception.

* fix: test setup for providers

  Reset db state on each test run.

* chore: improve config option naming for elecpricefixed.

* chore: adapt elecpricefixed test to changed time_windows naming

* chore: factorized common price provider helpers to priceabc.py

  Factorized common price provider helpers to priceabc.py. Add tests for these helpers.
  Reduce/ change testing of elecpriceabc.py and feedintariffabc.py to cover
  only specifics. Rest of testing is already covered by test_priceabc.py.

* chore: update version

Signed-off-by: Bobby Noelte <b0661n0e17e@gmail.com>
2026-08-23 02:26:16 +02:00
Bobby NoelteandGitHub 1905682113 chore: adapt pdf visualization (#1205)
Change PDF visualization to be created on demand and per optimization algorithm. The PDF
for the GENETIC0 optimization is provided by the /visualization_results.pdf endpoint.
There is no change in the interface.

By this the optimization algorithm is offloaded from the PDF generation which spares some
time.

To cope with several users may call the /visualization_results.pdf endpoint at the same
time the PDF is generated on the fly without any intermediate file taking the stored
GENETIC0 solution as an input. SVG picture generation is removed as this would again
create intermediate files. Chart pictures can easily be taken from the PDF.

To allow on demand creation of the optimization results visualization the optimisation
solution stored is extended by several new attributes. To keep the deprecated
/optimize endpoint compatible the optimization solution is stripped to the legacy
content before returned. Due to the extension of the solution the optimization tests were
adapted to cover the extended content.

The optimization tests are adapted to test the generated visualization report by
the pypdf reader. Pypdf is added to the development dependencies.

Besides the adaptation several fixes and improvements are added:

* feat: extend /v1/prediction/series endpoint by resampling and filling

  Add parameters for resampling and filling. Add the processing parameter
  to control wether raw data or resampled data shall be returned.

* feat: extend /v1/measurement/series endpoint by resampling and filling

  Add parameters for resampling and filling: Add the processing parameter
  to control wether raw data or resampled data shall be returned.

* feat: standardize and improve API error response

  Use FASTApi exception handlers to provide a standardized API exception handling.
  All exceptions are logged.

  Exception traces are only returned if the new logging configuration parameter
  logging.api_logging_level is set to "DEBUG" or "TRACE". Avoids unwanted leackage
  of server internals on exceptions.

* fix: align to intervall when resampling

  Ensure resampling is aligned to interval also when the buckets are shifted due to the
  align_to_intervall parameter is set.

* chore: make dropna mandatory and default to True

* chore: refactor key_to_xxx data management methods

  Make key_to_series the central method for data resampling and fill.
  Add a new key_to_raw_series to retrieve the data as it is stored
  (without resampling and filling).

  Users of key_to_series were mostly moved to key_to_raw_series as this resembles
  the former interface. Especially in predictions and tests this was done.

* chore: create test data sub-directory for each optimization algorithm

  To prevent cluttering the test data directory and ease test data management for
  optimization algorithms each algorithm got it's own sub-directory. The current
  test data was moved to these sub-directories.

* chore: update version

Signed-off-by: Bobby Noelte <b0661n0e17e@gmail.com>
2026-08-01 12:45:19 +02:00
Bobby NoelteandGitHub ed61918fe0 feat: add EnergyCharts feed-in tariff provider (#1165)
Bump Version / Bump Version Workflow (push) Has been cancelled
CodeQL Advanced / Analyze (actions) (push) Has been cancelled
CodeQL Advanced / Analyze (python) (push) Has been cancelled
docker-build / platform-excludes (push) Has been cancelled
docker-build / build (push) Has been cancelled
docker-build / merge (push) Has been cancelled
pre-commit / pre-commit (push) Has been cancelled
Run Pytest on Pull Request / test (push) Has been cancelled
Close stale pull requests/issues / Find Stale issues and PRs (push) Has been cancelled
The `FeedInTariffEnergyCharts` provider uses the raw Energy-Charts day-ahead market price as the
feed-in tariff. It stores prices in `feed_in_tariff_wh` without adding electricity import charges
or VAT. The data is loaded from the Energy-Charts `/price` endpoint for the configured bidding
zone. The native Energy-Charts resolution, including quarter-hour data, is retained.

Energy-Charts usually supplies prices only for the published day-ahead period. If that data does
not cover the complete configured prediction horizon, the provider extends it as follows:

- With more than 800 hours of history, an ETS (Holt-Winters exponential smoothing) forecast with
  weekly seasonality is used.
- With more than 168 hours of history, an ETS forecast with daily seasonality is used.
- With less history, the median of the available values is used as a constant fallback.

The seasonal periods are adjusted to the source resolution. For example, quarter-hour data uses
four values per hour. Values already supplied by Energy-Charts are kept unchanged; only missing
future slots after the last published price are forecast. Consequently, a 15-minute optimization
uses four forecast values per hour without converting them to hourly averages.

Signed-off-by: Andreas Schmitz <akkudoktor.net>
Signed-off-by: Bobby Noelte <b0661n0e17e@gmail.com>
2026-07-23 15:48:29 +02:00