feat: electricity fee provider framework and generic providers (#1235)

Add new provider class for electricity fees providers.

Add the generic providers:
- ElecFeeFixed
- ElecFeeImport

The providers provide predictions for:

- elecfee_consumption_amt_wh:
  Total fixed fee for consumed energy per Wh [amount/Wh]. This is the accumulation of all
  fixed per-Wh fees payable on "consumed energy - such as network charge, concession fee,
  and electricity charge - into a single amount.
- elecfee_consumption_percent_amt:
  Total fixed surcharge on consumed energy, given as a percentage of the monetary amount
  already charged for that energy [%]. This is the accumulation of all percentage-based
  surcharges payable on top of the consumed-energy fee - such as VAT - into a single
  percentage. This is a percentage of the fee amount, not a per-Wh rate.
- elecfee_feedin_amt_wh:
  Total fixed deduction from feed-in energy per Wh [amount/Wh]. This is the accumulation of
  all fixed per-Wh charges deducted from feed-in energy - such as metering fees or
  grid-operator handling "charges - into a single amount. Applied after the percentage-based
  deduction, i.e. it reduces the price by a flat amount per Wh rather than by a share of the
  raw price.
- elecfee_feedin_percent_amt:
  Total percentage deducted from the raw feed-in price (spot price) [%]. This is the
  accumulation of all percentage-based deductions payable on the feed-in tariff - such as a
  marketing or balancing fee retained by the aggregator - into a single percentage. It is
  applied as `raw_price * (100 - percent) / 100`, i.e. it scales down the raw price rather
  than adding a surcharge to it.

A new _apply_fee() method is added to the base class for ElecPrice and FeedInTariff to be used to
add the fees in a consistent way. Fees are taken from the active ElecFee provider and applied
to the raw prices given to the _apply_fee() method.

The optional application of fees is added to:

- ElecPriceAkkudoktor
- ElecPriceFixed
- ElecPriceEnergyCharts
- ElecPriceSMARD
- FeedInTariffEnergyCharts
- FeedInTariffFixed
- FeedInTariffSMARD

The import providers ElecPriceImport and FeedInTariffImport do not apply fees by intentention.

The following providers currently do not handle fees defined by ElecFee:

- ElecPriceTibber
- FeedInTariffAkkudoktor
- FeedInTariffDvhubOnline
- FeedInTariffTibber

The tests for this feature are either added or existing tests are extended.

The documentation was extended for the electricity fee provider settings.

Besides this feature further improvements are added:

* feat: add SMARD quarter-hour electricty price and feed-in tariff provider

* feat: to_series method for TimeWindows and ValueTimeWindows

  Additional to to_array the time window sequence can now also produce a pandas series.
  Test have been extended to cover the series generation.

* feat: use time windows in fixed feedin tariff provider

  Feedin tariff can now be configured by time windows - not a single value.

* feat: EOSdash select for PVLib inverters and modules

  Provide PVLib inverter and module names in config selection.

* feat: EOSdash lazy select for big option sets

  Add a new form for lazy selection of big option sets. Filtering and
  generation of the option set is done server-side.

* fix: use raw data for ETS/ median prediction

  Use to raw time series data for ETS/ median prediction to avoid interference
  by e.g. dynamic grid charges.

* fix: EOSdash config drops by type only on details resolve

  Drop configuration by type and path. Prevents dropping of configuration items
  with same type and level but different path.

* fix: EOSdash configuration section closes on update

  Open section if searching or if last update touched this category — including
  updates on deeply nested sub-fields.

* chore: make elecfeefixed, elecpricefixed and feedintarifffixed warn about no windows and default to 0

  Missining configuration creates default 0 value and a warning instead of an exception.

* fix: test setup for providers

  Reset db state on each test run.

* chore: improve config option naming for elecpricefixed.

* chore: adapt elecpricefixed test to changed time_windows naming

* chore: factorized common price provider helpers to priceabc.py

  Factorized common price provider helpers to priceabc.py. Add tests for these helpers.
  Reduce/ change testing of elecpriceabc.py and feedintariffabc.py to cover
  only specifics. Rest of testing is already covered by test_priceabc.py.

* chore: update version

Signed-off-by: Bobby Noelte <b0661n0e17e@gmail.com>

src/akkudoktoreos/config/config.py

@@ -43,6 +43,7 @@ from akkudoktoreos.core.version import __version__
 from akkudoktoreos.devices.devices import DevicesCommonSettings
 from akkudoktoreos.measurement.measurement import MeasurementCommonSettings
 from akkudoktoreos.optimization.optimization import OptimizationCommonSettings
+from akkudoktoreos.prediction.elecfee import ElecFeeCommonSettings
 from akkudoktoreos.prediction.elecprice import ElecPriceCommonSettings
 from akkudoktoreos.prediction.feedintariff import FeedInTariffCommonSettings
 from akkudoktoreos.prediction.load import LoadCommonSettings
@@ -264,6 +265,9 @@ class SettingsEOS(pydantic_settings.BaseSettings, PydanticModelNestedValueMixin)
     prediction: Optional[PredictionCommonSettings] = Field(
         default=None, json_schema_extra={"description": "Prediction Settings"}
     )
+    elecfee: Optional[ElecFeeCommonSettings] = Field(
+        default=None, json_schema_extra={"description": "Electricity Fee Settings"}
+    )
     elecprice: Optional[ElecPriceCommonSettings] = Field(
         default=None, json_schema_extra={"description": "Electricity Price Settings"}
     )
@@ -312,6 +316,7 @@ class SettingsEOSDefaults(SettingsEOS):
     measurement: MeasurementCommonSettings = Field(default_factory=MeasurementCommonSettings)
     optimization: OptimizationCommonSettings = Field(default_factory=OptimizationCommonSettings)
     prediction: PredictionCommonSettings = Field(default_factory=PredictionCommonSettings)
+    elecfee: ElecFeeCommonSettings = Field(default_factory=ElecFeeCommonSettings)
     elecprice: ElecPriceCommonSettings = Field(default_factory=ElecPriceCommonSettings)
     feedintariff: FeedInTariffCommonSettings = Field(default_factory=FeedInTariffCommonSettings)
     load: LoadCommonSettings = Field(default_factory=LoadCommonSettings)
@@ -621,6 +626,7 @@ class ConfigEOS(SingletonMixin, SettingsEOSDefaults):
                 # This should not happen
                 raise RuntimeError("Config file path not set.")
 
+            settings = {}
             try:
                 backup_file = config_file.with_suffix(f".{to_datetime(as_string='YYYYMMDDHHmmss')}")
                 if migrate_config_file(config_file, backup_file):
@@ -632,7 +638,6 @@ class ConfigEOS(SingletonMixin, SettingsEOSDefaults):
                 logger.error(
                     f"Error reading config file '{config_file}' (falling back to default config): {ex}"
                 )
-                settings = {}
 
             return settings
 

src/akkudoktoreos/config/configabc.py

@@ -7,6 +7,7 @@ from enum import StrEnum
 from typing import Any, ClassVar, Iterator, Optional, Union
 
 import numpy as np
+import pandas as pd
 import pendulum
 from babel.dates import get_day_names
 from pydantic import Field, field_serializer, field_validator, model_validator
@@ -668,6 +669,76 @@ class TimeWindowSequence(SettingsBaseModel):
 
         return np.array(result, dtype=np.float64)
 
+    def to_series(
+        self,
+        start_datetime: DateTime,
+        end_datetime: DateTime,
+        interval: Duration,
+        dropna: bool = True,
+        boundary: str = "context",
+        align_to_interval: bool = True,
+    ) -> pd.Series:
+        """Return a pandas Series indicating window coverage over a time grid.
+
+        The time grid is constructed from ``start_datetime`` to ``end_datetime``
+        (exclusive) in steps of ``interval``. Each element is ``1.0`` when the
+        corresponding step falls inside any window in this sequence, and ``0.0``
+        otherwise.
+
+        Args:
+            start_datetime: First step of the time grid (inclusive).
+            end_datetime: Upper bound of the time grid (exclusive).
+            interval: Fixed step size between consecutive grid points.
+            dropna: Unused for ``TimeWindowSequence`` (no NaN values are
+                produced). Accepted for signature compatibility.
+            boundary: Controls range enforcement. Only ``"context"`` is
+                currently supported.
+            align_to_interval: When ``True``, ``start_datetime`` is floored to
+                the nearest interval boundary in wall-clock time before
+                generating the grid. The timezone (or naivety) of
+                ``start_datetime`` is preserved exactly.
+
+        Returns:
+            ``pd.Series`` with a ``DatetimeIndex`` and ``float64`` values.
+            ``1.0`` means the timestamp is inside a window; ``0.0`` means it
+            is not.
+
+        Raises:
+            ValueError: If ``boundary`` is not ``"context"``.
+        """
+        if boundary != "context":
+            raise ValueError(f"Unsupported boundary {boundary!r}. Only 'context' is supported.")
+
+        interval_s = interval.total_seconds()
+
+        if align_to_interval and interval_s > 0:
+            # Floor purely in wall-clock seconds so the timezone (or naivety)
+            # of start_datetime is never touched and no UTC conversion occurs.
+            wall_s = (
+                start_datetime.hour * 3600
+                + start_datetime.minute * 60
+                + start_datetime.second
+                + start_datetime.microsecond / 1_000_000
+            )
+            remainder_s = wall_s % interval_s
+            if remainder_s:
+                start_datetime = start_datetime.subtract(seconds=remainder_s)
+
+        timestamps: list[DateTime] = []
+        values: list[float] = []
+
+        current = start_datetime
+        while current < end_datetime:
+            timestamps.append(current)
+            values.append(1.0 if self.contains(current) else 0.0)
+            current = current.add(seconds=interval_s)
+
+        return pd.Series(
+            values,
+            index=pd.DatetimeIndex(timestamps),
+            dtype=np.float64,
+        )
+
     def add_window(self, window: TimeWindow) -> None:
         """Add a new time window to the sequence.
 
@@ -861,3 +932,102 @@ class ValueTimeWindowSequence(TimeWindowSequence):
             current = current.add(seconds=interval_s)
 
         return np.array(result, dtype=np.float64)
+
+    def to_series(
+        self,
+        start_datetime: DateTime,
+        end_datetime: DateTime,
+        interval: Duration,
+        dropna: bool = True,
+        boundary: str = "context",
+        align_to_interval: bool = True,
+    ) -> pd.Series:
+        """Return a pandas Series of window values over a time grid.
+
+        The time grid is constructed from ``start_datetime`` to ``end_datetime``
+        (exclusive) in steps of ``interval``, matching the ``key_to_series``
+        signature used by the prediction store. Each element holds the
+        ``value`` of the first matching window at that step, ``0.0`` when no
+        window matches, or ``NaN`` when the matching window has ``value=None``
+        and ``dropna=False``.
+
+        When ``dropna=True``, steps whose matching window has ``value=None`` are
+        omitted from the resulting Series entirely, including their timestamps.
+
+        Args:
+            start_datetime: First step of the time grid (inclusive).
+            end_datetime: Upper bound of the time grid (exclusive).
+            interval: Fixed step size between consecutive grid points.
+            dropna: When ``True``, steps whose matching window carries
+                ``value=None`` are dropped from the Series. When ``False``,
+                those steps emit ``NaN``.
+            boundary: Controls range enforcement. Only ``"context"`` is
+                currently supported; the output is always clipped to
+                ``[start_datetime, end_datetime)``.
+            align_to_interval: When ``True``, ``start_datetime`` is floored to
+                the nearest interval boundary in wall-clock time before
+                generating the grid. The timezone (or naivety) of
+                ``start_datetime`` is preserved exactly — no UTC conversion
+                is performed. When ``False``, ``start_datetime`` is used as-is.
+
+        Returns:
+            ``pd.Series`` with a ``DatetimeIndex`` and ``float64`` values.
+            Positive values are window values; ``0.0`` means no window matched;
+            ``NaN`` means a window matched but its value was ``None`` (only when
+            ``dropna=False``).
+
+        Raises:
+            ValueError: If ``boundary`` is not ``"context"``.
+        """
+        if boundary != "context":
+            raise ValueError(f"Unsupported boundary {boundary!r}. Only 'context' is supported.")
+
+        interval_s = interval.total_seconds()
+
+        if align_to_interval and interval_s > 0:
+            # Floor purely in wall-clock seconds so the timezone (or naivety)
+            # of start_datetime is never touched and no UTC conversion occurs.
+            # This is correct regardless of the machine's local timezone.
+            wall_s = (
+                start_datetime.hour * 3600
+                + start_datetime.minute * 60
+                + start_datetime.second
+                + start_datetime.microsecond / 1_000_000
+            )
+            remainder_s = wall_s % interval_s
+            if remainder_s:
+                start_datetime = start_datetime.subtract(seconds=remainder_s)
+
+        timestamps: list[DateTime] = []
+        result: list[float] = []
+
+        current = start_datetime
+        while current < end_datetime:
+            step_value: Optional[float] = None
+            matched = False
+
+            for window in self.windows:
+                if window.contains(current):
+                    step_value = window.value
+                    matched = True
+                    break
+
+            if not matched:
+                timestamps.append(current)
+                result.append(0.0)
+            elif step_value is None:
+                if not dropna:
+                    timestamps.append(current)
+                    result.append(float("nan"))
+                # else: omit this step and its timestamp
+            else:
+                timestamps.append(current)
+                result.append(step_value)
+
+            current = current.add(seconds=interval_s)
+
+        return pd.Series(
+            result,
+            index=pd.DatetimeIndex(timestamps),
+            dtype=np.float64,
+        )

src/akkudoktoreos/config/configmigrate.py

@@ -62,11 +62,18 @@ MIGRATION_MAP: Dict[
     "devices/batteries/0/initial_soc_percentage": None,
     # - electric_vehicles
     "devices/electric_vehicles/0/initial_soc_percentage": None,
+    # elecfee
+    # =======
+    # - ElecFeeFixed
+    # - ElecFeeImport
     # elecprice
     # =========
+    "elecprice/charges_kwh": None,
+    "elecprice/vat_rate": None,
     # - ElecPriceAkkudoktor
     # - ElecPriceEnergyCharts
     # - ElecPriceFixed
+    "elecprice/elecpricefixed/time_windows": "elecprice/elecpricefixed/elecprice_marketprice_amt_kwh",
     # - ElecPriceImport
     "elecprice/provider_settings/ElecPriceImport/import_file_path": "elecprice/elecpriceimport/import_file_path",
     "elecprice/provider_settings/ElecPriceImport/import_json": "elecprice/elecpriceimport/import_json",
@@ -75,7 +82,22 @@ MIGRATION_MAP: Dict[
     # feedintariff
     # ============
     # - FeedInTariffFixed
-    "feedintariff/provider_settings/FeedInTariffFixed/feed_in_tariff_kwh": "feedintariff/feedintarifffixed/feed_in_tariff_kwh",
+    "feedintariff/feedintarifffixed/feed_in_tariff_kwh": (
+        "feedintariff/feedintarifffixed/feed_in_tariff_amt_kwh",
+        lambda v: {
+            "windows": [
+                {"start_time": "00:00", "duration": "24 hours", "value": float(v)},
+            ],
+        },
+    ),
+    "feedintariff/provider_settings/FeedInTariffFixed/feed_in_tariff_kwh": (
+        "feedintariff/feedintarifffixed/feed_in_tariff_amt_kwh",
+        lambda v: {
+            "windows": [
+                {"start_time": "00:00", "duration": "24 hours", "value": float(v)},
+            ],
+        },
+    ),
     # - FeedInTariffImport
     "feedintariff/provider_settings/FeedInTariffImport/import_file_path": "feedintariff/feedintariffimport/import_file_path",
     "feedintariff/provider_settings/FeedInTariffImport/import_json": "feedintariff/feedintariffimport/import_json",

src/akkudoktoreos/core/dataabc.py

@@ -1272,6 +1272,7 @@ class DataSequence(DataABC, DatabaseRecordProtocolMixin[DataRecord]):
 
         Raises:
             KeyError: If the specified key is not found in any of the DataRecords.
+            ValueError: If the fill_method, resample_method or boundary values are invalid
         """
         # Validate fill method
         if fill_method not in ("ffill", "bfill", "linear", "time", "none", None):

src/akkudoktoreos/optimization/genetic/geneticparams.py

@@ -293,7 +293,7 @@ class GeneticOptimizationParameters(
             # Assure predictions are uptodate
             await cls.prediction.update_data()
 
-            try:  # Try first - predition is also needed by the default PV forecast
+            try:  # Try weather first - predition is also needed by the default PV forecast
                 array = await cls.prediction.key_to_array(
                     key="weather_temp_air",
                     start_datetime=parameter_start_datetime,
@@ -311,6 +311,70 @@ class GeneticOptimizationParameters(
                 cls.config.weather.provider = "OpenMeteo"
                 # Retry
                 continue
+            # Try electricity fees next - predition is also needed by the default electricity price
+            # If no provider is set the fees default to 0 anyway
+            if cls.config.elecfee.provider:
+                try:
+                    array = await cls.prediction.key_to_array(
+                        key="elecfee_consumption_amt_kwh",
+                        start_datetime=parameter_start_datetime,
+                        end_datetime=parameter_end_datetime,
+                        interval=interval,
+                        fill_method="ffill",
+                    )
+                except Exception as e:
+                    logger.info(
+                        "No electricity fee data available - defaulting to demo data. Parameter preparation attempt {}: {}",
+                        attempt,
+                        e,
+                    )
+                    cls.config.merge_settings_from_dict(
+                        {
+                            "elecfee": {
+                                "provider": "ElecFeeFixed",
+                                "elecfeefixed": {
+                                    "consumption_amt_kwh": {
+                                        "windows": [
+                                            {
+                                                "start_time": "00:00",
+                                                "duration": "24 hours",
+                                                "value": 0.21,
+                                            },
+                                        ]
+                                    },
+                                    "consumption_percent_amt": {
+                                        "windows": [
+                                            {
+                                                "start_time": "00:00",
+                                                "duration": "24 hours",
+                                                "value": 19.0,
+                                            },
+                                        ]
+                                    },
+                                    "feedin_amt_kwh": {
+                                        "windows": [
+                                            {
+                                                "start_time": "00:00",
+                                                "duration": "24 hours",
+                                                "value": 0.0,
+                                            },
+                                        ]
+                                    },
+                                    "feedin_percent_amt": {
+                                        "windows": [
+                                            {
+                                                "start_time": "00:00",
+                                                "duration": "24 hours",
+                                                "value": 0.0,
+                                            },
+                                        ]
+                                    },
+                                },
+                            },
+                        }
+                    )
+                    # Retry
+                    continue
             try:
                 array = await cls.prediction.key_to_array(
                     key="pvforecast_ac_power",
@@ -401,7 +465,7 @@ class GeneticOptimizationParameters(
                     {
                         "elecprice": {
                             "elecpricefixed": {
-                                "time_windows": {
+                                "elecprice_marketprice_amt_kwh": {
                                     "windows": [
                                         {
                                             "duration": "1 day",
@@ -464,7 +528,15 @@ class GeneticOptimizationParameters(
                         "feedintariff": {
                             "provider": "FeedInTariffFixed",
                             "feedintarifffixed": {
-                                "feed_in_tariff_kwh": 0.078,
+                                "feed_in_tariff_amt_kwh": {
+                                    "windows": [
+                                        {
+                                            "start_time": "00:00",
+                                            "duration": "24 hours",
+                                            "value": 0.078,
+                                        },
+                                    ],
+                                },
                             },
                         },
                     }

src/akkudoktoreos/optimization/genetic0/genetic0params.py

@@ -302,6 +302,70 @@ class Genetic0OptimizationParameters(
                 cls.config.weather.provider = "OpenMeteo"
                 # Retry
                 continue
+            # Try electricity fees next - predition is also needed by the default electricity price
+            # If no provider is set the fees default to 0 anyway
+            if cls.config.elecfee.provider:
+                try:
+                    array = await cls.prediction.key_to_array(
+                        key="elecfee_consumption_amt_kwh",
+                        start_datetime=parameter_start_datetime,
+                        end_datetime=parameter_end_datetime,
+                        interval=interval,
+                        fill_method="ffill",
+                    )
+                except Exception as e:
+                    logger.info(
+                        "No electricity fee data available - defaulting to demo data. Parameter preparation attempt {}: {}",
+                        attempt,
+                        e,
+                    )
+                    cls.config.merge_settings_from_dict(
+                        {
+                            "elecfee": {
+                                "provider": "ElecFeeFixed",
+                                "elecfeefixed": {
+                                    "consumption_amt_kwh": {
+                                        "windows": [
+                                            {
+                                                "start_time": "00:00",
+                                                "duration": "24 hours",
+                                                "value": 0.21,
+                                            },
+                                        ]
+                                    },
+                                    "consumption_percent_amt": {
+                                        "windows": [
+                                            {
+                                                "start_time": "00:00",
+                                                "duration": "24 hours",
+                                                "value": 19.0,
+                                            },
+                                        ]
+                                    },
+                                    "feedin_amt_kwh": {
+                                        "windows": [
+                                            {
+                                                "start_time": "00:00",
+                                                "duration": "24 hours",
+                                                "value": 0.0,
+                                            },
+                                        ]
+                                    },
+                                    "feedin_percent_amt": {
+                                        "windows": [
+                                            {
+                                                "start_time": "00:00",
+                                                "duration": "24 hours",
+                                                "value": 0.0,
+                                            },
+                                        ]
+                                    },
+                                },
+                            },
+                        }
+                    )
+                    # Retry
+                    continue
             try:
                 array = await cls.prediction.key_to_array(
                     key="pvforecast_ac_power",
@@ -392,7 +456,7 @@ class Genetic0OptimizationParameters(
                     {
                         "elecprice": {
                             "elecpricefixed": {
-                                "time_windows": {
+                                "elecprice_marketprice_amt_kwh": {
                                     "windows": [
                                         {
                                             "duration": "1 day",
@@ -455,7 +519,15 @@ class Genetic0OptimizationParameters(
                         "feedintariff": {
                             "provider": "FeedInTariffFixed",
                             "feedintarifffixed": {
-                                "feed_in_tariff_kwh": 0.078,
+                                "feed_in_tariff_amt_kwh": {
+                                    "windows": [
+                                        {
+                                            "start_time": "00:00",
+                                            "duration": "24 hours",
+                                            "value": 0.078,
+                                        },
+                                    ],
+                                },
                             },
                         },
                     }

src/akkudoktoreos/prediction/elecfee.py

@@ -0,0 +1,65 @@
+from typing import Optional
+
+from pydantic import Field, computed_field, field_validator
+
+from akkudoktoreos.config.configabc import SettingsBaseModel
+from akkudoktoreos.core.coreabc import get_prediction
+from akkudoktoreos.prediction.elecfeeabc import ElecFeeProvider
+from akkudoktoreos.prediction.elecfeefixed import ElecFeeFixedCommonSettings
+from akkudoktoreos.prediction.elecfeeimport import ElecFeeImportCommonSettings
+
+
+def elecfee_provider_ids() -> list[str]:
+    """Valid elecfee provider ids."""
+    try:
+        prediction_eos = get_prediction()
+    except Exception:
+        # Prediction may not be initialized. Return static built-in provider ids.
+        return [
+            "ElecFeeFixed",
+            "ElecFeeImport",
+        ]
+
+    return [
+        provider.provider_id()
+        for provider in prediction_eos.providers
+        if isinstance(provider, ElecFeeProvider)
+    ]
+
+
+class ElecFeeCommonSettings(SettingsBaseModel):
+    """Electricity Price Prediction Configuration."""
+
+    provider: Optional[str] = Field(
+        default=None,
+        json_schema_extra={
+            "description": "Electricity fee provider id of provider to be used.",
+            "examples": ["ElecFeeFixed"],
+        },
+    )
+
+    elecfeefixed: ElecFeeFixedCommonSettings = Field(
+        default_factory=ElecFeeFixedCommonSettings,
+        json_schema_extra={"description": "Fixed electricity fees provider settings."},
+    )
+
+    elecfeeimport: ElecFeeImportCommonSettings = Field(
+        default_factory=ElecFeeImportCommonSettings,
+        json_schema_extra={"description": "Electricity fees import provider settings."},
+    )
+
+    @computed_field  # type: ignore[prop-decorator]
+    @property
+    def providers(self) -> list[str]:
+        """Available electricity fee provider ids."""
+        return elecfee_provider_ids()
+
+    # Validators
+    @field_validator("provider", mode="after")
+    @classmethod
+    def validate_provider(cls, value: Optional[str]) -> Optional[str]:
+        if value is None or value in elecfee_provider_ids():
+            return value
+        raise ValueError(
+            f"Provider '{value}' is not a valid electricity fees provider: {elecfee_provider_ids()}."
+        )

src/akkudoktoreos/prediction/elecfeeabc.py

@@ -0,0 +1,128 @@
+"""Abstract and base classes for electricity fee predictions.
+
+Notes:
+    - Ensure appropriate API keys or configurations are set up if required by external data sources.
+"""
+
+from abc import abstractmethod
+from typing import List, Optional
+
+from pydantic import Field, computed_field
+
+from akkudoktoreos.prediction.predictionabc import PredictionProvider, PredictionRecord
+
+
+class ElecFeeDataRecord(PredictionRecord):
+    """Represents a electricity price data record containing various price attributes at a specific datetime.
+
+    Attributes:
+        date_time (Optional[AwareDatetime]): The datetime of the record.
+
+    """
+
+    elecfee_consumption_amt_wh: Optional[float] = Field(
+        None,
+        json_schema_extra={
+            "description": (
+                "Total fixed fee for consumed energy per Wh [amount/Wh]. "
+                "This is the accumulation of all fixed per-Wh fees payable on "
+                "consumed energy - such as network charge, concession fee, "
+                "and electricity charge - into a single amount."
+            ),
+        },
+    )
+
+    elecfee_consumption_percent_amt: Optional[float] = Field(
+        None,
+        json_schema_extra={
+            "description": (
+                "Total fixed surcharge on consumed energy, given as a "
+                "percentage of the monetary amount already charged for that "
+                "energy [%]. This is the accumulation of all percentage-based "
+                "surcharges payable on top of the consumed-energy fee - such "
+                "as VAT - into a single percentage. This is a percentage of "
+                "the fee amount, not a per-Wh rate."
+            ),
+        },
+    )
+
+    elecfee_feedin_amt_wh: Optional[float] = Field(
+        None,
+        json_schema_extra={
+            "description": (
+                "Total fixed deduction from feed-in energy per Wh [amount/Wh]. "
+                "This is the accumulation of all fixed per-Wh charges deducted from "
+                "feed-in energy - such as metering fees or grid-operator handling "
+                "charges - into a single amount. Applied after the percentage-based "
+                "deduction, i.e. it reduces the price by a flat amount per Wh "
+                "rather than by a share of the raw price."
+            ),
+        },
+    )
+
+    elecfee_feedin_percent_amt: Optional[float] = Field(
+        None,
+        json_schema_extra={
+            "description": (
+                "Total percentage deducted from the raw feed-in price (spot price) "
+                "[%]. This is the accumulation of all percentage-based deductions "
+                "payable on the feed-in tariff - such as a marketing or balancing "
+                "fee retained by the aggregator - into a single percentage. It is "
+                "applied as `raw_price * (100 - percent) / 100`, i.e. it scales "
+                "down the raw price rather than adding a surcharge to it."
+            ),
+        },
+    )
+
+    # Computed fields
+    @computed_field  # type: ignore[prop-decorator]
+    @property
+    def elecfee_consumption_amt_kwh(self) -> Optional[float]:
+        """Electricity fee for consumed energy per kWh [amount/kWh].
+
+        This is the aggregation of fees that are to be paid by consumed energy per kWh - "
+        like network charge, concession fee, electricity charge."
+
+        Convenience attribute calculated from `elecfee_consumption_amt_wh`.
+        """
+        if self.elecfee_consumption_amt_wh is None:
+            return None
+        return self.elecfee_consumption_amt_wh * 1000.0
+
+    @computed_field  # type: ignore[prop-decorator]
+    @property
+    def elecfee_feedin_amt_kwh(self) -> Optional[float]:
+        """Electricity fee for feed-in energy per kWh [amount/kWh].
+
+        This is the aggregation of fees that are to be paid by feed-in energy per kWh - "
+        like network charge, concession fee, electricity charge."
+
+        Convenience attribute calculated from `elecfee_feedin_amt_wh`.
+        """
+        if self.elecfee_feedin_amt_wh is None:
+            return None
+        return self.elecfee_feedin_amt_wh * 1000.0
+
+
+class ElecFeeProvider(PredictionProvider):
+    """Abstract base class for electricity fee providers.
+
+    Electricity fee providers predict fees on consumed and feed-in electricity to be used by
+    electricity and feed-in price providers.
+
+    ElecFeeProvider is a thread-safe singleton, ensuring only one instance of this class is created.
+    """
+
+    # overload
+    records: List[ElecFeeDataRecord] = Field(
+        default_factory=list,
+        json_schema_extra={"description": "List of ElecFeeDataRecord records"},
+    )
+
+    @classmethod
+    @abstractmethod
+    def provider_id(cls) -> str:
+        return "ElecFeeProvider"
+
+    def enabled(self) -> bool:
+        return self.provider_id() == self.config.elecfee.provider

src/akkudoktoreos/prediction/elecfeefixed.py

@@ -0,0 +1,189 @@
+"""Provides fixed fee electricity fee data."""
+
+from typing import Optional
+
+from loguru import logger
+from pydantic import Field
+
+from akkudoktoreos.config.configabc import (
+    SettingsBaseModel,
+    ValueTimeWindowSequence,
+)
+from akkudoktoreos.prediction.elecfeeabc import ElecFeeProvider
+from akkudoktoreos.utils.datetimeutil import to_duration
+
+
+class ElecFeeFixedCommonSettings(SettingsBaseModel):
+    """Common settings for fixed electricity fees.
+
+    This model defines a fixed electricity fee schedule using a sequence
+    of time windows. Each window specifies a time interval and the electricity
+    fee applicable during that interval.
+    """
+
+    consumption_amt_kwh: ValueTimeWindowSequence = Field(
+        default_factory=ValueTimeWindowSequence,
+        json_schema_extra={
+            "description": (
+                "Sequence of time windows defining the total fixed per-kWh electricty fee "
+                "charged for consumed energy, accumulating all applicable fixed "
+                "per-kWh charges (e.g. network charge, metering fee, concession "
+                "fee) into a single amount [amount/kWh]. If not provided, no fixed "
+                "per-kWh consumption fee is applied."
+            ),
+            "examples": [
+                {
+                    "windows": [
+                        {"start_time": "00:00", "duration": "8 hours", "value": 0.00288},
+                        {"start_time": "08:00", "duration": "16 hours", "value": 0.0034},
+                    ],
+                }
+            ],
+        },
+    )
+
+    consumption_percent_amt: ValueTimeWindowSequence = Field(
+        default_factory=ValueTimeWindowSequence,
+        json_schema_extra={
+            "description": (
+                "Sequence of time windows defining the total fixed electricity surcharge "
+                "applied as a percentage of the monetary amount already charged "
+                "for consumed energy, accumulating all applicable percentage-based "
+                "surcharges (e.g. VAT, electricity tax) into a single percentage "
+                "[%]. This is a percentage of the fee amount, not a per-kWh rate. "
+                "If not provided, no percentage-based consumption surcharge is "
+                "applied."
+            ),
+            "examples": [
+                {
+                    "windows": [
+                        {"start_time": "00:00", "duration": "24 hours", "value": 19.0},
+                    ],
+                }
+            ],
+        },
+    )
+
+    feedin_amt_kwh: ValueTimeWindowSequence = Field(
+        default_factory=ValueTimeWindowSequence,
+        json_schema_extra={
+            "description": (
+                "Sequence of time windows defining the total deduction from feed-in energy "
+                "per Wh [amount/Wh]. This is the accumulation of all fixed per-Wh charges "
+                "deducted from feed-in energy - such as metering fees or grid-operator handling "
+                "charges - into a single amount. Applied after the percentage-based "
+                "deduction, i.e. it reduces the price by a flat amount per Wh "
+                "rather than by a share of the raw price. If not provided, no fixed per-kWh "
+                "feed-in fee is applied."
+            ),
+            "examples": [
+                {
+                    "windows": [
+                        {"start_time": "00:00", "duration": "8 hours", "value": 0.00288},
+                        {"start_time": "08:00", "duration": "16 hours", "value": 0.0034},
+                    ],
+                }
+            ],
+        },
+    )
+
+    feedin_percent_amt: ValueTimeWindowSequence = Field(
+        default_factory=ValueTimeWindowSequence,
+        json_schema_extra={
+            "description": (
+                "Sequence of time windows defining the total percentage deducted from the raw "
+                "feed-in price (spot price) [%]. This is the accumulation of all percentage-based "
+                "deductions payable on the feed-in tariff - such as a marketing or balancing "
+                "fee retained by the aggregator - into a single percentage. It is "
+                "applied as `raw_price * (100 - percent) / 100`, i.e. it scales "
+                "down the raw price rather than adding a surcharge to it. If not provided, no "
+                "percentage-based feed-in deduction is applied."
+            ),
+            "examples": [
+                {
+                    "windows": [
+                        {"start_time": "00:00", "duration": "24 hours", "value": 19.0},
+                    ],
+                }
+            ],
+        },
+    )
+
+
+class ElecFeeFixed(ElecFeeProvider):
+    """Fixed fee electricity fee data.
+
+    ElecFeeFixed is a singleton-based class that retrieves electricity fee data
+    from a fixed schedule defined by time windows.
+
+    The provider generates hourly electricity fees based on the configured time windows.
+    For each hour in the forecast period, it determines which time window applies and
+    assigns the corresponding fee.
+
+    Attributes:
+        time_windows: Sequence of time windows with associated electricity fees.
+    """
+
+    @classmethod
+    def provider_id(cls) -> str:
+        """Return the unique identifier for the ElecFeeFixed provider."""
+        return "ElecFeeFixed"
+
+    async def _update_data(self, force_update: Optional[bool] = False) -> None:
+        """Update electricity fee data from fixed schedule.
+
+        Generates electricity fees based on the configured time windows
+        at the optimization interval granularity. The fee sequence starts
+        synchronized to the wall clock at the next full interval boundary.
+
+        Args:
+            force_update: If True, forces update even if data exists.
+
+        Raises:
+            ValueError: If no time windows are configured.
+        """
+        elecfee_spec: dict[str, ValueTimeWindowSequence] = {
+            "elecfee_consumption_amt_wh": self.config.elecfee.elecfeefixed.consumption_amt_kwh,
+            "elecfee_consumption_percent_amt": self.config.elecfee.elecfeefixed.consumption_percent_amt,
+            "elecfee_feedin_amt_wh": self.config.elecfee.elecfeefixed.feedin_amt_kwh,
+            "elecfee_feedin_percent_amt": self.config.elecfee.elecfeefixed.feedin_percent_amt,
+        }
+
+        for prediction_key, time_window_seq in elecfee_spec.items():
+            if time_window_seq is None or not time_window_seq.windows:
+                warning_msg = f"No time windows configured for `{prediction_key}`, defaulting to 0."
+                logger.warning(warning_msg)
+                await self.update_value(self.ems_start_datetime, prediction_key, 0.0)
+                continue
+
+            start_datetime = self.ems_start_datetime
+            interval_seconds = 900  # Usual smallest time interval (15 min) used in electricty fees
+            total_hours = self.config.prediction.hours
+            interval = to_duration(interval_seconds)
+
+            end_datetime = start_datetime.add(hours=total_hours)
+
+            logger.debug(
+                f"Generating `{prediction_key}` for {total_hours} hours "
+                f"starting at {start_datetime}"
+            )
+
+            # Build the full fee array in one call — kWh values aligned to the
+            # optimization grid.  to_series mirrors the key_to_series signature so
+            # the grid is constructed identically to how prediction data is read.
+            fees = time_window_seq.to_series(
+                start_datetime=start_datetime,
+                end_datetime=end_datetime,
+                interval=interval,
+                dropna=True,
+                boundary="context",
+                align_to_interval=True,
+            )
+
+            if prediction_key.endswith("_wh"):
+                # Convert kWh → Wh
+                fees = fees / 1000.0
+
+            await self.key_from_series(prediction_key, fees)
+
+            logger.debug(f"Successfully generated {len(fees)} `{prediction_key}` entries")

src/akkudoktoreos/prediction/elecfeeimport.py

@@ -0,0 +1,77 @@
+"""Retrieves elecfee forecast data from an import file.
+
+This module provides classes and mappings to manage elecfee data obtained from
+an import file. The data is mapped to the `ElecFeeDataRecord` format, enabling consistent access
+to forecasted and historical elecfee attributes.
+"""
+
+from pathlib import Path
+from typing import Optional, Union
+
+from pydantic import Field, field_validator
+
+from akkudoktoreos.config.configabc import SettingsBaseModel
+from akkudoktoreos.prediction.elecfeeabc import ElecFeeProvider
+from akkudoktoreos.prediction.predictionabc import PredictionImportProvider
+
+
+class ElecFeeImportCommonSettings(SettingsBaseModel):
+    """Common settings for elecfee data import from file or JSON String."""
+
+    import_file_path: Optional[Union[str, Path]] = Field(
+        default=None,
+        json_schema_extra={
+            "description": "Path to the file to import elecfee data from.",
+            "examples": [None, "/path/to/prices.json"],
+        },
+    )
+
+    import_json: Optional[str] = Field(
+        default=None,
+        json_schema_extra={
+            "description": "JSON string, dictionary of electricity fee forecast value lists.",
+            "examples": ['{"elecfee_consumption_amt_wh": [0.0003384, 0.0003318, 0.0003284]}'],
+        },
+    )
+
+    # Validators
+    @field_validator("import_file_path", mode="after")
+    @classmethod
+    def validate_import_file_path(cls, value: Optional[Union[str, Path]]) -> Optional[Path]:
+        if value is None:
+            return None
+        if isinstance(value, str):
+            value = Path(value)
+        """Ensure file is available."""
+        value.resolve()
+        if not value.is_file():
+            raise ValueError(f"Import file path '{value}' is not a file.")
+        return value
+
+
+class ElecFeeImport(ElecFeeProvider, PredictionImportProvider):
+    """Fetch PV forecast data from import file or JSON string.
+
+    ElecFeeImport is a singleton-based class that retrieves elecfee forecast data
+    from a file or JSON string and maps it to `ElecFeeDataRecord` fields. It manages the forecast
+    over a range of hours into the future and retains historical data.
+    """
+
+    @classmethod
+    def provider_id(cls) -> str:
+        """Return the unique identifier for the ElecFeeImport provider."""
+        return "ElecFeeImport"
+
+    async def _update_data(self, force_update: Optional[bool] = False) -> None:
+        # Both _sequence_lock and _record_lock are already held by the caller.
+        # Use internal sync methods only — never await public async counterparts.
+        if self.config.elecfee.elecfeeimport.import_file_path:
+            await self._import_from_file(
+                self.config.elecfee.elecfeeimport.import_file_path,
+                key_prefix="elecfee",
+            )
+        if self.config.elecfee.elecfeeimport.import_json:
+            await self._import_from_json(
+                self.config.elecfee.elecfeeimport.import_json,
+                key_prefix="elecfee",
+            )

src/akkudoktoreos/prediction/elecprice.py

@@ -5,11 +5,15 @@ from pydantic import Field, computed_field, field_validator
 from akkudoktoreos.config.configabc import SettingsBaseModel
 from akkudoktoreos.core.coreabc import get_prediction
 from akkudoktoreos.prediction.elecpriceabc import ElecPriceProvider
+from akkudoktoreos.prediction.elecpriceakkudoktor import (
+    ElecPriceAkkudoktorCommonSettings,
+)
 from akkudoktoreos.prediction.elecpriceenergycharts import (
     ElecPriceEnergyChartsCommonSettings,
 )
 from akkudoktoreos.prediction.elecpricefixed import ElecPriceFixedCommonSettings
 from akkudoktoreos.prediction.elecpriceimport import ElecPriceImportCommonSettings
+from akkudoktoreos.prediction.elecpricesmard import ElecPriceSMARDCommonSettings
 from akkudoktoreos.prediction.elecpricetibber import ElecPriceTibberCommonSettings
 
 
@@ -23,6 +27,7 @@ def elecprice_provider_ids() -> list[str]:
             "ElecPriceAkkudoktor",
             "ElecPriceEnergyCharts",
             "ElecPriceFixed",
+            "ElecPriceSMARD",
             "ElecPriceImport",
             "ElecPriceTibber",
         ]
@@ -45,22 +50,9 @@ class ElecPriceCommonSettings(SettingsBaseModel):
         },
     )
 
-    charges_kwh: Optional[float] = Field(
-        default=None,
-        ge=0,
-        json_schema_extra={
-            "description": "Electricity price charges [amount/kWh]. Will be added to variable market price.",
-            "examples": [0.21],
-        },
-    )
-
-    vat_rate: Optional[float] = Field(
-        default=1.19,
-        ge=0,
-        json_schema_extra={
-            "description": "VAT rate factor applied to electricity price when charges are used.",
-            "examples": [1.19],
-        },
+    akkudoktor: ElecPriceAkkudoktorCommonSettings = Field(
+        default_factory=ElecPriceAkkudoktorCommonSettings,
+        json_schema_extra={"description": "Akkudoktor electricity price provider settings."},
     )
 
     elecpricefixed: ElecPriceFixedCommonSettings = Field(
@@ -78,6 +70,11 @@ class ElecPriceCommonSettings(SettingsBaseModel):
         json_schema_extra={"description": "Energy Charts provider settings."},
     )
 
+    smard: ElecPriceSMARDCommonSettings = Field(
+        default_factory=ElecPriceSMARDCommonSettings,
+        json_schema_extra={"description": "SMARD electricity price provider settings."},
+    )
+
     tibber: ElecPriceTibberCommonSettings = Field(
         default_factory=ElecPriceTibberCommonSettings,
         json_schema_extra={"description": "Tibber electricity price provider settings."},

src/akkudoktoreos/prediction/elecpriceabc.py

@@ -7,9 +7,11 @@ Notes:
 from abc import abstractmethod
 from typing import List, Optional
 
+import pandas as pd
 from pydantic import Field, computed_field
 
-from akkudoktoreos.prediction.predictionabc import PredictionProvider, PredictionRecord
+from akkudoktoreos.prediction.predictionabc import PredictionRecord
+from akkudoktoreos.prediction.priceabc import PricePredictionProviderBase
 
 
 class ElecPriceDataRecord(PredictionRecord):
@@ -20,8 +22,18 @@ class ElecPriceDataRecord(PredictionRecord):
 
     """
 
+    elecprice_marketprice_raw_wh: Optional[float] = Field(
+        None,
+        json_schema_extra={
+            "description": "Raw electricity market price per Wh, always excluding fees [amount/Wh]"
+        },
+    )
+
     elecprice_marketprice_wh: Optional[float] = Field(
-        None, json_schema_extra={"description": "Electricity market price per Wh [amount/Wh]"}
+        None,
+        json_schema_extra={
+            "description": "Electricity market price per Wh, including fees if configured [amount/Wh]"
+        },
     )
 
     # Computed fields
@@ -37,24 +49,10 @@ class ElecPriceDataRecord(PredictionRecord):
         return self.elecprice_marketprice_wh * 1000.0
 
 
-class ElecPriceProvider(PredictionProvider):
+class ElecPriceProvider(PricePredictionProviderBase):
     """Abstract base class for electricity price providers.
 
-    WeatherProvider is a thread-safe singleton, ensuring only one instance of this class is created.
-
-    Configuration variables:
-        electricity price_provider (str): Prediction provider for electricity price.
-
-    Attributes:
-        hours (int, optional): The number of hours into the future for which predictions are generated.
-        historic_hours (int, optional): The number of past hours for which historical data is retained.
-        latitude (float, optional): The latitude in degrees, must be within -90 to 90.
-        longitude (float, optional): The longitude in degrees, must be within -180 to 180.
-        start_datetime (datetime, optional): The starting datetime for predictions, defaults to the current datetime if unspecified.
-        end_datetime (datetime, computed): The datetime representing the end of the prediction range,
-            calculated based on `start_datetime` and `hours`.
-        keep_datetime (datetime, computed): The earliest datetime for retaining historical data, calculated
-            based on `start_datetime` and `historic_hours`.
+    ElecPriceProvider is a thread-safe singleton, ensuring only one instance of this class is created.
     """
 
     # overload
@@ -70,3 +68,45 @@ class ElecPriceProvider(PredictionProvider):
 
     def enabled(self) -> bool:
         return self.provider_id() == self.config.elecprice.provider
+
+    # --- PricePredictionProviderBase hooks -------------------------------
+    #
+    # Concrete for every electricity-price data source: the raw/gross record
+    # keys, the consumption-fee keys, and the fee formula itself don't vary
+    # by provider (ElecPriceEnergyCharts, ElecPriceAkkudoktor, ...), only by
+    # "electricity price" vs. "feed-in tariff" - so, unlike `provider_id`,
+    # these are NOT left abstract for concrete providers to fill in.
+
+    @property
+    def _raw_key(self) -> str:
+        """Record key holding the fee-free raw series."""
+        return "elecprice_marketprice_raw_wh"
+
+    @property
+    def _gross_key(self) -> str:
+        """Record key to write the fee-inclusive series to."""
+        return "elecprice_marketprice_wh"
+
+    @property
+    def _fee_keys(self) -> list[str]:
+        """Prediction keys to fetch for fee computation."""
+        return ["elecfee_consumption_amt_wh", "elecfee_consumption_percent_amt"]
+
+    def _compute_gross(self, raw_amt_wh: pd.Series, df_fee: pd.DataFrame) -> pd.Series:
+        """Add the per-Wh consumption fee, then apply the percent surcharge (e.g. VAT).
+
+        gross = (raw + elecfee_consumption_amt_wh) * (100 + elecfee_consumption_percent_amt) / 100
+
+        Args:
+            raw_amt_wh: Raw electricity market price (amount/Wh), fee-free.
+            df_fee: Fee dataframe aligned to `raw_amt_wh`'s index, with columns
+                matching `_fee_keys`.
+
+        Returns:
+            pd.Series: Gross electricity price (amount/Wh), same index as `raw_amt_wh`.
+        """
+        return (
+            (raw_amt_wh + df_fee["elecfee_consumption_amt_wh"])
+            * (100.0 + df_fee["elecfee_consumption_percent_amt"])
+            / 100.0
+        )

src/akkudoktoreos/prediction/elecpriceakkudoktor.py

@@ -8,17 +8,20 @@ format, enabling consistent access to forecasted and historical electricity pric
 
 from typing import Any, List, Optional, Union
 
-import numpy as np
 import pandas as pd
 import requests
 from loguru import logger
 from pydantic import ValidationError
-from statsmodels.tsa.holtwinters import ExponentialSmoothing
 
+from akkudoktoreos.config.configabc import SettingsBaseModel
 from akkudoktoreos.core.cache import cache_in_file
 from akkudoktoreos.core.pydantic import PydanticBaseModel
 from akkudoktoreos.prediction.elecpriceabc import ElecPriceProvider
-from akkudoktoreos.utils.datetimeutil import to_datetime, to_duration
+from akkudoktoreos.utils.datetimeutil import DateTime, to_datetime, to_duration
+
+
+class ElecPriceAkkudoktorCommonSettings(SettingsBaseModel):
+    """Common configuration settings for Akkodoktor electricity pricing."""
 
 
 class AkkudoktorElecPriceMeta(PydanticBaseModel):
@@ -64,6 +67,8 @@ class ElecPriceAkkudoktor(ElecPriceProvider):
         _update_data(): Processes and updates forecast data from Akkudoktor in ElecPriceDataRecord format.
     """
 
+    highest_orig_datetime: Optional[DateTime] = None
+
     @classmethod
     def provider_id(cls) -> str:
         """Return the unique identifier for the Akkudoktor provider."""
@@ -116,28 +121,7 @@ class ElecPriceAkkudoktor(ElecPriceProvider):
         self.update_datetime = to_datetime(in_timezone=self.config.general.timezone)
         return akkudoktor_data
 
-    def _cap_outliers(self, data: np.ndarray, sigma: int = 2) -> np.ndarray:
-        mean = data.mean()
-        std = data.std()
-        lower_bound = mean - sigma * std
-        upper_bound = mean + sigma * std
-        capped_data = data.clip(min=lower_bound, max=upper_bound)
-        return capped_data
-
-    def _predict_ets(self, history: np.ndarray, seasonal_periods: int, hours: int) -> np.ndarray:
-        clean_history = self._cap_outliers(history)
-        model = ExponentialSmoothing(
-            clean_history, seasonal="add", seasonal_periods=seasonal_periods
-        ).fit()
-        return model.forecast(hours)
-
-    def _predict_median(self, history: np.ndarray, hours: int) -> np.ndarray:
-        clean_history = self._cap_outliers(history)
-        return np.full(hours, np.median(clean_history))
-
-    async def _update_data(
-        self, force_update: Optional[bool] = False
-    ) -> None:  # tuple[np.ndarray, np.ndarray, np.ndarray]:
+    async def _update_data(self, force_update: Optional[bool] = False) -> None:
         """Update forecast data in the ElecPriceDataRecord format.
 
         Retrieves data from Akkudoktor, maps each Akkudoktor field to the corresponding
@@ -146,64 +130,81 @@ class ElecPriceAkkudoktor(ElecPriceProvider):
         The final mapped and processed data is inserted into the sequence as `ElecPriceDataRecord`.
         """
         # Get Akkudoktor electricity price data
-        akkudoktor_data = self._request_forecast(force_update=force_update)  # type: ignore
         if not self.ems_start_datetime:
             raise ValueError(f"Start DateTime not set: {self.ems_start_datetime}")
 
+        akkudoktor_data = self._request_forecast(force_update=force_update)  # type: ignore
+
         # Assumption that all lists are the same length and are ordered chronologically
         # in ascending order and have the same timestamps.
 
-        # Get charges_kwh in wh
-        charges_wh = (self.config.elecprice.charges_kwh or 0) / 1000
-
         highest_orig_datetime = None  # newest datetime from the api after that we want to update.
-        series_data = pd.Series(dtype=float)  # Initialize an empty series
+        prices_wh = pd.Series(dtype=float)  # Initialize an empty series
 
         for value in akkudoktor_data.values:
             orig_datetime = to_datetime(value.start, in_timezone=self.config.general.timezone)
             if highest_orig_datetime is None or orig_datetime > highest_orig_datetime:
                 highest_orig_datetime = orig_datetime
 
-            price_wh = value.marketpriceEurocentPerKWh / (100 * 1000) + charges_wh
+            price_wh = value.marketpriceEurocentPerKWh / (100 * 1000)
 
             # Collect all values into the Pandas Series
-            series_data.at[orig_datetime] = price_wh
+            prices_wh.at[orig_datetime] = price_wh
 
-        # Update values using key_from_series
-        await self.key_from_series("elecprice_marketprice_wh", series_data)
-
-        # Generate history array for prediction
-        history = await self.key_to_array(
-            key="elecprice_marketprice_wh", end_datetime=highest_orig_datetime, fill_method="linear"
-        )
-
-        amount_datasets = len(self.records)
         if not highest_orig_datetime:  # mypy fix
             error_msg = f"Highest original datetime not available: {highest_orig_datetime}"
             logger.error(error_msg)
             raise ValueError(error_msg)
+        self.highest_orig_datetime = highest_orig_datetime
 
-        # some of our data is already in the future, so we need to predict less. If we got less data we increase the prediction hours
+        # Every cycle fetches fresh data (no skip-fetch branch here, unlike
+        # ElecPriceEnergyCharts), so the gross-series recompute always needs to
+        # cover at least the freshly fetched range, in addition to the
+        # forward-looking window from ems_start_datetime.
+        gross_start_datetime = (
+            min(self.ems_start_datetime, prices_wh.index.min())
+            if not prices_wh.empty
+            else self.ems_start_datetime
+        )
+
+        # Always raw here - fees are applied once, later, over the complete
+        # raw+predicted series in _store_gross_series().
+        await self.key_from_series("elecprice_marketprice_raw_wh", prices_wh)
+
+        # Raw history only, so ETS/median always trains on the true
+        # wholesale-price signal, never on fee-inclusive values.
+        history = await self.key_to_array(
+            key="elecprice_marketprice_raw_wh",
+            end_datetime=highest_orig_datetime,
+            fill_method="linear",
+        )
+
+        # Some of our data is already in the future, so we need to predict less.
+        # If we got less data we increase the prediction hours
         needed_hours = int(
             self.config.prediction.hours
             - ((highest_orig_datetime - self.ems_start_datetime).total_seconds() // 3600)
         )
 
         if needed_hours <= 0:
+            # This might keep data longer than
+            # self.ems_start_datetime + self.config.prediction.hours in the records
             logger.warning(
-                f"No prediction needed. needed_hours={needed_hours}, hours={self.config.prediction.hours},highest_orig_datetime {highest_orig_datetime}, start_datetime {self.ems_start_datetime}"
-            )  # this might keep data longer than self.ems_start_datetime + self.config.prediction.hours in the records
+                f"No prediction needed. needed_hours={needed_hours}, "
+                f"hours={self.config.prediction.hours}, "
+                f"highest_orig_datetime {highest_orig_datetime}, "
+                f"start_datetime {self.ems_start_datetime}"
+            )
+            # Fee schedule may have changed since the last run even without new
+            # market data; recompute gross only for the window that was
+            # actually touched (or is still forward-looking) this cycle.
+            await self._store_gross_series(
+                start_datetime=gross_start_datetime,
+                end_datetime=self.highest_orig_datetime + to_duration("1 second"),
+            )
             return
 
-        if amount_datasets > 800:  # we do the full ets with seasons of 1 week
-            prediction = self._predict_ets(history, seasonal_periods=168, hours=needed_hours)
-        elif amount_datasets > 168:  # not enough data to do seasons of 1 week, but enough for 1 day
-            prediction = self._predict_ets(history, seasonal_periods=24, hours=needed_hours)
-        elif amount_datasets > 0:  # not enough data for ets, do median
-            prediction = self._predict_median(history, hours=needed_hours)
-        else:
-            logger.error("No data available for prediction")
-            raise ValueError("No data available")
+        prediction = self._predict(history, needed_hours)
 
         # write predictions into the records, update if exist.
         prediction_series = pd.Series(
@@ -213,46 +214,12 @@ class ElecPriceAkkudoktor(ElecPriceProvider):
                 for i in range(len(prediction))
             ],
         )
-        await self.key_from_series("elecprice_marketprice_wh", prediction_series)
+        await self.key_from_series("elecprice_marketprice_raw_wh", prediction_series)
 
-        # history2 = await self.key_to_array(key="elecprice_marketprice_wh", fill_method="linear") + 0.0002
-        # return history, history2, prediction  # for debug main
-
-
-"""
-def visualize_predictions(
-    history: np.ndarray[Any, Any],
-    history2: np.ndarray[Any, Any],
-    predictions: np.ndarray[Any, Any],
-) -> None:
-    import matplotlib.pyplot as plt
-
-    plt.figure(figsize=(28, 14))
-    plt.plot(range(len(history)), history, label="History", color="green")
-    plt.plot(range(len(history2)), history2, label="History_new", color="blue")
-    plt.plot(
-        range(len(history), len(history) + len(predictions)),
-        predictions,
-        label="Predictions",
-        color="red",
-    )
-    plt.title("Predictions ets")
-    plt.xlabel("Time")
-    plt.ylabel("Price")
-    plt.legend()
-    plt.savefig("predictions_vs_true.png")
-    plt.close()
-
-
-def main() -> None:
-    # Initialize ElecPriceAkkudoktor with required parameters
-    elec_price_akkudoktor = ElecPriceAkkudoktor()
-    history, history2, predictions = elec_price_akkudoktor._update_data()
-
-    visualize_predictions(history, history2, predictions)
-    # print(history, history2, predictions)
-
-
-if __name__ == "__main__":
-    main()
-"""
+        # Bounded to [gross_start_datetime, end of the freshly predicted tail) -
+        # covers exactly what was fetched and/or predicted this cycle, not the
+        # entire (potentially multi-year) retained history.
+        await self._store_gross_series(
+            start_datetime=gross_start_datetime,
+            end_datetime=prediction_series.index.max() + to_duration("1 second"),
+        )

src/akkudoktoreos/prediction/elecpriceenergycharts.py

@@ -6,22 +6,20 @@ humidity, cloud cover, and solar irradiance. The data is mapped to the `ElecPric
 format, enabling consistent access to forecasted and historical electricity price attributes.
 """
 
-from datetime import datetime
+import time
 from enum import StrEnum
 from typing import Any, List, Optional, Union
 
-import numpy as np
 import pandas as pd
 import requests
 from loguru import logger
 from pydantic import Field, ValidationError
-from statsmodels.tsa.holtwinters import ExponentialSmoothing
 
 from akkudoktoreos.config.configabc import SettingsBaseModel
 from akkudoktoreos.core.cache import cache_in_file
 from akkudoktoreos.core.pydantic import PydanticBaseModel
 from akkudoktoreos.prediction.elecpriceabc import ElecPriceProvider
-from akkudoktoreos.utils.datetimeutil import to_datetime, to_duration
+from akkudoktoreos.utils.datetimeutil import DateTime, to_datetime, to_duration
 
 
 class EnergyChartsBiddingZones(StrEnum):
@@ -89,7 +87,11 @@ class ElecPriceEnergyCharts(ElecPriceProvider):
         _update_data(): Processes and updates forecast data from Energy-Charts in ElecPriceDataRecord format.
     """
 
-    highest_orig_datetime: Optional[datetime] = None
+    highest_orig_datetime: Optional[DateTime] = None
+
+    def historic_hours_min(self) -> int:
+        """Keep enough history for weekly seasonal price extrapolation."""
+        return 24 * 35
 
     @classmethod
     def provider_id(cls) -> str:
@@ -112,6 +114,15 @@ class ElecPriceEnergyCharts(ElecPriceProvider):
             raise ValueError(error_msg)
         return energy_charts_data
 
+    def _bidding_zone(self) -> str:
+        settings = self.config.elecprice.energycharts
+        if settings is None:
+            return EnergyChartsBiddingZones.DE_LU.value
+        bidding_zone = settings.bidding_zone
+        if isinstance(bidding_zone, EnergyChartsBiddingZones):
+            return bidding_zone.value
+        return str(bidding_zone)
+
     @cache_in_file(with_ttl="1 hour")
     def _request_forecast(self, start_date: Optional[str] = None) -> EnergyChartsElecPrice:
         """Fetch electricity price forecast data from Energy-Charts API.
@@ -134,25 +145,43 @@ class ElecPriceEnergyCharts(ElecPriceProvider):
 
         last_date = to_datetime(self.end_datetime, as_string="YYYY-MM-DD")
         bidding_zone = str(self.config.elecprice.energycharts.bidding_zone)
-        url = f"{source}/price?bzn={bidding_zone}&start={start_date}&end={last_date}"
-        response = requests.get(url, timeout=30)
-        logger.debug(f"Response from {url}: {response}")
-        response.raise_for_status()  # Raise an error for bad responses
-        energy_charts_data = self._validate_data(response.content)
-        # We are working on fresh data (no cache), report update time
-        self.update_datetime = to_datetime(in_timezone=self.config.general.timezone)
-        return energy_charts_data
+        url = f"{source}/price?bzn={self._bidding_zone()}&start={start_date}&end={last_date}"
 
-    def _parse_data(self, energy_charts_data: EnergyChartsElecPrice) -> pd.Series:
+        # Retry transient network problems (timeouts / connection resets) a few
+        # times with a short backoff. Uses a (connect, read) timeout tuple so a
+        # slow-to-respond API does not block forever but also is not aborted
+        # after a too-short single read window.
+        max_attempts = 3
+        last_exc: Optional[Exception] = None
+        for attempt in range(1, max_attempts + 1):
+            try:
+                response = requests.get(url, timeout=(5, 60))
+                logger.debug(f"Response from {url}: {response}")
+                response.raise_for_status()
+                energy_charts_data = ElecPriceEnergyCharts._validate_data(response.content)
+                self.update_datetime = to_datetime(in_timezone=self.config.general.timezone)
+                return energy_charts_data
+            except (requests.exceptions.Timeout, requests.exceptions.ConnectionError) as exc:
+                last_exc = exc
+                logger.warning(
+                    "Energy-Charts request attempt {}/{} failed: {}",
+                    attempt,
+                    max_attempts,
+                    exc,
+                )
+                if attempt < max_attempts:
+                    time.sleep(2 * attempt)
+        # All attempts exhausted - re-raise the last transient error so the
+        # caller (_update_data) can decide whether to fall back to history.
+        raise last_exc  # type: ignore[misc]
+
+    async def _parse_data(self, energy_charts_data: EnergyChartsElecPrice) -> pd.Series:
         # Assumption that all lists are the same length and are ordered chronologically
         # in ascending order and have the same timestamps.
 
-        # Get charges_kwh in wh
-        charges_wh = (self.config.elecprice.charges_kwh or 0) / 1000
-
         # Initialize
         highest_orig_datetime = None  # newest datetime from the api after that we want to update.
-        series_data = pd.Series(dtype=float)  # Initialize an empty series
+        prices_wh = pd.Series(dtype=float)  # Initialize an empty series
 
         # Iterate over timestamps and prices together
         for unix_sec, price_eur_per_mwh in zip(
@@ -164,40 +193,17 @@ class ElecPriceEnergyCharts(ElecPriceProvider):
             if highest_orig_datetime is None or orig_datetime > highest_orig_datetime:
                 highest_orig_datetime = orig_datetime
 
-            # Convert EUR/MWh to EUR/Wh, apply charges and VAT if charges > 0
-            if charges_wh > 0:
-                vat_rate = self.config.elecprice.vat_rate or 1.19
-                price_wh = ((price_eur_per_mwh / 1_000_000) + charges_wh) * vat_rate
-            else:
-                price_wh = price_eur_per_mwh / 1_000_000
+            # Convert EUR/MWh to EUR/Wh
+            price_wh = price_eur_per_mwh / 1_000_000
 
             # Store in series
-            series_data.at[orig_datetime] = price_wh
+            prices_wh.at[orig_datetime] = price_wh
 
-        return series_data
+        # Always raw here — fees are applied once, later, over the complete
+        # raw+predicted series in _store_gross_series().
+        return prices_wh
 
-    def _cap_outliers(self, data: np.ndarray, sigma: int = 2) -> np.ndarray:
-        mean = data.mean()
-        std = data.std()
-        lower_bound = mean - sigma * std
-        upper_bound = mean + sigma * std
-        capped_data = data.clip(min=lower_bound, max=upper_bound)
-        return capped_data
-
-    def _predict_ets(self, history: np.ndarray, seasonal_periods: int, hours: int) -> np.ndarray:
-        clean_history = self._cap_outliers(history)
-        model = ExponentialSmoothing(
-            clean_history, seasonal="add", seasonal_periods=seasonal_periods
-        ).fit()
-        return model.forecast(hours)
-
-    def _predict_median(self, history: np.ndarray, hours: int) -> np.ndarray:
-        clean_history = self._cap_outliers(history)
-        return np.full(hours, np.median(clean_history))
-
-    async def _update_data(
-        self, force_update: Optional[bool] = False
-    ) -> None:  # tuple[np.ndarray, np.ndarray, np.ndarray]:
+    async def _update_data(self, force_update: Optional[bool] = False) -> None:
         """Update forecast data in the ElecPriceDataRecord format.
 
         Retrieves data from Energy-Charts, maps each Energy-Charts field to the corresponding
@@ -214,83 +220,158 @@ class ElecPriceEnergyCharts(ElecPriceProvider):
         if not self.ems_start_datetime:
             raise ValueError(f"Start DateTime not set: {self.ems_start_datetime}")
 
-        # Determine if update is needed and how many days
-        past_days = 35
-        if self.highest_orig_datetime:
-            history_series = await self.key_to_raw_series(
-                key="elecprice_marketprice_wh", start_datetime=self.ems_start_datetime
-            )
-            # If history lower, then start_datetime
-            if history_series.index.min() <= self.ems_start_datetime:
-                past_days = 0
+        # Lower bound for the gross-series recompute below: defaults to "from
+        # now", i.e. only the still-relevant forward-looking window, unless a
+        # fresh fetch widens it to cover newly-arrived historic data too.
+        gross_start_datetime = self.ems_start_datetime
 
-            needs_update = end > self.highest_orig_datetime
+        # Set default start_datetime - try to take data from 5 weeks back for prediction
+        past_days = 35
+        start_datetime = self.ems_start_datetime - to_duration(f"{past_days} days")
+
+        # Determine if update is needed and what start date is really necessary
+        needs_update = False
+        if self.highest_orig_datetime:
+            raw_history = await self.key_to_raw_series(
+                key="elecprice_marketprice_raw_wh",
+                start_datetime=start_datetime,
+                end_datetime=gross_start_datetime,
+            )
+
+            if raw_history.empty:
+                # We need the default start date (35 days in past)
+                needs_update = True
+            else:
+                # A later update must not mistake the current forecast window for
+                # sufficient ETS history. Require the same amount of data that the
+                # weekly prediction branch in _predict needs; otherwise fetch 35
+                # days again and repair an already-truncated in-memory history.
+                resolution_seconds = self._resolution_seconds(raw_history)
+                slots_per_hour = 3600 // resolution_seconds
+                if len(raw_history) <= 2 * 168 * slots_per_hour:
+                    # Not enough slots in history, default start date
+                    needs_update = True
+                elif force_update:
+                    # Use default start date in case of forced update
+                    needs_update = True
+                elif end > self.highest_orig_datetime:
+                    # We got enough history, but still not enough data to prediction end
+                    start_datetime = gross_start_datetime
+                    needs_update = True
         else:
             needs_update = True
 
         if needs_update:
             logger.info(
-                f"Update ElecPriceEnergyCharts is needed, last in history: {self.highest_orig_datetime}"
+                "Update ElecPriceEnergyCharts is needed, last in history: {}, "
+                "force_update={}, start_datetime={}",
+                self.highest_orig_datetime,
+                bool(force_update),
+                start_datetime,
             )
-            # Set start_date try to take data from 5 weeks back for prediction
-            start_date = to_datetime(
-                self.ems_start_datetime - to_duration(f"{past_days} days"), as_string="YYYY-MM-DD"
-            )
-            # Get Energy-Charts electricity price data
-            energy_charts_data = self._request_forecast(
-                start_date=start_date, force_update=force_update
-            )  # type: ignore
 
-            # Parse and store data
-            series_data = self._parse_data(energy_charts_data)
-            self.highest_orig_datetime = series_data.index.max()
-            await self.key_from_series("elecprice_marketprice_wh", series_data)
+            # Get Energy-Charts electricity price data
+            try:
+                energy_charts_data = self._request_forecast(
+                    start_date=to_datetime(start_datetime, as_string="YYYY-MM-DD"),
+                    force_update=force_update,
+                )  # type: ignore
+
+                # Parse and store data
+                series_data = await self._parse_data(energy_charts_data)
+                if series_data.empty:
+                    raise ValueError("No Energy-Charts electricity price data available")
+                self.highest_orig_datetime = to_datetime(series_data.index.max())
+                await self.key_from_series("elecprice_marketprice_raw_wh", series_data)
+                # Newly fetched data widens the window that needs its gross
+                # (fee-inclusive) values recomputed.
+                gross_start_datetime = to_datetime(series_data.index.min())
+            except Exception as exc:
+                if self.highest_orig_datetime is None:
+                    raise
+                logger.warning(
+                    "Energy-Charts electricity price update failed ({}); keeping "
+                    "existing history until {} and extrapolating the remaining "
+                    "slots via ETS.",
+                    exc,
+                    self.highest_orig_datetime,
+                )
         else:
             logger.info(
-                f"No Update ElecPriceEnergyCharts is needed, last in history: {self.highest_orig_datetime}"
+                "No Update ElecPriceEnergyCharts is needed, last in history: {}",
+                self.highest_orig_datetime,
             )
 
-        # Generate history array for prediction
-        history = await self.key_to_array(
-            key="elecprice_marketprice_wh",
-            end_datetime=self.highest_orig_datetime,
-            fill_method="linear",
-        )
-
-        amount_datasets = len(self.records)
-        if not self.highest_orig_datetime:  # mypy fix
+        if not self.highest_orig_datetime:
             error_msg = f"Highest original datetime not available: {self.highest_orig_datetime}"
             logger.error(error_msg)
             raise ValueError(error_msg)
 
-        # some of our data is already in the future, so we need to predict less. If we got less data we increase the prediction hours
-        needed_hours = int(
-            self.config.prediction.hours
-            - ((self.highest_orig_datetime - self.ems_start_datetime).total_seconds() // 3600)
+        raw_series = await self.key_to_raw_series(
+            key="elecprice_marketprice_raw_wh",
+            end_datetime=to_datetime(self.highest_orig_datetime).add(seconds=1),
+        )
+        resolution_seconds = self._resolution_seconds(raw_series)
+        slots_per_hour = 3600 // resolution_seconds
+
+        # Raw history only. Guaranteed fee-free regardless of which branch ran
+        # above, so ETS/median always trains on the true wholesale-price signal.
+        history = await self.key_to_array(
+            key="elecprice_marketprice_raw_wh",
+            end_datetime=self.highest_orig_datetime,
+            interval=to_duration(f"{resolution_seconds} seconds"),
+            fill_method="linear",
         )
 
-        if needed_hours <= 0:
+        # Signed gap: positive when existing raw data already reaches past
+        # ems_start_datetime (fewer slots left to predict); negative when the
+        # newest known data point (highest_orig_datetime) is older than
+        # ems_start_datetime, e.g. after a fetch outage - in that case we need
+        # extra slots to also backfill the gap up to ems_start_datetime, on top
+        # of the full prediction.hours horizon beyond it.
+        covered_slots = int(
+            (self.highest_orig_datetime - self.ems_start_datetime).total_seconds()
+            // resolution_seconds
+        )
+        needed_slots = self.config.prediction.hours * slots_per_hour - covered_slots
+
+        if needed_slots <= 0:
+            # This might keep data longer than
+            # self.ems_start_datetime + self.config.prediction.hours in the records
             logger.warning(
-                f"No prediction needed. needed_hours={needed_hours}, hours={self.config.prediction.hours},highest_orig_datetime {self.highest_orig_datetime}, start_datetime {self.ems_start_datetime}"
-            )  # this might keep data longer than self.ems_start_datetime + self.config.prediction.hours in the records
+                "No electricity price prediction needed. needed_slots={}, hours={}, "
+                "resolution_seconds={}, highest_orig_datetime={}, start_datetime={}",
+                needed_slots,
+                self.config.prediction.hours,
+                resolution_seconds,
+                self.highest_orig_datetime,
+                self.ems_start_datetime,
+            )
+            # Fee schedule may have changed since the last run even without new
+            # market data; recompute gross only for the window that was
+            # actually touched (or is still forward-looking) this cycle.
+            await self._store_gross_series(
+                start_datetime=gross_start_datetime,
+                end_datetime=to_datetime(self.highest_orig_datetime).add(seconds=1),
+            )
             return
 
-        if amount_datasets > 800:  # we do the full ets with seasons of 1 week
-            prediction = self._predict_ets(history, seasonal_periods=168, hours=needed_hours)
-        elif amount_datasets > 168:  # not enough data to do seasons of 1 week, but enough for 1 day
-            prediction = self._predict_ets(history, seasonal_periods=24, hours=needed_hours)
-        elif amount_datasets > 0:  # not enough data for ets, do median
-            prediction = self._predict_median(history, hours=needed_hours)
-        else:
-            logger.error("No data available for prediction")
-            raise ValueError("No data available")
+        prediction = self._predict(history, needed_slots, slots_per_hour=slots_per_hour)
 
         # write predictions into the records, update if exist.
         prediction_series = pd.Series(
             data=prediction,
             index=[
-                self.highest_orig_datetime + to_duration(f"{i + 1} hours")
+                self.highest_orig_datetime + to_duration(f"{(i + 1) * resolution_seconds} seconds")
                 for i in range(len(prediction))
             ],
         )
-        await self.key_from_series("elecprice_marketprice_wh", prediction_series)
+        await self.key_from_series("elecprice_marketprice_raw_wh", prediction_series)
+
+        # Bounded to [gross_start_datetime, end of the freshly predicted tail) -
+        # covers exactly what was fetched and/or predicted this cycle, not the
+        # entire (potentially multi-year) retained history.
+        await self._store_gross_series(
+            start_datetime=gross_start_datetime,
+            end_datetime=to_datetime(prediction_series.index.max()) + to_duration("1 second"),
+        )

src/akkudoktoreos/prediction/elecpricefixed.py

@@ -10,7 +10,7 @@ from akkudoktoreos.config.configabc import (
     ValueTimeWindowSequence,
 )
 from akkudoktoreos.prediction.elecpriceabc import ElecPriceProvider
-from akkudoktoreos.utils.datetimeutil import to_duration
+from akkudoktoreos.utils.datetimeutil import DateTime, to_duration
 
 
 class ElecPriceFixedCommonSettings(SettingsBaseModel):
@@ -21,7 +21,7 @@ class ElecPriceFixedCommonSettings(SettingsBaseModel):
     price applicable during that interval.
     """
 
-    time_windows: ValueTimeWindowSequence = Field(
+    elecprice_marketprice_amt_kwh: ValueTimeWindowSequence = Field(
         default_factory=ValueTimeWindowSequence,
         json_schema_extra={
             "description": (
@@ -54,6 +54,8 @@ class ElecPriceFixed(ElecPriceProvider):
         time_windows: Sequence of time windows with associated electricity prices.
     """
 
+    highest_orig_datetime: Optional[DateTime] = None
+
     @classmethod
     def provider_id(cls) -> str:
         """Return the unique identifier for the ElecPriceFixed provider."""
@@ -72,40 +74,60 @@ class ElecPriceFixed(ElecPriceProvider):
         Raises:
             ValueError: If no time windows are configured.
         """
-        time_windows_seq = self.config.elecprice.elecpricefixed.time_windows
-
-        if time_windows_seq is None or not time_windows_seq.windows:
-            error_msg = "No time windows configured for fixed electricity price"
-            logger.error(error_msg)
-            raise ValueError(error_msg)
+        prediction_key = "elecprice_marketprice_wh"
+        raw_prediction_key = "elecprice_marketprice_raw_wh"
+        time_windows_seq: ValueTimeWindowSequence = (
+            self.config.elecprice.elecpricefixed.elecprice_marketprice_amt_kwh
+        )
 
         start_datetime = self.ems_start_datetime
         interval_seconds = 900  # Usual smallest time interval (15 min) used in electricty prices
         total_hours = self.config.prediction.hours
         interval = to_duration(interval_seconds)
-
         end_datetime = start_datetime.add(hours=total_hours)
 
+        if time_windows_seq is None or not time_windows_seq.windows:
+            warning_msg = f"No time windows configured for `{raw_prediction_key}`, defaulting to 0."
+            logger.warning(warning_msg)
+            # Store two values to have a default interval to be used by _apply_fees
+            end_datetime = start_datetime + interval
+            await self.update_value(start_datetime, raw_prediction_key, 0.0)
+            await self.update_value(end_datetime, raw_prediction_key, 0.0)
+            self.highest_orig_datetime = end_datetime
+            await self._store_gross_series(
+                start_datetime=start_datetime,
+                end_datetime=end_datetime + to_duration("1 second"),
+            )
+            return
+
         logger.debug(
-            f"Generating fixed electricity prices for {total_hours} hours "
-            f"starting at {start_datetime}"
+            f"Generating {raw_prediction_key} for {total_hours} hours starting at {start_datetime}"
         )
 
         # Build the full price array in one call — kWh values aligned to the
-        # optimization grid.  to_array mirrors the key_to_array signature so
+        # optimization grid.  to_series mirrors the key_to_series signature so
         # the grid is constructed identically to how prediction data is read.
-        prices_kwh = time_windows_seq.to_array(
+        prices_kwh = time_windows_seq.to_series(
             start_datetime=start_datetime,
             end_datetime=end_datetime,
             interval=interval,
-            dropna=True,
+            dropna=False,
             boundary="context",
             align_to_interval=True,
         )
 
-        # Convert kWh → Wh and store one entry per interval step.
-        for idx, price_kwh in enumerate(prices_kwh):
-            current_dt = start_datetime.add(seconds=idx * interval_seconds)
-            await self.update_value(current_dt, "elecprice_marketprice_wh", price_kwh / 1000.0)
+        # Convert kWh → Wh
+        prices_wh = prices_kwh / 1000.0
 
-        logger.debug(f"Successfully generated {len(prices_kwh)} fixed electricity price entries")
+        await self.key_from_series(raw_prediction_key, prices_wh)
+        self.highest_orig_datetime = prices_wh.index.max()
+
+        # Bounded to exactly the window just generated - covers the whole
+        # forecast horizon in one shot, since ElecPriceFixed has no
+        # fetch/predict split to worry about like the other providers.
+        await self._store_gross_series(
+            start_datetime=prices_wh.index.min(),
+            end_datetime=prices_wh.index.max() + to_duration("1 second"),
+        )
+
+        logger.debug(f"Successfully generated {len(prices_wh)} `{prediction_key}` entries")

src/akkudoktoreos/prediction/elecpriceimport.py

@@ -1,9 +1,8 @@
 """Retrieves elecprice forecast data from an import file.
 
 This module provides classes and mappings to manage elecprice data obtained from
-an import file, including support for various elecprice attributes such as temperature,
-humidity, cloud cover, and solar irradiance. The data is mapped to the `ElecPriceDataRecord`
-format, enabling consistent access to forecasted and historical elecprice attributes.
+an import file. The data is mapped to the `ElecPriceDataRecord` format, enabling consistent access
+to forecasted and historical elecprice attributes.
 """
 
 from pathlib import Path

src/akkudoktoreos/prediction/elecpricesmard.py

@@ -0,0 +1,188 @@
+"""Retrieve German day-ahead electricity prices directly from SMARD."""
+
+import time
+from datetime import datetime
+from typing import List, Optional
+
+import requests
+from loguru import logger
+from pydantic import Field, ValidationError
+
+from akkudoktoreos.config.configabc import SettingsBaseModel
+from akkudoktoreos.core.cache import cache_in_file
+from akkudoktoreos.core.pydantic import PydanticBaseModel
+from akkudoktoreos.prediction.elecpriceenergycharts import (
+    ElecPriceEnergyCharts,
+    EnergyChartsElecPrice,
+)
+from akkudoktoreos.utils.datetimeutil import to_datetime, to_duration
+
+SMARD_BASE_URL = "https://www.smard.de/app/chart_data"
+
+
+class SmardIndex(PydanticBaseModel):
+    """Available SMARD data-chunk timestamps."""
+
+    timestamps: List[int]
+
+
+class SmardChunkMetadata(PydanticBaseModel):
+    """Metadata included in a SMARD data chunk."""
+
+    version: int
+    created: int
+
+
+class SmardChunk(PydanticBaseModel):
+    """SMARD data chunk with millisecond timestamps and EUR/MWh values."""
+
+    meta_data: SmardChunkMetadata
+    series: List[tuple[int, Optional[float]]]
+
+
+class ElecPriceSMARDCommonSettings(SettingsBaseModel):
+    """Common settings for the direct SMARD electricity-price provider."""
+
+    filter_id: int = Field(
+        default=4169,
+        gt=0,
+        json_schema_extra={
+            "description": "SMARD filter id for the German/Luxembourg day-ahead price.",
+            "examples": [4169],
+        },
+    )
+
+    region: str = Field(
+        default="DE",
+        min_length=2,
+        json_schema_extra={
+            "description": "SMARD market region used in the chart-data endpoint.",
+            "examples": ["DE"],
+        },
+    )
+
+
+class ElecPriceSMARD(ElecPriceEnergyCharts):
+    """Fetch SMARD day-ahead prices and extend them with the seasonal EOS forecast.
+
+    The provider uses the public SMARD chart-data endpoint directly. It reuses the
+    Energy-Charts parsing and ETS pipeline after normalizing the response because both
+    sources expose the same EUR/MWh day-ahead market-price concept.
+    """
+
+    @classmethod
+    def provider_id(cls) -> str:
+        """Return the unique identifier for the direct SMARD provider."""
+        return "ElecPriceSMARD"
+
+    @classmethod
+    def _validate_index(cls, json_data: bytes) -> SmardIndex:
+        """Validate a SMARD chunk index response."""
+        try:
+            return SmardIndex.model_validate_json(json_data)
+        except ValidationError as exc:
+            logger.error("SMARD index schema change: {}", exc)
+            raise ValueError(f"SMARD index schema change: {exc}") from exc
+
+    @classmethod
+    def _validate_chunk(cls, json_data: bytes) -> SmardChunk:
+        """Validate a SMARD price chunk response."""
+        try:
+            return SmardChunk.model_validate_json(json_data)
+        except ValidationError as exc:
+            logger.error("SMARD price schema change: {}", exc)
+            raise ValueError(f"SMARD price schema change: {exc}") from exc
+
+    @staticmethod
+    def _get(url: str) -> bytes:
+        """Request a SMARD JSON resource with bounded retries."""
+        last_exc: Optional[Exception] = None
+        for attempt in range(1, 4):
+            try:
+                response = requests.get(
+                    url,
+                    headers={"User-Agent": "Akkudoktor-EOS/SMARD price provider"},
+                    timeout=(5, 30),
+                )
+                response.raise_for_status()
+                return response.content
+            except (requests.exceptions.Timeout, requests.exceptions.ConnectionError) as exc:
+                last_exc = exc
+                logger.warning("SMARD request attempt {}/3 failed for {}: {}", attempt, url, exc)
+                if attempt < 3:
+                    time.sleep(2 * attempt)
+        if last_exc is not None:
+            raise last_exc
+        raise RuntimeError(f"SMARD request failed without an exception: {url}")
+
+    def _chunk_timestamps(
+        self, index: SmardIndex, start_datetime: datetime, end_datetime: datetime
+    ) -> list[int]:
+        """Select all weekly chunks overlapping the requested datetime range."""
+        start_ms = int(to_datetime(start_datetime).timestamp() * 1000)
+        end_ms = int(to_datetime(end_datetime).timestamp() * 1000)
+        timestamps = sorted(set(index.timestamps))
+        selected: list[int] = []
+        for position, chunk_start in enumerate(timestamps):
+            next_start = timestamps[position + 1] if position + 1 < len(timestamps) else None
+            overlaps_start = next_start is None or next_start > start_ms
+            if chunk_start <= end_ms and overlaps_start:
+                selected.append(chunk_start)
+        return selected
+
+    @cache_in_file(with_ttl="1 hour")
+    def _request_forecast(self, start_date: Optional[str] = None) -> EnergyChartsElecPrice:
+        """Fetch and normalize quarter-hourly German/Luxembourg day-ahead prices from SMARD."""
+        if not self.ems_start_datetime:
+            raise ValueError(f"Start DateTime not set: {self.ems_start_datetime}")
+        if start_date is None:
+            start_datetime = self.ems_start_datetime - to_duration("35 days")
+        else:
+            start_datetime = to_datetime(
+                start_date, in_timezone=self.config.general.timezone
+            ).start_of("day")
+        end_datetime = to_datetime(self.end_datetime).end_of("day")
+
+        settings = self.config.elecprice.smard
+        filter_id = settings.filter_id
+        region = settings.region
+        resolution = "quarterhour"
+        index_url = f"{SMARD_BASE_URL}/{filter_id}/{region}/index_{resolution}.json"
+        index = self._validate_index(self._get(index_url))
+        chunk_timestamps = self._chunk_timestamps(index, start_datetime, end_datetime)
+        if not chunk_timestamps:
+            raise ValueError("SMARD index contains no price chunks for the requested period")
+
+        values_by_timestamp: dict[int, float] = {}
+        latest_created = 0
+        for chunk_timestamp in chunk_timestamps:
+            chunk_url = (
+                f"{SMARD_BASE_URL}/{filter_id}/{region}/"
+                f"{filter_id}_{region}_{resolution}_{chunk_timestamp}.json"
+            )
+            chunk = self._validate_chunk(self._get(chunk_url))
+            latest_created = max(latest_created, chunk.meta_data.created)
+            for timestamp_ms, price_eur_mwh in chunk.series:
+                if price_eur_mwh is None:
+                    continue
+                if (
+                    int(start_datetime.timestamp() * 1000)
+                    <= timestamp_ms
+                    <= int(end_datetime.timestamp() * 1000)
+                ):
+                    values_by_timestamp[timestamp_ms] = price_eur_mwh
+
+        if not values_by_timestamp:
+            raise ValueError("SMARD response contains no usable day-ahead prices")
+
+        ordered_values = sorted(values_by_timestamp.items())
+        self.update_datetime = to_datetime(
+            latest_created / 1000, in_timezone=self.config.general.timezone
+        )
+        return EnergyChartsElecPrice(
+            license_info="CC BY 4.0 Bundesnetzagentur | SMARD.de",
+            unix_seconds=[timestamp_ms // 1000 for timestamp_ms, _ in ordered_values],
+            price=[price for _, price in ordered_values],
+            unit="EUR/MWh",
+            deprecated=False,
+        )

src/akkudoktoreos/prediction/elecpricetibber.py

@@ -7,7 +7,6 @@ import pandas as pd
 import requests
 from loguru import logger
 from pydantic import Field, ValidationError
-from statsmodels.tsa.holtwinters import ExponentialSmoothing
 
 from akkudoktoreos.config.configabc import SettingsBaseModel
 from akkudoktoreos.core.cache import cache_in_file
@@ -322,40 +321,6 @@ class ElecPriceTibber(ElecPriceProvider):
         series = series.groupby(level=0).mean().sort_index()
         return series.dropna()
 
-    def _resolution_seconds(self, series: pd.Series) -> int:
-        """Infer the native slot size in seconds from the series timestamps.
-
-        Uses the median of the timestamp differences so that a single outlier gap does
-        not distort the result. Falls back to hourly (3600 s) when fewer than two
-        timestamps are available.
-        """
-        if len(series) < 2:
-            return 3600
-        deltas = pd.DatetimeIndex(series.index).to_series().diff().dropna()
-        if deltas.empty:
-            return 3600
-        resolution = int(round(deltas.dt.total_seconds().median()))
-        return resolution if resolution > 0 else 3600
-
-    def _cap_outliers(self, data: np.ndarray, sigma: int = 2) -> np.ndarray:
-        mean = data.mean()
-        std = data.std()
-        lower_bound = mean - sigma * std
-        upper_bound = mean + sigma * std
-        capped_data = data.clip(min=lower_bound, max=upper_bound)
-        return capped_data
-
-    def _predict_ets(self, history: np.ndarray, seasonal_periods: int, hours: int) -> np.ndarray:
-        clean_history = self._cap_outliers(history)
-        model = ExponentialSmoothing(
-            clean_history, seasonal="add", seasonal_periods=seasonal_periods
-        ).fit()
-        return model.forecast(hours)
-
-    def _predict_median(self, history: np.ndarray, hours: int) -> np.ndarray:
-        clean_history = self._cap_outliers(history)
-        return np.full(hours, np.median(clean_history))
-
     def _predict_missing_prices(
         self, history: np.ndarray, slots: int, slots_per_hour: int
     ) -> np.ndarray:

src/akkudoktoreos/prediction/feedintariff.py

@@ -13,6 +13,7 @@ from akkudoktoreos.prediction.feedintariffenergycharts import (
 )
 from akkudoktoreos.prediction.feedintarifffixed import FeedInTariffFixedCommonSettings
 from akkudoktoreos.prediction.feedintariffimport import FeedInTariffImportCommonSettings
+from akkudoktoreos.prediction.feedintariffsmard import FeedInTariffSMARDCommonSettings
 
 
 def feedintariff_provider_ids() -> list[str]:
@@ -27,6 +28,7 @@ def feedintariff_provider_ids() -> list[str]:
             "FeedInTariffEnergyCharts",
             "FeedInTariffFixed",
             "FeedInTariffImport",
+            "FeedInTariffSMARD",
             "FeedInTariffTibber",
         ]
 
@@ -68,6 +70,11 @@ class FeedInTariffCommonSettings(SettingsBaseModel):
         json_schema_extra={"description": "EnergyCharts feed in tariff provider settings."},
     )
 
+    smard: FeedInTariffSMARDCommonSettings = Field(
+        default_factory=FeedInTariffSMARDCommonSettings,
+        json_schema_extra={"description": "SMARD feed in tariff provider settings."},
+    )
+
     @computed_field  # type: ignore[prop-decorator]
     @property
     def providers(self) -> list[str]:

src/akkudoktoreos/prediction/feedintariffabc.py

@@ -7,9 +7,11 @@ Notes:
 from abc import abstractmethod
 from typing import List, Optional
 
+import pandas as pd
 from pydantic import Field, computed_field
 
-from akkudoktoreos.prediction.predictionabc import PredictionProvider, PredictionRecord
+from akkudoktoreos.prediction.predictionabc import PredictionRecord
+from akkudoktoreos.prediction.priceabc import PricePredictionProviderBase
 
 
 class FeedInTariffDataRecord(PredictionRecord):
@@ -20,8 +22,18 @@ class FeedInTariffDataRecord(PredictionRecord):
 
     """
 
+    feed_in_tariff_raw_wh: Optional[float] = Field(
+        None,
+        json_schema_extra={
+            "description": "Raw feed in tariff per Wh, always excluding fees [amount/Wh]"
+        },
+    )
+
     feed_in_tariff_wh: Optional[float] = Field(
-        None, json_schema_extra={"description": "Feed in tariff per Wh [amount/Wh]"}
+        None,
+        json_schema_extra={
+            "description": "Feed in tariff per Wh, including fees if configured [amount/Wh]"
+        },
     )
 
     # Computed fields
@@ -37,13 +49,13 @@ class FeedInTariffDataRecord(PredictionRecord):
         return self.feed_in_tariff_wh * 1000.0
 
 
-class FeedInTariffProvider(PredictionProvider):
+class FeedInTariffProvider(PricePredictionProviderBase):
     """Abstract base class for feed in tariff providers.
 
     FeedInTariffProvider is a thread-safe singleton, ensuring only one instance of this class is created.
 
     Configuration variables:
-        feed in tariff_provider (str): Prediction provider for feed in tarif.
+        feedintariff.provider (str): Prediction provider for feed in tarif.
     """
 
     # overload
@@ -59,3 +71,44 @@ class FeedInTariffProvider(PredictionProvider):
 
     def enabled(self) -> bool:
         return self.provider_id() == self.config.feedintariff.provider
+
+    # --- PricePredictionProviderBase hooks -------------------------------
+    #
+    # Concrete for every feed-in tariff data source: the raw/gross record
+    # keys, the feed-in-fee keys, and the fee formula itself don't vary by
+    # provider, only by "electricity price" vs. "feed-in tariff" - so, unlike
+    # `provider_id`, these are NOT left abstract for concrete providers to
+    # fill in.
+
+    @property
+    def _raw_key(self) -> str:
+        """Record key holding the fee-free raw series."""
+        return "feed_in_tariff_raw_wh"
+
+    @property
+    def _gross_key(self) -> str:
+        """Record key to write the fee-inclusive series to."""
+        return "feed_in_tariff_wh"
+
+    @property
+    def _fee_keys(self) -> list[str]:
+        """Prediction keys to fetch for fee computation."""
+        return ["elecfee_feedin_amt_wh", "elecfee_feedin_percent_amt"]
+
+    def _compute_gross(self, raw_amt_wh: pd.Series, df_fee: pd.DataFrame) -> pd.Series:
+        """Apply the percent surcharge (e.g. VAT), then subtract the per-Wh feed-in fee.
+
+        gross = raw * (100 - elecfee_feedin_percent_amt) / 100 - elecfee_feedin_amt_wh
+
+        Args:
+            raw_amt_wh: Raw feed-in tariff (amount/Wh), fee-free.
+            df_fee: Fee dataframe aligned to `raw_amt_wh`'s index, with columns
+                matching `_fee_keys`.
+
+        Returns:
+            pd.Series: Gross feed-in tariff (amount/Wh), same index as `raw_amt_wh`.
+        """
+        return (
+            raw_amt_wh * (100.0 - df_fee["elecfee_feedin_percent_amt"]) / 100.0
+            - df_fee["elecfee_feedin_amt_wh"]
+        )

src/akkudoktoreos/prediction/feedintariffakkudoktor.py

@@ -95,21 +95,6 @@ class FeedInTariffAkkudoktor(FeedInTariffProvider):
             series.at[timestamp] = value.marketprice / 1_000_000
         return series
 
-    def _predict_prices(self, history: np.ndarray, hours: int) -> np.ndarray:
-        """Extend published prices to the configured prediction horizon."""
-        predictor = ElecPriceAkkudoktor()
-        if len(history) > 800:
-            return predictor._predict_ets(history, seasonal_periods=168, hours=hours)
-        if len(history) > 168:
-            return predictor._predict_ets(history, seasonal_periods=24, hours=hours)
-        if len(history) > 0:
-            logger.warning(
-                "Using median fallback for Akkudoktor feed-in tariff with only {} values.",
-                len(history),
-            )
-            return predictor._predict_median(history, hours=hours)
-        raise ValueError("No Akkudoktor feed-in tariff data available")
-
     async def _update_data(self, force_update: Optional[bool] = False) -> None:
         """Update raw prices and extrapolate any missing horizon values."""
         if not self.ems_start_datetime:
@@ -150,7 +135,8 @@ class FeedInTariffAkkudoktor(FeedInTariffProvider):
         if needed_hours <= 0:
             return
 
-        prediction = self._predict_prices(history, needed_hours)
+        prediction = self._predict(history, needed_hours)
+
         prediction_series = pd.Series(
             data=prediction,
             index=[

src/akkudoktoreos/prediction/feedintariffenergycharts.py

@@ -4,7 +4,6 @@ import time
 from datetime import datetime
 from typing import Optional
 
-import numpy as np
 import pandas as pd
 import requests
 from loguru import logger
@@ -40,8 +39,9 @@ class FeedInTariffEnergyCharts(FeedInTariffProvider):
     """Fetch Energy-Charts market prices as feed-in tariff data.
 
     This provider stores the raw Energy-Charts day-ahead market price as
-    ``feed_in_tariff_wh``. Unlike ``ElecPriceEnergyCharts`` it intentionally
-    does not add electricity import charges or VAT.
+    ``feed_in_tariff_raw_wh``, and derives ``feed_in_tariff_wh`` from it by
+    applying any configured feed-in fees (unconditionally; a fee-free result
+    requires leaving the ElecFee provider unconfigured).
     """
 
     highest_orig_datetime: Optional[datetime] = None
@@ -66,7 +66,17 @@ class FeedInTariffEnergyCharts(FeedInTariffProvider):
 
     @cache_in_file(with_ttl="1 hour")
     def _request_forecast(self, start_date: Optional[str] = None) -> EnergyChartsElecPrice:
-        """Fetch market price forecast data from Energy-Charts."""
+        """Fetch electricity price forecast data from Energy-Charts API.
+
+        This method sends a request to Energy-Charts API to retrieve forecast data for a specified
+        date range. The response data is parsed and returned as JSON for further processing.
+
+        Returns:
+            dict: The parsed JSON response from Energy-Charts API containing forecast data.
+
+        Raises:
+            ValueError: If the API response does not include expected `electricity price` data.
+        """
         source = "https://api.energy-charts.info"
         if start_date is None:
             start_date = to_datetime(
@@ -105,58 +115,41 @@ class FeedInTariffEnergyCharts(FeedInTariffProvider):
         raise last_exc  # type: ignore[misc]
 
     def _parse_data(self, energy_charts_data: EnergyChartsElecPrice) -> pd.Series:
-        series_data = pd.Series(dtype=float)
+        # Assumption that all lists are the same length and are ordered chronologically
+        # in ascending order and have the same timestamps.
+
+        # Initialize
+        highest_orig_datetime = None  # newest datetime from the api after that we want to update.
+        prices_wh = pd.Series(dtype=float)  # Initialize an empty series
+
+        # Iterate over timestamps and prices together
         for unix_sec, price_eur_per_mwh in zip(
-            energy_charts_data.unix_seconds, energy_charts_data.price, strict=False
+            energy_charts_data.unix_seconds, energy_charts_data.price
         ):
             orig_datetime = to_datetime(unix_sec, in_timezone=self.config.general.timezone)
-            series_data.at[orig_datetime] = price_eur_per_mwh / 1_000_000
-        return series_data
 
-    def _resolution_seconds(self, series: pd.Series) -> int:
-        """Infer the current native market interval from recent timestamps."""
-        if len(series) < 2:
-            return 3600
-        index = pd.DatetimeIndex(series.sort_index().index).drop_duplicates()
-        deltas = index.to_series().diff().dropna().dt.total_seconds()
-        deltas = deltas[deltas > 0].tail(96)
-        if deltas.empty:
-            return 3600
-        resolution = int(round(float(deltas.median())))
-        return resolution if resolution > 0 and 3600 % resolution == 0 else 3600
+            # Track the latest datetime
+            if highest_orig_datetime is None or orig_datetime > highest_orig_datetime:
+                highest_orig_datetime = orig_datetime
 
-    def _predict_prices(self, history: np.ndarray, slots: int, slots_per_hour: int) -> np.ndarray:
-        energycharts = ElecPriceEnergyCharts()
-        if len(history) > 800 * slots_per_hour:
-            logger.info(
-                "Using weekly seasonal ETS forecast for Energy-Charts feed-in tariff "
-                "with {} historical values.",
-                len(history),
-            )
-            return energycharts._predict_ets(
-                history, seasonal_periods=168 * slots_per_hour, hours=slots
-            )
-        if len(history) > 168 * slots_per_hour:
-            logger.info(
-                "Using daily seasonal ETS forecast for Energy-Charts feed-in tariff "
-                "with {} historical values.",
-                len(history),
-            )
-            return energycharts._predict_ets(
-                history, seasonal_periods=24 * slots_per_hour, hours=slots
-            )
-        if len(history) > 0:
-            logger.warning(
-                "Using constant median fallback for Energy-Charts feed-in tariff "
-                "with only {} historical values.",
-                len(history),
-            )
-            return energycharts._predict_median(history, hours=slots)
-        logger.error("No feed-in tariff data available for Energy-Charts prediction")
-        raise ValueError("No data available")
+            # Convert EUR/MWh to EUR/Wh
+            price_wh = price_eur_per_mwh / 1_000_000
+
+            # Store in series
+            prices_wh.at[orig_datetime] = price_wh
+
+        # Always raw here — fees are applied once, later, over the complete
+        # raw+predicted series in _store_gross_series().
+        return prices_wh
 
     async def _update_data(self, force_update: Optional[bool] = False) -> None:
-        """Update feed-in tariff forecast data from Energy-Charts."""
+        """Update feed-in tariff forecast data from Energy-Charts.
+
+        Retrieves data from Energy-Charts, maps each Energy-Charts field to the corresponding
+        `FeedInTariffDataRecord` and applies any necessary scaling.
+
+        The final mapped and processed data is inserted into the sequence as `FeedInTariffDataRecord`.
+        """
         # New prices are available every day at 14:00
         now = pd.Timestamp.now(tz=self.config.general.timezone)
         midnight = now.normalize()
@@ -166,65 +159,79 @@ class FeedInTariffEnergyCharts(FeedInTariffProvider):
         if not self.ems_start_datetime:
             raise ValueError(f"Start DateTime not set: {self.ems_start_datetime}")
 
-        # Determine if update is needed and how many days
+        # Lower bound for the gross-series recompute at the end of this method:
+        # defaults to "from now", widened to the fetched window's start if a
+        # fetch actually happens below.
+        gross_start_datetime = self.ems_start_datetime
+
+        # Set default start_datetime - try to take data from 5 weeks back for prediction
         past_days = 35
-        needs_history_refresh = False
+        start_datetime = self.ems_start_datetime - to_duration(f"{past_days} days")
+
+        # Determine if update is needed and what start date is really necessary
+        needs_update = False
         if self.highest_orig_datetime:
             raw_history = await self.key_to_raw_series(
-                key="feed_in_tariff_wh",
-                end_datetime=to_datetime(self.highest_orig_datetime).add(seconds=1),
+                key="feed_in_tariff_raw_wh",
+                start_datetime=start_datetime,
+                end_datetime=gross_start_datetime,
             )
 
-            # A later update must not mistake the current forecast window for
-            # sufficient ETS history. Require the same amount of data that the
-            # weekly prediction branch below needs; otherwise fetch 35 days
-            # again and repair an already-truncated in-memory history.
-            if not raw_history.empty:
+            if raw_history.empty:
+                # We need the default start date (35 days in past)
+                needs_update = True
+            else:
+                # A later update must not mistake the current forecast window for
+                # sufficient ETS history. Require the same amount of data that the
+                # weekly prediction branch in _predict needs; otherwise fetch 35
+                # days again and repair an already-truncated in-memory history.
                 resolution_seconds = self._resolution_seconds(raw_history)
                 slots_per_hour = 3600 // resolution_seconds
-                needs_history_refresh = len(raw_history) <= 800 * slots_per_hour
-            else:
-                needs_history_refresh = True
-
-            if not needs_history_refresh and not force_update:
-                past_days = 0
-            needs_update = (
-                bool(force_update) or end > self.highest_orig_datetime or needs_history_refresh
-            )
+                if len(raw_history) <= 2 * 168 * slots_per_hour:
+                    # Not enough slots in history, default start date
+                    needs_update = True
+                elif force_update:
+                    # Use default start date in case of forced update
+                    needs_update = True
+                elif end > self.highest_orig_datetime:
+                    # We got enough history, but still not enough data to prediction end
+                    start_datetime = gross_start_datetime
+                    needs_update = True
         else:
             needs_update = True
 
         if needs_update:
             logger.info(
                 "Update FeedInTariffEnergyCharts is needed, last in history: {}, "
-                "force_update={}, history_refresh={}",
+                "force_update={}, start_datetime={}",
                 self.highest_orig_datetime,
                 bool(force_update),
-                needs_history_refresh,
+                start_datetime,
             )
+            # Set start_date try to take data from 5 weeks back for prediction
             start_date = to_datetime(
                 self.ems_start_datetime - to_duration(f"{past_days} days"),
                 as_string="YYYY-MM-DD",
             )
+            # Get Energy-Charts electricity price data
             try:
                 energy_charts_data = self._request_forecast(
-                    start_date=start_date, force_update=force_update
+                    start_date=to_datetime(start_datetime, as_string="YYYY-MM-DD"),
+                    force_update=force_update,
                 )  # type: ignore
+
+                # Parse and store data
                 series_data = self._parse_data(energy_charts_data)
                 if series_data.empty:
                     raise ValueError("No Energy-Charts feed-in tariff data available")
-                self.highest_orig_datetime = series_data.index.max()
-                await self.key_from_series("feed_in_tariff_wh", series_data)
+                self.highest_orig_datetime = to_datetime(series_data.index.max())
+                await self.key_from_series("feed_in_tariff_raw_wh", series_data)
+                # Newly fetched data widens the window that needs its gross
+                # (fee-inclusive) values recomputed.
+                gross_start_datetime = to_datetime(series_data.index.min())
             except Exception as exc:
                 if self.highest_orig_datetime is None:
-                    # Cold start: no cached/historical data to fall back to, so a
-                    # failed fetch is fatal.
                     raise
-                # Transient API outage with existing history available: do not
-                # abort the whole prediction update. Keep the existing history
-                # and let the ETS/median branch below extrapolate the remaining
-                # slots, so downstream (e.g. /gesamtlast, optimization) still
-                # gets a usable feed-in tariff series.
                 logger.warning(
                     "Energy-Charts feed-in tariff update failed ({}); keeping "
                     "existing history until {} and extrapolating the remaining "
@@ -244,32 +251,36 @@ class FeedInTariffEnergyCharts(FeedInTariffProvider):
             raise ValueError(error_msg)
 
         raw_series = await self.key_to_raw_series(
-            key="feed_in_tariff_wh",
+            key="feed_in_tariff_raw_wh",
             end_datetime=to_datetime(self.highest_orig_datetime).add(seconds=1),
         )
         resolution_seconds = self._resolution_seconds(raw_series)
         slots_per_hour = 3600 // resolution_seconds
+
+        # Raw history only. Guaranteed fee-free regardless of which branch ran
+        # above, so ETS/median always trains on the true wholesale-price signal.
         history = await self.key_to_array(
-            key="feed_in_tariff_wh",
+            key="feed_in_tariff_raw_wh",
             end_datetime=self.highest_orig_datetime,
             interval=to_duration(f"{resolution_seconds} seconds"),
             fill_method="linear",
         )
 
-        # some of our data is already in the future, so we need to predict less.
-        # If we got less data we increase the prediction hours
-        covered_slots = 0
-        if self.highest_orig_datetime >= self.ems_start_datetime:
-            covered_slots = (
-                int(
-                    (self.highest_orig_datetime - self.ems_start_datetime).total_seconds()
-                    // resolution_seconds
-                )
-                + 1
-            )
+        # Signed gap: positive when existing raw data already reaches past
+        # ems_start_datetime (fewer slots left to predict); negative when the
+        # newest known data point (highest_orig_datetime) is older than
+        # ems_start_datetime, e.g. after a fetch outage - in that case we need
+        # extra slots to also backfill the gap up to ems_start_datetime, on top
+        # of the full prediction.hours horizon beyond it.
+        covered_slots = int(
+            (self.highest_orig_datetime - self.ems_start_datetime).total_seconds()
+            // resolution_seconds
+        )
         needed_slots = self.config.prediction.hours * slots_per_hour - covered_slots
 
         if needed_slots <= 0:
+            # This might keep data longer than
+            # self.ems_start_datetime + self.config.prediction.hours in the records
             logger.warning(
                 "No feed-in tariff prediction needed. needed_slots={}, hours={}, "
                 "resolution_seconds={}, highest_orig_datetime={}, start_datetime={}",
@@ -279,9 +290,18 @@ class FeedInTariffEnergyCharts(FeedInTariffProvider):
                 self.highest_orig_datetime,
                 self.ems_start_datetime,
             )
+            # Fee schedule may have changed since the last run even without new
+            # market data; recompute gross only for the window that was
+            # actually touched (or is still forward-looking) this cycle.
+            await self._store_gross_series(
+                start_datetime=gross_start_datetime,
+                end_datetime=to_datetime(self.highest_orig_datetime).add(seconds=1),
+            )
             return
 
-        prediction = self._predict_prices(history, needed_slots, slots_per_hour)
+        prediction = self._predict(history, needed_slots, slots_per_hour=slots_per_hour)
+
+        # write predictions into the records, update if exist.
         prediction_series = pd.Series(
             data=prediction,
             index=[
@@ -289,4 +309,12 @@ class FeedInTariffEnergyCharts(FeedInTariffProvider):
                 for i in range(len(prediction))
             ],
         )
-        await self.key_from_series("feed_in_tariff_wh", prediction_series)
+        await self.key_from_series("feed_in_tariff_raw_wh", prediction_series)
+
+        # Bounded to [gross_start_datetime, end of the freshly predicted tail) -
+        # covers exactly what was fetched and/or predicted this cycle, not the
+        # entire (potentially multi-year) retained history.
+        await self._store_gross_series(
+            start_datetime=gross_start_datetime,
+            end_datetime=to_datetime(prediction_series.index.max()) + to_duration("1 second"),
+        )

src/akkudoktoreos/prediction/feedintarifffixed.py

@@ -5,20 +5,32 @@ from typing import Optional
 from loguru import logger
 from pydantic import Field
 
-from akkudoktoreos.config.configabc import SettingsBaseModel
+from akkudoktoreos.config.configabc import (
+    SettingsBaseModel,
+    ValueTimeWindowSequence,
+)
 from akkudoktoreos.prediction.feedintariffabc import FeedInTariffProvider
-from akkudoktoreos.utils.datetimeutil import to_datetime
+from akkudoktoreos.utils.datetimeutil import DateTime, to_duration
 
 
 class FeedInTariffFixedCommonSettings(SettingsBaseModel):
     """Common settings for elecprice fixed price."""
 
-    feed_in_tariff_kwh: Optional[float] = Field(
-        default=None,
-        ge=0,
+    feed_in_tariff_amt_kwh: ValueTimeWindowSequence = Field(
+        default_factory=ValueTimeWindowSequence,
         json_schema_extra={
-            "description": "Electricity price feed in tariff [amount/kWh].",
-            "examples": [0.078],
+            "description": (
+                "Sequence of time windows defining the electricity feed in tariff [amount/kWh]. "
+                "If not provided, no fixed feed in tariff is applied."
+            ),
+            "examples": [
+                {
+                    "windows": [
+                        {"start_time": "00:00", "duration": "8 hours", "value": 0.028},
+                        {"start_time": "08:00", "duration": "16 hours", "value": 0.034},
+                    ],
+                }
+            ],
         },
     )
 
@@ -29,23 +41,80 @@ class FeedInTariffFixed(FeedInTariffProvider):
     FeedInTariffFixed is a singleton-based class that retrieves elecprice data.
     """
 
+    highest_orig_datetime: Optional[DateTime] = None
+
     @classmethod
     def provider_id(cls) -> str:
         """Return the unique identifier for the FeedInTariffFixed provider."""
         return "FeedInTariffFixed"
 
     async def _update_data(self, force_update: Optional[bool] = False) -> None:
-        error_msg = (
-            "Feed in tariff missing in configuration: "
-            "feedintariff.feedintarifffixed.feed_in_tariff_kwh"
+        """Update feed-in tariff data from fixed schedule.
+
+        Generates feed-in tariff based on the configured time windows
+        at the optimization interval granularity. The tariff sequence starts
+        synchronized to the wall clock at the next full interval boundary.
+
+        Args:
+            force_update: If True, forces update even if data exists.
+
+        Raises:
+            ValueError: If no time windows are configured.
+        """
+        prediction_key = "feed_in_tariff_wh"
+        raw_prediction_key = "feed_in_tariff_raw_wh"
+        time_windows_seq: ValueTimeWindowSequence = (
+            self.config.feedintariff.feedintarifffixed.feed_in_tariff_amt_kwh
         )
-        try:
-            feed_in_tariff = self.config.feedintariff.feedintarifffixed.feed_in_tariff_kwh
-        except Exception:
-            logger.exception(error_msg)
-            raise ValueError(error_msg)
-        if feed_in_tariff is None:
-            logger.error(error_msg)
-            raise ValueError(error_msg)
-        feed_in_tariff_wh = feed_in_tariff / 1000
-        await self.update_value(to_datetime(), "feed_in_tariff_wh", feed_in_tariff_wh)
+
+        start_datetime = self.ems_start_datetime
+        interval_seconds = 900  # Usual smallest time interval (15 min) used in electricty prices
+        total_hours = self.config.prediction.hours
+        interval = to_duration(interval_seconds)
+        end_datetime = start_datetime.add(hours=total_hours)
+
+        if time_windows_seq is None or not time_windows_seq.windows:
+            warning_msg = f"No time windows configured for `{raw_prediction_key}`, defaulting to 0."
+            logger.warning(warning_msg)
+            # Store two values to have a default interval to be used by _apply_fees
+            end_datetime = start_datetime + interval
+            await self.update_value(start_datetime, raw_prediction_key, 0.0)
+            await self.update_value(end_datetime, raw_prediction_key, 0.0)
+            self.highest_orig_datetime = end_datetime
+            await self._store_gross_series(
+                start_datetime=start_datetime,
+                end_datetime=end_datetime + to_duration("1 second"),
+            )
+            return
+
+        logger.debug(
+            f"Generating {prediction_key} for {total_hours} hours starting at {start_datetime}"
+        )
+
+        # Build the full tariff array in one call — kWh values aligned to the
+        # optimization grid.  to_series mirrors the key_to_series signature so
+        # the grid is constructed identically to how prediction data is read.
+        tariffs_kwh = time_windows_seq.to_series(
+            start_datetime=start_datetime,
+            end_datetime=end_datetime,
+            interval=interval,
+            dropna=False,
+            boundary="context",
+            align_to_interval=True,
+        )
+
+        # Convert kWh → Wh
+        tariffs_wh = tariffs_kwh / 1000.0
+
+        await self.key_from_series(raw_prediction_key, tariffs_wh)
+        self.highest_orig_datetime = tariffs_wh.index.max()
+
+        # Bounded to exactly the window just generated - covers the whole
+        # forecast horizon in one shot, since FeedInTariffFixed has no
+        # fetch/predict split to worry about like the other providers.
+        await self._store_gross_series(
+            start_datetime=tariffs_wh.index.min(),
+            end_datetime=tariffs_wh.index.max() + to_duration("1 second"),
+        )
+
+        logger.debug(f"Successfully generated {len(tariffs_wh)} `{prediction_key}` entries")

src/akkudoktoreos/prediction/feedintariffsmard.py

@@ -0,0 +1,29 @@
+"""Provide direct-marketing feed-in prices from SMARD day-ahead data."""
+
+from typing import Optional
+
+from akkudoktoreos.config.configabc import SettingsBaseModel
+from akkudoktoreos.prediction.elecpriceenergycharts import EnergyChartsElecPrice
+from akkudoktoreos.prediction.elecpricesmard import ElecPriceSMARD
+from akkudoktoreos.prediction.feedintariffenergycharts import FeedInTariffEnergyCharts
+
+
+class FeedInTariffSMARDCommonSettings(SettingsBaseModel):
+    """Settings for SMARD feed-in prices shared with ``elecprice.smard``."""
+
+
+class FeedInTariffSMARD(FeedInTariffEnergyCharts):
+    """Use raw SMARD day-ahead market prices for direct-marketing feed-in revenue."""
+
+    @classmethod
+    def provider_id(cls) -> str:
+        """Return the unique identifier for the direct SMARD feed-in provider."""
+        return "FeedInTariffSMARD"
+
+    def _request_forecast(
+        self, start_date: Optional[str] = None, force_update: Optional[bool] = False
+    ) -> EnergyChartsElecPrice:
+        """Reuse the cached direct SMARD request without import-price components."""
+        return ElecPriceSMARD()._request_forecast(  # type: ignore[call-arg]
+            start_date=start_date, force_update=force_update
+        )

src/akkudoktoreos/prediction/prediction.py

@@ -31,16 +31,20 @@ from typing import Optional, Union
 from pydantic import Field
 
 from akkudoktoreos.config.configabc import SettingsBaseModel
+from akkudoktoreos.prediction.elecfeefixed import ElecFeeFixed
+from akkudoktoreos.prediction.elecfeeimport import ElecFeeImport
 from akkudoktoreos.prediction.elecpriceakkudoktor import ElecPriceAkkudoktor
 from akkudoktoreos.prediction.elecpriceenergycharts import ElecPriceEnergyCharts
 from akkudoktoreos.prediction.elecpricefixed import ElecPriceFixed
 from akkudoktoreos.prediction.elecpriceimport import ElecPriceImport
+from akkudoktoreos.prediction.elecpricesmard import ElecPriceSMARD
 from akkudoktoreos.prediction.elecpricetibber import ElecPriceTibber
 from akkudoktoreos.prediction.feedintariffakkudoktor import FeedInTariffAkkudoktor
 from akkudoktoreos.prediction.feedintariffdvhubonline import FeedInTariffDvhubOnline
 from akkudoktoreos.prediction.feedintariffenergycharts import FeedInTariffEnergyCharts
 from akkudoktoreos.prediction.feedintarifffixed import FeedInTariffFixed
 from akkudoktoreos.prediction.feedintariffimport import FeedInTariffImport
+from akkudoktoreos.prediction.feedintariffsmard import FeedInTariffSMARD
 from akkudoktoreos.prediction.feedintarifftibber import FeedInTariffTibber
 from akkudoktoreos.prediction.loadakkudoktor import (
     LoadAkkudoktor,
@@ -81,16 +85,20 @@ class PredictionCommonSettings(SettingsBaseModel):
 
 
 # Initialize forecast providers, all are singletons.
+elecfee_fixed = ElecFeeFixed()
+elecfee_import = ElecFeeImport()
 elecprice_akkudoktor = ElecPriceAkkudoktor()
 elecprice_energy_charts = ElecPriceEnergyCharts()
 elecprice_fixed = ElecPriceFixed()
 elecprice_import = ElecPriceImport()
+elecprice_smard = ElecPriceSMARD()
 elecprice_tibber = ElecPriceTibber()
 feedintariff_akkudoktor = FeedInTariffAkkudoktor()
 feedintariff_dvhubonline = FeedInTariffDvhubOnline()
 feedintariff_energy_charts = FeedInTariffEnergyCharts()
 feedintariff_fixed = FeedInTariffFixed()
 feedintariff_import = FeedInTariffImport()
+feedintariff_smard = FeedInTariffSMARD()
 feedintariff_tibber = FeedInTariffTibber()
 loadforecast_akkudoktor = LoadAkkudoktor()
 loadforecast_akkudoktor_adjusted = LoadAkkudoktorAdjusted()
@@ -111,16 +119,20 @@ weather_import = WeatherImport()
 
 def prediction_providers() -> list[
     Union[
+        ElecFeeFixed,
+        ElecFeeImport,
         ElecPriceAkkudoktor,
         ElecPriceEnergyCharts,
         ElecPriceFixed,
         ElecPriceImport,
+        ElecPriceSMARD,
         ElecPriceTibber,
         FeedInTariffAkkudoktor,
         FeedInTariffDvhubOnline,
         FeedInTariffEnergyCharts,
         FeedInTariffFixed,
         FeedInTariffImport,
+        FeedInTariffSMARD,
         FeedInTariffTibber,
         LoadAkkudoktor,
         LoadAkkudoktorAdjusted,
@@ -144,16 +156,20 @@ def prediction_providers() -> list[
     Factory for prediction container.
     """
     global \
+        elecfee_fixed, \
+        elecfee_import, \
         elecprice_akkudoktor, \
         elecprice_energy_charts, \
         elecprice_fixed, \
         elecprice_import, \
+        elecprice_smard, \
         elecprice_tibber, \
         feedintariff_akkudoktor, \
         feedintariff_dvhubonline, \
         feedintariff_energy_charts, \
         feedintariff_fixed, \
         feedintariff_import, \
+        feedintariff_smard, \
         feedintariff_tibber, \
         loadforecast_akkudoktor, \
         loadforecast_akkudoktor_adjusted, \
@@ -176,20 +192,24 @@ def prediction_providers() -> list[
     # Inter provider dependencies:
     # - pvforecast_pvlib depends on weather
     return [
-        weather_brightsky,  # weather maybe needed by the pvforcast, keep it before
+        weather_brightsky,  # weather maybe needed by the pvforcast, keep before
         weather_clearoutside,
         weather_import,
         weather_openmeteo,
+        elecfee_fixed,  # elecfee maybe needed by elecprice and feedintariff, keep before
+        elecfee_import,
         elecprice_akkudoktor,
         elecprice_energy_charts,
         elecprice_fixed,
         elecprice_import,
+        elecprice_smard,
         elecprice_tibber,
         feedintariff_akkudoktor,
         feedintariff_dvhubonline,
         feedintariff_energy_charts,
         feedintariff_fixed,
         feedintariff_import,
+        feedintariff_smard,
         feedintariff_tibber,
         loadforecast_akkudoktor,
         loadforecast_akkudoktor_adjusted,
@@ -210,16 +230,20 @@ class Prediction(PredictionContainer):
 
     providers: list[
         Union[
+            ElecFeeFixed,
+            ElecFeeImport,
             ElecPriceAkkudoktor,
             ElecPriceEnergyCharts,
             ElecPriceFixed,
             ElecPriceImport,
+            ElecPriceSMARD,
             ElecPriceTibber,
             FeedInTariffAkkudoktor,
             FeedInTariffDvhubOnline,
             FeedInTariffEnergyCharts,
             FeedInTariffFixed,
             FeedInTariffImport,
+            FeedInTariffSMARD,
             FeedInTariffTibber,
             LoadAkkudoktor,
             LoadAkkudoktorAdjusted,

src/akkudoktoreos/prediction/priceabc.py

@@ -0,0 +1,312 @@
+"""Shared base for price-like predictions (electricity price, feed-in tariff)."""
+
+from abc import abstractmethod
+from typing import cast
+
+import numpy as np
+import pandas as pd
+from loguru import logger
+from statsmodels.tsa.holtwinters import ExponentialSmoothing
+
+from akkudoktoreos.core.coreabc import PredictionMixin
+from akkudoktoreos.prediction.predictionabc import PredictionProvider
+from akkudoktoreos.utils.datetimeutil import DateTime, to_datetime, to_duration
+
+
+class PricePredictionProviderBase(PredictionMixin, PredictionProvider):
+    """Common forecasting + fee-application logic shared by price-like providers.
+
+    Subclasses must supply the raw/gross record keys, the fee keys to pull from
+    the prediction store, and the formula that combines raw price + fees.
+    """
+
+    # --- identity hooks -------------------------------------------------
+
+    @property
+    @abstractmethod
+    def _raw_key(self) -> str:
+        """Record key holding the fee-free raw series."""
+
+    @property
+    @abstractmethod
+    def _gross_key(self) -> str:
+        """Record key to write the fee-inclusive series to."""
+
+    @property
+    @abstractmethod
+    def _fee_keys(self) -> list[str]:
+        """Prediction keys to fetch for fee computation."""
+
+    @abstractmethod
+    def _compute_gross(self, raw_amt_wh: pd.Series, df_fee: pd.DataFrame) -> pd.Series:
+        """Combine the raw series with the fetched fee dataframe."""
+
+    # --- forecasting helpers (verbatim, shared) --------------------------
+
+    def _resolution_seconds(self, series: pd.Series) -> int:
+        """Infer the native slot size in seconds from the series timestamps.
+
+        Uses the median of the timestamp differences so that a single outlier gap does
+        not distort the result. Falls back to hourly (3600 s) when fewer than two
+        timestamps are available.
+        """
+        if len(series) < 2:
+            return 3600
+        index = pd.DatetimeIndex(series.sort_index().index).drop_duplicates()
+        deltas = index.to_series().diff().dropna().dt.total_seconds()
+        deltas = deltas[deltas > 0].tail(96)
+        if deltas.empty:
+            return 3600
+        resolution = int(round(float(deltas.median())))
+        return resolution if resolution > 0 and 3600 % resolution == 0 else 3600
+
+    def _cap_outliers(self, data: np.ndarray, sigma: int = 2) -> np.ndarray:
+        """Clip extreme values in a price history to a range around the mean.
+
+        Values further than ``sigma`` standard deviations from the mean are clipped
+        to the corresponding bound. Used to keep single-point spikes (e.g. negative
+        price events or data glitches) from dominating seasonal decomposition or a
+        median fallback.
+
+        Args:
+            data: The raw price history to clip.
+            sigma: Number of standard deviations from the mean to allow before
+                clipping. Defaults to 2.
+
+        Returns:
+            A copy of ``data`` with outliers clipped to ``[mean - sigma * std,
+            mean + sigma * std]``.
+        """
+        mean = data.mean()
+        std = data.std()
+        lower_bound = mean - sigma * std
+        upper_bound = mean + sigma * std
+        return data.clip(min=lower_bound, max=upper_bound)
+
+    def _predict_ets(self, history: np.ndarray, seasonal_periods: int, hours: int) -> np.ndarray:
+        """Forecast future prices with additive Exponential Smoothing (ETS).
+
+        Fits a Holt-Winters model with an additive seasonal component to the
+        outlier-capped history and forecasts the requested number of hours ahead.
+
+        Args:
+            history: Historical price values, ordered oldest to newest.
+            seasonal_periods: Length of one seasonal cycle in the same unit as
+                ``history`` (e.g. 24 for daily seasonality, 168 for weekly
+                seasonality on hourly data).
+            hours: Number of hours to forecast beyond the end of ``history``.
+
+        Returns:
+            An array of ``hours`` forecasted values.
+
+        Raises:
+            ValueError: If ``history`` has fewer than ``2 * seasonal_periods``
+                observations, which ETS needs to reliably estimate the seasonal
+                component.
+        """
+        required_observations = 2 * seasonal_periods
+        if len(history) < required_observations:
+            raise ValueError(
+                f"Not enough history for ETS with seasonal_periods="
+                f"{seasonal_periods}: got {len(history)}, "
+                f"need at least {required_observations}"
+            )
+        clean_history = self._cap_outliers(history)
+        model = ExponentialSmoothing(
+            clean_history, seasonal="add", seasonal_periods=seasonal_periods
+        ).fit()
+        return model.forecast(hours)
+
+    def _predict_median(self, history: np.ndarray, hours: int) -> np.ndarray:
+        """Forecast future prices as a constant equal to the historical median.
+
+        Fallback used when there isn't enough history for a seasonal ETS forecast.
+
+        Args:
+            history: Historical price values, ordered oldest to newest.
+            hours: Number of hours to forecast.
+
+        Returns:
+            An array of ``hours`` values, all equal to the median of the
+            outlier-capped history.
+        """
+        clean_history = self._cap_outliers(history)
+        return np.full(hours, np.median(clean_history))
+
+    def _predict(self, history: np.ndarray, hours: int, slots_per_hour: int = 1) -> np.ndarray:
+        """Forecast future prices, choosing seasonality by available history length.
+
+        Uses weekly-seasonal ETS if there's enough history for it, falls back to
+        daily-seasonal ETS with less, and to a constant median with too little
+        history for either.
+
+        Args:
+            history: Historical price values, ordered oldest to newest.
+            hours: Number of forecast steps to produce, at the resolution implied
+                by ``slots_per_hour`` (despite the name, not necessarily clock hours).
+            slots_per_hour: Number of samples per hour in ``history`` (e.g. 4 for
+                15-minute data). Scales the seasonal period so a "week" or "day"
+                still spans the right number of samples at sub-hourly resolution.
+                Defaults to 1 (hourly data).
+
+        Returns:
+            An array of ``hours`` forecasted values.
+
+        Raises:
+            ValueError: If ``history`` is empty.
+        """
+        weekly_periods = 168 * slots_per_hour
+        daily_periods = 24 * slots_per_hour
+        history_length = len(history)
+        if history_length >= 2 * weekly_periods:
+            return self._predict_ets(history, seasonal_periods=weekly_periods, hours=hours)
+        elif history_length >= 2 * daily_periods:
+            return self._predict_ets(history, seasonal_periods=daily_periods, hours=hours)
+        elif history_length > 0:
+            logger.warning(
+                "Using median fallback to predict prices with only {} values.", len(history)
+            )
+            return self._predict_median(history, hours=hours)
+        logger.error("No data available for prediction")
+        raise ValueError("No data available")
+
+    # --- fee application (shared plumbing, subclass supplies formula) ----
+
+    async def _apply_fees(self, raw_price_amt_wh: pd.Series) -> pd.Series:
+        """Apply fees to a raw price-like time series to produce the gross series.
+
+        The raw series is first normalized to a strictly uniform, sorted,
+        duplicate-free DatetimeIndex (resampling to a fixed 15-minute grid with
+        forward-fill if the input spacing isn't already uniform), since fees are
+        fetched from the prediction store over the resulting `[start, end)` window
+        at that resolution. The fee values are then combined with the raw series
+        via `_compute_gross`, which each subclass implements with its own formula
+        (e.g. add-then-percent for consumption, percent-then-subtract for feed-in).
+
+        Args:
+            raw_price_amt_wh: Raw price-like series (amount/Wh), indexed by a
+                timezone-aware DatetimeIndex. Excludes fees.
+
+        Returns:
+            pd.Series: Raw series with fees applied (amount/Wh), named the same
+            as `raw_price_amt_wh`. If the input index was uniform, the returned
+            index matches it; otherwise the returned index is the fixed
+            15-minute, forward-filled resampling of the input index.
+
+        Raises:
+            ValueError: If `raw_price_amt_wh` is empty, or has fewer than two
+                entries (so no interval can be derived).
+            TypeError: If `raw_price_amt_wh` is not indexed by a DatetimeIndex,
+                or if the derived interval is not a `pd.Timedelta`.
+        """
+        if raw_price_amt_wh.empty:
+            raise ValueError("raw_price_amt_wh must not be empty.")
+        if len(raw_price_amt_wh.index) < 2:
+            raise ValueError(
+                "raw_price_amt_wh must have at least two entries to derive the interval."
+            )
+
+        # Normalize the index: sorted, unique timestamps only. Later duplicate
+        # timestamps win, since they're assumed to be the more recently written value.
+        index = raw_price_amt_wh.index.sort_values()
+        if not isinstance(index, pd.DatetimeIndex):
+            raise TypeError("raw_price_amt_wh must have a DatetimeIndex")
+        index = cast(pd.DatetimeIndex, index)
+
+        raw_price_amt_wh = raw_price_amt_wh.reindex(index)
+        raw_price_amt_wh = raw_price_amt_wh[~raw_price_amt_wh.index.duplicated(keep="last")]
+        index = cast(pd.DatetimeIndex, raw_price_amt_wh.index)
+
+        # Determine whether the (deduplicated) index has a single, uniform spacing.
+        diffs = index.to_series().diff().dropna().unique()
+        if len(diffs) != 1:
+            # Spacing is irregular (e.g. gaps or mixed resolutions): fall back to a
+            # fixed 15-minute grid spanning the same range, forward-filling gaps so
+            # every slot has a value before fees are fetched/applied.
+            diff0 = pd.Timedelta(minutes=15)
+            uniform_index = pd.date_range(start=index[0], end=index[-1], freq=diff0, tz=index.tz)
+            raw_price_amt_wh = (
+                raw_price_amt_wh.reindex(raw_price_amt_wh.index.union(uniform_index))
+                .sort_index()
+                .ffill()
+                .reindex(uniform_index)
+            )
+            index = uniform_index
+            logger.warning(
+                f"raw_price_amt_wh has non uniform spacing {diffs}; "
+                "resampled to fixed 15-minutes grid with forward filling gaps"
+            )
+        else:
+            # Already uniform: use the single observed spacing as-is.
+            diff = diffs[0]
+            if not isinstance(diff, pd.Timedelta):
+                raise TypeError("Expected a Timedelta")
+            diff0 = diff
+
+        # Window and resolution used to fetch fee data matching the raw series exactly.
+        # end_datetime is exclusive, so it's one interval past the last raw timestamp.
+        start_datetime = to_datetime(index[0].to_pydatetime())
+        end_datetime = to_datetime(index[-1].to_pydatetime() + diff0.to_pytimedelta())
+        interval = to_duration(f"{diff0.total_seconds()} seconds")
+
+        # Fetch the fee series/percentages this provider needs (subclass-specific keys).
+        keys = self._fee_keys
+        try:
+            df_fee = await self.prediction.keys_to_dataframe(
+                keys=keys,
+                start_datetime=start_datetime,
+                end_datetime=end_datetime,
+                interval=interval,
+                fill_method="linear",
+                resample_method="mean",
+                dropna=False,
+                boundary="context",
+                align_to_interval=True,
+            )
+        except KeyError:
+            # No fee provider enabled/configured for these keys: treat as zero
+            # fees rather than failing the whole price calculation.
+            df_fee = pd.DataFrame(0.0, index=raw_price_amt_wh.index, columns=keys)
+
+        # Guard against any boundary/resample mismatch between the fee dataframe
+        # and the raw price index (e.g. missing edge timestamps) by reindexing
+        # onto the raw index exactly and treating anything still missing as zero.
+        df_fee = df_fee.reindex(raw_price_amt_wh.index).fillna(0.0)
+
+        # Subclass-specific formula combining raw price and fees.
+        price_amt_wh = self._compute_gross(raw_price_amt_wh, df_fee)
+        price_amt_wh.name = raw_price_amt_wh.name
+        return price_amt_wh
+
+    async def _store_gross_series(
+        self,
+        start_datetime: DateTime | None = None,
+        end_datetime: DateTime | None = None,
+    ) -> None:
+        """Derive the fee-inclusive series from the fee-free (raw) series.
+
+        Recomputes the fee-inclusive series over `[start_datetime, end_datetime)`
+        only, so that historic and predicted values within that window get their
+        own correct time-window/weekday-specific fee amount. Deliberately bounded
+        rather than covering the entire retained history, which can span years -
+        callers are responsible for choosing bounds that cover every timestamp
+        written or possibly affected during the current update cycle.
+
+        Note: timestamps outside the given bounds keep whatever gross value was
+        computed for them in an earlier update cycle. If the fee schedule changes
+        in a way that should retroactively affect already-processed history, that
+        older range needs to be explicitly recomputed (e.g. via a forced refetch),
+        it will not happen automatically here.
+
+        Args:
+            start_datetime: Inclusive lower bound of the raw series to recompute.
+            end_datetime: Exclusive upper bound of the raw series to recompute.
+        """
+        # Read back only the fee-free slice that needs recomputing...
+        full_raw_series = await self.key_to_raw_series(
+            key=self._raw_key, start_datetime=start_datetime, end_datetime=end_datetime
+        )
+        # ...apply fees to it...
+        full_series_with_fees = await self._apply_fees(full_raw_series)
+        # ...and persist the result under the gross key.
+        await self.key_from_series(self._gross_key, full_series_with_fees)

src/akkudoktoreos/server/dash/components.py

@@ -410,6 +410,86 @@ def make_config_update_list_form(available_values: list[str]) -> Callable[[str,
     return ConfigUpdateListForm
 
 
+def make_config_update_lazy_select_form(options_source: str) -> Callable[[str, str], Grid]:
+    """Factory for a form that sets a value via a server-filtered select.
+
+    Unlike make_config_update_value_form, the <select> is never populated
+    with the full candidate list up front. A search Input drives an HTMX
+    GET against /eosdash/configuration/options/{options_source} which
+    returns a capped, filtered <select> fragment — safe for lists with
+    thousands of entries (e.g. the pvlib CEC module/inverter database).
+
+    Args:
+        options_source: Registry key into uihints.LAZY_OPTIONS_SOURCES.
+
+    Returns:
+        A function (config_name: str, value: str) -> Grid
+    """
+
+    def ConfigUpdateLazySelectForm(config_name: str, value: str) -> Grid:
+        config_id = config_name.lower().replace(".", "-")
+        select_id = f"{config_id}-lazy-select"
+        select_name = f"{config_id}_lazy_selected_value"
+
+        # value arrives JSON-encoded (e.g. '"SomeModule"') same as the
+        # other form factories receive it from ConfigCard.
+        try:
+            parsed = json.loads(value) if value else None
+        except (TypeError, ValueError):
+            parsed = value
+        current = "" if parsed is None else str(parsed)
+
+        return Grid(
+            DivRAligned(P("update value")),
+            DivHStacked(
+                ConfigButton(
+                    "Set",
+                    hx_put=request_url_for("/eosdash/configuration"),
+                    hx_target="#page-content",
+                    hx_swap="innerHTML",
+                    hx_vals=f"""js:{{
+                        action: "update",
+                        key: "{config_name}",
+                        value: document.querySelector("#{select_id}").value
+                    }}""",
+                    hx_include=HTMX_INCLUDE,
+                ),
+                Div(
+                    Input(
+                        placeholder="Type to search…",
+                        id=f"{config_id}-lazy-search",
+                        autocomplete="off",
+                        hx_get=request_url_for(f"/eosdash/configuration/options/{options_source}"),
+                        hx_trigger="load, keyup delay:300ms",
+                        # hx_trigger="load, keyup",
+                        hx_target=f"#{select_id}",
+                        hx_swap="outerHTML",
+                        hx_vals=f"""js:{{
+                            search: this.value,
+                            select_id: {json.dumps(select_id)},
+                            name: {json.dumps(select_name)},
+                            current: {json.dumps(current)}
+                        }}""",
+                        cls="border rounded px-3 py-2 mb-1 w-full",
+                    ),
+                    Select(
+                        Option(current, value=current, selected=True)
+                        if current
+                        else Option("Select a value...", value="", selected=True, disabled=True),
+                        id=select_id,
+                        name=select_name,
+                        required=True,
+                        cls="border rounded px-3 py-2 w-full",
+                    ),
+                    cls="col-span-4",
+                ),
+            ),
+            id=f"{config_id}-update-lazy-select-form",
+        )
+
+    return ConfigUpdateLazySelectForm
+
+
 def make_config_update_map_form(
     available_keys: list[str] | None = None,
     available_values: list[str] | None = None,

src/akkudoktoreos/server/dash/configuration.py

@@ -4,6 +4,7 @@ from collections.abc import Sequence
 from typing import Any, Dict, List, Optional, TypeVar, Union
 
 import requests
+from fasthtml.common import Select
 from loguru import logger
 from monsterui.franken import (
     Card,
@@ -11,6 +12,7 @@ from monsterui.franken import (
     Div,
     Grid,
     LabelCheckboxX,
+    Option,
 )
 from pydantic.fields import ComputedFieldInfo, FieldInfo
 from pydantic_core import PydanticUndefined
@@ -24,10 +26,13 @@ from akkudoktoreos.server.dash.components import (
     Input,
 )
 from akkudoktoreos.server.dash.context import request_url_for
+from akkudoktoreos.server.dash.eosstatus import eos_pvlib_cec_names, eos_server_address
 from akkudoktoreos.server.dash.itemscard import ConfigItemsCard
 from akkudoktoreos.server.dash.mapcard import ConfigMapCard
 from akkudoktoreos.server.dash.uihints import (
+    LAZY_OPTIONS_SOURCES,
     UI_HINTS,
+    register_lazy_options_source,
     resolve_form_factory,
 )
 
@@ -258,73 +263,68 @@ def create_config_details(
         dict[dict]: A dictionary of configuration details, each represented as a dictionary.
     """
     config_details: dict[str, dict] = {}
-    inner_types: set[type[PydanticBaseModel]] = set()
+
+    def extract_nested_models(
+        subfield_info: Union[ComputedFieldInfo, FieldInfo],
+        parent_types: list[str],
+        visited: frozenset,
+    ) -> None:
+        nonlocal values, values_prefix
+        regular_field = isinstance(subfield_info, FieldInfo)
+        subtype = subfield_info.annotation if regular_field else subfield_info.return_type
+
+        nested_types = resolve_nested_types(subtype, [])
+        found_basic = False
+        for nested_type, nested_parent_types in nested_types:
+            if not isinstance(nested_type, type) or not issubclass(nested_type, PydanticBaseModel):
+                if found_basic:
+                    continue
+                extra = get_field_extra_dict(subfield_info)
+
+                config: dict[str, Optional[Any]] = {}
+                config["name"] = ".".join(values_prefix + parent_types)
+                config["value"] = json.dumps(
+                    get_nested_value(values, values_prefix + parent_types, "<unknown>")
+                )
+                config["default"] = json.dumps(get_default_value(subfield_info, regular_field))
+                config["description"] = get_description(subfield_info, extra)
+                config["deprecated"] = get_deprecated(subfield_info, extra)
+                config["scope"] = get_scope(extra)
+                if isinstance(subfield_info, ComputedFieldInfo):
+                    config["read-only"] = "ro"
+                    type_description = str(subfield_info.return_type)
+                else:
+                    config["read-only"] = "rw"
+                    type_description = str(subfield_info.annotation)
+                config["type"] = (
+                    type_description.replace("typing.", "")
+                    .replace("pathlib.", "")
+                    .replace("NoneType", "None")
+                    .replace("<class 'float'>", "float")
+                )
+                config_details[str(config["name"])] = config
+                found_basic = True
+            else:
+                if nested_type in visited:
+                    # Genuine cycle along this path (self-referential model) — stop
+                    # recursing here, but do NOT block sibling branches elsewhere.
+                    continue
+                new_parent_types = parent_types + nested_parent_types
+                new_visited = visited | {nested_type}
+                for nested_field_name, nested_field_info in list(
+                    nested_type.model_fields.items()
+                ) + list(nested_type.model_computed_fields.items()):
+                    extract_nested_models(
+                        nested_field_info,
+                        new_parent_types + [nested_field_name],
+                        new_visited,
+                    )
 
     for field_name, field_info in list(model.model_fields.items()) + list(
         model.model_computed_fields.items()
     ):
+        extract_nested_models(field_info, [field_name], frozenset())
 
-        def extract_nested_models(
-            subfield_info: Union[ComputedFieldInfo, FieldInfo], parent_types: list[str]
-        ) -> None:
-            """Extract nested models from the given subfield information.
-
-            Args:
-                subfield_info (Union[ComputedFieldInfo, FieldInfo]): Field metadata from Pydantic.
-                parent_types (list[str]): A list of parent type names for hierarchical representation.
-            """
-            nonlocal values, values_prefix
-            regular_field = isinstance(subfield_info, FieldInfo)
-            subtype = subfield_info.annotation if regular_field else subfield_info.return_type
-
-            if subtype in inner_types:
-                return
-
-            nested_types = resolve_nested_types(subtype, [])
-            found_basic = False
-            for nested_type, nested_parent_types in nested_types:
-                if not isinstance(nested_type, type) or not issubclass(
-                    nested_type, PydanticBaseModel
-                ):
-                    if found_basic:
-                        continue
-                    extra = get_field_extra_dict(subfield_info)
-
-                    config: dict[str, Optional[Any]] = {}
-                    config["name"] = ".".join(values_prefix + parent_types)
-                    config["value"] = json.dumps(
-                        get_nested_value(values, values_prefix + parent_types, "<unknown>")
-                    )
-                    config["default"] = json.dumps(get_default_value(subfield_info, regular_field))
-                    config["description"] = get_description(subfield_info, extra)
-                    config["deprecated"] = get_deprecated(subfield_info, extra)
-                    config["scope"] = get_scope(extra)
-                    if isinstance(subfield_info, ComputedFieldInfo):
-                        config["read-only"] = "ro"
-                        type_description = str(subfield_info.return_type)
-                    else:
-                        config["read-only"] = "rw"
-                        type_description = str(subfield_info.annotation)
-                    config["type"] = (
-                        type_description.replace("typing.", "")
-                        .replace("pathlib.", "")
-                        .replace("NoneType", "None")
-                        .replace("<class 'float'>", "float")
-                    )
-                    config_details[str(config["name"])] = config
-                    found_basic = True
-                else:
-                    new_parent_types = parent_types + nested_parent_types
-                    inner_types.add(nested_type)
-                    for nested_field_name, nested_field_info in list(
-                        nested_type.model_fields.items()
-                    ) + list(nested_type.model_computed_fields.items()):
-                        extract_nested_models(
-                            nested_field_info,
-                            new_parent_types + [nested_field_name],
-                        )
-
-        extract_nested_models(field_info, [field_name])
     return config_details
 
 
@@ -366,6 +366,93 @@ def config_matches_search(config: dict, search: str) -> bool:
     )
 
 
+# ---------------------
+# Lazy Options Handling
+# ---------------------
+
+
+def _fetch_cec_names(kind: str) -> tuple[str, ...]:
+    """Fetch CEC module or inverter names from the EOS server.
+
+    Cached per kind for the EOSdash process lifetime — the CEC database
+    is static.
+
+    Args:
+        kind: "modules" or "inverters".
+    """
+    global eos_server_address, eos_pvlib_cec_names
+
+    if kind in eos_pvlib_cec_names:
+        # Already cached
+        return eos_pvlib_cec_names[kind]
+
+    if eos_server_address is None:
+        logger.warning(
+            f"EOS server address not yet known: `{eos_server_address}`; cannot fetch CEC {kind}"
+        )
+        return ()
+    host, port = eos_server_address
+    # host = "127.0.0.1"
+    # port = 8503
+    server = f"http://{host}:{port}"
+    path = f"/v1/prediction/pvforecast/pvlib/{kind}"
+    try:
+        result = requests.get(f"{server}{path}", timeout=10)
+        result.raise_for_status()
+        data = result.json()
+        names = tuple(data) if isinstance(data, list) else tuple(data.keys())
+        eos_pvlib_cec_names[kind] = names
+    except requests.exceptions.RequestException as e:
+        logger.warning(f"Can not retrieve CEC {kind} from {server}: {e}")
+        return ()
+
+    logger.info(f"Fetched CEC {kind}")
+
+    return names
+
+
+def _cec_modules_source() -> list[str]:
+    return list(_fetch_cec_names("modules"))
+
+
+def _cec_inverters_source() -> list[str]:
+    return list(_fetch_cec_names("inverters"))
+
+
+register_lazy_options_source("pvlib.modules", _cec_modules_source)
+register_lazy_options_source("pvlib.inverters", _cec_inverters_source)
+
+
+def config_options(
+    options_source: str, search: str, select_id: str, name: str, current: str
+) -> Select:
+    """Return a filtered, capped <select> fragment for a select_lazy field."""
+    source = LAZY_OPTIONS_SOURCES.get(options_source)
+    all_values = source() if source else []
+
+    search_l = (search or "").strip().lower()
+    matches = [v for v in all_values if search_l in v.lower()] if search_l else all_values
+    matches = matches[:50]  # hard cap regardless of source size
+
+    if current and current not in matches:
+        # keep the current selection visible even if it doesn't match
+        # the active filter, so switching searches never silently loses it
+        matches = [current] + matches
+
+    return Select(
+        *[Option(v, value=v, selected=(v == current)) for v in matches],
+        id=select_id,
+        name=name,
+        required=True,
+        cls="border rounded px-3 py-2 w-full",
+    )
+
+
+# -----------------------------------
+# Configuration Visual Representation
+# -----------------------------------
+
+
 def Configuration(
     eos_host: str,
     eos_port: Union[str, int],
@@ -381,9 +468,12 @@ def Configuration(
     Returns:
         rows:  Rows of configuration details.
     """
-    global config_visible
+    global config_visible, eos_server_address
     dark = False
 
+    # Remember for usage
+    eos_server_address = (eos_host, int(eos_port))
+
     if data and data.get("action", None):
         if data.get("dark", None) == "true":
             dark = True
@@ -526,11 +616,13 @@ def Configuration(
     logger.debug(f"devices_measurement_keys {devices_measurement_keys}")
 
     # build visual representation
-    sections: dict[str, list[Any]] = {}
+    sections: dict[str, dict[str, Any]] = {}
 
+    # Fill sections with config cards
     for config_key in sorted(config_details.keys()):
         config = config_details[config_key]
         category = config["name"].split(".")[0]
+        sections.setdefault(category, {})
 
         update_error = config_update_latest.get(config["name"], {}).get("error")
         update_value = config_update_latest.get(config["name"], {}).get("value")
@@ -594,23 +686,34 @@ def Configuration(
         else:
             continue
 
-        sections.setdefault(category, []).append(card)
+        sections[category][config["name"]] = card
 
     section_components = []
 
     for category in sorted(sections.keys()):
-        cards = sections[category]
-
-        # Open if searching OR if last update was here
+        # Open if searching OR if last update touched this category — including
+        # updates on deeply nested sub-fields (e.g. pvforecast.planes.2.module_model)
+        # whose full dotted name never appears as a key in the outer config_details
+        # dict, since that dict only holds top-level model fields. ConfigItemsCard
+        # resolves per-item sub-fields internally via its own create_config_details
+        # call, so we must match against config_update_latest's own keys instead.
         open_section = bool(search_value)
 
         if not open_section:
             open_section = any(
-                config_update_latest.get(c["name"], {}).get("open")
-                for c in config_details.values()
-                if c["name"].startswith(category)
+                info.get("open")
+                for key, info in config_update_latest.items()
+                if key == category or key.startswith(f"{category}.")
             )
 
+        cards: list[Card] = []
+        for name, card in sections[category].items():
+            if name.rsplit(".", 1)[-1] == "provider":
+                # Make provider the first config item in category
+                cards.insert(0, card)
+            else:
+                cards.append(card)
+
         section_components.append(ConfigSection(category, *cards, open=open_section))
 
     return Div(

src/akkudoktoreos/server/dash/context.py

@@ -87,11 +87,6 @@ class IngressMiddleware(BaseHTTPMiddleware):
             or request.headers.get("X-INGRESS-PATH", "")
         )
 
-        # Debug logging - remove after testing
-        logger.debug(f"All headers: {dict(request.headers)}")
-        logger.debug(f"Ingress path: {ingress_path}")
-        logger.debug(f"Request path: {request.url.path}")
-
         # Only set root_path if we have an ingress path
         if ingress_path:
             ROOT_PATH = ingress_path

src/akkudoktoreos/server/dash/eosstatus.py

@@ -11,3 +11,7 @@ eos_health: Optional[dict] = None
 eos_solution: Optional[OptimizationSolution] = None
 eos_plan: Optional[EnergyManagementPlan] = None
 eos_config: Optional[SettingsEOS] = None
+eos_server_address: Optional[tuple[str, int]] = None
+
+# dictionary for `modules` and `inverters` (str) names (tuple[str]).
+eos_pvlib_cec_names: dict[str, tuple[str]] = {}

src/akkudoktoreos/server/dash/uihints.py

@@ -26,6 +26,7 @@ from dataclasses import dataclass, field
 from typing import Any, Callable, Literal, Optional
 
 from akkudoktoreos.server.dash.components import (
+    make_config_update_lazy_select_form,
     make_config_update_list_form,
     make_config_update_map_form,
     make_config_update_time_windows_windows_form,
@@ -40,6 +41,7 @@ UiFormType = Literal[
     "text",  # plain text input (default)
     "select",  # single-value dropdown
     "select_list",  # add/delete multi-value list
+    "select_lazy",  # server-filtered select for large option sets
     "map",  # key/value pair editor
     "time_windows",  # time-window sequence editor
     "items",  # expandable list of sub-model cards
@@ -68,6 +70,12 @@ class UiHint:
             option list (JSON-encoded ``list[str]``).  Takes precedence
             over ``options`` when both are set.
 
+        options_source:
+            ``select_lazy`` only. Key into LAZY_OPTIONS_SOURCES identifying the
+            callable that lazily returns the full (potentially huge) candidate
+            list, e.g. "pvforecast.cec_modules". Distinct from options_from, which
+            reads a small already-resolved list out of config_details.
+
         param_from:
             Dotted config-field path for a secondary runtime parameter.
             Used by ``"map"`` for the *keys* dropdown.
@@ -108,6 +116,7 @@ class UiHint:
     # select / select_list / map
     options: list[str] = field(default_factory=list)
     options_from: Optional[str] = None
+    options_source: Optional[str] = None
     param_from: Optional[str] = None
     append_none: bool = False
 
@@ -120,14 +129,40 @@ class UiHint:
     max_items_from: Optional[str] = None
 
 
+# ---------------------------------------------------------------------------
+# Lazy Registry
+# ---------------------------------------------------------------------------
+
+LazyOptionsSource = Callable[[], list[str]]
+
+LAZY_OPTIONS_SOURCES: dict[str, LazyOptionsSource] = {}
+
+
+def register_lazy_options_source(key: str, source: LazyOptionsSource) -> None:
+    """Register a lazy options callable that provides the options list.
+
+    The callable shall provide the full candidate list for a "select_lazy" hint's
+    options_source.
+
+    The callable should be cheap to call repeatedly (e.g. wrapped in
+    caching) since it is invoked once per keystroke-driven search request —
+    filtering happens in this process, not inside the callable itself.
+    """
+    LAZY_OPTIONS_SOURCES[key] = source
+
+
 # ---------------------------------------------------------------------------
 # Registry
 # ---------------------------------------------------------------------------
 
 UI_HINTS: dict[str, UiHint] = {
     # ------------------------------------------------------------------
-    # Adapter - Home Assistant adapter
+    # Adapter
     # ------------------------------------------------------------------
+    "adapter.provider": UiHint(
+        form="select_list",
+        options_from="adapter.providers",
+    ),
     "adapter.homeassistant.config_entity_ids": UiHint(
         form="map",
         options_from="adapter.homeassistant.homeassistant_entity_ids",
@@ -162,6 +197,14 @@ UI_HINTS: dict[str, UiHint] = {
         options_from="adapter.homeassistant.eos_solution_entity_ids",
     ),
     # ------------------------------------------------------------------
+    # Database
+    # ------------------------------------------------------------------
+    "database.provider": UiHint(
+        form="select",
+        options_from="database.providers",
+        append_none=True,
+    ),
+    # ------------------------------------------------------------------
     # Devices
     # ------------------------------------------------------------------
     "devices.batteries": UiHint(
@@ -182,16 +225,40 @@ UI_HINTS: dict[str, UiHint] = {
         value_description="cycle index (0-based)",
     ),
     # ------------------------------------------------------------------
-    # Electricity price — fixed time windows
+    # Electricity fee
+    # ------------------------------------------------------------------
+    "elecfee.provider": UiHint(
+        form="select",
+        options_from="elecfee.providers",
+        append_none=True,
+    ),
+    "elecfee.elecfeefixed.consumption_amt_kwh.windows": UiHint(
+        form="time_windows",
+        value_description="consumption_amt_kwh [Amt/kWh]",
+    ),
+    "elecfee.elecfeefixed.consumption_percent_amt.windows": UiHint(
+        form="time_windows",
+        value_description="consumption_percent_amt [%]",
+    ),
+    "elecfee.elecfeefixed.feedin_amt_kwh.windows": UiHint(
+        form="time_windows",
+        value_description="feedin_amt_kwh [Amt/kWh]",
+    ),
+    "elecfee.elecfeefixed.feedin_percent_amt.windows": UiHint(
+        form="time_windows",
+        value_description="feedin_percent_amt [%]",
+    ),
+    # ------------------------------------------------------------------
+    # Electricity price
     # ------------------------------------------------------------------
     "elecprice.provider": UiHint(
         form="select",
         options_from="elecprice.providers",
         append_none=True,
     ),
-    "elecprice.elecpricefixed.time_windows.windows": UiHint(
+    "elecprice.elecpricefixed.elecprice_marketprice_amt_kwh.windows": UiHint(
         form="time_windows",
-        value_description="electricity_price_kwh [Amt/kWh]",
+        value_description="elecprice_marketprice_amt_kwh [Amt/kWh]",
     ),
     # ------------------------------------------------------------------
     # EMS
@@ -201,6 +268,18 @@ UI_HINTS: dict[str, UiHint] = {
         options_from="ems.modes",
     ),
     # ------------------------------------------------------------------
+    # Feed-in Tariff
+    # ------------------------------------------------------------------
+    "feedintariff.provider": UiHint(
+        form="select",
+        options_from="feedintariff.providers",
+        append_none=True,
+    ),
+    "feedintariff.feedintarifffixed.feed_in_tariff_amt_kwh.windows": UiHint(
+        form="time_windows",
+        value_description="feed_in_tariff_amt_kwh [Amt/kWh]",
+    ),
+    # ------------------------------------------------------------------
     # Load
     # ------------------------------------------------------------------
     "load.provider": UiHint(
@@ -238,6 +317,14 @@ UI_HINTS: dict[str, UiHint] = {
         form="select",
         options=["free", "building"],
     ),
+    "pvforecast.planes.inverter_model": UiHint(
+        form="select_lazy",
+        options_source="pvlib.inverters",
+    ),
+    "pvforecast.planes.module_model": UiHint(
+        form="select_lazy",
+        options_source="pvlib.modules",
+    ),
     # ------------------------------------------------------------------
     # Weather
     # ------------------------------------------------------------------
@@ -362,6 +449,9 @@ def resolve_form_factory(
             options = list(hint.options)
         return make_config_update_list_form(options)
 
+    if hint.form == "select_lazy":
+        return make_config_update_lazy_select_form(hint.options_source or "")
+
     if hint.form == "map":
         available_values: Optional[list[str]] = None
         available_keys: Optional[list[str]] = None

src/akkudoktoreos/server/eosdash.py

@@ -24,7 +24,7 @@ from akkudoktoreos.server.dash.admin import Admin
 # helpers
 from akkudoktoreos.server.dash.bokeh import BokehJS
 from akkudoktoreos.server.dash.components import Page
-from akkudoktoreos.server.dash.configuration import Configuration
+from akkudoktoreos.server.dash.configuration import Configuration, config_options
 from akkudoktoreos.server.dash.context import (
     IngressMiddleware,
     safe_asset_path,
@@ -323,6 +323,19 @@ def post_eosdash_admin(request: Request, data: dict):  # type: ignore
     return Admin(*eos_server(), data)
 
 
+@app.get("/eosdash/configuration/options/{options_source:path}")
+def get_eosdash_configuration_options(  # type: ignore
+    request: Request,
+    options_source: str,
+    search: str = "",
+    select_id: str = "",
+    name: str = "",
+    current: str = "",
+):
+    """Serve a filtered <select> fragment for a select_lazy config field."""
+    return config_options(options_source, search, select_id, name, current)
+
+
 @app.get("/eosdash/configuration")
 def get_eosdash_configuration(request: Request):  # type: ignore
     """Serve the EOSdash Configuration page.

src/akkudoktoreos/utils/datetimeutil.py

@@ -1125,7 +1125,7 @@ def to_duration(
         '15 minutes'
 
         >>> to_duration("90 seconds", as_string="pandas")
-        '90S'
+        '90s'
 
         >>> to_duration("15 minutes", as_string="{M}m")
         '15m'
@@ -1170,14 +1170,18 @@ def to_duration(
                 logger.error(error_msg)
                 raise ValueError(error_msg)
 
-            # Handle strings like "2 days 5 hours 30 minutes"
-            matches = re.findall(r"(\d+)\s*(days?|hours?|minutes?|seconds?)", input_value)
+            # Handle strings like "2 days 5 hours 30 minutes" or "900.0 seconds".
+            # NOTE: the numeric group must match decimals (e.g. "900.0"), not just
+            # integers -- otherwise a run like "900.0 seconds" mismatches on the
+            # "." after "900" and the regex instead matches only the trailing
+            # "0 seconds", silently truncating the value.
+            matches = re.findall(r"(\d+(?:\.\d+)?)\s*(days?|hours?|minutes?|seconds?)", input_value)
             if not matches:
                 error_msg = f"Invalid time string format '{input_value}'"
                 logger.error(error_msg)
                 raise ValueError(error_msg)
 
-            total_seconds = 0
+            total_seconds = 0.0
             time_units = {
                 "day": 86400,
                 "hour": 3600,
@@ -1187,7 +1191,7 @@ def to_duration(
             for value, unit in matches:
                 unit = unit.lower().rstrip("s")  # Normalize unit
                 if unit in time_units:
-                    total_seconds += int(value) * time_units[unit]
+                    total_seconds += float(value) * time_units[unit]
                 else:
                     error_msg = f"Unsupported time unit: {unit}"
                     logger.error(error_msg)
@@ -1216,6 +1220,13 @@ def to_duration(
 
     # Pandas frequency
     if as_string == "pandas":
+        if total_seconds <= 0:
+            error_msg = (
+                f"Cannot express a non-positive duration ({total_seconds} seconds) "
+                "as a pandas frequency string."
+            )
+            logger.error(error_msg)
+            raise ValueError(error_msg)
         # hours?
         if total_seconds % 3600 == 0:
             return f"{total_seconds // 3600}h"