feat: electricity fee provider framework and generic providers (#1235)

Add new provider class for electricity fees providers.

Add the generic providers:
- ElecFeeFixed
- ElecFeeImport

The providers provide predictions for:

- elecfee_consumption_amt_wh:
  Total fixed fee for consumed energy per Wh [amount/Wh]. This is the accumulation of all
  fixed per-Wh fees payable on "consumed energy - such as network charge, concession fee,
  and electricity charge - into a single amount.
- elecfee_consumption_percent_amt:
  Total fixed surcharge on consumed energy, given as a percentage of the monetary amount
  already charged for that energy [%]. This is the accumulation of all percentage-based
  surcharges payable on top of the consumed-energy fee - such as VAT - into a single
  percentage. This is a percentage of the fee amount, not a per-Wh rate.
- elecfee_feedin_amt_wh:
  Total fixed deduction from feed-in energy per Wh [amount/Wh]. This is the accumulation of
  all fixed per-Wh charges deducted from feed-in energy - such as metering fees or
  grid-operator handling "charges - into a single amount. Applied after the percentage-based
  deduction, i.e. it reduces the price by a flat amount per Wh rather than by a share of the
  raw price.
- elecfee_feedin_percent_amt:
  Total percentage deducted from the raw feed-in price (spot price) [%]. This is the
  accumulation of all percentage-based deductions payable on the feed-in tariff - such as a
  marketing or balancing fee retained by the aggregator - into a single percentage. It is
  applied as `raw_price * (100 - percent) / 100`, i.e. it scales down the raw price rather
  than adding a surcharge to it.

A new _apply_fee() method is added to the base class for ElecPrice and FeedInTariff to be used to
add the fees in a consistent way. Fees are taken from the active ElecFee provider and applied
to the raw prices given to the _apply_fee() method.

The optional application of fees is added to:

- ElecPriceAkkudoktor
- ElecPriceFixed
- ElecPriceEnergyCharts
- ElecPriceSMARD
- FeedInTariffEnergyCharts
- FeedInTariffFixed
- FeedInTariffSMARD

The import providers ElecPriceImport and FeedInTariffImport do not apply fees by intentention.

The following providers currently do not handle fees defined by ElecFee:

- ElecPriceTibber
- FeedInTariffAkkudoktor
- FeedInTariffDvhubOnline
- FeedInTariffTibber

The tests for this feature are either added or existing tests are extended.

The documentation was extended for the electricity fee provider settings.

Besides this feature further improvements are added:

* feat: add SMARD quarter-hour electricty price and feed-in tariff provider

* feat: to_series method for TimeWindows and ValueTimeWindows

  Additional to to_array the time window sequence can now also produce a pandas series.
  Test have been extended to cover the series generation.

* feat: use time windows in fixed feedin tariff provider

  Feedin tariff can now be configured by time windows - not a single value.

* feat: EOSdash select for PVLib inverters and modules

  Provide PVLib inverter and module names in config selection.

* feat: EOSdash lazy select for big option sets

  Add a new form for lazy selection of big option sets. Filtering and
  generation of the option set is done server-side.

* fix: use raw data for ETS/ median prediction

  Use to raw time series data for ETS/ median prediction to avoid interference
  by e.g. dynamic grid charges.

* fix: EOSdash config drops by type only on details resolve

  Drop configuration by type and path. Prevents dropping of configuration items
  with same type and level but different path.

* fix: EOSdash configuration section closes on update

  Open section if searching or if last update touched this category — including
  updates on deeply nested sub-fields.

* chore: make elecfeefixed, elecpricefixed and feedintarifffixed warn about no windows and default to 0

  Missining configuration creates default 0 value and a warning instead of an exception.

* fix: test setup for providers

  Reset db state on each test run.

* chore: improve config option naming for elecpricefixed.

* chore: adapt elecpricefixed test to changed time_windows naming

* chore: factorized common price provider helpers to priceabc.py

  Factorized common price provider helpers to priceabc.py. Add tests for these helpers.
  Reduce/ change testing of elecpriceabc.py and feedintariffabc.py to cover
  only specifics. Rest of testing is already covered by test_priceabc.py.

* chore: update version

Signed-off-by: Bobby Noelte <b0661n0e17e@gmail.com>

tests/test_configabc.py

@@ -10,7 +10,7 @@ Timezone contract under test:
 * When a timezone-aware datetime is supplied, ``start_time`` is
   interpreted as wall-clock time **in that timezone** — no tz
   conversion is applied to ``start_time`` itself.
-* Constructing a ``TimeWindow`` with an aware ``start_time`` raises
+* Constructing a ``TimeWindow`` with a naive ``start_time`` raises
   ``ValidationError``.
 """
 
@@ -20,7 +20,10 @@ import sys
 
 sys.path.insert(0, os.path.dirname(__file__))
 
+from typing import cast
+
 import numpy as np
+import pandas as pd
 import pendulum
 import pytest
 from pydantic import ValidationError
@@ -721,6 +724,152 @@ class TestTimeWindowSequenceToArray:
         assert np.all(arr_tue == 0.0)              # Tuesday — all outside
 
 
+# ===========================================================================
+# TimeWindowSequence.to_series
+# ===========================================================================
+
+class TestTimeWindowSequenceToSeries:
+    """Tests for TimeWindowSequence.to_series.
+
+    Window layout:
+        win1: 08:00–10:00
+        win2: 14:00–17:00
+    """
+
+    def setup_method(self, method):
+        self.seq = TimeWindowSequence(
+            windows=[
+                make_window(8, 2),
+                make_window(14, 3),
+            ]
+        )
+
+    def test_basic_1h_steps_naive(self):
+        start = naive_dt(2024, 6, 15, 0)
+        end = naive_dt(2024, 6, 16, 0)
+
+        series = self.seq.to_series(
+            start, end, pendulum.duration(hours=1)
+        )
+
+        assert isinstance(series, pd.Series)
+        assert series.shape == (24,)
+        assert isinstance(series.index, pd.DatetimeIndex)
+
+        assert series.iloc[8] == pytest.approx(1.0)
+        assert series.iloc[9] == pytest.approx(1.0)
+        assert series.iloc[10] == pytest.approx(0.0)
+
+        assert series.iloc[14] == pytest.approx(1.0)
+        assert series.iloc[15] == pytest.approx(1.0)
+        assert series.iloc[16] == pytest.approx(1.0)
+        assert series.iloc[17] == pytest.approx(0.0)
+
+    def test_values_match_to_array(self):
+        start = naive_dt(2024, 6, 15, 0)
+        end = naive_dt(2024, 6, 16, 0)
+        interval = pendulum.duration(hours=1)
+
+        arr = self.seq.to_array(start, end, interval)
+        series = self.seq.to_series(start, end, interval)
+
+        np.testing.assert_array_equal(series.to_numpy(), arr)
+
+    def test_dtype_is_float64(self):
+        start = naive_dt(2024, 6, 15, 0)
+        end = naive_dt(2024, 6, 15, 4)
+
+        series = self.seq.to_series(
+            start, end, pendulum.duration(hours=1)
+        )
+
+        assert series.dtype == np.float64
+
+    def test_end_is_exclusive(self):
+        start = naive_dt(2024, 6, 15, 6)
+        end = naive_dt(2024, 6, 15, 8)
+
+        series = self.seq.to_series(
+            start, end, pendulum.duration(hours=1)
+        )
+        index = cast(pd.DatetimeIndex, series.index)
+
+        assert series.shape == (2,)
+        assert list(index.hour) == [6, 7]
+        assert np.all(series.to_numpy() == 0.0)
+
+    def test_align_to_interval_false_preserves_start(self):
+        start = naive_dt(2024, 6, 15, 8, 10)
+        end = naive_dt(2024, 6, 15, 10, 10)
+
+        series = self.seq.to_series(
+            start,
+            end,
+            pendulum.duration(hours=1),
+            align_to_interval=False,
+        )
+        index = cast(pd.DatetimeIndex, series.index)
+
+        assert series.shape == (2,)
+        assert index[0] == pd.Timestamp(start)
+        assert index[1] == pd.Timestamp(start.add(hours=1))
+        assert np.all(series.to_numpy() == 1.0)
+
+    def test_align_to_interval_true_floors_start(self):
+        start = naive_dt(2024, 6, 15, 8, 10)
+        end = naive_dt(2024, 6, 15, 10, 10)
+
+        series = self.seq.to_series(
+            start,
+            end,
+            pendulum.duration(hours=1),
+            align_to_interval=True,
+        )
+        index = cast(pd.DatetimeIndex, series.index)
+
+        assert series.shape == (3,)
+        assert list(index.hour) == [8, 9, 10]
+        assert series.iloc[0] == pytest.approx(1.0)
+        assert series.iloc[1] == pytest.approx(1.0)
+        assert series.iloc[2] == pytest.approx(0.0)
+
+    def test_aware_datetime_preserves_timezone(self):
+        start = aware_dt(2024, 6, 15, 0, tz="Europe/Berlin")
+        end = aware_dt(2024, 6, 15, 4, tz="Europe/Berlin")
+
+        series = self.seq.to_series(
+            start, end, pendulum.duration(hours=1)
+        )
+        index = cast(pd.DatetimeIndex, series.index)
+
+        assert series.shape == (4,)
+        assert str(index.tz) == "Europe/Berlin"
+
+    def test_unsupported_boundary_raises(self):
+        start = naive_dt(2024, 6, 15, 0)
+        end = naive_dt(2024, 6, 15, 4)
+
+        with pytest.raises(ValueError, match="boundary"):
+            self.seq.to_series(
+                start,
+                end,
+                pendulum.duration(hours=1),
+                boundary="strict",
+            )
+
+    def test_empty_sequence_all_zeros(self):
+        seq = TimeWindowSequence()
+        start = naive_dt(2024, 6, 15, 0)
+        end = naive_dt(2024, 6, 15, 4)
+
+        series = seq.to_series(
+            start, end, pendulum.duration(hours=1)
+        )
+
+        assert series.shape == (4,)
+        assert np.all(series.to_numpy() == 0.0)
+
+
 # ===========================================================================
 # ValueTimeWindowSequence.to_array
 # ===========================================================================
@@ -881,6 +1030,254 @@ class TestValueTimeWindowSequenceToArray:
         assert arr[1] == pytest.approx(0.10)
 
 
+# ===========================================================================
+# ValueTimeWindowSequence.to_series
+# ===========================================================================
+
+class TestValueTimeWindowSequenceToSeries:
+    """Tests for ValueTimeWindowSequence.to_series."""
+
+    def setup_method(self, method):
+        self.seq = ValueTimeWindowSequence(
+            windows=[
+                ValueTimeWindow(
+                    start_time="08:00:00",
+                    duration="4 hours",
+                    value=0.25,
+                ),
+                ValueTimeWindow(
+                    start_time="18:00:00",
+                    duration="4 hours",
+                    value=0.35,
+                ),
+            ]
+        )
+
+    def test_basic_1h_steps_values(self):
+        start = naive_dt(2024, 6, 15, 0)
+        end = naive_dt(2024, 6, 16, 0)
+
+        series = self.seq.to_series(
+            start, end, pendulum.duration(hours=1)
+        )
+
+        assert isinstance(series, pd.Series)
+        assert isinstance(series.index, pd.DatetimeIndex)
+        assert series.shape == (24,)
+
+        assert series.iloc[8] == pytest.approx(0.25)
+        assert series.iloc[11] == pytest.approx(0.25)
+        assert series.iloc[12] == pytest.approx(0.0)
+
+        assert series.iloc[18] == pytest.approx(0.35)
+        assert series.iloc[21] == pytest.approx(0.35)
+        assert series.iloc[22] == pytest.approx(0.0)
+
+    def test_values_match_to_array(self):
+        start = naive_dt(2024, 6, 15, 0)
+        end = naive_dt(2024, 6, 16, 0)
+        interval = pendulum.duration(hours=1)
+
+        arr = self.seq.to_array(start, end, interval)
+        series = self.seq.to_series(start, end, interval)
+
+        np.testing.assert_array_equal(series.to_numpy(), arr)
+
+    def test_dtype_is_float64(self):
+        start = naive_dt(2024, 6, 15, 0)
+        end = naive_dt(2024, 6, 15, 4)
+
+        series = self.seq.to_series(
+            start, end, pendulum.duration(hours=1)
+        )
+
+        assert series.dtype == np.float64
+
+    def test_dropna_false_none_value_emits_nan(self):
+        seq = ValueTimeWindowSequence(
+            windows=[
+                ValueTimeWindow(
+                    start_time="08:00:00",
+                    duration="2 hours",
+                    value=None,
+                ),
+                ValueTimeWindow(
+                    start_time="12:00:00",
+                    duration="2 hours",
+                    value=0.5,
+                ),
+            ]
+        )
+
+        start = naive_dt(2024, 6, 15, 8)
+        end = naive_dt(2024, 6, 15, 15)
+
+        series = seq.to_series(
+            start,
+            end,
+            pendulum.duration(hours=1),
+            dropna=False,
+        )
+        index = cast(pd.DatetimeIndex, series.index)
+
+        assert series.shape == (7,)
+        assert list(index.hour) == [8, 9, 10, 11, 12, 13, 14]
+
+        assert np.isnan(series.iloc[0])
+        assert np.isnan(series.iloc[1])
+        assert series.iloc[2] == pytest.approx(0.0)
+        assert series.iloc[3] == pytest.approx(0.0)
+        assert series.iloc[4] == pytest.approx(0.5)
+        assert series.iloc[5] == pytest.approx(0.5)
+        assert series.iloc[6] == pytest.approx(0.0)
+
+    def test_dropna_true_none_value_omits_timestamp(self):
+        seq = ValueTimeWindowSequence(
+            windows=[
+                ValueTimeWindow(
+                    start_time="08:00:00",
+                    duration="2 hours",
+                    value=None,
+                ),
+                ValueTimeWindow(
+                    start_time="12:00:00",
+                    duration="2 hours",
+                    value=0.5,
+                ),
+            ]
+        )
+
+        start = naive_dt(2024, 6, 15, 8)
+        end = naive_dt(2024, 6, 15, 15)
+
+        series = seq.to_series(
+            start,
+            end,
+            pendulum.duration(hours=1),
+            dropna=True,
+        )
+        index = cast(pd.DatetimeIndex, series.index)
+
+        # 08:00 and 09:00 are omitted completely.
+        assert series.shape == (5,)
+        assert list(index.hour) == [10, 11, 12, 13, 14]
+
+        np.testing.assert_allclose(
+            series.to_numpy(),
+            [0.0, 0.0, 0.5, 0.5, 0.0],
+        )
+
+    def test_dropna_no_none_values_same_result(self):
+        start = naive_dt(2024, 6, 15, 0)
+        end = naive_dt(2024, 6, 15, 6)
+        interval = pendulum.duration(hours=1)
+
+        series_true = self.seq.to_series(
+            start, end, interval, dropna=True
+        )
+        series_false = self.seq.to_series(
+            start, end, interval, dropna=False
+        )
+
+        pd.testing.assert_series_equal(series_true, series_false)
+
+    def test_aware_datetime_preserves_timezone(self):
+        start = aware_dt(2024, 6, 15, 0, tz="Europe/Berlin")
+        end = aware_dt(2024, 6, 15, 4, tz="Europe/Berlin")
+
+        series = self.seq.to_series(
+            start, end, pendulum.duration(hours=1)
+        )
+        index = cast(pd.DatetimeIndex, series.index)
+
+        assert series.shape == (4,)
+        assert str(index.tz) == "Europe/Berlin"
+
+    def test_align_to_interval_true_floors_start(self):
+        start = naive_dt(2024, 6, 15, 8, 10)
+        end = naive_dt(2024, 6, 15, 10, 10)
+
+        series = self.seq.to_series(
+            start,
+            end,
+            pendulum.duration(hours=1),
+            align_to_interval=True,
+        )
+        index = cast(pd.DatetimeIndex, series.index)
+
+        assert series.shape == (3,)
+        assert list(index.hour) == [8, 9, 10]
+        assert series.iloc[0] == pytest.approx(0.25)
+        assert series.iloc[1] == pytest.approx(0.25)
+        assert series.iloc[2] == pytest.approx(0.25)
+
+    def test_align_to_interval_false_preserves_start(self):
+        start = naive_dt(2024, 6, 15, 8, 30)
+        end = naive_dt(2024, 6, 15, 12, 30)
+
+        series = self.seq.to_series(
+            start,
+            end,
+            pendulum.duration(hours=1),
+            align_to_interval=False,
+        )
+
+        assert series.shape == (4,)
+        assert series.index[0] == pd.Timestamp(start)
+        assert series.iloc[0] == pytest.approx(0.25)
+        assert np.all(series.to_numpy() == pytest.approx(0.25))
+
+    def test_unsupported_boundary_raises(self):
+        start = naive_dt(2024, 6, 15, 0)
+        end = naive_dt(2024, 6, 15, 4)
+
+        with pytest.raises(ValueError, match="boundary"):
+            self.seq.to_series(
+                start,
+                end,
+                pendulum.duration(hours=1),
+                boundary="inner",
+            )
+
+    def test_empty_sequence_all_zeros(self):
+        seq = ValueTimeWindowSequence()
+        start = naive_dt(2024, 6, 15, 0)
+        end = naive_dt(2024, 6, 15, 4)
+
+        series = seq.to_series(
+            start, end, pendulum.duration(hours=1)
+        )
+
+        assert series.shape == (4,)
+        assert np.all(series.to_numpy() == 0.0)
+
+    def test_overlapping_windows_first_wins(self):
+        seq = ValueTimeWindowSequence(
+            windows=[
+                ValueTimeWindow(
+                    start_time="08:00:00",
+                    duration="4 hours",
+                    value=0.10,
+                ),
+                ValueTimeWindow(
+                    start_time="09:00:00",
+                    duration="4 hours",
+                    value=0.99,
+                ),
+            ]
+        )
+
+        start = naive_dt(2024, 6, 15, 9)
+        end = naive_dt(2024, 6, 15, 11)
+
+        series = seq.to_series(
+            start, end, pendulum.duration(hours=1)
+        )
+
+        assert series.iloc[0] == pytest.approx(0.10)
+        assert series.iloc[1] == pytest.approx(0.10)
+
+
 # ===========================================================================
 # align_to_interval — timezone-invariance
 #
@@ -957,6 +1354,22 @@ class TestAlignToIntervalTimezoneInvariance:
         assert arr[1] == pytest.approx(1.0)
         assert arr[2] == pytest.approx(0.0)
 
+    def test_tws_series_naive_floor_non_utc(self, set_other_timezone):
+        set_other_timezone()
+
+        series = self._tws_naive().to_series(
+            self._tws_naive_start(),
+            self._tws_naive_end(),
+            pendulum.duration(hours=1),
+            align_to_interval=True,
+        )
+
+        assert series.shape == (3,)
+        assert list(series.index.hour) == [8, 9, 10]
+        assert series.iloc[0] == pytest.approx(1.0)
+        assert series.iloc[1] == pytest.approx(1.0)
+        assert series.iloc[2] == pytest.approx(0.0)
+
     # ------------------------------------------------------------------
     # TimeWindowSequence — naive datetime, 30-min steps
     # floor 08:10 → 08:00; expect steps 08:00(1), 08:30(1), 09:00(1), 09:30(1), 10:00(0)
@@ -1009,6 +1422,28 @@ class TestAlignToIntervalTimezoneInvariance:
         assert arr[1] == pytest.approx(1.0)
         assert arr[2] == pytest.approx(0.0)
 
+
+    def test_tws_series_aware_floor_non_utc(self, set_other_timezone):
+        set_other_timezone()
+
+        seq = self._tws_naive()
+        start = aware_dt(2024, 6, 15, 8, 10, tz="Europe/Berlin")
+        end = aware_dt(2024, 6, 15, 10, 10, tz="Europe/Berlin")
+
+        series = seq.to_series(
+            start,
+            end,
+            pendulum.duration(hours=1),
+            align_to_interval=True,
+        )
+
+        assert series.shape == (3,)
+        assert list(series.index.hour) == [8, 9, 10]
+        assert str(series.index.tz) == "Europe/Berlin"
+        assert series.iloc[0] == pytest.approx(1.0)
+        assert series.iloc[1] == pytest.approx(1.0)
+        assert series.iloc[2] == pytest.approx(0.0)
+
     # ------------------------------------------------------------------
     # ValueTimeWindowSequence — naive datetime, 1-hour steps
     # floor 08:10 → 08:00; values 0.25 at 08:00, 09:00; 0.0 at 10:00
@@ -1039,3 +1474,33 @@ class TestAlignToIntervalTimezoneInvariance:
         assert arr[0] == pytest.approx(0.25)
         assert arr[1] == pytest.approx(0.25)
         assert arr[2] == pytest.approx(0.0)
+
+    def test_vtws_series_naive_floor_non_utc(self, set_other_timezone):
+        set_other_timezone()
+
+        seq = ValueTimeWindowSequence(
+            windows=[
+                ValueTimeWindow(
+                    start_time="08:00:00",
+                    duration="2 hours",
+                    value=0.25,
+                )
+            ]
+        )
+
+        start = naive_dt(2024, 6, 15, 8, 10)
+        end = naive_dt(2024, 6, 15, 10, 10)
+
+        series = seq.to_series(
+            start,
+            end,
+            pendulum.duration(hours=1),
+            align_to_interval=True,
+        )
+        index = cast(pd.DatetimeIndex, series.index)
+
+        assert series.shape == (3,)
+        assert list(index.hour) == [8, 9, 10]
+        assert series.iloc[0] == pytest.approx(0.25)
+        assert series.iloc[1] == pytest.approx(0.25)
+        assert series.iloc[2] == pytest.approx(0.0)