mirror of
https://github.com/Akkudoktor-EOS/EOS.git
synced 2026-10-14 10:01:59 +00:00
feat(measurement): add typed energy, quality and capacity APIs (#1326)
* feat: adapt configuration for multi optimization algorithms Decouple configuration from optimization algorithm parameters. Add to_[algorithm]_param() methods to the configuration that derive optimization algorithm specific parameters from the configuration. Add x-scope tags to the configuration options that describe for which specific algorithms the configuration option is for. The whole device settings are restructured. There are now general settings for the device classes with the afore mentioned to_[algorithm]_param() methods. The general device settings got their own directory `devices/settings`. By this the parameter class also does not have to be a pydantic model which can be used for future optimization/ simulations speed up. Also the parameter class for a device is now part of the device module. This better decouples and also is the natural place for parameters of a device. Besides this feature there are also fixes and improvements: * feat: extend home appliance time window settings and simulation Home appliance can now be configured for multiple runs with per-cycle allowed time windows. The number of remaining cycles to plan is determined at runtime by reading the ``cycles_completed_measurement_key`` from the measurement store. * feat: specialiced CycleTimeWindowSequence for time window sequences Sequence of time windows associated to cycles. This model specializes ``ValueTimeWindowSequence`` so that the ``value`` field of each ``ValueTimeWindow`` encodes the **cycle index** (0-based integer) the window belongs to. Typical use: an appliance that must run ``n`` times per day, each run constrained to a distinct time window. Assign ``value=0`` to windows for the first cycle, ``value=1`` for the second, and so on. Multiple windows may share the same cycle index (their allowed regions are unioned). Windows with ``value=None`` are silently ignored by all cycle-aware methods. * fix: Make test_configmigrate also regard the _ANY_SENTENIEL in key values * chore: Make devices configurations a map instead of a list This makes config paths stable regardless of declaration order and lets each device settings class build its own config path from ``self.device_id`` without needing an external index. Tests are adapted likewise. Devices configurations are automatically migrated from lists to maps. * chore: rename levelized_cost_of_storage_kwh to levelized_cost_of_storage_amt kwh This better fits in the naming scheme and also makes clear the costs are money. Signed-off-by: Bobby Noelte <b0661n0e17e@gmail.com> * fix: runtime config update ignored by config file Runtime settings were handed back to pydantic-settings as init settings, which rank below the config file and the environment. Any key already present in EOS.config.json or in the environment silently discarded the update, so a bulk PUT /v1/config returned 200 without applying anything, while the granular PUT /v1/config/{path} endpoint kept working. Add a dedicated runtime settings source ranked directly below the command line arguments and record granular updates there as well, so both endpoints share one store that survives re-evaluation of the settings sources. Environment variables keep precedence over the config file for all keys that were not set at runtime. Also repairs revert_settings() and update(), which passed their data through the same init settings. Closes #1303 * fix: env vars ignored on first config build ConfigEOS.__init__ passed self as first positional argument to _setup, which forwards it to pydantic_settings.BaseSettings.__init__. Its first positional parameter is _case_sensitive, so the environment source matched the upper case variable names against the lower case field names and returned nothing. Environment settings only took effect after the next configuration setup. * docs: changelog for config priority fixes * fix(config): preserve device identities and storage costs during migration * fix(measurement): restore JSON records into the existing singleton * fix(devices): preserve charge-rate typing and public import compatibility * feat(measurement): integrate typed energy quality and capacity APIs Port the locally backed-up measurement extensions to main async storage and PR #1256 device maps. Preserve runtime capacity estimates across #1305 bulk updates. Confirm JSON singleton restore defect on unchanged main and add regression. No production configuration or measurements included. Co-authored-by: Andreas <drbacke@gmx.de> * docs(measurement): describe household settings and consolidate regression coverage * docs(measurement): regenerate configuration and API contracts * test(measurement): isolate capacity database state between tests * ruff format fix * test(measurement): assert restored timestamps before timezone conversion * docs(measurement): align API version with refreshed prerequisites * fix(config): satisfy typed device conversion and migration contracts * docs(config): refresh validated configuration prerequisite schemas * fix(measurement): enforce typed capacity and sample validation * style(measurement): normalize imports for CI * docs(measurement): refresh typed measurement API schemas --------- Signed-off-by: Bobby Noelte <b0661n0e17e@gmail.com> Co-authored-by: Bobby Noelte <b0661n0e17e@gmail.com> Co-authored-by: r0b2g1t <r0b2g1t@users.noreply.github.com>
This commit is contained in:
Andreas
co-authored by
Andreas
Bobby Noelte
r0b2g1t
Andreas
Bobby Noelte
r0b2g1t
parent
8224c64654
commit
1a18935667
@@ -0,0 +1,246 @@
|
||||
import pytest_asyncio
|
||||
|
||||
"""Capacity fits must preserve energy direction, coverage and independent anchors."""
|
||||
|
||||
# ruff: noqa: S101
|
||||
|
||||
from datetime import datetime, timedelta, timezone
|
||||
|
||||
import pytest
|
||||
from pydantic import ValidationError
|
||||
|
||||
from akkudoktoreos.measurement.batterycapacity import (
|
||||
BatteryCapacityEstimationSettings,
|
||||
BatteryCapacityRequest,
|
||||
estimate_capacity,
|
||||
)
|
||||
from akkudoktoreos.measurement.measurement import MeasurementChannelSettings
|
||||
from akkudoktoreos.measurement.quality import SampleQuality
|
||||
|
||||
START = datetime(2026, 9, 10, tzinfo=timezone.utc)
|
||||
|
||||
|
||||
def fit(
|
||||
points,
|
||||
*,
|
||||
start_soc=20,
|
||||
end_soc=100,
|
||||
efficiency=1,
|
||||
method="hold",
|
||||
polarity="charging",
|
||||
unit="W",
|
||||
end_seconds=3600,
|
||||
quality=None,
|
||||
max_gap=3600,
|
||||
):
|
||||
request = BatteryCapacityRequest.model_validate(dict(
|
||||
start=START,
|
||||
end=START + timedelta(seconds=end_seconds),
|
||||
start_soc_percentage=start_soc,
|
||||
end_soc_percentage=end_soc,
|
||||
soc_reference="voltage_current_anchor",
|
||||
))
|
||||
settings = BatteryCapacityEstimationSettings(power_key="dc", positive_power=polarity)
|
||||
channel = MeasurementChannelSettings(
|
||||
quantity="power",
|
||||
unit=unit,
|
||||
integration_method=method,
|
||||
max_gap_seconds=max_gap,
|
||||
)
|
||||
return estimate_capacity(
|
||||
request,
|
||||
settings,
|
||||
channel,
|
||||
[
|
||||
(START + timedelta(seconds=t), v, (quality or {}).get(t, SampleQuality()))
|
||||
for t, v in points
|
||||
],
|
||||
battery_id="battery",
|
||||
capacity_wh=12000,
|
||||
charging_efficiency=efficiency,
|
||||
discharging_efficiency=efficiency,
|
||||
)
|
||||
|
||||
|
||||
def test_charge_fit_and_unclipped_model_error():
|
||||
result = fit([(0, 8000), (3600, 8000)])
|
||||
assert result.estimated_capacity_wh == pytest.approx(10000)
|
||||
assert result.configured_capacity_wh == 12000
|
||||
assert result.model_end_soc_percentage_unclipped == pytest.approx(86.6666667)
|
||||
assert result.model_soc_error_percentage_points == pytest.approx(-13.3333333)
|
||||
assert result.charge_energy_wh == 8000
|
||||
assert result.coverage_seconds == 3600
|
||||
|
||||
|
||||
def test_efficiency_is_applied_once_on_dc_boundary():
|
||||
result = fit([(0, 10000), (3600, 10000)], efficiency=0.8)
|
||||
assert result.estimated_capacity_wh == pytest.approx(10000)
|
||||
assert result.stored_energy_change_wh == 8000
|
||||
|
||||
|
||||
def test_discharge_fit_and_reversed_sensor_sign():
|
||||
result = fit(
|
||||
[(0, 6400), (3600, 6400)], start_soc=100, end_soc=20, polarity="discharging", efficiency=0.8
|
||||
)
|
||||
assert result.discharge_energy_wh == 6400
|
||||
assert result.estimated_capacity_wh == pytest.approx(10000)
|
||||
|
||||
|
||||
def test_linear_zero_crossing_is_split_before_losses():
|
||||
result = fit([(0, -4000), (3600, 12000)], method="linear", efficiency=0.8)
|
||||
assert result.charge_energy_wh == pytest.approx(4500)
|
||||
assert result.discharge_energy_wh == pytest.approx(500)
|
||||
assert result.stored_energy_change_wh == pytest.approx(4500 * 0.8 - 500 / 0.8)
|
||||
|
||||
|
||||
def test_kw_and_clipped_boundary_interpolation():
|
||||
result = fit([(-3600, 0), (3600, 16)], method="linear", unit="kW", max_gap=7200)
|
||||
assert result.charge_energy_wh == pytest.approx(12000)
|
||||
# Deliberately do not saturate the old model at 100%.
|
||||
assert result.model_end_soc_percentage_unclipped == pytest.approx(120)
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"points", [[], [(0, 8000)], [(60, 8000), (3600, 8000)], [(0, 8000), (3500, 8000)]]
|
||||
)
|
||||
def test_missing_coverage_is_not_extrapolated(points):
|
||||
with pytest.raises(ValueError, match="coverage"):
|
||||
fit(points)
|
||||
|
||||
|
||||
def test_gap_is_rejected():
|
||||
with pytest.raises(ValueError, match="gap"):
|
||||
fit([(0, 8000), (3600, 8000)], max_gap=300)
|
||||
|
||||
|
||||
@pytest.mark.parametrize("value", [None, float("nan"), float("inf"), True])
|
||||
def test_bad_power_is_rejected(value):
|
||||
with pytest.raises(ValueError, match="finite measured"):
|
||||
fit([(0, value), (3600, 8000)])
|
||||
|
||||
|
||||
@pytest.mark.parametrize("status", ["estimated", "invalid", "unavailable"])
|
||||
def test_nonmeasured_quality_is_rejected(status):
|
||||
with pytest.raises(ValueError, match="finite measured"):
|
||||
fit([(0, 8000), (3600, 8000)], quality={0: SampleQuality(status=status)})
|
||||
|
||||
|
||||
def test_reset_and_sensor_change_are_rejected():
|
||||
for quality in (SampleQuality(reset=True), SampleQuality(generation="replacement")):
|
||||
with pytest.raises(ValueError, match="reset or generation"):
|
||||
fit([(0, 8000), (3600, 8000)], quality={3600: quality})
|
||||
|
||||
|
||||
@pytest.mark.parametrize("start_soc", [100, 99, 85])
|
||||
def test_full_to_full_and_small_soc_span_do_not_produce_estimates(start_soc):
|
||||
with pytest.raises(ValueError, match="SoC change is too small"):
|
||||
fit([(0, 8000), (3600, 8000)], start_soc=start_soc)
|
||||
|
||||
|
||||
def test_wrong_polarity_is_rejected():
|
||||
with pytest.raises(ValueError, match="disagrees"):
|
||||
fit([(0, -8000), (3600, -8000)])
|
||||
|
||||
|
||||
def test_hidden_saturation_cannot_be_fixed_by_end_point_fitting():
|
||||
with pytest.raises(ValueError, match="Fitted SoC leaves"):
|
||||
fit([(0, 24000), (1800, -8000), (3600, -8000)])
|
||||
|
||||
|
||||
def test_model_soc_is_not_an_accepted_reference():
|
||||
with pytest.raises(ValidationError):
|
||||
BatteryCapacityRequest.model_validate(dict(
|
||||
start=START,
|
||||
end=START + timedelta(hours=1),
|
||||
start_soc_percentage=20,
|
||||
soc_reference="calculated_soc",
|
||||
))
|
||||
|
||||
|
||||
@pytest_asyncio.fixture
|
||||
async def database_case(config_eos):
|
||||
from akkudoktoreos.core.coreabc import get_measurement
|
||||
from akkudoktoreos.measurement.quality import MeasurementSample
|
||||
|
||||
get_measurement()._db_reset_state()
|
||||
config_eos.merge_settings_from_dict(
|
||||
{
|
||||
"devices": {
|
||||
"batteries": {
|
||||
"battery": {
|
||||
"device_id": "battery",
|
||||
"capacity_wh": 12000,
|
||||
"charging_efficiency": 1,
|
||||
"discharging_efficiency": 1,
|
||||
"capacity_estimation": {
|
||||
"power_key": "battery_dc",
|
||||
"positive_power": "charging",
|
||||
},
|
||||
}
|
||||
}
|
||||
},
|
||||
"measurement": {
|
||||
"channels": {
|
||||
"battery_dc": {
|
||||
"quantity": "power",
|
||||
"unit": "W",
|
||||
"integration_method": "hold",
|
||||
"max_gap_seconds": 3600,
|
||||
}
|
||||
}
|
||||
},
|
||||
}
|
||||
)
|
||||
(await get_measurement().import_samples(
|
||||
[
|
||||
MeasurementSample(
|
||||
date_time=START + timedelta(seconds=t), key="battery_dc", value=8000.0
|
||||
)
|
||||
for t in (0, 1800, 3600)
|
||||
]
|
||||
))
|
||||
try:
|
||||
yield config_eos
|
||||
finally:
|
||||
get_measurement()._db_reset_state()
|
||||
|
||||
|
||||
def test_http_reads_database_and_stores_only_explicit_estimate(database_case):
|
||||
from fastapi.testclient import TestClient
|
||||
|
||||
from akkudoktoreos.server.eos import app
|
||||
|
||||
client = TestClient(app)
|
||||
body = {
|
||||
"start": START.isoformat(),
|
||||
"end": (START + timedelta(hours=1)).isoformat(),
|
||||
"start_soc_percentage": 20,
|
||||
"soc_reference": "voltage_current_anchor",
|
||||
}
|
||||
response = client.post("/v1/measurement/battery-capacity/battery", json=body)
|
||||
assert response.status_code == 200, response.text
|
||||
assert response.json()["estimated_capacity_wh"] == pytest.approx(10000)
|
||||
battery = database_case.devices.batteries["battery"]
|
||||
assert battery.capacity_estimate is None
|
||||
assert battery.capacity_wh == 12000
|
||||
body["store_estimate"] = True
|
||||
response = client.post("/v1/measurement/battery-capacity/battery", json=body)
|
||||
assert response.status_code == 200, response.text
|
||||
battery = database_case.devices.batteries["battery"]
|
||||
assert battery.capacity_estimate is not None
|
||||
assert battery.capacity_estimate.estimated_capacity_wh == pytest.approx(10000)
|
||||
assert battery.capacity_wh == 12000
|
||||
database_case.merge_settings_from_dict({"optimization": {"genetic": {"individuals": 100}}})
|
||||
battery = database_case.devices.batteries["battery"]
|
||||
assert battery.capacity_estimate is not None
|
||||
assert battery.capacity_estimate.estimated_capacity_wh == pytest.approx(10000)
|
||||
assert battery.capacity_wh == 12000
|
||||
|
||||
# The estimate survives normal config serialization without becoming capacity_wh.
|
||||
data = database_case.to_config_json()
|
||||
assert '"estimated_capacity_wh": 10000.0' in data
|
||||
assert '"capacity_wh": 12000' in data
|
||||
previous = battery.capacity_estimate
|
||||
body["start_soc_percentage"] = 100
|
||||
assert client.post("/v1/measurement/battery-capacity/battery", json=body).status_code == 422
|
||||
assert battery.capacity_estimate is previous
|
||||
@@ -0,0 +1,106 @@
|
||||
from unittest.mock import AsyncMock
|
||||
|
||||
"""Contracts for typed channels sharing the existing measurement storage."""
|
||||
|
||||
# ruff: noqa: S101
|
||||
|
||||
import pytest
|
||||
from pydantic import ValidationError
|
||||
|
||||
from akkudoktoreos.core.coreabc import get_measurement
|
||||
from akkudoktoreos.measurement.measurement import (
|
||||
MeasurementChannelSettings,
|
||||
MeasurementCommonSettings,
|
||||
MeasurementDataRecord,
|
||||
)
|
||||
from akkudoktoreos.utils.datetimeutil import to_datetime
|
||||
|
||||
CHANNELS = {
|
||||
"house_power": dict(quantity="power", unit="W", integration_method="hold", max_gap_seconds=120),
|
||||
"house_meter": dict(quantity="cumulative_energy", unit="kWh"),
|
||||
"house_interval": dict(
|
||||
quantity="interval_energy", unit="Wh", interval_seconds=900, timestamp_reference="start"
|
||||
),
|
||||
}
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"definition",
|
||||
[
|
||||
dict(quantity="power", unit="kWh", integration_method="hold", max_gap_seconds=60),
|
||||
dict(quantity="power", unit="W"),
|
||||
dict(quantity="power", unit="W", integration_method="hold", max_gap_seconds=0),
|
||||
dict(quantity="cumulative_energy", unit="W"),
|
||||
dict(quantity="cumulative_energy", unit="kWh", timestamp_reference="end"),
|
||||
dict(quantity="interval_energy", unit="Wh", interval_seconds=900),
|
||||
dict(
|
||||
quantity="interval_energy", unit="Wh", interval_seconds=True, timestamp_reference="end"
|
||||
),
|
||||
],
|
||||
)
|
||||
def test_reject_ambiguous_channel(definition):
|
||||
with pytest.raises(ValidationError):
|
||||
MeasurementChannelSettings(**definition)
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"legacy_field",
|
||||
["load_emr_keys", "grid_import_emr_keys", "grid_export_emr_keys", "pv_production_emr_keys"],
|
||||
)
|
||||
def test_legacy_keys_keep_meter_semantics(legacy_field):
|
||||
settings = MeasurementCommonSettings.model_validate(dict(**{legacy_field: ["legacy"]}, channels=CHANNELS))
|
||||
assert settings.keys == sorted(["legacy", *CHANNELS])
|
||||
compatible = MeasurementCommonSettings.model_validate(dict(
|
||||
**{legacy_field: ["legacy"]}, channels={"legacy": CHANNELS["house_meter"]}
|
||||
))
|
||||
assert compatible.keys == ["legacy"]
|
||||
with pytest.raises(ValidationError, match="must remain"):
|
||||
MeasurementCommonSettings.model_validate(dict(
|
||||
**{legacy_field: ["legacy"]}, channels={"legacy": CHANNELS["house_power"]}
|
||||
))
|
||||
|
||||
|
||||
@pytest.mark.parametrize("key", ["", " x", "date_time", "configured_data", "keys", "_private"])
|
||||
def test_reject_reserved_key(key):
|
||||
with pytest.raises(ValidationError):
|
||||
MeasurementCommonSettings.model_validate(dict(channels={key: CHANNELS["house_meter"]}))
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_existing_import_and_file_reload(config_eos, tmp_path, monkeypatch):
|
||||
"""All three quantities retain their raw values through the existing file path."""
|
||||
from akkudoktoreos.core.dataabc import DataSequence
|
||||
|
||||
measurement = get_measurement()
|
||||
previous_settings = config_eos.measurement
|
||||
previous_records = measurement.records
|
||||
previous_folder = config_eos.general.data_folder_path
|
||||
try:
|
||||
config_eos.measurement = MeasurementCommonSettings.model_validate(dict(channels=CHANNELS))
|
||||
config_eos.general.data_folder_path = tmp_path
|
||||
measurement._db_reset_state()
|
||||
values = dict(house_power=800.0, house_meter=12345.6, house_interval=200.0)
|
||||
for key, value in values.items():
|
||||
(await measurement.update_value(to_datetime("2026-09-10T18:00:00Z"), key, value))
|
||||
assert set(values).issubset(measurement.record_keys)
|
||||
monkeypatch.setattr(DataSequence, "save", AsyncMock(return_value=False))
|
||||
monkeypatch.setattr(DataSequence, "load", AsyncMock(return_value=False))
|
||||
assert (await measurement.save())
|
||||
measurement._db_reset_state()
|
||||
assert (await measurement.load())
|
||||
assert len(measurement.records) == 1
|
||||
for key, value in values.items():
|
||||
assert measurement.records[0][key] == value
|
||||
restored = MeasurementDataRecord.model_validate_json(
|
||||
measurement.records[0].model_dump_json()
|
||||
)
|
||||
assert restored.configured_data == values
|
||||
settings = MeasurementCommonSettings.model_validate_json(
|
||||
config_eos.measurement.model_dump_json()
|
||||
)
|
||||
assert settings.model_dump() == config_eos.measurement.model_dump()
|
||||
finally:
|
||||
measurement._db_reset_state()
|
||||
measurement.records = previous_records
|
||||
config_eos.measurement = previous_settings
|
||||
config_eos.general.data_folder_path = previous_folder
|
||||
@@ -0,0 +1,135 @@
|
||||
"""Physical and temporal contracts for interval energy."""
|
||||
|
||||
# ruff: noqa: S101
|
||||
|
||||
from datetime import datetime, timedelta, timezone
|
||||
from typing import Any
|
||||
from zoneinfo import ZoneInfo
|
||||
|
||||
import pytest
|
||||
|
||||
from akkudoktoreos.measurement.energy import energy_intervals
|
||||
from akkudoktoreos.measurement.measurement import MeasurementChannelSettings
|
||||
from akkudoktoreos.utils.datetimeutil import to_datetime
|
||||
|
||||
START = datetime(2026, 9, 10, tzinfo=timezone.utc)
|
||||
|
||||
|
||||
def channel(quantity="power", **kwargs):
|
||||
defaults: dict[str, dict[str, Any]] = {
|
||||
"power": dict(unit="W", integration_method="hold", max_gap_seconds=900),
|
||||
"cumulative_energy": dict(unit="kWh"),
|
||||
"interval_energy": dict(unit="Wh", interval_seconds=900, timestamp_reference="start"),
|
||||
}
|
||||
return MeasurementChannelSettings(quantity=quantity, **(defaults[quantity] | kwargs))
|
||||
|
||||
|
||||
def convert(points, config=None, seconds=900):
|
||||
return energy_intervals(
|
||||
[(START + timedelta(seconds=t), v) for t, v in points],
|
||||
config or channel(),
|
||||
START,
|
||||
START + timedelta(seconds=seconds),
|
||||
)
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"config,points",
|
||||
[
|
||||
(channel(), [(0, 800), (900, 800)]),
|
||||
(channel(unit="kW"), [(0, 0.8), (900, 0.8)]),
|
||||
(channel("cumulative_energy"), [(0, 10), (900, 10.2)]),
|
||||
(channel("interval_energy"), [(0, 200)]),
|
||||
(channel("interval_energy", timestamp_reference="end"), [(900, 200)]),
|
||||
],
|
||||
)
|
||||
def test_equivalent_measurements(config, points):
|
||||
result = convert(points, config)[0]
|
||||
assert result.energy_wh == pytest.approx(200)
|
||||
assert result.coverage_seconds == 900
|
||||
assert result.coverage_status == "complete"
|
||||
|
||||
|
||||
def test_time_weighting_and_linear_interpolation():
|
||||
assert convert([(0, 0), (600, 1200), (900, 1200)])[0].energy_wh == 100
|
||||
assert convert([(0, 0), (900, 1600)], channel(integration_method="linear"))[0].energy_wh == 200
|
||||
|
||||
|
||||
def test_gap_and_no_extrapolation():
|
||||
result = convert([(0, 800), (300, 800)])[0]
|
||||
assert result.energy_wh is None
|
||||
assert result.observed_energy_wh == pytest.approx(800 / 12)
|
||||
assert result.coverage_status == "partial"
|
||||
assert convert([(0, 800), (900, 800)], channel(max_gap_seconds=60))[0].energy_wh is None
|
||||
assert convert([])[0].coverage_status == "missing"
|
||||
|
||||
|
||||
def test_null_breaks_hold_at_outage():
|
||||
result = convert([(0, 800), (300, None), (600, 800), (900, 800)])[0]
|
||||
assert result.coverage_seconds == 600
|
||||
assert result.coverage_status == "partial"
|
||||
assert len(result.coverage_ranges) == 2
|
||||
|
||||
|
||||
def test_reset_does_not_create_negative_consumption():
|
||||
result = convert([(0, 10), (450, 0), (900, 0.1)], channel("cumulative_energy"))[0]
|
||||
assert result.energy_wh is None
|
||||
assert result.observed_energy_wh == pytest.approx(100)
|
||||
assert "meter_reset" in result.flags
|
||||
|
||||
|
||||
def test_hour_allocation_conserves_energy_and_is_labelled():
|
||||
result = convert([(0, 1000)], channel("interval_energy", interval_seconds=3600), 3600)
|
||||
assert all(r.energy_wh is not None for r in result)
|
||||
assert sum(r.energy_wh for r in result if r.energy_wh is not None) == 1000
|
||||
assert all("allocated_energy" in r.methods for r in result)
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"quantity,points",
|
||||
[
|
||||
("power", [(0, 1), (0, 2)]),
|
||||
("interval_energy", [(0, 100), (450, 100)]),
|
||||
],
|
||||
)
|
||||
def test_ambiguous_time_support_rejected(quantity, points):
|
||||
with pytest.raises(ValueError):
|
||||
convert(points, channel(quantity))
|
||||
|
||||
|
||||
def test_nan_and_signed_power():
|
||||
assert convert([(0, float("nan")), (900, 800)])[0].energy_wh is None
|
||||
assert convert([(0, -800), (900, -800)])[0].energy_wh == -200
|
||||
|
||||
|
||||
@pytest.mark.parametrize("month,day,hours", [(3, 29, 23), (10, 25, 25)])
|
||||
def test_dst_calendar_day(month, day, hours):
|
||||
start = datetime(2026, month, day, tzinfo=ZoneInfo("Europe/Berlin"))
|
||||
end = start + timedelta(days=1)
|
||||
result = energy_intervals([(start, 0), (end, hours)], channel("cumulative_energy"), start, end)
|
||||
assert len(result) == hours * 4
|
||||
assert all(r.energy_wh is not None for r in result)
|
||||
assert sum(r.energy_wh for r in result if r.energy_wh is not None) == pytest.approx(hours * 1000)
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_measurement_wrapper_preserves_asynchronous_channels(config_eos):
|
||||
from akkudoktoreos.core.coreabc import get_measurement
|
||||
from akkudoktoreos.measurement.measurement import MeasurementCommonSettings
|
||||
|
||||
measurement = get_measurement()
|
||||
previous, records = config_eos.measurement, measurement.records
|
||||
try:
|
||||
config_eos.measurement = MeasurementCommonSettings(
|
||||
channels={"p": channel(), "other": channel()}
|
||||
)
|
||||
measurement._db_reset_state()
|
||||
(await measurement.update_value(to_datetime(START), "p", 800))
|
||||
(await measurement.update_value(to_datetime(START + timedelta(seconds=450)), "other", 1))
|
||||
(await measurement.update_value(to_datetime(START + timedelta(seconds=900)), "p", 800))
|
||||
result = (await measurement.energy_intervals("p", START, START + timedelta(seconds=900)))
|
||||
assert result[0].energy_wh == 200
|
||||
finally:
|
||||
measurement._db_reset_state()
|
||||
measurement.records = records
|
||||
config_eos.measurement = previous
|
||||
@@ -0,0 +1,366 @@
|
||||
"""API, persistence and physical balance contracts for typed measurements."""
|
||||
|
||||
# ruff: noqa: S101
|
||||
|
||||
from datetime import datetime, timedelta, timezone
|
||||
|
||||
import pytest
|
||||
from fastapi import FastAPI
|
||||
from fastapi.testclient import TestClient
|
||||
from pydantic import ValidationError
|
||||
|
||||
from akkudoktoreos.core.coreabc import get_database, get_measurement
|
||||
from akkudoktoreos.measurement.measurement import MeasurementCommonSettings
|
||||
from akkudoktoreos.measurement.quality import MeasurementSample
|
||||
from akkudoktoreos.server.rest.measurement import router
|
||||
|
||||
START = datetime(2026, 9, 10, tzinfo=timezone.utc)
|
||||
END = START + timedelta(seconds=900)
|
||||
POWER = dict(quantity="power", unit="W", integration_method="hold", max_gap_seconds=900)
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def setup_measurement(config_eos):
|
||||
measurement = get_measurement()
|
||||
settings, records = config_eos.measurement, measurement.records
|
||||
measurement._db_reset_state()
|
||||
|
||||
def configure(topology="direct", inputs=None, channels=None):
|
||||
inputs = inputs or [dict(key="site", branch="house", role="site")]
|
||||
config_eos.measurement = MeasurementCommonSettings.model_validate(dict(
|
||||
channels=channels or {item["key"]: POWER for item in inputs},
|
||||
household=dict(topology=topology, inputs=inputs),
|
||||
))
|
||||
return measurement
|
||||
|
||||
yield configure
|
||||
measurement._db_reset_state()
|
||||
measurement.records = records
|
||||
config_eos.measurement = settings
|
||||
|
||||
|
||||
async def write(measurement, key, points):
|
||||
(await measurement.import_samples(
|
||||
[
|
||||
MeasurementSample(
|
||||
date_time=START + timedelta(seconds=t),
|
||||
key=key,
|
||||
value=value,
|
||||
quality=quality if quality else {},
|
||||
)
|
||||
for t, value, quality in points
|
||||
]
|
||||
))
|
||||
|
||||
|
||||
async def constant(measurement, key, watts):
|
||||
(await write(measurement, key, [(0, watts, None), (900, watts, None)]))
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
@pytest.mark.parametrize(
|
||||
"topology,values",
|
||||
[
|
||||
("direct", {"site": 4000}),
|
||||
("separate_ac", {"grid": 1000, "pv": 2000, "battery": 1000}),
|
||||
("hybrid_ac", {"grid": 1000, "inverter": 3000}),
|
||||
],
|
||||
)
|
||||
async def test_topologies_and_device_subtraction(setup_measurement, topology, values):
|
||||
values = values | {"ev": 2000, "device": 800}
|
||||
m = setup_measurement(topology, [dict(key=k, branch=k, role=k) for k in values])
|
||||
for key, value in values.items():
|
||||
(await constant(m, key, value))
|
||||
rows = (await m.household_intervals(START, END))
|
||||
assert rows["site"][0].energy_wh == 1000
|
||||
assert rows["household"][0].energy_wh == 500
|
||||
assert rows["base"][0].energy_wh == 300
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_export_charge_and_polarity(setup_measurement):
|
||||
m = setup_measurement(
|
||||
"separate_ac",
|
||||
[
|
||||
dict(key="grid", branch="grid", role="grid"),
|
||||
dict(key="solar", branch="solar", role="pv"),
|
||||
dict(key="charge", branch="battery", role="battery", polarity=-1),
|
||||
],
|
||||
)
|
||||
for key, watts in dict(grid=-1000, solar=4000, charge=1000).items():
|
||||
(await constant(m, key, watts))
|
||||
assert (await m.household_intervals(START, END))["site"][0].energy_wh == 500
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
@pytest.mark.parametrize("grid,inverter", [(-6000, 7200), (4200, -3000)])
|
||||
async def test_hybrid_with_independent_ac_pv_and_ev(setup_measurement, grid, inverter):
|
||||
inputs = [dict(key=k, branch=k, role=k) for k in ("grid", "inverter", "pv", "ev")]
|
||||
m = setup_measurement("hybrid_ac", inputs)
|
||||
for key, watts in dict(grid=grid, inverter=inverter, pv=400, ev=600).items():
|
||||
(await constant(m, key, watts))
|
||||
rows = (await m.household_intervals(START, END))
|
||||
assert rows["site"][0].energy_wh == 400
|
||||
assert rows["household"][0].energy_wh == 250
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_hybrid_missing_ac_pv_is_not_zero(setup_measurement):
|
||||
m = setup_measurement(
|
||||
"hybrid_ac", [dict(key=k, branch=k, role=k) for k in ("grid", "inverter", "pv")]
|
||||
)
|
||||
(await constant(m, "grid", -6000))
|
||||
(await constant(m, "inverter", 7200))
|
||||
row = (await m.household_intervals(START, END))["site"][0]
|
||||
assert row.energy_wh is None
|
||||
assert row.coverage_seconds == 0
|
||||
|
||||
|
||||
def test_hybrid_rejects_separate_battery_to_avoid_double_counting(setup_measurement):
|
||||
with pytest.raises(ValidationError):
|
||||
setup_measurement(
|
||||
"hybrid_ac",
|
||||
[dict(key=k, branch=k, role=k) for k in ("grid", "inverter", "battery")],
|
||||
)
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_missing_ev_does_not_destroy_site(setup_measurement):
|
||||
m = setup_measurement(inputs=[dict(key=k, branch=k, role=k) for k in ("site", "ev")])
|
||||
(await constant(m, "site", 800))
|
||||
rows = (await m.household_intervals(START, END))
|
||||
assert rows["site"][0].energy_wh == 200
|
||||
assert rows["household"][0].energy_wh is None
|
||||
assert rows["household"][0].coverage_seconds == 0
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_intersection_integrates_actual_shape(setup_measurement):
|
||||
m = setup_measurement(inputs=[dict(key=k, branch=k, role=k) for k in ("site", "ev")])
|
||||
(await write(m, "site", [(0, 1000, None), (300, 2000, None), (600, None, None)]))
|
||||
(await write(m, "ev", [(300, 500, None), (900, 500, None)]))
|
||||
row = (await m.household_intervals(START, END))["household"][0]
|
||||
assert row.coverage_seconds == 300
|
||||
assert row.energy_wh is None
|
||||
assert row.observed_energy_wh == pytest.approx(125)
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_nonoverlapping_coverage_is_missing(setup_measurement):
|
||||
m = setup_measurement(inputs=[dict(key=k, branch=k, role=k) for k in ("site", "ev")])
|
||||
(await write(m, "site", [(0, 1000, None), (300, 1000, None)]))
|
||||
(await write(m, "ev", [(600, 500, None), (900, 500, None)]))
|
||||
row = (await m.household_intervals(START, END))["household"][0]
|
||||
assert row.observed_energy_wh is None
|
||||
assert row.coverage_seconds == 0
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"inputs",
|
||||
[
|
||||
[dict(key="p", branch="x", role="grid"), dict(key="p", branch="y", role="pv")],
|
||||
[dict(key="p", branch="x", role="grid"), dict(key="q", branch="x", role="pv")],
|
||||
[dict(key="p", branch="x", role="site"), dict(key="q", branch="y", role="pv")],
|
||||
[dict(key="unknown", branch="x", role="grid")],
|
||||
],
|
||||
)
|
||||
def test_invalid_balance_configuration(setup_measurement, inputs):
|
||||
with pytest.raises(ValidationError):
|
||||
setup_measurement("separate_ac", inputs, {"p": POWER, "q": POWER})
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_quality_reset_even_with_increasing_meter(setup_measurement):
|
||||
m = setup_measurement(channels={"site": dict(quantity="cumulative_energy", unit="Wh")})
|
||||
(await write(
|
||||
m,
|
||||
"site",
|
||||
[
|
||||
(0, 0, {"generation": "old"}),
|
||||
(450, 1000, {"generation": "new", "reset": True}),
|
||||
(900, 1100, {"generation": "new", "status": "estimated"}),
|
||||
],
|
||||
))
|
||||
row = (await m.energy_intervals("site", START, END))[0]
|
||||
assert row.energy_wh is None
|
||||
assert row.observed_energy_wh == 100
|
||||
assert set(row.flags) == {"meter_reset", "estimated"}
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_api_upsert_outage_validation_and_readback(setup_measurement):
|
||||
m = setup_measurement()
|
||||
app = FastAPI()
|
||||
app.include_router(router)
|
||||
with TestClient(app) as client:
|
||||
payload = [
|
||||
dict(
|
||||
date_time=START.isoformat(), key="site", value=800, quality={"status": "estimated"}
|
||||
),
|
||||
dict(date_time=END.isoformat(), key="site", value=800),
|
||||
]
|
||||
assert client.put("/v1/measurement/samples", json=payload).status_code == 200
|
||||
params = dict(key="site", start=START.isoformat(), end=END.isoformat())
|
||||
row = client.get("/v1/measurement/energy", params=params).json()[0]
|
||||
assert row["energy_wh"] == 200
|
||||
assert "estimated" in row["flags"]
|
||||
raw = client.get("/v1/measurement/samples", params=params).json()
|
||||
assert raw[0]["quality"]["status"] == "estimated"
|
||||
payload[0]["value"] = None
|
||||
payload[0]["quality"] = {"status": "unavailable"}
|
||||
assert client.put("/v1/measurement/samples", json=payload[:1]).status_code == 200
|
||||
row = client.get("/v1/measurement/energy", params=params).json()[0]
|
||||
assert row["energy_wh"] is None
|
||||
assert "unavailable" in row["flags"]
|
||||
assert len(m.records) == 2
|
||||
payload[0]["value"] = 0
|
||||
payload[0]["quality"] = {}
|
||||
assert client.put("/v1/measurement/samples", json=payload[:1]).status_code == 200
|
||||
assert client.get("/v1/measurement/energy", params=params).json()[0]["energy_wh"] == 0
|
||||
assert (
|
||||
client.get("/v1/measurement/household", params=params).json()["site"][0]["energy_wh"]
|
||||
== 0
|
||||
)
|
||||
# Validate a whole batch before changing any value.
|
||||
bad = [payload[0] | {"value": 999}, payload[0] | {"key": "unknown"}]
|
||||
assert client.put("/v1/measurement/samples", json=bad).status_code == 422
|
||||
assert (await m.energy_intervals("site", START, END))[0].energy_wh == 0
|
||||
assert (
|
||||
client.get(
|
||||
"/v1/measurement/energy", params=params | {"interval_seconds": 0}
|
||||
).status_code
|
||||
== 422
|
||||
)
|
||||
assert (
|
||||
client.get(
|
||||
"/v1/measurement/energy", params=params | {"start": "2026-09-10T00:00:00"}
|
||||
).status_code
|
||||
== 422
|
||||
)
|
||||
assert (
|
||||
client.get(
|
||||
"/v1/measurement/energy",
|
||||
params=params | {"end": (START + timedelta(days=32)).isoformat()},
|
||||
).status_code
|
||||
== 422
|
||||
)
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
@pytest.mark.parametrize("provider", [None, "SQLite", "LMDB"])
|
||||
async def test_quality_persists_through_storage_restart(
|
||||
setup_measurement, config_eos, tmp_path, provider
|
||||
):
|
||||
m = setup_measurement()
|
||||
folder, previous_db = config_eos.general.data_folder_path, config_eos.database.provider
|
||||
try:
|
||||
config_eos.general.data_folder_path = tmp_path
|
||||
config_eos.database.provider = provider
|
||||
(await write(
|
||||
m,
|
||||
"site",
|
||||
[
|
||||
(0, 800, {"status": "estimated", "generation": "meter1"}),
|
||||
(450, None, {"status": "unavailable"}),
|
||||
(900, 800, None),
|
||||
],
|
||||
))
|
||||
before = (await m.energy_intervals("site", START, END))
|
||||
assert (await m.save())
|
||||
if provider:
|
||||
(await get_database().close())
|
||||
m._db_reset_state()
|
||||
if not provider:
|
||||
assert (await m.load())
|
||||
# DB loads on demand via the bounded query, without a full-history load.
|
||||
assert (await m.energy_intervals("site", START, END)) == before
|
||||
assert m.records[0].sample_quality["site"].generation == "meter1"
|
||||
finally:
|
||||
if provider:
|
||||
(await get_database().close())
|
||||
m._db_reset_state()
|
||||
config_eos.database.provider = previous_db
|
||||
config_eos.general.data_folder_path = folder
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_query_passes_bounded_storage_window(setup_measurement, monkeypatch):
|
||||
m = setup_measurement()
|
||||
calls = []
|
||||
original = type(m).db_iterate_records
|
||||
|
||||
def record_window(self, start_timestamp=None, end_timestamp=None):
|
||||
calls.append((start_timestamp, end_timestamp))
|
||||
return original(self, start_timestamp, end_timestamp)
|
||||
|
||||
monkeypatch.setattr(type(m), "db_iterate_records", record_window)
|
||||
(await m.energy_intervals("site", START, END))
|
||||
assert calls and all(a is not None and b is not None for a, b in calls)
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_actual_server_routes_and_legacy_value_api(setup_measurement):
|
||||
setup_measurement()
|
||||
from akkudoktoreos.server.eos import app
|
||||
|
||||
# No lifespan: this test must not start schedulers or write application state.
|
||||
client = TestClient(app)
|
||||
try:
|
||||
for time in (START, END):
|
||||
response = client.put(
|
||||
"/v1/measurement/value",
|
||||
params={
|
||||
"datetime": time.isoformat(),
|
||||
"key": "site",
|
||||
"value": 800,
|
||||
},
|
||||
)
|
||||
assert response.status_code == 200, response.text
|
||||
response = client.get(
|
||||
"/v1/measurement/energy",
|
||||
params={
|
||||
"key": "site",
|
||||
"start": START.isoformat(),
|
||||
"end": END.isoformat(),
|
||||
},
|
||||
)
|
||||
assert response.status_code == 200
|
||||
assert response.json()[0]["energy_wh"] == 200
|
||||
assert "sample_quality" not in get_measurement().record_keys
|
||||
finally:
|
||||
client.close()
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_quality_merge_keeps_other_channels(setup_measurement):
|
||||
from akkudoktoreos.measurement.measurement import MeasurementDataRecord
|
||||
|
||||
m = setup_measurement(inputs=[dict(key=k, branch=k, role=k) for k in ("site", "ev")])
|
||||
(await write(m, "site", [(0, 800, {"status": "estimated"})]))
|
||||
(await write(m, "ev", [(0, None, {"status": "unavailable"})]))
|
||||
(await m.insert_by_datetime(
|
||||
MeasurementDataRecord.model_validate(dict(date_time=START, sample_quality={"site": {"status": "measured"}}))
|
||||
))
|
||||
assert m.records[0].sample_quality["site"].status == "measured"
|
||||
assert m.records[0].sample_quality["ev"].status == "unavailable"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_hour_energy_allocation_and_short_last_interval(setup_measurement):
|
||||
m = setup_measurement(
|
||||
channels={
|
||||
"site": dict(
|
||||
quantity="interval_energy",
|
||||
unit="Wh",
|
||||
interval_seconds=3600,
|
||||
timestamp_reference="start",
|
||||
)
|
||||
}
|
||||
)
|
||||
(await write(m, "site", [(0, 1000, None)]))
|
||||
rows = (await m.household_intervals(START, START + timedelta(seconds=1800)))["site"]
|
||||
assert [row.energy_wh for row in rows] == [250, 250]
|
||||
assert all("allocated_energy" in row.methods for row in rows)
|
||||
row = (await m.energy_intervals("site", START, START + timedelta(seconds=60)))[0]
|
||||
assert row.coverage_seconds == 60
|
||||
assert row.energy_wh == pytest.approx(1000 / 60)
|
||||
Reference in New Issue
Block a user