fix(database): make delete_by_datetime(None, None) a true full wipe (#1476)

delete_by_datetime() with no bounds was documented as "delete all
records", but db_delete_records() assumed that
_db_ensure_loaded(start=None, end=None) loads every record into memory
before building the delete list. That is wrong: _db_ensure_loaded()
implements a NONE -> INITIAL -> FULL phase machine, and with both bounds
None in the NONE phase it loads only an *initial window* and leaves the
phase at INITIAL. Records outside that window never enter
_db_sorted_timestamps, so they are never marked for deletion and survive
in the physical store. Symptom: after a "wipe", db_count_records() still
returned N records (observed: 4 leftovers from a previous dense fixture).
A second delete_by_datetime() call happened to escalate INITIAL -> FULL
and deleted the rest, which is why the bug looked intermittent and why
stacked test runs accumulated rows.

Fix: in db_delete_records(), when both bounds are None, call
self._db_load_full() directly; otherwise keep the existing
_db_ensure_loaded(start, end) path. The delete list is then complete and
the "wipe" fixture actually wipes.

delete_by_datetime() remains a soft delete (rows are only added to
_db_deleted_timestamps; the physical DELETE happens on the next flush),
unchanged by this commit.

Besides this an additional fix was added:

* fix(tests): assert UTC-instant ordering in test_timestamps_remain_sorted

The test asserted `dts == sorted(dts)` on seq.records[].date_time, which
are tz-aware DateTime values in Europe/Berlin. Compaction buckets on a
UTC grid but records are materialized in local time, so across a DST
fall-back the local wall-clock sequence is non-monotonic even though the
UTC instants are ordered (e.g. 00:45 UTC -> 02:45 CEST, 01:00 UTC ->
02:00 CET). sorted() compares wall-clock fields, so it flags the pair at
the seam (index 1248 on 2026-10-25 for Europe/Berlin) as out of order.
The assertion is unsatisfiable for any 15-min grid spanning a fall-back
boundary when records are labelled in local time.

The actual invariant is "ascending by UTC instant", which is what
_db_sorted_timestamps guarantees and what the adjacent alignment tests
already assert via int(rec.date_time.timestamp()). Change the assertion
to compare integer UTC epochs:

    epochs = [int(r.date_time.timestamp()) for r in seq.records if r.date_time]
    assert epochs == sorted(epochs)

Note: this test was seasonally flaky. It only fails while a DST
transition lies inside the [now - 4 weeks, now] window used by
dense_energy_seq, which is why it newly started failing without any code
change. Upstream should ideally emit compacted records in UTC (or order
by UTC instant) so that wall-clock ordering is not silently assumed.

Signed-off-by: Bobby Noelte <b0661n0e17e@gmail.com>
This commit is contained in:
Bobby Noelte
2026-10-29 15:27:41 +01:00
committed by GitHub
parent c425fd58c4
commit 7554e2dc47
7 changed files with 62 additions and 23 deletions
+5 -5
View File
@@ -140,7 +140,7 @@ e.g. devices/batteries/0/capacity_wh |
| device_instruction_entity_ids | `Optional[list[str]]` | `rw` | `None` | Entity IDs for device (resource) instructions to be updated by EOS.
The device ids (resource ids) have to be prepended by 'sensor.eos_' to build the entity_id.
E.g. The instruction for device id 'battery1' becomes the entity_id 'sensor.eos_battery1'. |
| device_measurement_entity_ids | `Optional[dict[str, str]]` | `rw` | `None` | Mapping of EOS measurement keys used by device (resource) simulations to Home Assistant entity IDs. |
| device_measurement_entity_ids | `Optional[dict[str, str]]` | `rw` | `None` | Mapping of EOS device measurement keys to Home Assistant entity IDs. Registered keys such as 'battery1-soc-factor' are accepted; legacy underscore spellings are resolved when they uniquely match a registered key. |
| eos_device_instruction_entity_ids | `list[str]` | `ro` | `N/A` | Entity IDs for energy management instructions available at EOS. |
| eos_solution_entity_ids | `list[str]` | `ro` | `N/A` | Entity IDs for optimization solution available at EOS. |
| grid_export_emr_entity_ids | `Optional[list[str]]` | `rw` | `None` | Entity ID(s) of export to grid energy meter readings [kWh] |
@@ -179,8 +179,8 @@ E.g. solution key 'battery1_idle_op_mode' becomes the entity_id 'sensor.eos_batt
"sensor.pv_energy_total_kwh"
],
"device_measurement_entity_ids": {
"ev11_soc_factor": "sensor.ev11_soc_factor",
"battery1_soc_factor": "sensor.battery1_soc_factor"
"ev11-soc-factor": "sensor.ev11_soc_factor",
"battery1-soc-factor": "sensor.battery1_soc_factor"
},
"device_instruction_entity_ids": [
"sensor.eos_battery1"
@@ -219,8 +219,8 @@ E.g. solution key 'battery1_idle_op_mode' becomes the entity_id 'sensor.eos_batt
"sensor.pv_energy_total_kwh"
],
"device_measurement_entity_ids": {
"ev11_soc_factor": "sensor.ev11_soc_factor",
"battery1_soc_factor": "sensor.battery1_soc_factor"
"ev11-soc-factor": "sensor.ev11_soc_factor",
"battery1-soc-factor": "sensor.battery1_soc_factor"
},
"device_instruction_entity_ids": [
"sensor.eos_battery1"
+16 -2
View File
@@ -1,6 +1,6 @@
# Akkudoktor-EOS
**Version**: `v0.3.0.dev2610261491542381`
**Version**: `v0.3.0.dev2610291327175670`
<!-- pyml disable line-length -->
**Description**: This project provides a comprehensive solution for simulating and optimizing an energy system based on renewable energy sources. With a focus on photovoltaic (PV) systems, battery storage (batteries), load management (consumer requirements), heat pumps, electric vehicles, and consideration of electricity price data, this system enables forecasting and optimization of energy flow and costs over a specified period.
@@ -2023,7 +2023,7 @@ Return:
**Request Body**:
- `application/json`: {
"anyOf": [
"oneOf": [
{
"$ref": "#/components/schemas/PowerMeasurement"
},
@@ -2052,6 +2052,20 @@ Return:
"$ref": "#/components/schemas/DDBCActuatorStatus"
}
],
"discriminator": {
"propertyName": "type",
"mapping": {
"PowerMeasurement": "#/components/schemas/PowerMeasurement",
"EnergyMeasurement": "#/components/schemas/EnergyMeasurement",
"PPBCPowerProfileStatus": "#/components/schemas/PPBCPowerProfileStatus",
"OMBCStatus": "#/components/schemas/OMBCStatus",
"FRBCActuatorStatus": "#/components/schemas/FRBCActuatorStatus",
"FRBCEnergyStatus": "#/components/schemas/FRBCEnergyStatus",
"FRBCStorageStatus": "#/components/schemas/FRBCStorageStatus",
"FRBCTimerStatus": "#/components/schemas/FRBCTimerStatus",
"DDBCActuatorStatus": "#/components/schemas/DDBCActuatorStatus"
}
},
"description": "Resource Status.",
"title": "Status"
}