mirror of
https://github.com/Akkudoktor-EOS/EOS.git
synced 2026-10-30 09:41:57 +00:00
Bobby NoelteandGitHub
7554e2dc47
delete_by_datetime() with no bounds was documented as "delete all
records", but db_delete_records() assumed that
_db_ensure_loaded(start=None, end=None) loads every record into memory
before building the delete list. That is wrong: _db_ensure_loaded()
implements a NONE -> INITIAL -> FULL phase machine, and with both bounds
None in the NONE phase it loads only an *initial window* and leaves the
phase at INITIAL. Records outside that window never enter
_db_sorted_timestamps, so they are never marked for deletion and survive
in the physical store. Symptom: after a "wipe", db_count_records() still
returned N records (observed: 4 leftovers from a previous dense fixture).
A second delete_by_datetime() call happened to escalate INITIAL -> FULL
and deleted the rest, which is why the bug looked intermittent and why
stacked test runs accumulated rows.
Fix: in db_delete_records(), when both bounds are None, call
self._db_load_full() directly; otherwise keep the existing
_db_ensure_loaded(start, end) path. The delete list is then complete and
the "wipe" fixture actually wipes.
delete_by_datetime() remains a soft delete (rows are only added to
_db_deleted_timestamps; the physical DELETE happens on the next flush),
unchanged by this commit.
Besides this an additional fix was added:
* fix(tests): assert UTC-instant ordering in test_timestamps_remain_sorted
The test asserted `dts == sorted(dts)` on seq.records[].date_time, which
are tz-aware DateTime values in Europe/Berlin. Compaction buckets on a
UTC grid but records are materialized in local time, so across a DST
fall-back the local wall-clock sequence is non-monotonic even though the
UTC instants are ordered (e.g. 00:45 UTC -> 02:45 CEST, 01:00 UTC ->
02:00 CET). sorted() compares wall-clock fields, so it flags the pair at
the seam (index 1248 on 2026-10-25 for Europe/Berlin) as out of order.
The assertion is unsatisfiable for any 15-min grid spanning a fall-back
boundary when records are labelled in local time.
The actual invariant is "ascending by UTC instant", which is what
_db_sorted_timestamps guarantees and what the adjacent alignment tests
already assert via int(rec.date_time.timestamp()). Change the assertion
to compare integer UTC epochs:
epochs = [int(r.date_time.timestamp()) for r in seq.records if r.date_time]
assert epochs == sorted(epochs)
Note: this test was seasonally flaky. It only fails while a DST
transition lies inside the [now - 4 weeks, now] window used by
dense_energy_seq, which is why it newly started failing without any code
change. Upstream should ideally emit compacted records in UTC (or order
by UTC instant) so that wall-clock ordering is not silently assumed.
Signed-off-by: Bobby Noelte <b0661n0e17e@gmail.com>
239 lines
8.7 KiB
Markdown
239 lines
8.7 KiB
Markdown
## Adapter Configuration
|
||
|
||
<!-- pyml disable line-length -->
|
||
:::{table} adapter
|
||
:widths: 10 20 10 5 5 30
|
||
:align: left
|
||
|
||
| Name | Environment Variable | Type | Read-Only | Default | Description |
|
||
| ---- | -------------------- | ---- | --------- | ------- | ----------- |
|
||
| homeassistant | `EOS_ADAPTER__HOMEASSISTANT` | `HomeAssistantAdapterCommonSettings` | `rw` | `required` | Home Assistant adapter settings. |
|
||
| nodered | `EOS_ADAPTER__NODERED` | `NodeREDAdapterCommonSettings` | `rw` | `required` | NodeRED adapter settings. |
|
||
| provider | `EOS_ADAPTER__PROVIDER` | `Optional[list[str]]` | `rw` | `None` | List of adapter provider id(s) of provider(s) to be used. |
|
||
| providers | | `list[str]` | `ro` | `N/A` | Available adapter provider ids. |
|
||
:::
|
||
<!-- pyml enable line-length -->
|
||
|
||
<!-- pyml disable no-emphasis-as-heading -->
|
||
**Example Input**
|
||
<!-- pyml enable no-emphasis-as-heading -->
|
||
|
||
<!-- pyml disable line-length -->
|
||
```json
|
||
{
|
||
"adapter": {
|
||
"provider": [
|
||
"HomeAssistant"
|
||
],
|
||
"homeassistant": {
|
||
"config_entity_ids": null,
|
||
"load_emr_entity_ids": null,
|
||
"grid_export_emr_entity_ids": null,
|
||
"grid_import_emr_entity_ids": null,
|
||
"pv_production_emr_entity_ids": null,
|
||
"device_measurement_entity_ids": null,
|
||
"device_instruction_entity_ids": null,
|
||
"solution_entity_ids": null
|
||
},
|
||
"nodered": {
|
||
"host": "127.0.0.1",
|
||
"port": 1880
|
||
}
|
||
}
|
||
}
|
||
```
|
||
<!-- pyml enable line-length -->
|
||
|
||
<!-- pyml disable no-emphasis-as-heading -->
|
||
**Example Output**
|
||
<!-- pyml enable no-emphasis-as-heading -->
|
||
|
||
<!-- pyml disable line-length -->
|
||
```json
|
||
{
|
||
"adapter": {
|
||
"provider": [
|
||
"HomeAssistant"
|
||
],
|
||
"homeassistant": {
|
||
"config_entity_ids": null,
|
||
"load_emr_entity_ids": null,
|
||
"grid_export_emr_entity_ids": null,
|
||
"grid_import_emr_entity_ids": null,
|
||
"pv_production_emr_entity_ids": null,
|
||
"device_measurement_entity_ids": null,
|
||
"device_instruction_entity_ids": null,
|
||
"solution_entity_ids": null,
|
||
"homeassistant_entity_ids": [],
|
||
"eos_solution_entity_ids": [],
|
||
"eos_device_instruction_entity_ids": []
|
||
},
|
||
"nodered": {
|
||
"host": "127.0.0.1",
|
||
"port": 1880
|
||
},
|
||
"providers": [
|
||
"HomeAssistant",
|
||
"NodeRED"
|
||
]
|
||
}
|
||
}
|
||
```
|
||
<!-- pyml enable line-length -->
|
||
|
||
### Common settings for the NodeRED adapter
|
||
|
||
The Node-RED adapter sends to HTTP IN nodes.
|
||
|
||
This is the example flow:
|
||
|
||
`[HTTP In <URL>] -> [Function (parse payload)] -> [Debug] -> [HTTP Response]`
|
||
|
||
There are two URLs that are used:
|
||
|
||
- GET /eos/data_aquisition
|
||
The GET is issued before the optimization.
|
||
- POST /eos/control_dispatch
|
||
The POST is issued after the optimization.
|
||
|
||
<!-- pyml disable line-length -->
|
||
:::{table} adapter::nodered
|
||
:widths: 10 10 5 5 30
|
||
:align: left
|
||
|
||
| Name | Type | Read-Only | Default | Description |
|
||
| ---- | ---- | --------- | ------- | ----------- |
|
||
| host | `Optional[str]` | `rw` | `127.0.0.1` | Node-RED server IP address. Defaults to 127.0.0.1. |
|
||
| port | `Optional[int]` | `rw` | `1880` | Node-RED server IP port number. Defaults to 1880. |
|
||
:::
|
||
<!-- pyml enable line-length -->
|
||
|
||
<!-- pyml disable no-emphasis-as-heading -->
|
||
**Example Input/Output**
|
||
<!-- pyml enable no-emphasis-as-heading -->
|
||
|
||
<!-- pyml disable line-length -->
|
||
```json
|
||
{
|
||
"adapter": {
|
||
"nodered": {
|
||
"host": "127.0.0.1",
|
||
"port": 1880
|
||
}
|
||
}
|
||
}
|
||
```
|
||
<!-- pyml enable line-length -->
|
||
|
||
### Common settings for the home assistant adapter
|
||
|
||
<!-- pyml disable line-length -->
|
||
:::{table} adapter::homeassistant
|
||
:widths: 10 10 5 5 30
|
||
:align: left
|
||
|
||
| Name | Type | Read-Only | Default | Description |
|
||
| ---- | ---- | --------- | ------- | ----------- |
|
||
| config_entity_ids | `Optional[dict[str, str]]` | `rw` | `None` | Mapping of EOS config keys to Home Assistant entity IDs.
|
||
The config key has to be given by a ‘/’-separated path
|
||
e.g. devices/batteries/0/capacity_wh |
|
||
| device_instruction_entity_ids | `Optional[list[str]]` | `rw` | `None` | Entity IDs for device (resource) instructions to be updated by EOS.
|
||
The device ids (resource ids) have to be prepended by 'sensor.eos_' to build the entity_id.
|
||
E.g. The instruction for device id 'battery1' becomes the entity_id 'sensor.eos_battery1'. |
|
||
| device_measurement_entity_ids | `Optional[dict[str, str]]` | `rw` | `None` | Mapping of EOS device measurement keys to Home Assistant entity IDs. Registered keys such as 'battery1-soc-factor' are accepted; legacy underscore spellings are resolved when they uniquely match a registered key. |
|
||
| eos_device_instruction_entity_ids | `list[str]` | `ro` | `N/A` | Entity IDs for energy management instructions available at EOS. |
|
||
| eos_solution_entity_ids | `list[str]` | `ro` | `N/A` | Entity IDs for optimization solution available at EOS. |
|
||
| grid_export_emr_entity_ids | `Optional[list[str]]` | `rw` | `None` | Entity ID(s) of export to grid energy meter readings [kWh] |
|
||
| grid_import_emr_entity_ids | `Optional[list[str]]` | `rw` | `None` | Entity ID(s) of import from grid energy meter readings [kWh] |
|
||
| homeassistant_entity_ids | `list[str]` | `ro` | `N/A` | Entity IDs available at Home Assistant. |
|
||
| load_emr_entity_ids | `Optional[list[str]]` | `rw` | `None` | Entity ID(s) of load energy meter readings [kWh] |
|
||
| pv_production_emr_entity_ids | `Optional[list[str]]` | `rw` | `None` | Entity ID(s) of PV production energy meter readings [kWh] |
|
||
| solution_entity_ids | `Optional[list[str]]` | `rw` | `None` | Entity IDs for optimization solution keys to be updated by EOS.
|
||
The solution keys have to be prepended by 'sensor.eos_' to build the entity_id.
|
||
E.g. solution key 'battery1_idle_op_mode' becomes the entity_id 'sensor.eos_battery1_idle_op_mode'. |
|
||
:::
|
||
<!-- pyml enable line-length -->
|
||
|
||
<!-- pyml disable no-emphasis-as-heading -->
|
||
**Example Input**
|
||
<!-- pyml enable no-emphasis-as-heading -->
|
||
|
||
<!-- pyml disable line-length -->
|
||
```json
|
||
{
|
||
"adapter": {
|
||
"homeassistant": {
|
||
"config_entity_ids": {
|
||
"devices/batteries/0/capacity_wh": "sensor.battery1_capacity"
|
||
},
|
||
"load_emr_entity_ids": [
|
||
"sensor.load_energy_total_kwh"
|
||
],
|
||
"grid_export_emr_entity_ids": [
|
||
"sensor.grid_export_energy_total_kwh"
|
||
],
|
||
"grid_import_emr_entity_ids": [
|
||
"sensor.grid_import_energy_total_kwh"
|
||
],
|
||
"pv_production_emr_entity_ids": [
|
||
"sensor.pv_energy_total_kwh"
|
||
],
|
||
"device_measurement_entity_ids": {
|
||
"ev11-soc-factor": "sensor.ev11_soc_factor",
|
||
"battery1-soc-factor": "sensor.battery1_soc_factor"
|
||
},
|
||
"device_instruction_entity_ids": [
|
||
"sensor.eos_battery1"
|
||
],
|
||
"solution_entity_ids": [
|
||
"sensor.eos_battery1_idle_mode_mode"
|
||
]
|
||
}
|
||
}
|
||
}
|
||
```
|
||
<!-- pyml enable line-length -->
|
||
|
||
<!-- pyml disable no-emphasis-as-heading -->
|
||
**Example Output**
|
||
<!-- pyml enable no-emphasis-as-heading -->
|
||
|
||
<!-- pyml disable line-length -->
|
||
```json
|
||
{
|
||
"adapter": {
|
||
"homeassistant": {
|
||
"config_entity_ids": {
|
||
"devices/batteries/0/capacity_wh": "sensor.battery1_capacity"
|
||
},
|
||
"load_emr_entity_ids": [
|
||
"sensor.load_energy_total_kwh"
|
||
],
|
||
"grid_export_emr_entity_ids": [
|
||
"sensor.grid_export_energy_total_kwh"
|
||
],
|
||
"grid_import_emr_entity_ids": [
|
||
"sensor.grid_import_energy_total_kwh"
|
||
],
|
||
"pv_production_emr_entity_ids": [
|
||
"sensor.pv_energy_total_kwh"
|
||
],
|
||
"device_measurement_entity_ids": {
|
||
"ev11-soc-factor": "sensor.ev11_soc_factor",
|
||
"battery1-soc-factor": "sensor.battery1_soc_factor"
|
||
},
|
||
"device_instruction_entity_ids": [
|
||
"sensor.eos_battery1"
|
||
],
|
||
"solution_entity_ids": [
|
||
"sensor.eos_battery1_idle_mode_mode"
|
||
],
|
||
"homeassistant_entity_ids": [],
|
||
"eos_solution_entity_ids": [],
|
||
"eos_device_instruction_entity_ids": []
|
||
}
|
||
}
|
||
}
|
||
```
|
||
<!-- pyml enable line-length -->
|