mirror of
https://github.com/Akkudoktor-EOS/EOS.git
synced 2026-10-10 08:26:38 +00:00
docs(integration): record tested checkpoint and remaining consolidation work
This commit is contained in:
Andreas
@@ -0,0 +1,168 @@
|
||||
# EOS consolidation — integration record
|
||||
|
||||
Status: in progress, not a completed optimizer port. Updated 2026-09-16.
|
||||
|
||||
## Pinned sources
|
||||
|
||||
- Official main: `afb7bcb8bc84769ec2febd34e2dea1b1ad44ddbe`.
|
||||
- Feature branch: `d2e2d58237339454dd8bb226f92677c5987f8b27`.
|
||||
- PR #1256: `6ebd343047b87819b57778359784a616452b5f76`, open, conflicts.
|
||||
- PR #1305: `6fb84765c4b6d2c032fdf83effcb2ea6597f18f9`, open, main target.
|
||||
- PR #1224: `0b12a34c11685822d97c3bc9f56c6292774cd930`, open, feature target.
|
||||
- PR #1304: `4b4b49f29902579a8deb34999d71eb2142c0620d`, open, feature target.
|
||||
- PR #1190 merged into main: GENETIC0 retained separately.
|
||||
|
||||
Sources: https://github.com/Akkudoktor-EOS/EOS/issues/1192 and linked PRs.
|
||||
Issue body, all issue comments (none), and PR metadata were retrieved from the GitHub API.
|
||||
Use `refs/remotes/origin/main`: local `refs/heads/origin/main` is ambiguous.
|
||||
|
||||
## Isolation and recovery
|
||||
|
||||
Original EOS worktree remains on the feature branch. No reset, stash or checkout there.
|
||||
Integration branch: `integration/eos-consolidation-20260916`, sibling directory
|
||||
`EOS-integration-20260916`. Unmodified main comparison worktree: `EOS-reference-20260916`.
|
||||
|
||||
Private backup: `%USERPROFILE%/.codex/backups/eos-20260916-120324`.
|
||||
102 modified/untracked files copied byte-for-byte with SHA256 verification; binary
|
||||
tracked and index patches, original status/refs, verified `repository.bundle`.
|
||||
See private `RESTORE.md` and `manifest.json` for recovery into a NEW checkout.
|
||||
Ignored runtime files stay in the original worktree; this is not an archive of its
|
||||
virtual environment, caches or ignored production datasets. Backup is private and
|
||||
may include credentials or real measurements; never stage or publish it.
|
||||
Only explicitly reviewed source/test/document paths are staged for new commits.
|
||||
|
||||
## Functional matrix
|
||||
|
||||
| Area | Evidence | Integration state |
|
||||
| --- | --- | --- |
|
||||
| Providers, SMARD, fee model | #1192 checked; main prediction modules and #1235 | Keep main implementation; provider equivalence not yet established |
|
||||
| GENETIC0 / legacy API | #1190; `core/ems.py`, `server/eos.py`, `optimization/genetic0` | Preserved; device regressions pass, full endpoint validation pending |
|
||||
| Device maps / algorithm conversion | #1256; `devices/settings`, `config/configmigrate.py` | Merged with original history; four merge conflicts resolved |
|
||||
| Runtime updates | #1305; `config/config.py`, `core/pydantic.py` | Merged with original history; configuration tests pass |
|
||||
| Stable IDs / LCOS migration | New `test_consolidation_config.py` | Four initial regression failures fixed; 55 configuration tests pass |
|
||||
| 15-minute battery/inverter physics | Feature `devices/genetic`, `prediction/interpolator.py` | Ported in c0796e1; parameter classes retained in device modules, export/slot/efficiency tests pass |
|
||||
| New GENETIC, adaptive evolution, export states | Feature `optimization/genetic/genetic.py` | Pending; current optimizer is not the complete feature optimizer |
|
||||
| Warm start alignment | Feature d2e2d58, `test_genetic_warm_start_alignment.py` | Pending |
|
||||
| Control horizon / forecast tail / terminal value | Feature a2f4ef6, `tailvalue.py`, `terminalvalue.py` | Primitives ported in f70a786; 22 tests pass. Optimizer/horizon/API integration pending |
|
||||
| EV deadlines / flexible consumers | Feature f24d9ea; #1256 cycle windows | Pending reconciliation: preserve both per-cycle windows and profile/deadline semantics |
|
||||
| Imported feed-in revenue | #1224 + #1304, 38 feature regression cases | Neither PR targets main; adapt after GENETIC preparation port, keep author credit |
|
||||
| Local calibrated PV | Feature f976335/6dc58c3/faed0fd | Pending; keep public provider ID PVForecastAkkudoktor per #1192 |
|
||||
| Algorithm-specific PDF | Main #1205 vs feature utils/visualize.py | Keep on-demand API; feature slot/tail fields pending |
|
||||
| Measurement channels/energy/quality/capacity/household | Original uncommitted source + five new tests | Ported in 7338baf to async storage and keyed devices; 127 related tests passed |
|
||||
| Request configuration learning | Original uncommitted `genetic/configrequest.py` | Secured only; pending |
|
||||
| HA-private core differences | Separate private repo | Not ported; no HA files modified, no deployment |
|
||||
|
||||
## Completed packages and tests
|
||||
|
||||
1. `d3f96c4`: merge #1256 onto pinned main. Conflicts in devices.py and three generated
|
||||
documents; preserve maps/settings and main evolution. Regenerate OpenAPI from
|
||||
merged code. Do not adopt PR build version bumps. 244 configuration/time-window
|
||||
tests + 81 device/simulation tests passed before subsequent packages.
|
||||
2. `b43c8df`: merge #1305 without conflicts. 275 config/configabc/configmigrate/pydantic
|
||||
tests passed.
|
||||
3. `e3987b2`: stable map identities, reject key/ID mismatch, preserve renamed LCOS,
|
||||
repair legacy EV charge-rate migration. 4 regression cases failed before fix;
|
||||
55 related tests passed afterwards. These are integration/PR defects, not claimed
|
||||
as unchanged-main defects.
|
||||
|
||||
Tests currently use the existing Python 3.11.9 environment read-only; pytest adds
|
||||
this worktree's src. No mutation of the original venv. Missing pypdf 6.19.0 installed into the private
|
||||
backup test-packages directory and exposed only through test-process PYTHONPATH.
|
||||
This is not yet validation against every pinned dependency / CI Python version.
|
||||
Synthetic pytest fixtures; no real devices, server deployment or production config.
|
||||
|
||||
## HA handoff and acceptance
|
||||
|
||||
Do not relocate active feature development to this integration branch yet.
|
||||
Device collections are keyed maps, with identical map key and device_id. New settings
|
||||
live in devices/settings; conversion methods derive algorithm parameters. LCOS
|
||||
configuration is `levelized_cost_of_storage_amt_kwh`; old files migrate.
|
||||
Runtime config bulk/granular precedence follows #1305. Both algorithms remain
|
||||
separate; `/optimize` uses GENETIC0 and cannot demonstrate the new GENETIC port.
|
||||
Pin a final tested commit for HA only after all remaining packages and end-to-end
|
||||
API/forecast/output tests. No public push, PR, comment or main update is authorized.
|
||||
|
||||
|
||||
4. `c0796e1`: slot-duration-aware battery/inverter simulation, bounded per-slot
|
||||
discharge/export, probabilistic direct-use energy model and GENETIC converter
|
||||
support for LCOS/export levels. Original author co-authorship retained.
|
||||
Fixed monetary goldens from the previous model are intentionally replaced with
|
||||
independent grid-flow repricing. Original simulation test passed on unchanged
|
||||
main; the changed result is not labelled a baseline failure.
|
||||
5. `f70a786`: bounded forecast reader, concave terminal value and deterministic
|
||||
forecast-tail primitives. 22 direct primitive tests passed. Does NOT activate
|
||||
the new optimization algorithm, warmstart, horizon handling or diagnostic API.
|
||||
6. `d9c434f`: keep main's charge-rate ndarray contract and exported constant after
|
||||
#1256. The first broad collection found this missed overlap.
|
||||
7. `7338baf`: typed measurement channels, sample quality, energy integration,
|
||||
household accounting and battery capacity estimation. Adapt all storage calls
|
||||
and endpoints to main's async API; map battery IDs through device collections.
|
||||
Capacity estimates survive subsequent runtime config updates and never change
|
||||
the active capacity_wh. 127 new/existing measurement tests passed, including
|
||||
JSON/SQLite/LMDB restart, partial coverage and real FastAPI routes with no lifespan.
|
||||
JSON measurement reload regression reproduced on unchanged afb7bcb (0 records
|
||||
after reload); applying the previously local singleton fix resolves it.
|
||||
8. `876756b`: replace stochastic GENETIC output equality with schema, independent
|
||||
per-slot cost/revenue accounting and physical-range assertions. GENETIC0 goldens
|
||||
remain unchanged. Short optimizer/PDF runs: 4 GENETIC0 and 4 GENETIC pass; one
|
||||
400-generation case for each algorithm skipped by the existing --finalize rule.
|
||||
New household configuration field has documentation metadata.
|
||||
|
||||
### Remaining integration risks and concrete next package
|
||||
|
||||
The current GENETIC orchestration is still the main-era optimizer with the newly
|
||||
ported device physics. It must NOT be described as the complete feature optimizer.
|
||||
Next port `optimization/genetic/genetic.py`, its parameter preparation and solution
|
||||
model together, carrying feature d2e2d58 warmstart and #1304 revenue fixes. Translate
|
||||
old top-level optimization settings to `optimization.genetic` and main's `_amt_kwh`
|
||||
prediction keys with explicit /1000 conversion. Use #1256 `to_genetic_*` converters
|
||||
instead of reintroducing parallel settings. Reconcile flexible profiles/deadlines
|
||||
with #1256 per-cycle windows/completed cycles before exposing the combined API.
|
||||
Both porting sides must retain their regression cases. Follow with local PV and
|
||||
algorithm-specific on-demand PDF; do not copy feature's synchronous/automatic-PDF
|
||||
server paths over main. Then adapt local configrequest.py to the current API.
|
||||
|
||||
No feature-provider robustness fixes are claimed ported merely because a provider
|
||||
with the same name exists on main. HA private-core differences remain unaudited.
|
||||
No full 2030-test suite run, --finalize optimization run, dependency-pin CI matrix,
|
||||
or physical installation validation has been completed.
|
||||
|
||||
### Backup verification
|
||||
|
||||
An independent clone of repository.bundle at the original feature HEAD was restored
|
||||
using the backup files. All 102 restored SHA256 hashes match. The original worktree
|
||||
HEAD and all 102 file hashes were also rechecked unchanged after the source ports.
|
||||
|
||||
|
||||
## Reproducing the focused acceptance run
|
||||
|
||||
Use a disposable environment with the repository dependencies plus pytest,
|
||||
pytest-asyncio, pytest-xprocess, pytest-cov and pypdf. The private backup contains
|
||||
`test-environment.json`, `integration-final.xml` and the complete collection log.
|
||||
Run from the integration worktree (not the original feature worktree):
|
||||
|
||||
```powershell
|
||||
python -m pytest tests/test_typingmodels.py tests/test_config.py tests/test_configabc.py tests/test_configmigrate.py tests/test_configfile.py tests/test_pydantic.py tests/test_consolidation_config.py tests/test_genetichomeappliance.py tests/test_genetic0battery.py tests/test_genetic0inverterefficiency.py tests/test_genetic0simulation.py tests/test_battery.py tests/test_inverter.py tests/test_inverter_efficiency.py tests/test_geneticsimulation.py tests/test_geneticsimulation2.py tests/test_terminalvalue.py tests/test_tailvalue_physics.py tests/test_interpolator.py tests/test_measurement_channels.py tests/test_measurement_energy.py tests/test_measurement_household.py tests/test_battery_capacity.py tests/test_consolidation_file_restore.py tests/test_measurement.py tests/test_genetic0optimize.py tests/test_geneticoptimize.py tests/test_doc.py -q --tb=short
|
||||
```
|
||||
|
||||
Do not inherit EOS_DIR/EOS_CONFIG_DIR from documentation generation when running
|
||||
pytest: these conflict with fixture-controlled temporary config directories.
|
||||
Generate OpenAPI after committing source changes; source-dirty version timestamps
|
||||
otherwise make exact documentation comparisons nondeterministic.
|
||||
|
||||
|
||||
## Final verified checkpoint for this work session
|
||||
|
||||
Focused combined run: **633 passed, 2 skipped in 65.46 s**. The skips are the
|
||||
existing 400-generation --finalize cases. Zero remaining failures in that run.
|
||||
All 2030 collected tests were collectable after supplying pypdf and resolving the
|
||||
charge-rate import overlap; collection alone is not a pass of the full suite.
|
||||
Ruff passed for the ported measurement/device/interpolator/primitive source files;
|
||||
git diff --check passed. OpenAPI/config documentation generation and the documentation
|
||||
comparison tests passed. The first combined run caught fixture state leakage in the
|
||||
new battery test; reset-before/after fixture isolation resolved it on the repeated run.
|
||||
|
||||
This is a tested partial integration checkpoint, NOT completion of the consolidation.
|
||||
The source packages are locally committed; release, full feature acceptance and
|
||||
upstream submission remain pending. Do not move active development or HA deployment
|
||||
here until the remaining optimizer/prognosis/output/request packages are integrated.
|
||||
@@ -0,0 +1,36 @@
|
||||
# EOS interface handoff to Home Assistant
|
||||
|
||||
2026-09-16 — intermediate integration only; do not deploy or switch HA dependency yet.
|
||||
|
||||
Source branch: `integration/eos-consolidation-20260916` on main `afb7bcb8`.
|
||||
See eos-consolidation.md for source commits, tests, backup, and remaining work.
|
||||
|
||||
- Configuration collections `devices.batteries`, `electric_vehicles`, `inverters`,
|
||||
`home_appliances` are maps keyed by stable device ID. A supplied `device_id` must
|
||||
match its map key; omitted IDs use that key. Old lists migrate. LCOS configuration
|
||||
uses `levelized_cost_of_storage_amt_kwh`; old files retain their values.
|
||||
- Algorithm settings are separate under `optimization.genetic` / `genetic0`.
|
||||
Existing main configuration uses `interval_sec` and `horizon_hours` there.
|
||||
Do not send feature-branch top-level interval/horizon settings without migration.
|
||||
- #1305 runtime bulk/granular changes are retained above file/environment sources;
|
||||
config persistence is still an explicit existing save operation.
|
||||
- New measurement routes: PUT/GET `/v1/measurement/samples`, GET
|
||||
`/v1/measurement/energy`, GET `/v1/measurement/household`, POST
|
||||
`/v1/measurement/battery-capacity/{battery_id}`. OpenAPI describes request schemas.
|
||||
Sample times require timezone; values remain in declared raw units, derived energy
|
||||
is Wh and includes coverage/quality. Missing data is not implicitly zero energy.
|
||||
- Capacity estimates are separate evidence; even store_estimate=true never replaces
|
||||
devices.batteries[id].capacity_wh. Explicitly stored estimates survive runtime
|
||||
bulk changes; existing config save controls disk persistence.
|
||||
- Python measurement storage-facing methods are async and must be awaited. The HTTP
|
||||
schema remains ordinary JSON; HA does not need to mirror EOS internals.
|
||||
- Legacy POST `/optimize` remains GENETIC0. The existing algorithm-specific raw result
|
||||
route is GET `/v1/energy-management/optimization/solution/{algorithm}`. The local
|
||||
feature ConfigOptimizationRequest and its `/v1/optimize` route are NOT yet ported.
|
||||
- New GENETIC orchestration/warmstart, imported-tariff #1304 fix, flexible-consumer
|
||||
reconciliation, calibrated local PV, complete output/PDF and forecast handling
|
||||
remain pending. Passing GENETIC0 or primitive tests does not cover these.
|
||||
|
||||
No HA repository changes, lab deployment, real device control or production config
|
||||
were performed. Retire the private HA core only after its differences are audited
|
||||
and a final EOS commit passes the full feature/API acceptance scenarios.
|
||||
Reference in New Issue
Block a user