# EOS consolidation — integration record Historical integration record, updated 2026-09-16. The completed optimizer port and current merge instructions are in [review handoff](review-handoff.md). Pending-work statements below describe earlier checkpoints. Current PR-ready packages, dependency order and development guidance: [PR workflow](pr-workflow.md). The first independent fix is ready locally on main; the complete feature consolidation remains open. ## Pinned sources - Official main: `4a3724424f98b5ce814d0340ddf3b6b5f1364752` (refreshed after initial checkpoint). - Feature branch: `d2e2d58237339454dd8bb226f92677c5987f8b27`. - PR #1256: `6ebd343047b87819b57778359784a616452b5f76`, open, conflicts. - PR #1305: `60b77f6da2da3a0f59518873fe7fbe9a92e733f4`, open, main target. Latest added commit is formatting only. - PR #1224: `0b12a34c11685822d97c3bc9f56c6292774cd930`, open, feature target. - PR #1304: `4b4b49f29902579a8deb34999d71eb2142c0620d`, open, feature target. - PR #1190 merged into main: GENETIC0 retained separately. Sources: [issue #1192](https://github.com/Akkudoktor-EOS/EOS/issues/1192) and linked PRs. Issue body, all issue comments (none), and PR metadata were retrieved from the GitHub API. Use `refs/remotes/origin/main`: local `refs/heads/origin/main` is ambiguous. ## Isolation and recovery Original EOS worktree remains on the feature branch. No reset, stash or checkout there. Integration branch: `integration/eos-consolidation-20260916`, sibling directory `EOS-integration-20260916`. Unmodified main comparison worktree: `EOS-reference-20260916`. Private backup: `%USERPROFILE%/.codex/backups/eos-20260916-120324`. 102 modified/untracked files copied byte-for-byte with SHA256 verification; binary tracked and index patches, original status/refs, verified `repository.bundle`. See private `RESTORE.md` and `manifest.json` for recovery into a NEW checkout. Ignored runtime files stay in the original worktree; this is not an archive of its virtual environment, caches or ignored production datasets. Backup is private and may include credentials or real measurements; never stage or publish it. Only explicitly reviewed source/test/document paths are staged for new commits. ## Functional matrix | Area | Evidence | Integration state | | --- | --- | --- | | Providers, SMARD, fee model | #1192 checked; main prediction modules and #1235 | Keep main implementation; provider equivalence not yet established | | GENETIC0 / legacy API | #1190; `core/ems.py`, `server/eos.py`, `optimization/genetic0` | Preserved; device regressions pass, full endpoint validation pending | | Device maps / algorithm conversion | #1256; `devices/settings`, `config/configmigrate.py` | Merged with original history; four merge conflicts resolved | | Runtime updates | #1305; `config/config.py`, `core/pydantic.py` | Merged with original history; configuration tests pass | | Stable IDs / LCOS migration | New `test_consolidation_config.py` | Four initial regression failures fixed; 55 configuration tests pass | | 15-minute battery/inverter physics | Feature `devices/genetic`, `prediction/interpolator.py` | Ported in c0796e1; parameter classes retained in device modules, export/slot/efficiency tests pass | | New GENETIC, adaptive evolution, export states | Feature `optimization/genetic/genetic.py` | Pending; current optimizer is not the complete feature optimizer | | Warm start alignment | Feature d2e2d58, `test_genetic_warm_start_alignment.py` | Pending | | Control horizon / forecast tail / terminal value | Feature a2f4ef6, `tailvalue.py`, `terminalvalue.py` | Primitives ported in f70a786; 22 tests pass. Optimizer/horizon/API integration pending | | EV deadlines / flexible consumers | Feature f24d9ea; #1256 cycle windows | Pending reconciliation: preserve both per-cycle windows and profile/deadline semantics | | Imported feed-in revenue | #1224 + #1304, 38 feature regression cases | Neither PR targets main; adapt after GENETIC preparation port, keep author credit | | Local calibrated PV | Feature f976335/6dc58c3/faed0fd | Pending; keep public provider ID PVForecastAkkudoktor per #1192 | | Algorithm-specific PDF | Main #1205 vs feature utils/visualize.py | Keep on-demand API; feature slot/tail fields pending | | Measurement channels/energy/quality/capacity/household | Original uncommitted source + five new tests | Ported in 7338baf to async storage and keyed devices; 127 related tests passed | | Request configuration learning | Original uncommitted `genetic/configrequest.py` | Secured only; pending | | HA-private core differences | Separate private repo | Not ported; no HA files modified, no deployment | ## Completed packages and tests 1. `d3f96c4`: merge #1256 onto pinned main. Conflicts in devices.py and three generated documents; preserve maps/settings and main evolution. Regenerate OpenAPI from merged code. Do not adopt PR build version bumps. 244 configuration/time-window tests + 81 device/simulation tests passed before subsequent packages. 2. `b43c8df`: merge #1305 without conflicts. 275 config/configabc/configmigrate/pydantic tests passed. 3. `e3987b2`: stable map identities, reject key/ID mismatch, preserve renamed LCOS, repair legacy EV charge-rate migration. 4 regression cases failed before fix; 55 related tests passed afterwards. These are integration/PR defects, not claimed as unchanged-main defects. Tests currently use the existing Python 3.11.9 environment read-only; pytest adds this worktree's src. No mutation of the original venv. Missing pypdf 6.19.0 installed into the private backup test-packages directory and exposed only through test-process PYTHONPATH. This is not yet validation against every pinned dependency / CI Python version. Synthetic pytest fixtures; no real devices, server deployment or production config. ## HA handoff and acceptance Do not relocate active feature development to this integration branch yet. Device collections are keyed maps, with identical map key and device_id. New settings live in devices/settings; conversion methods derive algorithm parameters. LCOS configuration is `levelized_cost_of_storage_amt_kwh`; old files migrate. Runtime config bulk/granular precedence follows #1305. Both algorithms remain separate; `/optimize` uses GENETIC0 and cannot demonstrate the new GENETIC port. Pin a final tested commit for HA only after all remaining packages and end-to-end API/forecast/output tests. No public push, PR, comment or main update is authorized. 1. `c0796e1`: slot-duration-aware battery/inverter simulation, bounded per-slot discharge/export, probabilistic direct-use energy model and GENETIC converter support for LCOS/export levels. Original author co-authorship retained. Fixed monetary goldens from the previous model are intentionally replaced with independent grid-flow repricing. Original simulation test passed on unchanged main; the changed result is not labelled a baseline failure. 2. `f70a786`: bounded forecast reader, concave terminal value and deterministic forecast-tail primitives. 22 direct primitive tests passed. Does NOT activate the new optimization algorithm, warmstart, horizon handling or diagnostic API. 3. `d9c434f`: keep main's charge-rate ndarray contract and exported constant after `#1256`. The first broad collection found this missed overlap. 4. `7338baf`: typed measurement channels, sample quality, energy integration, household accounting and battery capacity estimation. Adapt all storage calls and endpoints to main's async API; map battery IDs through device collections. Capacity estimates survive subsequent runtime config updates and never change the active capacity_wh. 127 new/existing measurement tests passed, including JSON/SQLite/LMDB restart, partial coverage and real FastAPI routes with no lifespan. JSON measurement reload regression reproduced on unchanged afb7bcb (0 records after reload); applying the previously local singleton fix resolves it. 5. `876756b`: replace stochastic GENETIC output equality with schema, independent per-slot cost/revenue accounting and physical-range assertions. GENETIC0 goldens remain unchanged. Short optimizer/PDF runs: 4 GENETIC0 and 4 GENETIC pass; one 400-generation case for each algorithm skipped by the existing --finalize rule. New household configuration field has documentation metadata. ### Remaining integration risks and concrete next package The current GENETIC orchestration is still the main-era optimizer with the newly ported device physics. It must NOT be described as the complete feature optimizer. Next port `optimization/genetic/genetic.py`, its parameter preparation and solution model together, carrying feature d2e2d58 warmstart and #1304 revenue fixes. Translate old top-level optimization settings to `optimization.genetic`. Check the actual prediction record units: current `elecprice_marketprice_wh` and `feed_in_tariff_wh` arrays already contain amount/Wh; only amount/kWh configuration is divided by 1000. Do not apply a second conversion to the existing *_wh arrays. Use #1256 `to_genetic_*` converters instead of reintroducing parallel settings. Reconcile flexible profiles/deadlines with #1256 per-cycle windows/completed cycles before exposing the combined API. Both porting sides must retain their regression cases. Follow with local PV and algorithm-specific on-demand PDF; do not copy feature's synchronous/automatic-PDF server paths over main. Then adapt local configrequest.py to the current API. No feature-provider robustness fixes are claimed ported merely because a provider with the same name exists on main. HA private-core differences remain unaudited. No full 2030-test suite run, --finalize optimization run, dependency-pin CI matrix, or physical installation validation has been completed. ### Backup verification An independent clone of repository.bundle at the original feature HEAD was restored using the backup files. All 102 restored SHA256 hashes match. The original worktree HEAD and all 102 file hashes were also rechecked unchanged after the source ports. ## Reproducing the focused acceptance run Use a disposable environment with the repository dependencies plus pytest, pytest-asyncio, pytest-xprocess, pytest-cov and pypdf. The private backup contains `test-environment.json`, `integration-final.xml` and the complete collection log. Run from the integration worktree (not the original feature worktree): ```powershell python -m pytest ` tests/test_typingmodels.py ` tests/test_config.py ` tests/test_configabc.py ` tests/test_configmigrate.py ` tests/test_configfile.py ` tests/test_pydantic.py ` tests/test_consolidation_config.py ` tests/test_genetichomeappliance.py ` tests/test_genetic0battery.py ` tests/test_genetic0inverterefficiency.py ` tests/test_genetic0simulation.py ` tests/test_battery.py ` tests/test_inverter.py ` tests/test_inverter_efficiency.py ` tests/test_geneticsimulation.py ` tests/test_geneticsimulation2.py ` tests/test_terminalvalue.py ` tests/test_tailvalue_physics.py ` tests/test_interpolator.py ` tests/test_measurement_channels.py ` tests/test_measurement_energy.py ` tests/test_measurement_household.py ` tests/test_battery_capacity.py ` tests/test_measurement_file_restore.py ` tests/test_measurement.py ` tests/test_genetic0optimize.py ` tests/test_geneticoptimize.py ` tests/test_doc.py ` -q ` --tb=short ``` Do not inherit EOS_DIR/EOS_CONFIG_DIR from documentation generation when running pytest: these conflict with fixture-controlled temporary config directories. Generate OpenAPI after committing source changes; source-dirty version timestamps otherwise make exact documentation comparisons nondeterministic. ## Final verified checkpoint for this work session Focused combined run: **633 passed, 2 skipped in 65.46 s**. The skips are the existing 400-generation --finalize cases. Zero remaining failures in that run. All 2030 collected tests were collectable after supplying pypdf and resolving the charge-rate import overlap; collection alone is not a pass of the full suite. Ruff passed for the ported measurement/device/interpolator/primitive source files; git diff --check passed. OpenAPI/config documentation generation and the documentation comparison tests passed. The first combined run caught fixture state leakage in the new battery test; reset-before/after fixture isolation resolved it on the repeated run. This is a tested partial integration checkpoint, NOT completion of the consolidation. The source packages are locally committed; release, full feature acceptance and upstream submission remain pending. Do not move active development or HA deployment here until the remaining optimizer/prognosis/output/request packages are integrated. ## PR-readiness verification, 2026-09-16 Current main `7ebe6d7` is incorporated. Two reviewable local packages now exist: `fix/measurement-json-reload` (`dba0c9c`, independently based on main) and `feat/measurement-energy-quality-capacity` (`ea3383e`, depends on the configuration integration base `d546f08`). The latter excludes the new optimizer/device physics. See [PR workflow](pr-workflow.md) and its concrete draft descriptions. The expanded integration run completed with 764 passed, 3 skipped and 2 documentation failures in 453.99 seconds. Both failures were solely stale generated OpenAPI version metadata, not schema or functional differences. Regenerated the two files; the focused rerun of all 5 documentation tests passed. No remaining failure from that selection. The complete expanded selection was not rerun after this documentation-only correction. Three skips: the two existing long --finalize optimizer cases and the development-only Energy-Charts forecast case. All 128 active Energy-Charts regressions passed. Independent package checks: JSON fix 49 tests; isolated measurement package 453 tests plus 5 documentation tests; capacity fixture isolation followed by 74 passing tests. Ruff and source formatting passed for the independent fix; measurement source Ruff passed. Full pinned Linux/Python 3.13 CI remains outstanding. All original 102 saved file hashes and original feature HEAD were checked unchanged. Both PR worktrees are clean, locally committed, and unpublished. This enables small independent PRs now; it does not complete the still-open optimizer/PV/output port. ## First upstream PR published, 2026-09-16 Explicit user approval received for publishing the standalone JSON restore fix. Fetched main `4a37244` (dependency update #1321) and rebased the standalone branch; its new head is `bdc754d12fd08e0da18d3156642c695f4bf67bd2`. All 49 measurement tests, source Ruff and formatting checks passed again. Pushed only `fix/measurement-json-reload` and created [PR #1322](https://github.com/Akkudoktor-EOS/EOS/pull/1322) against main. Verified the remote head and PR patch: one commit, exactly two files. The other integration/measurement branches were not pushed. The PR is conflict-free, not merged; GitHub CI was started and is being checked. Original HEAD and all 102 backup hashes were rechecked unchanged. Earlier notes saying all packages are unpublished are historical checkpoints; this section supersedes them for this fix. ## Parallel package preparation and compatibility On the user's explicit request, independent PV and tariff ports and Optimize-mode compatibility coverage are being prepared in separate worktrees. The main/integration configuration prerequisite was refreshed to main4a37244 and PR1305head60b77f6; 285 configuration/migration/Pydantic tests pass. The latest 1305 change only formats three files. See [PR integration matrix](pr-integration-matrix.md) for the seven remaining functional packages, existing upstream prerequisites, and combined Optimize acceptance gates. Parallel work is not authorization to publish every lane. ## Completed parallel checkpoint and first green CI Source checkpoint b684748 now combines local PV, imported tariff protection, device physics/cache corrections and the confirmed Optimize failure-path fix. The combined selection passed 277 tests (3 regular skips); measurement APIs and persistence then passed 132 tests; regenerated documentation passed all 5 checks. These are targeted local checks, not a full integration CI run. The complete new GENETIC orchestration, config-owned request and result/PDF port remain outstanding. See the updated integration matrix for exact dependencies and limitations. PR #1322 now has two commits through ce132ea. Its initial mypy error in a test's optional timestamp assertion was fixed; all current checks are green: 1,884 passed, 16 skipped in pinned Linux/Python 3.13 pytest, plus pre-commit/mypy, CodeQL and Docker build. PR remains open and mergeable. Only this branch has been published; no merge or HA deployment occurred. The finite CI follow-up was paused after verification. Original feature HEAD and all 102 backed-up file hashes remain intact.