EOS/docs/development/eos-consolidation.md

EOS consolidation — integration record

Status: in progress, not a completed optimizer port. Updated 2026-09-16.

Pinned sources

• Official main: afb7bcb8bc84769ec2febd34e2dea1b1ad44ddbe.
• Feature branch: d2e2d58237339454dd8bb226f92677c5987f8b27.
• PR #1256: 6ebd343047b87819b57778359784a616452b5f76, open, conflicts.
• PR #1305: 6fb84765c4b6d2c032fdf83effcb2ea6597f18f9, open, main target.
• PR #1224: 0b12a34c11685822d97c3bc9f56c6292774cd930, open, feature target.
• PR #1304: 4b4b49f29902579a8deb34999d71eb2142c0620d, open, feature target.
• PR #1190 merged into main: GENETIC0 retained separately.

Sources: https://github.com/Akkudoktor-EOS/EOS/issues/1192 and linked PRs. Issue body, all issue comments (none), and PR metadata were retrieved from the GitHub API. Use refs/remotes/origin/main: local refs/heads/origin/main is ambiguous.

Isolation and recovery

Original EOS worktree remains on the feature branch. No reset, stash or checkout there. Integration branch: integration/eos-consolidation-20260916, sibling directory EOS-integration-20260916. Unmodified main comparison worktree: EOS-reference-20260916.

Private backup: %USERPROFILE%/.codex/backups/eos-20260916-120324. 102 modified/untracked files copied byte-for-byte with SHA256 verification; binary tracked and index patches, original status/refs, verified repository.bundle. See private RESTORE.md and manifest.json for recovery into a NEW checkout. Ignored runtime files stay in the original worktree; this is not an archive of its virtual environment, caches or ignored production datasets. Backup is private and may include credentials or real measurements; never stage or publish it. Only explicitly reviewed source/test/document paths are staged for new commits.

Functional matrix

Area	Evidence	Integration state
Providers, SMARD, fee model	#1192 checked; main prediction modules and #1235	Keep main implementation; provider equivalence not yet established
GENETIC0 / legacy API	#1190; core/ems.py, server/eos.py, optimization/genetic0	Preserved; device regressions pass, full endpoint validation pending
Device maps / algorithm conversion	#1256; devices/settings, config/configmigrate.py	Merged with original history; four merge conflicts resolved
Runtime updates	#1305; config/config.py, core/pydantic.py	Merged with original history; configuration tests pass
Stable IDs / LCOS migration	New test_consolidation_config.py	Four initial regression failures fixed; 55 configuration tests pass
15-minute battery/inverter physics	Feature devices/genetic, prediction/interpolator.py	Ported in c0796e1; parameter classes retained in device modules, export/slot/efficiency tests pass
New GENETIC, adaptive evolution, export states	Feature optimization/genetic/genetic.py	Pending; current optimizer is not the complete feature optimizer
Warm start alignment	Feature d2e2d58, test_genetic_warm_start_alignment.py	Pending
Control horizon / forecast tail / terminal value	Feature a2f4ef6, tailvalue.py, terminalvalue.py	Primitives ported in f70a786; 22 tests pass. Optimizer/horizon/API integration pending
EV deadlines / flexible consumers	Feature f24d9ea; #1256 cycle windows	Pending reconciliation: preserve both per-cycle windows and profile/deadline semantics
Imported feed-in revenue	#1224 + #1304, 38 feature regression cases	Neither PR targets main; adapt after GENETIC preparation port, keep author credit
Local calibrated PV	Feature f976335/6dc58c3/faed0fd	Pending; keep public provider ID PVForecastAkkudoktor per #1192
Algorithm-specific PDF	Main #1205 vs feature utils/visualize.py	Keep on-demand API; feature slot/tail fields pending
Measurement channels/energy/quality/capacity/household	Original uncommitted source + five new tests	Ported in 7338baf to async storage and keyed devices; 127 related tests passed
Request configuration learning	Original uncommitted genetic/configrequest.py	Secured only; pending
HA-private core differences	Separate private repo	Not ported; no HA files modified, no deployment

Completed packages and tests

1. d3f96c4: merge #1256 onto pinned main. Conflicts in devices.py and three generated documents; preserve maps/settings and main evolution. Regenerate OpenAPI from merged code. Do not adopt PR build version bumps. 244 configuration/time-window tests + 81 device/simulation tests passed before subsequent packages.
2. b43c8df: merge #1305 without conflicts. 275 config/configabc/configmigrate/pydantic tests passed.
3. e3987b2: stable map identities, reject key/ID mismatch, preserve renamed LCOS, repair legacy EV charge-rate migration. 4 regression cases failed before fix; 55 related tests passed afterwards. These are integration/PR defects, not claimed as unchanged-main defects.

Tests currently use the existing Python 3.11.9 environment read-only; pytest adds this worktree's src. No mutation of the original venv. Missing pypdf 6.19.0 installed into the private backup test-packages directory and exposed only through test-process PYTHONPATH. This is not yet validation against every pinned dependency / CI Python version. Synthetic pytest fixtures; no real devices, server deployment or production config.

HA handoff and acceptance

Do not relocate active feature development to this integration branch yet. Device collections are keyed maps, with identical map key and device_id. New settings live in devices/settings; conversion methods derive algorithm parameters. LCOS configuration is levelized_cost_of_storage_amt_kwh; old files migrate. Runtime config bulk/granular precedence follows #1305. Both algorithms remain separate; /optimize uses GENETIC0 and cannot demonstrate the new GENETIC port. Pin a final tested commit for HA only after all remaining packages and end-to-end API/forecast/output tests. No public push, PR, comment or main update is authorized.

4. c0796e1: slot-duration-aware battery/inverter simulation, bounded per-slot discharge/export, probabilistic direct-use energy model and GENETIC converter support for LCOS/export levels. Original author co-authorship retained. Fixed monetary goldens from the previous model are intentionally replaced with independent grid-flow repricing. Original simulation test passed on unchanged main; the changed result is not labelled a baseline failure.
5. f70a786: bounded forecast reader, concave terminal value and deterministic forecast-tail primitives. 22 direct primitive tests passed. Does NOT activate the new optimization algorithm, warmstart, horizon handling or diagnostic API.
6. d9c434f: keep main's charge-rate ndarray contract and exported constant after #1256. The first broad collection found this missed overlap.
7. 7338baf: typed measurement channels, sample quality, energy integration, household accounting and battery capacity estimation. Adapt all storage calls and endpoints to main's async API; map battery IDs through device collections. Capacity estimates survive subsequent runtime config updates and never change the active capacity_wh. 127 new/existing measurement tests passed, including JSON/SQLite/LMDB restart, partial coverage and real FastAPI routes with no lifespan. JSON measurement reload regression reproduced on unchanged afb7bcb (0 records after reload); applying the previously local singleton fix resolves it.
8. 876756b: replace stochastic GENETIC output equality with schema, independent per-slot cost/revenue accounting and physical-range assertions. GENETIC0 goldens remain unchanged. Short optimizer/PDF runs: 4 GENETIC0 and 4 GENETIC pass; one 400-generation case for each algorithm skipped by the existing --finalize rule. New household configuration field has documentation metadata.

Remaining integration risks and concrete next package

The current GENETIC orchestration is still the main-era optimizer with the newly ported device physics. It must NOT be described as the complete feature optimizer. Next port optimization/genetic/genetic.py, its parameter preparation and solution model together, carrying feature d2e2d58 warmstart and #1304 revenue fixes. Translate old top-level optimization settings to optimization.genetic and main's _amt_kwh prediction keys with explicit /1000 conversion. Use #1256 to_genetic_* converters instead of reintroducing parallel settings. Reconcile flexible profiles/deadlines with #1256 per-cycle windows/completed cycles before exposing the combined API. Both porting sides must retain their regression cases. Follow with local PV and algorithm-specific on-demand PDF; do not copy feature's synchronous/automatic-PDF server paths over main. Then adapt local configrequest.py to the current API.

No feature-provider robustness fixes are claimed ported merely because a provider with the same name exists on main. HA private-core differences remain unaudited. No full 2030-test suite run, --finalize optimization run, dependency-pin CI matrix, or physical installation validation has been completed.

Backup verification

An independent clone of repository.bundle at the original feature HEAD was restored using the backup files. All 102 restored SHA256 hashes match. The original worktree HEAD and all 102 file hashes were also rechecked unchanged after the source ports.

Reproducing the focused acceptance run

Use a disposable environment with the repository dependencies plus pytest, pytest-asyncio, pytest-xprocess, pytest-cov and pypdf. The private backup contains test-environment.json, integration-final.xml and the complete collection log. Run from the integration worktree (not the original feature worktree):

python -m pytest tests/test_typingmodels.py tests/test_config.py tests/test_configabc.py tests/test_configmigrate.py tests/test_configfile.py tests/test_pydantic.py tests/test_consolidation_config.py tests/test_genetichomeappliance.py tests/test_genetic0battery.py tests/test_genetic0inverterefficiency.py tests/test_genetic0simulation.py tests/test_battery.py tests/test_inverter.py tests/test_inverter_efficiency.py tests/test_geneticsimulation.py tests/test_geneticsimulation2.py tests/test_terminalvalue.py tests/test_tailvalue_physics.py tests/test_interpolator.py tests/test_measurement_channels.py tests/test_measurement_energy.py tests/test_measurement_household.py tests/test_battery_capacity.py tests/test_consolidation_file_restore.py tests/test_measurement.py tests/test_genetic0optimize.py tests/test_geneticoptimize.py tests/test_doc.py -q --tb=short

Do not inherit EOS_DIR/EOS_CONFIG_DIR from documentation generation when running pytest: these conflict with fixture-controlled temporary config directories. Generate OpenAPI after committing source changes; source-dirty version timestamps otherwise make exact documentation comparisons nondeterministic.

Final verified checkpoint for this work session

Focused combined run: 633 passed, 2 skipped in 65.46 s. The skips are the existing 400-generation --finalize cases. Zero remaining failures in that run. All 2030 collected tests were collectable after supplying pypdf and resolving the charge-rate import overlap; collection alone is not a pass of the full suite. Ruff passed for the ported measurement/device/interpolator/primitive source files; git diff --check passed. OpenAPI/config documentation generation and the documentation comparison tests passed. The first combined run caught fixture state leakage in the new battery test; reset-before/after fixture isolation resolved it on the repeated run.

This is a tested partial integration checkpoint, NOT completion of the consolidation. The source packages are locally committed; release, full feature acceptance and upstream submission remain pending. Do not move active development or HA deployment here until the remaining optimizer/prognosis/output/request packages are integrated.