A single 503 from Open-Meteo answered /v1/prediction/update with 400 and left
every provider after PVForecastAkkudoktorLocal unrun: the provider raised on the
first failed request, and PredictionContainer.update_data re-raises whatever an
enabled provider raises. Open-Meteo returns 503 while it rotates its model runs
and 429 when the free tier is briefly saturated; both clear within seconds.
Retryable responses (429, 500, 502, 503, 504) and connection errors are now
retried three times with a growing pause, matching what the SMARD provider
already does. If the fetch still fails and the stored forecast reaches past the
run start, that forecast is kept for one more run rather than failing the update
- a forecast one run old beats no forecast at all. A cold start with nothing
stored still fails, because then there really is no PV forecast.
Every morning before the day-ahead auction is published, /v1/prediction/update
answered 400 and no prediction was produced at all.
The provider asks for prices starting at the run day, because an existing
history sets past_days to 0. The optimization horizon always reaches past the
last published price, so an update is always considered necessary - and SMARD
publishes the next day around midday. Between midnight and publication the
requested window therefore contains nothing, and ElecPriceSMARD raised
"SMARD response contains no usable day-ahead prices", which failed the whole
prediction update rather than only that provider.
ElecPriceEnergyCharts and its SMARD subclass now keep their existing history and
let the ETS/median branch extrapolate the remaining slots, the same fallback
FeedInTariffEnergyCharts already had. A cold start without any history stays
fatal. ElecPriceSMARD also separates the two cases it used to conflate: a period
the source has not published yet now reports the latest value it does have, and
only a response without a single price still reads as unusable.
Fixes the cache noise this produced as well. cache_in_file claimed its cache
entry before calling the wrapped function, so a raising function left an empty
file behind and every later call within the TTL logged "Read failed: Ran out of
input" before refetching. The entry is now created only after the call returns.
On a converged population the boost was permanently on, so there was nothing
left for it to intervene in. Its trigger, DIVERSITY_BOOST_THRESHOLD at 0.35, sat
above SELECTION_DIVERSITY_FLOOR at 0.30 - the floor the selection itself
guarantees - so `diversity < threshold` was true in every generation after
convergence. The threshold now sits below the floor.
The immigrants the boost injects are by construction the worst individuals in
the pool, and `_select_diverse` ran a plain tournament over parents and
offspring together, so they were removed in the very generation that created
them and their genes never recombined. A bounded share of seats
(IMMIGRANT_PROTECTION_FRACTION) is now reserved for them for
IMMIGRANT_PROTECTION_GENERATIONS selections. The incumbent is protected by
genome key, so no immigrant can evict the best solution or an equal-genome twin,
and offspring do not inherit the protection.
The log line was edge-triggered on `diversity_boost_active`, which was also
cleared on every fitness improvement, so a running boost re-announced itself
with "stagnation 0" while a boost that never stopped looked like several short
ones. It now tracks the boost alone, and the end of a boost is logged too.
Measured on a converged population: without protection 0 of 12 immigrants
survive the selection, with it all 12 do, and the incumbent is kept either way.
A battery or inverter failure limits PV to local demand for days. The
calibration read that as the plant's true output and learned it as a permanent
model loss, so one outage degraded the forecast long after the hardware was
fixed.
Calibration now estimates the healthy plant ratio over
`calibration_reference_days`, excludes days below `calibration_outage_threshold`
of it, and falls back to the most recent `calibration_min_healthy_days` when the
normal window is contaminated. `calibration_outage_filter_enabled` turns this
off for plants where measured curtailment, not available potential, is the
prediction target.
The fit also uses native 15-minute meter readings when every configured PV meter
supplies them - never interpolating hourly counters into an invented
quarter-hour profile - interpolates azimuth factors smoothly between bin centres
instead of stepping the EMS input curve, and normalizes the shape per forecast
day so it redistributes energy without changing that day's kWh correction. The
default azimuth bin widens from 15 to 45 degrees, which is what a typical
window actually supports.
Fixes the calibration window itself: it was derived from the measurement store
as a whole rather than from the configured PV production meters. A load meter
reaching further than the PV meter placed the window where no PV reading exists,
so calibration silently fell back to hourly fitting or skipped itself.
Also fixes `Measurement.load()`, which discarded every stored record. It
validated the file into a "temporary" Measurement, but Measurement is a
singleton, so that instance was the live one and the parsed records were
dropped.
The optimizer treated the end of `optimization.horizon_hours` as the end of the
world: energy left in the battery there was worth a single configured price per
kWh, so it either dumped the battery into the last hours or hoarded it,
depending on that one number.
The horizon is now two spans. `horizon_hours` still receives every control
command. The new `optimization.tail_horizon_hours` (default 48 h) is a pure
lookahead that never produces a command. In AUTO terminal-value mode a
deterministic dynamic program solves that tail backwards on a 101-point SoC
grid using the production battery and inverter models - SoC bounds, power caps,
conversion losses, configured charge and export rates, direct-marketing
permission and LCOS on delivered DC energy - and the existing AUTO proxy
supplies the continuation value at the tail end. Genetic fitness reads the
resulting curve. `tail_horizon_hours: 0` restores the plain proxy at the control
end, FIXED is unchanged.
The forecast budget is reported, never enforced by refusal: a tail that does not
fit is shortened to what the forecast covers and reported as
`effective_tail_hours`, and a control horizon that does not fit is warned about
at configuration time and rejected by the optimizer at run time, which knows
which series ran out. `prediction.hours` defaults to 72 so the new defaults fit
out of the box; existing shorter configurations keep starting.
Control arrays and warm-start genomes now begin at the run timestamp rather than
midnight, flagged by `controls_start_at_now` so the adapters still read older
solutions. `forecast_interval_seconds` declares the resolution of shortened
native quarter-hour inputs.
Required forecasts are no longer silently replaced by demo providers. A missing
PV, price, load, feed-in or weather forecast used to rewrite the configured
provider and retry, so a run could quietly optimize against invented data.
Missing values now stay missing, and provider values are held only within their
own source interval instead of being extended indefinitely.
Also fixes a config update that could leave EOS half-updated: the merged
candidate is validated before the singleton is reinitialized.
Four provider tests that hard-coded the old 48 h prediction default are rewritten
to derive their expectations from the configured horizon.
Add PVForecastAkkudoktorLocal, which runs the modelling chain inside EOS on
raw Open-Meteo irradiance instead of calling a forecast service: solar
position, horizon shading, plane transposition, incidence-angle modifier,
cell temperature, PVWatts DC and inverter AC.
It needs no API key and serves up to 16 days at 15-minute resolution from a
single hourly request, which is what keeps `optimization.tail_horizon_hours`
fed - services wrapping Open-Meteo cut the horizon much shorter. Several
Open-Meteo models can be listed in `weather_models` and are averaged per
variable at no extra request cost.
With `calibration_enabled` the provider fits itself against
`measurement.pv_production_emr_keys` over the past `calibration_days`: a
global scale factor plus optional per-solar-azimuth factors, each weighted by
modelled energy, shrunk toward the global factor by `calibration_prior_kwh`
and clamped to `[calibration_min_factor, calibration_max_factor]`. The
comparison runs on past intervals, where Open-Meteo serves analysed rather
than forecast weather, so it corrects the error of the PV model and not that
of the weather forecast.
Calibration is a scale factor on the output and never touches `userhorizon`,
`surface_tilt`, `surface_azimuth` or `peakpower`. The docs say so, and say
why a short window and a plant fault inside it are the two ways to end up
with a misleading factor.
Also add `Measurement.pv_production_total_kwh()` alongside the existing load
total, and `scripts/pvforecast_backtest.py`, which scores configuration
variants against the stored meter readings without waiting for new forecasts
to come true.
Everything up to the knee is backed by residual load, everything beyond
it only by an export at the median feed-in tariff - the two halves have
very different confidence. The breakpoint list alone does not say where
the split is, so consumers had to guess it from the segment lengths.
A run whose price forecast is all zeros produces no priced residual load,
so AUTO cannot derive a curve and quietly credits the request scalar
instead. The reported mode was then "FIXED" - indistinguishable from a
run actually configured that way.
The solution now carries a reason, and the fallback is logged as a
warning instead of passing unnoticed.
The energy still stored when the horizon ends keeps its worth: it replaces
grid imports that are paid for afterwards. Crediting that with a single
price per kWh cannot describe it, because the value is not linear in the
amount stored. The first kWh replaces the most expensive hour that PV
cannot cover, the next one the second most expensive, and once every such
hour is served, further energy replaces nothing.
A scalar has to pick one slope for all of it. High enough for the first kWh
means hoarding a full battery; low enough for the last kWh means running it
empty by the end of the horizon - which is exactly what the previous default
of 0 EUR/kWh did.
terminal_value_mode = AUTO (the new default) builds the curve instead. There
is no forecast beyond the horizon, so its trailing window stands in for the
day that follows: residual load max(load - PV, 0) per slot, priced at its
import price, sorted and accumulated. LCOS is subtracted from every marginal
value so stored energy is not credited twice, and the tail beyond the
residual load is only credited when direct marketing allows an export. The
curve is built once per run; the search only interpolates on it.
The solution reports what a run used as terminal_value, curve included, so
the shape can be inspected instead of guessed. FIXED restores the previous
scalar behaviour.
In a 48 h scenario with two cheap slots at the end, AUTO keeps the battery
at 50 % and credits 3.85 EUR where FIXED with 0 EUR/kWh drains it to empty.
The stored optimization results move accordingly - the objective changed.
With grid charging switched off (inverter.max_ac_charge_power_w = 0) the
simulation zeroes the AC charge array before using it, but it did so by
rebinding a local name. The solution is read back from the original array,
so it still carried the optimizer's AC charge genes - genes the fitness
never evaluated, because the simulation ignored them, and which are
therefore arbitrary.
The effect was visible as a slot with ac_charge = 0.8 where the battery SoC
does not move. Harmless inside EOS, but a controller that follows the plan
would grid-charge the battery at a time nobody planned or paid for.
Zero the array in place so the reported plan matches what was simulated.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Three related scheduling improvements, all opt-in and behaviour-preserving
when the new fields are not set.
Flexible consumers get absolute time bounds next to the recurring
time_windows: earliest_start_datetime and deadline_datetime, where the
deadline requires the complete run to have *finished* before that moment
("clean dishes by 03:00 tonight"). When no start can meet it,
deadline_policy decides between BEST_EFFORT (run as early as possible, so
the delay rather than the cost is minimized) and STRICT (keep the
deadline; a ONCE consumer then fails the optimization). The solution
reports appliance_deadline_missed per device.
The EV charging target can be given the same kind of deadline, as an
absolute min_soc_deadline_datetime and/or a relative min_soc_max_duration_h
("full in 6 hours"), the earlier of the two winning. The ev_soc_miss
penalty is then evaluated at that slot instead of at the end of the
horizon, and the seeding heuristic only proposes charge slots before it.
Battery-to-grid export under direct marketing is no longer all-or-nothing:
grid_export_rates configures the selectable export levels as a factor of
the rated discharge power (default [0.25, 0.5, 0.75, 1.0]). Each rate is
its own optimizer state, with the full-power state keeping its previous
index so existing seeds and heuristics are unaffected. The chosen level
per slot is reported in battery_grid_export_factor and as the
GRID_SUPPORT_EXPORT operation factor.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
A plane's userhorizon uses the same convention as the Forecast.Solar
horizon query parameter (evenly distributed heights in degrees, starting
north, clockwise), so pass it through instead of dropping it. Without it
a shaded plane is forecast as if it had a free horizon.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Replace the single hourly "dishwasher" home appliance with a list of
flexible consumers (home_appliances). Each consumer defines its load
either as an explicit power profile (energy-preservingly resampled onto
the optimization slot grid, incl. 15-min and non-integer interval ratios)
or the flat consumption_wh/duration_h fallback, and runs ONCE or DAILY
within its time windows and the optimization horizon.
- ConsumerScheduleMode + shared load-definition validation (XOR of
profile/fallback, reject negative/NaN/inf, unique device_id)
- ApplianceGeneLayout: variable appliance gene block (index into
allowed_start_slots), ONCE/DAILY calendar-day based, no snapping
- per-device output: result.home_appliance_energy_wh, appliance_starts
(absolute local times), per-device solution columns and DDBC RUN/OFF
instructions on state transitions only
- deprecate dishwasher/washingstart/Home_appliance_wh_per_hour with
backward-compatible mapping and explicit conflict rejection
- max_home_appliances is now an upper bound only; no demo appliance and
no on/off behaviour
- docs, openapi.json, CHANGELOG and optimize_result_2* fixtures updated;
new tests/test_homeappliance.py covers the mandatory test matrix
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Request QUARTER_HOURLY exchange prices from Tibber and store them at their
native resolution instead of pre-averaging to hourly values. EOS resamples the
stored records onto the optimization grid on demand (key_to_array), so keeping
the native step size lets both the hourly (interval=3600) and the 15-minute
(interval=900) optimizer be fed the correct grid automatically.
- GraphQL: priceInfoRange resolution HOURLY -> QUARTER_HOURLY (last 960).
- _hourly_series -> _normalize_series: dedupe by timestamp (mean) + sort, no
1h aggregation; add _resolution_seconds (median of timestamp diffs, fallback
3600s).
- Resolution-agnostic ETS extrapolation: seasonal windows and history
thresholds are scaled by slots_per_hour, needed forecast length and the
prediction index step are computed in slots. Hourly behaviour is unchanged
(slots_per_hour=1 -> 168/24 seasonal periods, hourly steps).
- Tests: replace the 1h-averaging test with resolution-preserving + dedup
tests, assert QUARTER_HOURLY in the query, add a 15-min end-to-end test
(native storage stays 15min, slot-based seasonal periods = 96, 15-min
forecast index). Hourly backward-compat tests stay green unchanged.
The genetic optimizer was hard-wired to an hourly grid and forced
optimization.interval to 3600 s. Generalize it to a configurable slot grid
of length prediction.hours * (3600 / interval), accepting 900 (15 min) in
addition to the default 3600 (1 hour) so the optimizer can schedule on a
quarter-hour grid for 15-minute dynamic electricity tariffs.
- genetic.py: slot_duration_h / slots_per_hour / total_slots helpers; all GA
vectors sized by total_slots; simulate()/evaluate() indexed by start slot.
- geneticparams.py: allow {900, 3600}; scale the load power series to per-slot
energy, mirroring the PV series.
- battery.py / inverter.py: scale power caps to per-slot energy caps via
slot_duration_h; homeappliance.py carries the hook.
- geneticsolution.py: serialize solution and plan on the slot grid (interval
freq, start-slot offset, second-based instruction instants).
The default 3600 s interval keeps the previous hourly behaviour; the genetic
regression suite is unchanged. Adds tests for the 15-minute slot grid.
Fügt einen Direktvermarktungs-Modus (feedintariff.direct_marketing_enabled)
hinzu, der den Börsenpreis als Einspeisevergütung nutzt und aktive
Batterie-Entladung ins Netz (battery_grid_export_allowed) sowie
DC-Charge-Bypass optimiert.
- FeedInTariffEnergyCharts-Provider (Börsen-Einspeisetarif inkl. Prognose)
- Inverter: DC/AC-Wirkungsgrade und Batterie-Grid-Export in process_energy
- Genetik: Export-/DC-Charge-Zustände, Restwert-Bewertung des Akkus
- Solution-Result: neues Feld Feed_in_tariff (verwendeter Tarif je Stunde)
- Tests für neue Provider, Solution und Simulation
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- Use urllib.parse.quote instead of requests.utils.quote in the pvnode and
Solcast providers: the runtime re-export exists, but the requests type stubs
do not declare it, so the pre-commit mypy hook failed with
'Module has no attribute "quote"'.
- Mark the force_update keyword in the new provider tests with `# type: ignore`
— it is consumed by the cache_in_file decorator at runtime; same call
convention and ignore style as pvforecastakkudoktor.py.
- Add PVForecastPVNode, PVForecastForecastSolar and PVForecastSolcast to the
expected provider sequence in tests/test_prediction.py (fixture + index
assertions) — the two sequence tests failed because the new providers were
registered in prediction.py but missing from the hardcoded expectations.
Add PVForecastSolcast for the Solcast rooftop-site API. The operator registers a
site in the Solcast web app and enters the API key + resource (site) id:
GET /rooftop_sites/{site_id}/forecasts. pv_estimate (kW) is converted to watts
and fed as pvforecast_ac_power; the timestamp is normalised to the period start
(period_end - period) so it aligns with the resample axis. period_end is UTC.
Completes the set of selectable cloud PV forecast providers
(Akkudoktor, VRM, Import, pvnode, Forecast.Solar, Solcast). Adds tests for the
kW->W conversion, period-start normalisation, ISO-8601 period parsing, the
request URL/auth and HTTP-error handling.
Add PVForecastForecastSolar, a PV forecast provider for the free Forecast.Solar
API (https://forecast.solar), giving operators a no-account forecast source in
addition to Akkudoktor, VRM, Import and pvnode. An optional API key raises the
rate limit.
result.watts is the instantaneous AC power per timestamp, fed directly as
pvforecast_ac_power. Plants with several roof planes issue one request per plane
(Forecast.Solar is single-plane) and the powers are summed per timestamp.
Forecast.Solar azimuth (-180=N..0=S..90=W) is converted from EOS surface_azimuth
(north=0..south=180); local wall-clock timestamps are resolved via the response
timezone before resampling.
Registered in pvforecast.py and prediction.py. Adds tests for timezone
resolution, azimuth conversion, multi-plane summation and HTTP-error handling.
Add PVForecastPVNode, a native 15-minute PV forecast provider for the
pvnode.com V2 API, giving operators another forecast source to choose from
alongside Akkudoktor, VRM and Import.
Two request modes, selected by configuration:
* site_id set -> GET /v2/forecast/{site_id} (a saved, possibly calibrated
site managed on pvnode.com — the operator enters site id + API key)
* site_id empty -> POST /v2/forecast/inline (geometry sent inline from the
configured pvforecast.planes; no web-app setup required)
V2 response timestamps are site-local wall-clock accompanied by an IANA
timezone; they are resolved to absolute instants before EOS resamples them.
Nullable pv_power (e.g. at night) is treated as 0 W so the optimizer's linear
resampling does not interpolate phantom production across the night.
Registered in pvforecast.py (provider settings + id list) and prediction.py
(singleton, factory list, container union). Adds tests covering timezone
resolution, null handling, both request modes and HTTP-error propagation.
Make openmeteo test robust against timing issues.
Also update:
- types-docutils==0.22.3.20260322
- pytest-cov==7.1.0
- ruff-pre-commit v0.15.7
- uvlock
Signed-off-by: Bobby Noelte <b0661n0e17e@gmail.com>
* record battery SOC at the start of the interval for accurate display
* map battery charge rates to GeneticOptimizationParameters and update related logic
Instead of using the EV charge rates, the battery now has its own charge rate defined
in the GeneticOptimizationParameters to better separate those entities.
* separate raw gene values from SOC-clamped op factors
The genetic_*_factor columns in the OptimizationSolution dataframe now
always carry the raw gene values (optimizer intent), while the
battery1_*_op_mode / battery1_*_op_factor columns and FRBCInstruction
operation_mode_factor reflect SOC-clamped effective values that can
actually be executed given the battery's state of charge at each hour.
Adds GeneticSolution._soc_clamped_operation_factors():
- AC charge factor: proportionally scaled down when battery headroom
(max_soc - current_soc) is less than what the commanded factor
would store in one hour; zeroed when battery is full.
- DC charge factor: zeroed when battery is at or above max SOC.
- Discharge: blocked when SOC is at or below min SOC.
Both optimization_solution() and energy_management_plan() pass the
clamped values to _battery_operation_from_solution(), so the plan
instructions the HEMS receives reflect physically achievable targets.
* ensure max AC charge power is only used if defined
* update fixtures for PV suffix + SOC-clamp algorithm changes
* handle None case for home appliance start hour in simulation
if the hour is zero, the the home appliance wont start
* update handling of invalid charge indices in autocharge hours
Seems to be a bug, since all invalid indexes needs to be invalidated (also the first one)
* remove double code in simulation preparation and improve comments
* update version
Signed-off-by: Bobby Noelte <b0661n0e17e@gmail.com>
Co-authored-by: Christopher Nadler <christopher.nadler@gmail.com>
Ensure that the bidding zone in the request is correctly set to a
string value (not an enum).
This seems to be also an issue with python version < 3.11. Add safeguards
to only use python >= 3.11. Still keep a regression test for the enum
conversion to string.
Signed-off-by: Bobby Noelte <b0661n0e17e@gmail.com>
Add a fixed electricity prediction that supports prices per time window.
The time windows may flexible be defined by day or date.
The prediction documentation is updated to also cover the ElecPriceFixed
provider.
The feature includes several changes that are not directly related to the
electricity price prediction implementation but are necessary to keep
EOS running properly and to test and document the changes.
* feat: add value time windows
Add time windows with an associated float value.
* feat: harden eos measurements endpoints error detection and reporting
Cover more errors that may be raised during endpoint access. Report the
errors including trace information to ease debugging.
* feat: extend server configuration to cover all arguments
Make the argument controlled options also available in server configuration.
* fix: eos config configuration by cli arguments
Move the command line argument handling to config eos so that it is
excuted whenever eos config is rebuild or reset.
* chore: extend measurement endpoint system test
* chore: refactor time windows
Move time windows to configabc as they are only used in configurations.
Also move all tests to test_configabc.
* chore: provide config update errors in eosdash with summarized error text
If there is an update error provide the error text as a summary. On click
provide the full error text.
* chore: force eosdash ip address and port in makefile dev run
Ensure eosdash ip address and port are correctly set for development runs.
Signed-off-by: Bobby Noelte <b0661n0e17e@gmail.com>
Validate solution prediction data before processing.
If required prediction data is missing, the prediction is skipped
instead of raising an exception.
Introduce a new configuration file saving policy to improve loading robustness:
- Exclude computed fields
- Exclude fields set to their default values
- Exclude fields with value None
- Use field aliases
- Recursively remove empty dictionaries and lists
- Ensure general.version is always present and correctly set
When loading older configuration files, computed fields are now stripped
before migration. This further improves backward compatibility and loading
robustness.
Signed-off-by: Bobby Noelte <b0661n0e17e@gmail.com>
Adapt uvicorn log level to allowed levels.
Ensure that EOSdash is started after EOS configuration is available.
Signed-off-by: Bobby Noelte <b0661n0e17e@gmail.com>
* fix: improve error handling for provider updates
Distinguishes failures of active providers from inactive ones.
Propagates errors only for enabled providers, allowing execution
to continue if a non-active provider fails, which avoids unnecessary
interruptions and improves robustness.
* fix: add provider settings validation for forecast requests
Prevents potential runtime errors by checking if provider settings are configured
before accessing forecast credentials.
Raises a clear error when settings are missing to help with debugging misconfigurations.
* refactor(load): move provider settings to top-level fields
Transitions load provider settings from a nested "provider_settings" object with provider-specific keys to dedicated top-level fields.\n\nRemoves the legacy "provider_settings" mapping and updates migration logic to ensure backward compatibility with existing configurations.
* docs: update version numbers and documantation
---------
Co-authored-by: Normann <github@koldrack.com>
Startup retention manager for asynchronous tasks. Handle gracefully
exceptions in these tasks or the configuration for them.
Remove tasks.py as repeated tasks are now handled by the retention
manager.
When running on GitHub, only the version date file is checked. The
development tag is merely a label, so any date set during development suffices.
The test_doc is also skipped on GitHub actions.
Ensure that the snapping sequence generated in the test fixture
is within the boundaries expected by the test.
Ensure we read the _version_date.py info as UTC datetime and do
no localtime conversion.
Prevent and guard test_version.py to modify the version date file.
Signed-off-by: Bobby Noelte <b0661n0e17e@gmail.com>
Upgrade to:
- pandas==3.01
- fastapi[standard-no-fastapi-cloud-cli]==0.132.0
- fastapi_cli==0.0.23
- MonsterUI==1.0.44
- uvicorn==0.41.0
Close database in database fixture on teardown.
Fix file exclusion in hash and version date generation.
Update version information in documentation.
Signed-off-by: Bobby Noelte <b0661n0e17e@gmail.com>
Home Assistant expects versioning always increases numbers. Add
a date component to the development version to comply with this
expectation. The scheme is now 0.0.0.dev<date><hash>.
Use uv for creating and managing the virtual environment for developement.
This enourmously speeds up dependency updates. For this change
dependency requirements are now solely handled in pyproject.toml.
requirements.tx and requirements-dev.txt are deleted.
Signed-off-by: Bobby Noelte <b0661n0e17e@gmail.com>
The database supports backend selection, compression, incremental data load,
automatic data saving to storage, automatic vaccum and compaction.
Make SQLite3 and LMDB database backends available.
Update tests for new interface conventions regarding data sequences,
data containers, data providers. This includes the measurements provider and
the prediction providers.
Add database documentation.
The fix includes several bug fixes that are not directly related to the database
implementation but are necessary to keep EOS running properly and to test and
document the changes.
* fix: config eos test setup
Make the config_eos fixture generate a new instance of the config_eos singleton.
Use correct env names to setup data folder path.
* fix: startup with no config
Make cache and measurements complain about missing data path configuration but
do not bail out.
* fix: soc data preparation and usage for genetic optimization.
Search for soc measurments 48 hours around the optimization start time.
Only clamp soc to maximum in battery device simulation.
* fix: dashboard bailout on zero value solution display
Do not use zero values to calculate the chart values adjustment for display.
* fix: openapi generation script
Make the script also replace data_folder_path and data_output_path to hide
real (test) environment pathes.
* feat: add make repeated task function
make_repeated_task allows to wrap a function to be repeated cyclically.
* chore: removed index based data sequence access
Index based data sequence access does not make sense as the sequence can be backed
by the database. The sequence is now purely time series data.
* chore: refactor eos startup to avoid module import startup
Avoid module import initialisation expecially of the EOS configuration.
Config mutation, singleton initialization, logging setup, argparse parsing,
background task definitions depending on config and environment-dependent behavior
is now done at function startup.
* chore: introduce retention manager
A single long-running background task that owns the scheduling of all periodic
server-maintenance jobs (cache cleanup, DB autosave, …)
* chore: canonicalize timezone name for UTC
Timezone names that are semantically identical to UTC are canonicalized to UTC.
* chore: extend config file migration for default value handling
Extend the config file migration handling values None or nonexisting values
that will invoke a default value generation in the new config file. Also
adapt test to handle this situation.
* chore: extend datetime util test cases
* chore: make version test check for untracked files
Check for files that are not tracked by git. Version calculation will be
wrong if these files will not be commited.
* chore: bump pandas to 3.0.0
Pandas 3.0 now performs inference on the appropriate resolution (a.k.a. unit)
for the output dtype which may become datetime64[us] (before it was ns). Also
numeric dtype detection is now more strict which needs a different detection for
numerics.
* chore: bump pydantic-settings to 2.12.0
pydantic-settings 2.12.0 under pytest creates a different behaviour. The tests
were adapted and a workaround was introduced. Also ConfigEOS was adapted
to allow for fine grain initialization control to be able to switch
off certain settings such as file settings during test.
* chore: remove sci learn kit from dependencies
The sci learn kit is not strictly necessary as long as we have scipy.
* chore: add documentation mode guarding for sphinx autosummary
Sphinx autosummary excecutes functions. Prevent exceptions in case of pure doc
mode.
* chore: adapt docker-build CI workflow to stricter GitHub handling
Signed-off-by: Bobby Noelte <b0661n0e17e@gmail.com>
Andreas
ChristinandAndreas
Andreas
Bobby NoelteandGitHub
Christopher NadlerandGitHub
