mirror of https://github.com/Akkudoktor-EOS/EOS.git synced 2026-02-24 18:06:19 +00:00

Files

Bobby Noelte 6498c7dc32 Add database support for measurements and historic prediction data. (#848 )

The database supports backend selection, compression, incremental data load,
automatic data saving to storage, automatic vaccum and compaction.

Make SQLite3 and LMDB database backends available.

Update tests for new interface conventions regarding data sequences,
data containers, data providers. This includes the measurements provider and
the prediction providers.

Add database documentation.

The fix includes several bug fixes that are not directly related to the database
implementation but are necessary to keep EOS running properly and to test and
document the changes.

* fix: config eos test setup

  Make the config_eos fixture generate a new instance of the config_eos singleton.
  Use correct env names to setup data folder path.

* fix: startup with no config

  Make cache and measurements complain about missing data path configuration but
  do not bail out.

* fix: soc data preparation and usage for genetic optimization.

  Search for soc measurments 48 hours around the optimization start time.
  Only clamp soc to maximum in battery device simulation.

* fix: dashboard bailout on zero value solution display

  Do not use zero values to calculate the chart values adjustment for display.

* fix: openapi generation script

  Make the script also replace data_folder_path and data_output_path to hide
  real (test) environment pathes.

* feat: add make repeated task function

  make_repeated_task allows to wrap a function to be repeated cyclically.

* chore: removed index based data sequence access

  Index based data sequence access does not make sense as the sequence can be backed
  by the database. The sequence is now purely time series data.

* chore: refactor eos startup to avoid module import startup

  Avoid module import initialisation expecially of the EOS configuration.
  Config mutation, singleton initialization, logging setup, argparse parsing,
  background task definitions depending on config and environment-dependent behavior
  is now done at function startup.

* chore: introduce retention manager

  A single long-running background task that owns the scheduling of all periodic
  server-maintenance jobs (cache cleanup, DB autosave, …)

* chore: canonicalize timezone name for UTC

  Timezone names that are semantically identical to UTC are canonicalized to UTC.

* chore: extend config file migration for default value handling

  Extend the config file migration handling values None or nonexisting values
  that will invoke a default value generation in the new config file. Also
  adapt test to handle this situation.

* chore: extend datetime util test cases

* chore: make version test check for untracked files

  Check for files that are not tracked by git. Version calculation will be
  wrong if these files will not be commited.

* chore: bump pandas to 3.0.0

  Pandas 3.0 now performs inference on the appropriate resolution (a.k.a. unit)
  for the output dtype which may become datetime64[us] (before it was ns). Also
  numeric dtype detection is now more strict which needs a different detection for
  numerics.

* chore: bump pydantic-settings to 2.12.0

  pydantic-settings 2.12.0 under pytest creates a different behaviour. The tests
  were adapted and a workaround was introduced. Also ConfigEOS was adapted
  to allow for fine grain initialization control to be able to switch
  off certain settings such as file settings during test.

* chore: remove sci learn kit from dependencies

  The sci learn kit is not strictly necessary as long as we have scipy.

* chore: add documentation mode guarding for sphinx autosummary

  Sphinx autosummary excecutes functions. Prevent exceptions in case of pure doc
  mode.

* chore: adapt docker-build CI workflow to stricter GitHub handling

Signed-off-by: Bobby Noelte <b0661n0e17e@gmail.com>

2026-02-22 14:12:42 +01:00

38 KiB

Raw Blame History

Akkudoktor-EOS

Version: v0.2.0.dev58204789

Description: This project provides a comprehensive solution for simulating and optimizing an energy system based on renewable energy sources. With a focus on photovoltaic (PV) systems, battery storage (batteries), load management (consumer requirements), heat pumps, electric vehicles, and consideration of electricity price data, this system enables forecasting and optimization of energy flow and costs over a specified period.

Base URL: No base URL provided.

Endpoints:

POST /gesamtlast

Links: local, eos

Fastapi Gesamtlast

"""
Deprecated: Total Load Prediction with adjustment.

Endpoint to handle total load prediction adjusted by latest measured data.

Total load prediction starts at 00.00.00 today and is provided for 48 hours.
If no prediction values are available the missing ones at the start of the series are
filled with the first available prediction value.

Note:
    Use '/v1/prediction/list?key=loadforecast_power_w' instead.
    Load energy meter readings to be added to EOS measurement by:
    '/v1/measurement/value' or
    '/v1/measurement/series' or
    '/v1/measurement/dataframe' or
    '/v1/measurement/data'
"""

Request Body:

application/json: { "$ref": "#/components/schemas/GesamtlastRequest" }

Responses:

200: Successful Response
422: Validation Error

GET /gesamtlast_simple

Links: local, eos

Fastapi Gesamtlast Simple

"""
Deprecated: Total Load Prediction.

Endpoint to handle total load prediction.

Total load prediction starts at 00.00.00 today and is provided for 48 hours.
If no prediction values are available the missing ones at the start of the series are
filled with the first available prediction value.

Args:
    year_energy (float): Yearly energy consumption in Wh.

Note:
    Set LoadAkkudoktor as provider, then update data with
    '/v1/prediction/update'
    and then request data with
    '/v1/prediction/list?key=loadforecast_power_w' instead.
"""

Parameters:

year_energy (query, required): No description provided.

Responses:

200: Successful Response
422: Validation Error

POST /optimize

Links: local, eos

Fastapi Optimize

"""
Deprecated: Optimize.

Endpoint to handle optimization.

Note:
    Use automatic optimization instead.
"""

Parameters:

start_hour (query, optional): Defaults to current hour of the day.
ngen (query, optional): Number of indivuals to generate for genetic algorithm.

Request Body:

application/json: { "$ref": "#/components/schemas/GeneticOptimizationParameters" }

Responses:

200: Successful Response
422: Validation Error

GET /pvforecast

Links: local, eos

Fastapi Pvforecast

"""
Deprecated: PV Forecast Prediction.

Endpoint to handle PV forecast prediction.

PVForecast starts at 00.00.00 today and is provided for 48 hours.
If no forecast values are available the missing ones at the start of the series are
filled with the first available forecast value.

Note:
    Set PVForecastAkkudoktor as provider, then update data with
    '/v1/prediction/update'
    and then request data with
    '/v1/prediction/list?key=pvforecast_ac_power' and
    '/v1/prediction/list?key=pvforecastakkudoktor_temp_air' instead.
"""

Responses:

200: Successful Response

GET /strompreis

Links: local, eos

Fastapi Strompreis

"""
Deprecated: Electricity Market Price Prediction per Wh (€/Wh).

Electricity prices start at 00.00.00 today and are provided for 48 hours.
If no prices are available the missing ones at the start of the series are
filled with the first available price.

Note:
    Electricity price charges are added.

Note:
    Set ElecPriceAkkudoktor as provider, then update data with
    '/v1/prediction/update'
    and then request data with
    '/v1/prediction/list?key=elecprice_marketprice_wh' or
    '/v1/prediction/list?key=elecprice_marketprice_kwh' instead.
"""

Responses:

200: Successful Response

GET /v1/admin/cache

Links: local, eos

Fastapi Admin Cache Get

"""
Current cache management data.

Returns:
    data (dict): The management data.
"""

Responses:

200: Successful Response

POST /v1/admin/cache/clear

Links: local, eos

Fastapi Admin Cache Clear Post

"""
Clear the cache.

Deletes all cache files.

Returns:
    data (dict): The management data after cleanup.
"""

Responses:

200: Successful Response

POST /v1/admin/cache/clear-expired

Links: local, eos

Fastapi Admin Cache Clear Expired Post

"""
Clear the cache from expired data.

Deletes expired cache files.

Returns:
    data (dict): The management data after cleanup.
"""

Responses:

200: Successful Response

POST /v1/admin/cache/load

Links: local, eos

Fastapi Admin Cache Load Post

"""
Load cache management data.

Returns:
    data (dict): The management data that was loaded.
"""

Responses:

200: Successful Response

POST /v1/admin/cache/save

Links: local, eos

Fastapi Admin Cache Save Post

"""
Save the current cache management data.

Returns:
    data (dict): The management data that was saved.
"""

Responses:

200: Successful Response

GET /v1/admin/database/stats

Links: local, eos

Fastapi Admin Database Stats Get

"""
Get statistics from database.

Returns:
    data (dict): The database statistics
"""

Responses:

200: Successful Response

POST /v1/admin/database/vacuum

Links: local, eos

Fastapi Admin Database Vacuum Post

"""
Remove old records from database.

Returns:
    data (dict): The database stats after removal of old records.
"""

Responses:

200: Successful Response

POST /v1/admin/server/restart

Links: local, eos

Fastapi Admin Server Restart Post

"""
Restart the server.

Restart EOS properly by starting a new instance before exiting the old one.
"""

Responses:

200: Successful Response

POST /v1/admin/server/shutdown

Links: local, eos

Fastapi Admin Server Shutdown Post

"""
Shutdown the server.
"""

Responses:

200: Successful Response

GET /v1/config

Links: local, eos

Fastapi Config Get

"""
Get the current configuration.

Returns:
    configuration (ConfigEOS): The current configuration.
"""

Responses:

200: Successful Response

PUT /v1/config

Links: local, eos

Fastapi Config Put

"""
Update the current config with the provided settings.

Note that for any setting value that is None or unset, the configuration will fall back to
values from other sources such as environment variables, the EOS configuration file, or default
values.

Args:
    settings (SettingsEOS): The settings to write into the current settings.

Returns:
    configuration (ConfigEOS): The current configuration after the write.
"""

Request Body:

application/json: { "$ref": "#/components/schemas/SettingsEOS" }

Responses:

200: Successful Response
422: Validation Error

GET /v1/config/backup

Links: local, eos

Fastapi Config Backup Get

"""
Get the EOS configuration backup identifiers and backup metadata.

Returns:
    dict[str, dict[str, Any]]: Mapping of backup identifiers to metadata.
"""

Responses:

200: Successful Response

PUT /v1/config/file

Links: local, eos

Fastapi Config File Put

"""
Save the current configuration to the EOS configuration file.

Returns:
    configuration (ConfigEOS): The current configuration that was saved.
"""

Responses:

200: Successful Response

POST /v1/config/reset

Links: local, eos

Fastapi Config Reset Post

"""
Reset the configuration to the EOS configuration file.

Returns:
    configuration (ConfigEOS): The current configuration after update.
"""

Responses:

200: Successful Response

PUT /v1/config/revert

Links: local, eos

Fastapi Config Revert Put

"""
Revert the configuration to a EOS configuration backup.

Returns:
    configuration (ConfigEOS): The current configuration after revert.
"""

Parameters:

backup_id (query, required): EOS configuration backup ID.

Responses:

200: Successful Response
422: Validation Error

GET /v1/config/{path}

Links: local, eos

Fastapi Config Get Key

"""
Get the value of a nested key or index in the config model.

Args:
    path (str): The nested path to the key (e.g., "general/latitude" or "optimize/nested_list/0").

Returns:
    value (Any): The value of the selected nested key.
"""

Parameters:

path (path, required): The nested path to the configuration key (e.g., general/latitude).

Responses:

200: Successful Response
422: Validation Error

PUT /v1/config/{path}

Links: local, eos

Fastapi Config Put Key

"""
Update a nested key or index in the config model.

Args:
    path (str): The nested path to the key (e.g., "general/latitude" or "optimize/nested_list/0").
    value (Any): The new value to assign to the key or index at path.

Returns:
    configuration (ConfigEOS): The current configuration after the update.
"""

Parameters:

path (path, required): The nested path to the configuration key (e.g., general/latitude).

Request Body:

application/json: { "anyOf": [ {}, { "type": "null" } ], "description": "The value to assign to the specified configuration path (can be None).", "title": "Value" }

Responses:

200: Successful Response
422: Validation Error

GET /v1/energy-management/optimization/solution

Links: local, eos

Fastapi Energy Management Optimization Solution Get

"""
Get the latest solution of the optimization.
"""

Responses:

200: Successful Response

GET /v1/energy-management/plan

Links: local, eos

Fastapi Energy Management Plan Get

"""
Get the latest energy management plan.
"""

Responses:

200: Successful Response

GET /v1/health

Links: local, eos

Fastapi Health Get

"""
Health check endpoint to verify that the EOS server is alive.
"""

Responses:

200: Successful Response

GET /v1/logging/log

Links: local, eos

Fastapi Logging Get Log

"""
Get structured log entries from the EOS log file.

Filters and returns log entries based on the specified query parameters. The log
file is expected to contain newline-delimited JSON entries.

Args:
    limit (int): Maximum number of entries to return.
    level (Optional[str]): Filter logs by severity level (e.g., DEBUG, INFO).
    contains (Optional[str]): Return only logs that include this string in the message.
    regex (Optional[str]): Return logs that match this regular expression in the message.
    from_time (Optional[str]): ISO 8601 timestamp to filter logs not older than this.
    to_time (Optional[str]): ISO 8601 timestamp to filter logs not newer than this.
    tail (bool): If True, fetch the most recent log entries (like `tail`).

Returns:
    JSONResponse: A JSON list of log entries.
"""

Parameters:

limit (query, optional): Maximum number of log entries to return.
level (query, optional): Filter by log level (e.g., INFO, ERROR).
contains (query, optional): Filter logs containing this substring.
regex (query, optional): Filter logs by matching regex in message.
from_time (query, optional): Start time (ISO format) for filtering logs.
to_time (query, optional): End time (ISO format) for filtering logs.
tail (query, optional): If True, returns the most recent lines (tail mode).

Responses:

200: Successful Response
422: Validation Error

PUT /v1/measurement/data

Links: local, eos

Fastapi Measurement Data Put

"""
Merge the measurement data given as datetime data into EOS measurements.
"""

Request Body:

application/json: { "$ref": "#/components/schemas/PydanticDateTimeData" }

Responses:

200: Successful Response
422: Validation Error

PUT /v1/measurement/dataframe

Links: local, eos

Fastapi Measurement Dataframe Put

"""
Merge the measurement data given as dataframe into EOS measurements.
"""

Request Body:

application/json: { "$ref": "#/components/schemas/PydanticDateTimeDataFrame" }

Responses:

200: Successful Response
422: Validation Error

GET /v1/measurement/keys

Links: local, eos

Fastapi Measurement Keys Get

"""
Get a list of available measurement keys.
"""

Responses:

200: Successful Response

GET /v1/measurement/series

Links: local, eos

Fastapi Measurement Series Get

"""
Get the measurements of given key as series.
"""

Parameters:

key (query, required): Measurement key.

Responses:

200: Successful Response
422: Validation Error

PUT /v1/measurement/series

Links: local, eos

Fastapi Measurement Series Put

"""
Merge measurement given as series into given key.
"""

Parameters:

key (query, required): Measurement key.

Request Body:

application/json: { "$ref": "#/components/schemas/PydanticDateTimeSeries" }

Responses:

200: Successful Response
422: Validation Error

PUT /v1/measurement/value

Links: local, eos

Fastapi Measurement Value Put

"""
Merge the measurement of given key and value into EOS measurements at given datetime.
"""

Parameters:

datetime (query, required): Datetime.
key (query, required): Measurement key.
value (query, required): No description provided.

Responses:

200: Successful Response
422: Validation Error

GET /v1/prediction/dataframe

Links: local, eos

Fastapi Prediction Dataframe Get

"""
Get prediction for given key within given date range as series.

Args:
    key (str): Prediction key
    start_datetime (Optional[str]): Starting datetime (inclusive).
        Defaults to start datetime of latest prediction.
    end_datetime (Optional[str]: Ending datetime (exclusive).

Defaults to end datetime of latest prediction.
"""

Parameters:

keys (query, required): Prediction keys.
start_datetime (query, optional): Starting datetime (inclusive).
end_datetime (query, optional): Ending datetime (exclusive).
interval (query, optional): Time duration for each interval. Defaults to 1 hour.

Responses:

200: Successful Response
422: Validation Error

PUT /v1/prediction/import/{provider_id}

Links: local, eos

Fastapi Prediction Import Provider

"""
Import prediction for given provider ID.

Args:
    provider_id: ID of provider to update.
    data: Prediction data.
    force_enable: Update data even if provider is disabled.
        Defaults to False.
"""

Parameters:

provider_id (path, required): Provider ID.
force_enable (query, optional): No description provided.

Request Body:

application/json: { "anyOf": [ { "$ref": "#/components/schemas/PydanticDateTimeDataFrame" }, { "$ref": "#/components/schemas/PydanticDateTimeData" }, { "type": "object", "additionalProperties": true }, { "type": "null" } ], "title": "Data" }

Responses:

200: Successful Response
422: Validation Error

GET /v1/prediction/keys

Links: local, eos

Fastapi Prediction Keys Get

"""
Get a list of available prediction keys.
"""

Responses:

200: Successful Response

GET /v1/prediction/list

Links: local, eos

Fastapi Prediction List Get

"""
Get prediction for given key within given date range as value list.

Args:
    key (str): Prediction key
    start_datetime (Optional[str]): Starting datetime (inclusive).
        Defaults to start datetime of latest prediction.
    end_datetime (Optional[str]: Ending datetime (exclusive).
        Defaults to end datetime of latest prediction.
    interval (Optional[str]): Time duration for each interval.
        Defaults to 1 hour.
"""

Parameters:

key (query, required): Prediction key.
start_datetime (query, optional): Starting datetime (inclusive).
end_datetime (query, optional): Ending datetime (exclusive).
interval (query, optional): Time duration for each interval. Defaults to 1 hour.

Responses:

200: Successful Response
422: Validation Error

GET /v1/prediction/providers

Links: local, eos

Fastapi Prediction Providers Get

"""
Get a list of available prediction providers.

Args:
    enabled (bool): Return enabled/disabled providers. If unset, return all providers.
"""

Parameters:

enabled (query, optional): No description provided.

Responses:

200: Successful Response
422: Validation Error

GET /v1/prediction/series

Links: local, eos

Fastapi Prediction Series Get

"""
Get prediction for given key within given date range as series.

Args:
    key (str): Prediction key
    start_datetime (Optional[str]): Starting datetime (inclusive).
        Defaults to start datetime of latest prediction.
    end_datetime (Optional[str]: Ending datetime (exclusive).
        Defaults to end datetime of latest prediction.
"""

Parameters:

key (query, required): Prediction key.
start_datetime (query, optional): Starting datetime (inclusive).
end_datetime (query, optional): Ending datetime (exclusive).

Responses:

200: Successful Response
422: Validation Error

POST /v1/prediction/update

Links: local, eos

Fastapi Prediction Update

"""
Update predictions for all providers.

Args:
    force_update: Update data even if it is already cached.
        Defaults to False.
    force_enable: Update data even if provider is disabled.
        Defaults to False.
"""

Parameters:

force_update (query, optional): No description provided.
force_enable (query, optional): No description provided.

Responses:

200: Successful Response
422: Validation Error

POST /v1/prediction/update/{provider_id}

Links: local, eos

Fastapi Prediction Update Provider

"""
Update predictions for given provider ID.

Args:
    provider_id: ID of provider to update.
    force_update: Update data even if it is already cached.
        Defaults to False.
    force_enable: Update data even if provider is disabled.
        Defaults to False.
"""

Parameters:

provider_id (path, required): No description provided.
force_update (query, optional): No description provided.
force_enable (query, optional): No description provided.

Responses:

200: Successful Response
422: Validation Error

GET /v1/resource/status

Links: local, eos

Fastapi Devices Status Get

"""
Get the latest status of a resource/ device.

Return:
    latest_status: The latest status of a resource/ device.
"""

Parameters:

resource_id (query, required): Resource ID.
actuator_id (query, optional): Actuator ID.

Responses:

200: Successful Response
422: Validation Error

PUT /v1/resource/status

Links: local, eos

Fastapi Devices Status Put

"""
Update the status of a resource/ device.

Return:
    latest_status: The latest status of a resource/ device.
"""

Parameters:

resource_id (query, required): Resource ID.
actuator_id (query, optional): Actuator ID.

Request Body:

application/json: { "anyOf": [ { "$ref": "#/components/schemas/PowerMeasurement-Input" }, { "$ref": "#/components/schemas/EnergyMeasurement-Input" }, { "$ref": "#/components/schemas/PPBCPowerProfileStatus-Input" }, { "$ref": "#/components/schemas/OMBCStatus" }, { "$ref": "#/components/schemas/FRBCActuatorStatus" }, { "$ref": "#/components/schemas/FRBCEnergyStatus-Input" }, { "$ref": "#/components/schemas/FRBCStorageStatus" }, { "$ref": "#/components/schemas/FRBCTimerStatus" }, { "$ref": "#/components/schemas/DDBCActuatorStatus" } ], "description": "Resource Status.", "title": "Status" }

Responses:

200: Successful Response
422: Validation Error

GET /visualization_results.pdf

Links: local, eos

Get Pdf

Responses:

200: Successful Response

Auto generated from openapi.json.

38 KiB Raw Blame History

Akkudoktor-EOS

POST /gesamtlast

GET /gesamtlast_simple

POST /optimize

GET /pvforecast

GET /strompreis

GET /v1/admin/cache

POST /v1/admin/cache/clear

POST /v1/admin/cache/clear-expired

POST /v1/admin/cache/load

POST /v1/admin/cache/save

GET /v1/admin/database/stats

POST /v1/admin/database/vacuum

POST /v1/admin/server/restart

POST /v1/admin/server/shutdown

GET /v1/config

PUT /v1/config

GET /v1/config/backup

PUT /v1/config/file

POST /v1/config/reset

PUT /v1/config/revert

GET /v1/config/{path}

PUT /v1/config/{path}

GET /v1/energy-management/optimization/solution

GET /v1/energy-management/plan

GET /v1/health

GET /v1/logging/log

PUT /v1/measurement/data

PUT /v1/measurement/dataframe

GET /v1/measurement/keys

GET /v1/measurement/series

PUT /v1/measurement/series

PUT /v1/measurement/value

GET /v1/prediction/dataframe

PUT /v1/prediction/import/{provider_id}

GET /v1/prediction/keys

GET /v1/prediction/list

GET /v1/prediction/providers

GET /v1/prediction/series

POST /v1/prediction/update

POST /v1/prediction/update/{provider_id}

GET /v1/resource/status

PUT /v1/resource/status

GET /visualization_results.pdf

38 KiB

Raw Blame History