chore: adapt deprecated endpoints to 15-minutes predictions (#1195)
Bump Version / Bump Version Workflow (push) Waiting to run
CodeQL Advanced / Analyze (actions) (push) Waiting to run
CodeQL Advanced / Analyze (python) (push) Waiting to run
docker-build / platform-excludes (push) Waiting to run
docker-build / build (push) Blocked by required conditions
docker-build / merge (push) Blocked by required conditions
pre-commit / pre-commit (push) Waiting to run
Run Pytest on Pull Request / test (push) Waiting to run

Ensure the deprecated endpoints get /strompreis, post /gesamtlast, get /gesamtlast_simple,
get /pvforecast to work on 1-hour intervalls even if the prediction provides 15-minutes
intervall data. This keeps the interface compliant to the legacy functionality.

Besides this adaptations several other improvements and fixes are included in this PR.

* feat: extend data management method key_to array by resample_method

  One can now define the resample method on how to aggregate the values in an interval
  for resampling. Three methods are provided:

  - "first": Use the first value in each interval.
  - "mean": Compute the arithmetic mean of all samples in each interval.
  - "interval_mean": Compute the time-weighted mean assuming each value remains valid
     until the next timestamp (piecewise-constant signal).

* feat: extend the get /v1/prediction/dataframe endpoint with resampling parameters

  Make all parameters for resampling available at the endpoint.

* feat: extend the get /v1/prediction/list endpoint with resampling parameters

  Make all parameters for resampling available at the endpoint.

* feat: add new delete /v1/prediction/range endpoint

  The endpoint allows to delete prediction values for a given time span.

* fix: adapt for PVForecastAkkudoktor server side cache handling

  /api.akkudoktor/forecast does it's own caching on requests. Call it with slightly
  randomized request values to avoid getting cached values in the case we need fresh
  data. The requests are anyway rate limited to one request per hour on our side.

* chore: add core.types

  This module centralizes reusable type definitions shared across multiple
  packages. Defining common types here avoids duplication of complex type
  annotations (such as Literal aliases), ensures consistent typing across the
  code base, and helps prevent circular import dependencies between modules.

* chore: extend cache testing

* chore: add system test for deprecated /strompreis endpoint

* chore: add unit test module for server endpoints

  Add a new test module to do unit tests on server endpoints. First test added
  for deprecated get /strompreis endpoint.

Signed-off-by: Bobby Noelte <b0661n0e17e@gmail.com>
This commit is contained in:
Bobby Noelte
2026-07-30 19:54:05 +02:00
committed by GitHub
parent 52fe489d4e
commit 8344974c16
16 changed files with 1232 additions and 95 deletions
+1 -1
View File
@@ -11,7 +11,7 @@ DOCKER_COMPOSE_DATA_DIR=${HOME}/.local/share/net.akkudoktor.eos
# -----------------------------------------------------------------------------
# Image / build
# -----------------------------------------------------------------------------
VERSION=0.3.0.dev2607290944830210
VERSION=0.3.0.dev2607301181843779
PYTHON_VERSION=3.13.9
# -----------------------------------------------------------------------------
+1 -1
View File
@@ -6,7 +6,7 @@
# the root directory (no add-on folder as usual).
name: "Akkudoktor-EOS"
version: "0.3.0.dev2607290944830210"
version: "0.3.0.dev2607301181843779"
slug: "eos"
description: "Akkudoktor-EOS add-on"
url: "https://github.com/Akkudoktor-EOS/EOS"
+145 -16
View File
@@ -1,6 +1,6 @@
# Akkudoktor-EOS
**Version**: `v0.3.0.dev2607290944830210`
**Version**: `v0.3.0.dev2607301181843779`
<!-- pyml disable line-length -->
**Description**: This project provides a comprehensive solution for simulating and optimizing an energy system based on renewable energy sources. With a focus on photovoltaic (PV) systems, battery storage (batteries), load management (consumer requirements), heat pumps, electric vehicles, and consideration of electricity price data, this system enables forecasting and optimization of energy flow and costs over a specified period.
@@ -38,9 +38,9 @@ Deprecated: Total Load Prediction with adjustment.
Endpoint to handle total load prediction adjusted by latest measured data.
Total load prediction starts at 00.00.00 today and is provided for 48 hours.
If no prediction values are available the missing ones at the start of the series are
filled with the first available prediction value.
Total load prediction starts at 00.00.00 today and is provided for 48 hours
in 1-hour intervals. If no prediction values are available the missing ones
at the start of the series are filled with the first available prediction value.
Note:
Use '/v1/prediction/list?key=loadforecast_power_w' instead.
@@ -82,9 +82,9 @@ Deprecated: Total Load Prediction.
Endpoint to handle total load prediction.
Total load prediction starts at 00.00.00 today and is provided for 48 hours.
If no prediction values are available the missing ones at the start of the series are
filled with the first available prediction value.
Total load prediction starts at 00.00.00 today and is provided for 48 hours
in 1-hour intervals. If no prediction values are available the missing ones
at the start of the series are filled with the first available prediction value.
Args:
year_energy (float): Yearly energy consumption in Wh.
@@ -169,9 +169,9 @@ Deprecated: PV Forecast Prediction.
Endpoint to handle PV forecast prediction.
PVForecast starts at 00.00.00 today and is provided for 48 hours.
If no forecast values are available the missing ones at the start of the series are
filled with the first available forecast value.
PVForecast starts at 00.00.00 today and is provided for 48 hours
in 1-hour intervals. If no forecast values are available the missing ones
at the start of the series are filled with the first available forecast value.
Note:
Set PVForecastAkkudoktor as provider, then update data with
@@ -202,9 +202,9 @@ Fastapi Strompreis
"""
Deprecated: Electricity Market Price Prediction per Wh [amount/Wh].
Electricity prices start at 00.00.00 today and are provided for 48 hours.
If no prices are available the missing ones at the start of the series are
filled with the first available price.
Electricity prices start at 00.00.00 today and are provided for 48 hours
in 1-hour intervals. If no prices are available the missing ones at the
start of the series are filled with the first available price.
Note:
Electricity price charges are added.
@@ -1091,15 +1091,54 @@ Fastapi Prediction Dataframe Get
<!-- pyml disable line-length -->
```python
"""
Get prediction for given key within given date range as series.
Get prediction for given keys within given date range as dataframe.
Args:
key (str): Prediction key
key (list[str]): Prediction keys
start_datetime (Optional[str]): Starting datetime (inclusive).
Defaults to start datetime of latest prediction.
end_datetime (Optional[str]: Ending datetime (exclusive).
Defaults to end datetime of latest prediction.
interval (Optional[str]): Time duration for each interval.
Defaults to 1 hour.
fill_method (str): Method to handle missing values during resampling.
Defaults to end datetime of latest prediction.
- 'linear': Linearly interpolate missing values (for numeric data only).
- 'time': Interpolate missing values (for numeric data only).
- 'ffill': Forward fill missing values.
- 'bfill': Backward fill missing values.
- Defaults to 'linear' for numeric values, otherwise 'ffill'.
resample_method (str):
Method used to aggregate values within a resampling interval.
- "first": Use the first value in each interval.
- "mean": Compute the arithmetic mean of all samples in each interval.
- "interval_mean": Compute the time-weighted mean assuming each
value remains valid until the next timestamp (piecewise-constant
signal).
dropna: (bool, optional): Whether to drop NAN/ None values before processing.
Defaults to True.
boundary (Literal["strict", "context"]): resampling boundary
"strict" → only values inside [start, end)
"context" → include one value before and after for proper resampling
align_to_interval (bool): When True, snap the resample origin to the nearest
UTC epoch-aligned boundary of ``interval`` before resampling. This ensures
that bucket timestamps always fall on wall-clock-round times regardless of
when ``start_datetime`` falls:
- 15-minute interval → buckets on :00, :15, :30, :45
- 1-hour interval → buckets on the hour
When False (default), the origin is ``query_start`` (or ``"start_day"`` when
no start is given), preserving the existing behaviour where buckets are
aligned to the query window rather than the clock.
Set to True when storing compacted records back to the database so that the
resulting timestamps are predictable and human-readable. Leave False for
forecast or reporting queries where alignment to the exact query window is
more important than clock-round boundaries.
"""
```
<!-- pyml enable line-length -->
@@ -1114,6 +1153,16 @@ Defaults to end datetime of latest prediction.
- `interval` (query, optional): Time duration for each interval. Defaults to 1 hour.
- `fill_method` (query, optional): Method to handle missing values during resampling.
- `resample_method` (query, optional): Method used to aggregate values within a resampling interval.
- `dropna` (query, optional): Drop NAN/ None values before processing.
- `boundary` (query, optional): Resampling boundary mode.
- `align_to_interval` (query, optional): Snap resample origin to the nearest UTC epoch-aligned boundary of interval.
**Responses**:
- **200**: Successful Response
@@ -1222,6 +1271,44 @@ Args:
Defaults to end datetime of latest prediction.
interval (Optional[str]): Time duration for each interval.
Defaults to 1 hour.
fill_method (str): Method to handle missing values during resampling.
- 'linear': Linearly interpolate missing values (for numeric data only).
- 'time': Interpolate missing values (for numeric data only).
- 'ffill': Forward fill missing values.
- 'bfill': Backward fill missing values.
- Defaults to 'linear' for numeric values, otherwise 'ffill'.
resample_method (str):
Method used to aggregate values within a resampling interval.
- "first": Use the first value in each interval.
- "mean": Compute the arithmetic mean of all samples in each interval.
- "interval_mean": Compute the time-weighted mean assuming each
value remains valid until the next timestamp (piecewise-constant
signal).
dropna: (bool, optional): Whether to drop NAN/ None values before processing.
Defaults to True.
boundary (Literal["strict", "context"]): resampling boundary
"strict" → only values inside [start, end)
"context" → include one value before and after for proper resampling
align_to_interval (bool): When True, snap the resample origin to the nearest
UTC epoch-aligned boundary of ``interval`` before resampling. This ensures
that bucket timestamps always fall on wall-clock-round times regardless of
when ``start_datetime`` falls:
- 15-minute interval → buckets on :00, :15, :30, :45
- 1-hour interval → buckets on the hour
When False (default), the origin is ``query_start`` (or ``"start_day"`` when
no start is given), preserving the existing behaviour where buckets are
aligned to the query window rather than the clock.
Set to True when storing compacted records back to the database so that the
resulting timestamps are predictable and human-readable. Leave False for
forecast or reporting queries where alignment to the exact query window is
more important than clock-round boundaries.
"""
```
<!-- pyml enable line-length -->
@@ -1236,6 +1323,16 @@ Args:
- `interval` (query, optional): Time duration for each interval. Defaults to 1 hour.
- `fill_method` (query, optional): Method to handle missing values during resampling.
- `resample_method` (query, optional): Method used to aggregate values within a resampling interval.
- `dropna` (query, optional): Drop NAN/ None values before processing.
- `boundary` (query, optional): Resampling boundary mode.
- `align_to_interval` (query, optional): Snap resample origin to the nearest UTC epoch-aligned boundary of interval.
**Responses**:
- **200**: Successful Response
@@ -1275,6 +1372,38 @@ Args:
---
## DELETE /v1/prediction/range
<!-- pyml disable line-length -->
**Links**: [local](http://localhost:8503/docs#/default/fastapi_prediction_range_delete_v1_prediction_range_delete), [eos](https://petstore3.swagger.io/?url=https://raw.githubusercontent.com/Akkudoktor-EOS/EOS/refs/heads/main/openapi.json#/default/fastapi_prediction_range_delete_v1_prediction_range_delete)
<!-- pyml enable line-length -->
Fastapi Prediction Range Delete
<!-- pyml disable line-length -->
```python
"""
Delete prediction values for a key within a datetime range.
"""
```
<!-- pyml enable line-length -->
**Parameters**:
- `key` (query, required): Prediction key.
- `start_datetime` (query, optional): Start datetime.
- `end_datetime` (query, optional): End datetime.
**Responses**:
- **200**: Successful Response
- **422**: Validation Error
---
## GET /v1/prediction/series
<!-- pyml disable line-length -->
+262 -7
View File
@@ -8,7 +8,7 @@
"name": "Apache 2.0",
"url": "https://www.apache.org/licenses/LICENSE-2.0.html"
},
"version": "v0.3.0.dev2607290944830210"
"version": "v0.3.0.dev2607301181843779"
},
"paths": {
"/v1/admin/cache/clear": {
@@ -1470,7 +1470,7 @@
"prediction"
],
"summary": "Fastapi Prediction Dataframe Get",
"description": "Get prediction for given key within given date range as series.\n\nArgs:\n key (str): Prediction key\n start_datetime (Optional[str]): Starting datetime (inclusive).\n Defaults to start datetime of latest prediction.\n end_datetime (Optional[str]: Ending datetime (exclusive).\n\nDefaults to end datetime of latest prediction.",
"description": "Get prediction for given keys within given date range as dataframe.\n\nArgs:\n key (list[str]): Prediction keys\n start_datetime (Optional[str]): Starting datetime (inclusive).\n Defaults to start datetime of latest prediction.\n end_datetime (Optional[str]: Ending datetime (exclusive).\n Defaults to end datetime of latest prediction.\n interval (Optional[str]): Time duration for each interval.\n Defaults to 1 hour.\n fill_method (str): Method to handle missing values during resampling.\n\n - 'linear': Linearly interpolate missing values (for numeric data only).\n - 'time': Interpolate missing values (for numeric data only).\n - 'ffill': Forward fill missing values.\n - 'bfill': Backward fill missing values.\n - Defaults to 'linear' for numeric values, otherwise 'ffill'.\n\n resample_method (str):\n Method used to aggregate values within a resampling interval.\n\n - \"first\": Use the first value in each interval.\n - \"mean\": Compute the arithmetic mean of all samples in each interval.\n - \"interval_mean\": Compute the time-weighted mean assuming each\n value remains valid until the next timestamp (piecewise-constant\n signal).\n\n dropna: (bool, optional): Whether to drop NAN/ None values before processing.\n Defaults to True.\n boundary (Literal[\"strict\", \"context\"]): resampling boundary\n \"strict\" \u2192 only values inside [start, end)\n \"context\" \u2192 include one value before and after for proper resampling\n align_to_interval (bool): When True, snap the resample origin to the nearest\n UTC epoch-aligned boundary of ``interval`` before resampling. This ensures\n that bucket timestamps always fall on wall-clock-round times regardless of\n when ``start_datetime`` falls:\n\n - 15-minute interval \u2192 buckets on :00, :15, :30, :45\n - 1-hour interval \u2192 buckets on the hour\n\n When False (default), the origin is ``query_start`` (or ``\"start_day\"`` when\n no start is given), preserving the existing behaviour where buckets are\n aligned to the query window rather than the clock.\n\n Set to True when storing compacted records back to the database so that the\n resulting timestamps are predictable and human-readable. Leave False for\n forecast or reporting queries where alignment to the exact query window is\n more important than clock-round boundaries.",
"operationId": "fastapi_prediction_dataframe_get_v1_prediction_dataframe_get",
"parameters": [
{
@@ -1540,6 +1540,93 @@
"title": "Interval"
},
"description": "Time duration for each interval. Defaults to 1 hour."
},
{
"name": "fill_method",
"in": "query",
"required": false,
"schema": {
"anyOf": [
{
"enum": [
"linear",
"time",
"ffill",
"bfill"
],
"type": "string"
},
{
"type": "null"
}
],
"description": "Method to handle missing values during resampling.",
"title": "Fill Method"
},
"description": "Method to handle missing values during resampling."
},
{
"name": "resample_method",
"in": "query",
"required": false,
"schema": {
"enum": [
"first",
"mean",
"interval_mean"
],
"type": "string",
"description": "Method used to aggregate values within a resampling interval.",
"default": "mean",
"title": "Resample Method"
},
"description": "Method used to aggregate values within a resampling interval."
},
{
"name": "dropna",
"in": "query",
"required": false,
"schema": {
"anyOf": [
{
"type": "boolean"
},
{
"type": "null"
}
],
"description": "Drop NAN/ None values before processing.",
"title": "Dropna"
},
"description": "Drop NAN/ None values before processing."
},
{
"name": "boundary",
"in": "query",
"required": false,
"schema": {
"enum": [
"strict",
"context"
],
"type": "string",
"description": "Resampling boundary mode.",
"default": "context",
"title": "Boundary"
},
"description": "Resampling boundary mode."
},
{
"name": "align_to_interval",
"in": "query",
"required": false,
"schema": {
"type": "boolean",
"description": "Snap resample origin to the nearest UTC epoch-aligned boundary of interval.",
"default": false,
"title": "Align To Interval"
},
"description": "Snap resample origin to the nearest UTC epoch-aligned boundary of interval."
}
],
"responses": {
@@ -1572,7 +1659,7 @@
"prediction"
],
"summary": "Fastapi Prediction List Get",
"description": "Get prediction for given key within given date range as value list.\n\nArgs:\n key (str): Prediction key\n start_datetime (Optional[str]): Starting datetime (inclusive).\n Defaults to start datetime of latest prediction.\n end_datetime (Optional[str]: Ending datetime (exclusive).\n Defaults to end datetime of latest prediction.\n interval (Optional[str]): Time duration for each interval.\n Defaults to 1 hour.",
"description": "Get prediction for given key within given date range as value list.\n\nArgs:\n key (str): Prediction key\n start_datetime (Optional[str]): Starting datetime (inclusive).\n Defaults to start datetime of latest prediction.\n end_datetime (Optional[str]: Ending datetime (exclusive).\n Defaults to end datetime of latest prediction.\n interval (Optional[str]): Time duration for each interval.\n Defaults to 1 hour.\n fill_method (str): Method to handle missing values during resampling.\n\n - 'linear': Linearly interpolate missing values (for numeric data only).\n - 'time': Interpolate missing values (for numeric data only).\n - 'ffill': Forward fill missing values.\n - 'bfill': Backward fill missing values.\n - Defaults to 'linear' for numeric values, otherwise 'ffill'.\n\n resample_method (str):\n Method used to aggregate values within a resampling interval.\n\n - \"first\": Use the first value in each interval.\n - \"mean\": Compute the arithmetic mean of all samples in each interval.\n - \"interval_mean\": Compute the time-weighted mean assuming each\n value remains valid until the next timestamp (piecewise-constant\n signal).\n\n dropna: (bool, optional): Whether to drop NAN/ None values before processing.\n Defaults to True.\n boundary (Literal[\"strict\", \"context\"]): resampling boundary\n \"strict\" \u2192 only values inside [start, end)\n \"context\" \u2192 include one value before and after for proper resampling\n align_to_interval (bool): When True, snap the resample origin to the nearest\n UTC epoch-aligned boundary of ``interval`` before resampling. This ensures\n that bucket timestamps always fall on wall-clock-round times regardless of\n when ``start_datetime`` falls:\n\n - 15-minute interval \u2192 buckets on :00, :15, :30, :45\n - 1-hour interval \u2192 buckets on the hour\n\n When False (default), the origin is ``query_start`` (or ``\"start_day\"`` when\n no start is given), preserving the existing behaviour where buckets are\n aligned to the query window rather than the clock.\n\n Set to True when storing compacted records back to the database so that the\n resulting timestamps are predictable and human-readable. Leave False for\n forecast or reporting queries where alignment to the exact query window is\n more important than clock-round boundaries.",
"operationId": "fastapi_prediction_list_get_v1_prediction_list_get",
"parameters": [
{
@@ -1639,6 +1726,93 @@
"title": "Interval"
},
"description": "Time duration for each interval. Defaults to 1 hour."
},
{
"name": "fill_method",
"in": "query",
"required": false,
"schema": {
"anyOf": [
{
"enum": [
"linear",
"time",
"ffill",
"bfill"
],
"type": "string"
},
{
"type": "null"
}
],
"description": "Method to handle missing values during resampling.",
"title": "Fill Method"
},
"description": "Method to handle missing values during resampling."
},
{
"name": "resample_method",
"in": "query",
"required": false,
"schema": {
"enum": [
"first",
"mean",
"interval_mean"
],
"type": "string",
"description": "Method used to aggregate values within a resampling interval.",
"default": "mean",
"title": "Resample Method"
},
"description": "Method used to aggregate values within a resampling interval."
},
{
"name": "dropna",
"in": "query",
"required": false,
"schema": {
"anyOf": [
{
"type": "boolean"
},
{
"type": "null"
}
],
"description": "Drop NAN/ None values before processing.",
"title": "Dropna"
},
"description": "Drop NAN/ None values before processing."
},
{
"name": "boundary",
"in": "query",
"required": false,
"schema": {
"enum": [
"strict",
"context"
],
"type": "string",
"description": "Resampling boundary mode.",
"default": "context",
"title": "Boundary"
},
"description": "Resampling boundary mode."
},
{
"name": "align_to_interval",
"in": "query",
"required": false,
"schema": {
"type": "boolean",
"description": "Snap resample origin to the nearest UTC epoch-aligned boundary of interval.",
"default": false,
"title": "Align To Interval"
},
"description": "Snap resample origin to the nearest UTC epoch-aligned boundary of interval."
}
],
"responses": {
@@ -1891,6 +2065,87 @@
}
}
},
"/v1/prediction/range": {
"delete": {
"tags": [
"prediction"
],
"summary": "Fastapi Prediction Range Delete",
"description": "Delete prediction values for a key within a datetime range.",
"operationId": "fastapi_prediction_range_delete_v1_prediction_range_delete",
"parameters": [
{
"name": "key",
"in": "query",
"required": true,
"schema": {
"type": "string",
"description": "Prediction key.",
"title": "Key"
},
"description": "Prediction key."
},
{
"name": "start_datetime",
"in": "query",
"required": false,
"schema": {
"anyOf": [
{
"type": "string"
},
{
"type": "null"
}
],
"description": "Start datetime.",
"title": "Start Datetime"
},
"description": "Start datetime."
},
{
"name": "end_datetime",
"in": "query",
"required": false,
"schema": {
"anyOf": [
{
"type": "string"
},
{
"type": "null"
}
],
"description": "End datetime.",
"title": "End Datetime"
},
"description": "End datetime."
}
],
"responses": {
"200": {
"description": "Successful Response",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/PydanticDateTimeSeries"
}
}
}
},
"422": {
"description": "Validation Error",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/HTTPValidationError"
}
}
}
}
}
}
},
"/v1/energy-management/optimization/solution": {
"get": {
"tags": [
@@ -1992,7 +2247,7 @@
"prediction"
],
"summary": "Fastapi Strompreis",
"description": "Deprecated: Electricity Market Price Prediction per Wh [amount/Wh].\n\nElectricity prices start at 00.00.00 today and are provided for 48 hours.\nIf no prices are available the missing ones at the start of the series are\nfilled with the first available price.\n\nNote:\n Electricity price charges are added.\n\nNote:\n Set ElecPriceAkkudoktor as provider, then update data with\n '/v1/prediction/update'\n and then request data with\n '/v1/prediction/list?key=elecprice_marketprice_wh' or\n '/v1/prediction/list?key=elecprice_marketprice_kwh' instead.",
"description": "Deprecated: Electricity Market Price Prediction per Wh [amount/Wh].\n\nElectricity prices start at 00.00.00 today and are provided for 48 hours\nin 1-hour intervals. If no prices are available the missing ones at the\nstart of the series are filled with the first available price.\n\nNote:\n Electricity price charges are added.\n\nNote:\n Set ElecPriceAkkudoktor as provider, then update data with\n '/v1/prediction/update'\n and then request data with\n '/v1/prediction/list?key=elecprice_marketprice_wh' or\n '/v1/prediction/list?key=elecprice_marketprice_kwh' instead.",
"operationId": "fastapi_strompreis_strompreis_get",
"responses": {
"200": {
@@ -2019,7 +2274,7 @@
"prediction"
],
"summary": "Fastapi Gesamtlast",
"description": "Deprecated: Total Load Prediction with adjustment.\n\nEndpoint to handle total load prediction adjusted by latest measured data.\n\nTotal load prediction starts at 00.00.00 today and is provided for 48 hours.\nIf no prediction values are available the missing ones at the start of the series are\nfilled with the first available prediction value.\n\nNote:\n Use '/v1/prediction/list?key=loadforecast_power_w' instead.\n Load energy meter readings to be added to EOS measurement by:\n '/v1/measurement/value' or\n '/v1/measurement/series' or\n '/v1/measurement/dataframe' or\n '/v1/measurement/data'",
"description": "Deprecated: Total Load Prediction with adjustment.\n\nEndpoint to handle total load prediction adjusted by latest measured data.\n\nTotal load prediction starts at 00.00.00 today and is provided for 48 hours\nin 1-hour intervals. If no prediction values are available the missing ones\nat the start of the series are filled with the first available prediction value.\n\nNote:\n Use '/v1/prediction/list?key=loadforecast_power_w' instead.\n Load energy meter readings to be added to EOS measurement by:\n '/v1/measurement/value' or\n '/v1/measurement/series' or\n '/v1/measurement/dataframe' or\n '/v1/measurement/data'",
"operationId": "fastapi_gesamtlast_gesamtlast_post",
"requestBody": {
"content": {
@@ -2066,7 +2321,7 @@
"prediction"
],
"summary": "Fastapi Gesamtlast Simple",
"description": "Deprecated: Total Load Prediction.\n\nEndpoint to handle total load prediction.\n\nTotal load prediction starts at 00.00.00 today and is provided for 48 hours.\nIf no prediction values are available the missing ones at the start of the series are\nfilled with the first available prediction value.\n\nArgs:\n year_energy (float): Yearly energy consumption in Wh.\n\nNote:\n Set LoadAkkudoktor as provider, then update data with\n '/v1/prediction/update'\n and then request data with\n '/v1/prediction/list?key=loadforecast_power_w' instead.",
"description": "Deprecated: Total Load Prediction.\n\nEndpoint to handle total load prediction.\n\nTotal load prediction starts at 00.00.00 today and is provided for 48 hours\nin 1-hour intervals. If no prediction values are available the missing ones\nat the start of the series are filled with the first available prediction value.\n\nArgs:\n year_energy (float): Yearly energy consumption in Wh.\n\nNote:\n Set LoadAkkudoktor as provider, then update data with\n '/v1/prediction/update'\n and then request data with\n '/v1/prediction/list?key=loadforecast_power_w' instead.",
"operationId": "fastapi_gesamtlast_simple_gesamtlast_simple_get",
"deprecated": true,
"parameters": [
@@ -2114,7 +2369,7 @@
"prediction"
],
"summary": "Fastapi Pvforecast",
"description": "Deprecated: PV Forecast Prediction.\n\nEndpoint to handle PV forecast prediction.\n\nPVForecast starts at 00.00.00 today and is provided for 48 hours.\nIf no forecast values are available the missing ones at the start of the series are\nfilled with the first available forecast value.\n\nNote:\n Set PVForecastAkkudoktor as provider, then update data with\n '/v1/prediction/update'\n and then request data with\n '/v1/prediction/list?key=pvforecast_ac_power' and\n '/v1/prediction/list?key=pvforecastakkudoktor_temp_air' instead.",
"description": "Deprecated: PV Forecast Prediction.\n\nEndpoint to handle PV forecast prediction.\n\nPVForecast starts at 00.00.00 today and is provided for 48 hours\nin 1-hour intervals. If no forecast values are available the missing ones\nat the start of the series are filled with the first available forecast value.\n\nNote:\n Set PVForecastAkkudoktor as provider, then update data with\n '/v1/prediction/update'\n and then request data with\n '/v1/prediction/list?key=pvforecast_ac_power' and\n '/v1/prediction/list?key=pvforecastakkudoktor_temp_air' instead.",
"operationId": "fastapi_pvforecast_pvforecast_get",
"responses": {
"200": {
+185 -21
View File
@@ -21,7 +21,6 @@ from typing import (
Any,
Dict,
Iterator,
Literal,
Optional,
Tuple,
Type,
@@ -61,6 +60,11 @@ from akkudoktoreos.core.pydantic import (
PydanticDateTimeData,
PydanticDateTimeDataFrame,
)
from akkudoktoreos.core.types import (
BoundaryMode,
FillMethod,
ResampleMethod,
)
from akkudoktoreos.utils.datetimeutil import (
DateTime,
Duration,
@@ -69,6 +73,8 @@ from akkudoktoreos.utils.datetimeutil import (
to_duration,
)
# ==================== Base Class ====================
class DataABC(ConfigMixin, StartMixin, PydanticBaseModel):
"""Base class for handling generic data.
@@ -1196,9 +1202,10 @@ class DataSequence(DataABC, DatabaseRecordProtocolMixin[DataRecord]):
start_datetime: Optional[DateTime] = None,
end_datetime: Optional[DateTime] = None,
interval: Optional[Duration] = None,
fill_method: Optional[str] = None,
fill_method: Optional[FillMethod] = None,
resample_method: ResampleMethod = "mean",
dropna: Optional[bool] = True,
boundary: Literal["strict", "context"] = "context",
boundary: BoundaryMode = "context",
align_to_interval: bool = False,
) -> NDArray[Shape["*"], Any]:
"""Extract an array indexed by fixed time intervals from data records within an optional date range.
@@ -1209,14 +1216,25 @@ class DataSequence(DataABC, DatabaseRecordProtocolMixin[DataRecord]):
end_datetime (datetime, optional): The end date for filtering the records (exclusive).
interval (duration, optional): The fixed time interval. Defaults to 1 hour.
fill_method (str): Method to handle missing values during resampling.
- 'linear': Linearly interpolate missing values (for numeric data only).
- 'time': Interpolate missing values (for numeric data only).
- 'ffill': Forward fill missing values.
- 'bfill': Backward fill missing values.
- 'none': Defaults to 'linear' for numeric values, otherwise 'ffill'.
- Defaults to 'linear' for numeric values, otherwise 'ffill'.
resample_method (str):
Method used to aggregate values within a resampling interval.
- "first": Use the first value in each interval.
- "mean": Compute the arithmetic mean of all samples in each interval.
- "interval_mean": Compute the time-weighted mean assuming each
value remains valid until the next timestamp (piecewise-constant
signal).
dropna: (bool, optional): Whether to drop NAN/ None values before processing.
Defaults to True.
boundary (Literal["strict", "context"]):
boundary (Literal["strict", "context"]): resampling boundary
"strict" → only values inside [start, end)
"context" → include one value before and after for proper resampling
align_to_interval (bool): When True, snap the resample origin to the nearest
@@ -1249,6 +1267,9 @@ class DataSequence(DataABC, DatabaseRecordProtocolMixin[DataRecord]):
if fill_method not in ("ffill", "bfill", "linear", "time", "none", None):
raise ValueError(f"Unsupported fill method: {fill_method}")
if resample_method not in ("first", "mean", "interval_mean"):
raise ValueError(f"Unsupported resample method: {resample_method}")
if boundary not in ("strict", "context"):
raise ValueError(f"Unsupported boundary mode: {boundary}")
@@ -1368,10 +1389,32 @@ class DataSequence(DataABC, DatabaseRecordProtocolMixin[DataRecord]):
if is_numeric:
# Step 1: aggregate — collapses sub-interval data (e.g. 4x 15min → 1h mean).
# Produces NaN for buckets where no data existed at all.
resampled = pd.to_numeric(
series.resample(resample_freq, origin=resample_origin).mean(),
errors="coerce", # ← ensures float64, not object dtype
)
numeric_series = pd.to_numeric(
series, errors="coerce"
) # ← ensures float64, not object dtype
if resample_method == "first":
resampled = numeric_series.resample(
resample_freq,
origin=resample_origin,
).first()
elif resample_method == "mean":
resampled = numeric_series.resample(
resample_freq,
origin=resample_origin,
).mean()
elif resample_method == "interval_mean":
# Treat each value as valid until the next timestamp.
expanded = numeric_series.resample("1s").ffill()
resampled = expanded.resample(
resample_freq,
origin=resample_origin,
).mean()
else:
raise ValueError(f"Unsupported resample method: {resample_method}")
# Step 2: fill gaps — interpolates or fills the NaN buckets from step 1.
if fill_method in ("linear", "time"):
@@ -2385,23 +2428,59 @@ class DataContainer(SingletonMixin, DataABC):
start_datetime: Optional[DateTime] = None,
end_datetime: Optional[DateTime] = None,
interval: Optional[Duration] = None,
fill_method: Optional[str] = None,
boundary: Optional[str] = "context",
fill_method: Optional[FillMethod] = None,
resample_method: ResampleMethod = "mean",
dropna: Optional[bool] = True,
boundary: BoundaryMode = "context",
align_to_interval: bool = False,
) -> NDArray[Shape["*"], Any]:
"""Retrieve an array indexed by fixed time intervals for a specified key from the data in each DataProvider.
Iterates through providers to find and return the first available array for the specified key.
Args:
key (str): The field name to retrieve, representing a data attribute in DataRecords.
key (str): The field name in the DataRecord from which to extract values.
start_datetime (datetime, optional): The start date for filtering the records (inclusive).
end_datetime (datetime, optional): The end date for filtering the records (exclusive).
interval (duration, optional): The fixed time interval. Defaults to 1 hour.
fill_method (str): Method to handle missing values during resampling.
- 'linear': Linearly interpolate missing values (for numeric data only).
- 'time': Interpolate missing values (for numeric data only).
- 'ffill': Forward fill missing values.
- 'bfill': Backward fill missing values.
- 'none': Defaults to 'linear' for numeric values, otherwise 'ffill'.
- Defaults to 'linear' for numeric values, otherwise 'ffill'.
resample_method (str):
Method used to aggregate values within a resampling interval.
- "first": Use the first value in each interval.
- "mean": Compute the arithmetic mean of all samples in each interval.
- "interval_mean": Compute the time-weighted mean assuming each
value remains valid until the next timestamp (piecewise-constant
signal).
dropna: (bool, optional): Whether to drop NAN/ None values before processing.
Defaults to True.
boundary (Literal["strict", "context"]):
"strict" → only values inside [start, end)
"context" → include one value before and after for proper resampling
align_to_interval (bool): When True, snap the resample origin to the nearest
UTC epoch-aligned boundary of ``interval`` before resampling. This ensures
that bucket timestamps always fall on wall-clock-round times regardless of
when ``start_datetime`` falls:
- 15-minute interval → buckets on :00, :15, :30, :45
- 1-hour interval → buckets on the hour
When False (default), the origin is ``query_start`` (or ``"start_day"`` when
no start is given), preserving the existing behaviour where buckets are
aligned to the query window rather than the clock.
Set to True when storing compacted records back to the database so that the
resulting timestamps are predictable and human-readable. Leave False for
forecast or reporting queries where alignment to the exact query window is
more important than clock-round boundaries.
Returns:
np.ndarray: A NumPy array containing aggregated data for the specified key.
@@ -2421,7 +2500,10 @@ class DataContainer(SingletonMixin, DataABC):
end_datetime=end_datetime,
interval=interval,
fill_method=fill_method,
resample_method=resample_method,
dropna=dropna,
boundary=boundary,
align_to_interval=align_to_interval,
)
break
except KeyError:
@@ -2437,23 +2519,60 @@ class DataContainer(SingletonMixin, DataABC):
keys: list[str],
start_datetime: Optional[DateTime] = None,
end_datetime: Optional[DateTime] = None,
interval: Optional[Any] = None, # Duration assumed
fill_method: Optional[str] = None,
interval: Optional[Duration] = None,
fill_method: Optional[FillMethod] = None,
resample_method: ResampleMethod = "mean",
dropna: Optional[bool] = True,
boundary: BoundaryMode = "context",
align_to_interval: bool = False,
) -> pd.DataFrame:
"""Retrieve a dataframe indexed by fixed time intervals for specified keys from the data in each DataProvider.
Generates a pandas DataFrame using the NumPy arrays for each specified key, ensuring a common time index.
Args:
keys (list[str]): A list of field names to retrieve.
start_datetime (datetime, optional): Start date for filtering records (inclusive).
end_datetime (datetime, optional): End date for filtering records (exclusive).
keys (list[str]): The field names in the DataRecords from which to extract values.
start_datetime (datetime, optional): The start date for filtering the records (inclusive).
end_datetime (datetime, optional): The end date for filtering the records (exclusive).
interval (duration, optional): The fixed time interval. Defaults to 1 hour.
fill_method (str, optional): Method to handle missing values during resampling.
fill_method (str): Method to handle missing values during resampling.
- 'linear': Linearly interpolate missing values (for numeric data only).
- 'time': Interpolate missing values (for numeric data only).
- 'ffill': Forward fill missing values.
- 'bfill': Backward fill missing values.
- 'none': Defaults to 'linear' for numeric values, otherwise 'ffill'.
- Defaults to 'linear' for numeric values, otherwise 'ffill'.
resample_method (str):
Method used to aggregate values within a resampling interval.
- "first": Use the first value in each interval.
- "mean": Compute the arithmetic mean of all samples in each interval.
- "interval_mean": Compute the time-weighted mean assuming each
value remains valid until the next timestamp (piecewise-constant
signal).
dropna: (bool, optional): Whether to drop NAN/ None values before processing.
Defaults to True.
boundary (Literal["strict", "context"]):
"strict" → only values inside [start, end)
"context" → include one value before and after for proper resampling
align_to_interval (bool): When True, snap the resample origin to the nearest
UTC epoch-aligned boundary of ``interval`` before resampling. This ensures
that bucket timestamps always fall on wall-clock-round times regardless of
when ``start_datetime`` falls:
- 15-minute interval → buckets on :00, :15, :30, :45
- 1-hour interval → buckets on the hour
When False (default), the origin is ``query_start`` (or ``"start_day"`` when
no start is given), preserving the existing behaviour where buckets are
aligned to the query window rather than the clock.
Set to True when storing compacted records back to the database so that the
resulting timestamps are predictable and human-readable. Leave False for
forecast or reporting queries where alignment to the exact query window is
more important than clock-round boundaries.
Returns:
pd.DataFrame: A DataFrame where each column represents a key's array with a common time index.
@@ -2503,7 +2622,15 @@ class DataContainer(SingletonMixin, DataABC):
for key in keys:
try:
array = await self.key_to_array(
key, start_datetime, end_datetime, interval, fill_method
key=key,
start_datetime=start_datetime,
end_datetime=end_datetime,
interval=interval,
fill_method=fill_method,
resample_method=resample_method,
dropna=dropna,
boundary=boundary,
align_to_interval=align_to_interval,
)
if len(array) != len(reference_index):
@@ -2520,6 +2647,43 @@ class DataContainer(SingletonMixin, DataABC):
return pd.DataFrame(data, index=reference_index)
async def key_delete_by_datetime(
self,
key: str,
start_datetime: Optional[DateTime] = None,
end_datetime: Optional[DateTime] = None,
) -> None:
"""Delete an attribute specified by `key` from records in the sequence within a given datetime range.
This method removes the attribute identified by `key` from records that have a `date_time` value falling
within the specified `start_datetime` (inclusive) and `end_datetime` (exclusive) range.
- If only `start_datetime` is specified, attributes will be removed from records from that date onward.
- If only `end_datetime` is specified, attributes will be removed from records up to that date.
- If neither `start_datetime` nor `end_datetime` is given, the attribute will be removed from all records.
Args:
key (str): The attribute name to delete from each record.
start_datetime (datetime, optional): The start datetime to begin attribute deletion (inclusive).
end_datetime (datetime, optional): The end datetime to stop attribute deletion (exclusive).
Raises:
KeyError: If `key` is not a valid attribute of the records.
"""
key_error = True
for provider in self.enabled_providers:
try:
await provider.key_delete_by_datetime(
key=key, start_datetime=start_datetime, end_datetime=end_datetime
)
key_error = False
except KeyError:
key_error = True
continue
if key_error:
raise KeyError(f"key `{key}` is not in predictions")
def provider_by_id(self, provider_id: str) -> DataProvider:
"""Retrieves a data provider by its unique identifier.
+8 -3
View File
@@ -17,7 +17,6 @@ from typing import (
Generic,
Iterable,
Iterator,
Literal,
Optional,
Protocol,
Self,
@@ -34,6 +33,11 @@ from akkudoktoreos.core.coreabc import (
DatabaseMixin,
SingletonMixin,
)
from akkudoktoreos.core.types import (
BoundaryMode,
FillMethod,
ResampleMethod,
)
from akkudoktoreos.utils.datetimeutil import (
DateTime,
Duration,
@@ -543,9 +547,10 @@ class DatabaseRecordProtocolMixin(
start_datetime: Optional[DateTime] = None,
end_datetime: Optional[DateTime] = None,
interval: Optional[Duration] = None,
fill_method: Optional[str] = None,
fill_method: Optional[FillMethod] = None,
resample_method: ResampleMethod = "mean",
dropna: Optional[bool] = True,
boundary: Literal["strict", "context"] = "context",
boundary: BoundaryMode = "context",
align_to_interval: bool = False,
) -> NDArray[Shape["*"], Any]: ...
+59
View File
@@ -0,0 +1,59 @@
"""Common type aliases used throughout AkkudoktorEOS.
This module centralizes reusable type definitions shared across multiple
packages. Defining common types here avoids duplication of complex type
annotations (such as Literal aliases), ensures consistent typing across the
code base, and helps prevent circular import dependencies between modules.
The aliases defined in this module describe common concepts and API contracts,
including resampling methods, interpolation and fill methods, boundary
handling, and other shared parameter types.
Guidelines:
- Import shared type aliases from this module instead of redefining them.
- Keep this module lightweight and free of runtime dependencies wherever
possible.
- Only define reusable types here. Implementation-specific types should
remain in the modules where they are used.
The contents of this module are intended for static type checking and
documentation and have no significant runtime behavior.
"""
from typing import Literal, TypeAlias
FillMethod: TypeAlias = Literal[
"linear",
"time",
"ffill",
"bfill",
]
"""Method used to fill missing values before or after resampling.
- "linear": Linear interpolation.
- "time": Time-based interpolation.
- "ffill": Forward-fill using the previous value.
- "bfill": Backward-fill using the next value.
"""
ResampleMethod: TypeAlias = Literal[
"first",
"mean",
"interval_mean",
]
"""Method used to aggregate multiple samples within a resampling interval.
- "first": Use the first sample.
- "mean": Arithmetic mean of the samples.
- "interval_mean": Time-weighted mean assuming piecewise-constant values.
"""
BoundaryMode: TypeAlias = Literal[
"strict",
"context",
]
"""Controls whether resampling includes context outside the requested time range.
- "strict": Use only data inside the requested interval.
- "context": Include one sample before and after for correct interpolation/resampling.
"""
@@ -18,6 +18,9 @@ from akkudoktoreos.core.emplan import (
FRBCInstruction,
)
from akkudoktoreos.core.pydantic import PydanticDateTimeDataFrame
from akkudoktoreos.core.types import (
FillMethod,
)
from akkudoktoreos.devices.devicesabc import (
ApplianceOperationMode,
BatteryOperationMode,
@@ -673,7 +676,7 @@ class GeneticSolution(ConfigMixin, GeneticParametersBaseModel):
)
pred = get_prediction()
for pred_key, pred_fill_method, pred_solution_key, pred_solution_factor in [
prediction_specs: list[tuple[str, FillMethod, str, float]] = [
(
"pvforecast_ac_power",
"linear",
@@ -722,7 +725,9 @@ class GeneticSolution(ConfigMixin, GeneticParametersBaseModel):
"loadakkudoktor_mean_energy_wh",
power_to_energy_per_interval_factor,
),
]:
]
for pred_key, pred_fill_method, pred_solution_key, pred_solution_factor in prediction_specs:
if pred_key in pred.record_keys:
array = await pred.key_to_array(
key=pred_key,
@@ -18,6 +18,9 @@ from akkudoktoreos.core.emplan import (
FRBCInstruction,
)
from akkudoktoreos.core.pydantic import PydanticDateTimeDataFrame
from akkudoktoreos.core.types import (
FillMethod,
)
from akkudoktoreos.devices.devicesabc import (
ApplianceOperationMode,
BatteryOperationMode,
@@ -675,7 +678,7 @@ class Genetic0Solution(ConfigMixin, Genetic0ParametersBaseModel):
)
pred = get_prediction()
for pred_key, pred_fill_method, pred_solution_key, pred_solution_factor in [
prediction_specs: list[tuple[str, FillMethod, str, float]] = [
(
"pvforecast_ac_power",
"linear",
@@ -724,7 +727,9 @@ class Genetic0Solution(ConfigMixin, Genetic0ParametersBaseModel):
"loadakkudoktor_mean_energy_wh",
power_to_energy_per_interval_factor,
),
]:
]
for pred_key, pred_fill_method, pred_solution_key, pred_solution_factor in prediction_specs:
if pred_key in pred.record_keys:
array = await pred.key_to_array(
key=pred_key,
@@ -77,6 +77,7 @@ Methods:
"""
import random
from typing import Any, List, Optional, Union
import requests
@@ -230,7 +231,9 @@ class PVForecastAkkudoktor(PVForecastProvider):
"""
base_url = "https://api.akkudoktor.net/forecast"
query_params: dict[str, Any] = {
"lat": self.config.general.latitude,
# Randomize lat a little bit to circumvent caching by api.akkudoktor.
# Caching used to provide very old data
"lat": round(float(self.config.general.latitude) + random.uniform(0.000, 0.001), 6), # noqa: S311
"lon": self.config.general.longitude,
}
@@ -332,11 +335,22 @@ class PVForecastAkkudoktor(PVForecastProvider):
"""
akkudoktor_data: Optional[AkkudoktorForecast] = None
# Hopefully skip internediate caches
headers = {
"Accept": "application/json",
"Cache-Control": "no-cache",
"Pragma": "no-cache",
}
for plane in self.config.pvforecast.planes:
plane_url = self._url(plane)
response = requests.get(plane_url, timeout=10)
response = requests.get(plane_url, headers=headers, timeout=10)
logger.info("[PVForecastAkkudoktor] URL: {}", response.request.url)
logger.info("[PVForecastAkkudoktor] Headers: {}", response.request.headers)
logger.info("[PVForecastAkkudoktor] Status: {}", response.status_code)
logger.info("[PVForecastAkkudoktor] Response headers: {}", response.headers)
response.raise_for_status() # Raise an error for bad responses
logger.debug(f"Response from {plane_url}: {response}")
plane_data = self._validate_data(response.content)
if akkudoktor_data is None:
@@ -369,6 +383,8 @@ class PVForecastAkkudoktor(PVForecastProvider):
raise ValueError(error_msg)
# Get Akkudoktor PV Forecast data for the given configuration.
if force_update:
logger.info("[PVForecastAkkudoktor] force update.")
akkudoktor_data = self._request_forecast(force_update=force_update) # type: ignore
# Timezone of the PV system
@@ -377,16 +393,35 @@ class PVForecastAkkudoktor(PVForecastProvider):
logger.error(f"Akkudoktor schema change: {error_msg}")
raise ValueError(error_msg)
prediction_horizon = self.ems_start_datetime.start_of("day")
prediction_horizon = prediction_horizon.add(hours=self.config.prediction.hours)
# Assumption that all lists are the same length and are ordered chronologically
# in ascending order and have the same timestamps.
if len(akkudoktor_data.values[0]) < self.config.prediction.hours:
# Expect one value set per prediction hour
error_msg = (
f"The forecast must cover at least {self.config.prediction.hours} hours, "
f"but only {len(akkudoktor_data.values[0])} data sets are given in forecast data."
)
logger.error(f"Akkudoktor schema change: {error_msg}")
raise ValueError(error_msg)
for plane_idx, plane_values in enumerate(akkudoktor_data.values):
last_datetime = plane_values[-1].datetime
dt = to_datetime(last_datetime, in_timezone=self.config.general.timezone)
prediction_horizon = self.ems_start_datetime.start_of("day")
prediction_horizon = prediction_horizon.add(hours=self.config.prediction.hours - 1)
if compare_datetimes(dt, prediction_horizon).lt:
error_msg = (
f"The forecast must cover at least the `{prediction_horizon}` prediction horizon, "
f"but only data up to `{dt}` is given in "
f"forecast data for plane `{plane_idx}`."
)
logger.error(f"Akkudoktor schema change: {error_msg}")
raise ValueError(error_msg)
if len(plane_values) < self.config.prediction.hours:
# Expect one value set per prediction hour
error_msg = (
f"The forecast must cover at least `{self.config.prediction.hours}` hours, "
f"but only `{len(plane_values)}` data sets are given in "
f"forecast data for plane `{plane_idx}`."
)
logger.error(f"Akkudoktor schema change: {error_msg}")
raise ValueError(error_msg)
if not self.ems_start_datetime:
raise ValueError(f"Start DateTime not set: {self.ems_start_datetime}")
@@ -416,7 +451,8 @@ class PVForecastAkkudoktor(PVForecastProvider):
raise ValueError(
f"The forecast must cover at least {self.config.prediction.hours} hours, "
f"but only {len(self)} hours starting from {self.ems_start_datetime} "
f"were predicted."
f"were predicted.\n"
f"{akkudoktor_data}"
)
def report_ac_power_and_measurement(self) -> str:
+246 -26
View File
@@ -46,6 +46,11 @@ from akkudoktoreos.core.pydantic import (
PydanticDateTimeDataFrame,
PydanticDateTimeSeries,
)
from akkudoktoreos.core.types import (
BoundaryMode,
FillMethod,
ResampleMethod,
)
from akkudoktoreos.core.version import __version__
from akkudoktoreos.devices.devices import ResourceKey
from akkudoktoreos.optimization.genetic0.genetic0params import (
@@ -973,16 +978,77 @@ async def fastapi_prediction_dataframe_get(
Optional[str],
Query(description="Time duration for each interval. Defaults to 1 hour."),
] = None,
fill_method: Annotated[
Optional[FillMethod],
Query(description="Method to handle missing values during resampling."),
] = None,
resample_method: Annotated[
ResampleMethod,
Query(description="Method used to aggregate values within a resampling interval."),
] = "mean",
dropna: Annotated[
Optional[bool],
Query(description="Drop NAN/ None values before processing."),
] = None,
boundary: Annotated[
BoundaryMode,
Query(description="Resampling boundary mode."),
] = "context",
align_to_interval: Annotated[
bool,
Query(
description="Snap resample origin to the nearest UTC epoch-aligned boundary of interval."
),
] = False,
) -> PydanticDateTimeDataFrame:
"""Get prediction for given key within given date range as series.
"""Get prediction for given keys within given date range as dataframe.
Args:
key (str): Prediction key
key (list[str]): Prediction keys
start_datetime (Optional[str]): Starting datetime (inclusive).
Defaults to start datetime of latest prediction.
end_datetime (Optional[str]: Ending datetime (exclusive).
Defaults to end datetime of latest prediction.
interval (Optional[str]): Time duration for each interval.
Defaults to 1 hour.
fill_method (str): Method to handle missing values during resampling.
Defaults to end datetime of latest prediction.
- 'linear': Linearly interpolate missing values (for numeric data only).
- 'time': Interpolate missing values (for numeric data only).
- 'ffill': Forward fill missing values.
- 'bfill': Backward fill missing values.
- Defaults to 'linear' for numeric values, otherwise 'ffill'.
resample_method (str):
Method used to aggregate values within a resampling interval.
- "first": Use the first value in each interval.
- "mean": Compute the arithmetic mean of all samples in each interval.
- "interval_mean": Compute the time-weighted mean assuming each
value remains valid until the next timestamp (piecewise-constant
signal).
dropna: (bool, optional): Whether to drop NAN/ None values before processing.
Defaults to True.
boundary (Literal["strict", "context"]): resampling boundary
"strict" → only values inside [start, end)
"context" → include one value before and after for proper resampling
align_to_interval (bool): When True, snap the resample origin to the nearest
UTC epoch-aligned boundary of ``interval`` before resampling. This ensures
that bucket timestamps always fall on wall-clock-round times regardless of
when ``start_datetime`` falls:
- 15-minute interval → buckets on :00, :15, :30, :45
- 1-hour interval → buckets on the hour
When False (default), the origin is ``query_start`` (or ``"start_day"`` when
no start is given), preserving the existing behaviour where buckets are
aligned to the query window rather than the clock.
Set to True when storing compacted records back to the database so that the
resulting timestamps are predictable and human-readable. Leave False for
forecast or reporting queries where alignment to the exact query window is
more important than clock-round boundaries.
"""
for key in keys:
if key not in get_prediction().record_keys:
@@ -995,10 +1061,25 @@ async def fastapi_prediction_dataframe_get(
end_datetime = get_prediction().end_datetime
else:
end_datetime = to_datetime(end_datetime)
df = await get_prediction().keys_to_dataframe(
keys=keys, start_datetime=start_datetime, end_datetime=end_datetime, interval=interval
)
return PydanticDateTimeDataFrame.from_dataframe(df, tz=get_config().general.timezone)
try:
prediction_df = await get_prediction().keys_to_dataframe(
keys=keys,
start_datetime=start_datetime,
end_datetime=end_datetime,
interval=interval,
fill_method=fill_method,
resample_method=resample_method,
dropna=dropna,
boundary=boundary,
align_to_interval=align_to_interval,
)
except Exception as e:
raise HTTPException(
status_code=400, detail=f"Error on prediction dataframe for '{key}': {e}"
)
return PydanticDateTimeDataFrame.from_dataframe(prediction_df, tz=get_config().general.timezone)
@app.get("/v1/prediction/list", tags=["prediction"])
@@ -1016,6 +1097,28 @@ async def fastapi_prediction_list_get(
Optional[str],
Query(description="Time duration for each interval. Defaults to 1 hour."),
] = None,
fill_method: Annotated[
Optional[FillMethod],
Query(description="Method to handle missing values during resampling."),
] = None,
resample_method: Annotated[
ResampleMethod,
Query(description="Method used to aggregate values within a resampling interval."),
] = "mean",
dropna: Annotated[
Optional[bool],
Query(description="Drop NAN/ None values before processing."),
] = None,
boundary: Annotated[
BoundaryMode,
Query(description="Resampling boundary mode."),
] = "context",
align_to_interval: Annotated[
bool,
Query(
description="Snap resample origin to the nearest UTC epoch-aligned boundary of interval."
),
] = False,
) -> List[Any]:
"""Get prediction for given key within given date range as value list.
@@ -1027,6 +1130,44 @@ async def fastapi_prediction_list_get(
Defaults to end datetime of latest prediction.
interval (Optional[str]): Time duration for each interval.
Defaults to 1 hour.
fill_method (str): Method to handle missing values during resampling.
- 'linear': Linearly interpolate missing values (for numeric data only).
- 'time': Interpolate missing values (for numeric data only).
- 'ffill': Forward fill missing values.
- 'bfill': Backward fill missing values.
- Defaults to 'linear' for numeric values, otherwise 'ffill'.
resample_method (str):
Method used to aggregate values within a resampling interval.
- "first": Use the first value in each interval.
- "mean": Compute the arithmetic mean of all samples in each interval.
- "interval_mean": Compute the time-weighted mean assuming each
value remains valid until the next timestamp (piecewise-constant
signal).
dropna: (bool, optional): Whether to drop NAN/ None values before processing.
Defaults to True.
boundary (Literal["strict", "context"]): resampling boundary
"strict" → only values inside [start, end)
"context" → include one value before and after for proper resampling
align_to_interval (bool): When True, snap the resample origin to the nearest
UTC epoch-aligned boundary of ``interval`` before resampling. This ensures
that bucket timestamps always fall on wall-clock-round times regardless of
when ``start_datetime`` falls:
- 15-minute interval → buckets on :00, :15, :30, :45
- 1-hour interval → buckets on the hour
When False (default), the origin is ``query_start`` (or ``"start_day"`` when
no start is given), preserving the existing behaviour where buckets are
aligned to the query window rather than the clock.
Set to True when storing compacted records back to the database so that the
resulting timestamps are predictable and human-readable. Leave False for
forecast or reporting queries where alignment to the exact query window is
more important than clock-round boundaries.
"""
if key not in get_prediction().record_keys:
raise HTTPException(status_code=404, detail=f"Key '{key}' is not available.")
@@ -1042,13 +1183,23 @@ async def fastapi_prediction_list_get(
interval = to_duration("1 hour")
else:
interval = to_duration(interval)
prediction_array = await get_prediction().key_to_array(
key=key,
start_datetime=start_datetime,
end_datetime=end_datetime,
interval=interval,
)
prediction_list = prediction_array.tolist()
try:
prediction_array = await get_prediction().key_to_array(
key=key,
start_datetime=start_datetime,
end_datetime=end_datetime,
interval=interval,
fill_method=fill_method,
resample_method=resample_method,
dropna=dropna,
boundary=boundary,
align_to_interval=align_to_interval,
)
prediction_list = prediction_array.tolist()
except Exception as e:
raise HTTPException(status_code=400, detail=f"Error on prediction list for '{key}': {e}")
return prediction_list
@@ -1150,6 +1301,53 @@ async def fastapi_prediction_update_provider(
return Response()
@app.delete("/v1/prediction/range", tags=["prediction"])
async def fastapi_prediction_range_delete(
key: Annotated[str, Query(description="Prediction key.")],
start_datetime: Annotated[Optional[str], Query(description="Start datetime.")] = None,
end_datetime: Annotated[Optional[str], Query(description="End datetime.")] = None,
) -> PydanticDateTimeSeries:
"""Delete prediction values for a key within a datetime range."""
try:
if key not in get_prediction().record_keys:
raise HTTPException(
status_code=status.HTTP_404_NOT_FOUND,
detail=f"Key '{key}' not found in predictions",
)
try:
start_dt = to_datetime(start_datetime) if start_datetime else None
end_dt = to_datetime(end_datetime) if end_datetime else None
except Exception as e:
raise HTTPException(
status_code=status.HTTP_400_BAD_REQUEST,
detail=f"Invalid datetime: {e}",
)
try:
await get_prediction().key_delete_by_datetime(
key=key,
start_datetime=start_dt,
end_datetime=end_dt,
)
except KeyError:
# No data for key in predictions
pass
pdseries = await get_prediction().key_to_series(key=key)
return PydanticDateTimeSeries.from_series(pdseries)
except HTTPException:
raise
except Exception as e:
trace = "".join(traceback.TracebackException.from_exception(e).format())
logger.exception(f"Unexpected error deleting prediction range: {key}")
raise HTTPException(
status_code=status.HTTP_500_INTERNAL_SERVER_ERROR,
detail=f"Internal server error:\n{e}\n{trace}",
)
@app.get("/v1/energy-management/optimization/solution", tags=["energy-management"])
def fastapi_energy_management_optimization_solution_get() -> OptimizationSolution:
"""Get the latest solution of the optimization."""
@@ -1210,9 +1408,9 @@ def fastapi_energy_management_plan_get() -> EnergyManagementPlan:
async def fastapi_strompreis() -> list[float]:
"""Deprecated: Electricity Market Price Prediction per Wh [amount/Wh].
Electricity prices start at 00.00.00 today and are provided for 48 hours.
If no prices are available the missing ones at the start of the series are
filled with the first available price.
Electricity prices start at 00.00.00 today and are provided for 48 hours
in 1-hour intervals. If no prices are available the missing ones at the
start of the series are filled with the first available price.
Note:
Electricity price charges are added.
@@ -1251,7 +1449,9 @@ async def fastapi_strompreis() -> list[float]:
key="elecprice_marketprice_wh",
start_datetime=start_datetime,
end_datetime=end_datetime,
interval=to_duration("1 hour"),
fill_method="ffill",
resample_method="interval_mean",
)
elecprice_list = elecprice_array.tolist()
except Exception as e:
@@ -1275,9 +1475,9 @@ async def fastapi_gesamtlast(request: GesamtlastRequest) -> list[float]:
Endpoint to handle total load prediction adjusted by latest measured data.
Total load prediction starts at 00.00.00 today and is provided for 48 hours.
If no prediction values are available the missing ones at the start of the series are
filled with the first available prediction value.
Total load prediction starts at 00.00.00 today and is provided for 48 hours
in 1-hour intervals. If no prediction values are available the missing ones
at the start of the series are filled with the first available prediction value.
Note:
Use '/v1/prediction/list?key=loadforecast_power_w' instead.
@@ -1355,6 +1555,11 @@ async def fastapi_gesamtlast(request: GesamtlastRequest) -> list[float]:
key="loadforecast_power_w",
start_datetime=start_datetime,
end_datetime=end_datetime,
interval=to_duration("1 hour"),
fill_method="ffill",
resample_method="interval_mean",
dropna=True,
boundary="context",
)
prediction_list = prediction_array.tolist()
except Exception as e:
@@ -1372,9 +1577,9 @@ async def fastapi_gesamtlast_simple(year_energy: float) -> list[float]:
Endpoint to handle total load prediction.
Total load prediction starts at 00.00.00 today and is provided for 48 hours.
If no prediction values are available the missing ones at the start of the series are
filled with the first available prediction value.
Total load prediction starts at 00.00.00 today and is provided for 48 hours
in 1-hour intervals. If no prediction values are available the missing ones
at the start of the series are filled with the first available prediction value.
Args:
year_energy (float): Yearly energy consumption in Wh.
@@ -1415,6 +1620,11 @@ async def fastapi_gesamtlast_simple(year_energy: float) -> list[float]:
key="loadforecast_power_w",
start_datetime=start_datetime,
end_datetime=end_datetime,
interval=to_duration("1 hour"),
fill_method="ffill",
resample_method="interval_mean",
dropna=True,
boundary="context",
)
prediction_list = prediction_array.tolist()
except Exception as e:
@@ -1437,9 +1647,9 @@ async def fastapi_pvforecast() -> ForecastResponse:
Endpoint to handle PV forecast prediction.
PVForecast starts at 00.00.00 today and is provided for 48 hours.
If no forecast values are available the missing ones at the start of the series are
filled with the first available forecast value.
PVForecast starts at 00.00.00 today and is provided for 48 hours
in 1-hour intervals. If no forecast values are available the missing ones
at the start of the series are filled with the first available forecast value.
Note:
Set PVForecastAkkudoktor as provider, then update data with
@@ -1471,12 +1681,22 @@ async def fastapi_pvforecast() -> ForecastResponse:
key="pvforecast_ac_power",
start_datetime=start_datetime,
end_datetime=end_datetime,
interval=to_duration("1 hour"),
fill_method="ffill",
resample_method="interval_mean",
dropna=True,
boundary="context",
)
ac_power_list = ac_power_array.tolist()
temp_air_array = await get_prediction().key_to_array(
key="pvforecastakkudoktor_temp_air",
start_datetime=start_datetime,
end_datetime=end_datetime,
interval=to_duration("1 hour"),
fill_method="ffill",
resample_method="interval_mean",
dropna=True,
boundary="context",
)
temp_air_list = temp_air_array.tolist()
except Exception as e:
+43 -2
View File
@@ -631,8 +631,8 @@ class TestCacheFileDecorators:
cache_file.seek(0) # Move to the start of the file
assert cache_file.read() == result1
def test_cache_in_file_handles_ttl(self, cache_file_store):
"""Test that the cache_infile decorator handles the with_ttl parameter."""
def test_cache_in_file_handles_ttl_in_call(self, cache_file_store):
"""Test that the cache_infile decorator handles the with_ttl parameter in decorator."""
# Define a simple function to decorate
@cache_in_file(mode="w+")
@@ -672,6 +672,47 @@ class TestCacheFileDecorators:
result = my_function(with_ttl="1 second") # type: ignore[call-arg]
assert result == result1
def test_cache_in_file_handles_ttl_in_decorator(self, cache_file_store):
"""Test that the cache_infile decorator handles the with_ttl parameter in call."""
# Define a simple function to decorate
@cache_in_file(mode="w+", with_ttl="1 second")
def my_function():
return "New result"
# Call the decorated function
result1 = my_function()
assert result1 == "New result"
assert len(cache_file_store._store) == 1
key = list(cache_file_store._store.keys())[0]
# Assert result was written to cache file
key = next(iter(cache_file_store._store))
cache_file = cache_file_store._store[key].cache_file
assert cache_file is not None
cache_file.seek(0) # Move to the start of the file
assert cache_file.read() == result1
# Modify cache file
result2 = "Cached result"
cache_file.seek(0)
cache_file.write(result2)
cache_file.seek(0) # Move to the start of the file
assert cache_file.read() == result2
# Call the decorated function again
result = my_function(with_ttl="1 second") # type: ignore[call-arg]
cache_file.seek(0) # Move to the start of the file
assert cache_file.read() == result2
assert result == result2
# Wait one second to let the cache time out
sleep(2)
# Call again - cache should be timed out
result = my_function(with_ttl="1 second") # type: ignore[call-arg]
assert result == result1
def test_cache_in_file_handles_bytes_return(self, cache_file_store):
"""Test that the cache_infile decorator handles bytes returned from the function."""
# Clear store to assure it is empty
+93
View File
@@ -427,6 +427,34 @@ class TestDataSequence:
fill_method="invalid",
)
async def test_key_to_array_resample_first(self, sequence):
"""Test that resample_method='first' returns the first sample in each interval."""
interval = to_duration("1 hour")
for minute, value in (
(0, 1.0),
(15, 2.0),
(30, 3.0),
(45, 4.0),
):
await sequence.insert_by_datetime(
self.create_test_record(
pendulum.datetime(2023, 11, 6, 0, minute),
value,
)
)
array = await sequence.key_to_array(
key="data_value",
start_datetime=pendulum.datetime(2023, 11, 6, 0),
end_datetime=pendulum.datetime(2023, 11, 6, 1),
interval=interval,
resample_method="first",
)
assert len(array) == 1
assert array[0] == 1.0
async def test_key_to_array_resample_mean(self, sequence):
"""Test that numeric resampling uses mean when multiple values fall into one interval."""
interval = to_duration("1 hour")
@@ -447,6 +475,7 @@ class TestDataSequence:
start_datetime=pendulum.datetime(2023, 11, 6, 0),
end_datetime=pendulum.datetime(2023, 11, 6, 1),
interval=interval,
resample_method="mean",
)
assert isinstance(array, np.ndarray)
@@ -454,6 +483,70 @@ class TestDataSequence:
# The first interval mean = (1+2+3+4)/4 = 2.5
assert array[0] == pytest.approx(2.5)
async def test_key_to_array_resample_interval_mean(self, sequence):
"""Test that interval_mean computes a time-weighted mean."""
interval = to_duration("1 hour")
await sequence.insert_by_datetime(
self.create_test_record(
pendulum.datetime(2023, 11, 6, 0, 0),
10.0,
)
)
await sequence.insert_by_datetime(
self.create_test_record(
pendulum.datetime(2023, 11, 6, 0, 45),
20.0,
)
)
await sequence.insert_by_datetime(
self.create_test_record(
pendulum.datetime(2023, 11, 6, 1, 0),
20.0,
)
)
array = await sequence.key_to_array(
key="data_value",
start_datetime=pendulum.datetime(2023, 11, 6, 0),
end_datetime=pendulum.datetime(2023, 11, 6, 1),
interval=interval,
resample_method="interval_mean",
fill_method="none",
)
assert len(array) == 1
# 10 for 45 min, 20 for 15 min
expected = (10 * 45 + 20 * 15) / 60
assert array[0] == pytest.approx(expected)
async def test_key_to_array_invalid_resample_method(self, sequence):
"""Test invalid resample_method raises an error."""
interval = to_duration("1 hour")
await sequence.insert_by_datetime(
self.create_test_record(
pendulum.datetime(2023, 11, 6),
1.0,
)
)
with pytest.raises(
ValueError,
match="Unsupported resample method: invalid",
):
await sequence.key_to_array(
key="data_value",
start_datetime=pendulum.datetime(2023, 11, 6),
end_datetime=pendulum.datetime(2023, 11, 6, 1),
interval=interval,
resample_method="invalid",
)
# ------------------------------------------------------------------
# key_to_array — align_to_interval parameter
# ------------------------------------------------------------------
+8 -2
View File
@@ -27,6 +27,11 @@ from akkudoktoreos.core.databaseabc import (
DatabaseTimestamp,
_DatabaseTimestampUnbound,
)
from akkudoktoreos.core.types import (
BoundaryMode,
FillMethod,
ResampleMethod,
)
from akkudoktoreos.utils.datetimeutil import (
DateTime,
Duration,
@@ -258,9 +263,10 @@ class SampleSequence(DatabaseRecordProtocolMixin[SampleRecord]):
start_datetime: Optional[DateTime] = None,
end_datetime: Optional[DateTime] = None,
interval: Optional[Duration] = None,
fill_method: Optional[str] = None,
fill_method: Optional[FillMethod] = None,
resample_method: ResampleMethod = "mean",
dropna: Optional[bool] = True,
boundary: Literal["strict", "context"] = "context",
boundary: BoundaryMode = "context",
align_to_interval: bool = False,
) -> NDArray[Shape["*"], Any]:
"""Minimal resampling stub sufficient for compaction tests."""
+73
View File
@@ -0,0 +1,73 @@
"""Test server endpoints directly calling the endpoint function (unit tests)."""
import numpy as np
import pytest
from akkudoktoreos.server.eos import fastapi_strompreis
from akkudoktoreos.utils.datetimeutil import to_duration
class _FakeConfig:
def __init__(self):
self.settings = None
def merge_settings(self, settings):
self.settings = settings
class _FakeEms:
def __init__(self):
self.run_kwargs = None
async def run(self, **kwargs):
self.run_kwargs = kwargs
class TestServerEndpointDirect:
@pytest.mark.asyncio
async def test_fastapi_strompreis(self, monkeypatch):
class _FakePrediction:
def __init__(self):
self.key_to_array_kwargs = None
async def key_to_array(self, **kwargs):
self.key_to_array_kwargs = kwargs
# two hours of hourly values
return np.array([4.0, 16.0])
config = _FakeConfig()
ems = _FakeEms()
prediction = _FakePrediction()
monkeypatch.setattr(
"akkudoktoreos.server.eos.get_config",
lambda: config,
)
monkeypatch.setattr(
"akkudoktoreos.server.eos.get_ems",
lambda: ems,
)
monkeypatch.setattr(
"akkudoktoreos.server.eos.get_prediction",
lambda: prediction,
)
result = await fastapi_strompreis()
assert ems.run_kwargs is not None
assert prediction.key_to_array_kwargs is not None
assert result == [4.0, 16.0]
assert ems.run_kwargs == {
"mode": ems.run_kwargs["mode"], # see below
"force_update": True,
}
kwargs = prediction.key_to_array_kwargs
assert kwargs["key"] == "elecprice_marketprice_wh"
assert kwargs["interval"] == to_duration("1 hour")
assert kwargs["fill_method"] == "ffill"
assert kwargs["resample_method"] == "interval_mean"
+46
View File
@@ -786,3 +786,49 @@ class TestSystem:
assert result.status_code == HTTPStatus.OK
cache = result.json()
assert cache == {}
def test_deprecated_strompreis(self, server_setup_for_class, is_system_test):
"""Test deprecated /strompreis endpoint.
Deprecated /strompreis aggregates 15-minute spot prices to hourly means.
"""
server = server_setup_for_class["server"]
eos_dir = server_setup_for_class["eos_dir"]
start_datetime = to_datetime().start_of("day")
end_datetime = start_datetime.add(days=2)
# Reset electricity market price
result = requests.delete(
f"{server}/v1/prediction/range",
params = {
"key": "elecprice_marketprice_wh",
}
)
assert result.status_code in (HTTPStatus.OK, HTTPStatus.NOT_FOUND)
if not is_system_test:
return
# Call /strompreis
result = requests.get(f"{server}/strompreis")
assert result.status_code == HTTPStatus.OK
strompreis_data = result.json()
assert len(strompreis_data) == 48
# Get the same data by v1 interface
result = requests.get(
f"{server}/v1/prediction/list",
params = {
"key": "elecprice_marketprice_wh",
"start_datetime": to_datetime(start_datetime, as_string=True),
"end_datetime": to_datetime(end_datetime, as_string=True),
"interval" : "1 hour",
"fill_method" : "ffill",
"resample_method" : "interval_mean",
}
)
assert result.status_code == HTTPStatus.OK
v1_data = result.json()
assert strompreis_data == pytest.approx(v1_data)