Mirrors/EOS
1
0
Fork 0
mirror of https://github.com/Akkudoktor-EOS/EOS.git synced 2026-08-09 10:26:11 +00:00
Code Issues Actions 1 Packages Projects Releases Wiki Activity

chore: adapt pdf visualization (#1205)

Browse Source 
Change PDF visualization to be created on demand and per optimization algorithm. The PDF
for the GENETIC0 optimization is provided by the /visualization_results.pdf endpoint.
There is no change in the interface.

By this the optimization algorithm is offloaded from the PDF generation which spares some
time.

To cope with several users may call the /visualization_results.pdf endpoint at the same
time the PDF is generated on the fly without any intermediate file taking the stored
GENETIC0 solution as an input. SVG picture generation is removed as this would again
create intermediate files. Chart pictures can easily be taken from the PDF.

To allow on demand creation of the optimization results visualization the optimisation
solution stored is extended by several new attributes. To keep the deprecated
/optimize endpoint compatible the optimization solution is stripped to the legacy
content before returned. Due to the extension of the solution the optimization tests were
adapted to cover the extended content.

The optimization tests are adapted to test the generated visualization report by
the pypdf reader. Pypdf is added to the development dependencies.

Besides the adaptation several fixes and improvements are added:

* feat: extend /v1/prediction/series endpoint by resampling and filling

  Add parameters for resampling and filling. Add the processing parameter
  to control wether raw data or resampled data shall be returned.

* feat: extend /v1/measurement/series endpoint by resampling and filling

  Add parameters for resampling and filling: Add the processing parameter
  to control wether raw data or resampled data shall be returned.

* feat: standardize and improve API error response

  Use FASTApi exception handlers to provide a standardized API exception handling.
  All exceptions are logged.

  Exception traces are only returned if the new logging configuration parameter
  logging.api_logging_level is set to "DEBUG" or "TRACE". Avoids unwanted leackage
  of server internals on exceptions.

* fix: align to intervall when resampling

  Ensure resampling is aligned to interval also when the buckets are shifted due to the
  align_to_intervall parameter is set.

* chore: make dropna mandatory and default to True

* chore: refactor key_to_xxx data management methods

  Make key_to_series the central method for data resampling and fill.
  Add a new key_to_raw_series to retrieve the data as it is stored
  (without resampling and filling).

  Users of key_to_series were mostly moved to key_to_raw_series as this resembles
  the former interface. Especially in predictions and tests this was done.

* chore: create test data sub-directory for each optimization algorithm

  To prevent cluttering the test data directory and ease test data management for
  optimization algorithms each algorithm got it's own sub-directory. The current
  test data was moved to these sub-directories.

* chore: update version

Signed-off-by: Bobby Noelte <b0661n0e17e@gmail.com>
This commit is contained in:
Bobby Noelte
2026-08-01 12:45:19 +02:00
committed by GitHub
parent 8344974c16
commit 1905682113
61 changed files with 29450 additions and 9083 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/_generated/configexample.md
+1
View File
@@ -162,6 +162,7 @@
}
},
"logging": {
"api_level": "TRACE",
"console_level": "TRACE",
"file_level": "TRACE"
},
docs/_generated/configlogging.md
+5 -2
View File
@@ -7,8 +7,9 @@
| Name | Environment Variable | Type | Read-Only | Default | Description |
| ---- | -------------------- | ---- | --------- | ------- | ----------- |
| console_level | `EOS_LOGGING__CONSOLE_LEVEL` | `str | None` | `rw` | `None` | Logging level when logging to console. |
| file_level | `EOS_LOGGING__FILE_LEVEL` | `str | None` | `rw` | `None` | Logging level when logging to file. |
| api_level | `EOS_LOGGING__API_LEVEL` | `str | None` | `rw` | `None` | Logging level for API response. |
| console_level | `EOS_LOGGING__CONSOLE_LEVEL` | `str | None` | `rw` | `None` | Logging level for logging to console. |
| file_level | `EOS_LOGGING__FILE_LEVEL` | `str | None` | `rw` | `None` | Logging level for logging to file. |
| file_path | | `pathlib.Path | None` | `ro` | `N/A` | Computed log file path based on data output path. |
:::
<!-- pyml enable line-length -->
@@ -21,6 +22,7 @@
```json
{
"logging": {
"api_level": "TRACE",
"console_level": "TRACE",
"file_level": "TRACE"
}
@@ -36,6 +38,7 @@
```json
{
"logging": {
"api_level": "TRACE",
"console_level": "TRACE",
"file_level": "TRACE",
"file_path": "/home/user/.local/share/net.akkudoktor.eos/output/eos.log"
docs/_generated/openapi.md
+170 -5
View File
@@ -1,6 +1,6 @@
# Akkudoktor-EOS
**Version**: `v0.3.0.dev2607301181843779`
**Version**: `v0.3.0.dev2608010961051894`
<!-- pyml disable line-length -->
**Description**: This project provides a comprehensive solution for simulating and optimizing an energy system based on renewable energy sources. With a focus on photovoltaic (PV) systems, battery storage (batteries), load management (consumer requirements), heat pumps, electric vehicles, and consideration of electricity price data, this system enables forecasting and optimization of energy flow and costs over a specified period.
@@ -143,7 +143,7 @@ Note:
**Request Body**:
- `application/json`: {
"$ref": "#/components/schemas/Genetic0OptimizationParameters"
"$ref": "#/components/schemas/Genetic0OptimizationParameters-Input"
}
**Responses**:
@@ -948,6 +948,14 @@ Merge the measurement data given as dataframe into EOS measurements.
Fastapi Measurement Keys Get
<!-- pyml disable line-length -->
```python
"""
Get a list of available measurement keys.
"""
```
<!-- pyml enable line-length -->
**Responses**:
- **200**: Successful Response
@@ -997,7 +1005,73 @@ Fastapi Measurement Series Get
<!-- pyml disable line-length -->
```python
"""
Get the measurements of given key as series.
Get measurements for given key within given date range as series.
Args:
key (str): Measurement key
start_datetime (Optional[str]): Starting datetime (inclusive).
Defaults to datetime of first measurement.
end_datetime (Optional[str]: Ending datetime (exclusive).
Defaults to datetime after latest measurement.
interval (Optional[str]): Time duration for each interval.
Defaults to 1 hour.
fill_method (str): Method to handle missing values during resampling.
- 'linear': Linearly interpolate missing values (for numeric data only).
- 'time': Interpolate missing values (for numeric data only).
- 'ffill': Forward fill missing values.
- 'bfill': Backward fill missing values.
- Defaults to 'linear' for numeric values, otherwise 'ffill'.
resample_method (str):
Method used to aggregate values within a resampling interval.
- "first": Use the first value in each interval.
- "mean": Compute the arithmetic mean of all samples in each interval.
- "interval_mean": Compute the time-weighted mean assuming each
value remains valid until the next timestamp (piecewise-constant
signal).
dropna: (bool): Whether to drop NAN/ None values before processing.
Defaults to True.
boundary (Literal["strict", "context"]): resampling boundary
"strict" → only values inside [start, end)
"context" → include one value before and after for proper resampling
align_to_interval (bool): When True, snap the resample origin to the nearest
UTC epoch-aligned boundary of ``interval`` before resampling. This ensures
that bucket timestamps always fall on wall-clock-round times regardless of
when ``start_datetime`` falls:
- 15-minute interval → buckets on :00, :15, :30, :45
- 1-hour interval → buckets on the hour
When False (default), the origin is ``query_start`` (or ``"start_day"`` when
no start is given), preserving the existing behaviour where buckets are
aligned to the query window rather than the clock.
Set to True when storing compacted records back to the database so that the
resulting timestamps are predictable and human-readable. Leave False for
forecast or reporting queries where alignment to the exact query window is
more important than clock-round boundaries.
processing (SeriesProcessing):
Processing mode for the returned series.
- ``SeriesProcessing.RESAMPLED``: Return a processed series.
Measurements are first filtered by ``start_datetime``,
``end_datetime``, and ``dropna``, then resampled according to
``interval`` and ``resample_method``, and finally missing values
are filled using ``fill_method``.
- ``SeriesProcessing.RAW``: Return the original measurement series.
Measurements are filtered by ``start_datetime``,
``end_datetime``, and ``dropna`` only. No resampling or filling is
performed, and ``interval``, ``fill_method``,
``resample_method``, ``boundary``, and
``align_to_interval`` are ignored.
Defaults to ``SeriesProcessing.RAW``.
Returns:
Series
"""
```
<!-- pyml enable line-length -->
@@ -1006,6 +1080,24 @@ Get the measurements of given key as series.
- `key` (query, required): Measurement key.
- `start_datetime` (query, optional): Starting datetime (inclusive).
- `end_datetime` (query, optional): Ending datetime (exclusive).
- `interval` (query, optional): Time duration for each interval. Defaults to 1 hour.
- `fill_method` (query, optional): Method to handle missing values during resampling.
- `resample_method` (query, optional): Method used to aggregate values within a resampling interval.
- `dropna` (query, optional): Drop NAN/ None values before processing.
- `boundary` (query, optional): Resampling boundary mode.
- `align_to_interval` (query, optional): Snap resample origin to the nearest UTC epoch-aligned boundary of interval.
- `processing` (query, optional): Processing mode. 'raw' returns original measurement data without resampling or filling.
**Responses**:
- **200**: Successful Response
@@ -1118,7 +1210,7 @@ Args:
value remains valid until the next timestamp (piecewise-constant
signal).
dropna: (bool, optional): Whether to drop NAN/ None values before processing.
dropna: (bool): Whether to drop NAN/ None values before processing.
Defaults to True.
boundary (Literal["strict", "context"]): resampling boundary
"strict" → only values inside [start, end)
@@ -1288,7 +1380,7 @@ Args:
value remains valid until the next timestamp (piecewise-constant
signal).
dropna: (bool, optional): Whether to drop NAN/ None values before processing.
dropna: (bool): Whether to drop NAN/ None values before processing.
Defaults to True.
boundary (Literal["strict", "context"]): resampling boundary
"strict" → only values inside [start, end)
@@ -1423,6 +1515,65 @@ Args:
Defaults to start datetime of latest prediction.
end_datetime (Optional[str]: Ending datetime (exclusive).
Defaults to end datetime of latest prediction.
interval (Optional[str]): Time duration for each interval.
Defaults to 1 hour.
fill_method (str): Method to handle missing values during resampling.
- 'linear': Linearly interpolate missing values (for numeric data only).
- 'time': Interpolate missing values (for numeric data only).
- 'ffill': Forward fill missing values.
- 'bfill': Backward fill missing values.
- Defaults to 'linear' for numeric values, otherwise 'ffill'.
resample_method (str):
Method used to aggregate values within a resampling interval.
- "first": Use the first value in each interval.
- "mean": Compute the arithmetic mean of all samples in each interval.
- "interval_mean": Compute the time-weighted mean assuming each
value remains valid until the next timestamp (piecewise-constant
signal).
dropna: (bool): Whether to drop NAN/ None values before processing.
Defaults to True.
boundary (Literal["strict", "context"]): resampling boundary
"strict" → only values inside [start, end)
"context" → include one value before and after for proper resampling
align_to_interval (bool): When True, snap the resample origin to the nearest
UTC epoch-aligned boundary of ``interval`` before resampling. This ensures
that bucket timestamps always fall on wall-clock-round times regardless of
when ``start_datetime`` falls:
- 15-minute interval → buckets on :00, :15, :30, :45
- 1-hour interval → buckets on the hour
When False (default), the origin is ``query_start`` (or ``"start_day"`` when
no start is given), preserving the existing behaviour where buckets are
aligned to the query window rather than the clock.
Set to True when storing compacted records back to the database so that the
resulting timestamps are predictable and human-readable. Leave False for
forecast or reporting queries where alignment to the exact query window is
more important than clock-round boundaries.
processing (SeriesProcessing):
Processing mode for the returned series.
- ``SeriesProcessing.RESAMPLED``: Return a processed series.
Measurements are first filtered by ``start_datetime``,
``end_datetime``, and ``dropna``, then resampled according to
``interval`` and ``resample_method``, and finally missing values
are filled using ``fill_method``.
- ``SeriesProcessing.RAW``: Return the original measurement series.
Measurements are filtered by ``start_datetime``,
``end_datetime``, and ``dropna`` only. No resampling or filling is
performed, and ``interval``, ``fill_method``,
``resample_method``, ``boundary``, and
``align_to_interval`` are ignored.
Defaults to ``SeriesProcessing.RAW``.
Returns:
Array
"""
```
<!-- pyml enable line-length -->
@@ -1435,6 +1586,20 @@ Args:
- `end_datetime` (query, optional): Ending datetime (exclusive).
- `interval` (query, optional): Time duration for each interval. Defaults to 1 hour.
- `fill_method` (query, optional): Method to handle missing values during resampling.
- `resample_method` (query, optional): Method used to aggregate values within a resampling interval.
- `dropna` (query, optional): Drop NAN/ None values before processing.
- `boundary` (query, optional): Resampling boundary mode.
- `align_to_interval` (query, optional): Snap resample origin to the nearest UTC epoch-aligned boundary of interval.
- `processing` (query, optional): Processing mode. 'raw' returns original measurement data without resampling or filling.
**Responses**:
- **200**: Successful Response