4 Commits

Include renames

Author	SHA1	Message	Date
4 people	9189fc890e	fix(dataabc): drop NaN values in key_to_dict/key_to_lists (#1211) ... The dropna filter compared values against float("nan") using ==, which is always False (NaN != NaN). As a result NaN values were never dropped when dropna=True, letting them leak into key_to_series/key_to_array and downstream resampling. Use pd.isna() to detect NaN, matching the rest of the module. Add regression tests that fail before and pass after the fix. Co-authored-by: Cornelius Mund <cornim@users.noreply.github.com> Co-authored-by: Normann <github@koldrack.com>	2026-08-01 13:13:24 +02:00
Bobby NoelteandGitHub	1905682113	chore: adapt pdf visualization (#1205) ... Change PDF visualization to be created on demand and per optimization algorithm. The PDF for the GENETIC0 optimization is provided by the /visualization_results.pdf endpoint. There is no change in the interface. By this the optimization algorithm is offloaded from the PDF generation which spares some time. To cope with several users may call the /visualization_results.pdf endpoint at the same time the PDF is generated on the fly without any intermediate file taking the stored GENETIC0 solution as an input. SVG picture generation is removed as this would again create intermediate files. Chart pictures can easily be taken from the PDF. To allow on demand creation of the optimization results visualization the optimisation solution stored is extended by several new attributes. To keep the deprecated /optimize endpoint compatible the optimization solution is stripped to the legacy content before returned. Due to the extension of the solution the optimization tests were adapted to cover the extended content. The optimization tests are adapted to test the generated visualization report by the pypdf reader. Pypdf is added to the development dependencies. Besides the adaptation several fixes and improvements are added: * feat: extend /v1/prediction/series endpoint by resampling and filling Add parameters for resampling and filling. Add the processing parameter to control wether raw data or resampled data shall be returned. * feat: extend /v1/measurement/series endpoint by resampling and filling Add parameters for resampling and filling: Add the processing parameter to control wether raw data or resampled data shall be returned. * feat: standardize and improve API error response Use FASTApi exception handlers to provide a standardized API exception handling. All exceptions are logged. Exception traces are only returned if the new logging configuration parameter logging.api_logging_level is set to "DEBUG" or "TRACE". Avoids unwanted leackage of server internals on exceptions. * fix: align to intervall when resampling Ensure resampling is aligned to interval also when the buckets are shifted due to the align_to_intervall parameter is set. * chore: make dropna mandatory and default to True * chore: refactor key_to_xxx data management methods Make key_to_series the central method for data resampling and fill. Add a new key_to_raw_series to retrieve the data as it is stored (without resampling and filling). Users of key_to_series were mostly moved to key_to_raw_series as this resembles the former interface. Especially in predictions and tests this was done. * chore: create test data sub-directory for each optimization algorithm To prevent cluttering the test data directory and ease test data management for optimization algorithms each algorithm got it's own sub-directory. The current test data was moved to these sub-directories. * chore: update version Signed-off-by: Bobby Noelte <b0661n0e17e@gmail.com>	2026-08-01 12:45:19 +02:00
Bobby NoelteandGitHub	8344974c16	chore: adapt deprecated endpoints to 15-minutes predictions (#1195) ... Ensure the deprecated endpoints get /strompreis, post /gesamtlast, get /gesamtlast_simple, get /pvforecast to work on 1-hour intervalls even if the prediction provides 15-minutes intervall data. This keeps the interface compliant to the legacy functionality. Besides this adaptations several other improvements and fixes are included in this PR. * feat: extend data management method key_to array by resample_method One can now define the resample method on how to aggregate the values in an interval for resampling. Three methods are provided: - "first": Use the first value in each interval. - "mean": Compute the arithmetic mean of all samples in each interval. - "interval_mean": Compute the time-weighted mean assuming each value remains valid until the next timestamp (piecewise-constant signal). * feat: extend the get /v1/prediction/dataframe endpoint with resampling parameters Make all parameters for resampling available at the endpoint. * feat: extend the get /v1/prediction/list endpoint with resampling parameters Make all parameters for resampling available at the endpoint. * feat: add new delete /v1/prediction/range endpoint The endpoint allows to delete prediction values for a given time span. * fix: adapt for PVForecastAkkudoktor server side cache handling /api.akkudoktor/forecast does it's own caching on requests. Call it with slightly randomized request values to avoid getting cached values in the case we need fresh data. The requests are anyway rate limited to one request per hour on our side. * chore: add core.types This module centralizes reusable type definitions shared across multiple packages. Defining common types here avoids duplication of complex type annotations (such as Literal aliases), ensures consistent typing across the code base, and helps prevent circular import dependencies between modules. * chore: extend cache testing * chore: add system test for deprecated /strompreis endpoint * chore: add unit test module for server endpoints Add a new test module to do unit tests on server endpoints. First test added for deprecated get /strompreis endpoint. Signed-off-by: Bobby Noelte <b0661n0e17e@gmail.com>	2026-07-30 19:54:05 +02:00
Bobby NoelteandGitHub	eb9e966de9	fix: move data management to async (#1015) ... FAstAPI is an async framework. Data may be imported and exported, load and save, set and get asynchronously. Prevent interleaving data operations to corrupt the data. In the previous design sync and async data access was intermixed leading to data corruption. The basic data classes DataSequence and DataContainer and the derived classes like Provider and Measurement now are async. Data access is protected by several async locks. To support the async design of the data classes the database interface became async. The energy management is also adapted to the new async design. Optimization is still off-loaded to another thread, but the prepration for the optimization and the post optimization actions now follow the async design. Adapter operations are now also protected by async locks. Tests were adapted to the async design and new tests were created. Besides this major fix several other improvements and fixes are included in this PR. * fix: key_to_dict/list/array only regard data records with key value set. Before the exclusion of no value data records was only done if the dropna flag was set. * fix: test for visual result pdf generation Due to updates in the library the generated charts text was a little bit different. Adapt the test to create the comaprison pdf in the test data durectory and update the reference pdf. * chore: Remove MutableMapping from DataSequence and DataContainer. Mutable Mapping does not fit to the now async design. * chore: Add NoDB database backend This backend implements the full database backend interface but performs no actual persistence. It is intended for configurations where database persistence is disabled (`provider=None`). * chore: Improve measurement data import testing with real world scenarios. Added two new endpoints to support testing. * chore: Add mermaid to supported documentation tools * chore: Add documentation about async design * chore: Add documentation about generic data handling Covers the basics of measurement and prediction time series data handling. * chore: Add empty lines around markdown lists. * chore: sync pre-commit config to updated package versions Signed-off-by: Bobby Noelte <b0661n0e17e@gmail.com>	2026-07-15 16:38:53 +02:00